From 67a42fc97c64c83e02f6f0d68e5a4a22c71138d3 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Thu, 25 Jul 2024 00:50:30 +0300 Subject: libcurl: update to 8.9.0 --- libs/libcurl/docs/CHANGES | 14711 +++++++++++++++++++++++--------------------- libs/libcurl/docs/THANKS | 38 + 2 files changed, 7905 insertions(+), 6844 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 5553b0a845..8ea9d27b12 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,10704 +6,11727 @@ Changelog -Version 8.8.0 (22 May 2024) +Version 8.9.0 (24 Jul 2024) -Daniel Stenberg (22 May 2024) +Daniel Stenberg (24 Jul 2024) - RELEASE-NOTES: synced -- THANKS: add contributors from 8.8.0 +- THANKS: updates from the 8.9.0 release -Nathan Moinvaziri (21 May 2024) +- curl_easy_escape.md: move historic details to HISTORY -- url: remove duplicate call to Curl_conncache_remove_conn when pruning + Closes #14261 - - remove unnecessary prunedead struct from prune_dead_connections - - rename extract_if_dead to prune_if_dead for clarity +- docs/libcurl: add to cleanup docs that their inputs go invalid - Closes #13710 + Reported-by: icy17 on github + Fixes #14248 + Closes #14258 -Joseph Chen (21 May 2024) +dependabot[bot] (23 Jul 2024) -- curl_setup.h: add support for IAR compiler +- GHA: bump github/codeql-action from 3.25.11 to 3.25.13 - Closes #13728 + Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3. + 25.11 to 3.25.13. + - [Release notes](https://github.com/github/codeql-action/releases) + - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) + - [Commits](https://github.com/github/codeql-action/compare/b611370bb5703a7ef + b587f9d136a52ea24c5c38c...2d790406f505036ef40ecba973cc774a50395aac) -Stephen Farrell (21 May 2024) + --- + updated-dependencies: + - dependency-name: github/codeql-action + dependency-type: direct:production + update-type: version-update:semver-patch + ... -- docs/ECH: typo/clarification + Signed-off-by: dependabot[bot] + Closes #14255 - Closes #13727 +Stefan Eissing (23 Jul 2024) -Viktor Szakats (21 May 2024) +- conncache: connection shutdown, multi_socket handling -- hash: delete unused debug function + - implement the socket hash user/reader/writer processing also + for connections that are being shut down by the connection cache. + - split out handling of current vs. last pollset socket event handling + into a function available in other code parts + - add `shutdown_poll` pollset to `connectdata` struct so that changes + in the pollset can be recorded during shutdown. (The internal handle + cannot keep it since it might be used for many connections) - It had no use in the curl codebase and was also protected by the macro - `AGGRESSIVE_TEST` (renamed in 2020), also with no local reference. + Reported-by: calvin2021y on github + Fixes #14252 + Closes #14257 - Added in ca6e77083768858aa34207f8c5dce38b3c05336d (2002-11-11) +Daniel Stenberg (22 Jul 2024) - Closes #13729 +- tool_cb_prg: output "flying saucers" with leading carriage return -Stefan Eissing (21 May 2024) + Because that is how the progress-bar is output, so when the progress-bar + has been shown at least once and the information is reset, like for a + redirect, there might be a moment where the size goes from known to + unknown and then the flying saucerts are shown after a brief display of + the progress-bar. -- content_encoding: reject transfer-encoding after chunked + It could previously cause accidental character leftovers on the right + side of the bar when using a narrow display. - reject a response that applies a transfer-encoding after a 'chunked' - encoding. RFC 9112 ch. 6.1 required chunked to be the final encoding. + Reported-by: Chris Webb + Fixes #14213 + Closes #14246 - Closes #13733 +- lib: Curl_posttransfer => multi_posttransfer -- http: HEAD response body tolerance + Moved from transfer.c to multi.c as it was only used within multi.c - - as reported in #13725, some servers wrongly send body bytes in - responses to a HEAD request. This used to be tolerated in curl - 8.4 and before and leads to failed transfers in newer versions. - - restore previous behaviour for HTTP/1.1 and HTTP/2: - * 1.1: do not add 'Transfer-Encoding' writers from HEAD - responses. RFC 9112 says they do not apply. - * 2: when the transfer expects 'no_body', to not report stream - resets as error when all response headers have been received. + Made a void, as it returned a fixed return code nothing checked. - Reported-by: Jeroen Ooms - Fixes #13725 - Closes #13732 + Closes #14240 -Viktor Szakats (20 May 2024) +- CURLOPT_SSL_VERIFYHOST.md: refresh -- tests: fix TFTP test 2305 on Windows + Move mentions of old behavior to the HISTORY section to make it easier + to read about modern behavior. - Ref: #13692 - Closes #13724 + Added a MATCHING section. -Jay Satiro (20 May 2024) + Closes #14241 -- openssl: revert keylog_callback support for LibreSSL +- multi: do a final progress update on connect failure - - Revert to the legacy TLS 1.2 key logging code for LibreSSL. + To fix timing metric etc - - Document SSLKEYLOGFILE for LibreSSL is TLS 1.2 max. + Co-authored-by: Justin Maggard + Fixes #14204 + Closes #14239 - Prior to this change if the user specified a filename in the - SSLKEYLOGFILE environment variable and was using LibreSSL 3.5.0+ then - an empty file would be created and no keys would be logged. +Orgad Shaneh (19 Jul 2024) - This is effectively a revert of e43474b4 which changed openssl.c to use - SSL_CTX_set_keylog_callback for LibreSSL 3.5.0+. Unfortunately LibreSSL - added that function only as a stub that doesn't actually do anything. +- md4: fix compilation with OpenSSL 1.x with md4 disabled - Reported-by: Gonçalo Carvalho + If OpenSSL 1.x is used, and it is configured with md4 disabled, + OPENSSL_NO_MD4 is defined in opensslconf.h, but this header was not + included before checking for this define. - Fixes https://github.com/curl/curl/issues/13672 - Closes https://github.com/curl/curl/pull/13682 + Later in md4.c, openssl/md4.h is included, and it includes that header + indirectly, leading to inconsistency within md4.c. -renovate[bot] (19 May 2024) + Since the md4.h branch was taken, wincrypt.h (or others) is not + included, and later below the USE_WIN32_CRYPTO branch is taken, but the + types are not defined. -- GHA: pin dependencies + Closes #14218 - Closes #13712 +martinevsky (19 Jul 2024) -Viktor Szakats (19 May 2024) +- ftp: remove redundant null pointer check in loop condition -- appveyor: drop unnecessary `--clean-first` cmake option + Closes #14234 - In CI all machines are fresh on startup, making the `clean` operation - unnecessary. This can save some time/energy for each job run. +Justin Maggard (19 Jul 2024) - Closes #13707 +- mbedtls: check version before getting tls version -- cmake: merge two `if(BUILD_TESTING)` branches + mbedtls_ssl_get_version_number() was added in mbedtls 3.2.0. Check for + that version before using it. - Closes #13708 + Closes #14228 -Tatsuhiro Tsujikawa (19 May 2024) +martinevsky (19 Jul 2024) -- GHA: bump nghttp2 to v1.62.1 +- urlapi: remove unused definition of HOST_BAD - Use gcc-12 explicitly to compile C++20 source files. + Closes #14235 - Closes #13702 +Daniel Stenberg (19 Jul 2024) -Viktor Szakats (19 May 2024) +- curldown: fixups -- GHA: add NetBSD, OpenBSD, FreeBSD/arm64 and OmniOS jobs + - make DEFAULT sections less repetitive - Add these jobs to GHA: - - NetBSD, cmake-unity, clang, OpenSSL, x86_64, with tests, w/o python, - no parallelism (was flaky sometimes) - - OpenBSD, cmake-unity, clang, LibreSSL, x86_64, with tests, - with python, -j8, TFTP results ignored due to #13623. - - FreeBSD, cmake-unity and autotools, clang, OpenSSL, arm64 - (Tests disabled for arm64, because they are slow. It's available for - x86_64 with python, -j12.) - Configuration matches our existing Cirrus CI one. - - OmniOS, autotools, gcc, OpenSSL, x86_64, with tests, -j12. + - make historic mentions use HISTORY - All build with websockets and examples. + - generate the protocols section on `# %PROTOCOLS%` instead of guessing + where to put it - Closes #13583 + - generate the availability section on `# %AVAILABILITY%` instead of + guessing where to put it -- GHA: disable TFTP test on native Windows + - make the protocols section more verbose - Some TFTP tests seem to enter into a loop and maybe hang? + Closes #14227 - E.g. 1007, 1009, 1238 +Tal Regev (19 Jul 2024) - Try fixing it by skipping all TFTP tests. +- GHA/windows: enable libssh in !ssl MSVC job - Ref: https://github.com/curl/curl/actions/runs/9141987545/job/25137038249?pr= - 13698 + Closes #14232 - Also drop mingw-w64 test exclusions copy-pasted from MSYS jobs. +- GHA/windows: enable libidn2 in !ssl MSVC job - Possibly related: cffbcc3110c1eda2e333f9cfe2e269154618793a #5364 + Closes #14200 - Close #13699 +Viktor Szakats (19 Jul 2024) -renovate[bot] (18 May 2024) +- GHA/macos: improve, fix gcc/llvm, add new test matrix -- GHA: pin dependencies + This PR began as an attempt to drop GCC support, after repeated reports + on fallouts when trying to use it on macOS. - Closes #13691 + Then it transformed into a 3-week project turning up the issues causing + the fallouts, ending up including llvm and all available Xcode / macOS + SDK, macOS runner image, build tools and compiler vendors and versions. + Accumulating 400 sub-commits. -Viktor Szakats (18 May 2024) + I developed and tested all fixes under this PR, then merged them as + separate patches. -- cmake: do not pass linker flags to the static library tool + This PR retained CI jobs updates, extensively reworking and extending + them: [1] - Do not add linker flags to the global CMake static library tool (aka - "static linker") (e.g. `ar`) flags list. They don't mix well. This was - only done after successfully detecting GSSAPI. + At first it seemed GCC and the Apple SDK is "naturally" growing more + incompatible, as Apple added further non-standard features to their + headers. This is partly true, but reality is more complicated. - Linker flags seen on Old Linux CI: - ``` - -- |GSS_LINKER_FLAGS|-Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib/x86_64-li - nux-gnu/heimdal| - -- |CMAKE_STATIC_LINKER_FLAGS| -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib - /x86_64-linux-gnu/heimdal| - ``` - Ref: https://github.com/curl/curl/actions/runs/9138988036/job/25130791712#ste - p:6:85 + Besides some issues local to curl, there were bugs in Apple SDK + headers, Homebrew GCC builds, feature missing in the old llvm version + pre-installed on GitHub CI runner images, and subtle incompatibilities + between GCC and llvm/clang when handling language extensions. - Causing: - ``` - /usr/bin/ar qc libcurltool.a -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib/ - x86_64-linux-gnu/heimdal - CMakeFiles/curltool.dir/slist_wc.c.o CMakeFiles/curltool.dir/tool_binmode.c - .o CMakeFiles/curltool.dir/tool_bname.c.o - [...] - CMakeFiles/curltool.dir/tool_writeout_json.c.o CMakeFiles/curltool.dir/tool - _xattr.c.o CMakeFiles/curltool.dir/var.c.o - CMakeFiles/curltool.dir/__/lib/base64.c.o CMakeFiles/curltool.dir/__/lib/dy - nbuf.c.o - /usr/bin/ar: invalid option -- 'W' - Usage: /usr/bin/ar [emulation options] [-]{dmpqrstx}[abcDfilMNoPsSTuvV] [--pl - ugin ] [member-name] [count] archive-file file... - /usr/bin/ar -M [now - c->timestamp; - | ^~~ - curl/lib/hostip.c: In function 'Curl_hostcache_prune': - curl/lib/hostip.c:241:10: note: 'now' was declared here - 241 | time_t now; - | ^~~ - In function 'hostcache_timestamp_remove', - inlined from 'fetch_addr' at curl/lib/hostip.c:310:8: - curl/lib/hostip.c:205:23: error: 'user.now' may be used uninitialized [-Werro - r=maybe-uninitialized] - 205 | time_t age = prune->now - c->timestamp; - | ~~~~~^~~~~ - curl/lib/hostip.c: In function 'fetch_addr': - curl/lib/hostip.c:304:33: note: 'user' declared here - 304 | struct hostcache_prune_data user; - | ^~~~ - In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity - _0_c.c:40: - curl/lib/cf-socket.c: In function 'cf_socket_send': - curl/lib/cf-socket.c:1294:10: error: 'c' may be used uninitialized [-Werror=m - aybe-uninitialized] - 1294 | if(c >= ((100-ctx->wblock_percent)*256/100)) { - | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - curl/lib/cf-socket.c:1292:19: note: 'c' was declared here - 1292 | unsigned char c; - | ^ - In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity - _0_c.c:364: - In function 'tftp_state_timeout', - inlined from 'tftp_multi_statemach' at curl/lib/tftp.c:1230:27: - curl/lib/tftp.c:1208:5: error: 'current' may be used uninitialized [-Werror=m - aybe-uninitialized] - 1208 | if(current > state->rx_time + state->retry_time) { - | ^ - curl/lib/tftp.c: In function 'tftp_multi_statemach': - curl/lib/tftp.c:1192:10: note: 'current' was declared here - 1192 | time_t current; - | ^~~~~~~ + macOS Xcode gcc gcc SDK hacks Xcode SDK SDK major Build + Compile + (*def) (Homebrew) (CommandLineTools) versions + error + -------- -------- ---------- ------------------ ---------- --------- ----- + --------------------- + macos-12 13.1 GCC 11.4.0 MacOSX12 MacOSX12.0 + macos-12 13.2.1 GCC 11.4.0 MacOSX12 MacOSX12.1 + macos-12 13.3.1 GCC 11.4.0 MacOSX12 MacOSX12.3 + macos-12 13.4.1 GCC 11.4.0 MacOSX12 MacOSX12.3 + macos-12 14.0.1 GCC 11.4.0 MacOSX12 MacOSX12.3 + macos-12 14.1 GCC 11.4.0 MacOSX12 MacOSX13.0 MISMATCH FAIL + /Applications/Xcode_14.1.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' be + fore 'extern' + macos-12 *14.2 GCC 11.4.0 MacOSX12 MacOSX13.1 MISMATCH FAIL + /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' be + fore 'extern' + macos-13 14.1 GCC 11.4.0 MacOSX13 MacOSX13.0 + macos-13 14.2 GCC 11.4.0 MacOSX13 MacOSX13.1 + macos-13 14.3.1 GCC 11.4.0 MacOSX13 MacOSX13.3 + macos-13 *15.0.1 GCC 11.4.0 MacOSX13 MacOSX14.0 MISMATCH FAIL + /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/ + Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown + type name 'dispatch_queue_t' + macos-13 15.1 GCC 11.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown ty + pe name 'dispatch_queue_t' + macos-13 15.2 GCC 11.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown ty + pe name 'dispatch_queue_t' + macos-14 14.3.1 GCC 11.4.0 MacOSX14 MacOSX13.3 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-14 *15.0.1 GCC 11.4.0 MacOSX14 MacOSX14.0 + macos-14 15.1 GCC 11.4.0 MacOSX14 MacOSX14.2 + macos-14 15.2 GCC 11.4.0 MacOSX14 MacOSX14.2 + macos-14 15.3 GCC 11.4.0 MacOSX14 MacOSX14.4 + macos-14 15.4 GCC 11.4.0 MacOSX14 MacOSX14.5 + macos-14 16.0 GCC 11.4.0 MacOSX14 MacOSX15.0 MISMATCH FAIL + /opt/homebrew/Cellar/gcc@11/11.4.0/lib/gcc/11/gcc/aarch64-apple-darwin23/11/ + include-fixed/stdio.h:83:8: error: unknown type name 'FILE' + macos-12 13.1 GCC 12.4.0 MacOSX12 MacOSX12.0 + macos-12 13.2.1 GCC 12.4.0 MacOSX12 MacOSX12.1 + macos-12 13.3.1 GCC 12.4.0 MacOSX12 MacOSX12.3 + macos-12 13.4.1 GCC 12.4.0 MacOSX12 MacOSX12.3 + macos-12 14.0.1 GCC 12.4.0 MacOSX12 MacOSX12.3 + macos-12 14.1 GCC 12.4.0 MacOSX12 MacOSX13.0 MISMATCH FAIL + /Applications/Xcode_14.1.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' be + fore 'extern' + macos-12 *14.2 GCC 12.4.0 MacOSX12 MacOSX13.1 MISMATCH FAIL + /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' be + fore 'extern' + macos-13 14.1 GCC 12.4.0 MacOSX13 MacOSX13.0 + macos-13 14.2 GCC 12.4.0 MacOSX13 MacOSX13.1 + macos-13 14.3.1 GCC 12.4.0 MacOSX13 MacOSX13.3 + macos-13 *15.0.1 GCC 12.4.0 MacOSX13 MacOSX14.0 MISMATCH FAIL + /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/ + Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown + type name 'dispatch_queue_t' + macos-13 15.1 GCC 12.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown ty + pe name 'dispatch_queue_t' + macos-13 15.2 GCC 12.4.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:103:1: error: unknown ty + pe name 'dispatch_queue_t' + macos-14 14.3.1 GCC 12.4.0 MacOSX14 MacOSX13.3 MISMATCH + macos-14 *15.0.1 GCC 12.4.0 MacOSX14 MacOSX14.0 + macos-14 15.1 GCC 12.4.0 MacOSX14 MacOSX14.2 + macos-14 15.2 GCC 12.4.0 MacOSX14 MacOSX14.2 + macos-14 15.3 GCC 12.4.0 MacOSX14 MacOSX14.4 + macos-14 15.4 GCC 12.4.0 MacOSX14 MacOSX14.5 + macos-14 16.0 GCC 12.4.0 MacOSX14 MacOSX15.0 MISMATCH FAIL + /opt/homebrew/Cellar/gcc@12/12.4.0/lib/gcc/12/gcc/aarch64-apple-darwin23/12/ + include-fixed/stdio.h:83:8: error: unknown type name 'FILE' + macos-12 13.1 GCC 13.3.0 MacOSX12 MacOSX12.0 + macos-12 13.2.1 GCC 13.3.0 MacOSX12 MacOSX12.1 + macos-12 13.3.1 GCC 13.3.0 MacOSX12 MacOSX12.3 + macos-12 13.4.1 GCC 13.3.0 MacOSX12 MacOSX12.3 + macos-12 14.0.1 GCC 13.3.0 MacOSX12 MacOSX12.3 + macos-12 14.1 GCC 13.3.0 MacOSX12 MacOSX13.0 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-12 *14.2 GCC 13.3.0 MacOSX12 MacOSX13.1 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-13 14.1 GCC 13.3.0 MacOSX13 MacOSX13.0 + macos-13 14.2 GCC 13.3.0 MacOSX13 MacOSX13.1 + macos-13 14.3.1 GCC 13.3.0 MacOSX13 MacOSX13.3 + macos-13 *15.0.1 GCC 13.3.0 MacOSX13 MacOSX14.0 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-13 15.1 GCC 13.3.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-13 15.2 GCC 13.3.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-14 14.3.1 GCC 13.3.0 MacOSX14 MacOSX13.3 MISMATCH FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-14 *15.0.1 GCC 13.3.0 MacOSX14 MacOSX14.0 FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-14 15.1 GCC 13.3.0 MacOSX14 MacOSX14.2 FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-14 15.2 GCC 13.3.0 MacOSX14 MacOSX14.2 FAIL + /Users/runner/work/curl/curl/bld/lib/curl_config.h:792:19: error: two or mor + e data types in declaration specifiers + macos-14 15.3 GCC 13.3.0 MacOSX14 MacOSX14.4 + macos-14 15.4 GCC 13.3.0 MacOSX14 MacOSX14.5 + macos-14 16.0 GCC 13.3.0 MacOSX14 MacOSX15.0 MISMATCH FAIL + /opt/homebrew/Cellar/gcc@13/13.3.0/lib/gcc/13/gcc/aarch64-apple-darwin23/13/ + include-fixed/stdio.h:83:8: error: unknown type name 'FILE' + macos-12 13.1 GCC 14.1.0 MacOSX12 MacOSX12.0 + macos-12 13.2.1 GCC 14.1.0 MacOSX12 MacOSX12.1 + macos-12 13.3.1 GCC 14.1.0 MacOSX12 MacOSX12.3 + macos-12 13.4.1 GCC 14.1.0 MacOSX12 MacOSX12.3 + macos-12 14.0.1 GCC 14.1.0 MacOSX12 MacOSX12.3 + macos-12 14.1 GCC 14.1.0 MacOSX12 MacOSX13.0 MISMATCH FAIL + /Applications/Xcode_14.1.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' be + fore 'extern' + macos-12 *14.2 GCC 14.1.0 MacOSX12 MacOSX13.1 MISMATCH FAIL + /Applications/Xcode_14.2.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/os/object.h:275:1: error: expected ';' be + fore 'extern' + macos-13 14.1 GCC 14.1.0 MacOSX13 MacOSX13.0 + macos-13 14.2 GCC 14.1.0 MacOSX13 MacOSX13.1 + macos-13 14.3.1 GCC 14.1.0 MacOSX13 MacOSX13.3 + macos-13 *15.0.1 GCC 14.1.0 MacOSX13 MacOSX14.0 MISMATCH FAIL + /Applications/Xcode_15.0.1.app/Contents/Developer/Platforms/MacOSX.platform/ + Developer/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:70:1: error: type defa + ults to 'int' in declaration of 'DISPATCH_DECL_FACTORY_CLASS_SWIFT' [-Wimplic + it-int] + macos-13 15.1 GCC 14.1.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:70:1: error: type defaul + ts to 'int' in declaration of 'DISPATCH_DECL_FACTORY_CLASS_SWIFT' [-Wimplicit + -int] + macos-13 15.2 GCC 14.1.0 MacOSX13 MacOSX14.2 MISMATCH FAIL + /Applications/Xcode_15.2.app/Contents/Developer/Platforms/MacOSX.platform/De + veloper/SDKs/MacOSX.sdk/usr/include/dispatch/queue.h:70:1: error: type defaul + ts to 'int' in declaration of 'DISPATCH_DECL_FACTORY_CLASS_SWIFT' [-Wimplicit + -int] + macos-14 14.3.1 GCC 14.1.0 MacOSX14 MacOSX13.3 MISMATCH + macos-14 *15.0.1 GCC 14.1.0 MacOSX14 MacOSX14.0 + macos-14 15.1 GCC 14.1.0 MacOSX14 MacOSX14.2 + macos-14 15.2 GCC 14.1.0 MacOSX14 MacOSX14.2 + macos-14 15.3 GCC 14.1.0 MacOSX14 MacOSX14.4 + macos-14 15.4 GCC 14.1.0 MacOSX14 MacOSX14.5 + macos-14 16.0 GCC 14.1.0 MacOSX14 MacOSX15.0 MISMATCH FAIL + /opt/homebrew/Cellar/gcc/14.1.0_1/lib/gcc/current/gcc/aarch64-apple-darwin23 + /14/include-fixed/stdio.h:83:8: error: unknown type name 'FILE' ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49792835/job/91c8dj5 - qb36spfe0#L112 - Ref: https://github.com/curl/curl/actions/runs/9082968838/job/24960616145#ste - p:12:62 + Source: https://github.com/curl/curl/actions/runs/9883956647/job/27299564218 - Ref: #13592 - Closes #13643 + This commit fixes earlier commit + 1e75edd372868048c9f805ac4ca6d2cb5a88ff5a, reverted in + 41a7e0dcc9681afd91e066411bcee4f369c23366, where I cut the commit + message in half by accident. The patch itself is identical. -Andrew (16 May 2024) + Closes #14097 -- wakeup_create: use FD_CLOEXEC/SOCK_CLOEXEC +- Revert "GHA/macos: improve, fix gcc/llvm, add new test matrix" - for `pipe()`/`socketpair()` + This reverts commit 1e75edd372868048c9f805ac4ca6d2cb5a88ff5a. - Fixes #13618 - Closes #13625 + Due to some parts of the commit message missing (my bad.) -Stefan Eissing (16 May 2024) +Daniel Stenberg (19 Jul 2024) -- rustls: fix partial send handling +- Revert "lib: send eos flag" - When TLS bytes could not completely sent off, the amount of plain bytes - already added to rustls were forgotten. This lead to send those byte - duplicate, corrupting the request send to the server. + This reverts commit be93299f10ef0b2bf7fe5c82140120073831867a. - Closes #13676 +Viktor Szakats (19 Jul 2024) -- pytest: add DELETE tests, check server version +- GHA/windows: use default shell CI feature - - add tests for DELETE working - - check apache version in keepalive test - - fix some comments + It makes repeating a line in each step unnecessary. - Closes #13679 + Closes #14206 -Juliusz Sosinowicz (16 May 2024) +- GHA/macos: improve, fix gcc/llvm, add new test matrix -- vquic-tls: use correct cert name check API for wolfSSL + This PR began as an attempt to drop GCC support, after repeated reports + on fallouts when trying to use it on macOS. - wolfSSL_X509_check_host checks the peer name against the alt names and - the common name. + Then it transformed into a 3-week project turning up the issues causing + the fallouts, ending up including llvm and all available Xcode / macOS + SDK, macOS runner image, build tools and compiler vendors and versions. + Accumulating 400 sub-commits. - Fixes #13487 - Closes #13680 + I developed and tested all fixes under this PR, then merged them as + separate patches. -Viktor Szakats (16 May 2024) + This PR retained CI jobs updates, extensively reworking and extending + them: [1] -- cmake: initialize `BUILD_TESTING` before first use + At first it seemed GCC and the Apple SDK is "naturally" growing more + incompatible, as Apple added further non-standard features to their + headers. This is partly true, but reality is more complicated. - Before this patch `BUILD_TESTING` was used once, then initialized, then - used again. This caused the `curlu` library not being built when relying - on an implicit `BUILD_TESTING=ON` setting, and ending up with a link - error when building the `testdeps` target. + Besides some issues local to curl, there were bugs in Apple SDK + headers, Homebrew GCC builds, feature missing in the old llvm version + pre-installed on GitHub CI runner images, and subtle incompatibilities + between GCC and llvm/clang when handling language extensions. - It did not cause issues when `BUILD_TESTING` was explicitly set. + Resulting compiler errors seldom pointed to a useful direction, and + internet search was silent about these issues too. Thus, I had to peel + them off layer by layer, using trial and error, and by recognizing + patterns of failures accross 150-200 builds combinations. Exposing + configure logs, and curl_config.h in the CI logs helped too. - Move the initialization before the first use to fix it. + 1. GCC header compatibility layer ("hack" as GCC calls it) - Regression from aace27b0965c10394544d1dacc9c2cb2fe0de3d3 #12287 - Closes #13668 + The toughest issue is GCC's built-in compatibility layer: + https://github.com/gcc-mirror/gcc/tree/master/fixincludes -Daniel Stenberg (16 May 2024) + This patch layer is further patched by a "Darwin compatibility" project + applied on top by Homebrew GCC via: + https://github.com/iains/gcc-12-branch + https://github.com/iains/gcc-13-branch + https://github.com/iains/gcc-14-branch -- libtest: 2308 verifies CURLE_WRITE_ERROR after write callback error + The hack layer is designed in a way that breaks more builds than it + fixes, esp. in context of GHA runners. The idea is to build GCC + specifically for the SDK for the target macOS version. The problem with + this approach is that the Xcode + SDK installed on the local/CI machine + often does not match with the SDK used on while building GCC on + Homebrew's build machines. In these cases the GCC compatibility layer + turns into an "uncompatibility" layer and consistently breaks builds. + curl cannot offer a fix for this, because the solution (I found) is to + patch the toolchain on the local machine. I implemented this for our CI + builds and curl-for-win. In other case the user must do this patching + manually, or choose a compatible GCC + Xcode/SDK combination. - Verifies that the issue in #13669 actually is fixed. This return code is - what the CURLOPT_WRITEFUNCTION manpage documents should be returned. + An upstream fix doesn't seem trivial either, because the issue is + ingrained in the compatibility layer's design. Offering an `-fapplesdk` + (or recognizing `-target`) option and/or fixing them within the compiler + would seem like a more robust option, and also how mainline llvm solves + this. - This code is mostly from the - Source-written-by: Trumeet on github - Closes #13671 + Here's a table summarizing the GCC + SDK combinations and curl build + failures: [2] -Antoine Bollengier (16 May 2024) + More info: https://github.com/curl/curl/issues/10356#issuecomment-2222734103 -- socketpair: fix compilation when USE_UNIX_SOCKETS is not defined + db135f8d7207b20d531e7e2100a49f3e16bdcfab #14119 macos: add workaround for gcc + , non-c-ares, IPv6, compile error + Ref: https://github.com/curl/curl-for-win/commit/e2db3c475f5981352e6e6a79854a + 255805b28deb + Ref: https://github.com/curl/curl-for-win/commit/f5c58d7fef78e972be33ca2355dc + b42ba56622a6 - Closes #13666 + 2. Homebrew GCC's `availability` extension -Stefan Eissing (16 May 2024) + A recent minor Homebrew GCC upgrade caused major breakage. The "Darwin + compatibility" patch applied to GCC implemented the `availability` + compiler attribute in GCC. Apple SDK detected this and enabled using + them, but as it turns out GCC accepts compiler attributes with slightly + different rules than llvm/clang, and how the Apple SDK uses them, + breaking builds. -- rustsls: fix error code on receive + Affected Homebrew GCC versions are: 12.4.0, 13.3.0 and 14.1.0. - - use CURLE_RECV_ERROR instead of CURLE_READ_ERROR when receiving - data fails. + Possibly tracked here: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108796 + More info: https://github.com/llvm/llvm-project/issues/81767 - Closes #13670 + Commit implementing the `availability` macro: + gcc-12: https://github.com/iains/gcc-12-branch/commit/fd5530b7cb0012bf4faeddd + 45e13054a1dfa6783 + gcc-13: https://github.com/iains/gcc-13-branch/commit/cb7e4eca68cfc4763474e2e + b0935a844458842a8 + gcc-14: https://github.com/iains/gcc-14-branch/commit/ff62a108865a6403f501738 + 0d7018250c1d3306f -Max Dymond (16 May 2024) + That applied to Homebrew GCC (12.4.0): + https://github.com/Homebrew/homebrew-core/commit/b904223d9893f62bec2a8f7483bf + 5992747fc6c7#diff-89dd0b4176eca7fcc24b591943509bf8a8d6ea904d71e5dfcd6b78fed62 + fc574R44-R48 -- ci: disable Renovate dashboard + Ref: #13700 + More info: https://github.com/curl/curl/pull/14091#issuecomment-2222703468 - The Renovate dashboard insists on an open issue, - which is a problem. Disable the dashboard. Status - can still be seen at https://developer.mend.io/github/curl/curl. + e91fcbac7d86292858718a0bfebad57978761af4 #14155 macos: undo `availability` ma + cro enabled by Homebrew gcc - Fixes #13630 - Closes #13673 + 3. Proprietary Apple SDK macros -Daniel Stenberg (16 May 2024) + Apple SDK expects certain macros predefined by the compiler. Missing + them may causes odd issues. Mainline llvm is keeping up with Apple + clang, but it needs a fresh version, while the one installed on GitHub + runners is old (v15). I patched these in `lib/curl_setup.h`. -- RELEASE-NOTES: synced + baa3270846b2a7307cbd0dd5c02c4e5f00e388dd #14134 build: fix llvm 16 or older + + Xcode 15 or newer, and gcc -renovate[bot] (16 May 2024) + 4. Apple SDK header bug -- GHA: update awslabs/aws-lc to v1.27.0 + Without certain predefined macros, SDK headers can take a codepath where + it mis-defines its own `TARGET_OS_OSX` macro, which make it break its + own headers later. I patched it in `lib/curl_setup.h`. - Closes #13667 + ff784af461175584c73e7e2b65af00b1a5a6f67f #14159 build: fix llvm 17 and older + + macOS SDK 14.4 and newer -Daniel Stenberg (15 May 2024) + 5. `TargetConditionals.h` requires `sys/types.h` -- curl_easy_pause.md: use correct defines in example + Fixed in curl. It caused feature-detection failurs with autotools, and + could break builds in certain configurations. - Spotted-by: Harry Sintonen - Closes #13664 + e1f6192939c9c5ab2310b60bedf4c07d635193f6 #14130 configure: fix `SystemConfigu + ration` detection -Viktor Szakats (15 May 2024) + 6. Differences between autotools and CMake compiler options -- appveyor: more tidy-ups + Fixed it by syncing compiler warning options. - - use `--disable` when calling `curl --version`. Just in case. + 59cadacfcc1d39472245979cdbd614c7a9af6f0d #14128 build: sync warning options b + etween autotools, cmake & compilers - - use single-quotes for a constant. + 7. Differences between autotools and CMake dependency detection - Closes #13662 + Fixed it by improving detection of libidn2, with some more fixes + pending for the next feature window. -- reuse: migrate standalone license file to dep5 + f43adc2c4978f7f82a359e89186e58a31d17b0ad #14137 cmake: detect `libidn2` also + via `pkg-config` + Ref: #14136 cmake: detect `nghttp2` via `pkg-config`, enable by default - Follow-up to 73a36021207284ad2b4340ffde34a51b0ba4d47a - Closes #13660 + 8. libidn2 detection bug with CMake -- appveyor: guard against crash-build with VS2008 + Fixed the root cause and also the trigger in the CI config. - The combination of `-DDEBUGBUILD`, a shared `curl.exe`, and the VS2008 - compiler creates a `curl.exe` segfaulting on startup: + 764fbabf6ed4c1d36c0ab2033ac8df52d9923cd7 #14175 cmake: fix builds with detect + ed libidn2 lib but undetected header - ``` - + _bld/src/curl.exe --version - ./appveyor.sh: line 122: 793 Segmentation fault "${curl}" --version - Command exited with code 139 - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49817266/job/651iy6q - n1e238pqj#L191 + 9. Suppressed compiler warnings inside Apple-specific curl code - Add job that triggers the issue and add the necessary logic to skip - running the affected `curl.exe`. + Fixed these warnings, which allowed to stop silencing them. - Ref: #13592 - Closes #13654 + b05dc7eb3592305de9fa85640767f3dd2a8d4c93 #14122 sectransp: fix `HAVE_BUILTIN_ + AVAILABLE` checks to not emit warnings + 5fa534b0dacdc120aaab0766047e0ecac37be4b3 #14162 sectransp: fix clang compiler + warnings, stop silencing them -renovate[bot] (15 May 2024) + 10. CMake mis-detecting a CA bundle path on macOS -- GHA: pin dependencies + d2ef6255f4a040397d2f40ab7cbf65a203201cd9 #14182 cmake: sync CA bundle/path de + tection with autotools - Closes #13628 + 11. Failure to build tests with LibreSSL or wolfSSL with CMake -Orgad Shaneh (15 May 2024) + Fixed by dropping unnecessary includes, makign test builds dependent + on dependency headers. -- socket: remove redundant call to getsockname + 3765d75ce47b66289f946382b649d0e99389dc77 #14172 cmake: fix building `unit1600 + ` due to missing `ssl/openssl.h` - The result "add" is unused. + 12. curl tests with CMake - Closes #13655 + curl's CMake was missing bits for running the C preprocessor accurately. + It made tests 1119 and 1167 fail. I implemented the missing bits. -renovate[bot] (15 May 2024) + efc2c5184d008fe2e5910fd03263e1ab0331d4e6 #14124 tests: include current direct + ory when running test Perl commands + c09db8b51b88ee6ad55bd637dcb4b47678e30906 #14129 cmake: create `configurehelp. + pm` like autotools does + 67cc1e3400b77536a3ca529c986247e1ef985e6e #14125 test1119: adapt for `.md` inp + ut -- CI: renovate updates + 13. GCC missing `__builtin_available()` support - - GHA: update actions/checkout action to v4 - - GHA: update wolfSSL/wolfssh to v1.4.17 - - GHA: update wolfSSL/wolfssl to v5.7.0 - - Update the regex config in renovate.json + curl source code assumes this is available to enable certain codepaths. + It's also intermixed with monotonic timer support. - Closes #13632 - Closes #13641 - Closes #13658 - Closes #13659 + 14. Monotonic timer support with GCC -Max Dymond (15 May 2024) + Detected by GCC, while it probably shouldn't be. llvm/clang detects it + depending on target OS version. I've been playing with this, but so far + without a conclusion or fix. -- ci: fix renovate config for WolfSSL/WolfSSH tagging scheme + 15. Runtime/test failures with GCC - WolfSSL/WolfSSH use a different versioning scheme; - stable builds end with `-stable`. Renovate requires - some extra configuration to extract the version - from these types of tags. + I couldn't find the reason for most of this. A bunch of RTSP tests fail + with GCC. SecureTransport + HTTP/2 is failing a bunch of tests. With + OpenSSL it fails two of those. SecureTransport builds also fail one DoH + test. - Closes #13644 + 16. Runtime/test failure in llvm/clang -- ci: set semantic type as CI and include digests as CI operations + AppleIDN support received a fix with two more remaining. - Replace "chore" with "ci" for renovate's semantic - type, and include digests with "pin" and - "pinDigest" as ci operations. + fd0250869f7837e4a48d7e6f54cc0801ad3820e8 #14179 #14176 IDN: fix ß with Apple + IDN - Closes #13644 + 17. Other issues found and fixed while working on this: -Daniel Stenberg (15 May 2024) + 2c15aa5765900d4351e733671a1c8c3785beee1a GHA/macos: delete misplaced ` + CFLAGS`, drop redundant CMake option + 80fb7c0bef209735ab352bf4afa31193a7bc65f1 #14126 configure: limit `SystemConfi + guration` test to non-c-ares, IPv6 builds + cfd6f43d6ca7e57670b422bab7bbf10221a2cf3e #14127 build: tidy up `__builtin_ava + ilable` feature checks (Apple) + bae555359979016999a9425a2d489f219a78abdd #14174 runtests: show name and keywo + rds for failed tests in summary + 09cdf7e5315711dea4ce7dcf5d99a4d41e7f658b #14178 cmake: delete unused `HAVE_LI + BSSH2`, `HAVE_LIBSOCKET` macros + d3595c74fab829f07ef44da1b7fc2f5668767020 #14186 configure: CA bundle/path det + ection fixes + 58772b0e082eda333e0a5fc8fb0bc7f17a3cd99c #14187 runtests: set `SOURCE_DATE_EP + OCH` to fix failing around midnight + 18f1cd7a77c4beecfd977d43f55634eb593ac99e #14183 tests: sync feature names wit + h `curl -V` + 4c22d97be786ed801e050da6872dd3143d6d0a59 #14181 build: use `#error` instead o + f invalid syntax -- DEPRECATE.md: TLS libraries without 1.3 support + Pending merge: - curl drops support for TLS libraries without TLS 1.3 capability after - May 2025. +Daniel Stenberg (19 Jul 2024) - It requires that a curl build using the library should be able to - negotiate and use TLS 1.3, or else it is not good enough. We support a - vast amount of other TLS libraries that are likely to satisfy users - better. +- RELEASE-NOTES: synced - Closes #13544 +Stefan Eissing (18 Jul 2024) -- Revert "ci: update nghttp2/nghttp2 to v1.62.0" +- lib: send eos flag - This reverts commit 14f2c767555b7598d7783ccd9093670b84d28488. + Adds a `bool eos` flag to send methods to indicate that the data is the + last chunk the invovled transfer wants to send to the server. - We need to also upgrade the C++ compiler for that bump to work. + This will help protocol filters like HTTP/2 and 3 to forward the + stream's EOF flag and also allow to EAGAIN such calls when buffers are + not yet fully flushed. - Closes #13656 + Closes #14220 -renovate[bot] (15 May 2024) +Bhanu Prakash (18 Jul 2024) -- Dockerfile: update debian digest to 911821c +- mbedtls: correct the error message for cert blob parsing failure - Closes #13629 + Closes #14224 -- ci: update gnutls/gnutls to v3.8.5 +Daniel Stenberg (18 Jul 2024) - Closes #13640 +- curldown: make 'added-in:' a mandatory header field -- ci: update awslabs/aws-lc to v1.26.0 + - generate AVAILABILITY manpage sections automatically - for consistent + wording - Closes #13647 + - allows us to double-check against other documumentation (symbols-in-version + s + etc) -- ci: update cloudflare/quiche to v0.21.0 + - enables proper automation/scripting based on this data - Closes #13648 + - lots of them were wrong or missing in the manpages -- ci: update libressl-portable/portable to v3.9.2 + - several of them repeated (sometimes mismatching) backend support info - Closes #13649 + Add test 1488 to verify "added-in" version numbers against + symbols-in-versions. -- ci: update nghttp2/nghttp2 to v1.62.0 + Closes #14217 - Closes #13650 +Stefan Eissing (18 Jul 2024) -- ci: update ngtcp2/nghttp3 to v1.3.0 +- doh: fix cleanup - Closes #13651 + When removing an easy handle that had DoH sub-easy handles going, those + were not removed from the multi handle. Their memory was reclaimed on + curl_easy_cleanup() of the owning handle, but multi still had them in + their list. -- ci: update ngtcp2/ngtcp2 to v1.5.0 + Add `Curl_doh_close()` and `Curl_doh_cleanup()` as common point for + handling the DoH resource management. Use the `multi` present in the doh + handles (if so), for removal, as the `data->multi` might already have + been NULLed at this time. - Closes #13652 + Reported-by: 罗朝辉 + Fixes #14207 + Closes #14212 -Max Dymond (14 May 2024) +Daniel Stenberg (18 Jul 2024) -- ci: handle git submodules for mbedTLS +- tests/scripts: call it 'manpage' (single word) -- ci: reconfigure renovate + Mostly in comments - - set prefix for github actions updates to be gha: - - set prefix for other renovate actions to be ci: - - disable debian updates in linux-old.yml + Closes #14216 -Viktor Szakats (14 May 2024) +Alex Snast (18 Jul 2024) -- tidy-up: whitespace [ci skip] +- http/3: resume upload on ack if we have more data to send -- warnless: delete orphan declarations + Currently we're waiting for sendbuf_len_in_flight to hit zero before + resuming upload which means we're blocking and waiting for _all_ acks to + arrive before sending more data. This causes significant delays especially + when ack delay is used on the server side. - Follow-up to 358f7e757781857c4b498a68634726609fa3884a #11932 - Closes #13639 + The fix addresses several issues in h3 over ngtcp2: + - On ack we now call nghttp3_conn_resume_stream() when we have more + data to send. + - upload_left was incorrectly computed on CF_CTRL_DATA_DONE_SEND as + we need to subtract the ammount of data we have in flight. + - Remove upload_blocked_len as we Curl_bufq_write call will do the + right thing when called from cf_ngtcp2_send. -Daniel Stenberg (14 May 2024) + Fixes #14198 + Closes #14209 -- BUG-BOUNTY.md: clarify the third party situation +Daniel Stenberg (18 Jul 2024) - We do not pay bounties for problems in other libraries. +- idn: make macidn fail before trying conversion if name too long - Closes #13560 + - double the max name length to 512 bytes -Stefan Eissing (14 May 2024) + Closes #14215 -- http tests: in CI skip test_02_23* for quiche +z2_ (18 Jul 2024) - For unknown reasons, these tests fail in CI often, but run fine locally. - Skip them in CI to avoid unrelated PRs to have failures. +- idn: tweak buffer use when converting with macidn - Closes #13638 + Closes #14215 -Daniel Gustafsson (14 May 2024) +Orgad Shaneh (18 Jul 2024) -- hsts: explicitly skip blank lines +- lib: add failure reason on bind errors - Keep blank lines or lines containing only whitespace to make it all - the way to the more expensive sscanf call in hsts_add. + Closes #14221 - Closes: #13603 - Reviewed-by: Daniel Stenberg +Stefan Eissing (18 Jul 2024) -- autotools: Only probe for SGI MIPS compilers on IRIX +- pytests: scorecard upload tests - MIPSPro and the predecessor compiler which was part of the IDO (IRIS - Development Option) were only ever shipped on the SGI IRIX operating - system (with MIPSPro on 6.0+ which was released in 1994). Limit the - autoconf check to IRIX when probing for these compilers to save some - cycles on other platforms. + - add upload tests to scorecard, invoke with + > python3 tests/http/scorecard.py -u h1|h2|h3 + - add a reverse proxy setup from Caddy to httpd for + upload tests since Caddy does not have other PUT/POST handling + - add caddy tests in test_08 for POST/PUT + - increase read buffer in mod_curltest for larger reads - Closes: #13611 - Reviewed-by: Daniel Stenberg + Closes #14208 -Viktor Szakats (14 May 2024) +Viktor Szakats (18 Jul 2024) -- tests: fix test 1167 to skip digit-only symbols +- runtests: fix newline glitch in FAIL details - This avoids mistaking symbols with their numeric value when using - certain C preprocessors which output these numeric values at the - beginning of the line as part of an expression. + Follow-up to bae555359979016999a9425a2d489f219a78abdd #14174 - Seen on OpenBSD 7.5 + clang. +- runtests: show name and keywords for failed tests in summary - Example `test1167.pl -v` output, before this patch: + Useful to see what the numbers listed in the `TESTFAIL:` and `IGNORED:` + lines mean. Also list test keywords to help catching failure patterns. + + Example: ``` - Source: cpp /home/runner/work/curl/curl/tests/../include/curl/curl.h - Symbol: 20000 - Line #3835: 20000 + 142, - [...] - Bad symbols in public header files: - 20000 - [...] + FAIL 1034: 'HTTP over proxy with malformatted IDN host name' HTTP, HTTP GET, + HTTP proxy, IDN, FAILURE, config file + FAIL 1035: 'HTTP over proxy with too long IDN host name' HTTP, HTTP GET, HTTP + proxy, IDN, FAILURE + + TESTFAIL: These test cases failed: 1034 1035 ``` - Ref: https://github.com/curl/curl/actions/runs/9069136530/job/24918015357#ste - p:3:7513 - Ref: #13583 - Closes #13634 + Closes #14174 -Daniel Stenberg (14 May 2024) +Tal Regev (16 Jul 2024) -- lib: call Curl_strntolower instead of doing crafted loops +- GHA/windows: add MSVC wolfSSL job with test - Closes #13627 + Fix the file of wolfssl.c because of this warning/error: + ``` + curl\lib\vtls\wolfssl.c(1017,42): error C2220: the following warning is treat + ed as an error [curl\bld\lib\libcurl_object.vcxproj] + curl\lib\vtls\wolfssl.c(1017,42): warning C4267: 'function': conversion from + 'size_t' to 'unsigned long', possible loss of data [curl\bld\lib\libcurl_obje + ct.vcxproj] + ``` -- setopt: acknowledge errors proper for CURLOPT_COOKIEJAR + `size_t` in MSVC is different. Change it to `unsigned long` because + `wolfSSL_ERR_error_string_n` last argument is defined as + `unsigned long`. - Error out on error, do not continue. + Closes #14193 - Closes #13624 +Viktor Szakats (16 Jul 2024) -- vtls: remove duplicate assign +- cmake: delete unused `HAVE_LIBSSH2`, `HAVE_LIBSOCKET` macros - Curl_ssl_peer_cleanup() already clears the ->sni field, no point in - assigning it again. + - `HAVE_LIBSSH2`: unused in source. Not defined in CMake. - Spotted by CodeSonar + - `HAVE_LIBSOCKET`: unused in source. Used internally in CMake. - Closes #13626 + autotools sets them implicitly, so add them to the flag comparison + ignore-list. -Max Dymond (13 May 2024) + Closes #14178 -- Group all non-major updates together to reduce PR spam +- cmake: create `configurehelp.pm` like autotools does -- Add the remainder of the workflows + Required by tests 1119 and 1167 to run a C preprocessor. -- Add some basic versioning for some workflows to check whether this is detecte - d properly + Tested OK: https://github.com/curl/curl/actions/runs/9915343826 -renovate[bot] (13 May 2024) + Besides Apple, it also supports any gcc and clang builds, and MSVC. + For other platforms, it defaults to `cpp` (like autotools). -- Add renovate.json + Follow-up to efc2c5184d008fe2e5910fd03263e1ab0331d4e6 #14124 + Cherry-picked from #14097 + Closes #14129 -Daniel Stenberg (13 May 2024) +- cmake: sync CA bundle/path detection with autotools -- vauth: make two functions void that always just returned OK + - skip the entire CA logic if no selected TLS backend support CA + certs/bundles. + Follow-up to 082bb41311a832ae1b83bb8fe1dfdefcf4e68ea5 #2545 - Removes the need to check return values when they can never fail. + - sync bundle path detection logic with `./configure`. - Pointed out by CodeSonar + - fix to not auto-detect CA bundle/path on Windows. - Closes #13621 + - fix to reflect that BearSSL has CA bundle support. -- setopt: remove check for 'option' that is always true + - show the detected bundle path (as with the cert bundle). - - make sure that passing in option set to NULL clears the fields - correctly + - tidy up CMake syntax, fix typos in comments. - - remove the weird second take if Curl_parse_login_details() returns - error + Closes #14182 - Follow-up to 7333faf00bf25db7cd1e0012d6b140 +- configure: CA bundle/path detection fixes - Spotted by CodeSonar + - fix to not auto-detect CA bundle/path on Windows. - Closes #13619 + - two checks missed BearSSL, but they were only run for supported + TLS backends anyway. Delete these redundant checks. -Viktor Szakats (13 May 2024) + - fix typos in a comment nearby. -- tests: tidy up types in server code + Follow-up to 082bb41311a832ae1b83bb8fe1dfdefcf4e68ea5 #2545 + Closes #14186 - Cherry-picked from #13489 - Closes #13610 +- runtests: set `SOURCE_DATE_EPOCH` to fix failing around midnight -Daniel Stenberg (13 May 2024) + To make sure that `managen` called by test 1706 uses the same date as + the test expects in the `%DATE` macro. -- setopt: make the setstropt_userpwd args compulsory + Before this patch when tests started running before UTC midnight and + reached test 1706 after, these dates were different and the test failed. - They were always used so no point in allowing them to be optional. + Follow-up to 0e73b69b3dd6d174226c60406d3c4266754d70f8 + Fixes #14173 + Closes #14187 - follow-up to 0e37b42dc956bd8a +- GHA/windows: verify 1448 2046 2047 in IDN Unicode jobs - Closes #13608 - Reviewed-by: Daniel Gustafsson + These IDN tests pass with Unicode and fail without. -- RELEASE-NOTES: synced + Follow-up to cb22cfca69bded45bf7f9c72c8e6764990490f11 #14077 + Closes #14188 -Daniel Gustafsson (13 May 2024) +- tests: sync feature names with `curl -V` -- websocket: Avoid memory leak in error path + Some feature names used in tests had minor differences compared to + the well-known ones from `curl -V`. This patch syncs them to make test + results easier to grok. - In the errorpath for randstr being too long to copy into the buffer - we leak the randstr when returning CURLE_FAILED_INIT. Fix by using - an explicit free on randstr in the errorpath. + Closes #14183 - Closes: #13602 - Reviewed-by: Daniel Stenberg +Stefan Eissing (15 Jul 2024) -- hsts: Remove single-use single-line function +- sendf: fix CRLF conversion of input - The hsts_entry() function contains of a single line and is only - used in a single place in the code, so move the allocation into - hsts_create instead to improve code readability. C code usually - don't use the factory abstraction for object creation, and this - small example wasn't following our usual code style. + When CRLF line end conversion was enabled (--crlf), input after the last + newline in the upload buffer was not sent, if the buffer contained a + newline. - Closes: #13604 - Reviewed-by: Daniel Stenberg + Reported-by: vuonganh1993 on github + Fixes #14165 + Closes #14169 -Viktor Szakats (12 May 2024) +- test2600: disable on win32 -- lib: bump hash sizes to `size_t` + - disbable this test on WIN32 platforms. It uses the file describtor '1' + as valid socket without events. Not portable. + - reduce trace output somewhat on other runs - Follow-up to cc907e80a2498c0599253271a6f657f614b52a4e #13502 - Cherry-picked from #13489 - Closes #13601 + Fixes #14177 + Reported-by: Viktor Szakats + Closes #14191 -- tests: make the unit test result type `CURLcode` +- smtp: for starttls, do full upgrade - Before this patch, the result code was a mixture of `int` and - `CURLcode`. + - make sure the TLS handshake after a successful STARTTLS command is + fully done before further sending/receiving on the connection. - Also adjust casts and fix a couple of minor issues found along the way. + Reported-by: tomy2105 on github + Fixes #14166 + Closes #14190 - Cherry-picked from #13489 - Closes #13600 +Daniel Stenberg (14 Jul 2024) -- appveyor: tidy-ups +- RELEASE-NOTES: synced - - delete a duplicate line. - - simplify a `make` call. - - merge two `if` branches. - - reorder autotools options for clarity. - - add `--enable-warnings` where missing (it's also the default.) - - add empty lines to YAML for readability. - - use lowercase install prefix/directory. +Viktor Szakats (14 Jul 2024) - Closes #13598 +- build: use `#error` instead of invalid syntax -Daniel Stenberg (12 May 2024) + It reduces configure log noise. -- docs/cmdline-opts: mention STARTTLS for --ssl and --ssl-reqd + Follow-up to 20c1b2d75ee38189ffa75d21ed04108e1e0630ae #13287 + Closes #14181 - ... since users might look for those terms in the manpage. +Daniel Stenberg (14 Jul 2024) - Closes #13590 +- libcurl-docs: make option lists alpha-sorted -- setopt: warn on Curl_set*opt() uses not using the return value + The man pages for curl_easy_getinfo, curl_easy_setopt and + curl_multi_setopt now feature the lists of options alphabetically + sorted. Test 1139 verify that they are. - And switch the invokes that would "set" NULL to instead just plainly - free the pointer, as those were otherwise the invokes that would ignore - the return code. And possibly confuse static code analyzers. + The curl_multi_setopt page also got brief explanations of the listed + options. - Closes #13591 + Closes #14156 -Orgad Shaneh (12 May 2024) +Christian Schmitz (14 Jul 2024) -- autotools: delete unused functions +- IDN: fix ß with AppleIDN - Closes #13605 + Add flags UIDNA_NONTRANSITIONAL_TO_ASCII and + UIDNA_NONTRANSITIONAL_TO_UNICODE to encode ß correctly. -Viktor Szakats (11 May 2024) + It fixes test 165. -- examples: fix/silence `-Wsign-conversion` + Reported-by: Viktor Szakats + Bug: #14176 + Closes #14179 - - extend `FD_SET()` hack to all platforms (was only Cygwin). - Warnings may also happen in other envs, e.g. OmniOS. - Ref: https://github.com/libssh2/libssh2/actions/runs/8854199687/job/2431676 - 2831#step:3:2021 +Viktor Szakats (14 Jul 2024) - - tidy-up `CURLcode` vs `int` use. +- cmake: fix builds with detected libidn2 lib but undetected header - - cast an unsigned to `long` before passing to `curl_easy_setopt()`. + It caused IDN to appear in `curl-config`, `libidn2` referenced from + `libcurl.pc`, fail to fallback to `pkg-config` detection. But libidn2 + not actually used. - Cherry-picked from #13489 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - Closes #13501 + It came up in macOS CI builds after enabling cmake build tests. It + remained hidden for a while due to setting `-DUSE_APPLE_IDN=ON`. -Orgad Shaneh (11 May 2024) + (The half-detection of Homebrew libidn2 was the result of configuring + with `-DCMAKE_EXE_LINKER_FLAGS=-L$(brew --prefix)/lib`, to fix + linking GnuTLS that needs the `nettle` lib from the brew prefix.) -- cmake: fix `HAVE_IOCTLSOCKET_FIONBIO` test with gcc 14 + ``` + FAIL 1014: [Compare curl --version with curl-config --features] curl-config + ``` + Ref: https://github.com/curl/curl/actions/runs/9919357748/job/27405080722 - The function signature has had u_long flags since ever. This is how it - is defined in the documentation, and implemented in MinGW. + Cherry-picked from #14097 + Closes #14175 - The code that uses ioctlsocket in nonblock.c also has unsigned long. +- cmake: fix building `unit1600` due to missing `ssl/openssl.h` - Error: - CurlTests.c:275:41: error: passing argument 3 of 'ioctlsocket' from incompati - ble pointer type [-Wincompatible-pointer-types] - 275 | if(0 != ioctlsocket(0, FIONBIO, &flags)) - | ^~~~~~ - | | - | int * - In file included from CurlTests.c:266: - /opt/mxe/usr/i686-w64-mingw32.static/include/winsock2.h:1007:76: note: expect - ed 'u_long *' {aka 'long unsigned int *'} but argument is of type 'int *' - 1007 | WINSOCK_API_LINKAGE int WSAAPI ioctlsocket(SOCKET s,__LONG32 cmd,u_ - long *argp); - | ~~ - ~~~~~~^~~~ + In specific builds configs, cmake failed to build test `unit1600`, + due missing an OpenSSL (or wolfSSL) header. - Closes #13578 + The test code relies on `lib/curl_ntlm_core.h`, which in turn included + TLS library headers. But, dependency header directories are not setup + in cmake for tests, because they should not normally be needed. -Jay Satiro (10 May 2024) + The issue was hidden in most builds because TLS headers are usually + found under the system prefix. One counterexample is macOS + Homebrew + LibreSSL builds, where OpenSSL is purposefully unlinked from there to + avoid a mixup with LibreSSL that resides under its own prefix. It was + also hidden in autotools, possibly because it sets up header directories + globally, tests included. -- ftp: fix build for CURL_DISABLE_VERBOSE_STRINGS + The actual bug however is that `lib/curl_ntlm_core.h` should not include + TLS headers. None of its internal users need it, and `curl_ntlm_core.c` + included them already directly. - This is a follow-up to b7c7dffe which changed the FTP state change - verbose debug text (aka infof) to tracing debug text (aka trc). + Fix it by deleting the TLS header includes from this internal header. - Prior to this change if libcurl was without DEBUGBUILD and built with - CURL_DISABLE_VERBOSE_STRINGS (ie --disable-verbose) the build would - error. + Fixes: + ``` + In file included from curl/tests/unit/unit1600.c:27: + curl/lib/curl_ntlm_core.h:32:12: fatal error: 'openssl/ssl.h' file not found + # include + ^~~~~~~~~~~~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9912684737/job/27388041520#ste + p:12:1694 - Caught by Circle CI job openssl-no-verbose. + Follow-up to 48eb71ade41d4b37f416b643063cab846ac027a2 #10322 + Cherry-picked from #14097 + Closes #14172 -- lib: clear the easy handle's saved errno before transfer +- sectransp: fix clang compiler warnings, stop silencing them - - Clear data->state.os_errno before transfer. + Fix `-Wpointer-bool-conversion` warnings with the method suggested by + both Apple clang and mainline llvm. This was already tried and dropped + in #1705 (in year 2017), but the issue reported there no longer + replicates. - - Explain the change in behavior in the CURLINFO_OS_ERRNO doc. + Verified with Apple clang 14, llvm 15, llvm 18 and gcc 11, 14 that the + generated objects are bit by bit identical before and after this patch. - - Add to the CURLINFO_OS_ERRNO doc the list of libcurl network-related - errors that may cause the errno to be saved. + Also: + - stop silencing `-Wtautological-pointer-compare`. This warning don't + seem to be appearing anymore (with or without this patch), at least + with the tested compilers and SDKs (clang 13.1.6-16.0.0beta, llvm 15, + 18, gcc 11, 14) and minimum macOS target of 10.8. Older targets fail + to build curl with SecureTransport. + + - silence `-Wunreachable-code` for clang only. Previously I applied it + also to GCC, by mistake. + Ref: https://github.com/curl/curl/pull/12331/commits/8d7172d20a48ebc6c1b1d9 + 4a76e2c5fb19dd9bfa + + Apple clang `-Wpointer-bool-conversion`: + ``` + curl/lib/vtls/sectransp.c:1103:6: error: address of function 'SSLCreateContex + t' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] + if(SSLCreateContext) { /* use the newer API if available */ + ~~ ^~~~~~~~~~~~~~~~ + curl/lib/vtls/sectransp.c:1103:6: note: prefix with the address-of operator t + o silence this warning + if(SSLCreateContext) { /* use the newer API if available */ + ^ + & + ``` + Ref: https://github.com/curl/curl/actions/runs/9819538439/job/27113201384#ste + p:8:382 - data->state.os_errno is saved before libcurl returns a network-related - failure such as connection failure. It is accessible to the user via - CURLINFO_OS_ERRNO so they can get more information about the failure. + llvm `-Wpointer-bool-conversion`: + ``` + curl/lib/vtls/sectransp.c:2663:8: error: address of function 'SSLCreateContex + t' will always evaluate to 'true' [-Werror,-Wpointer-bool-conversion] + if(SSLCreateContext) + ~~ ^~~~~~~~~~~~~~~~ + curl/lib/vtls/sectransp.c:2663:8: note: prefix with the address-of operator t + o silence this warning + if(SSLCreateContext) + ^ + & + ``` + Ref: https://github.com/curl/curl/actions/runs/9819538439/job/27113200291#ste + p:8:417 - Prior to this change it wasn't cleared before transfer, so if a user - retrieved the saved errno it could be from a previous transfer. That is - because an errno is not always saved for network-related errors. + gcc still needs `-Waddress` suppressed to avoid these: + ``` + curl/lib/vtls/n/sectransp.c: In function 'getsubject': + curl/lib/vtls/n/sectransp.c:379:6: warning: the address of 'SecCertificateCop + yLongDescription' will always evaluate as 'true' [-Waddress] + 379 | if(&SecCertificateCopyLongDescription) + | ^ + [...] + ``` - Closes https://github.com/curl/curl/pull/13574 + Follow-up to 59cadacfcc1d39472245979cdbd614c7a9af6f0d #14128 + Follow-up to af271ce9b9717ba289417e9cbb7f278c2a12f959 #1722 + Follow-up to 2b7ce3f56dfede107113c6de7d0ca457109d3eda #1706 + Cherry-picked from #14097 + Closes #14162 -Stefan Eissing (10 May 2024) +- CI/circleci: config tidy-ups, bump up test parallelism -- ftp: add tracing support + - bump parallel test for Linux jobs. + Credit-to: Dan Fandrich + Cherry-picked from #11510 + - bump parallel test for macOS jobs. + - drop no longer necessary `-Wno-vla` option. + - fold long lines. + - drop `--enable-maintainer-mode` `./configure` option. + - replace a hard-coded prefix with `brew --prefix`. + - update documentation link. + - move `--enable-debug` in front. + - tidy up quotes. - - add `Curl_trc_feat_ftp` for tracing via trace config - - add macro CURL_TRC_FTP(data, fmt, ...) - - replace DEBUGF(infof()) statements in ftp.c by CURL_TRC_FTP() - - always trace FTP connection state + Closes #14171 - Closes #13580 +- GHA/windows: re-add gsasl to MSVC jobs -Daniel Stenberg (10 May 2024) + Now that the package reached the CI runner image. -- http: remove redundant check + Follow-up to f99c08dba40307c07341013ff5f71fa8e3464ffc #14090 + Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979 - Spotted by CodeSonar + Closes #14170 - Closes #13582 +- tidy-up: adjust casing of project names -Viktor Szakats (10 May 2024) + Mostly TLS/SSH project name. -- ldap: fix unused variables (seen on OmniOS) + Closes #14160 - ``` - ../../lib/ldap.c: In function 'ldap_do': - ../../lib/ldap.c:380:11: error: unused variable 'ldap_ca' [-Werror=unused-v - ariable] - 380 | char *ldap_ca = conn->ssl_config.CAfile; - | ^~~~~~~ - ../../lib/ldap.c:379:9: error: unused variable 'ldap_option' [-Werror=unuse - d-variable] - 379 | int ldap_option; - | ^~~~~~~~~~~ - ``` - Ref: https://github.com/curl/curl/actions/runs/9033564377/job/24824192730#ste - p:3:6059 +Daniel Stenberg (12 Jul 2024) - Ref: #13583 - Closes #13588 +- ISSUE_TEMPLATE/docs: correct the field identifiers -Daniel Stenberg (10 May 2024) +Stephen Farrell (12 Jul 2024) -- url: make parse_login_details use memdup0 +- doh: fix leak and zero-length HTTPS RR crash - Also make the user and password arguments mandatory, since all code - paths in libcurl used them anyway. + This PR fixes a leak and a crash that can happen when curl encounters + bad HTTPS RR values in DNS. We're starting to do better testing of that + kind of thing and e.g. have published bad HTTPS RR values at + dodgy.test.defo.ie. - Adapted unit test case 1620 to the new rules. + Closes #14151 - Closes #13584 +Daniel Stenberg (12 Jul 2024) -Orgad Shaneh (10 May 2024) +- curl_global_init.md: polish the thread-safe wording -- digest: replace strcpy for empty string with simple assignment + Since this has been thread-safe for two years now, few users actually + are hurt by the previous unsafe ways. - Closes #13586 + Closes #14158 -Viktor Szakats (10 May 2024) +Viktor Szakats (12 Jul 2024) -- autotools: fix `HAVE_IOCTLSOCKET_FIONBIO` test for gcc 14 +- GHA: FreeBSD 14.1, actions bump + - bump FreeBSD to 14.1 + + - update cross-platform-actions/action action to v0.25.0 + + Closes #14157 + Closes #14164 + +- build: fix llvm 17 and older + macOS SDK 14.4 and newer + + Fixup faulty target macro initialization in macOS SDK since v14.4 (as of + 15.0 beta). The SDK target detection in `TargetConditionals.h` correctly + detects macOS, but fails to set the macro's old name `TARGET_OS_OSX`, + then continues to set it to a default value of 0. Other parts of the SDK + still rely on the old name, and with this inconsistency our builds fail + due to missing declarations. It happens when using mainline llvm older + than v18. Later versions fixed it by predefining these target macros, + avoiding the faulty dynamic detection. gcc is not affected (for now) + because it lacks the necessary dynamic detection features, so the SDK + falls back to a codepath that sets both the old and new macro to 1. + + Also move the `TargetConditionals.h` include to the top of to make sure + including it also for c-ares builds, combined with SecureTransport or + other curl features that may call use an Apple SDK. + + Before this patch, affected build combinations (e.g. in GHA runners, + llvm@15 + Xcode 15.3, 15.4, 16.0 with their default SDKs + + SecureTransport) fail with: ``` - conftest.c:152:41: error: passing argument 3 of 'ioctlsocket' from incompatib - le pointer type [-Wincompatible-pointer-types] - 152 | if(0 != ioctlsocket(0, FIONBIO, &flags)) - | ^~~~~~ - | | - | int * + error: use of undeclared identifier 'noErr' + or 'SecCertificateCopyLongDescription' + or 'SecItemImportExportKeyParameters' + or 'SecExternalFormat' + or 'SecExternalItemType' + or 'SEC_KEY_IMPORT_EXPORT_PARAMS_VERSION' ``` - Reported-by: LigH - Fixes #13579 - Closes #13587 + Example: + ``` + curl/lib/vtls/sectransp.c:311:18: error: use of undeclared identifier 'noErr' + OSStatus rtn = noErr; + ^ + curl/lib/vtls/sectransp.c:379:7: error: use of undeclared identifier 'SecCert + ificateCopyLongDescription' + if(&SecCertificateCopyLongDescription) + ^ + curl/lib/vtls/sectransp.c:381:7: error: call to undeclared function 'SecCerti + ficateCopyLongDescription'; ISO C99 and later do not support implicit functio + n declarations [-Werror,-Wimplicit-function-declaration] + SecCertificateCopyLongDescription(NULL, cert, NULL); + ^ + curl/lib/vtls/sectransp.c:380:25: error: incompatible integer to pointer conv + ersion assigning to 'CFStringRef' (aka 'const struct __CFString *') from 'int + ' [-Wint-conversion] + server_cert_summary = + ^ + [...] + ``` + Ref: https://github.com/curl/curl/actions/runs/9893867519/job/27330135969#ste + p:10:22 -- CI: ignore test 286 on Appveyor gcc 7 build + llvm v18 patches implementing the predefined macros: + https://github.com/llvm/llvm-project/pull/74676 + https://github.com/llvm/llvm-project/commit/6e1f19168bca7e3bd4eefda50ba03eac8 + 441dbbf + https://github.com/llvm/llvm-project/pull/82833 + https://github.com/llvm/llvm-project/commit/e5ed7b6e2fd368b722b6359556cd01258 + 81e7638 - Disabled earlier for gcc 9 builds. gcc 7 uses the same runner and - prone to similar intermittent failures. + Cherry-picked from #14097 + Closes #14159 - Follow-up to f1e05a6e6e7225fa09952abb2c935ae1abe44f45 #12106 #12040 - Closes #13575 +- macos: undo `availability` macro enabled by Homebrew gcc -Daniel Stenberg (10 May 2024) + Homebrew gcc builds starting with 12.4.0, 13.3.0 and 14.1.0 enabled + the `availability` attribute. -- cf-socket: don't try getting local IP without socket + This broke builds because the way the Apple SDK uses attributes (when + available) are incompatible with how gcc accepts them. Causing these + errors: + ``` + error: attributes should be specified before the declarator in a function d + efinition + error: expected ',' or '}' before + ``` - In cf_tcp_connect(), it might fail and not get a socket assigned to - ctx->sock but set_local_ip() is still called which would make - getsockname() get invoked with a negative file desriptor and fail. + Upstream commits implementing the `availability` macro: + gcc-12: https://github.com/iains/gcc-12-branch/commit/fd5530b7cb0012bf4faeddd + 45e13054a1dfa6783 + gcc-13: https://github.com/iains/gcc-13-branch/commit/cb7e4eca68cfc4763474e2e + b0935a844458842a8 + gcc-14: https://github.com/iains/gcc-14-branch/commit/ff62a108865a6403f501738 + 0d7018250c1d3306f - By adding this check, set_local_ip() will now instead blank out the - fields correctly. + The project above is a Darwin gcc compatibility pack, that is applied + to Homebrew gcc builds. - Spotted by CodeSonar + This patch works by redefining the `availability` macro to an invalid + value, making `__has_attribute(availability)` checks fail, stopping + Apple SDK from inserting the incompatible attributes. - Closes #13577 + It also replaces the previous, local workaround for `lib/macos.c`. -- tool_getparam: remove two redundant conditions + Example with gcc 12.4.0 with macOS SDK 14.0 (Xcode 15.0.1): + ``` + In file included from /MacOSX14.0.sdk/System/Library/Frameworks/ + CoreFoundation.framework/Headers/CoreFoundation.h:54, + from /MacOSX14.0.sdk/System/Library/Frameworks/ + SystemConfiguration.framework/Headers/SCDynamicStoreCopySpecific.h:30, + from /Users/runner/work/curl/curl/lib/macos.c:33, + from /Users/runner/work/curl/curl/build/lib/CMakeFiles/libcu + rl_shared.dir/Unity/unity_0_c.c:244: + /MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framewo + rk/Headers/CFUserNotification.h:126:1: error: attributes should be specified + before the declarator in a function definition + 126 | CF_INLINE CFOptionFlags CFUserNotificationCheckBoxChecked(CFIndex i) + API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOp + tionFlags)(1UL << (8 + i)));} + | ^~~~~~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9787982387/job/27025351601?pr= + 14096#step:7:18 - When getstr() does not return error, it returns a valid pointer. + The gcc vs. llvm/clang incompatibility possibly tracked here upstream: + https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108796 + More info: + https://github.com/llvm/llvm-project/issues/81767 + https://github.com/gcc-mirror/gcc/commit/8433baadec88e5f31fa141b6d78094e912 + 56079d + https://discourse.llvm.org/t/changing-attribute-ast-printing-location-for-g + cc-compatibility/73215 + https://reviews.llvm.org/D159362 - Spotted by CodeSonar + Follow-up to db135f8d7207b20d531e7e2100a49f3e16bdcfab #14119 + Ref: https://github.com/curl/curl/pull/14091#issuecomment-2222703468 + Fixes #13700 + Cherry-picked from #14097 + Closes #14155 - Closes #13576 +Daniel Stenberg (11 Jul 2024) -Stefan Eissing (10 May 2024) +- ISSUE_TEMPLATE/docs: add a separate GitHub issue template for documentation -- quiche: trust its timeout handling + As such problems don't really fit the code related template - - set the idle timeout transport parameter - in milliseconds as documented by quiche - - do not calculate the idle timeout, rely on - quiche handling it + Closes #14161 - Closes #13581 +Dan Fandrich (11 Jul 2024) -Daniel Stenberg (10 May 2024) +- DISTROS: add AlmaLinux package source link -- dmaketgz: accept a SOURCE_DATE_EPOCH as an second argument +Viktor Szakats (11 Jul 2024) - to make it easier to reproduce a tarball +- GHA/windows: ignore FTP test results for old-mingw-w64 [ci skip] - Closes #13573 + Missed from previous commit. They are flaky here as well. -- RELEASE-NOTES: synced + Follow-up to 0b81eccd22fb915aa6b679c0fd23a8a89332dc9e -Stefan Eissing (10 May 2024) +Daniel Stenberg (11 Jul 2024) -- h3/ngtcp2: improve error handling +- libcurl-easy.md: now *more* than 300 options - - identify ngtcp2 and nghttp3 error codes that are fatal - - close quic connection on fatal errors - - refuse further filter operations once connection is closed - - confusion about the nghttp3 API. We should close the QUIC stream on - cancel and not use the nghttp3 calls intended to be invoked when the - QUIC stream was closed by the peer. + it previously said "almost 300". - Closes #13562 + Also cleaned up the language somewhat. -Jay Satiro (10 May 2024) + Closes #14153 -- docs: fix some CURLINFO examples +Martin Peck (10 Jul 2024) - - improve getinfo result check for example sections: - CURLINFO_ACTIVESOCKET, CURLINFO_LASTSOCKET, CURLINFO_SSL_VERIFYRESULT, - CURLINFO_PROXY_SSL_VERIFYRESULT +- MANUAL.md: wrap two example urls that overrun styling - - fix getinfo result check for example sections: - CURLINFO_NUM_CONNECTS, CURLINFO_OS_ERRNO + Closes #14149 - - fix verify result check for example sections: - CURLINFO_PROXY_SSL_VERIFYRESULT +renovate[bot] (10 Jul 2024) - Bug: https://github.com/curl/curl/discussions/13557#discussion-6625507 - Reported-by: farazrbx@users.noreply.github.com +- GHA: update wolfSSL and mod_h2 - Closes https://github.com/curl/curl/pull/13559 + - wolfSSL/wolfssl to v5.7.2 + - icing/mod_h2 to v2.0.29 -Daniel Stenberg (9 May 2024) + Closes #14131 + Closes #14148 -- KNOWN_BUGS: gssapi library name + version is missing in curl_version_info() +Dominik Piątkowski (10 Jul 2024) - Closes #13492 - Closes #13570 +- docs: start markdown headers with capital letter where applicable -- krb5: use dynbuf + Closes #14115 - Closes #13568 +CMD (10 Jul 2024) -- managen: fix the option sort order +- hostip: skip error check for infallible function call - ... it used to strip off the .d file extension to sort correctly but - ever since the extension changed to .md the operation failed and the - sort got wrong. + Closes #14147 - Follow-up to 2494b8dd5175cee7f2e +Daniel Stenberg (10 Jul 2024) - Closes #13567 +- cf-socket: remove two "useless" assignments -Stefan Eissing (8 May 2024) + 'nread' is already -1, no need to assign it again -- GHA: repair the linux-old job + Pointed out by CodeSonar - package libc6_2.28-10+deb10u2_amd64.deb changed to - libc6_2.28-10+deb10u3_amd64.deb + Closes #14145 - Closes #13564 +Viktor Szakats (10 Jul 2024) -Viktor Szakats (8 May 2024) +- cmake: detect `libidn2` also via `pkg-config` -- appveyor: make gcc 6 mingw64 job build-only + Also: + - GHA/non-native: install `pkg-config` to detect libidn2 with cmake + on NetBSD and FreeBSD. + - GHA/non-native: tidy-up `curl --version` command if here. - This job has proven to be the flakiest of all, and it's also the oldest - Windows runner we had tests running on: 'Visual Studio 2015', that is - running on Windows Server 2012 R2: - https://www.appveyor.com/docs/windows-images-software/ + Cherry-picked from #14097 + Closes #14137 - Turn off tests on this job to help stabilizing CI runs. +- build: fix llvm 16 or older + Xcode 15 or newer, and gcc - This was also one of the slowest running job amongst the AppVeyor CI ones. + Xcode v15 (2023) or newer requires the built-in macro + `__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__`. This macro is missing from + mainline llvm versions released earlier. llvm v17 introduced it here: + https://github.com/llvm/llvm-project/commit/c8e2dd8c6f490b68e41fe663b44535a8a + 21dfeab - Flakiness data: - https://testclutch.curl.se/static/reports/summary.html - Entries: - Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity - (curl) [current] - Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static (curl) [fo - rmer] + This patch defines the missing macro when the necessary conditions + align, by using the value via the macro's old name. - Closes #13566 + The issue affected SecureTransport builds: The SecureTransport code, + `lib/md4.c` and `lib/md5.c`. -Stefan Eissing (8 May 2024) + Existing gcc versions (as of v14) also don't define this macro, so apply + the patch to it as well. Even though gcc is incompatible in other ways, + so this isn't fixing an actual curl build case that I could find yet. -- unit2604: use alloc instead of overlong string const + GHA macOS runner images have llvm v15 pre-installed, which broke builds + when building with an affected Xcode: + ``` + curl/lib/md4.c:80:14: error: '__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__' is not + defined, evaluates to 0 [-Werror,-Wundef] + (__MAC_OS_X_VERSION_MIN_REQUIRED < 101500)) || \ + ^ + /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Dev + eloper/SDKs/MacOSX14.2.sdk/usr/include/AvailabilityInternal.h:40:53: note: ex + panded from macro '__MAC_OS_X_VERSION_MIN_REQUIRED' + #define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_OS_VERSION_ + MIN_REQUIRED__ + ^ + In file included from curl/build/lib/CMakeFiles/libcurl_shared.dir/Unity/unit + y_0_c.c:250: + curl/lib/md5.c:75:14: error: '__ENVIRONMENT_OS_VERSION_MIN_REQUIRED__' is not + defined, evaluates to 0 [-Werror,-Wundef] + (__MAC_OS_X_VERSION_MIN_REQUIRED < 101500)) || \ + ^ + /Applications/Xcode_15.1.app/Contents/Developer/Platforms/MacOSX.platform/Dev + eloper/SDKs/MacOSX14.2.sdk/usr/include/AvailabilityInternal.h:40:53: note: ex + panded from macro '__MAC_OS_X_VERSION_MIN_REQUIRED' + #define __MAC_OS_X_VERSION_MIN_REQUIRED __ENVIRONMENT_OS_VERSION_ + MIN_REQUIRED__ + ^ + 2 errors generated. + ``` + Ref: https://github.com/curl/curl/actions/runs/9811974634/job/27095218578#ste + p:4:20 - Closes #13563 + Cherry-picked from #14097 + Closes #14134 -Daniel Gustafsson (8 May 2024) +- build: tidy up `__builtin_available` feature checks (Apple) -- bufq: remove duplicate word in comment + - sync detection snippet between autotools and cmake + It wasn't causing issues, but it's useful to avoid unnecessary + differences while debugging. - Inspired by 13552. + - cmake: limit check to `APPLE`. - Closes: #13554 - Reviewed-by: Daniel Stenberg + Ref: b05dc7eb3592305de9fa85640767f3dd2a8d4c93 #14122 + Cherry-picked from #14097 + Closes #14127 -Viktor Szakats (8 May 2024) +- configure: limit `SystemConfiguration` test to non-c-ares, IPv6 builds -- lib/cf-h1-proxy: silence compiler warnings (gcc 14) + The framework this check detects is necessary for the function + `SCDynamicStoreCopyProxies()` used in `lib/macos.c`. Non-c-ares, + IPv6-enabled builds touch this codepath. - They came up ealier with gcc 12 (Windows), but apparently gcc 14 is - still reporting them, also under Linux. + Limit the feature check for builds that actually need it. + It brings this in sync with CMake which already worked this way. + + Cherry-picked from #14097 + Closes #14126 + +- configure: fix `SystemConfiguration` detection + + Before this patch, `SystemConfiguration` detection failed due to this + error when compiling the detection snippet: ``` - /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c: In functi - on 'cf_h1_proxy_close': - /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1060:17: w - arning: null pointer dereference [-Wnull-dereference] - 1060 | cf->connected = FALSE; - /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1061:8: wa - rning: null pointer dereference [-Wnull-dereference] - 1061 | if(cf->ctx) { - | ~~^~~~~ - In function 'tunnel_free', - inlined from 'cf_h1_proxy_destroy' at /home/runner/work/curl-for-win/curl - -for-win/curl/lib/cf-h1-proxy.c:1053:3: - /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:198:27: wa - rning: null pointer dereference [-Wnull-dereference] - 198 | struct h1_tunnel_state *ts = cf->ctx; - | ^~ + /Applications/Xcode_15.3.app/Contents/Developer/Platforms/MacOSX.platform/Dev + eloper/SDKs/MacOSX.sdk/usr/include/TargetConditionals.h:140:50: error: missin + g binary operator before token "(" + 140 | #if !defined(__has_extension) || !__has_extension(define_target_os_ma + cros) + | ^ ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/8985369476/job/2467921 - 9528#step:3:6320 + Ref: https://github.com/curl/curl/actions/runs/9821817534/job/27117929218#ste + p:6:1079 - Fixes #13237 - Closes #13555 + It occured with gcc-11 when combined with macOS SDK 14.4 and 14.5 + (default SDKs in Xcode 15.3 and 15.4 respectively). It did not happen + with earlier releases. -Michał Antoniak (8 May 2024) + Despite the failure in `./configure`, `lib/macos.c` compiled with + Apple's `TargetConditionals.h` just fine. -- mbedtls: support TLS 1.3 + Turns out that including the `sys/types.h` header before the SDK + header fixes the error and makes the detection snippet compile. - Closes #13539 + Cherry-picked from #14097 + Closes #14130 -Daniel Stenberg (8 May 2024) +- build: sync warning options between autotools, cmake & compilers -- version: use msnprintf instead of strncpy + - cmake: enable Apple-specific `-Werror=partial-availability` to match + autotools. - - to ensure a terminating null byte - - to avoid zero-padding the target + - autotools: enable `-pedantic-errors` with llvm/clang to match gcc and + CMake. - debug code only + - autotools: enable `-Werror-implicit-function-declaration` for + llvm/clang to match gcc. - Closes #13549 + - cmake: enable `-Werror-implicit-function-declaration` to match + autotools. -- curl_path: make Curl_get_pathname use dynbuf + - move `-Wpointer-bool-conversion` from autotools to the local file + (`sectransp.c`) it was meant to apply. This way it applies to all + build methods. - ... instead of malloc and memcpy + - autotoos: show `CURL_CFLAG_EXTRAS` in the `./configure` summary. + (it may contain `-Werror` and/or `-pedentic-errors`.) - - unit test 2604 verifies Curl_get_pathname() + Cherry-picked from #14097 + Closes #14128 - Closes #13550 +- CI: simplify running curl with DLLs -- lib: make protocol handlers store scheme name lowercase + - update `PATH` instead of copying DLLs around. + - drop redundant `export` from `export PATH`. + - delete ending pathseps. - - saves a lowercase operation when the "[scheme]_proxy" name is - generated - - appears less "shouting" - - update test 970, 972, 1438 and 1536 + Closes #14143 - Closes #13553 +Alex Snast (9 Jul 2024) -- lib: remove two instances of "only only" messages +- wolfssl: use larger error buffer when formatting errors - Fixes #13551 - Reported-by: Lucas Nussbaum - Closes #13552 + Currently we're using WOLFSSL_MAX_ERROR_SZ to define the error buffer + size, this value is user defined which means it can be overwritten with + -DWOLFSSL_MAX_ERROR_SZ=512 when building wolfssl and this overwrite is + not exported to the users of wolfssl. -Pavel Pavlov (7 May 2024) + Instead of relying on WOLFSSL_MAX_ERROR_SZ we'll just use a 256 bytes + error buffer and use wolfSSL_ERR_error_string_n to fill it thus dropping + the dependency on WOLFSSL_MAX_ERROR_SZ altogether. -- asyn-thread: fix curl_global_cleanup crash in Windows + Closes #14114 - - Make sure that asynchronous resolves handled by Winsock are stopped - before WSACleanup is called. +Viktor Szakats (9 Jul 2024) - This is implemented by ensuring that when Curl_resolver_kill is called - (eg via multi_done) it will cancel the Winsock asynchronous resolve and - wait for the cancellation to complete. Winsock runs the asynchronous - completion routine immediately when a resolve is canceled. +- CI: bump FreeBSD Python packages - Prior to this change it was possible that during curl_global_cleanup - "a DNS resolver thread created by GetAddrInfoExW did not terminate yet, - however curl is already shutting down, deinitializing Winsock with - WSACleanup() leading to an access violation." + Closes #14141 - Background: +- GHA/curl-for-win: don't run if only another CI was changed - If libcurl is built with the asynchronous threaded resolver option for - Windows then it resolves in one of two ways. For Windows 8.1 and later, - libcurl resolves by using the Winsock asynchronous resolver which does - its own thread management. For older versions of Windows, libcurl - resolves by creating a separate thread that calls getaddrinfo. This - change only affects the former and it's already handled for the latter. + Closes #14142 - Reported-by: Ch40zz@users.noreply.github.com +Daniel Stenberg (9 Jul 2024) - Fixes https://github.com/curl/curl/issues/13509 - Closes https://github.com/curl/curl/pull/13518 +- RELEASE-NOTES: synced -Jay Satiro (7 May 2024) +Stefan Eissing (9 Jul 2024) -- asyn-thread: fix Curl_thread_create result check +- vtls: replace addsessionid with set_sessionid - - Compare to curl_thread_t_null instead of 0 for error. + - deduplicate the code in many tls backends that check + for an existing id and delete it before adding the new one + - rename ssl_primary_config's `sessionid` bool to `cache_session` - Currently for both supported thread libraries (pthreads and Windows) - curl_thread_t_null is defined as 0. However, the pattern throughout the - code is to check against curl_thread_t_null and not 0 since for - posterity some thread library may not use 0 for error. + Closes #14121 - Closes https://github.com/curl/curl/pull/13542 +Daniel Stenberg (9 Jul 2024) -- curl_multibyte: remove access() function wrapper for Windows +- test1175: scan libcurl-errors.md, not the generated .3 version - - Remove curlx_win32_access() which was a wrapper to use access() in - Windows. + Closes #14133 - This is a follow-up to 602fc213, one of two commits which removed - access() calls from the codebase and banned use of the function. +- test1139: scan .md files instead of .3 ones - Closes https://github.com/curl/curl/pull/13529 + As they are the canonical sources. -Daniel Gustafsson (6 May 2024) + It still uses the curl.1 for command line option info. -- tls: Remove EXAMPLEs from deprecated options + Closes #14132 - CURLOPT_EGDSOCKET and CURLOPT_RANDOM_FILE are both completely dead - so remove their example sections since the code there is useless. - There is still a way to inject a random file for OpenSSL older than - 1.1.0 but it's not what the example showed (and it's not even done - with this option) so we refrain from documenting it here. +Stefan Eissing (9 Jul 2024) - Closes: #13540 - Reviewed-by: Daniel Stenberg +- cf-socket: remove obsolete recvbuf -- tests: Only require EXAMPLE for non-deprecated options + - recvbuf was never enabled, remove all its code + - remove `fdsave`ing the socket as that is not longer needed - Manpages which document deprecated CURLOPT_ or CURLINFO_ are not - required to have an EXAMPLE section since they might effectively - be dead no-ops which we don't want to trick users into believing - they can use by copying example code. + Closes #14138 - Closes: #13540 - Reviewed-by: Daniel Stenberg +Viktor Szakats (9 Jul 2024) -Daniel Stenberg (6 May 2024) +- test1119: adapt for `.md` input -- EXPERIMENTAL: add graduation requirements for each feature + Replace logic dealing with `.3` files to handle the Markdown syntax. - Starting now, experimental features should have a set of documentated - requirements of what is needed for the feature to graduate. + Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 + Cherry-picked from #14097 + Closes #14125 - This adds requirements to all existing experiments. +- tests: include current directory when running test Perl commands - Closes #13541 + Necessary to find generated files in the out-of-tree build directory. + E.g. `tests/configurehelp.pm`, for tests 1119 and 1167. -Ivan (6 May 2024) + Before this patch macOS autotools builds were failing these two tests + due to falling back to the default preprocessor (`cpp`) instead of + the actual one configured. Then `cpp` failing to compile Apple SDK + headers referenced by curl headers. -- misc: fix typos, quoting and spelling + Cherry-picked from #14097 + Closes #14124 - Fix wording of comments, and misquotings where `' is markdown parsed - where it shouldn't be, and remove a misspelled preprocessor comment - which really isn't needed (and removing it makes it match surrounding - code better). +- configure: sort feature list, lowercase protocols, use backticks - Closes: #13538 - Reviewed-by: Daniel Gustafsson + - sort features case-insensitively to match `curl -V` and cmake. + `sort -f` is POSIX, but check if it's available anyway. -Daniel Gustafsson (6 May 2024) + - make protocols lowercase to match `curl -V` and cmake. -- tests: Mark tftpd timer function as noreturn + - replace two outlier `$()` with backticks. - This avoids the below compiler warning: + Closes #14117 - tftpd.c:280:1: warning: function 'timer' could be declared with - attribute 'noreturn' [-Wmissing-noreturn] +Yedaya Katsman (8 Jul 2024) - Closes: #13534 - Reviewed-by: Daniel Stenberg +- variable.md: make example use expand -- doh: Remove unused function prototype + I used double quotes since it seemed required for powershell, so this + example works in both (ba)sh and powershell as well as cmd.exe. - Closes: #13536 - Reviewed-by: Daniel Stenberg + Closes #14118 -Daniel Stenberg (6 May 2024) +Andy Reitz (8 Jul 2024) -- doh: cleanups in ECH related functions +- GIT-INFO.md: remove version requirements - - make local_decode_rdata_name use dynbuf instead of calloc + memcpy - - avoid extra memdup in local_decode_rdata_alpn - - no need to if() before free() - - use memdup instead of calloc + memcpy in Curl_doh_decode_httpsrr + Keep them in docs/INTERNALS.md - Reviewed-by: Stephen Farrell - Closes #13526 + Bump lowest perl to 5.8 -Viktor Szakats (5 May 2024) + Closes #14112 -- libssh2: delete redundant feature guard +Viktor Szakats (8 Jul 2024) - Delete `HAVE_LIBSSH2_VERSION` (equivalent to - `LIBSSH2_VERSION_NUM` > 0x010100) guard surrounding - a `LIBSSH2_VERSION_NUM` > 0x010B00 one. +- sectransp: fix `HAVE_BUILTIN_AVAILABLE` checks to not emit warnings - Reviewed-by: Daniel Gustafsson - Closes #13537 + `HAVE_BUILTIN_AVAILABLE` is a curl macro set via autotools and cmake. + Like other `HAVE_`s it signals availability if defined. -Jan Venekamp (5 May 2024) + SecureTransport code was specifically looking for the value 1, which + triggered compiler warnings when the feature was not present. -- tool_cfgable: free {proxy_}cipher13_list on exit + Replace the existing workaround of locally suppressing the compiler + warning with using `defined()`. - Author: Jan Venekamp - Reviewed-by: Daniel Gustafsson - Closes: #13531 + autotools: + ``` + 767 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILAB + LE == 1 + | ^~~~~~~~~~~~~~~~~~ + ~~~~ + ../../lib/vtls/sectransp.c: In function 'sectransp_connect_step1': + ../../lib/vtls/sectransp.c:1140:52: error: "HAVE_BUILTIN_AVAILABLE" is not de + fined, evaluates to 0 [-Werror=undef] + 1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAIL + ABLE == 1 + | ^~~~~~~~~~~~~~~~~~ + ~~~~ + ../../lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILTIN_AVAILABLE" is not de + fined, evaluates to 0 [-Werror=undef] + 1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAIL + ABLE == 1 + | ^~~~~~~~~~~~~~~~~~ + ~~~~ + ../../lib/vtls/sectransp.c: In function 'sectransp_connect_step2': + ``` + Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104448045#ste + p:6:499 -RainRat (4 May 2024) + cmake gcc: + ``` + 1140 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAIL + ABLE == 1 + | ^~~~~~~~~~~~~~~~~~ + ~~~~ + /Users/runner/work/curl/curl/lib/vtls/sectransp.c:1240:52: error: "HAVE_BUILT + IN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef] + 1240 | #if (CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAIL + ABLE == 1 + | ^~~~~~~~~~~~~~~~~~ + ~~~~ + /Users/runner/work/curl/curl/lib/vtls/sectransp.c: In function 'sectransp_con + nect_step2': + /Users/runner/work/curl/curl/lib/vtls/sectransp.c:2231:51: error: "HAVE_BUILT + IN_AVAILABLE" is not defined, evaluates to 0 [-Werror=undef] + 2231 | #if(CURL_BUILD_MAC_10_13 || CURL_BUILD_IOS_11) && HAVE_BUILTIN_AVAILA + BLE == 1 + | ^~~~~~~~~~~~~~~~~~~ + ~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9815428701/job/27104445425#ste + p:8:355 -- doh: Fix typo in comment + Cherry-picked from #14097 + Closes #14122 - Closes: #13504 - Author: RainRat on Github - Reviewed-by: Daniel Stenberg - Reviewed-by: Daniel Gustafsson +- examples: suppress deprecation warnings locally -Christian Schmitz (4 May 2024) + Simplify making clean builds by silencing deprecation warnings inside + the example code where these may occur. -- dynbuf: Fix returncode on memory error + Drop related build tweaks/comments from GHA jobs. - Curl_dyn_vaddf should return a proper error code in case allocating - memory failed. + Example warning: + ``` + curl/docs/examples/postit2-formadd.c:65:16: error: 'CURLFORM_COPYNAME' is dep + recated: since 7.56.0. Use curl_mime_name() [-Werror=deprecated-declarations] + 65 | CURLFORM_COPYNAME, "sendfile", + | ^~~~~~~~~~~~~~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9841099503/job/27166970904#ste + p:10:829 - Closes: #13533 - Author: Christian Schmitz - Reviewed-by: Daniel Gustafsson + Closes #14123 -Daniel Stenberg (3 May 2024) +- GHA/macos: bump parallel tests to -j5 -- RELEASE-NOTES: synced + Credit-to: Dan Fandrich + Cherry-picked from #11510 #14097 -Jan Venekamp (2 May 2024) +- GHA/windows: usability improvements -- bearssl: use common code for cipher suite lookup + - move `curl --version` into separate step. - Take advantage of the Curl_cipher_suite_walk_str() and - Curl_cipher_suite_get_str() functions introduced in commit fba9afeb. + - move configure log to separate step. Run on success, too. - This also fixes CURLOPT_SSL_CIPHER_LIST not working at all for bearssl - due to commit ff74cef5. + - add step with `curl_config.h` dump (full and brief/sorted). - Closes #13464 + - make `autoreconf` a separate step. -Daniel Stenberg (2 May 2024) + - add each job configuration a short name. -- curl.h: change CURL_SSLVERSION_* from enum to defines + - shorten job names. + Dedupe/drop redundant info, introduce abbreviations: + AM = autotools, CM = CMake, U = Unicode, R = Release, not -> `!`, etc. + Instead of mentioning `debug`, mentioned when it's not. - C++20 and later compilers emit a deprecation warning if values from two - different enums are combined with a bitwise operation the way the - CURL_SSLVERSION_* values were previously created. + - simplify `PATH` forming for MSVC jobs. + It's sufficient to add the release binary directory of vcpkg, the debug one + is redundant. + Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979 - Reported-by: Michael Kaufmann - Fixes #13510 - Closes #13511 + - other minor tidy-ups. -- configure: error on missing perl if docs or manual is enabled + Closes #14116 - Fixes #13508 - Reported-by: Harmen Stoppels - Closes #13514 +- GHA/macos: delete misplaced `CFLAGS`, drop redundant CMake option -- tool_cb_rea: limit rate unpause for -T . uploads + With macOS there is a long-term struggle with deprecation warnings. + In curl they occur with LDAP, SecureTransport and in docs/examples. - To avoid getting stuck in a busy-loop when nothing is read from stdin, - this function now checks the call rate and might enforce a short sleep - when called repeatedly without uploading anything. It is a crude - work-around to avoid a 100% busy CPU. + There are three ways to fix them: + - by CFLAGS `-Wno-deprecated-declarations` as a workaround. + - by CFLAGS `-mmacosx-version-min` set to a version where the the + feature was not deprecated. + - by CMake option `-DCMAKE_OSX_DEPLOYMENT_TARGET=`. - Reported-by: magisterquis on hackerone - Fixes #13174 - Closes #13506 + In GHA CMake jobs, all three were used, and `-mmacosx-version-min` was + set in a bogus way. Delete that bogus option, and delete the lone, + redundant CMake option too. -Viktor Szakats (1 May 2024) + In a future commit I might replace the suppression option to properly + setting the target OS. -- appveyor: enable websockets for VS2017 jobs + Follow-up to dfdd978f7c60224dffe2aac25b436dc0a5cd0186 #13491 + Cherry-picked from #14097 - Follow-up to eb4fe6c6340c3d5b0c347c6e30be004d4f9117d7 #13232 - Closes #13513 +- macos: add workaround for gcc, non-c-ares, IPv6, compile error -Daniel Stenberg (30 Apr 2024) + Apple macOS SDK 13.0 and later are increasingly incompatible with gcc, + which started causing CI errors with the 20240701.9 revision of the + `macos-latest` (= `macos-14-arm64`) runner image. -- if2ip: make the buf_size arg a size_t + This error is happening inside an Apple SDK header. We use the header + for calling a function in a resolver-related hack, in non-c-ares, IPv6 + builds. You can avoid the problem by using c-ares or disabling IPv6 + (or using clang, llvm, or a compatible gcc + SDK combination). - sizes should be size_t + This patch fixes affected builds by declaring the ncessary framework + function manually, and not including the problematic header. - Ref: #13489 - Closes #13505 + This workaround is ugly, doesn't cover all combinations, and fragile. -- cf-https-connect: use timeouts as unsigned ints + Other options are to disable this resolver-related hack for GCC, or to + replace it with a solution that doesn't rely on Apple SDK. - To match the type used in 'set.happy_eyeballs_timeout'. + If you are aware of a stable fix or workaround, let us know. - Ref: #13489 - Closes #13503 + gcc 12.4.0 + macOS SDK 14.0 (Xcode 15.0.1) error example: + ``` + In file included from /Applications/Xcode.app/Contents/Developer/Platforms/Ma + cOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFou + ndation.framework/Headers/CoreFoundation.h:54, + from /Applications/Xcode.app/Contents/Developer/Platforms/Ma + cOSX.platform/Developer/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/SystemC + onfiguration.framework/Headers/SCDynamicStoreCopySpecific.h:30, + from /Users/runner/work/curl/curl/lib/macos.c:33, + from /Users/runner/work/curl/curl/build/lib/CMakeFiles/libcu + rl_shared.dir/Unity/unity_0_c.c:244: + /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Develope + r/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Head + ers/CFUserNotification.h:126:1: error: attributes should be specified before + the declarator in a function definition + 126 | CF_INLINE CFOptionFlags CFUserNotificationCheckBoxChecked(CFIndex i) + API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOp + tionFlags)(1UL << (8 + i)));} + | ^~~~~~~~~ + /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Develope + r/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Head + ers/CFUserNotification.h:127:1: error: attributes should be specified before + the declarator in a function definition + 127 | CF_INLINE CFOptionFlags CFUserNotificationSecureTextField(CFIndex i) + API_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOp + tionFlags)(1UL << (16 + i)));} + | ^~~~~~~~~ + /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Develope + r/SDKs/MacOSX14.0.sdk/System/Library/Frameworks/CoreFoundation.framework/Head + ers/CFUserNotification.h:128:1: error: attributes should be specified before + the declarator in a function definition + 128 | CF_INLINE CFOptionFlags CFUserNotificationPopUpSelection(CFIndex n) A + PI_AVAILABLE(macos(10.0)) API_UNAVAILABLE(ios, watchos, tvos) {return ((CFOpt + ionFlags)(n << 24));} + | ^~~~~~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9787982387/job/27025351601?pr= + 14096#step:7:18 -- hash: change 'slots' to size_t from int + The exact conditions are fuzzy. Oddly enough gcc 12.3.0 and the SDK + same as above are _compatible_: + https://github.com/curl/curl/actions/runs/9791701214/job/27036037162 - - an unsigned type makes more sense - - size_t seems suitable - - on 64 bit args, the struct alignment makes the new Curl_hash remain - the same size + Also notice that similar errors can also happen in SecureTransport + builds, due to the SDK headers required. - Closes #13502 + Ref: https://github.com/curl/curl/pull/14097#issuecomment-2208639046 + Ref: https://github.com/curl/curl/pull/14091#issuecomment-2205870854 + Cherry-picked from #14097 + Closes #14119 -Viktor Szakats (30 Apr 2024) +- cmake: feature casing fix and tidy-ups -- libssh2: replace `access()` with `stat()` + - fix casing of a feature (`Unicode`) in the feature list. + - sort TLS backends case-insensitively. + - sync feature/protocol list heading with `curl -V` and autotools. - Prefer `stat()` to verify the presence of key files. + Closes #14120 - This drops the last uses of `access()` in the codebase, which was - reported to cause issues in some cases. +- GHA: ignore FTP test result in Windows jobs - Also add `access()` to the list of banned functions in checksrc. + They are flaky. - Ref: https://github.com/curl/curl/pull/13412#issuecomment-2065505415 - Ref: https://github.com/curl/curl/pull/13482#issuecomment-2078980522 - Ref: #13497 - Co-authored-by: Jay Satiro - Closes #13498 + E.g.: + - old-mingw-w64 7.3.0: 2001, 2039, 2083 + - msvc: 1501, 593 (multiple) -Daniel Stenberg (30 Apr 2024) + Ref: https://github.com/curl/curl/pull/13599#issuecomment-2119372376 + Cherry-picked from #14116 -- multi: remove useless assignment +- GHA: improve vcpkg cache, add BoringSSL ECH and LibreSSL MSVC jobs - Spotted by CodeSonar + - cache on a per-package basis. + Replace manual caching with a built-in solution. It shares cached + package builds between jobs, e.g. libssh2 only builds once + per platform (instead of once per job). Individual packages are built + as needed (not the whole per-job tree). It also fixes the duplicate + cache entry issues. + Ref: https://learn.microsoft.com/en-us/vcpkg/consume/binary-caching-github- + actions-cache + Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979 + Follow-up to cb22cfca69bded45bf7f9c72c8e6764990490f11 #14077 - Closes #13500 + - add BoringSSL job with ECH enabled. The first such job in the curl CI. -- RELEASE-NOTES: synced + - add LibreSSL job. -fuzzard (29 Apr 2024) + - use vcpkg pre-installed on the runner image, instead of rolling our + own. This is quicker, simpler and more robust. + Follow-up to e26cbe20cbedbea0ca743dd33880517309315cb2 #13979 -- cmake: FindNGHTTP2 add static lib name to find_library call + - show pre-installed vcpkg and ports version. - Add the static library name, nghttp2_static as a name to search. + - drop `gsasl` dependency till it reaches the pre-installed vcpkg ports. - This provides cmake parity with the winbuild Makefile.vc allowing - the cmake build to find and allow the link to static nghttp2 library. + - re-add `find .` to see the binaries generated. -Viktor Szakats (29 Apr 2024) + - simplify setting up `PATH`. -- DISTROS: add patch and issues link for curl-for-win + - exclude failing tests for any job enabling WinIDN. - curl-for-win sometimes includes curl patches that were already merged in - master, but not yet part of a stable release. + - drop collecting and uploading log archives. We already dump CMake + logs, and our build doesn't use Ninja. Rest of files weren't generated + by the curl build. We don't aim to debug vcpkg package builds. - Also include the Issues link. Build-specific issues are handled there. + Closes #14090 - Ref: #13493 - Closes #13499 +Tal Regev (7 Jul 2024) -Daniel Stenberg (29 Apr 2024) +- GHA: add MSVC UWP job, expand jobs with more options -- mime: avoid using access() + - add new dependencies: brotli, libpsl (requires libicu2) and gsasl. - If stat() fails, there is no point in calling access() + - enable WinIDN in a job. Exclude failing tests. - Also: return error immediately if the stat() fails. + - add UWP job and fix the build logic to support it. - Ref: #13482 - Closes #13497 + - increase timeouts to build the new dependencies. -Stefan Eissing (29 Apr 2024) + Assisted-by: Viktor Szakats + Closes #14077 -- tests: add SNI and peer name checks +Dan Fandrich (6 Jul 2024) - - connect to DNS names with trailing dot - - connect to DNS names with double trailing dot - - rustls, always give `peer->hostname` and let it - figure out SNI itself - - add SNI tests for ip address and localhost - - document in code and TODO that QUIC with ngtcp2+wolfssl - does not do proper peer verification of the certificate - - mbedtls, skip tests with ip address verification as not - supported by the library +- tests: fix sshd UserKnownHostsFile path for MinGW/Cygwin - Closes #13486 + This is the same thing as the previous commit fd194f46 but on the next + line. -Daniel Stenberg (29 Apr 2024) + Follow-up to 70d2fca2 -- curl_getdate.md: document two-digit year handling + Ref: #10818 - Mentioned-by: Paul Gilmartin - Ref: https://curl.se/mail/archive-2024-04/0014.html - Closes #13494 +- tests: fix sshd IdentityFile path for MinGW/Cygwin -Viktor Szakats (29 Apr 2024) + This was missed during some refactoring more than a year ago and is + causing a warning "Use of uninitialized value $path in pattern match". -- cmake: add `BUILD_EXAMPLES` option to build examples + Follow-up to 70d2fca2 - You can enable it with `-DBUILD_EXAMPLES=ON`. + Ref: #10818 + Closes #14113 - To match autotools' `make examples` feature. - Windows (static) builds not tested. +Viktor Szakats (7 Jul 2024) - Also enable examples in a pair of CI jobs. +- build: add Debug, TrackMemory, ECH to feature list - Apply related updates to the macOS CI workflow: - - drop unused `CXX` envs. - - drop no longer needed `-Wno-error=undef -Wno-error=conversion` flags. - - pass `-Wno-deprecated-declarations` to GCC too (for `BUILD_EXAMPLES`). - - document why `-Wno-deprecated-declarations` is necessary. + Also: - Closes #13491 + - remove stray `ECH` and `HTTPSRR` from cmake protocol list. -Stefan Eissing (26 Apr 2024) + - stop excluding `Debug` and `TrackMemory` in `test1013.pl`. -- http3: quiche+ngtcp2 improvements + - configure: delete `CURL_CHECK_CURLDEBUG` check. + Ref: 065047dc62cba3efde597fa5420d112fc2f4c500 + This check was effectively doing nothing, except disabling + `--enable-curldebug` in `curl-config` for + Cygwin/MSYS/cegcc/OS2/AIX targets with c-ares enabled. - - quiche: error transfers that try to receive on a closed - or draining connection - - ngtcp2: use callback for extending max bidi streams. This - allows more precise calculation of MAX_CONCURRENT as we - only can start a new stream when the server acknowledges - the close - not when we locally have closed it. - - remove a fprintf() from h2-download client to avoid excess - log files on tests timing out. + Closes #14096 - Closes #13475 +Dan Fandrich (5 Jul 2024) -- vtls: TLS session storage overhaul +- CI: bump the libc6 on the linux-old build - - add session with destructor callback - - remove vtls `session_free` method - - let `Curl_ssl_addsessionid()` take ownership - of session object, freeing it also on failures - - change tls backend use - - test_17, add tests for SSL session resumption + This contains some security fixes for nscd. - Closes #13386 +Viktor Szakats (6 Jul 2024) -- multi: multi_wait improvements +- reuse: fix typo in comment - - only call `multi_getsock()` once for all transfers - - realloc pollset array on demand - - fold repeated sockets + Follow-up to 9104bad82004d908e1fe66a425f8ca78f975045d #14107 - Closes #13150 +Dan Fandrich (5 Jul 2024) -Philip Heiduck (25 Apr 2024) +- CI: Fix typo in comment -- ci: remove microsoft-prod.list +- curl: follow-up to fix categories in --help - This is added by default, and it is often broken, but we don't need - anything from it. + The commit 6483813b was missing changes necessitated by 2abfc75 that + causes a crash. Also, use ARRAYSIZE() for cleaner code. - Closes #13473 + Follow-up to 6483813b -Evgeny Grin (Karlson2k) (25 Apr 2024) + Ref #14055 -- curl_setup.h: detect 'inline' support +- curl: list categories in --help - Closes #13355 + This eliminates the need to run an extra help subcommand to get the + possible categories, reducing the friction in getting relevant help. The + help wording was also slightly tweaked for grammatical accuracy. -Daniel Stenberg (25 Apr 2024) + Closes #14055 -- multi: avoid memory-leak risk +Daniel Stenberg (5 Jul 2024) - 'newurl' is allocated in some conditions and used in a few scenarios, - but there were theoretical combinations in which it would not get freed. - Move the free to happen unconditionally. Never triggered by tests, but - spotted by Coverity. +- RELEASE-NOTES: synced - Closes #13471 +renovate[bot] (5 Jul 2024) -Johann Sebastian Schicho (25 Apr 2024) +- GHA: update actions/upload-artifact and actions/download-artifact -- sendf: Curl_cwriter_write: remove comment disallowing zero length writes + update actions/upload-artifact digest to 0b2256b + update actions/download-artifact digest to fa0a91b - They are needed to pass CLIENTWRITE_EOS. + Closes #14111 + Closes #14110 - Closes #13477 +Max Mehl (5 Jul 2024) -Stefan Eissing (25 Apr 2024) +- reuse: switch to REUSE 3.2 and REUSE.toml -- CI: macos fixes for new ARM GHA images + - remove scripts/copyright.pl - - based on #13478 with additions from #13476 - - make homebrew install path flexible - - fix OpenSSL pkgconfig files libdir - - add path to --with-libssh2 target - - disable gcc securetransport due to linker - errors (missing symbols), probably because - the os version is no longer low enough + Closes #14107 - Assisted-by: Viktor Szakats +Yedaya Katsman (5 Jul 2024) - Closes #13479 +- curl: move more options to deprecated category -- content_encoding: ignore duplicate chunked encoding + --no-npn, --sslv2, --sslv3 - - ignore duplicate "chunked" transfer-encodings from - a server to accomodate for broken implementations - - add test1482 and test1483 + Closes #14109 - Reported-by: Mel Zuser - Fixes #13451 - Closes #13461 +Stefan Eissing (5 Jul 2024) -Daniel Stenberg (25 Apr 2024) +- multi: pollset assertion only when IP connected -- tool: move tool_ftruncate64 to tool_util.c + Give warning for an empty pollset only when the connection has at least + IP connectivity. There are cases where the connect in QUIC makes another + attempt on a timeout and no socket will be available during that. - ... and the prototype to tool_setup.h, to make them both available more - widely and accurately. + Closes #14108 - Follow-up to 00bef95946d3511 +Daniel Stenberg (5 Jul 2024) - Fixes #13458 - Closes #13459 +- cmdline-opts: category cleanup -Viktor Szakats (24 Apr 2024) + Option cleanups: -- lib: silence `-Wsign-conversion` in base64, strcase, mprintf + --get is not upload + --form* are post + - added several options into ldap, smtp, imap and pop3 + - shortened the category descriptions in the list - Closes #13467 + category curl fixes: -- CI: retain failure code after `./configure` with Circle CI + --create-dirs removed from 'curl' + --ftp-create-dirs removed from 'curl' + --netrc moved to 'auth' from 'curl' + --netrc-file moved to 'auth' from 'curl' + --netrc-optional moved to 'auth' from 'curl' + --no-buffer moved to 'output' from 'curl' + --no-clobber removed from 'curl' + --output removed from 'curl' + --output-dir removed from 'curl' + --remove-on-error removed from 'curl' - Suggested-by: Dan Fandrich - Follow-up to 43299e93c06b96fea8a8dc9b1c2e49c82bc21801 #13462 - Follow-up to d7332e3e46c3ef401b34e6a1a129eb4dd846c452 #12635 - Closes #13468 + Add a "global" category: -Daniel Stenberg (24 Apr 2024) + - Made all "global" options set this category -- RELEASE-NOTES: synced + Add a "deprecated" category: -Jan Venekamp (24 Apr 2024) + - Moved the deprecated options to it (maybe they should not be in any + category long term) -- mbedTLS: implement CURLOPT_SSL_CIPHER_LIST option + Add a 'timeout' category - Use a lookup list to set the cipher suites, allowing the - ciphers to be set by either openssl or IANA names. + - Put a number of appropriate options in it - To keep the binary size of the lookup list down we compress - each entry in the cipher list down to 2 + 6 bytes using the - C preprocessor. + Add an 'ldap' category - Closes #13442 + - Put the LDAP related option in there -Viktor Szakats (24 Apr 2024) + Remove categories "ECH" and "ipfs" -- CI: show more failed `config.log` on Circle CI + - They should not be categories. Had only one single option each. - Show last 1000 lines of `config.log` if `./configure` fails. This was - already done for one job, this patch extends it to all. + Remove category "misc" - Ref: #13438 - Closes #13462 + - It should not be a category as it is impossible to know when to browse + it. -Daniel Stenberg (24 Apr 2024) + --use-ascii moved to ftp and output + --xattr moved to output + --service-name moved to auth -- telnet: check return code from fileno() + Managen fixes: - and return error if necessary + - errors if an option is given a category name that is not already setup + for in code - Spotted by CodeSonar + - verifies that options set `scope: global` also is put in category + `global´ - Closes #13457 + Closes #14101 -Viktor Szakats (24 Apr 2024) +Stefan Eissing (5 Jul 2024) -- tls: fix SecureTransport + BearSSL cmake unity builds +- GHA: configure OpenSSL's libdir as 'lib' only - Avoid clashing static function names by namespacing them. + Also mention in HTTP3.md - Pointed-out-by: Jan Venekamp - Ref: https://github.com/curl/curl/pull/13442#discussion_r1576350700 - Closes #13450 + OpenSSL has a bug that messes the config `--libdir=path` to become the + wrong path in its pkgconfig files. If we just pass `--libdir=lib` it + should avoid this. -Jay Satiro (24 Apr 2024) + Ref: #14099 + See also: https://github.com/openssl/openssl/issues/23569 -- dllmain: Call OpenSSL thread cleanup for Windows and Cygwin + Closes #14102 - - Call OPENSSL_thread_stop on thread termination (DLL_THREAD_DETACH) - to prevent a memory leak in case OpenSSL is linked statically. +Daniel Stenberg (5 Jul 2024) - - Warn in libcurl-thread.3 that if OpenSSL is linked statically then it - may require thread cleanup. +- tool_operate: simplify return code handling from url_proto() - OpenSSL may need per-thread cleanup to stop a memory leak. For Windows - and Cygwin if libcurl was built as a DLL then we can do that for the - user by calling OPENSSL_thread_stop on thread termination. However, if - libcurl was built statically then we do not have notification of thread - termination and cannot do that for the user. + The additional checks were superfluous as it would only ever return + error if one of those protocols were set. Also: a returned error + *should* mean get out of there, without having to check more conditions. - Also, there are several other unusual cases where it may be necessary - for the user to call OPENSSL_thread_stop, so in the libcurl-thread - warning I added a link to the OpenSSL documentation. + Closes #14104 - Co-authored-by: Viktor Szakats +- tool_operate: check for --disable case *sensitively* - Reported-by: southernedge@users.noreply.github.com - Reported-by: zmcx16@users.noreply.github.com + curl command line options are specified with the correct casing or they + don't match - Ref: https://www.openssl.org/docs/man3.0/man3/OPENSSL_thread_stop.html#NOTES + Closes #14103 - Fixes https://github.com/curl/curl/issues/12327 - Closes https://github.com/curl/curl/pull/12408 +Stefan Eissing (4 Jul 2024) -Jan Venekamp (24 Apr 2024) +- transfer: avoid polling socket every transfer loop -- rustls: remove incorrect SSLSUPP_TLS13_CIPHERSUITES flag + Improve download performance, minimal effort. - The rustls backend advertises SSLSUPP_TLS13_CIPHERSUITES, but - the code does not actually seem to support it (yet?). Removed - the flag and corrected documentation. + Do not poll the socket for pending data every transfer loop iteration. + This gives 10-20% performance gains on large HTTP/1.1 downloads (on my + machine). - Closes #13452 + Closes #14098 -Stefan Eissing (24 Apr 2024) +Viktor Szakats (4 Jul 2024) -- quiche: expire all active transfers on connection close +- tests: delete `CharConv` remains - - when a connection close is detected, all ongoing transfers - need to expire bc no more POLL events are likely to happen - for them. + Closes #14100 - Fixes #13439 - Reported-by: Jay Satiro - Closes #13447 +- GHA: bump macOS CMake job parallelism to 4 (nproc+1) [ci skip] -Dan Fandrich (23 Apr 2024) + To match autotools ones and the rest of workflows. -- tests: fix feature case in test1481 + Follow-up to 464282ddfb214917be3d143c035f178f3b77f209 #13807 - This test was being skipped everywhere because the feature never - matched. +Yedaya Katsman (4 Jul 2024) - Closes #13445 +- help: add flags to output and ssh categories -Gusted (23 Apr 2024) + - Add --output, --remove-on-error, --output-dir and --created-dirs to + the output help category -- tool_operate: don't truncate the etag save file by default + - Add --hostpubmd5, --hostpubsha256, --insecure (-k), and --pubkey to + the ssh help category - This fixes a regression of 75d79a4486b279100209ddf8c7fdb12955fb66e9. The - code in tool-operate truncated the etag save file, under the assumption - that the file would be written with a new etag value. However since - 75d79a4486b279100209ddf8c7fdb12955fb66e9 that might not be the case - anymore and could result in the file being truncated when --etag-compare - and --etag-save was used and that the etag value matched with what the - server responded. Instead the truncation should not be done when a new - etag value should be written. + Closes #14076 - Test 3204 was added to verify that the file with the etag value doesn't - change the contents when used by --etag-compare and --etage-save and - that value matches with what the server returns on a non 2xx response. +Stefan Eissing (4 Jul 2024) - Closes #13432 +- TODO: remove item about 'SSL_peak' -Abdullah Alyan (22 Apr 2024) + GnuTLS todo item about using an equivalent of `SSL_peak()`, which + nicely escaped the word checks, is no longer relevant. -- tests: enable test 1117 for hyper + We do not use `SSL_peek()` anymore since connection filters were + introduced. - Closes #13436 + Closes #14091 -Daniel Stenberg (22 Apr 2024) +renovate[bot] (4 Jul 2024) -- sendf: useless assignment in cr_lc_read() +- GHA: update dependency gnutls/gnutls to v3.8.6 - Spotted by CodeSonar + Closes #14094 - Closes #13437 +- GHA: update fsfe/reuse-action action to v4 -- tool_paramhlp: remove duplicate assign + Closes #14095 - Spotted by CodeSonar +Viktor Szakats (3 Jul 2024) - Closes #13433 +- GHA: Windows job exclusions tweaks -- transfer: remove useless assignment + - disable SMTP tests in MSYS2/mingw-w64 and MSVC jobs. + On the suspicion of sometimes hanging: + https://github.com/curl/curl/actions/runs/9346162475/job/25720437944?pr=138 + 55#step:14:2838 + https://github.com/curl/curl/actions/runs/9758011305/job/26931678639?pr=140 + 84#step:14:2834 + https://github.com/curl/curl/actions/runs/9774468536/job/26982805294#step:1 + 1:4731 - in Curl_xfer_recv_resp + - run TFTP, MQTT, WebSockets tests in MSYS2/msys jobs again. - Spotted by CodeSonar + - switch hanging old-mingw-w64 7.3.0 job to Release (from Debug). + Guessing here, 9.5.0 is more solid, and one difference is + Debug/Release mode. Let's match 7.3.0 with that and see how it changes + hangs and flakiness. + The other difference is Unicode ON in 7.3.0. Flaky 6.3.0 was also + Debug, with Unicode OFF: + 217878bade884202ee5fb2e80186c5fd130392e8 #13566. + (Unicode unlikely to play a role here IMO.) + If 7.3.0 keeps hanging / remains flaky I'll consider disabling its + test runs. - Closes #13435 + - opt-out from vcpkg telemetry. -- http: acknowledge a returned error code + Ref: https://github.com/curl/curl/pull/13599#issuecomment-2119372376 + Closes #14085 - ... and do not overwrite it with a new value that could then hide the - problem. +renovate[bot] (3 Jul 2024) - Spotted by CodeSonar +- Dockerfile: update debian:bookworm-slim to 39868a6 - Closes #13434 + Closes #14083 -- tool_operate: init vars unconditionally in post_per_transfer +Daniel Stenberg (3 Jul 2024) - In case of (the unlikely) early return, they could otherwise remain - uninitialized +- FEATURES.md: refresh - Spotted by CodeSonar + - added lots of missing stuff + - rearranged a little + - remove all footnotes - Closes #13430 + Closes #14086 - RELEASE-NOTES: synced -- urlapi: allow setting port number zero +- curl_easy_perform.md: call it network transfer, not file transfer - Also set and check errno when strtoul() parsing numbers for better error - checking. +Viktor Szakats (2 Jul 2024) - Updated test 1560 +- winbuild: MS-DOS batch tidy-ups - Closes #13427 + - prefer `.bat` extension over `.cmd` for MS-DOS batch, which also + avoids confusion with OS/400 `.cmd` files. + - cleanup `echo` quotes, drop them consistently. + - delete empty output line from one of the error branches. + - prefer lowercase commands like the rest of MS-DOS batches. + - delete a contraction. + - drop backticks from error message. + - use `nmake.exe` consistently. + - use equal/not-equal operator style consistently. + - inline a single-line `if` branch. + - delete exceptions and rules dealing with Windows `.cmd` extension. -- http_aws_sigv4: remove useless assignment + Closes #14084 - This code assigned the variable the same value it already had +Stefan Eissing (2 Jul 2024) - Spotted by CodeSonar +- multi: fix pollset during RESOLVING phase - Closes #13426 + - add a DEBUGASSERT for when a transfer's pollset should not be empty. + - move write unpausing from transfer loop into curl_easy_pause. This + make sure that the url_updatesocket() finds the correct state when + updating socket events. + - fix HTTP/2 proxy during connect phase to set sockets correctly + - fix test2600 to simulate a socket set + - move write unpausing from transfer loop into curl_easy_pause. This + make sure that the url_updatesocket() finds the correct state when + updating socket events. + - waiting for the resolver to deliver might not involve any sockets to + wait for. Do not generate a warning. -- file: remove useless assignment + Fixes #14047 + Closes #14074 - This code assigned the variable the same value it already had. +Daniel Stenberg (2 Jul 2024) - Spotted by CodeSonar +- cmdline-opts: shorten six help texts - Closes #13425 + o --location-trusted + o --next + o --parallel-immmediate + o --pinnedpubkey + o --proxy-pass + o --proxy-ssl-allow-beast -- test2406: verify -f with HTTP/2 + Closes #14075 -Stefan Eissing (19 Apr 2024) +- managen: fix removing backticks from subtitles -- http2 + ngtcp2: pass CURLcode errors from callbacks + It erroneously removed them from the wrong variable. - - errors returned by Curl_xfer_write_resp() and the header variant are - not errors in the protocol. The result needs to be returned on the - next recv() from the protocol filter. + Closes #14081 - - make xfer write errors for response data cause the stream to be - cancelled +Viktor Szakats (2 Jul 2024) - - added pytest test_02_14 and test_02_15 to verify that also for - parallel processing +- cmake: show protocols, then features - Reported-by: Laramie Leavitt - Fixes #13411 - Closes #13424 + To match the order used by `curl -V` and `./configure`. -Daniel Stenberg (19 Apr 2024) + Closes #14082 -- request: make Curl_req_init return void +- cmdline-docs: fix `--proxy-ca-native` example + tidy-ups - Since it could not return error and therefore this change removes dead - code for the caller. + Also: + - fix an indentation. + - fix capitalized option in comment. - Spotted by CodeSonar. + Closes #14078 - Closes #13423 +- cmake: sync protocol/feature list with `curl -V` output -- multi: remove the unused Curl_preconnect function + - sort features case-insensitively. + Requires CMake v3.13.0. + Follow-up to 0f26abeef1dd1d1a02f8e12dbc3d51e73e9d2e9c #14063 - The implementation has been removed, no point in keeping it around. + - convert protocol list to lowercase. + But leave it uppercase in `curl-config`. - Follow-up to 476adfeac019ed + Closes #14066 - Closes #13422 +- GHA/badwords.yml: fixup indent for yamllint [ci skip] -- Curl_creader_read: init two variables to avoid using them uninited +renovate[bot] (1 Jul 2024) - Spotted by CodeSonar +- GHA: update dependency awslabs/aws-lc to v1.31.0 - Closes #13419 + Closes #14080 -- http: reject HTTP major version switch mid connection +Daniel Stenberg (1 Jul 2024) - A connection that has seen an HTTP major version now refuses any other - major HTTP version in future responses. Previously, a HTTP/1.x - connection would just silently accept HTTP/2 or HTTP/3 in the status - lines as long as it had support for those built-in. It would then just - lead to confusion and badness. +- GHA/badwords.yml: check source code wording - Indirectly Spotted by CodeSonar which identified a duplicate assignment - in this function. + Closes #14073 - Add test 471 to verify +- code: language cleanup in comments - Closes #13421 + Based on the standards and guidelines we use for our documentation. -- mqtt: when Curl_xfer_recv returns error, don't use nread + - expand contractions (they're => they are etc) + - host name = > hostname + - file name => filename + - user name = username + - man page => manpage + - run-time => runtime + - set-up => setup + - back-end => backend + - a HTTP => an HTTP + - Two spaces after a period => one space after period - A returned error code makes other return value unreliable, and in this - case potentially uninitialized. On error, do not read other return - values like the nread counter. + Closes #14073 - Spotted by CodeSonar +Yedaya Katsman (1 Jul 2024) - Closes #13418 +- docs: add RELEASE-TOOLS.md.dist to .gitignore -- ftp: fix socket leak on rare error + Closes #14079 - In the function AcceptServerConnect() the newly created socket would - leak if Curl_conn_tcp_accepted_set() returns error. Which basically - should never happen. +Viktor Szakats (1 Jul 2024) - Spotted by CodeSonar. +- libcurl.pc: add more `Requires.private`/`Requires` dependencies - Closes #13417 + - add `libmsh3` reference from cmake and autotools. -- urlapi: remove unused flags argument from Curl_url_set_authority + - add `mit-krb5-gssapi` reference from cmake. - The function is only called from a single place (for HTTP/2 server push) - so might as well just assume this fixed option every time. + It leaves GSS not set from autotools. The handling of heimdal in cmake + is fuzzy, that's probably missing too. - Closes #13409 + Follow-up to f057de5a1a950a90d1920021db152a4b695f1a8a #13911 + Closes #14072 -- github/ISSUE_TEMPLATE: tweak the commericual support text +- cmake: improve wolfSSL detection -- github/ISSUE_TEMPLATE: link the GitHub discussions too + - support detecting wolfSSL via pkg-config (like autotools.) - ... and move the feature request line to the bottom. + - detect wolfSSL version. -- curl_url_get.md: clarify queries and fragments and CURLU_GET_EMPTY + - detect `HAVE_WOLFSSL_DES_ECB_ENCRYPT`. + (needs e.g. `--enable-curl` when building wolfSSL) - Follow-up to 3eac21d86bc5 + - detect `HAVE_WOLFSSL_FULL_BIO` and enable HTTPS-proxy feature. + (needs e.g. `--enable-opensslall` when building wolfSSL) - Closes #13407 + - fix to show `HTTPS-proxy` in cmake feature list. + Ref: 55807e6c056f27846d70cec70ee6ac3f0e5b3bbe #9962 -Stefan Eissing (18 Apr 2024) + - fix to show `NTLM` in cmake feature list. -- tests: check caddy server version to match test expectations + - fix to show `smb` and `smbs` in cmake protocol list. - - new caddy servers no longer return 200 on POSTs, but 405 - as they should + - add wolfSSL CMake job to GHA (for macOS). - Closes #13405 + - fix mqtt and wolfSSL symbol clash. + ``` + ./curl/lib/mqtt.c: In function 'mqtt_doing': + ./curl/lib/mqtt.c:746:17: error: declaration of 'byte' shadows a global dec + laration [-Werror=shadow] + 746 | unsigned char byte; + | ^~~~ + /opt/homebrew/Cellar/wolfssl/5.7.0_1/include/wolfssl/wolfcrypt/types.h:85:3 + 6: note: shadowed declaration is here + 85 | typedef unsigned char byte; + | ^~~~ + ``` -Daniel Stenberg (18 Apr 2024) + - format `FindWolfSSL.cmake` closer to neighbours. -- curl_url_set.md: extended + Closes #14064 - Closes #13404 +Daniel Stenberg (1 Jul 2024) -- urlapi: add CURLU_GET_EMPTY for empty queries and fragments +- curl_url_set: elaborate on scheme guessing - By default the API inhibits empty queries and fragments extracted. - Unless this new flag is set. + Explain a little more and refer to the CURLU_NO_GUESS_SCHEME flag + for getting scheme or URL. - This also makes the behavior more consistent: without it set, zero - length queries and fragments are considered not present in the URL. With - the flag set, they are returned as a zero length strings if they were in - fact present in the URL. + Closes #14071 - This applies when extracting the individual query and fragment - components and for the full URL. +- docs: misc language polish - Closes #13396 + - CURLINFO_FILETIME*: improve language + - add '32bit' and '64bit' as bad words, use 32-bit and 64-bit + - mksymbolsmanpage.pl: avoid "will" -- RELEASE-NOTES: synced + Closes #14070 -- lib1560: test with leading zeroes and more IPv4 versions +- curl_easy_escape: elaborate a little on encoding a URL - Inspired by WHATWG URL Spec test inputs + Closes #14069 - Closes #13400 +Viktor Szakats (1 Jul 2024) -Christian Schmitz (17 Apr 2024) +- cmake: fix feature and protocol lists for SecureTransport -- smtp: result of Curl_bufq_cread was not used + NTLM was missing from the features list, and SMB/SMBS from + the protocols list in SecureTransport builds. - return the result back to the caller. + Follow-up to 76a9c3c4be10b3d4d379d5b23ca76806bbae536a #3619 - Closes #13398 + Reported-by: Tal Regev + Bug: https://github.com/curl/curl/pull/13963#issuecomment-2178791390 + Closes #14065 -Daniel Stenberg (17 Apr 2024) +Daniel Stenberg (1 Jul 2024) -- urlapi: fix relative redirects to fragment-only +- curl_str[n]equal.md: tidy up text to make them stand-alone - Using the URL API for a redirect URL when the redirected-to string - starts with a hash, ie is only a fragment, the API would produce the - wrong final URL. + Previously this was one single manpage for two functions but as they are + two separate ones since a while back, they should each clearly document + their single specific functions. - Adjusted test 1560 to test for several new redirect cases. + Follow-up to eefcc1bda4bc - Closes #13394 + Closes #14068 -Jiwoo Park (17 Apr 2024) +- RELEASE-NOTES: synced -- url: fix use of an uninitialized variable +Tal Regev (30 Jun 2024) - Closes #13399 +- GHA: use vcpkg to install packages for MSVC jobs -Patrick Monnerat (17 Apr 2024) + - enable new dependencies for existing jobs. -- os400: sync with latest changes + - add cache for vcpkg packages. - - Conversion support for new version info character field rtmp_version. - - New ILE/RPG declarations. + - tidy-up CMake options and environment for vcpkg. - Closes #13402 + Closes #13979 -Daniel Stenberg (17 Apr 2024) +Daniel Stenberg (30 Jun 2024) -- ngtcp2: fix macro use +- curl_mprintf.md: add missing comma - macro "H3_STREAM_CTX" requires 2 arguments, but only 1 given +- CURLOPT_TLSAUTH_PASSWORD/USERNAME.md: language fixups - Follow-up to c6655f7029ec5c128561e3ecf1f93db3ed0432a4 + - relies *on* TLS SRP + - *for* the specific TLS backends - Closes #13401 + Closes #14061 -Christian Schmitz (17 Apr 2024) +- docs/libcurl: polish the single-line descriptions -- sendf: fix two typos in comments + - use imperative form + - use lowercase + - no period + - unify some phrases + - fix curl_multi_socket and curl_multi_socket_all to keep their own + descriptions - The parameters are named data, not date. + Closes #14062 - Closes #13393 +Viktor Szakats (30 Jun 2024) -- lib: silence warnings on comma misuse +- cmake: alpha-sort feature list - Building curl with -Wcomma, I see warnings about "possible misuse of - comma operator here" and moving fields assignment out of the for() fixes - it. + Like autotools does. - Closes #13392 + Closes #14063 -Stefan Eissing (17 Apr 2024) +renovate[bot] (29 Jun 2024) -- http/2, http/3: decouple stream state from easy handle +- GHA: update github/codeql-action digest to b611370 - - add `Curl_hash_offt` as hashmap between a `curl_off_t` and - an object. Use this in h2+h3 connection filters to associate - `data->id` with the internal stream state. - - changed implementations of all affected connection filters - - removed `h2_ctx*` and `h3_ctx*` from `struct HTTP` and thus - the easy handle - - solves the problem of attaching "foreign protocol" easy handles - during connection shutdown + Closes #14058 - Test 1616 verifies the new hash functions. +Tatsuhiro Tsujikawa (29 Jun 2024) - Closes #13204 +- vquic: fix UDP_GRO struct cmsghdr data type -Daniel Stenberg (17 Apr 2024) + The data type for UDP_GRO in struct cmsghdr is int. Limit the usage of + UDP_GRO to linux only because it is not portable. -- ROADMAP: remove completed entries, mention websocket + Closes #14056 -- THANKS-filter: name fixes +Sertonix (29 Jun 2024) -Christian Schmitz (17 Apr 2024) +- mk-ca-bundle.pl: delay 'curl -V' execution until it is needed -- winbuild: add ENABLE_WEBSOCKETS option + Avoid an `Can't exec "curl"` message when curl is not actually needed. - Closes #13232 + Closes #14060 -Daniel Stenberg (17 Apr 2024) +Daniel Stenberg (29 Jun 2024) -- dmaketgz: compacter +- src/Makefile.am: remove SUBDIRS assignment - Removes the need for disabling shellcheck warnings. + It was once used to continue into ../docs but is just leftovers now. - Follow-up to d28f74913c2 - Proposed-by: Viktor Szakats - Closes #13391 + Closes #14054 -Dan Fandrich (16 Apr 2024) +z2_ (28 Jun 2024) -- tests: Fix uninitialized value warning +- x509asn1: remove superfluous free() - The check for an option must be predicated on options existing at all. +Stefan Eissing (28 Jun 2024) - Follow-up to f7cc9e91 +- ngtcp2+quictls: fix cert-status use -Christian Schmitz (17 Apr 2024) + - add test for --cert-status on all http versions -- idn: add native AppleIDN (icucore) support for macOS/iOS + Reported-by: Dexter Gerig + Fixes #14049 + Closes #14050 - I implemented the IDN functions for macOS and iOS using Unicode - libraries coming with macOS and iOS. +Daniel Stenberg (28 Jun 2024) - Builds and runs here on macOS 14.2.1. Also verified to load and - run on older macOS version 10.13. +- RELEASE-PROCEDURE.md: update release date - Build requires macOS SDK 13 or equivalent. +- managen: insert final .fi for files ending with a quote - Set `-DUSE_APPLE_IDN=ON` CMake option to enable it. - With autotools and other build tools, set these manual options: - ``` - CPPFLAGS=-DUSE_APPLE_IDN - LIBS=-licucore - ``` + When an individual file ended with a quote (typically an example), the + render function would return without ending the quote correctly with a + ".fi" (fill in) in the manpage output. - Completes TODO 1.6. + This made the additional text provided below to render wrongly. - TODO: add autotools option and feature-detection. + Closes #14048 - Refs: #5330 #5371 - Co-authored-by: Viktor Szakats - Closes #13246 +Junho Choi (28 Jun 2024) -Stefan Eissing (16 Apr 2024) +- quic: update to quiche 0.22.0 -- http3: extend download abort tests, fixes in ngtcp2 + quiche 0.22.0 will set SONAME in libquiche.so (libquiche.so.0) for + linux/BSDs. Install a symlink with SONAME. - - fix flow handling in ngtcp2 to ACK data on streams - we abort ourself. - - extend test_02_23* cases to also run for h3 - - skip test_02_23* for OpenSSL QUIC as it gets stalled - on progressing the connection + Closes #14030 + Closes #14046 - Closes #13374 +Daniel Stenberg (28 Jun 2024) -Daniel Stenberg (16 Apr 2024) +- managen: introduce "Multi: per-URL" -- tests: add -q as first option when invoking curl for tests + For -O, -o and -T that are used once per specified URL. - To reduce the risk that the user running the tests has a .curlrc present - that messes things up. + Closes #14045 - Support 'option="no-q"' for the tag to switch it off on demand. - Use this new feature in test 433 and 436. +- quiche: fix operand of ‘?:’ changes signedness - Ref: #13284 - Closes #13387 + ... from ‘int’ to ‘curl_uint64_t’ -- dmaketgz: release tarball generation using docker + Closes #14041 - For easier reproducibility. +- GHA: add --enable-werror to the quiche job - Mention using this script in RELEASE-PROCEDURE + Closes #14041 - Closes #13388 +- KNOWN_BUGS: three new bugs -Viktor Szakats (16 Apr 2024) + These have lingered in the issue tracker for a long time without action. + We don't expect any fixes in the near term either. Move them to the + KNOWN_BUGS document. -- cmake: update ECH code and minor fixups + Closes #12177 + Closes #12171 + Closes #13350 - - `openssl_check_symbol_exists()` expects a 4th argument now. - Follow-up to edc2702a1fe3a4a5386ffd9aa4f240f0c0197fa2 #13373 + Closes #14042 - - minor comment/script touch-ups. - Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 +Viktor Szakats (27 Jun 2024) - - fix indentation. +- CI: add whitespace checker - Closes #13383 + Fix issues detected. -- tests: fix shellcheck issues in `ech_tests.sh` + Also: - Add double-quotes where missing. + - One of the `.vc` files used LF EOLs, while the other didn't. + Make that one also use LF EOLs, as this is apparently supported by + `nmake`. - Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 - Closes #13382 + - Drop `.dsw` and `.btn` types from `.gitattributes`. + The repository doesn't use them. -- dist: add ECH files to tarball + - Sync section order with the rest of files in + `tests/certs/EdelCurlRoot-ca.prm`. - Also sort `EXTRA_DIST` list in `tests/Makefile.am` and make it diffable. + - Indent/align `.prm` and `.pem` files. - Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 - Closes #13381 + - Delete dummy `[something]` section from `.prm` and `.pem` files. -- openvms: look for `USE_IPV6` in `config.h` (was: `ENABLE_IPV6`) + Mental note: + MSVC `.sln` files seem to accept spaces for indentation and also support + LF line-endings. I cannot test this and I don't know what's more + convenient when updating them, so left them as-is, with specific + exclusions. - The OpenVMS script `config_h.com` is parsing the config header - generated by autotools. Let's make it look for the macro name we now - use universally across the codebase. + Closes #14031 - Follow-up to e411c98f702f0fb38dceec95e7507ef15a00d12c #13349 - Closes #13360 +- CI: fix typo in job name -daniel-j-h (16 Apr 2024) + Closes #14040 -- Dockerfile: for release automation and reproducibility +Stefan Eissing (27 Jun 2024) - Closes #13250 +- tests/httpd: adjust ReadBufferSize for better performance -Stefan Eissing (16 Apr 2024) + - list httpd and caddy versions in scorecard run -- cw-out: improved error handling + Closes #14039 - - remember error encountered in invoking write callback and always fail - afterwards without further invokes +Daniel Stenberg (27 Jun 2024) - - check behaviour in test_02_17 with h2-pausing client +- runtests: fix %VERNUM - Reported-by: Pavel Kropachev - Fixes #13337 - Closes #13340 + It needs to be set to the leading digits and dots only, so that the + `-[date]` suffix strings are not included, as those used in the daily + snapshots. -Daniel Stenberg (16 Apr 2024) + Fixes #14035 + Reported-by: Marcel Raad + Closes #14036 -- version: add "ECH" as a feature +Philip Heiduck (27 Jun 2024) - If available +- CI/synopsis.yml: run on `.md` files - Follow-up to a362962b7 - Closes #13378 + Reported-by: Viktor Szakats + Fixes #14032 + Closes #14037 -- CURLOPT_ECH: polish +Daniel Stenberg (27 Jun 2024) - - remove the pointer to build instructions, it won't work in manpages - - add see-also - - minor white space edits +- verify-synopsis.pl: work with .md files - Closes #13379 + Ref: #14037 + Closes #14038 -Viktor Szakats (16 Apr 2024) +- conncache: done always evaluates to false -- tidy-up: whitespace [ci skip] + Follow-up to c9b95c0bb30f88bf00e1ac -- mbedtls: fix building with v3 in CMake Unity mode + Spotted by CodeSonar - Before this patch the internal feature detection macro - `HAS_MBEDTLS_RESULT_CODE_BASED_FUNCTIONS` was defined in three files, - with an incomplete logic in one of them. In Unity mode that spilled - into another source file and broke the build. + Reviewed-by: Stefan Eissing + Closes #14034 - Closes #13377 +- lib: add a few DEBUGASSERT(data) to aid code analyzers -- cmake: add librtmp/rtmpdump option and detection + ... where 'data' is assumed to always work. - Add CMake option `USE_LIBRTMP`. Disabled by default. + Closes #14033 - This library requires OpenSSL TLS-backend when linked statically. +- RELEASE-NOTES: synced - Follow-up to 6eb9e65781fa1fd8a0bcfe0715187a3a35f09ae4 #13364 - Closes #13373 +Viktor Szakats (26 Jun 2024) -Stephen Farrell (16 Apr 2024) +- tidy-up: use `/usr/bin/env perl` shebang -- TLS: add support for ECH (Encrypted Client Hello) + Most Perl scripts already used it. Sync up the few outliers. - An EXPERIMENTAL feature used with CURLOPT_ECH and --ech. + Closes #14029 - Closes #11922 +Stefan Eissing (26 Jun 2024) -Daniel Stenberg (15 Apr 2024) +- quic: openssl quic, cmake and doc version update to 3.3.0 -- RELEASE-NOTES: synced + Closes #14028 -- multi: introduce SETUP state for better timeouts +- http/3: add shutdown support - Since we can go to the CONNECT state from PENDING, potentially multiple - times for a single transfer, this change introdues a SETUP state that - happens before CONNECT when doing a new transfer. + - openssl-quic shutdown handling + - ngtcp2 shutdown handling + - quiche shutdown handling + - add test_19_06 for verfication - Now, doing a redirect on a handle goes back to SETUP (not CONNECT like - before) and we initilize the connect timeout etc in SETUP. Previously, - we would do it in CONNECT but that would make it unreliable in cases - where a transfer goes in and out between CONNECT and PENDING multiple - times. + Reported-by: Dexter Gerig + Closes #14027 + Fixes #14022 - SETUP is transient, so the handle never actually stays in that state. +Daniel Stenberg (26 Jun 2024) - Additionally: take care of timeouts of PENDING transfers in - curl_multi_perform() +- tests: verify managen - Ref: #13227 - Closes #13371 + 1705: verifies the manpage output -Tal Regev (15 Apr 2024) + 1706: verifies the ascii output -- cmake: forward `USE_LIBRTMP` option to C + Closes #14025 - Define in C `USE_LIBRTMP` if user requested it from cmake. +- runtests: support %DATE for YYYY-MM-DD of right now - Closes #13364 +- runtests: support %VERNUM -Daniel Stenberg (15 Apr 2024) + For the plain version number of the built curl without -DEV etc. Only + digits and dots. -- curl_version_info: provide librtmp version +- managen: only output .RE for manpage output - Ref: https://github.com/curl/curl/pull/13364#issuecomment-2054151942 - Reported-by: talregev on github - Closes #13368 + For ascii they are just rubbish. -blankie (15 Apr 2024) + Closes #14025 -- docs: clarify CURLOPT_MAXFILESIZE and CURLOPT_MAXFILESIZE_LARGE +Tatsuhiro Tsujikawa (26 Jun 2024) - The bounds of the size parameter were not specified, and nor was it - specified how to disable the maximum file size check. +- quic: enable UDP GRO - The documentation also incorrectly stated that CURLOPT_MAXFILESIZE - always returns CURLE_OK and that CURLOPT_MAXFILESIZE_LARGE only returns - CURLE_OK or CURLE_UNKNOWN_OPTION. + Closes #14012 - It also did not mention what the default value is, which is zero. This - commit updates the documentation to make note of all these things. +Stefan Eissing (26 Jun 2024) - Closes #13372 +- quic: require at least OpenSSL 3.3 for QUIC -Patrick Monnerat (15 Apr 2024) + - when checking for QUIC support in OpenSSL, also check + for it being at least 3.3.0 + - remove workarounds for features buggy or missing in 3.2 -- OS400: post-shellcheck changes adjustments + Closes #14026 - Build scripts must be executed by the os/400 shell (sh), not bash which - is a PASE program. +Daniel Stenberg (26 Jun 2024) - Shell function get_make_vars() escaping reworked to match $() subcommand - construct. +- FILEFORMAT.md: mentioned for "client" - Follow-up to 8a622baf9e9233241bbe93d6599c99cb46478614 - Closes #13366 + They can be used to create more files. -Viktor Szakats (15 Apr 2024) + Closes #14024 -- OS400: tidy-up +Marcel Raad (26 Jun 2024) - Drop/fixup mods trying to make some syntax highlighters happier. +- system_win32: add missing curl.h include - Follow-up to 8a622baf9e9233241bbe93d6599c99cb46478614 #13309 - Closes #13362 + It's required for `CURLcode`. -Daniel Stenberg (15 Apr 2024) + Closes https://github.com/curl/curl/pull/14019 -- multi: timeout handles even without connection +Daniel Stenberg (26 Jun 2024) - When there is a "change" in a multi handle and pending handles are moved - back to the main list to be retested if they can proceed further (for - example a previous transfer completed or a connection has a confirmed - multiplexed state), the timeout check in multi_runsingle() would not - trigger because it required an established connection. +- TODO: specify which response codes that make -f/--fail return error - This could make a pending tranfer go back to pending state even though - it had been "in progress" for a longer time than permitted. By removing - the requirement for an associated connection, the timeout check will be - done proper even for transfers that has not yet been assigned one. + Suggestion from the user survey 2024 - Ref #13227 - Reported-by: Rahul Krishna M - Closes #13276 + Closes #14020 -Patrick Monnerat (15 Apr 2024) +Stefan Eissing (26 Jun 2024) -- mprintf: check fputc error rather than matching returned character +- lib: graceful connection shutdown - OS/400 ascii fputc wrapper deviates from the posix standard by the - fact that it returns the ebcdic encoding of the original ascii - character. Testing for a matching value for success will then always - fail. + When libcurl discards a connection there are two phases this may go + through: "shutdown" and "closing". If a connection is aborted, the + shutdown phase is skipped and it is closed right away. - This commit replaces the chariacter comparison by an explicit error - return check. + The connection filters attached to the connection implement the phases + in their `do_shutdown()` and `do_close()` callbacks. Filters carry now a + `shutdown` flags next to `connected` to keep track of the shutdown + operation. - Follow-up to ef2cf58 - Closes #13367 + Filters are shut down from top to bottom. If a filter is not connected, + its shutdown is skipped. Notable filters that *do* something during + shutdown are HTTP/2 and TLS. HTTP/2 sends the GOAWAY frame. TLS sends + its close notify and expects to receive a close notify from the server. -Viktor Szakats (14 Apr 2024) + As sends and receives may EAGAIN on the network, a shutdown is often not + successful right away and needs to poll the connection's socket(s). To + facilitate this, such connections are placed on a new shutdown list + inside the connection cache. -- ci: add CMake build variation, fixup libssh detection in `linux-old` + Since managing this list requires the cooperation of a multi handle, + only the connection cache belonging to a multi handle is used. If a + connection was in another cache when being discarded, it is removed + there and added to the multi's cache. If no multi handle is available at + that time, the connection is shutdown and closed in a one-time, + best-effort attempt. - To test without c-ares and hit `easy_lock.h` on an old system. Use this - new build step to introduce small variations, and also test libssh2. + When a multi handle is destroyed, all connection still on the shutdown + list are discarded with a final shutdown attempt and close. In curl + debug builds, the environment variable `CURL_GRACEFUL_SHUTDOWN` can be + set to make this graceful with a timeout in milliseconds given by the + variable. - Also add workaround to existing job to enable libssh. (CMake's generic - auto-detection doesn't seem to work here.): - ``` - CMake Warning at CMakeLists.txt:908 (find_package): - Could not find a package configuration file provided by "libssh" with any - of the following names: + The shutdown list is limited to the max number of connections configured + for a multi cache. Set via CURLMOPT_MAX_TOTAL_CONNECTIONS. When the + limit is reached, the oldest connection on the shutdown list is + discarded. - libsshConfig.cmake - libssh-config.cmake - ``` - Ref: https://github.com/curl/curl/actions/runs/8661316091/job/23750974358#ste - p:5:69 + - In multi_wait() and multi_waitfds(), collect all connection caches + involved (each transfer might carry its own) into a temporary list. + Let each connection cache on the list contribute sockets and + POLLIN/OUT events it's connections are waiting for. - Closes #13361 + - in multi_perform() collect the connection caches the same way and let + them peform their maintenance. This will make another non-blocking + attempt to shutdown all connections on its shutdown list. -- lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3` + - for event based multis (multi->socket_cb set), add the sockets and + their poll events via the callback. When `multi_socket()` is invoked + for a socket not known by an active transfer, forward this to the + multi's cache for processing. On closing a connection, remove its + socket(s) via the callback. - Before this patch `lib/curl_setup.h` defined these two macros right - next to each other, then the source code used them interchangeably. + TLS connection filters MUST NOT send close nofity messages in their + `do_close()` implementation. The reason is that a TLS close notify + signals a success. When a connection is aborted and skips its shutdown + phase, the server needs to see a missing close notify to detect + something has gone wrong. - After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features. - (Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used. + A graceful shutdown of FTP's data connection is performed implicitly + before regarding the upload/download as complete and continuing on the + control connection. For FTP without TLS, there is just the socket close + happening. But with TLS, the sent/received close notify signals that the + transfer is complete and healthy. Servers like `vsftpd` verify that and + reject uploads without a TLS close notify. - This patch doesn't change the way HTTP/3 is enabled via autotools - or CMake. Builders who enabled HTTP/3 manually by defining both of - these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`. + - added test_19_* for shutdown related tests + - test_19_01 and test_19_02 test for TCP RST packets + which happen without a graceful shutdown and should + no longer appear otherwise. + - add test_19_03 for handling shutdowns by the server + - add test_19_04 for handling shutdowns by curl + - add test_19_05 for event based shutdowny by server + - add test_30_06/07 and test_31_06/07 for shutdown checks + on FTP up- and downloads. - Closes #13352 + Closes #13976 -- build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`) +Daniel Stenberg (25 Jun 2024) - Before this patch, two macros were used to guard IPv6 features in curl - sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use - the latter for consistency with other similar switches. +- managen: fix blank line detection - `-DENABLE_IPV6` remains accepted for compatibility as a synonym for - `-DUSE_IPV6`, when passed to the compiler. + Follow-up to d14a53eea7b87 which ruined the output somewhat. - `ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc` - options to control this feature. + Closes #14017 - Closes #13349 +- managen: output tabs for each 8 leading spaces -Dan Fandrich (12 Apr 2024) + This replacing of eight leading spaces into tabs was already done for + the embedded uncompressed version in tool_hugehelp.c so it does not save + anything there. But the gzip compressed version ends up almost 2K + smaller. -- DISTROS: mark rolling release distros + The output in a terminal should be identical. - These are ones that are unlikely to have back-ported curl patches. + Before using TABs: - Closes #13353 + curl.txt 282492 bytes + curl.txt.gz 73261 bytes -Daniel Stenberg (12 Apr 2024) + With this change applied: -- mbedtls: cut off trailing newlines from debug logs + curl.txt 249382 bytes + curl.txt.gz 71470 bytes - To avoid double newlines in the output. + Closes #14016 - Reported-by: Gisle Vanem - Fixes #13321 - Closes #13356 +- managen: error on trailing blank lines in input files -- RELEASE-NOTES: synced + Ref: #14014 + Closes #14015 -Stefan Eissing (12 Apr 2024) +Viktor Szakats (25 Jun 2024) -- CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported +- tidy-up: more whitespace - - tests for 'size_request' and other stats reported, for - presence and consistency + Closes #14014 - Reported-by: Jonatan Vela - Fixes #13269 - Closes #13275 +Stefan Eissing (25 Jun 2024) -Viktor Szakats (11 Apr 2024) +- multi: multi_getsock(), check correct socket -- dist: add files missing from release tarball + - in phase CONNECTING/TUNNELING/PROTOCONNECT, retrieve + the socket from the connection filters and do not rely + on `conn->sockfd` being already set by the transfer. + - this applies to the default behaviour, a protocol handler + may override this via its callbacks. + - add a warning message in multi_getsock() when the transfer + is expected to have something in its pollset, but instead + it is empty. - Closes #13346 + Reported-by: saurabhsingh-dev on github + Fixes #13998 + Closes #14011 -- ci: parallelize more, tidy up cmake commands (distcheck, macos) +Daniel Stenberg (25 Jun 2024) - Also enable `-DCURL_WERROR=ON` in the Linux cmake build test. +- managen: fix each options footer to end with newline - Closes #13343 + A previous change sometimes made a command line option's description not + end with a newline immediately before the next command line. -Toon Claes (11 Apr 2024) + Also widened the lines to wrap on column 79 instead of 78. -- docs: add CURLOPT_NOPROGRESS to CURLOPT_XFERINFOFUNCTION example + Closes #14010 - It's important to set `CURLOPT_NOPROGRESS` to `0` if you want your - transfer callback function, set by `CURLOPT_XFERINFOFUNCTION`, getting - called. To emphasize this to the users, add this to the code example. +Alex Snast (25 Jun 2024) - Closes #13348 +- wolfssl: assume key_file equal to clientcert in the absence of key_file -RainRat (11 Apr 2024) + When user sets CURLOPT_SSLCERT but leaves CURLOPT_SSLKEY unset assume + the path passed in CURLOPT_SSLCERT holds the ssl key which is what we do + in openssl implementation. -- misc: fix typos + Fixes #14007 + Closes #14008 - Closes #13344 +Viktor Szakats (24 Jun 2024) -Colin Leroy-Mira (11 Apr 2024) +- autotools: fix pkg-config names (zstd, ngtcp2*) -- file: add support for getting basic directory listings + Also verified that all names now match up with CMake. - Not supported on Windows (yet) + Follow-up to f057de5a1a950a90d1920021db152a4b695f1a8a #13911 + Follow-up to eeab0ea7aa19af61af881e8a0bf9ff1f2e28ef79 #13994 + Reported-by: 李四 + Fixes #14005 + Closes #14006 - Closes #13137 +- tidy-up: whitespace [ci skip] -Viktor Szakats (11 Apr 2024) +Daniel Stenberg (24 Jun 2024) -- ci: add curl-for-win builds: Linux MUSL, macOS, Windows +- cmdline-docs: "added in" cleanups - Linux MUSL (llvm/clang), macOS Apple clang, Windows (llvm/clang). + - markup fixes + - remove some mentions of < 7.60.0 changes - Configured with HTTP/2 and HTTP/3 and other dependencies (the default - curl-for-win) for a comprehensive build test. + Closes #14003 - ``` - curl 8.8.0-DEV (x86_64-unknown-linux-musl) libcurl/8.8.0-DEV LibreSSL/3.9.1 z - lib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 - ngtcp2/1.4.0 nghttp3/1.2.0 - Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns - mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss - Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefil - e libz NTLM PSL SSL threadsafe UnixSockets zstd +- RELEASE-NOTES: synced - curl 8.8.0-DEV (x86_64-apple-darwin) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1. - 3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp - 2/1.4.0 nghttp3/1.2.0 - Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns - ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws w - ss - Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefil - e libz NTLM PSL SSL threadsafe UnixSockets zstd +- managen: "added in" fixes - curl 8.8.0-DEV (x86_64-w64-mingw32) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3 - .1 brotli/1.1.0 zstd/1.5.6 WinIDN libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 - ngtcp2/1.4.0 nghttp3/1.2.0 - Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns - ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws w - ss - Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerb - eros Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd - ``` + - up the limit: remove all mentions of 7.60 or earlier from manpage + 7.60 is 6 years old now. + - warn on "broken" added in lines, as they avoid detection + - fixup added in markup in a few curldown files - Limited to x64, because for build testing the additional CPUs don't add - much value compared to the extra build time. They can be enabled easily - if deemed useful. + Closes #14002 - To the extent of curl-for-win configuration options, it's trivial to add - further build combinations. +Matt Jolly (24 Jun 2024) - Closes #13335 +- configure: fix pkg-config library name 'libnghttp3' -- OS400: fix shellcheck warnings in scripts + Closes #13994 - - use `$()` instead of backticks, and re-arrange double-quotes inside. - - add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu` - .) - - add `-n` to a few `if`s. - - shorten redirections by using `{} >` (as shellcheck recommended). - - silence warnings where variables were detected as unused (SC2034). - - a couple misc updates to silence warnings. - - switch to bash shebang for `-ot` feature. - - split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, ` - $(dirname \`) +Daniel Stenberg (24 Jun 2024) - Also enable CI checks for OS/400 shell scripts. +- managen: cleanups to generate nicer-looking output - Ref: #13307 - Closes #13309 + - output "see also" last + - when there are multiple mutex items, use commas between all of them + except the last. + - call them mututally exclusive WITH not TO other options. + - remove trailing space from added in, add newline prefix + - smoother language for requires -Stefan Eissing (11 Apr 2024) + Closes #14001 -- lib: add Curl_xfer_write_resp_hd +- configure: require a QUIC library if nghttp3 is used - Add method in protocol handlers to allow writing of a single, - 0-terminated header line. Avoids parsing and copying these lines. + Instead of just silently disabling HTTP/3. - Closes #13165 + Reported-by: Matt Jolly + Fixes #13995 + Closes #13999 -- llist: add Curl_llist_append() +- docs/cmdline-opts: remove two superfluous "Added in" mentions - - use for better readability in all places where the "insert_next" - actually performs an append to the list - - add some tests in unit1300 + The key "added in" phrase for the option itself is added automatically. - Closes #13336 + Closes #14000 -- gnutls: lazy init the trust settings +- cookie-jar.md: see also --junk-session-cookies - - delay loading of trust anchors and CRLs after the ClientHello - has been sent off - - add tracing to IO operations - - on IO errors, return the CURLcode of the underlying filter + Closes #13996 - Closes #13339 +- runtests: support crlf="yes" for the section -Marcel Raad (10 Apr 2024) +- TODO: -h option -- http_negotiate: fix `CURL_DISABLE_PROXY` build + Support "curl -h --insecure" etc to output the manpage section for the + --insecure command line option in the terminal. Should be possible to + work with either long or short versions of command line options. - `proxyuserpwd` was removed from `dynamically_allocated_data` in commit - f46385d36df. + Closes #13990 - Closes https://github.com/curl/curl/pull/13334 +- trace-ascii.md: mention "%" for stderr -Viktor Szakats (10 Apr 2024) + Closes #13991 -- quic: fixup duplicate static function name (for cmake unity) +- connect-to.md: expand with examples - Visible in daily curl-for-win builds: - https://github.com/curl/curl-for-win/actions/runs/8621925870 + - add referer from the resolve section to connect-to if user wants + wildcard for the port number - ``` - lib/vquic/curl_ngtcp2.c:1916:12: error: redefinition of 'ossl_new_session_cb' - static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid) - ^ - lib/vtls/openssl.c:2978:12: note: previous definition is here - static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid) - ^ - ``` - https://github.com/curl/curl-for-win/actions/runs/8621925870/job/23631885439# - step:3:6965 + Closes #13989 - Follow-up to 3210101088dfa3d6a125d213226b092f2f866722 #13172 - Closes #13332 +- TODO: connect to multiple IPs in parallel -- appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings + Closes #13986 - Tests were consistently flaky for a while. +- dump-header.md: mention minus for stdout - Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers: - ``` - C:/projects/curl/lib/vtls/schannel.c(688): - warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater s - ize - C:/projects/curl/lib/vtls/schannel_verify.c(642): - warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater s - ize - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44 - kymgc0nif#L106 - - Closes #13330 + Closes #13985 -Daniel Stenberg (10 Apr 2024) +- CURLOPT_RESOLVE.md: mention hostname can be wildcard ('*') -- projects: drop MSVC project files for recent versions + Closes #13983 - We encourage users to generate visual studio project files using CMake. +Andy Pan (22 Jun 2024) - We keep project files in git for ancient visual studio versions that - cmake cannot generate files for, but we no longer ship the project files - in the tarballs. +- cf-socket: optimize curlx_nonblock() and check its return error - appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013) + Reviewed-by: Stefan Eissing + Closes #13942 - Co-Authored-by: Viktor Szakats - Co-Authored-by: Jay Satiro +z2_ (22 Jun 2024) - Closes #13311 +- x509asn1: prevent NULL dereference -Viktor Szakats (9 Apr 2024) + Closes #13978 -- cmake: use namespaced custom target names +Daniel Stenberg (19 Jun 2024) - Rename custom target to namespaced (unique) names to avoid colliding - with 3rd-party projects (e.g. libzip) built together with curl. +- unit2604: use 'unitfail' instead of 'error' variable - Reported-by: hammlee96 on github - Fixes #13324 - Closes #13326 + Since the framework is already returning that variable by default. + Avoids a warning for unreachable code. -- appveyor: re-enable OpenSSL 3, bump to 3.2.1 + Reported-by: Tal Regev + Fixes #13967 + Closes #13973 - Ref: b62454a875d70f93ab5347c050903596feb45a23 #13266 - Closes #13329 +- KNOWN_BUGS: TFTP tests fail on OpenBSD -Stefan Eissing (9 Apr 2024) + Closes #13623 + Closes #13975 -- CI: upgrade openssl version to 3.3.0 for openssl-quic +- VULN-DISCLOSURE-POLICY: NULL dereferences and crashes - Closes #13328 + If a malicious server can trigger a NULL dereference in curl or + otherwise cause curl to crash (and nothing worse), chances are big that + we do not consider that a security problem. -Daniel Stenberg (9 Apr 2024) + Closes #13974 - RELEASE-NOTES: synced - Bump to 8.8.0-DEV +Sergey Markelov (19 Jun 2024) -- curl_multi_waitfds.md: add protocol mention +- mbedtls: support CURLOPT_CERTINFO - Follow-up to 02beac6bb6b + Closes #13113 -Dmitry Karpov (9 Apr 2024) +Daniel Stenberg (19 Jun 2024) -- lib: add curl_multi_waitfds +- x509asn1: ASN1tostr() should fail when 'constructed' is set - New function call, similar to curl_multi_fdset() + This is a regression from my refactor in 623c3a8fa0bdb (#12808) - Closes #13135 + Follow-up to 623c3a8fa0bdb2751f14b37417 -Viktor Szakats (9 Apr 2024) + Closes #13972 -- dist: verify tarball reproducibility in CI +- x509asn1: remove two static variables - Closes #13327 + cnOID and sanOID were not used outside of the OID table anyway -Stefan Eissing (9 Apr 2024) + Closes #13971 -- tests: stabilitze test_02_23* +brian m. carlson (18 Jun 2024) - - h2-download now always opens the output file on first write callback - invocation, if it will pause the transfer or not. - - Checks on output files then does not depend on the amount of data curl - has collected for the first write. +- TODO: TLS channel binding - Closes #13323 + Closes #13483 -- tls: fix compile issues on old-linux CI +Tal Regev (17 Jun 2024) - Follow-up to 3210101088dfa - Closes #13325 +- cmake: add CURL_USE_GSASL option with detection + CI test -Viktor Szakats (9 Apr 2024) + Reviewed-by: Viktor Szakats + Closes #13948 -- dist: add reproducible dir entries to tarballs +Daniel Stenberg (16 Jun 2024) - In the initial implementation of reproducible tarballs, they were - missing directory entries, while .zip archives had them. It meant - that on extracting the tarball, on-disk directory entries got the - current timestamp. +- x509asn1: make Curl_extract_certinfo store error message - This patch fixes this by including directory entries in the tarball, - with reproducible timestamps. It also moves sorting inside tar, - to ensure reproducible directory entry timestamps on extract - (without the need of `--delay-directory-restore` option, when - extracting with GNU tar. BSD tar got that right by default.) + To help us all better understand where the error actually comes from. - GNU tar 1.28 (2014-07-28) introduced `--sort=`. + Ref: #13958 + Closes #13959 - Ref: https://github.com/curl/curl/pull/13299#discussion_r1555957350 - Follow-up to 860cd5fc2dc8e165fadd2c19a9b7c73b3ae5069d #13299 - Closes #13322 +Viktor Szakats (15 Jun 2024) -Stefan Eissing (9 Apr 2024) +- appveyor: dump build logs on failure in VS2008 jobs -- tls: use shared init code for TCP+QUIC + This seems to be the only way to see what actual toolchain commands were + run, and with what arguments. - Closes #13172 + Without `dos2unix`, `cat` output comes out empty. -Daniel Stenberg (9 Apr 2024) + Closes #13957 -- .mailmap: update Gisle's preferred email +- cmake: fix quotes when appending multiple options (SecureTransport) -Jan Macku (9 Apr 2024) + Copied from a vcpkg distro patch: + https://github.com/microsoft/vcpkg/blob/02745e0f4749d1f51d2025824209408f5a6c3 + 614/ports/curl/dependencies.patch#L43C38-L44 -- doc: pytest `--repeat` -> `--count` + Ref: https://github.com/microsoft/vcpkg/pull/38847 + Ref: https://github.com/microsoft/vcpkg/commit/795f2f137e6cf6d985fcc927bffcaf + 9c0a96e4ac + Ref: https://github.com/microsoft/vcpkg/pull/38847/commits/36f0c917de5319e953 + 61451fc0aef0698b264874#diff-ab5c23e5dc5df412539cc93e24b37abbc588e1918236f8abc + 019d676b270c85fR39 (sub-commit) - Pytest doesn't have a `--repeat` option, but it does have a `--count` - option. + Authored-by: Kai Pastor + Closes #13953 - ``` - --count=COUNT Number of times to repeat each test - ``` +Daniel Stenberg (15 Jun 2024) - Closes #13218 +- CURLOPT_NETRC.md: clarify what it does on Windows -Daniel Stenberg (9 Apr 2024) + Closes #13956 -- src/Makefile.am: access curl.txt using a relative path, not abs +- KNOWN_BUGS: "HTTP/2 + TLS spends a lot of time in recv" - ... to make it work when mounted using different mount points. Like when - generated/used inside and outside of a docker image. + Closes #13416 + Closes #13955 - Closes #13320 +- RELEASE-NOTES: synced -- build: remove MacOSX-Framework script +Yedaya Katsman (14 Jun 2024) - I don't think this is much used these days. +- examples: add missing binaries to .gitignore - Also remove the libcurl.plist file used (only) by this script + They were showing as changed when built. Add them sorted alphabetically, + while also moving a few more entries to sorted order. - Closes #13313 + Closes #13952 -- release-tools.sh: store the timestamp and release tag too +- docs: reference non deprecated libcurl options - When maketgz invokes this script to generate the docs/RELEASE-TOOLS.md - file that gets bundled in the release, it now also passes on the exact - timestamp and version number so that those details also get mentioned in - the document. They will help users reproduce an identical tarball. + There are a places where man pages reference deprecated CURLOPT options, + where it doesn't make sense, replace them with the reccomended + replacement option. - Closes #13319 + also remove reference to the removed mesalink TLS backend -Viktor Szakats (8 Apr 2024) + Closes #13951 -- GHA: disable permissions where missing +Daniel Stenberg (14 Jun 2024) - Reviewed-by: Daniel Stenberg - Closes #13306 +- gnutls: pass in SNI name, not hostname when checking cert -Stefan Eissing (8 Apr 2024) + The function we use is called 'gnutls_x509_crt_check_hostname()' but if + we pass in the hostname with a trailing dot, the check fails. If we pass + in the SNI name, which cannot have a trailing dot, it succeeds for + https://pyropus.ca./ -- CI: update component versions + I consider this as a flaw in GnuTLS and have submitted this issue + upstream: - - ngtcp2: v1.4.0 - - nghttp3: v1.2.0 - - nghttp2: v1.61.0 - - mod_h2: v2.0.27 + https://gitlab.com/gnutls/gnutls/-/issues/1548 - Closes #13316 + In order to work with old and existing GnuTLS versions, we still need + this change no matter how they view the issue or might change it in the + future. -Jérôme Leclercq (8 Apr 2024) + Fixes #13428 + Reported-by: Ryan Carsten Schmidt + Closes #13949 -- CMake: check fseeko after detecting HAVE_FILE_OFFSET_BITS +- BINDINGS: update java link to one that exists - Closes #13264 + The previous java binding seems to have vanished. Link to one that still + exists. -Stefan Eissing (8 Apr 2024) + Bug: https://github.com/curl/everything-curl/issues/456 + Reported-by: Jiang Wenjian + Closes #13950 -- http2: emit RST when client write fails +renovate[bot] (14 Jun 2024) - - When the writing of response data fails, reset the stream - and do not return a callback error to nghttp2. That would - be a fatal error for the connection and harm other requests. - - add test cases for various abort scenarios +- GHA: update pinned actions - Reported-by: Konstantin Kuzov - Fixes #13292 - Closes #13298 + - github/codeql-action digest to 23acc5c + - actions/checkout digest to 692973e + - rojopolis/spellcheck-github-actions digest to d354a4d -Kailun Qin (8 Apr 2024) + Closes #13935 + Closes #13945 + Closes #13946 -- mbedtls: call mbedtls_ssl_setup() after RNG callback is set +Jay Satiro (14 Jun 2024) - Since mbedTLS v3.6.0, the RNG check added in ssl_conf_check() will fail - if no RNG is provided when calling mbedtls_ssl_setup(). +- tool_cb_hdr: allow etag and content-disposition for 3xx reply - Therefore, mbedtls_ssl_conf_rng() needs to be called before the SSL - context is passed to mbedtls_ssl_setup(). + - Parse etag and content-disposition headers for 3xx replies. - Ref: https://github.com/Mbed-TLS/mbedtls/commit/b422cab052b51ec84758638d6783d - 6ba4fc60613 + For example, a server may send a content-disposition filename header + with a redirect reply (3xx) but not with the final response (2xx). + Without this change curl would ignore the server's specified filename + and continue to use the filename extracted from the user-specified URL. - Signed-off-by: Kailun Qin - Closes #13314 + Prior to this change, 75d79a4 had limited etag and content-disposition + to 2xx replies only. -Daniel Stenberg (8 Apr 2024) + Tests-by: Daniel Stenberg -- NTLM_WB: drop support + Reported-by: Morgan Willcock + Fixes https://github.com/curl/curl/issues/13302 + Closes #13484 - The feature has not worked for months and has been marked as DEPRECATED - for six+ months. +Daniel Stenberg (13 Jun 2024) - Closes #13249 +- transfer: set CSELECT_IN if there is data pending -- curl_trc: fix build error when lacking verbose messages + When aborting the transfer loop early, like when there is rate limiting + in effect, there might be buffered data already read off the socket so + the socket might not signal reability. Therefore we must set the + CSELECT_IN manually if data_pending_() suggests there might be more data + to get. This is particularly noticeable with SSH when the underlying + library has drained the socket and holds pending data in its buffer. - Follow-up from 0b28ece657b2273 - Closes #13312 + Reported-by: alervd on github + Fixes #13695 + Closes #13943 -Viktor Szakats (8 Apr 2024) +Viktor Szakats (13 Jun 2024) -- contrithanks: honor `CURLWWW` variable +- cmake: enable SOVERSION for Cygwin and `CMAKE_DLL_NAME_WITH_SOVERSION` - Reviewed-by: Daniel Stenberg - Closes #13315 + - enable SOVERSION when `CMAKE_DLL_NAME_WITH_SOVERSION=ON` is set. + Ref: https://cmake.org/cmake/help/v3.27/variable/CMAKE_DLL_NAME_WITH_SOVERS + ION.html + Use: https://github.com/search?q=-DCMAKE_DLL_NAME_WITH_SOVERSION&type=code -- GHA: add shellcheck job and fix warnings, shell tidy-ups + - enable SOVERSION for Cygwin builds by default. - Reviewed-by: Daniel Stenberg - Closes #13307 + Ref: #13936 + Ref: #13944 + Closes #13898 -- dist: do not require Perl in `maketgz` +- cmake: allow SOVERSION override with `CURL_LIBCURL_SOVERSION` - Perl remains required for the tarball build process. + Allow overriding SOVERSION with the new CMake option: + `CURL_LIBCURL_SOVERSION=ON/OFF` - Follow-up to 860cd5fc2dc8e165fadd2c19a9b7c73b3ae5069d #13299 + For certain target platforms the shared libcurl library filename + contains the SOVERSION. This new option allows to enable/disable + this behavior manually. If set, it takes precedence over the default + setting. - Reviewed-by: Daniel Stenberg - Closes #13310 + Ref: #13898 + Closes #13944 -Daniel Stenberg (8 Apr 2024) +renovate[bot] (13 Jun 2024) -- RELEASE-NOTES: synced +- Dockerfile: update debian:bookworm-slim to 84d83b2 -- docs/cmdline-opts: invoke managen using a relative path + Closes #13934 - ... no need to use an absolute path, that makes the build unncessarily - fail if invoked using a different mount point. managen now takes options - to find the input files. +Daniel Stenberg (13 Jun 2024) - Update test1478 to provide the dir arguments to managen +- configure: use AC_MSG_WARN for TLS/experimental warning texts - Closes #13281 + - no longer warns for mbedtls + - warns for each item on individual lines + - no longer shows irrelevant TLS libraries when multiple are selected + - removes ech repetition -- GHA: add valgrind to a wolfSSL build + Closes #13941 - Closes #13274 +- GHA: detect and warn for more English contractions -Viktor Szakats (7 Apr 2024) + As we try to avoid them in curl documentation -- dist: `set -eu`, fix shellcheck, make reproducible and smaller tarballs + Closes #13940 - - set bash `-eu` and fix fallouts. - - fix shellcheck warnings. - - set and use `SOURCE_DATE_EPOCH` for reproducibility. - Authored-by: Daniel J. H. - Ref: #13280 - - set `TZ=UTC` and `LC_ALL=C` for reproducibility. - - make file timestamps in tarball/zip reproducible. - - make directory timestamps in zip reproducible. - - make timestamps of tarballs/zip reproducible. - - make file order in tarball/zip reproducible. - - omit extra file metadata from zip for reproducibility. - - use maximum zip compression. - - use POSIX `ustar` tarball format to avoid supply chain vulnerability: - https://seclists.org/oss-sec/2021/q4/0 - - make uid/gid in tarball reproducible. - - omit owner user/group names from tarball for reproducibility and privacy. - - omit current timestamp from .gz header for reproducibility. - - display SHA-256 hashes of produced tarballs/zip. - - fix whitespace. +Stefan Eissing (13 Jun 2024) - `.tar.gz` also became smaller in the process: 4,462,311 -> 4,148,249 bytes (8 - .7.1) +- transfer: do not use EXPIRE_NOW while blocked - Requires GNU tar, GNU date, `sha256sum`. + - When a transfer sets `data->state.select_bits`, it is + scheduled for rerun with EXPIRE_NOW. If such a transfer + is blocked (due to PAUSE, for example), this will lead to + a busy loop. + - multi.c: check for transfer block + - sendf.*: add Curl_xfer_is_blocked() + - sendf.*: add client reader `is_paused()` callback + - implement is_paused()` callback where needed - Reviewed-by: Daniel Stenberg - Ref: #13250 - Closes #13299 + Closes #13908 -Gisle Vanem (7 Apr 2024) +renovate[bot] (13 Jun 2024) -- tests/http: fix compiler warning +- ci: update dependency ngtcp2/ngtcp2 to v1.6.0 - - Init result code variable to fix clang warning that it may be used - uninitialized. + Closes #13939 - Fixes https://github.com/curl/curl/issues/13301 - Closes https://github.com/curl/curl/pull/13304 +- ci: update ngtcp2/nghttp3 to v1.4.0 -Stefan Eissing (6 Apr 2024) + Closes #13938 -- vquic: use new curl_int64_t type +Viktor Szakats (13 Jun 2024) - - add curl_int64_t signed 64-bit type for lib use +- cmake: stop setting SOVERSION for the static lib target - - define CURL_PRId64, CURL_PRIu64 format ids + Also move the logic closer to its use and related tidy-ups. - - use curl_int64_t in vquic + Cherry-picked from #13898 + Closes #13936 - curl_int64_t signed complements the existing curl_uint64_t unsigned. +Patrick Monnerat (13 Jun 2024) - Note that `curl_int64_t` and `int64_t` are assignable from each other - but not identical. Some platforms with 64 long type defint int64_t as - "long long" (staring at macOS) which messes up things like pointers and - format identifiers. +- os400: make it compilable again - Closes https://github.com/curl/curl/pull/13293 + A newly introduced use of getsockname() in the cli tool makes it require + the ascii wrapper module, which is not available outside of the library: + as the tool only uses the address family field (binary), disable + wrappers outside of libcurl. -Jay Satiro (5 Apr 2024) + Fix setsockopt() parameter type mismatch using a (void *) cast. -- lib: use multi instead of multi_easy for the active multi + Sync ILE/RPG binding. - - Use data->multi and not data->multi_easy to refer to the active multi. + Closes #13930 - The easy handle's active multi is always data->multi. +Viktor Szakats (13 Jun 2024) - This is a follow up to 757dfdf which changed curl so that an easy handle - used with the easy interface and then multi interface cannot have two - different multi handles associated with it at the same time - (data->multi_easy from the easy interface and data->multi from the multi - interface). +- libcurl.pc: add `Requires.private`, `Requires` for static linking - Closes https://github.com/curl/curl/pull/12665 + - cmake: populate for dependencies. + - autotools: populate for dependencies. + (including mbedtls, though the script does not detect + mbedtls through pkgconfig. mbedtls 3.6.0 now supports it.) -Viktor Szakats (5 Apr 2024) + Skip dealing with gssapi in this patch. -- tidy-up: whitespace [ci skip] + Fixes #864 + Closes #13911 -Daniel Stenberg (5 Apr 2024) +- cmake: bring `curl-config.cmake` closer to `FindCURL` -- makefile: remove the sorting from the vc-ide action + Set `CURL_LIBRARIES` and `CURL_INCLUDE_DIRS` variables + for compatibility with CMake's `FindCURL.cmake`: + https://github.com/Kitware/CMake/blob/b411d0146c2e06acfb0c823bb039e99f0191b61 + 1/Modules/FindCURL.cmake#L209 - This target generates the MSVC project files. This change removes the - extra sorting and instead makes the script use the order of the files as - listed in the variables - which are mostly sorted anyway. + For dependent projects, CMake's suggestion is to replace + `CURL_LIBRARIES` with `CURL::libcurl`, and drop `CURL_INCLUDE_DIRS`. - This is an attempt to make the project file generation more easily - reproducible. + Reported-by: Aurélien Pierre + Ref: https://curl.se/mail/lib-2024-06/0014.html + Ref: https://gitlab.kitware.com/cmake/cmake/-/issues/24580 + Closes #13897 - Ref: #13250 - Closes #13294 +Daniel Stenberg (13 Jun 2024) -Gisle Vanem (5 Apr 2024) +- tool_getparam: fix the bsearch call for ip-tos names -- bearssl: fix compiler warnings + Follow-up to 3c20ae08b9591 + Reported-by: Samuel Chiang + Fixes #13932 + Closes #13933 - "variables may be uninitialized when used" +- request: change the struct field bodywrites to a bool, only for hyper - Fixes #13290 - Closes #13297 + Only hyper needs to know this, and it can use it as a boolean. -Daniel Stenberg (5 Apr 2024) + Closes #13928 -- DISTROS: Cygwin updates +Andy Pan (12 Jun 2024) - Brought-by: Brian Inglis - Fixes #13258 - Co-authored-by: Viktor Szakats - Closes #13279 +- test: fix CURLOPT_TCP_KEEPCNT typo -Stefan Eissing (5 Apr 2024) + Follow up to b77d627d242 -- lib: add trace support for client reads and writes + Closes #13931 - - add `CURL_TRC_READ()` and `CURL_TRC_WRITE()` - - use in generic client writers and readers, as well - as http headers, chunking and websockets +Daniel Stenberg (12 Jun 2024) - Closes #13223 +- http: remove "struct HTTP" -Michał Antoniak (5 Apr 2024) + It is not actually used anymore and only contained a dummy struct field. + Remove all traces and uses of it. -- urldata: remove fields not used depending on used features + Closes #13927 - Reduced size of dynamically_allocated_data structure. +- cd2nroff: convert two warnings to errors - Reduced number of stored values in enum dupstring and enum dupblob. This - affects the reduced array placed in the UserDefined structure. + Since the warnings tend to get missed too easily and these are problems + we rather want addressed than letting slide. - Closes #13188 + Closes #13929 -Viktor Szakats (5 Apr 2024) +- urlapi: use a correct value for CURLU_NO_GUESS_SCHEME -- cmake: enable `-pedantic-errors` for clang when `CURL_WERROR=ON` + It was mistakenly set to the same value as CURLU_GET_EMPTY uses. - clang doesn't have the issues of GCC and old CMake versions. + Reported-by: Patrick Monnerat + Bug: https://github.com/curl/curl/commit/655d44d139489625e77cf6790d36 + Closes #13926 - Note: This introduces asymmetry with autotools, which only enables - this for GCC. +- file: separate fake headers and body with a stand-alone CRLF - Reviewed-by: Daniel Stenberg - Closes #13286 + Instead of bolting on the extra CRLF to the final header - as that makes + the behavior inconsistent and not as documented. The final CRLF is now + also made unconditional, just like it is for HTTP. -- cmake: fix `CURL_WERROR=ON` for old CMake and use it in GHA/linux-old + Reported-by: dogma + Bug: https://curl.se/mail/lib-2024-06/0033.html + Closes #13925 - - cmake: fix `-pedantic-errors` for old CMake with `CURL_WERROR=ON` set. +- RELEASE-NOTES: synced - `-pedantic-errors` option throws a warning with GCC (all versions) and - makes `check_symbol_exists()` fail in CMake versions older than - v3.23.0 (2022-03-29), when CMake introduced a workaround: +Andy Pan (12 Jun 2024) - https://gitlab.kitware.com/cmake/cmake/-/issues/13208 - https://gitlab.kitware.com/cmake/cmake/-/commit/eeb45401163d831b8c841ef6eba - 81466b4067b68 - https://gitlab.kitware.com/cmake/cmake/-/commit/1ab7c3cd28b27ca162c4559e102 - 6e5cad1898ade +- tcpkeepalive: add CURLOPT_TCP_KEEPCNT and --keepalive-cnt - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + Closes #13885 - - set `CURL_WERROR=ON` for the `linux-old` job in CI. +Daniel Stenberg (12 Jun 2024) - Closes #13282 +- TODO: make it "Add missing features to TLS backends" -- lib: use `#error` instead of invalid syntax in `curl_setup_once.h` + ... instead of just mentioning CA caching. - Reviewed-by: Daniel Stenberg - Closes #13287 + Closes #13924 -Daniel Stenberg (5 Apr 2024) +Orgad Shaneh (11 Jun 2024) -- GHA: on macOS remove $HOME/.curlrc +- curl: support VLAN Priority: --vlan-priority - A recent image upgrade added a $HOME/.curlrc by default using --ipv4. + Add --vlan-priority option to the command line tool for setting VLAN + priority. - Ref: https://github.com/actions/runner-images/pull/9586 - Fixes #13284 - Closes #13285 + Closes #13907 -Viktor Szakats (4 Apr 2024) +RainRat (11 Jun 2024) -- cmake: fixup `DEPENDS` filename +- misc: fix typos - Fixing: - ``` - make[2]: Circular docs/curl-config.1 <- docs/curl-config.1 dependency dropped - . - make[2]: Circular docs/mk-ca-bundle.1 <- docs/mk-ca-bundle.1 dependency dropp - ed. - ``` - Ref: https://github.com/curl/curl/actions/runs/8559617487/job/23456740844?pr= - 13282#step:6:18 + Closes #13923 - Follow-up to 5023ffad2c27d4b916ddb91800f99ecc5d3aad07 #13197 - Closes #13283 +Daniel Stenberg (11 Jun 2024) -- GHA: enable unity mode for cmake jobs + tidy-ups +- CURLOPT_ECH.md: remove repeated 'if' - Unity mode is not supported by CMake v3.7.2 used in linux-old, but - enable it anyway for consistency and to kick in automatically once - migrating to a newer old Linux in the future. + Closes #13922 - Also: - - replace `CMAKE_COMPILE_WARNING_AS_ERROR` with `CURL_WERROR`. - - delete default build option `PICKY_COMPILER=ON`. +- vms: fixed language in comment - Closes #13277 + It started with me fixing a repeated "are are" but the wording was + incomprehensible so I tried to untangle it. -Dan Fandrich (4 Apr 2024) + Closes #13921 -- CI: Add CI build on Debian stretch to test old support +Stefan Eissing (11 Jun 2024) - This version still has ELTS support and contains some old versions of - key components like cmake to help prevent us from breaking that support. +- lib: xfer_setup and non-blocking shutdown - Closes #13029 + - clarify Curl_xfer_setup() with RECV/SEND flags and different calls for + which socket they operate on. Add a shutdown flag for secondary + sockets + - change Curl_xfer_setup() calls to new functions + - implement non-blocking connection shutdown at the end of receiving or + sending a transfer -Stefan Eissing (4 Apr 2024) + Closes #13913 -- request: paused upload on completed download, assess connection +Daniel Stenberg (11 Jun 2024) - A transfer with a completed download that is still uploading needs to - check the connection state when it is PAUSEd, since connection - close/errors would otherwise go unnoticed. +- test1486: verify that write-out.md and tool_writeout.c are in sync - Reported-by: Sergey Bronnikov - Fixes #13260 - Closes #13271 + - also verify alphabetialal order in the source + - add two missing variables to write-out.md -Daniel Stenberg (4 Apr 2024) + Closes #13920 -- url: do not URL decode proxy crendentials +Viktor Szakats (11 Jun 2024) - The two options CURLOPT_PROXYUSERNAME and CURLOPT_PROXYPASSWORD set the - actual names as-is, not URL encoded. +- GHA: add cmake MSYS2 native job - Modified test 503 to use percent-encoded strings in the credential - strings that should be passed on as-is. + curl, libcurl, examples, build-only. - Reported-by: Sergey Ogryzkov - Fixes #13265 - Closes #13270 + To compare build behaviour with autotools. -Viktor Szakats (4 Apr 2024) + Closes #13917 -- appveyor: enable cmake unity mode by default +Daniel Stenberg (11 Jun 2024) - Leave one non-unity cmake job. This makes the jobs finish slightly - quicker, while giving more coverage for unity issues. +- openssl: shortcut store_expired for negative timeouts - Before: - https://ci.appveyor.com/project/curlorg/curl/builds/49496977 - https://ci.appveyor.com/project/curlorg/curl/builds/49500372 - After: - https://ci.appveyor.com/project/curlorg/curl/builds/49500338 + Avoid some unnecessary computation if the timeout is negative. - Also fixup unrelated whitespace. + Spotted by CodeSonar + Closes #13919 - Reviewed-by: Daniel Stenberg - Closes #13217 +- RELEASE-NOTES: synced -Daniel Stenberg (4 Apr 2024) +- curl: support -w '%{num_retries} -- RELEASE-NOTES: synced + Suggested-by: Jay Guerette + Ref: https://github.com/curl/curl/discussions/13901 + Closes #13910 -Viktor Szakats (4 Apr 2024) +Guilherme Puida (11 Jun 2024) -- cmake: speed up libcurl doc building again +- pytest: include testenv/vsftpd.py in dist tarball - This time limit the number of files per command to avoid exceeding - limitations of certain OS/shell envs. + Closes #13918 - Such known env is Windows with the `cmd.exe` shell, which features an - 8K command-line length limit to this day. +Viktor Szakats (11 Jun 2024) - Allowlisting `UNIX` to have no limit and using a limit of 200 for other - envs to be safe. If there is a way to detect `cmd.exe` and/or we know - which precise envs are sensitive to this, we can tweak these conditions - further. +- DISTROS: add MSYS2 (native) links - Even with the low limit, this patch reduces external commands by 200x, - making builds much faster. + Also rename existing 'MSYS2' to 'MSYS2 (mingw-w64)'. - Ref: #12762 2620aa930bc73af1e4c70b10e3125b957b96ecfb (initial) - Ref: #13047 f03c85635f35269f1f45b983bf216624f541760a (revert) + Closes #13915 - Reviewed-by: Daniel Stenberg - Closes #13207 +Daniel Stenberg (10 Jun 2024) -- cmake: tidy-up to use `WORKING_DIRECTORY` +- tool_writeout: get certinfo only when needing it - Reviewed-by: Daniel Stenberg - Closes #13206 + Removes a fairly expensive libcurl call when not necessary -- cmake: generate misc manpages and install `mk-ca-bundle.pl` + Closes #13914 - - install `mk-ca-bundle.pl` like autotools does. +- tool_writeout: bsearch the variable name - - generate and install `mk-ca-bundle.1` and `curl-config.1` like - autotools. This fixes tests 1140 and 1173. + As the list of variable names grows, doing a simple loop to find the + name get increasingly worse. This switches to a bsearch. - Reported-by: Dan Fandrich - Fixes #13194 + Also: do a case sensitive check for the variable name. The names have + not been documented to be case insensitive and there is no point in + having them so. - - add option `BUILD_MISC_DOCS` to control building the above two - manpages. Enabled by default. + Closes #13914 - - appveyor: stop disabling tests 1140 and 1173. +Stefan Eissing (10 Jun 2024) - Reviewed-by: Daniel Stenberg - Closes #13197 +- multi: prepare multi_wait() for future shutdown usage -Fabian Keil (4 Apr 2024) + - new struct curl_pollfds and struct curl_waitfds + - add structs and methods to init/add/cleanup an array of pollfd and + struct curl_waitfd. Use in multi_wait() and multi_waitfds() to + populate the sets for polling. + - place USE_WINSOCK WSAEventSelect() setting into a separate loop over + all collected pfds -- wolfssl: plug memory leak in wolfssl_connect_step2() + Closes #13900 - Fixes: +- connection: shutdown TLS (for FTP) better - test 2034...[simple HTTPS GET with DER public key pinning] - ==61829== 22,610 (3,744 direct, 18,866 indirect) bytes in 1 blocks are d - efinitely lost in loss record 51 of 54 - ==61829== at 0x484BB74: malloc (vg_replace_malloc.c:446) - ==61829== by 0x4B53A80: wolfSSL_Malloc (memory.c:344) - ==61829== by 0x4C1C8E1: wolfSSL_X509_new (x509.c:5326) - ==61829== by 0x4C3977D: d2i_X509orX509REQ (x509.c:3628) - ==61829== by 0x4C1D1F4: wolfSSL_X509_d2i (x509.c:3664) - ==61829== by 0x4C1C37B: wolfSSL_X509_dup (x509.c:13425) - ==61829== by 0x4C197DB: wolfSSL_get_peer_certificate (ssl.c:18765) - ==61829== by 0x33297C: wolfssl_connect_step2 (wolfssl.c:875) - ==61829== by 0x331669: wolfssl_connect_common (wolfssl.c:1287) - ==61829== by 0x3303E9: wolfssl_connect_nonblocking (wolfssl.c:1319) - ==61829== by 0x32FE89: ssl_connect_nonblocking (vtls.c:510) - ==61829== by 0x32DBE5: ssl_cf_connect (vtls.c:1679) - ==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307) - ==61829== by 0x27D9CF: cf_setup_connect (connect.c:1199) - ==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307) - ==61829== by 0x283CEA: cf_hc_baller_connect (cf-https-connect.c:135) + This adds connection shutdown infrastructure and first use for FTP. FTP + data connections, when not encountering an error, are now shut down in a + blocking way with a 2sec timeout. - Closes #13272 + - add cfilter `Curl_cft_shutdown` callback + - keep a shutdown start timestamp and timeout at connectdata + - provide shutdown timeout default and member in + `data->set.shutdowntimeout`. + - provide methods for starting, interrogating and clearing + shutdown timers + - provide `Curl_conn_shutdown_blocking()` to shutdown the + `sockindex` filter chain in a blocking way. Use that in FTP. + - add `Curl_conn_cf_poll()` to wait for socket events during + shutdown of a connection filter chain. + This gets the monitoring sockets and events via the filters + "adjust_pollset()" methods. This gives correct behaviour when + shutting down a TLS connection through a HTTP/2 proxy. + - Implement shutdown for all socket filters + - for HTTP/2 and h2 proxying to send GOAWAY + - for TLS backends to the best of their capabilities + - for tcp socket filter to make a final, nonblocking + receive to avoid unwanted RST states + - add shutdown forwarding to happy eyeballers and + https connect ballers when applicable. -Viktor Szakats (3 Apr 2024) + Closes #13904 -- appveyor: OpenSSL 3 no longer found by CMake, revert to 1.1.1 +Daniel Stenberg (7 Jun 2024) - OpenSSL moved directories, and bumped versions in AppVeyor CI. +- CURLOPT_CONNECTTIMEOUT*: clarify, document the milliseond version - Downgrading is not an ideal solution, but however trivial the solution - may be, I failed to come with anything that made CMake recognize either - OpenSSL 3.1 or 3.2. + Provide an explanation in the CURLOPT_CONNECTTIMEOUT_MS page instead of + just referring to the non-MS version. - Possibly caused by: - https://github.com/appveyor/build-images/commit/702e8cdca01f28f6a40687783f493 - c786cebbe2c - https://github.com/appveyor/build-images/pull/149 + Closes #13905 - Closes #13266 +- cmdline-opts: tidy up --ip-tos and --mptcp -hongfei.li (3 Apr 2024) + To make them render nicer in the manpage and minor polish. -- winbuild: use $(RC) correctly + Closes #13906 - Cloes #13267 +- RELEASE-NOTES: synced -Daniel Stenberg (3 Apr 2024) +Dorian Craps (7 Jun 2024) -- dist: remove the curl-config.1 from the tarball +- curl: (on linux) add MPTCP support - The markdown file is already there and the .1 file gets generated in the - build. + Multipath TCP (MPTCP), standardized in RFC8684 [1], is a TCP extension + that enables a TCP connection to use different paths. - Ref: #13250 - Closes #13268 + Multipath TCP has been used for several use cases. On smartphones, MPTCP + enables seamless handovers between cellular and Wi-Fi networks while + preserving established connections. This use-case is what pushed Apple + to use MPTCP since 2013 in multiple applications [2]. On dual-stack + hosts, Multipath TCP enables the TCP connection to automatically use the + best performing path, either IPv4 or IPv6. If one path fails, MPTCP + automatically uses the other path. -- curl_global_trace.md: shorten the description + To benefit from MPTCP, both the client and the server have to support + it. Multipath TCP is a backward-compatible TCP extension that is enabled + by default on recent Linux distributions (Debian, Ubuntu, Redhat, ...). + Multipath TCP is included in the Linux kernel since version 5.6 [3]. To + use it on Linux, an application must explicitly enable it when creating + the socket. No need to change anything else in the application. - Closes #13263 + This attached patch adds an --mptcp option which allows the creation of + an MPTCP socket instead of TCP on Linux. If Multipath TCP is not + supported on the system, an error will be reported. It is important to + note that if the end server doesn't support MPTCP, the connection will + continue after a seamless fallback to TCP. -- test1901: verify chunked POST from callback with CURLOPT_POSTFIELDSIZE set + Link: https://www.rfc-editor.org/rfc/rfc8684.html [1] + Link: https://www.tessares.net/apples-mptcp-story-so-far/ [2] + Link: https://www.mptcp.dev [3] + Co-developed-by: Dorian Craps (@CrapsDorian) + Co-developed-by: Olivier Bonaventure (@obonaventure) + Co-developed-by: Matthieu Baerts (@matttbe) + Signed-off-by: Dorian Craps - Follow-up to 721941aadf4ad + Closes #13278 - Ref: #13257 - Closes #13262 +Orgad Shaneh (7 Jun 2024) -Stefan Eissing (2 Apr 2024) +- curl: support IP Type of Service / Traffic Class: --ip-tos -- http: with chunked POST forced, disable length check on read callback + Add --ip-tos option to the command line tool for setting TOS for IPv4 or + Traffic Class for IPv6. - - when an application forces HTTP/1.1 chunked transfer encoding - by setting the corresponding header and instructs curl to use - the CURLOPT_READFUNCTION, disregard any POST length information. - - this establishes backward compatibility with previous curl versions + Closes #13606 - Applications are encouraged to not force "chunked", but rather - set length information for a POST. By setting -1, curl will - auto-select chunked on HTTP/1.1 and work properly on other HTTP - versions. +Andy Pan (7 Jun 2024) - Reported-by: Jeff King - Fixes #13229 - Closes #13257 +- socketpair: provide `Curl_socketpair` only when `!CURL_DISABLE_SOCKETPAIR` -Jay Satiro (1 Apr 2024) + Ref: https://curl.se/dev/log.cgi?id=20240605035856-3529577 -- INSTALL-CMAKE.md: explain `cmake -G ` + Reported-by: Marcel Raad + Closes #13888 - - Explain that CMake's -G option can be used to specify which build - system to generate files for. +Daniel Stenberg (7 Jun 2024) - Example: cmake ../curl -G "MinGW Makefiles" +- noproxy: test bad ipv6 net size first - Ref: https://github.com/curl/curl/pull/12224#issuecomment-2026813645 + No need to parse anything if the size is out of range. - Closes https://github.com/curl/curl/pull/13244 + Added some tests to this effect to test 1614. -Daniel Stenberg (1 Apr 2024) + Closes #13902 -- libcurl-opts: mention pipelining less +- managen: warn on excessively long help texts - libcurl has not supported HTTP pipelining since many years. Remove a few - (more) mentions of the feature. + Help texts at 49 characters or longer get a warning displayed because + they make --help output uglier and we should make an effort to keep the + help texts short and succinct. - Closes #13254 + The warning is only for display, it does not break the build. That is + left for the future if necessary. -Daniel McCarney (31 Mar 2024) + I picked 49 because the longest current text is 48. -- m4: reposition USE_RUSTLS="yes" for pkg-config + Closes #13895 - It's necessary to set this var to "yes" _after_ AC_DEFINE and AC_SUBST - in order for a later `test` to pass so that `check_for_ca_bundle=1` ends - up being set. This is in turn required for the default CA certificate - bundle to be set when building w/ rustls & pkg-config. +Viktor Szakats (5 Jun 2024) - Reported-by: Matt Jolly - Fixes #13248 - Closes #13251 +- lib: tidy up types and casts -Daniel Stenberg (31 Mar 2024) + Cherry-picked from #13489 + Closes #13862 -- maketgz: put docs/RELEASE-TOOL.md into the tarball +Daniel Stenberg (5 Jun 2024) - Generated with scripts/release-tools.sh +- cmdline-opts/ech.md: shorten the help text - The script lists the exact Debian package names and version numbers for - the tools that are used to generate the tarball. + To make --help look sensible again - Closes #13239 + Closes #13894 -- cd2nroff/manage: use UTC when SOURCE_DATE_EPOCH is set +- cmdline-opts/_PROTOCOLS.md: mention WS(S) - Make them independent of the TZ setting. Also set a date string like - YYYY-MM-DD to avoid a local month name in the date. + Closes #13891 - Reported-by: Carlos Henrique Lima Melara - Fixes #13242 - Closes #13243 +Viktor Szakats (5 Jun 2024) -- RELEASE-NOTES: synced +- GHA: disable TFTP and WebSockets tests in old-mingw-w64 -- docs/MAIL-ETIQUETTE: convert to markdown + Follow-up to 03bd16e5339b069aa9409b75fcab2b21fd3a4b16 #13860 + Follow-up to def7d05382743ea7aa1d356d1e41dcb22ecdd4d7 - To render nicer. To get spellchecked. +Daniel Stenberg (5 Jun 2024) - Closes #13247 +- cmdline-opts/fail.md: expand and clarify -- reuse: add copyright + license info to individual docs/*.md files + Closes #13890 - Instead of use 'docs/*.md' in dep5. For clarity and avoiding a wide- - matching wildcard. +- doh-insecure.md: expand - + Remove mention of old files from .reuse/dep5 - + add info to .github/dependabot.yml - + make scripts/copyright.pl warn on non-matching patterns + Closes #13889 - Closes #13245 +- cmdline: expand proxy option explanations -- test470: warn about unicode quote character read from config file + - do less references to other options + - provide more specific text about proxies + - added more see-also references - Idea-by: Emanuele Torre + Closes #13887 -- test469: verify warning when argument has unicode quote +- cmdline-opts: expand the parallel explanations -- tool_getparam: output warning for leading unicode quote character + Closes #13886 - ... in the option argument. +- RELEASE-NOTES: synced - Typically this is a mistake done when copying example command lines from - online documentation using the wrong quote character. +Stefan Eissing (5 Jun 2024) - Presumably there are also other potential quote characters that might be - used, and this check is done without even knowing that unicode is used! +- vtls: new io_need flags for poll handling - Reported-by: Sanjay Pujare - Fixes #13214 - Closes #13215 + - decouple need to recv/send from negotiation state, we need + this later in shutdown handling as well + - move ssl enums from urldata.h to vtls_int.h + - implement use of `connssl->io_need` in vtls.c. and all backends -- tool: follow-up getenv fix + Closes #13879 - Remove a double free. Change the IPFS env use to a plain getenv() simply - because coverity gets confused. +Daniel Stenberg (5 Jun 2024) - Follow-up to 9126b141c9398fe - Closes #13241 +- cfilters: make Curl_conn_connect always assign 'done' -- idn: make Curl_idnconvert_hostname() use Curl_idn_decode() + It could return error without assigning it, and we have a caller in + multi.c that assumes it gets set. - In the name of less code duplication + Spotted by CodeSonar + Closes #13884 - Closes #13236 +- CURLOPT_INTERFACE.md: quote the less-than and larger-than -- curl-confopts.m4: define CARES_NO_DEPRECATED when c-ares is used + Fixes the warnings shown on stderr. - Starting in 1.28.0 c-ares added deprecation warnings for some API calls - libcurl uses. + Follow-up from 3060557af702dd5 - Closes #13240 + Closes #13883 -- vquic: use CURL_FORMAT_CURL_OFF_T for 64 bit printf output +- cmdline-opts/interface.md: expand the documentation - Reported-by: Keitagit-kun on github - Fixes #13224 - Closes #13231 + Explain the syntax it supports. -- openldap: create ldap URLs correctly for IPv6 addresses + Closes #13882 - Reported-by: Sergio Durigan Junior - Fixes #13228 - Closes #13235 +- url: allow DoH transfers to override max connection limit -- curl: use curl_getenv instead of the curlx_ version + When reaching the set maximum limit of allowed connections, allow a new + connection anyway if the transfer is created for the (internal) purpose + of doing a DoH name resolve. Otherwise, unrelated "normal" transfers can + starve out new DoH requests making it impossible to name resolve for new + transfers. - The curlx one was once introduced when we still considered dropping the - libcurl function at some point. To reduce confusion and to make it - easier to understand when curl_free() should be used, use the actual - libcurl function call directly instead. + Bug: https://curl.se/mail/lib-2024-06/0001.html + Reported-by: kartatz + Closes #13880 - Closes #13230 +Viktor Szakats (5 Jun 2024) -Evgeny Grin (Karlson2k) (30 Mar 2024) +- windows: fix UWP builds, add GHA job -- curl_sha512_256: do not use workaround for NetBSD when not needed + Add new job to test building for UWP (aka `CURL_WINDOWS_APP`). - Assisted-by: riastradh on github - Assisted-by: Michael Kaufmann - Closes #13225 + Fix fallouts when building for UWP: + - rand: do not use `BCryptGenRandom()`. + - cmake: disable using win32 LDAP. + - cmake: disable telnet. + - version_win32: fix code before declaration. + - schannel: disable `HAS_MANUAL_VERIFY_API`. + - schannel: disable `SSLSUPP_PINNEDPUBKEY` + and make `schannel_checksum()` a stub. + Ref: e178fbd40a896f2098278ae61e1166c88e7b31d0 #1429 + - schannel: make `cert_get_name_string()` a failing stub. + - system_win32: make `Curl_win32_impersonating()` a failing stub. + - system_win32: try to fix `Curl_win32_init()` (untested). + - threads: fix to use `CreateThread()`. + - src: disable searching `PATH` for the CA bundle. + - src: disable bold text support and capability detection. + - src: disable `getfiletime()`/`setfiletime()`. + - tests: make `win32_load_system_library()` a failing stub. + - tests/server/util: make it compile. + - tests/server/sockfilt: make it compile. + - tests/lib3026: fix to use `CreateThread()`. -Matt Jolly (30 Mar 2024) + See individual commits for build error details. -- m4: fix rustls pkg-config codepath + Some of these fixes may have better solutions, and some may not work + as expected. The goal of this patch is to make curl build for UWP. - The previous pkg-config code would successfully detect rustls but did - not set all appropriate variables and call the right macros to properly - configure cURL. + Closes #13870 - Reported-by: kpcyrd on github - Fixes #13200 - Closes #13202 +Orgad Shaneh (4 Jun 2024) -Daniel McCarney (30 Mar 2024) +- socket: support binding to interface *AND* IP -- deps: update librustls 0.12.0 -> 0.13.0 + Introduce new notation for CURLOPT_INTERFACE / --interface: + ifhost!! - This commit updates the optional rustls-ffi librustls dependency from - 0.12.0 to 0.13.0. This version is based on the latest available rustls - release (0.23.4). + Binding to an interface doesn't set the address, and an interface can + have multiple addresses. - The breaking API changes from 0.12.0 to 0.13.0 are in API surface unused - by curl, so this is an in-place update without any code changes. + When binding to an address (without interface), the kernel is free to + choose the route, and it can route through any device that can access + the target address, not necessarily the one with the chosen address. - The `RUSTLS.md` documentation is updated to reflect the new version in - use, and to clarify that `cbindgen` isn't required to build `librustls` - - it's only used by developers to update the vendored `rustls.h` header - file maintained upstream. + Moreover, it is possible for different interfaces to have the same IP + address, on which case we need to provide a way to be more specific. - Closes #13238 + Factor out the parsing part of interface option, and add unit tests: + 1663. -Daniel Stenberg (28 Mar 2024) + Closes #13719 -- RELEASE-NOTES: synced +Andy Pan (4 Jun 2024) -- tool_xattr: "guess" URL scheme if none is provided +- socketpair: add `eventfd` and use `SOCK_NONBLOCK` for `socketpair()` - ... when figuring out the source URL to store. + Currently, we use `pipe` for `wakeup_create`, which requires ***two*** + file descriptors. Furthermore, given its complexity inside, `pipe` is a + bit heavyweight for just a simple event wait/notify mechanism. - Reported-by: Dagfinn Ilmari Mannsåker - Fixes #13205 - Closes #13221 + `eventfd` would be a more suitable solution for this kind of scenario, + kernel also advocates for developers to use `eventfd` instead of `pipe` + in some simple use cases: -- tool_xattr: in debug builds, act normally if CURL_FAKE_XATTR is not set + Applications can use an eventfd file descriptor instead of a pipe + (see pipe(2) in all cases where a pipe is used simply to signal + events. The kernel overhead of an eventfd file descriptor is much + lower than that of a pipe, and only one file descriptor is required + (versus the two required for a pipe). - Closes #13220 + This change adds the new backend of `eventfd` for `wakeup_create` and + uses it where available, eliminating the overhead of `pipe`. Also, it + optimizes the `wakeup_create` to eliminate the system calls that make + file descriptors non-blocking by moving the logic of setting + non-blocking flags on file descriptors to `socketpair.c` and using + `SOCK_NONBLOCK` for `socketpair(2)`, `EFD_NONBLOCK` for `eventfd(2)`. -Stefan Eissing (28 Mar 2024) + Ref: + https://man7.org/linux/man-pages/man7/pipe.7.html + https://man7.org/linux/man-pages/man2/eventfd.2.html + https://man7.org/linux/man-pages/man2/socketpair.2.html + https://www.gnu.org/software/gnulib/manual/html_node/eventfd.html -- content_encoding: brotli and others, pass through 0-length writes + Closes #13874 - - curl's transfer handling may write 0-length chunks at the end of the - download with an EOS flag. (HTTP/2 does this commonly) +renovate[bot] (4 Jun 2024) - - content encoders need to pass-through such a write and not count this - as error in case they are finished decoding +- ci: update github/codeql-action digest to 2e230e8 - Fixes #13209 - Fixes #13212 - Closes #13219 + Closes #13881 -Tobias Stoeckmann (28 Mar 2024) +Jay Satiro (4 Jun 2024) -- libssh2: set length to 0 if strdup failed +- examples/threaded-ssl: remove locking callback code - Internally, libssh2 dereferences the NULL pointer if length is non-zero. - The callback function cannot return the error condition, so at least - prevent subsequent crash. + - Remove the locking callback code that demonstrates how to meet + requirements of threading libraries (mainly OpenSSL). - Closes #13213 + Locking callback code has not been needed for many years. According to + the documentation for OpenSSL and GnuTLS they are thread-safe by design, + assuming support for the underlying OS threading API is built-in. -Daniel Stenberg (28 Mar 2024) + Ref: https://github.com/curl/curl/pull/13850#issuecomment-2143538458 -- RELEASE-PROCEDURE: mention an initial working build + Closes https://github.com/curl/curl/pull/13851 - This is the step that was not done and caused the 8.7.0 mishap (it - lacked the correctly generated hugehelp file). +Viktor Szakats (4 Jun 2024) - Remove the mention of the copyright script as this is verified by a CI - job these days: the REUSE one. +- tests: delete redundant `!MSDOS` guard - Closes #13216 + This fix was supposed to be committed earlier, but ended up missing from + the final commit. -Paul Howarth (28 Mar 2024) + Follow-up to e9a7d4a1c8377dbcf9a2d94365f60e3e5dff48f8 #12376 + Closes #13878 -- curl_sha512_255: fix detection of OpenSSL 1.1.1 or later +- lib: fix thread entry point to return `DWORD` on WinCE - Use the same OPENSSL_VERSION_NUMBER comparison as in lib/vtls/openssl.c. + We already do this in `tests/server/util.c`: + https://github.com/curl/curl/blob/97e5e37cc8269660bc5d4a1936f10f2390b97c5a/te + sts/server/util.c#L604-L606 + and in `sockfilt.c`, `lib3026.c`. - Closes #13208 + Before this patch it returned `unsigned int`. -Robert Moreton (28 Mar 2024) + Closes #13877 -- cf-socket: remove references to l_ip, l_port +Andy Pan (4 Jun 2024) - Fixes #13210 - Closes #13211 +- socket: use SOCK_NONBLOCK to eliminate extra system call -Daniel Stenberg (28 Mar 2024) + Every time function `cf_socket_open()` is called to create a socket, + `curlx_nonblock()` is called to make that socket non-blocking. And + `curlx_nonblock()` will cost us 1 or 2 system calls (2 for `fcntl()`, 1 + for `ioctl()`, etc.), meanwhile, tucking `SOCK_NONBLOCK` and + `SOCK_CLOEXEC` into the `type` argument for `socket()` is widely + supported across UNIX-like OS: Linux, *BSD, Solaris, etc. With that + ability, we can save 1 or 2 system calls on each socket. -- openssl: do not set SSL_MODE_RELEASE_BUFFERS + Another change in this PR is to eliminate the redundant + `curlx_nonblock()` call on the socket in `cf_udp_setup_quic()` as that + socket created by `cf_socket_open()` is already non-blocking. - While it might save some memory, it causes OpenSSL to instead do a huge - amount of allocations. + Ref: + https://man7.org/linux/man-pages/man2/socket.2.html + https://man.freebsd.org/cgi/man.cgi?socket(2) + https://man.dragonflybsd.org/?command=socket§ion=2 + https://man.netbsd.org/socket.2 + https://man.openbsd.org/socket + https://docs.oracle.com/cd/E88353_01/html/E37843/socket-3c.html + https://illumos.org/man/3SOCKET/socket + ... - Ref: #13136 - Closes #13203 + Closes #13855 -- curl: make --help adapt to the terminal width +Viktor Szakats (4 Jun 2024) - Instead of assuming and working with 80 colums, try figuring out what - width is actually used. +- GHA: show cmake error log in Windows and non-native workflows - Ref: #13141 + CMake configure doesn't fail often, but when it does, it helps to see + its `CMakeFiles/CMakeConfigureLog.yaml` output. This file is present + since CMake v3.26: + https://cmake.org/cmake/help/v3.26/manual/cmake-configure-log.7.html - Closes #13171 + (Older CMake versions save similar contend to + `CMakeFiles\CMakeOutput.log` and + `CMakeFiles\CMakeError.log`. This patch doesn't deal with that because + the workflows touched are all running a newer CMake.) -- RELEASE-NOTES: synced + After this patch, we dump the content if cmake fails. Syncing this with + autotools, where we already did that. - and bump to 8.7.2 for now + Closes #13872 -- configure: make --disable-docs imply --disable-manual +- GHA: switch a Windows job to UCRT (gcc) - Because when the docs is not built, the necesary curl.txt file is not - present so then the manual cannot get built. + Cherry-picked from #13870 - Reported-by: Harry Sintonen - Closes #13191 +- curl-config: revert to backticks to support old target envs -Chris Webb (27 Mar 2024) + Make an exception for `curl-config` because this script that may be + running on any target system, including old ones, e.g. SunOS 5.10. -- cmdline-docs: fix make install with configure --disable-docs + Reported-by: Alejandro R. Sedeño + Ref: https://github.com/curl/curl/pull/13307#issuecomment-2146427358 + Follow-up to fa69b41c7790fab86fd363242c81d8ef2e89e183 #13307 + Closes #13871 - make -C docs/cmdline-opts install depends on all-am, which in turn - depends on $(MANS), unconditionally defined to be $(man_MANS). +Stefan Eissing (4 Jun 2024) - As with CLEANFILES, only add curl.1 to man_MANS when BUILD_DOCS is true - so we don't try to build curl.1 unnecessarily. +- mbedtls: v3.6.0 workarounds - Closes #13198 + - add special sauce to disable unwanted peer verification by mbedtls + when negotiating TLS v1.3 + - add special sauce for MBEDTLS_ERR_SSL_RECEIVED_NEW_SESSION_TICKET + return code on *writing* TLS data. We assume the data had not been + written and EAGAIN. + - return correct Curl error code when peer verification failed. + - disable test_08_05 with 50 HTTP/1.1 connections, as mbedtls reports a + memory allocation failed during handshake. + - bump CI mbedtls version to 3.6.0 -Version 8.7.1 (27 Mar 2024) + Fixes #13653 + Closes #13838 -Daniel Stenberg (27 Mar 2024) +- gnutls: support CA caching -- RELEASE-PROCEDURE: remove old release dates, add new pending ones + - similar to openssl, use a shared 'credentials' instance + among TLS connections with a plain configuration. + - different to openssl, a connection with a client certificate + is not eligible to sharing. + - document CURLOPT_CA_CACHE_TIMEOUT in man page -Version 8.7.0 (27 Mar 2024) + Closes #13795 -Daniel Stenberg (27 Mar 2024) +Dan Fandrich (3 Jun 2024) -- RELEASE-NOTES: synced +- tests: don't log buffer length in throwing away message - curl 8.7.0 release + It's not available at that point, and it will be written in the + non-error case right afterward. -- THANKS: new contributors from the 8.7.0 release +- tests: log "Throwing away" messages before throwing away -- CURLOPT_POSTFIELDS.md: used for MQTT as well + In case the read that follows hangs we'll get a clue as to what it was + doing. - Closes #13189 +- CI: reduce memory request for FreeBSD builds -- http: remove stale comment about rewindbeforesend + Also, add a comment with link to the Cirrus credit page since it's not + easy to find otherwise. - ... because that struct field exists no more. +Andy Pan (3 Jun 2024) - Follow-up to 14bcea074a782272. +- tcpkeepalive: support setting TCP keep-alive parameters on Solaris <11.4 - Closes #13187 + Solaris didn't support TCP_KEEPIDLE and TCP_KEEPINTVL until 11.4, + before that it use TCP_KEEPALIVE_THRESHOLD and TCP_KEEPALIVE_ABORT_THRESHOLD + as the substitute. Therefore, for Solaris <11.4 we need to use this substitut + e + for setting TCP keep-alive parameters. -- DISTROS: add document with distro pointers + Ref: + https://docs.oracle.com/cd/E86824_01/html/E54777/tcp-7p.html + https://docs.oracle.com/cd/E88353_01/html/E37851/tcp-4p.html - Lots of organizations distribute curl packages to end users. This is a - collection of pointers to where to learn more about curl on and with - each distro. + Closes #13864 - Assisted-by: Alan Coopersmith - Assisted-by: Andrew Kaster - Assisted-by: Andy Fiddaman - Assisted-by: Arjan van de Ven - Assisted-by: Brian Clemens - Assisted-by: chrysos349 on github - Assisted-by: Dan Fandrich - Assisted-by: Dan McDonald - Assisted-by: Gaelan Steele - Assisted-by: graywolf on github - Assisted-by: Jan Macku - Assisted-by: John Marshall - Assisted-by: Jonathan Perkin - Assisted-by: Kevin Daudt - Assisted-by: Marcus Müller - Assisted-by: Michał Górny - Assisted-by: Outvi V - Assisted-by: Ross Burton - Assisted-by: Sean Molenaar - Assisted-by: Till Wegmüller - Assisted-by: Viktor Szakats - Assisted-by: Winni Neessen +Daniel Stenberg (3 Jun 2024) - Closes #13178 +- KNOWN_BUGS: quiche: QUIC connection is draining -Fabian Keil (25 Mar 2024) + Closes #12037 + Closes #13867 -- wolfSSL: do not call the stub function wolfSSL_BIO_set_init() +- KNOWN_BUGS: aws-sigv4 has problems with particular URLs - Calling the function isn't necessary and causes the build - to fail when wolfSSL has been compiled with NO_WOLFSSL_STUB: + Closes #13058 + Closes #13866 - Making all in opts - CCLD curl - ld: error: undefined symbol: wolfSSL_BIO_set_init - >>> referenced by wolfssl.c:235 (vtls/wolfssl.c:235) - >>> libcurl_la-wolfssl.o:(wolfssl_bio_cf_create) in archiv - e ../lib/.libs/libcurl.a - cc: error: linker command failed with exit code 1 (use -v to see invocat - ion) - *** Error code 1 +- KNOWN_BUGS: aws-sigv4 does not handle multipart/form-data correctly - Closes #13164 + Closes #13351 + Closes #13866 -Daniel Stenberg (25 Mar 2024) +- RELEASE-NOTES: synced -- cmdline-opts: shorter help texts +Viktor Szakats (3 Jun 2024) - In an effort to increase the readability of the "--help all" output on - narrow (80 column) terminals. +- GHA: fix old mingw-w64 32-bit job - Co-authored-by: Jay Satiro + This toolchain resides in the `mingw32` directory. Make sure to + configure `PATH` accordingly. - Closes #13169 + Before this patch, it pointed to a non-existing `mingw64` directory, + making the job use the wrong compiler (gcc 12, 64-bit). -Matt Jolly (25 Mar 2024) + Follow-up to e838b341a08b44d4a8486fb0d3f15d12fc794c62 #12927 + Closes #13863 -- curl-rustls.m4: add pkg-config support to rustls detection +Daniel Stenberg (2 Jun 2024) - Based on the existing openssl pkg-config detection, this commit tries to - use pkg-config to find `rustls` then falls back to the current approach - if that fails. +- tool_cb_hdr: return error for failed header writes - We use the following logic: + By checking that fflush() works. - - if no path is provided, just use pkg-config, if it's not there we have - a problem! - - if a path is provided, try pkg-config - + if pkg-config fails, try and find rustls directly + Reported-by: Sebastian Andersson + Fixes #13836 + Closes #13859 - Closes #13179 +Viktor Szakats (2 Jun 2024) -Mohammadreza Hendiani (25 Mar 2024) +- GHA: bump all build jobs to nproc+1 -- TODO: update 13.11 with more information + - bump rest of the workflows (windows, macos, distrocheck). - Closes #13173 + - non-native virtualized envs have 2 CPUs, bump down accordingly. + (for `vmactions/omnios-vm` it's just a guess.) -Daniel Stenberg (23 Mar 2024) + - bump all to nproc + 1. -- docs/libcurl: generate PROTOCOLS from meta-data + Follow-up to e838b341a08b44d4a8486fb0d3f15d12fc794c62 #12927 + Closes #13807 - Remove the PROTOCOLS section from the source files completely and - instead generate them based on the header data in the curldown files. +- GHA: disable MQTT and WebSocket tests in Windows jobs - It also generates TLS backend information for options marked for TLS as - protocol. + Trying to figure out which category is causing the remaining hangs. - Closes #13175 + Follow-up to def7d05382743ea7aa1d356d1e41dcb22ecdd4d7 + Closes #13860 -- CURLMOPT_MAX*: mention what happens if changed mid-transfer +- lib/v*: tidy up types and casts - For CURLMOPT_MAXCONNECTS and CURLMOPT_MAX_HOST_CONNECTIONS + Also add a couple of negative checks. - Ref: #13158 - Closes #13176 + Cherry-picked from #13489 + Closes #13622 -- docs/libcurl: add TLS backend info for all TLS options +- GHA: fix caching old mingw-w64 toolchains in the Windows workflow - All man pages that are listed to be for TLS now must also specify - exactly what TLS backends the option works for, or use All if they all - work. + - stop altering the `PATH` via `GITHUB_ENV`. This confused the + `actions/cache` post-job, which needs to run in the exact same + environment as its pre-job, to have a consistent cache entry "version" + hash. Altering the `PATH` via `GITHUB_ENV` spills into the the + post-job and breaks this hash. GHA doesn't reset the env automatically + and I have not found a way to do it manually. - cd2nroff makes sure this is done and that the listed backends exist. + - add double-quotes where missing. - Closes #13168 + - move cache directory under `USERPROFILE` to not rely on absolute + paths. -- docs/libcurl: cleanups + - make cache directory flatter and versionless. - - CURLINFO_TLS_SESSION.md: remove mention of NSS - - CURLINFO_TLS_SSL_PTR.md: remove NSS leftover - - CURLOPT_CAINFO.md: drop mention of backends not supporting this - - CURLOPT_CAPATH.md: wolfSSL also supports this + Follow-up to 0914d8aadddac0d1459673d5b7f77e8f3378b22b #13759 + Closes #13856 - Closes #13166 +renovate[bot] (2 Jun 2024) -- docs: make each libcurl man specify protocol(s) +- ci: pin actions/github-script action to 60a0d83 - The mandatory header now has a mandatory list of protocols for which the - manpage is relevant. + Closes #13846 - Most man pages already has a "PROTOCOLS" section, but this introduces a - stricter way to specify the relevant protocols. +Bo Anderson (2 Jun 2024) - cd2nroff verifies that at least one protocol is mentioned (which can be - `*`). +- x509asn1: add some common ECDSA OIDs - This information is not used just yet, but A) the PROTOCOLS section can - now instead get generated and get a unified wording across all manpages - and B) this allows us to more reliably filter/search for protocol - specific manpages/options. + Closes #13857 - Closes #13166 +renovate[bot] (2 Jun 2024) -Stefan Eissing (21 Mar 2024) +- ci: update rojopolis/spellcheck-github-actions digest to e36f662 -- http2, http3: only return CURLE_PARTIAL_FILE when bytes were received + Closes #13852 - - should resolve spurious pytest failures when stream were reset - right after response header were received +Bo Anderson (2 Jun 2024) - Clsoes #13151 +- x509asn1: fallback to dotted OID representation -- http: separate response parsing from response action + Reported-by: Luke Hamburg + Fixes #13845 + Closes #13858 - - move code that triggers on end-of-response into separate function from - parsing - - simplify some headp/headerlen usage - - add `httpversion` to SingleRequest to indicate the version of the - current response +Lee Li (2 Jun 2024) - Closes #13134 +- request.md: language fix -Daniel Stenberg (21 Mar 2024) + improved for better readability and correctness -- http2: remove the third (unused) argument from http2_data_done() + Closes #13854 - Closes #13154 +Christian Schmitz (2 Jun 2024) -- RELEASE-NOTES: synced +- vtls: deprioritize Secure Transport -Evgeny Grin (Karlson2k) (21 Mar 2024) + Moved Secure Transport behind OpenSSL, so we can build CURL with both + and prefer using OpenSSL over Secure Transport by default. -- RELEASE-NOTES: corrected + Closes #13547 - Corrected link for item 118 +Daniel Stenberg (1 Jun 2024) - Closes #13157 +- urlapi: add CURLU_NO_GUESS_SCHEME -Daniel Stenberg (19 Mar 2024) + Used for extracting: -- CURLOPT_INTERFACE.md: remove spurious amp, add see-also + - when used asking for a scheme, it will return CURLUE_NO_SCHEME if the + stored information was a guess - Closes #13149 + - when used asking for a URL, the URL is returned without a scheme, like + when previously given to the URL parser when it was asked to guess -Stefan Eissing (19 Mar 2024) + - as soon as the scheme is set explicitly, it is no longer internally + marked as guessed -- http: improve response header handling, save cpu cycles + The idea being: - Saving some cpu cycles in http response header processing: - - pass the length of the header line along - - use string constant sizeof() instead of strlen() - - check line length if prefix is possible - - switch on first header char to limit checks + 1. allow a user to figure out if a URL's scheme was set as a result of + guessing - Closes #13143 + 2. extract the URL without a guessed scheme -Daniel Stenberg (19 Mar 2024) + 3. this makes it work similar to how we already deal with port numbers -- tool_getparam: accept a blank -w "" + Extend test 1560 to verify. - Added test 468 to verify. + Closes #13616 - Regression from 07bcae89d5d00 (shipped in 8.6.0) - Reported-by: Thomas Pyle - Fixes #13144 - Closes #13145 +- wolfssl: support CA caching -Evgeny Grin (Karlson2k) (18 Mar 2024) + As a bonus, add SSLSUPP_CA_CACHE to let TLS backends signal its support + for this so that *setopt() return error if there is no support. -- curl_sha512_256: work around a NetBSD bug + Closes #13786 - Based on Michael Kaufmann analysis and suggestion +Andy Pan (1 Jun 2024) - Closes #13133 +- socket: change TCP keepalive from ms to seconds on DragonFly BSD -Stefan Eissing (18 Mar 2024) + DragonFly BSD changed the time unit for TCP keep-alive from milliseconds + to seconds since v5.8, thus setting the keepalive options with + milliseconds with curl/libcurl will result in unexpected behaviors on + DragonFlyBSD 5.8+ -- http: expect 100 rework + Distinguish the DragonFly BSD versions and use the proper time units + accordingly. - Move all handling of HTTP's `Expect: 100-continue` feature into a client - reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer - sending on general events like a timer. + Ref: + https://lists.dragonflybsd.org/pipermail/commits/2019-July/719125.html + https://github.com/DragonFlyBSD/DragonFlyBSD/blob/965b380e960908836b97aa034 + fa2753091e0172e/sys/sys/param.h#L207 - HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect: - 100-continue`. That reader works as follows: + Fixes #13847 + Closes #13848 - - on first invocation, records time, starts the `EXPIRE_100_TIMEOUT` - timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0, - eos=FALSE like a paused upload. +Daniel Stenberg (1 Jun 2024) - - on subsequent invocation it checks if the timer has expired. If so, it - enables `KEEP_SEND` and switches to passing through reads to the - underlying readers. +- curlver.h: aiming for 8.9.0 - Transfer handling's `readwrite()` will be invoked when a timer expires - (like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing - `KEEP_SEND_TIMER`, it will try to upload more data, which triggers - reading from the client readers again. Which then may lead to a new - pausing or cause the upload to start. +- noproxy: patterns need to be comma separated - Flags and timestamps connected to this have been moved from - `SingleRequest` into the reader's context. + or they will not parse correctly. - Closes #13110 + Mentioned in DEPRECATED since Janurary 2023 (in 7ad8a7ba9ebdedc). -- mbedtls: fix pytest for newer versions + Closes #13789 - Fix the expectations in pytest for newer versions of mbedtls +Jan Venekamp (1 Jun 2024) - Closes #13132 +- sectransp: remove large cipher table -Daniel Stenberg (15 Mar 2024) + Previously a large table of ciphers was used to determine the default + ciphers and to lookup manually selected ciphers names. -- ipv6.md: mention IPv4 mapped addresses + With the lookup of the manually selected cipher names moved to + Curl_cipher_suite_walk_str() the large table is no longer needed for + that purpose. - Reported-by: Josh Soref - Assisted-by: Jay Satiro - Fixes #13112 - Closes #13131 + The list of manually selected cipher can now be intersected with the + ciphers supported by Secure Transport (SSLGetSupportedCiphers()), + instead of using the fixed table for that. -Stefan Eissing (15 Mar 2024) + The other use of the table was to filter the list of all supported + ciphers offered by Secure Transport to create a list of ciphers to + use by default, excluding ciphers in the table marked as weak. -- http: revisit http_perhapsrewind() + Instead of using a complement based approach (exclude weak), switch + to using an intersection with a smaller list of ciphers deemed + appropriate. - - use facilities provided by client readers better - - work also for non-uploading requests like GET/HEAD - - update documentation + Closes #13823 - Closes #13117 +Tatsuhiro Tsujikawa (1 Jun 2024) -- test 1541: verify getinfo values on first header callback +- GHA: unify http3 workflows into one - Reported-by: chensong1211 on github - Ref: #13125 - Closes #13128 + This commit unifies the following http3 workflows into http3-linux.yml: -- TLS: start shutdown only when peer did not already close + - ngtcp2-linux.yml + - osslq-linux.yml + - quiche-linux.yml - - When curl sees a TCP close from the peer, do not start a TLS shutdown. - TLS shutdown is a handshake and if the peer already closed the - connection, it is not interested in participating. + The idea is better use of the build cache. Previously, they + independently create caches with the same key. Some of the caches + include source code and intermediate object files, which makes cache + quite large. In this commit, only built artifacts are cached, which + drastically reduces the cache size. OpenSSL v3, mod_h2 and quiche caches + still include all stuff, but they are left for the later improvement. + Because the contents of the cache have been changed, the cache keys are + also changed to include the word "http3". - Reported-by: dfdity on github - Assisted-by: Jiří Bok - Assisted-by: Pēteris Caune - Fixes #10290 - Closes #13087 + Closes #13841 -Daniel Stenberg (14 Mar 2024) +Stephen Farrell (1 Jun 2024) -- RELEASE-NOTES: synced +- openSSL: fix hostname handling when using ECH -- curl: make --libcurl output better CURLOPT_*SSLVERSION + Reported-by: vvb2060 + Fixes #13818 + Closes #13822 - The option is really two enums ORed together, so it needs special - attention to make the code output nice. +renovate[bot] (1 Jun 2024) - Added test 1481 to verify. Both the server and the proxy versions. +- ci: update github/codeql-action digest to f079b84 - Reported-by: Boris Verkhovskiy - Fixes #13127 - Closes #13129 + Closes #13837 -- GHA/linux: add sysctl trick to work-around GitHub runner issue +Daniel Stenberg (1 Jun 2024) - The GitHub image runner update from 20240304.1.0 to 20240310.1 - introduces a problem for clang-14. The issue is caused by - incompatibility between llvm 14 provided in ubuntu-22.04 image and the - much newer kernel configured with high-entropy ASLR. +- RELEASE-NOTES: synced - As a work-around, we issue a sysctl command to lower the entropy and get - clang-14 to work again. +- curl_multi_poll.md: expand the example with an custom file descriptor - URL: https://github.com/actions/runner-images/issues/9491 + Closes #13842 - Closes #13124 +Christian Heusel (1 Jun 2024) -- SPONSORS: describe the basics +- DISTROS: add a link to the list archive - Closes #13119 + Related to https://github.com/curl/curl/discussions/13833 -- GOVERNANCE: document the core team + Signed-off-by: Christian Heusel + Closes #13843 - Closes #13118 +Matt Jolly (31 May 2024) -Jay Satiro (13 Mar 2024) +- autoconf: remove 'deeper' checks for `AC_CHECK_FUNCS` -- vquic-tls: fix the error code returned for bad CA file + The net effect of the deeper checks is to raise implicit function decls + on modern compilers. - - Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem - reading the cert file or path. + These checks appear to have been added ~20 years ago, relating to an + unverifiable claim about HP-UX. Autoconf support for the platform has + grown in leaps and bounds since. - This is a follow-up to the parent commit aedbbdf1. + It didn't cause a real problem here, but when investigating a FP this + came up. No evidence has been identified that this was actually broken + in the past, and there is no evidence that this is necessary now. - Reported-by: Karthikdasari0423@users.noreply.github.com + `-Werror=implicit-function-declarations` is enabled for both checks; + without a working prototype they will both fail regardless. In the + second case there will in fact never be a working prototype and + therefore it will always fail unconditionally. - Fixes https://github.com/curl/curl/issues/13115 + `AC_CHECK_FUNCS` does effectively the same thing as the removed checks, + except it actually defines a dummy prototype to see if it links. -Daniel Stenberg (12 Mar 2024) + If `AC_CHECK_FUNCS` is broken on a given platform we have bigger + problems than trying to build cURL. This should also be faster. -- vquic-tls: return appropirate errors on wolfSSL errors + Bug: https://bugs.gentoo.org/932827 + Reviewed-By: Eli Schwartz + Closes #13830 - Reported-by: Dexter Gerig - Closes #13107 +Jay Satiro (30 May 2024) -Viktor Szakats (12 Mar 2024) +- cf-socket: improve SO_SNDBUF update for Winsock -- tidy-up: one comment and EOF newlines + - Rename: Curl_sndbufset => Curl_sndbuf_init - Reviewed-by: Daniel Stenberg - Closes #13108 + - Rename: win_update_buffer_size => win_update_sndbuf_size -Daniel Stenberg (12 Mar 2024) + - Save the last set SO_SNDBUF size to compare against so that we can + avoid setsockopt calls every second. -- cmdline-opts: language cleanups + This is a follow-up to 0b520e12 which moved the SO_SNDBUF update check + into cf-socket. This change improves it further by making the function + names easier to understand and reducing the amount of setsockopt calls. - Use imperative mood consistently for the first sentence describing an - option. + Closes https://github.com/curl/curl/pull/13827 - "Set this" instead "tell curl to set" or "this sets..." +Viktor Szakats (30 May 2024) - Plus some extra cleanups and rephrasing. +- tidy-up: use consistent casing for Windows directories - Closes #13106 + C:\Windows\System32 -- managen: remove space before protocols + Closes #13832 - For options that are listed for specific protocols, the protocols (shown - first within parentheses) are now output without the leading space in the - manpage output. +- GHA: use ubuntu-latest with OmniOS job - Closes #13105 + It's the same as ubuntu-22.04. -Jay Satiro (12 Mar 2024) + Also update OmniOS package search link. -- mbedtls: properly cleanup the thread-shared entropy + Closes #13831 - - Store the state of the thread-shared entropy for global init/cleanup. +Ayesh Karunaratne (30 May 2024) - - Use curl's thread support of mbedtls for all Windows builds instead of - just when the threaded resolver is used via USE_THREADS_WIN32. +- GHA: adjust parallel job counts - Prior to this change on global cleanup curl builds that have curl thread - support for mbedtls freed the entropy (8b1d2298) but failed to mark that - it had been freed, which caused problems on subsequent init + transfer. + Adjusts the `make -j` flag to match the latest GitHub-hosted runner + hardware specs[^1]: - Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105 - Reported-by: awesomekosm@users.noreply.github.com + - `ubuntu-latest` on 4 CPU cores + - `macos-latest` on 3 CPU cores - Closes https://github.com/curl/curl/pull/13071 + The processor count is ideally obtained from `nproc`, but setting env + vars from the current CI yaml files is not possible because they expect + literal strings. -Daniel Stenberg (12 Mar 2024) + [^1]: https://docs.github.com/en/actions/using-github-hosted-runners/about-gi + thub-hosted-runners/about-github-hosted-runners#standard-github-hosted-runner + s-for-public-repositories -- tool_getparam: handle non-existing (out of range) short-options + Closes #12927 - ... correctly, even when they follow an existing one without a space in - between. +pszlazak (30 May 2024) - Verify with test 467 +- get.d: clarify the explanation - Follow-up to 07dd60c05b - Reported-by: Geeknik Labs - Fixes #13101 - Closes #13102 + Closes #13706 -Stefan Eissing (11 Mar 2024) +Daniel Stenberg (30 May 2024) -- lib: move 'done' parameter to SingleRequests +- curl_url_set.md: libcurl only parses :// URLs - A transfer may do several `SingleRequest`s for its success. This happens - regularly for authentication, follows and retries on failed connections. - The "readwrite()" calls and functions connected to those carried a `bool - *done` parameter to indicate that the current `SingleRequest` is over. - This may happen before `upload_done` or `download_done` bits of - `SingleRequest` are set. + Make it clearer in the documentation. - The problem with that is now `write_resp()` protocol handlers are - invoked in places where the `bool *done` cannot be passed up to the - caller. Instead of being a bool in the call chain, it needs to become a - member of `SingleRequest`, reflecting its state. + Closes #13821 - This removes the `bool *done` parameter and adds the `done` bit to - `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a - `SingleRequest` in a follow up, clearing `done` and other - flags/counters. +Stefan Eissing (30 May 2024) - Closes #13096 +- multi: fix multi_wait() timeout handling -- request: clarify message when request has been sent off + - determine the actual poll timeout *after* all sockets + have been collected. Protocols and connection filters may + install new timeouts during collection. + - add debug logging to test1533 where the mistake was noticed - Change the "uploaded and fine" message for requests without a body + Reported-by: Matt Jolly + Fixes #13782 + Closes #13825 - Reported-by: Karthikdasari0423 on github - Fixes #13093 - Closes #13095 +Viktor Szakats (29 May 2024) -Daniel Stenberg (11 Mar 2024) +- lib: prefer `var = time(NULL)` over `time(&var)` -- RELEASE-NOTES: synced + Following up on previous occurrences showing up as gcc warnings, replace + the remaining `time(&var)` calls with `var = time(NULL)`, though these + aren't specifically causing compiler warnings. These are in the TFTP + client code (`lib/tftp.c`), except one which is in a debug branch in + `lib/http_aws_sigv4.c`. -Stefan Eissing (9 Mar 2024) + What's unexplainable is that this patch seems to mitigate TFTP tests + often hanging or going into an infinite loop on GHA windows workflows + with MSYS2, mingw-w64 and MSVC (Cygwin is unaffected): + https://github.com/curl/curl/pull/13599#issuecomment-2119372376 + TFTP hangs did not entirely disappear though, so could be unrelated. -- lib: keep conn IP information together + `time()` docs: + https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/time-time32 + -time64 + https://manpages.debian.org/bookworm/manpages-dev/time.2.en.html - new struct ip_quadruple for holding local/remote addr+port + Follow-up to 58ca0a2f0743a586716ca357c382b29e3f08db69 #13800 + Follow-up to d0728c9109629ee82b855b350a4c3f1f52ee61df #13643 + Closes #13815 - - used in data->info and conn and cf-socket.c - - copy back and forth complete struct - - add 'secondary' to conn - - use secondary in reporting success for ftp 2nd connection +Stefan Eissing (29 May 2024) - Reported-by: DasKutti on github - Fixes #13084 - Closes #13090 +- winsock: move SO_SNDBUF update into cf-socket -Daniel Stenberg (8 Mar 2024) + - Move the code that updates the SO_SNDBUF size for Windows to + cf_socket_send. -- scripts/managen: the new name and home for the manpage generator + Prior to this change the code was in readwrite_upload but the socket + filter is the more appropriate place because it applies to all sends. - It was previously docs/cmdline-opts/gen.pl + Background: - Closes #13089 + For Windows users SO_SNDBUF (the total per-socket buffer size reserved + by Winsock for sends) is updated dynamically by libcurl during the + transfer. This is because Windows does not do it automatically for + non-blocking sockets and without it the performance of large transfers + may suffer. -- VULN-DISCLOSURE-POLICY.md: update detail about CVE requests + Closes https://github.com/curl/curl/pull/13763 - curl is a CNA now +Jan Venekamp (29 May 2024) - Closes #13088 +- sectransp: use common code for cipher suite lookup -Stefan Eissing (8 Mar 2024) + Take advantage of the Curl_cipher_suite_walk_str() and + Curl_cipher_suite_get_str() functions introduced in commit fba9afe. -- lib: client reader polish + Closes #13521 - - seek_func/seek_client, use transfer values only - - remove copies held in `struct connectdata`, use only - ever `data->set.seek_func` - - resolves possible issues in multiuse connections - - new mime post reader eliminates need to ever overwriting this +Matthias Gatto (29 May 2024) - - websockets, remove empty Curl_ws_done() function +- aws-sigv4: url encode the canonical path - Closes #13079 + Refactors canon_query, so it could use the encoding part of the function + to use it in the path. -Marcel Raad (8 Mar 2024) + As the path doesn't encode '/', but encode '=', I had to add some + conditions to know If I was doing the query or path encoding. -- lib1598: fix `CURLOPT_POSTFIELDSIZE` usage + Also, instead of adding a `bool in_path` variable, I use `bool + *found_equals` to know if the function was called for the query or path, + as found_equals is used only in query_encoding. - It requires a `long` argument. + Test 472 verifies. - Closes https://github.com/curl/curl/pull/13085 + Reported-by: Alexander Shtuchkin + Fixes #13754 + Closes #13814 -Daniel Stenberg (8 Mar 2024) + Signed-off-by: Matthias Gatto -- docs/cmdline-opts: drop the curl.1 from the dist tarball +Daniel Stenberg (29 May 2024) - Since it is no longer needed for building tool_hugehelp.c and all the - docs is available in readable markdown format in the tarball, the peeps - that don't want to build the manpage still do good. +- cd2nroff: use an empty "##" to signal end of .IP sequence - Removing it also fixes the complexity of out-of-tree builds when the - curl.1 exists in the source tree. + Like when we list a series of options and then want to add "normal" text + again afterwards. -- test1140/1173: extend wildcards to find curl.1 + Without this, the indentation level wrongly continues even after the + final "##" header, making following text wrongly appear to belong to the + header above. - ... in its new build path. + Adjusted several curldown files to use this. - Also update the test scripts to be more precise in error messages to - help us understand CI errors better. + Fixes #13803 + Reported-by: Jay Satiro + Closes #13806 - Follow-up to f03c85635f35269f1 - Ref: #13029 - Closes #13083 +vvb2060 (28 May 2024) -- http2: minor tweaks to optimize two struct sizes +- openssl: fix %-specifier in infof() call - - use BIT() instead of bool - - place the struct fields in (roughly) size order + Closes #13816 - Closes #13082 +Daniel Stenberg (28 May 2024) -- buildconf.bat: remove outdated groff/nroff use +- curl: make warnings and other messages aware of terminal width - - don't try to generate the real hugehelp file, because it requires - curl.txt which needs a build - - don't attempt to do anything in a c-ares subdirectory + This removes unnecessary line wraps when the terminal is wider than 79 + columns and it also makes messages look better in narrower terminals. - Follow-up to f03c85635f35269 - Closes #13078 + The get_terminal_columns() function is not split out into its own source + file. -- http2: memory errors in the push callbacks are fatal + Suggested-by: Elliott Balsley + Fixes #13804 + Closes #13808 - Use the correct nghttp2 error code accordingly. +Viktor Szakats (28 May 2024) - Closes #13081 +- GHA: enable tests 1139, 1177, 1477 on Windows -Viktor Szakats (7 Mar 2024) + These exclusions came from the AppVeyor CI config, but they do pass now + and they are static tests with no flakiness risk. -- mkhelp: rename variable to fix compiler warnings + Follow-up to 0914d8aadddac0d1459673d5b7f77e8f3378b22b #13759 + Closes #13817 - ``` - src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global de - claration [_bld\src\curl.vcxproj] - _bld\src\tool_hugehelp.c(8,27): - see declaration of 'm' - src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global d - eclaration [_bld\src\curl.vcxproj] - src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global d - eclaration [_bld\src\curl.vcxproj] - src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global d - eclaration [_bld\src\curl.vcxproj] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75c - d2n0wj6lc#L614 +Dan Fandrich (28 May 2024) - Reviewed-by: Daniel Stenberg - Closes #13077 +- CI: Improve labeler tag detection -Daniel Stenberg (7 Mar 2024) + Also, simplify patterns with a single glob. -- KNOWN_BUGS: POP3 issue when reading small chunks +Viktor Szakats (28 May 2024) - Closes #12063 +- GHA: disable TFTP tests in Windows jobs -- RELEASE-NOTES: synced + Shot in the dark trying to find out which tests are + hanging / going to an infinite loop. -Robert Moreton (7 Mar 2024) + The ones failing after 45 minutes (mingw-w64) or 30 minutes (MSVC). -- asyn-ares: fix data race warning + Ref: https://github.com/curl/curl/pull/13599#issuecomment-2119372376 - - Store the c-ares version during global init. +renovate[bot] (28 May 2024) - Prior to this change several threads could write the same data to a - static int variable at the same time. Though in practice it's not a - problem ThreadSanitizer may warn. +- ci: update vmactions/omnios-vm digest to a61ca1e - Reported-by: Nikita Taranov - Assisted-by: Jay Satiro + Closes #13801 - Fixes #13065 - Closes #13000 +Daniel Stenberg (28 May 2024) -Stefan Eissing (7 Mar 2024) +- openssl/gnutls: rectify the TLS version checks for QUIC -- hyper: implement unpausing via client reader + The versions check wrongly complained and return error if the *minimum* + version was set to something less than 1.3. QUIC is always TLS 1.3, but + that means minimum 1.2 is still fine to ask for. - Just a tidy up to contain 'ifdef' pollution of common - code parts with implementation specifics. + This also renames the local variable to make the mistake harder to make + in the future. - - remove the ifdef hyper unpausing in easy.c - - add hyper client reader for CURL_CR_PROTOCOL phase - that implements the unpause method for calling - the hyper waker if it is set + Regression shipped in 8.8.0 - Closes #13075 + Follow-up to 3210101088dfa3d6a125 -- ngtcp2: no recvbuf for stream + Reported-by: fds242 on github + Fixes #13799 + Closes #13802 - - write response data directly to the transfer via - `Curl_xfer_write_resp()` like we do in HTTP/2. +Stefan Eissing (28 May 2024) - Closes #13073 +- gnutls: improve TLS shutdown -- docs/cmdline-opts/.gitignore: ignore curl.txt + local ftp upload tests sometimes failed with an invalid TLS record being + reported by gnutls. vsftp did log that the shutdown was not regarded as + clean, failing the control connection thereafter. - Closes #13076 + These changes make test_31_05 work reliable locally. -Evgeny Grin (Karlson2k) (7 Mar 2024) + - on closing the SSL filter, shutdown READ *and* WRITE + - on closing, try a receive after shutdown is sent + - convert to DEBUGF to CURL_TRC_CF -- sha512_256: add support for GnuTLS and OpenSSL + Closes #13790 - This is a follow-up for PR #12897. +Daniel Stenberg (28 May 2024) - Add support for SHA-512/256 digest calculation by TLS backends. - Currently only OpenSSL and GnuTLS (actually, nettle) support - SHA-512/256. +- RELEASE-NOTES: synced - Closes #13070 +- tests: run with "--trace-config all" to provide even more info -- digest: add check for hashing error + in case of problems. - Closes #13072 + Closes #13791 -Viktor Szakats (7 Mar 2024) +Viktor Szakats (28 May 2024) -- cmake: enable `ENABLE_CURL_MANUAL` by default +- build: untangle `CURLDEBUG` and `DEBUGBUILD` macros - Meaning `curl.1` and `src/tool_hugehelp.c` are built by default, - and `--manual` in curl tool is also enabled by default. + `CURLDEBUG` is meant to enable memory tracking, but in a bunch of cases, + it was protecting debug features that were supposed to be guarded with + `DEBUGBUILD`. - This syncs behaviour with autotools. + Replace these uses with `DEBUGBUILD`. - For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set - to a consistent date, e.g. the timestamp of `CHANGES`. + This leaves `CURLDEBUG` uses solely for its intended purpose: to enable + the memory tracking debug feature. - A pre-built manual (e.g. the one distributed in the official source - tarball) will be ignored and rebuilt after this patch, unless - explicitly disabling this option. + Also: + - autotools: rely on `DEBUGBUILD` to enable `checksrc`. + Instead of `CURLDEBUG`, which worked in most cases because debug + builds enable `CURLDEBUG` by default, but it's not accurate. + - include `lib/easyif.h` instead of keeping a copy of a declaration. + - add CI test jobs for the build issues discovered. - Fixes #13028 - Closes #13069 + Ref: https://github.com/curl/curl/pull/13694#issuecomment-2120311894 + Closes #13718 -Stefan Eissing (7 Mar 2024) +- examples: delete unused includes -- http2: push headers better cleanup + Delete a bunch of unnecessary-looking headers from some examples. This + is known to be tricky on AIX (perhaps also in other less-tested envs). - - provide common cleanup method for push headers + Let me know if any of this looks incorrect or outright fails on some + systems. - Closes #13054 + Follow-up to d4b85890555388bec212b75f47a5c1a48705b156 #13771 + Closes #13785 -Daniel Stenberg (7 Mar 2024) +- appveyor: fixup job name [ci skip] -- GIT-INFO: convert to markdown + Follow-up to fc8e0dee3045658f293452121f5290d81ba3aa1e #13694 - Closes #13074 +- cmake: fix `-Wredundant-decls` in unity/mingw-w64/gcc/curldebug/DLL builds -Richard Levitte (7 Mar 2024) + It affected cmake-unity shared-curltool curldebug mingw-w64 gcc builds + when building the `testdeps` target. -- cmake: fix libcurl.pc and curl-config library specifications + Apply the solution already used in `lib/base64.c` and `lib/dynbuf.c` + to fix it. - Letting CMake figure out where libraries are located gives you full - paths. When generating libcurl.pc and curl-config, getting libraries as - full paths is unusual when one expects to get a list of -l. + Also update an existing GHA CI job to test the issue fixed. - To meet expectations, an effort is made to convert the full paths into - -l, possibly with -L before it. + ``` + In file included from curl/lib/version_win32.c:35, + from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:145 + : + curl/lib/memdebug.h:52:14: error: redundant redeclaration of 'curl_dbg_logfil + e' [-Werror=redundant-decls] + 52 | extern FILE *curl_dbg_logfile; + | ^~~~~~~~~~~~~~~~ + In file included from curl/src/slist_wc.c:32, + from curl/_bld/src/CMakeFiles/curl.dir/Unity/unity_0_c.c:4: + curl/lib/memdebug.h:52:14: note: previous declaration of 'curl_dbg_logfile' w + ith type 'FILE *' {aka 'struct _iobuf *'} + 52 | extern FILE *curl_dbg_logfile; + | ^~~~~~~~~~~~~~~~ + curl/lib/memdebug.h:55:44: error: redundant redeclaration of 'curl_dbg_malloc + ' [-Werror=redundant-decls] + 55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t siz + e, + | ^~~~~~~~~~~~~~~ + curl/lib/memdebug.h:55:44: note: previous declaration of 'curl_dbg_malloc' wi + th type 'void *(size_t, int, const char *)' {aka 'void *(long long unsigned + int, int, const char *)'} + 55 | CURL_EXTERN ALLOC_FUNC ALLOC_SIZE(1) void *curl_dbg_malloc(size_t siz + e, + | ^~~~~~~~~~~~~~~ + [...] + curl/lib/memdebug.h:110:17: error: redundant redeclaration of 'curl_dbg_fclos + e' [-Werror=redundant-decls] + 110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *sou + rce); + | ^~~~~~~~~~~~~~~ + curl/lib/memdebug.h:110:17: note: previous declaration of 'curl_dbg_fclose' w + ith type 'int(FILE *, int, const char *)' {aka 'int(struct _iobuf *, int, c + onst char *)'} + 110 | CURL_EXTERN int curl_dbg_fclose(FILE *file, int line, const char *sou + rce); + | ^~~~~~~~~~~~~~~ + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49840554/job/a4aoet1 + 7e9qnqx1a#L362 - Fixes #6169 - Fixes #12748 - Closes #12930 + After: https://ci.appveyor.com/project/curlorg/curl/builds/49843735/job/hbo2u + ah2vj0ns523 -Daniel Stenberg (7 Mar 2024) + Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG`/shared-static c + ombinations) + Depends-on: #13694 + Depends-on: #13800 + Closes #13705 -- test463: HTTP with -d @file with file containing CR, LF and null byte +- lib: fix gcc warning in certain debug builds -- paramhlp: fix CRLF-stripping files with "-d @file" + ``` + curl/lib/http_aws_sigv4.c:536:10: error: 'clock' may be used uninitialized [- + Werror=maybe-uninitialized] + 536 | time_t clock; + | ^~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9158755123/job/25177765000#ste + p:13:79 - All CR and LF bytes should be stripped, as documented, and all other - bytes are inluded in the data. Starting now, it also excludes null bytes - as they would otherwise also cut the data short. + Cherry-picked from #13718 + Closes #13800 - Reported-by: Simon K - Fixes #13063 - Closes #13064 +- cmake: always build unit tests with the `testdeps` target -Viktor Szakats (7 Mar 2024) + Before this patch, the `testdeps` build target required `-DCURLDEBUG` + be set either via `ENABLE_DEBUG=ON` or `ENABLE_CURLDEBUG=ON` to build + the curl unit tests. -- cmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled + After fixing build issues in #13694, we can drop this requirement and + build unit tests unconditionally. - Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF` - when building without the Schannel TLS backend. + Depends-on: #13694 + Depends-on: #13697 (fix unit test issue revealed by Old Linux CI job) + Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 #11446 + Closes #13698 - This in turn may have caused Kerberos, SPNEGO and SSPI features - disappearing even with `CURL_WINDOWS_SSPI=ON` set. +- CI: disable dependency tracking in most autotools builds - This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a - default for `CURL_WINDOWS_SSPI`, but allowing a manual override. + For better build performance. Dependency tracking causes a build + overhead while compiling to help a subsequent build, but in CI there is + never one and the extra work is discarded. - Also update the option text to better tell its purpose. + Closes #13794 - Thanks-to: Andreas Loew - Reviewed-by: Daniel Stenberg - Ref: #13056 - Closes #13061 +- build: untangle `UNITTESTS` and `DEBUGBUILD` macros -Jay Satiro (6 Mar 2024) + - fix `DEBUGBUILD` guards that should be `UNITTESTS`, in libcurl code + used by unit tests. + - fix guards for libcurl functions used in unit tests only. + - sync `UNITTEST` attribute between declarations and definitions. + - drop `DEBUGBUILD` guard from test `unit2600`. + - fix guards for libcurl HSTS code used by both a unit test (`unit1660`) + and `test0446`. + - update an existing AppVeyor CI job to test the issues fixed. -- KNOWN_BUGS: FTPS server compatibility on Windows with Schannel + This fixes building tests with `CURLDEBUG` enabled but `DEBUGBUILD` + disabled. This can happen when building tests with CMake with + `ENABLE_DEBUG=ON` in Release config, or with `ENABLE_CURLDEBUG=ON` + and _without_ `ENABLE_DEBUG=ON`. Possibly also with autotools + when using `--enable-curldebug` without `--enable-debug`. - - Remove "2.12 FTPS with Schannel times out file list operation" + Test results: + - before: + https://ci.appveyor.com/project/curlorg/curl/builds/49835609 + https://ci.appveyor.com/project/curlorg/curl/builds/49898529/job/k8qpbs8idb + y70smw + https://github.com/curl/curl/actions/runs/9259078835/job/25470318167?pr=137 + 98#step:13:821 + - after: https://ci.appveyor.com/project/curlorg/curl/builds/49839255 + (the two failures are unrelated, subject to PR #13705) - - Remove "7.12 FTPS directory listing hangs on Windows with Schannel" + Ref: #13592 (issue discovery) + Ref: #13689 (CI testing this PR with `DEBUGBUILD`/`CURLDEBUG` combinations) + Closes #13694 - - Add "7.12 FTPS server compatibility on Windows with Schannel" +- GHA: ignore flaky MQTT and FTP test results [ci skip] - This change adds a more generic bug description that explains FTPS with - the latest curl and Schannel is not widely used and may have more bugs - than other TLS backends. + MQTT / OmniOS: + ``` + TESTFAIL: These test cases failed: 1190 1198 3017 + ``` + Ref: https://github.com/curl/curl/actions/runs/9258522297/job/25468730731?pr= + 13694#step:3:10251 - The two removed FTPS Schannel bugs can't be reproduced any longer and - were likely fixed by 24d6c288. + MQTT / OmniOS: + ``` + TESTFAIL: These test cases failed: 1194 2200 2203 2205 + ``` + Ref: https://github.com/curl/curl/actions/runs/9150523540/job/25155409832#ste + p:3:10233 - Ref: https://github.com/curl/curl/issues/5284 - Ref: https://github.com/curl/curl/issues/9161 - Ref: https://github.com/curl/curl/issues/12894 + FTP / OmniOS: + ``` + TESTFAIL: These test cases failed: 1096 + ``` + Ref: https://github.com/curl/curl/actions/runs/9150702711/job/25155793948#ste + p:3:10247 - Closes https://github.com/curl/curl/pull/13032 + FTP / OmniOS: + ``` + TESTFAIL: These test cases failed: 381 + ``` + Ref: https://github.com/curl/curl/actions/runs/9163863822/job/25193897640#ste + p:3:10230 -- trace-config.md: remove the mutexed options list + FTP / OmniOS: + ``` + TESTFAIL: These test cases failed: 340 + ``` + Ref: https://github.com/curl/curl/actions/runs/9233804752/job/25406671742?pr= + 13771#step:3:10245 - - Remove the rendered manpage message that says: - "[--trace-config] is mutually exclusive to --trace and -v, --verbose". + Ref: https://github.com/curl/curl/pull/13583#issuecomment-2119376898 - Actually it can be used with either of those options, which are mutually - exclusive to each other but not to --trace-config. +- CI: tidy up skipping tests build/run in Windows jobs - Ref: https://curl.se/docs/manpage.html#--trace-config + Simplify controlling whether to build and/run tests in a CI job. - Closes https://github.com/curl/curl/pull/13031 + Apply the TFLAGS='skipall' (do not build nor run tests) or + 'skiprun' (build, but do not run) method already used with old-mingw-w64 + and msvc jobs to existing Windows jobs in GHA and AppVeyor. -Daniel Stenberg (6 Mar 2024) + Also: + - add Cygwin/cmake test build and run steps while here. + - replace `DISABLED_TESTS` with `TFLAGS` in AppVeyor. -- mkhelp: simplify the generated hugehelp program + Closes #13796 - Use a plain array and puts() every line, also allows us to provide the - strings without ending newlines. +- cmake: use `APPLE` instead of `CMAKE_SYSTEM_NAME` string - - merge blank lines into the next one as a prefixed newline. - - turn eight consecutive spaces into a tab (since they can only be on the - left side of text) - - the newly generated tool_hugehelp is 3K lines shorter and 50K smaller - - modifies the top logo layout a little by reducing the indent + Follow-up to a86254b39307af1a53735b065a382567805cd9b8 #12515 + Closes #13713 - Closes #13047 +- cmake: whitespace, formatting/tidy-up in comments -- docs: ascii version of manpage without nroff + Also correct casing in a few option descriptions. - Create ASCII version of manpage without nroff + Closes #13711 - - build src/tool_hugegelp.c from the ascii manpage - - move the the manpage and the ascii version build to docs/cmdline-opts - - remove all use of nroff from the build process - - should make the build entirely reproducible (by avoiding nroff) +- cmake: allow `ENABLE_CURLDEBUG=OFF` with `ENABLE_DEBUG=ON` - - partly reverts 2620aa9 to build libcurl option man pages one by one - in cmake because the appveyor builds got all crazy until I did + Before this patch, `ENABLE_CURLDEBUG` (memory tracking) was + unconditionally enabled when `ENABLE_DEBUGBUILD` was set. This made + testing some build configurations complicated. To fix it, this patch + makes `ENABLE_CURLDEBUG` to receive the value of `ENABLE_DEBUG` by + default, while allowing free override by the user. - The ASCII version of the manpage + This allows to use the config: + `ENABLE_DEBUGBUILD=ON ENABLE_CURLDEBUG=OFF` + to enable debug features, without also enabling memory tracking. - - is built with gen.pl, just like the manpage is - - has a right-justified column making the appearance similar to the previous - version - - uses a 4-space indent per level (instead of the old version's 7) - - does not do hyphenation of words (which nroff does) + This is important because some other build methods allow to set one of + these features but not the other. This patch allows to test any + combination with CMake. - History + This makes it unnecessary to use the workaround of passing + `-DDEBUGBUILD` via `CMAKE_C_FLAGS`. Which has the disadvantage that our + CMake logic cannot easily detect it, e.g. for disabling symbol hiding on + Windows for `ENABLE_DEBUG`/`DEBUGBUILD` builds. - We first made the curl build use nroff for building the hugehelp file in - December 1998, for curl 5.2. + Cherry-picked from #13718 + Closes #13792 - Closes #13047 +- cmake: `ENABLE_DEBUG=ON` to always set `-DDEBUGBUILD` -Stefan Eissing (6 Mar 2024) + Before this patch `ENABLE_DEBUG=ON` always enabled the TrackMemory + (aka `ENABLE_CURLDEBUG=ON`) feature, but required the `Debug` CMake + configration to actually enable curl debug features + (aka `-DDEBUGBUILD`). -- lib: add `void *ctx` to reader/writer instances + Curl debug features do not require compiling with C debug options. This + also made enabling debug features unintuitive and complicated to use. + Due to other issues (subject to PR #13694) it also caused an error in + default (and `Release`/`MinSizeRel`/`RelWithDebInfo`) configs, when + building the `testdeps` target: + ``` + ld: CMakeFiles/unit1395.dir/unit1395.c.o: in function `test': + unit1395.c:(.text+0x1a0): undefined reference to `dedotdotify' + ``` + Ref: https://github.com/curl/curl/actions/runs/9037287098/job/24835990826#ste + p:3:2483 - - `struct Curl_cwriter` and `struct Curl_creader` now carry a - `void *ctx` member that points to the instance as allocated. - - using `r->ctx` and `w->ctx` as pointer to the instance specific - struct that has been allocated + Fix it by always defining `DEBUGBUILD` when setting `ENABLE_DEBUG=ON`. + Decoupling this option from the selected CMake configuration. - Reported-by: Rudi Heitbaum - Fixes #13035 - Closes #13059 + Note that after this patch `ENABLE_DEBUG=ON` unconditionally enables + curl debug features. These features are insecure and unsuited for + production. Make sure to omit this option when building for production + in default, `Release` (and other not-`Debug`) modes. -- http: fix dead code in setting post client reader + Also delete a workaround no longer necessary in GHA CI jobs. - - postsize was always 0, thus the check's else never happened - after the mime client reader was introduced + Ref: 1a62b6e68c08c7e471ff22dd92932aba7e026817 (2015-03-03) + Ref: #13583 + Closes #13592 - Follow-up to 0ba47146f7ff3d - Closes #13060 +- GHA: add autotools mingw-64, build-only job -- http2: fix push discard + Cherry-picked from #13718 + Closes #13793 - - fix logic in discarding a failed pushed stream so that - stream context is properly cleaned up +- GHA: add three MSVC jobs - Closes #13055 + Continuing the theme, add 3 MSVC jobs with tests, matching + configurations used on AppVeyor. MSVC versions are identical: + 19.39.33523.0 + Windows SDK 10.0.22621.0. -- transfer.c: break receive loop in speed limited transfers + Also enable websockets, and build examples. Tests are run in parallel + (`-j14`), with improved performance. - - the change breaks looping in transfer.c receive for transfers that are - speed limited on having gotten *some* bytes. - - the overall speed limit timing is done in multi.c + Job performance: + ``` + AppVeyor GHA + w/examples + -------- ---------- + CMake, VS2022, Debug, x64, Schannel, Static, Unicode 38m 4s 11m57s + CMake, VS2022, Debug, x64, no SSL, Static 35m15s 12m 6s + CMake, VS2022, Debug, x64, no SSL, Static, HTTP only 25m25s 10m36s + ``` + Based on these runs: + https://ci.appveyor.com/project/curlorg/curl/builds/49884748 + https://github.com/curl/curl/actions/runs/9229448468 + + This is the first time examples are built in CI with MSVC: Fix all + warnings and errors that came up via + d4b85890555388bec212b75f47a5c1a48705b156 #13771. + + Closes #13766 + +- GHA: add three old (gcc 6, 7, 9) mingw-w64 jobs + + Re-implement old mingw-w64 jobs in GHA. This allows to use the latest + Windows runners, replacing Windows Server 2012 R2 (gcc 6) and Windows + Server 2016 (gcc 7, 9) with Windows Server 2022. + + GHA runners are also significantly faster, and allow running tests in + parallel (`-j14`). It also offloads 3 more long-running jobs from + AppVeyor CI. + + These jobs download (then cache) the mingw-w64 packages from their + original location, which allows flexibility in choosing which versions + and flavours (win32/POSIX, SEH/DWARF, 64/32-bit) we want to test in CI. + The new jobs use these distros: + - https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20 + Win64/Personal%20Builds/mingw-builds/ (for gcc 7, same as on AppVeyor) + - https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20 + Win32/Personal%20Builds/mingw-builds/ (for gcc 6, same as on AppVeyor) + - https://winlibs.com/ (for gcc 9) + + I matched existing AppVeyor job configs, with these differences: + - gcc 6.4.0 instead of 6.3.0. + (same distro as on AppVeyor, but the latest bugfix release) + - gcc 9.5.0 instead of 9.1.0 and a different (but compatible) binary distro. + (in AppVeyor this relies on an old MSYS2 pre-installed on the runner) + - using win32 builds instead of posix for gcc 6.4.0 and 7.3.0. + - websockets enabled. + - always build examples. + - always build tests (this wasn't done for 6.4.0 with AppVeyor CI). + + I did not replicate existing test exclusions, and oddly enough the few + failures (so far) were different from MSYS2 jobs and also from their + AppVeyor CI counterparts. - Reported-by: Dmitry Karpov - Bug: https://curl.se/mail/lib-2024-03/0001.html - Closes #13050 + Also: + - delete redundant (default) `-u` option from `cygpath` calls. + - allow matrix options to override default ones in CMake. + - detect and use Windows-supplied curl for `TFLAGS` `-ac` option. + (it's available in modern runners.) + - delete the 3 AppVeyor CI jobs now replicated in GHA. + - appveyor: prefer `SYSTEMROOT` over `WINDIR`. + - tidy-up quotes. + + Job performance: + ``` + AppVeyor GH + A + w/ + examples + w/ + tests + -------- -- + -------- + CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity 1m25s 8 + m50s + CMake, mingw-w64, gcc 7, Debug, x64, Schannel, Static, Unicode 31m45s 9 + m39s + CMake, mingw-w64, gcc 9, Debug, x64, Schannel, Static 28m25s 13 + m38s + ``` + Based on these runs: + https://ci.appveyor.com/project/curlorg/curl/builds/49880799 + https://github.com/curl/curl/actions/runs/9218292508 -- mime: add client reader + Notice that building examples and tests is time consuming. - Add `mime` client reader. Encapsulates reading from mime parts, getting - their length, rewinding and unpausing. + We can tweak any build parameter as necessary to make them more useful + and/or without clogging the job queue or introducing flakiness. - - remove special mime handling from sendf.c and easy.c - - add general "unpause" method to client readers - - use new reader in http/imap/smtp - - make some mime functions static that are now only used internally + Closes #13759 - In addition: - - remove flag 'forbidchunk' as no longer needed +Daniel Stenberg (27 May 2024) - Closes #13039 +- TODO: remove some old, clarify, add something -Daniel Stenberg (5 Mar 2024) + Closes #13788 -- RELEASE-NOTES: synced +- TODO: Add "Share CA cache" + "CA caching to more TLS backends" -- TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake" + Closes #13787 - Follow-up to 8e741644a229c37 +Viktor Szakats (26 May 2024) -Tal Regev (5 Mar 2024) +- runtests: sort test IDs in summary lines -- cmake: add USE_OPENSSL_QUIC support + Changing this output: + ``` + TESTFAIL: These test cases failed: 2301 2303 2302 2307 + ``` + Ref: https://github.com/curl/curl/actions/runs/9228638364/job/25393106631#ste + p:6:21181 - Closes #13034 + To: + ``` + TESTFAIL: These test cases failed: 2301 2302 2303 2307 + ``` -Stefan Eissing (5 Mar 2024) + Cherry-picked from #13766 + Closes #13774 -- TIMER_STARTTRANSFER: set the same for everyone +- examples: fix compiling with MSVC - - set TIMER_STARTTRANSFER on seeing the first response bytes - in the download client writer, not coming from a CONNECT - - initialized the timer the same way for all protocols - - remove explicit setting of TIMER_STARTTRANSFER in file.c - and c-hyper.c + - `websocket.c`: use `Sleep()` on Windows. + `sleep()` and `unistd.h` are not available in MSVC. - Closes #13052 + - `http2-upload.c`: use local `gettimeofday()` implementation when + compiled with MSVC. + (Alternate solution is to disable the trace function for MSVC.) + Public domain code copied and adapted from libssh2: + https://github.com/libssh2/libssh2/blob/e973493f992313b3be73f51d3f7ca6d52e2 + 88558/src/misc.c#L719-L743 -Michael Kaufmann (5 Mar 2024) + - silence compiler warning for deprecated `inet_addr()`. + Also drop duplicate winsock2 include. + ``` + curl\docs\examples\externalsocket.c(125,32): error C2220: the following war + ning is treated as an error [curl\bld\docs\examples\curl-example-externalsock + et.vcxproj] + curl\docs\examples\externalsocket.c(125,32): warning C4996: 'inet_addr': Us + e inet_pton() or InetPton() instead or define _WINSOCK_DEPRECATED_NO_WARNINGS + to disable deprecated API warnings [curl\bld\docs\examples\curl-example-e + ``` + Ref: https://github.com/curl/curl/actions/runs/9227337318/job/25389073450#s + tep:4:95 -- http: better error message for HTTP/1.x response without status line + - silence an MSVC compiler warning. This is in conflict with `checksrc` + rules, so silence the rule in favour of the warning-free C syntax. + ``` + curl\docs\examples\multi-legacy.c(152,1): error C2220: the following warnin + g is treated as an error [curl\bld\docs\examples\curl-example-multi-legacy.vc + xproj] + curl\docs\examples\multi-legacy.c(152,1): warning C4706: assignment within + conditional expression [curl\bld\docs\examples\curl-example-multi-legacy.vcxp + roj] + ``` + Ref: https://github.com/curl/curl/actions/runs/9227337318/job/25389073450#s + tep:4:226 - If a response without a status line is received, and the connection is - known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status - line" instead of "Received HTTP/0.9 when not allowed". + - do not use `sys/time.h` and `unistd.h` in Windows builds. + Some of these includes look unnecessary. Subject to another PR. - Closes #13045 + Cherry-picked from #13766 + Closes #13771 -Viktor Szakats (5 Mar 2024) +Jonathan Matthews (26 May 2024) -- KNOWN_BUGS: fix typo +- docs/cmdline-opts: fix mail-auth example TLD typo - Reviewed-by: Daniel Stenberg - Closes #13051 + Closes: #13784 + Reviewed-by: Daniel Gustafsson -Sebastian Neubauer (5 Mar 2024) +Daniel Stenberg (26 May 2024) -- smpt: fix starttls +- libssh: remove CURLOPT_SSL_VERIFYHOST check - In cases where the connection was fast, curl sometimes failed to open a - connection. This fixes a regression of c2d973627bab12abc5486a3f3. + It was never meant for SSH: it should rely on the knownhosts file (if + set) in the same way libssh2 already does. - The regression triggered in these steps: + Reported-by: James Abbatiello + Fixes #13767 + Closes #13781 - 1. Create an smtp connection - 2. Use STARTTLS - 3. Receive the response - 4. We are inside the loop in `smtp_statemachine`, calling - `smtp_state_starttls_resp` - 5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and - run `smtp_perform_upgrade_tls` at the start of the function. +Stefan Eissing (26 May 2024) - In the bad flow, we stay in the while loop, calling - `Curl_pp_readresp`, which reads part of the TLS handshake and things - go wrong. +- multi: add multi->proto_hash, a key-value store for protocol data - The reason is that `Curl_pp_moredata` changed behavior and always - returns `true`, so we stay in the loop in `smtp_statemachine`. With a - slow connection `Curl_pp_readresp` cannot read new data and returns - `CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`. + - add `Curl_hash_add2()` that passes a destructor function for + the element added. Call element destructor instead of hash + destructor if present. + - multi: add `proto_hash` for protocol related information, + remove `struct multi_ssl_backend_data`. + - openssl: use multi->proto_hash to keep x509 shared store + - schannel: use multi->proto_hash to keep x509 shared store + - vtls: remove Curl_free_multi_ssl_backend_data() and its + equivalents in the TLS backends - With a fast connection, `Curl_pp_readresp` reads new data from the tcp - connection, which is part of the TLS handshake. + Closes #13345 - The fix is in `Curl_pp_moredata`, which needs to take the final line - into account and return `false` if only the final line is stored. +Jan Venekamp (25 May 2024) - Closes #13048 +- tests: add pytest for --ciphers and --tls13-ciphers options -Stefan Eissing (5 Mar 2024) + Closes #13530 -- lib: enhance client reader resume + rewind +Orgad Shaneh (25 May 2024) - - update client reader documentation - - client reader, add rewind capabilities - - tell creader to rewind on next start - - Curl_client_reset() will keep reader for future rewind if requested - - add Curl_client_cleanup() for freeing all resources independent of - rewinds - - add Curl_client_start() to trigger rewinds - - move rewind code from multi.c to sendf.c and make part of - "cr-in"'s implementation - - http, move the "resume_from" handling into the client readers - - the setup of a HTTP request is reshuffled to follow: - * determine method, target, auth negotiation - * install the client reader(s) for the request, including crlf - conversions and "chunked" encoding - * apply ranges to client reader - * concat request headers, upgrades, cookies, etc. - * complete request by determining Content-Length of installed - readers in combination with method - * send - - add methods for client readers to - * return the overall length they will generate (or -1 when unknown) - * return the amount of data on the CLIENT level, so that - expect-100 can decide if it want to apply itself - * set a "resume_from" offset or fail if unsupported - - struct HTTP has become largely empty now - - rename `Client_reader_*` to `Curl_creader_*` +- tool_operate: avoid explicitly setting verifypeer to 1 - Closes #13026 + Also for the proxy verison. It is the default, just like verifyhost, + since a long time. -Viktor Szakats (5 Mar 2024) + Closes #13704 -- openssl-quic: fix BIO leak and Windows warning +- tests: extend user/password parsing test1620 - Caused by an accidentally duplicated line in - d6825df334def106f735ce7e0c1a2ea87bddffb0. + Closes #13756 - ``` - .../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses intege - r precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-6 - 4-to-32] - 1095 | bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE); - | ~~~~~~~~~~~~~ ~~~~~~~^~~~~~ - 1 warning and 2 errors generated. - ``` +Alejandro R. Sedeño (25 May 2024) - Reviewed-by: Stefan Eissing - Closes #13043 +- configure: use `$EGREP` in place of `grep -E` -- openssl-quic: fix unity build, casing, indentation + `$EGREP` is set based on an earlier test in configure so that we can + work with systems that have `egrep` and a `grep` that does not support + `-E`. - - rename static functions to avoid duplicate symbols in unity mode. - - windows -> Windows/window in error message and comment. - - fix indentation. + Closes #13780 - Reviewed-by: Stefan Eissing - Closes #13044 +renovate[bot] (25 May 2024) -Daniel Stenberg (5 Mar 2024) +- ci: update dependency awslabs/aws-lc to v1.28.0 -- gen.pl: make the "manpageification" faster + Closes #13770 - The function that replaces occurances of "--longoption" with "-Z, - --longoption" etc with the proper highlight applied, no longer loops - over the options. +Jan Venekamp (25 May 2024) - Closes #13041 +- tests: test_17_ssl_use.py clarify mbedtls TLSv1.3 support -- CONTRIBUTE: update the section on documentation format + Closes #13779 - ... since most of it is markdown now. +Stefan Eissing (25 May 2024) - Closes #13046 +- http: write last header line late -- smtp: free a temp resource + - HEADERFUNCTIONS might inspect response properties like + CURLINFO_CONTENT_LENGTH_DOWNLOAD_T on seeing the last header line. If + the line is being written before this is initialized, values are not + available. - The returned address needs to be freed. + - write the last header line late when analyzing a HTTP response so that + all information is available at the time of the writing. - Follow-up to e3905de8196d67b89df1602feb84c1f993211b20 - Spotted by Coverity + - add test1485 to verify that CURLINFO_CONTENT_LENGTH_DOWNLOAD_T works + on seeing the last header. - Closes #13038 + Fixes #13752 + Reported-by: Harry Sintonen + Closes #13757 -- _VARIABLES.md: improve the description +Dan Fandrich (24 May 2024) - Closes #13040 +- tests: use exec when spawning nghttpx -dependabot[bot] (4 Mar 2024) + This stops keeping perl and shell processes around that are no longer + needed, plus it eliminates an unneeded shell message when the server is + later terminated. -- build(deps): bump fsfe/reuse-action from 2 to 3 + Closes #13772 - Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3. - - [Release notes](https://github.com/fsfe/reuse-action/releases) - - [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3) +Viktor Szakats (24 May 2024) - --- - updated-dependencies: - - dependency-name: fsfe/reuse-action - dependency-type: direct:production - update-type: version-update:semver-major - ... +- GHA: ignore flaky test 3017 (MQTT) on OpenBSD - Signed-off-by: dependabot[bot] + ``` + TESTFAIL: These test cases failed: 3017 + ``` + Ref: https://github.com/curl/curl/actions/runs/9223543272/job/25376999226?pr= + 13759#step:3:16326 + Ref: https://github.com/curl/curl/actions/runs/9230183764/job/25397883193?pr= + 13766#step:3:16345 -Stefan Eissing (4 Mar 2024) + Ref: https://github.com/curl/curl/pull/13583#issuecomment-2119376898 -- pytest: adapt to API change +Joseph Chen (24 May 2024) - - pytest has changed the signature of the hook pytest_report_header() - for some obscure reason and that change landed in our CI now +- build: add more supported attributes to the IAR compiler - - remove the changed param that we never used anyway + Closes #13744 - Closes #13037 +Viktor Szakats (24 May 2024) -Daniel Stenberg (4 Mar 2024) +- cmake: fix test 1013 with websockets enabled and no TLS -- cookie: if psl fails, reject the cookie + test 1013 is 'Compare curl --version with curl-config --protocols'. - A libpsl install without data and no built-in database is now considered - bad enough to reject all cookies since they cannot be checked. It is - somewhat of a user error, but still. + Ref: https://github.com/curl/curl/actions/runs/9228363859/job/25392251955 - Reported-by: Dan Fandrich - Closes #13033 + Closes #13769 -Stefan Eissing (4 Mar 2024) +- GHA: stop deleting curl in non-native workflows -- lib: further send/upload handling polish + We do it in Cirrus CI, but for some platforms it's not possible to + delete it and tests work anyway. - - Move all the "upload_done" handling to request.c + The test runner also runs `../src/curl` by default, which is always the + one freshly built. The runner may also need the system curl to talk to + APIs when needed. - - add possibility to abort sending of a request - - add `Curl_req_done_sending()` for checks - - transfer.c: readwrite_upload() now clean + Also: + - stop setting `CURL` env. This isn't picked up by the runners, + and works out of the box anyway. + - quote an option just in case. - - removing data->state.ulbuf and data->req.upload_fromhere + Follow-up to 90e644f944969bb11c6448bf50c6d441b5c0b1e6 #13583 + Closes #13765 - - as well as data->req.upload_present - - set data->req.upload_done on having read all from - the client and completely flushed the send buffer +Jay Satiro (24 May 2024) - - tftp, remove setting of data->req.upload_fromhere +- openssl: stop duplicate ssl key logging for legacy OpenSSL - - serves no purpose as `upload_present` is not set - and the data itself is directly `sendto()` anyway + - Don't call the keylog function if it has already logged the key. - - smtp, make upload EOB conversion a client reader - - xfer_ulbuf addition + For old OpenSSL versions and its forks that do not have support for + OpenSSL's keylog callback, libcurl has its own legacy key logging + function that logs the TLS 1.2 (and earlier) key (client random + master + key) on a single line. - - add xfer_ulbuf for borrowing, similar to xfer_buf - - use in file upload - - use in c-hyper body sending + Prior to this change, since e7de80e8 (precedes 8.8.0), the legacy key + logging function could write the same key line more than once (usually + twice) due to some incorrect logic. - - h1-proxy, remove init of data->state.uilbuf that is never used - - smb, add own send_buf instead of using data->state.ulbuf + Closes https://github.com/curl/curl/pull/13683 - Closes #13010 +Stefan Eissing (24 May 2024) -Daniel Stenberg (4 Mar 2024) +- transfer: remove curl_upload_refill_watermark, no longer used -- RELEASE-NOTES: synced + the define applied to upload buffers which we removed -kpcyrd (3 Mar 2024) + Closes #13764 -- rustls: fix two warnings related to number types +Daniel Stenberg (24 May 2024) - Reported-by: Gisle Vanem - Follow-up to #12989 - Closes #13017 +- RELEASE-NOTES: synced -Stefan Eissing (3 Mar 2024) +Viktor Szakats (24 May 2024) -- bufq: writing into a softlimit queue cannot be partial +- cmake: fix brotli lib order - - when unable to obtain a new chunk on a softlimit bufq, - this is an allocation error and needs to be reported as - such. - - writes into a soflimit bufq never must be partial success + Fix root cause that caused missing symbols when linking brotli + statically with e.g. binutils `ld` (and any other "picky" linker, + or "traditional" linker as CMake now calls them). - Reported-by: Dan Fandrich - Fixes #13020 - Closes #13023 + Also drop existing workaround that added brotli libs twice to the lib + list. -Dan Fandrich (2 Mar 2024) + ``` + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(decode.c.o + bj):decode.c:(.text$ProcessCommands[ProcessCommands]+0xbb5): undefined refere + nce to `BrotliTransformDictionaryWord' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(decode.c.o + bj):decode.c:(.text$SafeProcessCommands[SafeProcessCommands]+0xe8a): undefine + d reference to `BrotliTransformDictionaryWord' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(decode.c.o + bj):decode.c:(.rdata$.refptr._kBrotliContextLookupTable[.refptr._kBrotliConte + xtLookupTable]+0x0): undefined reference to `_kBrotliContextLookupTable' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(decode.c.o + bj):decode.c:(.rdata$.refptr._kBrotliPrefixCodeRanges[.refptr._kBrotliPrefixC + odeRanges]+0x0): undefined reference to `_kBrotliPrefixCodeRanges' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(state.c.ob + j):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x21): undef + ined reference to `BrotliDefaultAllocFunc' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(state.c.ob + j):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x2f): undef + ined reference to `BrotliDefaultFreeFunc' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(state.c.ob + j):state.c:(.text$BrotliDecoderStateInit[BrotliDecoderStateInit]+0x10e): unde + fined reference to `BrotliSharedDictionaryCreateInstance' + x86_64-w64-mingw32-ld: .../curl/brotli/_bld/usr/lib/libbrotlidec.a(state.c.ob + j):state.c:(.text$BrotliDecoderStateCleanup[BrotliDecoderStateCleanup]+0xf4): + undefined reference to `BrotliSharedDictionaryDestroyInstance' + collect2: error: ld returned 1 exit status + ``` -- configure: Don't build shell completions when disabled + Breakage reproducible with curl-for-win config "`win-gcc`" and deleting + the `LDFLAGS+=' -Wl,--start-group'` line from its `curl.sh` script. + (Above line still required for some non-brotli cases, e.g. libssh2 and + zlib.) - With the recent changes to completion file building, the files were - built always and only installation was selectively disabled. Now, when - they are disabled they aren't even built, avoiding a build-time error in - environments where it's not possible to run the curl binary that was - just created (e.g. if library paths were not set up correctly). + Assisted-by: Kai Pastor + Ref: https://github.com/curl/curl/pull/10857#discussion_r1611714989 + Follow-up to 1e3319a167d2f32d295603167486e9e88af9bb4e #10857 + Closes #13761 - Follow-up to 0f7aba83c +Pavel Pavlov (24 May 2024) - Reported-by: av223119 on github - Fixes #13027 - Closes #13030 +- cmake: fix building in unity mode -Jay Satiro (2 Mar 2024) + - Fix sha256 and sha512 duplicate macro names (eg function-like macro Ch + is now Sha256_Ch and Sha512_Ch). -- cmdline-opts/_EXITCODES: sync with libcurl-errors + - Avoid defining short defines like R, S. (eg S is now Sha256_S). - - Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that - can be returned by the curl tool. + Closes https://github.com/curl/curl/pull/13751 - Closes https://github.com/curl/curl/pull/13015 +Jay Satiro (24 May 2024) -Stefan Eissing (1 Mar 2024) +- winbuild: remove outdated WIN32 defines -- hyper: disable test1598 due to lack of trailer support + - Remove all instances in the makefile of compiler option /DWIN32. - Follow-up to 50838095 + This is a follow-up to e9a7d4a1 which replaced all defined(WIN32) checks + with defined(_WIN32) in the codebase, since only the latter is + automatically defined by all compilers for Windows builds. - Closes #13016 + Bug: https://github.com/curl/curl/pull/13739#issuecomment-2123937859 + Reported-by: Viktor Szakats -Dan Fandrich (1 Mar 2024) + Closes https://github.com/curl/curl/pull/13742 -- ftp: Mark a const buffer as const +renovate[bot] (24 May 2024) -- appveyor: Properly skip if only CircleCI is changed +- ci: update github/codeql-action digest to 9fdb3e4 -- docs: Update minimal binary size in INSTALL.md + Closes #13726 - Include more options to reduce binary size. +Pavel Pavlov (23 May 2024) -- configure: Don't make shell completions without perl +- asyn-thread: avoid using GetAddrInfoExW with impersonation - The code that attempted to skip building the shell completions didn't - work properly and tried to build them even if perl wasn't available. - This step, as well as the install step, is now properly skipped without - perl. + Multiple reports suggest that GetAddrInfoExW fails when impersonation is + used. This PR checks if thread is impersonating and avoids using + GetAddrInfoExW api. - Follow-up to 89733e2dd + Reported-by: Keerthi Timmaraju + Assisted-by: edmcln on github + Fixes #13612 + Closes #13738 - Closes #13022 +Stefan Eissing (23 May 2024) -RainRat (1 Mar 2024) +- transfer: conn close on paused upload -- misc: Fix typos in docs and lib + - add 2 variations on test_07_42 which PAUSEs uploads + and response connections terminating either right away + or after the 100-continue response + - when detecting the connection being closed in transfer.c + readwrite_data(), clear ALL send bits in data->req.keepon. + It no longer makes send to wait for a KEEP_SEND_PAUSE or HOLD. + - in the protocol client writer add the check for incomplete + response bodies. When an EOS is seen and the length is known, + check that and fail if bytes are missing. - This fixes miscellaneous typos and duplicated words in the docs, lib - and test comments and a few user facing errorstrings. + Reported-by: Sergey Bronnikov + Fixes #13740 + Closes #13750 - Author: RainRat on Github - Reviewed-by: Daniel Gustafsson - Reviewed-by: Dan Fandrich - Closes: #13019 +- CI GHA: add vsftpd to ngtcp2-linux runs -Dan Fandrich (29 Feb 2024) + - not using HTTP/3, but gnutls does not seem to run + somewhere else right now -- configure: build & install shell completions when enabled + Closes #13760 - The --with-fish-functions-dir and --with-zsh-functions-dir options - currently have no effect on a normal build because the scripts/ directory - where they're used is not built. Add scripts/ to a normal build and - change the completion options to default to off to preserve the existing - behaviour. +Orgad Shaneh (23 May 2024) - Closes: #12906 +- GHA: increase timeout for Cygwin autotools build tests step -- github/labeler: improve the match patterns + Apparently 10 minutes are not (always) enough: + https://github.com/curl/curl/actions/runs/9197003907/job/25296439556#step:8:1 + 936 -Stefan Eissing (28 Feb 2024) + Closes #13753 -- tests: add test1598 for POST with trailers +Stefan Eissing (22 May 2024) - - test POST fields with trailers and chunked encoding +- mbedtls: send close-notify on close - Ref: #12938 - Closes #13009 + - send the TLS close notify message when cloding down + the mbedtls connection filter + - this is a "least" effort version and, as other TLS filters, + is lacking a graceful send/receive/timeout for a really + clean shutdown. -Daniel Stenberg (28 Feb 2024) + Closes #13745 -- cmdline-opts/_VERSION: provide %VERSION correctly +- mbedtls: check version for cipher id - ... so that it does not get included verbatim in the output. Fixes a - regression shipped in 8.6.0. + mbedtls_ssl_get_ciphersuite_id_from_ssl() seems to have been added in + mbedtls 3.2.0. Check for that version. - Also fix a format mistake in form.md + Closes #13749 - Closes #13008 +Viktor Szakats (22 May 2024) -Stefan Eissing (28 Feb 2024) +- cmake: fix building with both md4 and md5 in unity mode -- lib: Curl_read/Curl_write clarifications + Macro and static function names were colliding between + `lib/md4.c` and + `lib/md5.c`. - - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to - clarify when and at what level they operate - - send/recv of transfer related data is now done via - `Curl_xfer_send()/Curl_xfer_recv()` which no longer has - socket/socketindex as parameter. It decides on the transfer - setup of `conn->sockfd` and `conn->writesockfd` on which - connection filter chain to operate. - - send/recv on a specific connection filter chain is done via - `Curl_conn_send()/Curl_conn_recv()` which get the socket index - as parameter. - - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for - naming consistency - - clarify that the special CURLE_AGAIN hangling to return - `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` - and CURLE_AGAIN is returned by all other send() variants. - - fix a bug in websocket `curl_ws_recv()` that mixed up data - when it arrived in more than a single chunk (to be made - into a sperate PR, also) + Fix it by namespacing these symbols. - Added as documented [in - CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68 - aa06dc738cb75d4/docs/CLIENT-READERS.md). + Seen with a basic macOS build using these options: + `-DCMAKE_UNITY_BUILD=ON -DCURL_USE_SECTRANSP=ON` - - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` - - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` - - HTTP chunked uploads are now formatted in a client reader added when - needed. - - FTP line-end conversions are done in a client reader added when - needed. - - when sending requests headers, remaining buffer space is filled with - body data for sending in "one go". This is independent of the request - body size. Resolves #12938 as now small and large requests have the - same code path. + Closes #13737 - Changes done to test cases: +Daniel Stenberg (22 May 2024) - - test513: now fails before sending request headers as this initial - "client read" triggers the setup fault. Behaves now the same as in - hyper build - - test547, test555, test1620: fix the length check in the lib code to - only fail for reads *smaller* than expected. This was a bug in the - test code that never triggered in the old implementation. +- docs/Makefile.am: make curl-config.1 install - Closes #12969 + on "make install" like it should -Daniel Gustafsson (28 Feb 2024) + Follow-up to 60971d665b9b1df87082 -- curldown: Fix email address in Copyright + Closes #13741 - The curldown conversion accidentally replaced daniel@haxx.se with - just daniel.se. This reverts back to the proper email address in - the curldown docs as well as in a few other stray places where it - was incorrect (while unrelated to curldown). +dependabot[bot] (22 May 2024) - Reviewed-by: Daniel Stenberg - Closes: #12997 +- GHA: bump actions/checkout from 4.1.4 to 4.1.6 -Daniel Stenberg (28 Feb 2024) + Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.4 to 4 + .1.6. + - [Release notes](https://github.com/actions/checkout/releases) + - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) + - [Commits](https://github.com/actions/checkout/compare/0ad4b8fadaa221de15dce + c353f45205ec38ea70b...a5ac7e51b41094c92402da3b24376905380afc29) -- getparam: make --ftp-ssl work again + --- + updated-dependencies: + - dependency-name: actions/checkout + dependency-type: direct:production + update-type: version-update:semver-patch + ... - Follow-up to 9e4e527 which accidentally broke it + Signed-off-by: dependabot[bot] - Reported-by: Jordan Brown - Fixes #13006 - Closes #13007 + Closes #13720 -- KNOWN_BUGS: IMAPS connection fails with rustls error +Stefan Eissing (22 May 2024) - Closes #10457 +- pytest: add ftp upload tests -- KNOWN_BUGS: FTPS upload, FileZilla, GnuTLS and close_notify + - refs #13556 + - allow anon uploads on vsftpd test server + - add test_30_05 for plain upload of 1k, 100k, 1m + - add test_31_05 for SSL upload of 1k, 100k, 1m + - verify file size and contents - Closes #11383 + Closes #13734 -- KNOWN_BUGS: Implicit FTPS upload timeout +- test: add test1546, chunked not last transfer encoding - Closes #11720 + with more than one transfer-encoding, 'chunked' must be the last added + to the writer stack (and therefore the first to decode). RFC 9112, ch. + 6.1. -- KNOWN_BUGS: HTTP/2 prior knowledge over proxy + Closes #13736 - Closes #12641 +- test: add test1484, for HEAD with content -- TODO: build HTTP/3 with OpenSSL and nghttp3 using cmake + - test HEAD request with 'Transfer-Encoding:chunked' and + non-encoded response content + - verifies #13725 - Closes #12988 + Closes #13735 -- TODO: Select signature algorithms +Daniel Stenberg (22 May 2024) - Closes #12982 +- RELEASE-NOTES: synced -- examples: use present tense in comments + bump to 8.8.1 for now - remove "will" and some other word fixes +Viktor Szakats (22 May 2024) - Closes #13003 +- (lib)curl.rc: set debug flag also for `CURLDEBUG` and `UNITTESTS` -- docs: more language cleanups + These macros also enable debug features in both libcurl and curl. + Enable `VS_FF_DEBUG` version resource flag when they are set. - - present tense - - avoid bad words + Closes #13730 - Closes #13003 +Jay Satiro (22 May 2024) -Daniel Gustafsson (27 Feb 2024) +- winbuild: fix PE version info debug flag -- setopt: Fix disabling all protocols + - Only set PE file flag VS_FF_DEBUG if curl.exe and libcurl.dll were + built with winbuild option DEBUG=yes which builds with debug info. - When disabling all protocols without enabling any, the resulting - set of allowed protocols remained the default set. Clearing the - allowed set before inspecting the passed value from --proto make - the set empty even in the errorpath of no protocols enabled. + VS_FF_DEBUG is a PE flag (Portable Executable file flag - dll, exe, etc) + that indicates the file contains or was built with debug info. - Co-authored-by: Dan Fandrich - Reported-by: Dan Fandrich - Reviewed-by: Daniel Stenberg - Closes: #13004 + Prior to this change when winbuild was used to build curl, curl.exe + and libcurl.dll always had VS_FF_DEBUG set, regardless of build option + DEBUG=yes/no, due to some bad logic. -Andreas Kiefer (27 Feb 2024) + Closes https://github.com/curl/curl/pull/13739 -- fopen: fix narrowing conversion warning on 32-bit Android +Version 8.8.0 (22 May 2024) - This was fixed in commit 06dc599405f, but came back in commit - 03cb1ff4d62. +Daniel Stenberg (22 May 2024) - When building for 32-bit ARM or x86 Android, `st_mode` is defined as - `unsigned int` instead of `mode_t`, resulting in a - `-Wimplicit-int-conversion` clang warning because `mode_t` is - `unsigned short`. Add a cast to silence the warning, but only for - 32-bit Android builds, because other architectures and platforms are - not affected. +- RELEASE-NOTES: synced + +- THANKS: add contributors from 8.8.0 + +Nathan Moinvaziri (21 May 2024) + +- url: remove duplicate call to Curl_conncache_remove_conn when pruning + + - remove unnecessary prunedead struct from prune_dead_connections + - rename extract_if_dead to prune_if_dead for clarity + + Closes #13710 + +Joseph Chen (21 May 2024) + +- curl_setup.h: add support for IAR compiler + + Closes #13728 + +Stephen Farrell (21 May 2024) + +- docs/ECH: typo/clarification + + Closes #13727 + +Viktor Szakats (21 May 2024) + +- hash: delete unused debug function + + It had no use in the curl codebase and was also protected by the macro + `AGGRESSIVE_TEST` (renamed in 2020), also with no local reference. + + Added in ca6e77083768858aa34207f8c5dce38b3c05336d (2002-11-11) + + Closes #13729 + +Stefan Eissing (21 May 2024) + +- content_encoding: reject transfer-encoding after chunked + + reject a response that applies a transfer-encoding after a 'chunked' + encoding. RFC 9112 ch. 6.1 required chunked to be the final encoding. + + Closes #13733 + +- http: HEAD response body tolerance + + - as reported in #13725, some servers wrongly send body bytes in + responses to a HEAD request. This used to be tolerated in curl + 8.4 and before and leads to failed transfers in newer versions. + - restore previous behaviour for HTTP/1.1 and HTTP/2: + * 1.1: do not add 'Transfer-Encoding' writers from HEAD + responses. RFC 9112 says they do not apply. + * 2: when the transfer expects 'no_body', to not report stream + resets as error when all response headers have been received. + + Reported-by: Jeroen Ooms + Fixes #13725 + Closes #13732 + +Viktor Szakats (20 May 2024) + +- tests: fix TFTP test 2305 on Windows + + Ref: #13692 + Closes #13724 + +Jay Satiro (20 May 2024) + +- openssl: revert keylog_callback support for LibreSSL + + - Revert to the legacy TLS 1.2 key logging code for LibreSSL. + + - Document SSLKEYLOGFILE for LibreSSL is TLS 1.2 max. + + Prior to this change if the user specified a filename in the + SSLKEYLOGFILE environment variable and was using LibreSSL 3.5.0+ then + an empty file would be created and no keys would be logged. + + This is effectively a revert of e43474b4 which changed openssl.c to use + SSL_CTX_set_keylog_callback for LibreSSL 3.5.0+. Unfortunately LibreSSL + added that function only as a stub that doesn't actually do anything. + + Reported-by: Gonçalo Carvalho + + Fixes https://github.com/curl/curl/issues/13672 + Closes https://github.com/curl/curl/pull/13682 + +renovate[bot] (19 May 2024) + +- GHA: pin dependencies + + Closes #13712 + +Viktor Szakats (19 May 2024) + +- appveyor: drop unnecessary `--clean-first` cmake option + + In CI all machines are fresh on startup, making the `clean` operation + unnecessary. This can save some time/energy for each job run. + + Closes #13707 + +- cmake: merge two `if(BUILD_TESTING)` branches + + Closes #13708 + +Tatsuhiro Tsujikawa (19 May 2024) + +- GHA: bump nghttp2 to v1.62.1 + + Use gcc-12 explicitly to compile C++20 source files. + + Closes #13702 + +Viktor Szakats (19 May 2024) + +- GHA: add NetBSD, OpenBSD, FreeBSD/arm64 and OmniOS jobs + + Add these jobs to GHA: + - NetBSD, cmake-unity, clang, OpenSSL, x86_64, with tests, w/o python, + no parallelism (was flaky sometimes) + - OpenBSD, cmake-unity, clang, LibreSSL, x86_64, with tests, + with python, -j8, TFTP results ignored due to #13623. + - FreeBSD, cmake-unity and autotools, clang, OpenSSL, arm64 + (Tests disabled for arm64, because they are slow. It's available for + x86_64 with python, -j12.) + Configuration matches our existing Cirrus CI one. + - OmniOS, autotools, gcc, OpenSSL, x86_64, with tests, -j12. + + All build with websockets and examples. + + Closes #13583 + +- GHA: disable TFTP test on native Windows + + Some TFTP tests seem to enter into a loop and maybe hang? + + E.g. 1007, 1009, 1238 + + Try fixing it by skipping all TFTP tests. + + Ref: https://github.com/curl/curl/actions/runs/9141987545/job/25137038249?pr= + 13698 + + Also drop mingw-w64 test exclusions copy-pasted from MSYS jobs. + + Possibly related: cffbcc3110c1eda2e333f9cfe2e269154618793a #5364 + + Close #13699 + +renovate[bot] (18 May 2024) + +- GHA: pin dependencies + + Closes #13691 + +Viktor Szakats (18 May 2024) + +- cmake: do not pass linker flags to the static library tool + + Do not add linker flags to the global CMake static library tool (aka + "static linker") (e.g. `ar`) flags list. They don't mix well. This was + only done after successfully detecting GSSAPI. + + Linker flags seen on Old Linux CI: + ``` + -- |GSS_LINKER_FLAGS|-Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib/x86_64-li + nux-gnu/heimdal| + -- |CMAKE_STATIC_LINKER_FLAGS| -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib + /x86_64-linux-gnu/heimdal| + ``` + Ref: https://github.com/curl/curl/actions/runs/9138988036/job/25130791712#ste + p:6:85 + + Causing: + ``` + /usr/bin/ar qc libcurltool.a -Wl,--enable-new-dtags -Wl,-rpath -Wl,/usr/lib/ + x86_64-linux-gnu/heimdal + CMakeFiles/curltool.dir/slist_wc.c.o CMakeFiles/curltool.dir/tool_binmode.c + .o CMakeFiles/curltool.dir/tool_bname.c.o + [...] + CMakeFiles/curltool.dir/tool_writeout_json.c.o CMakeFiles/curltool.dir/tool + _xattr.c.o CMakeFiles/curltool.dir/var.c.o + CMakeFiles/curltool.dir/__/lib/base64.c.o CMakeFiles/curltool.dir/__/lib/dy + nbuf.c.o + /usr/bin/ar: invalid option -- 'W' + Usage: /usr/bin/ar [emulation options] [-]{dmpqrstx}[abcDfilMNoPsSTuvV] [--pl + ugin ] [member-name] [count] archive-file file... + /usr/bin/ar -M [now - c->timestamp; + | ^~~ + curl/lib/hostip.c: In function 'Curl_hostcache_prune': + curl/lib/hostip.c:241:10: note: 'now' was declared here + 241 | time_t now; + | ^~~ + In function 'hostcache_timestamp_remove', + inlined from 'fetch_addr' at curl/lib/hostip.c:310:8: + curl/lib/hostip.c:205:23: error: 'user.now' may be used uninitialized [-Werro + r=maybe-uninitialized] + 205 | time_t age = prune->now - c->timestamp; + | ~~~~~^~~~~ + curl/lib/hostip.c: In function 'fetch_addr': + curl/lib/hostip.c:304:33: note: 'user' declared here + 304 | struct hostcache_prune_data user; + | ^~~~ + In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity + _0_c.c:40: + curl/lib/cf-socket.c: In function 'cf_socket_send': + curl/lib/cf-socket.c:1294:10: error: 'c' may be used uninitialized [-Werror=m + aybe-uninitialized] + 1294 | if(c >= ((100-ctx->wblock_percent)*256/100)) { + | ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + curl/lib/cf-socket.c:1292:19: note: 'c' was declared here + 1292 | unsigned char c; + | ^ + In file included from curl/_bld/lib/CMakeFiles/libcurl_object.dir/Unity/unity + _0_c.c:364: + In function 'tftp_state_timeout', + inlined from 'tftp_multi_statemach' at curl/lib/tftp.c:1230:27: + curl/lib/tftp.c:1208:5: error: 'current' may be used uninitialized [-Werror=m + aybe-uninitialized] + 1208 | if(current > state->rx_time + state->retry_time) { + | ^ + curl/lib/tftp.c: In function 'tftp_multi_statemach': + curl/lib/tftp.c:1192:10: note: 'current' was declared here + 1192 | time_t current; + | ^~~~~~~ + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49792835/job/91c8dj5 + qb36spfe0#L112 + Ref: https://github.com/curl/curl/actions/runs/9082968838/job/24960616145#ste + p:12:62 + + Ref: #13592 + Closes #13643 + +Andrew (16 May 2024) - Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li - bc/include/sys/stat.h#86 - Closes https://github.com/curl/curl/pull/12998 +- wakeup_create: use FD_CLOEXEC/SOCK_CLOEXEC -Stefan Eissing (27 Feb 2024) + for `pipe()`/`socketpair()` -- lib: Curl_read/Curl_write clarifications + Fixes #13618 + Closes #13625 - - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to - clarify when and at what level they operate - - send/recv of transfer related data is now done via - `Curl_xfer_send()/Curl_xfer_recv()` which no longer has - socket/socketindex as parameter. It decides on the transfer - setup of `conn->sockfd` and `conn->writesockfd` on which - connection filter chain to operate. - - send/recv on a specific connection filter chain is done via - `Curl_conn_send()/Curl_conn_recv()` which get the socket index - as parameter. - - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for - naming consistency - - clarify that the special CURLE_AGAIN hangling to return - `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` - and CURLE_AGAIN is returned by all other send() variants. - - fix a bug in websocket `curl_ws_recv()` that mixed up data - when it arrived in more than a single chunk +Stefan Eissing (16 May 2024) - The method for sending not just raw bytes, but bytes that are either - "headers" or "body". The send abstraction stack, to to bottom, now is: +- rustls: fix partial send handling - * `Curl_req_send()`: has parameter to indicate amount of header bytes, - buffers all data. - * `Curl_xfer_send()`: knows on which socket index to send, returns - amount of bytes sent. - * `Curl_conn_send()`: called with socket index, returns amount of bytes - sent. + When TLS bytes could not completely sent off, the amount of plain bytes + already added to rustls were forgotten. This lead to send those byte + duplicate, corrupting the request send to the server. - In addition there is `Curl_req_flush()` for writing out all buffered - bytes. + Closes #13676 - `Curl_req_send()` is active for requests without body, - `Curl_buffer_send()` still being used for others. This is because the - special quirks need to be addressed in future parts: +- pytest: add DELETE tests, check server version - * `expect-100` handling - * `Curl_fillreadbuffer()` needs to add directly to the new - `data->req.sendbuf` - * special body handlings, like `chunked` encodings and line end - conversions will be moved into something like a Client Reader. + - add tests for DELETE working + - check apache version in keepalive test + - fix some comments - In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, - replace the `ssize_t` with a `size_t`. It makes no sense to allow for negativ - e - values as the returned `CURLcode` already specifies error conditions. This - allows easier handling of lengths without casting. + Closes #13679 - Closes #12964 +Juliusz Sosinowicz (16 May 2024) -Daniel Stenberg (27 Feb 2024) +- vquic-tls: use correct cert name check API for wolfSSL -- multi: make add_handle free any multi_easy + wolfSSL_X509_check_host checks the peer name against the alt names and + the common name. - If the easy handle that is being added to a multi handle has previously - been used for curl_easy_perform(), there is a private multi handle here - that we can kill off. While it flushes some caches etc for the easy - handle would it be used for an easy interface transfer again after being - used in the multi stack, this cleanup simplifies behavior and uses less - memory. + Fixes #13487 + Closes #13680 - Closes #12992 +Viktor Szakats (16 May 2024) -- docs: use present tense +- cmake: initialize `BUILD_TESTING` before first use - avoid "will", detect "will" as a bad word in the CI + Before this patch `BUILD_TESTING` was used once, then initialized, then + used again. This caused the `curlu` library not being built when relying + on an implicit `BUILD_TESTING=ON` setting, and ending up with a link + error when building the `testdeps` target. - Also line wrapped a bunch of paragraphs + It did not cause issues when `BUILD_TESTING` was explicitly set. - Closes #13001 + Move the initialization before the first use to fix it. -- CURLOPT_SSL_CTX_FUNCTION.md: no promises of lifetime after return + Regression from aace27b0965c10394544d1dacc9c2cb2fe0de3d3 #12287 + Closes #13668 - ... and cleanup other language. +Daniel Stenberg (16 May 2024) - Closes #12999 +- libtest: 2308 verifies CURLE_WRITE_ERROR after write callback error -Stefan Eissing (27 Feb 2024) + Verifies that the issue in #13669 actually is fixed. This return code is + what the CURLOPT_WRITEFUNCTION manpage documents should be returned. -- lib: send rework + This code is mostly from the + Source-written-by: Trumeet on github + Closes #13671 - Curl_read/Curl_write clarifications +Antoine Bollengier (16 May 2024) - - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify - when and at what level they operate +- socketpair: fix compilation when USE_UNIX_SOCKETS is not defined - - send/recv of transfer related data is now done via - `Curl_xfer_send()/Curl_xfer_recv()` which no longer has - socket/socketindex as parameter. It decides on the transfer setup of - `conn->sockfd` and `conn->writesockfd` on which connection filter - chain to operate. + Closes #13666 - - send/recv on a specific connection filter chain is done via - `Curl_conn_send()/Curl_conn_recv()` which get the socket index as - parameter. +Stefan Eissing (16 May 2024) - - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming - consistency +- rustsls: fix error code on receive - - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` - with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is - returned by all other send() variants. + - use CURLE_RECV_ERROR instead of CURLE_READ_ERROR when receiving + data fails. - SingleRequest reshuffling + Closes #13670 - - move functions into request.[ch] - - differentiate between reset and free - - add Curl_req_done() to perform last actions - - add a send `bufq` to SingleRequest for future use in keeping upload data +Max Dymond (16 May 2024) - Closes #12963 +- ci: disable Renovate dashboard -Daniel Stenberg (26 Feb 2024) + The Renovate dashboard insists on an open issue, + which is a problem. Disable the dashboard. Status + can still be seen at https://developer.mend.io/github/curl/curl. + + Fixes #13630 + Closes #13673 + +Daniel Stenberg (16 May 2024) - RELEASE-NOTES: synced -- http_chunks: remove unused 'endptr' variable +renovate[bot] (16 May 2024) - Closes #12996 +- GHA: update awslabs/aws-lc to v1.27.0 -Louis Solofrizzo (26 Feb 2024) + Closes #13667 -- lib: initialize output pointers to NULL before calling strto[ff,l,ul] +Daniel Stenberg (15 May 2024) - In order to make MSAN happy: +- curl_easy_pause.md: use correct defines in example - ==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x596f3b3ed246 in curlx_strtoofft [...]/libcurl/src/lib/strtoofft.c:23 - 9:11 - #1 0x596f3b402156 in Curl_httpchunk_read [...]/libcurl/src/lib/http_chunk - s.c:149:12 - #2 0x596f3b348550 in readwrite_data [...]/libcurl/src/lib/transfer.c:607: - 11 - [...] + Spotted-by: Harry Sintonen + Closes #13664 - ==2202041==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x5a3fab66a72a in Curl_parse_port [...]/libcurl/src/lib/urlapi.c:547:8 - #1 0x5a3fab650645 in parse_authority [...]/libcurl/src/lib/urlapi.c:796:1 - 2 - #2 0x5a3fab6740f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 - #3 0x5a3fab664fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: - 1342:12 - [...] +Viktor Szakats (15 May 2024) - ==2202320==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x569076a0d6b0 in ipv4_normalize [...]/libcurl/src/lib/urlapi.c:683:12 - #1 0x5690769f2820 in parse_authority [...]/libcurl/src/lib/urlapi.c:803:1 - 0 - #2 0x569076a160f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 - #3 0x569076a06fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: - 1342:12 - [...] +- appveyor: more tidy-ups + + - use `--disable` when calling `curl --version`. Just in case. + + - use single-quotes for a constant. + + Closes #13662 + +- reuse: migrate standalone license file to dep5 + + Follow-up to 73a36021207284ad2b4340ffde34a51b0ba4d47a + Closes #13660 + +- appveyor: guard against crash-build with VS2008 + + The combination of `-DDEBUGBUILD`, a shared `curl.exe`, and the VS2008 + compiler creates a `curl.exe` segfaulting on startup: + + ``` + + _bld/src/curl.exe --version + ./appveyor.sh: line 122: 793 Segmentation fault "${curl}" --version + Command exited with code 139 + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49817266/job/651iy6q + n1e238pqj#L191 + + Add job that triggers the issue and add the necessary logic to skip + running the affected `curl.exe`. + + Ref: #13592 + Closes #13654 + +renovate[bot] (15 May 2024) + +- GHA: pin dependencies + + Closes #13628 + +Orgad Shaneh (15 May 2024) + +- socket: remove redundant call to getsockname + + The result "add" is unused. + + Closes #13655 + +renovate[bot] (15 May 2024) + +- CI: renovate updates + + - GHA: update actions/checkout action to v4 + - GHA: update wolfSSL/wolfssh to v1.4.17 + - GHA: update wolfSSL/wolfssl to v5.7.0 + - Update the regex config in renovate.json + + Closes #13632 + Closes #13641 + Closes #13658 + Closes #13659 + +Max Dymond (15 May 2024) + +- ci: fix renovate config for WolfSSL/WolfSSH tagging scheme + + WolfSSL/WolfSSH use a different versioning scheme; + stable builds end with `-stable`. Renovate requires + some extra configuration to extract the version + from these types of tags. + + Closes #13644 + +- ci: set semantic type as CI and include digests as CI operations + + Replace "chore" with "ci" for renovate's semantic + type, and include digests with "pin" and + "pinDigest" as ci operations. - Signed-off-by: Louis Solofrizzo - Closes #12995 + Closes #13644 -Stefan Eissing (26 Feb 2024) +Daniel Stenberg (15 May 2024) -- lib: move client writer into own source +- DEPRECATE.md: TLS libraries without 1.3 support - Refactoring of the client writer that passes the data to the - client/application's callback functions. + curl drops support for TLS libraries without TLS 1.3 capability after + May 2025. - - split out into own source cw-out.[ch] from sendf.c + It requires that a curl build using the library should be able to + negotiate and use TLS 1.3, or else it is not good enough. We support a + vast amount of other TLS libraries that are likely to satisfy users + better. - - move tempwrite and tempcount from data->state into the context of the - client writer + Closes #13544 - - redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On - paused transfers, this allows to "record" interleaved HEADER/BODY - chunks to be "played back" in the same order on unpausing. +- Revert "ci: update nghttp2/nghttp2 to v1.62.0" - - keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. - On exceeding that, return CURLE_TOO_LARGE instead of - CURLE_OUT_OF_MEMORY as before. + This reverts commit 14f2c767555b7598d7783ccd9093670b84d28488. - - add method to be called when a transfer is DONE to allow writing of - any data still buffered + We need to also upgrade the C++ compiler for that bump to work. - - when paused, record HEADER writes exactly as they come for later - playback. HEADERs are documented to be written one-by-one. + Closes #13656 - Closes #12898 +renovate[bot] (15 May 2024) -- urldata: move authneg bit from conn to Curl_easy +- Dockerfile: update debian digest to 911821c - - from `conn->bits.authneg` to `data->req.authneg` - - this is a property of the request about to be made - and not a property of the connection - - in multiuse connections, transfer could step on each others - toes here potentially. + Closes #13629 - Closes #12949 +- ci: update gnutls/gnutls to v3.8.5 -- c-hyper: add header collection writer in hyper builds + Closes #13640 - Closes #12880 +- ci: update awslabs/aws-lc to v1.26.0 -- http: move headers collecting to writer + Closes #13647 - - add a client writer that does "push" response - headers written to the client if the headers api - is enabled - - remove special handling in sendf.c - - needs to be installed very early on connection - setup to catch CONNECT response headers +- ci: update cloudflare/quiche to v0.21.0 - Closes #12880 + Closes #13648 -- sendf: Curl_client_write(), make passed in buf const +- ci: update libressl-portable/portable to v3.9.2 -Michał Antoniak (26 Feb 2024) + Closes #13649 -- lib: remove curl_mimepart object when CURL_DISABLE_MIME +- ci: update nghttp2/nghttp2 to v1.62.0 - Remove curl_mimepart object from UserDefined structure when - CURL_DISABLE_MIME flag is active. Reduce size of UserDefined structure. + Closes #13650 - Also remove unreachable code: when CURL_DISABLE_MIME is set, httpreq can - never have HTTPREQ_POST_MIME value and the same goes for the - CURL_DISABLE_FORM_API flag and the HTTPREQ_POST_FORM value +- ci: update ngtcp2/nghttp3 to v1.3.0 - Closes #12948 + Closes #13651 -kpcyrd (26 Feb 2024) +- ci: update ngtcp2/ngtcp2 to v1.5.0 -- rustls: make curl compile with 0.12.0 + Closes #13652 - Closes #12989 +Max Dymond (14 May 2024) -Daniel Stenberg (26 Feb 2024) +- ci: handle git submodules for mbedTLS -- strtoofft: fix the overflow check +- ci: reconfigure renovate - ... to not rely on wrapping, since it is an undefined behavior that is - not what always might happen. This is in our private strtoff() parser - function, used only on platforms without a native version. + - set prefix for github actions updates to be gha: + - set prefix for other renovate actions to be ci: + - disable debian updates in linux-old.yml - Reported-by: vulnerabilityspotter on hackerone - Closes #12990 +Viktor Szakats (14 May 2024) -- libssh/libssh2: return error on too big range +- tidy-up: whitespace [ci skip] - If trying to get the range 0 - 2^63 and the remote file is 2^63 bytes or - larger. +- warnless: delete orphan declarations - Fixes #12983 - Closes #12984 + Follow-up to 358f7e757781857c4b498a68634726609fa3884a #11932 + Closes #13639 -Scott Talbert (24 Feb 2024) +Daniel Stenberg (14 May 2024) -- setopt: fix check for CURLOPT_PROXY_TLSAUTH_TYPE value +- BUG-BOUNTY.md: clarify the third party situation - Prior to this change CURLOPT_PROXY_TLSAUTH_TYPE would return - CURLE_BAD_FUNCTION_ARGUMENT on any type other than NULL. Since there is - only one type of TLS auth and it is also the default (SRP) the TLS auth - would work anyway. + We do not pay bounties for problems in other libraries. - Closes https://github.com/curl/curl/pull/12981 + Closes #13560 -Jay Satiro (24 Feb 2024) +Stefan Eissing (14 May 2024) -- mprintf: fix format prefix I32/I64 for windows compilers +- http tests: in CI skip test_02_23* for quiche - - Support I32 & I64 (eg: %I64d) for all Win32 builds. + For unknown reasons, these tests fail in CI often, but run fine locally. + Skip them in CI to avoid unrelated PRs to have failures. - Prior to this change mprintf support for the I format prefix, which is a - Microsoft extension, was dependent on the compiler used. + Closes #13638 - When Borland compiler support was removed in fd7ef00f the prefix was - then no longer supported for that compiler; however since it's still - possible to build with Borland I'm restoring support for the prefix in - this way. +Daniel Gustafsson (14 May 2024) - Reported-by: Paweł Witas +- hsts: explicitly skip blank lines - Fixes https://github.com/curl/curl/issues/12944 - Closes https://github.com/curl/curl/pull/12950 + Keep blank lines or lines containing only whitespace to make it all + the way to the more expensive sscanf call in hsts_add. -Daniel Stenberg (23 Feb 2024) + Closes: #13603 + Reviewed-by: Daniel Stenberg -- cd2nroff: gen: make `\>` in input to render as plain '>' in output +- autotools: Only probe for SGI MIPS compilers on IRIX - The same (copy and pasted) fix/mistake as in gen.pl + MIPSPro and the predecessor compiler which was part of the IDO (IRIS + Development Option) were only ever shipped on the SGI IRIX operating + system (with MIPSPro on 6.0+ which was released in 1994). Limit the + autoconf check to IRIX when probing for these compilers to save some + cycles on other platforms. -- gen: make `\>` in input to render as plain '>' in output + Closes: #13611 + Reviewed-by: Daniel Stenberg - Reported-by: Gisle Vanem - Fixes #12977 - Closes #12978 +Viktor Szakats (14 May 2024) -Fabrice Fontaine (23 Feb 2024) +- tests: fix test 1167 to skip digit-only symbols -- configure.ac: find libpsl with pkg-config + This avoids mistaking symbols with their numeric value when using + certain C preprocessors which output these numeric values at the + beginning of the line as part of an expression. - Find libpsl with pkg-config to avoid static build failures. + Seen on OpenBSD 7.5 + clang. - Ref: http://autobuild.buildroot.org/results/1fb15e1a99472c403d0d3b1a688902f32 - e78d002 + Example `test1167.pl -v` output, before this patch: + ``` + Source: cpp /home/runner/work/curl/curl/tests/../include/curl/curl.h + Symbol: 20000 + Line #3835: 20000 + 142, + [...] + Bad symbols in public header files: + 20000 + [...] + ``` + Ref: https://github.com/curl/curl/actions/runs/9069136530/job/24918015357#ste + p:3:7513 - Signed-off-by: Fabrice Fontaine - Closes #12947 + Ref: #13583 + Closes #13634 -Daniel Stenberg (23 Feb 2024) +Daniel Stenberg (14 May 2024) -- BUG-BOUNTY.md: clarify that the curl security team decides +- lib: call Curl_strntolower instead of doing crafted loops - Closes #12975 + Closes #13627 -- THANKS: add bug reporter from #740 +- setopt: acknowledge errors proper for CURLOPT_COOKIEJAR - Ref: https://github.com/curl/curl/issues/740 + Error out on error, do not continue. -Stefan Eissing (22 Feb 2024) + Closes #13624 -- multi: fix multi_sock handling of select_bits +- vtls: remove duplicate assign - - OR the event bitmask to data->state.select_bits instead of overwriting - them. They are cleared again on use. + Curl_ssl_peer_cleanup() already clears the ->sni field, no point in + assigning it again. - Reported-by: 5533asdg on github - Fixes #12971 - Closes #12972 + Spotted by CodeSonar -Daniel Stenberg (22 Feb 2024) + Closes #13626 -- curlver: bump to 8.7.0 for next release +Max Dymond (13 May 2024) -- RELEASE-NOTES: synced +- Group all non-major updates together to reduce PR spam -- write-out: add '%{proxy_used}' +- Add the remainder of the workflows - Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to - for example determine if a `NOPROXY` pattern matched the hostname or - not. +- Add some basic versioning for some workflows to check whether this is detecte + d properly - Extended test 970 and 972 +renovate[bot] (13 May 2024) -- CURLINFO_USED_PROXY: return bool whether the proxy was used +- Add renovate.json - Adds test536 to verify +Daniel Stenberg (13 May 2024) - Closes #12719 +- vauth: make two functions void that always just returned OK -- sha512_256: remove the cast macro, minor language/format edits + Removes the need to check return values when they can never fail. - Follow-up to cbe41d151d6a100c + Pointed out by CodeSonar - Closes #12966 + Closes #13621 -Stefan Eissing (20 Feb 2024) +- setopt: remove check for 'option' that is always true -- DoH: add trace configuration + - make sure that passing in option set to NULL clears the fields + correctly - - refs #12397 where it is dicussed how to en-/disable verbose output - of DoH operations - - introducing `struct curl_trc_feat` to track a curl feature for - tracing - - adding `data->state.feat` optionally pointing to the feature a - transfer belongs to - - adding trace functions and verbosity checks on features - - using trace feature in DoH code - - documenting `doh` as feature for `--trace-config` + - remove the weird second take if Curl_parse_login_details() returns + error - Closes #12411 + Follow-up to 7333faf00bf25db7cd1e0012d6b140 -- websocket: fix curl_ws_recv() + Spotted by CodeSonar - - when data arrived in several chunks, the collection into - the passed buffer always started at offset 0, overwriting - the data already there. + Closes #13619 + +Viktor Szakats (13 May 2024) + +- tests: tidy up types in server code - adding test_20_07 to verify fix + Cherry-picked from #13489 + Closes #13610 - - debug environment var CURL_WS_CHUNK_SIZE can be used to - influence the buffer chunk size used for en-/decoding. +Daniel Stenberg (13 May 2024) - Closes #12945 +- setopt: make the setstropt_userpwd args compulsory -Evgeny Grin (Karlson2k) (20 Feb 2024) + They were always used so no point in allowing them to be optional. -- digest: support SHA-512/256 + follow-up to 0e37b42dc956bd8a - Also fix the tests. New implementation tested with GNU libmicrohttpd. - The new numbers in tests are real SHA-512/256 numbers (not just some - random ;) numbers ). + Closes #13608 + Reviewed-by: Daniel Gustafsson -- tests: add SHA-512/256 unit test +- RELEASE-NOTES: synced -- SHA-512/256: implement hash algorithm +Daniel Gustafsson (13 May 2024) - Closes #12897 +- websocket: Avoid memory leak in error path -- curl_setup.h: add curl_uint64_t internal type + In the errorpath for randstr being too long to copy into the buffer + we leak the randstr when returning CURLE_FAILED_INIT. Fix by using + an explicit free on randstr in the errorpath. - The unsigned version of curl_off_t basically + Closes: #13602 + Reviewed-by: Daniel Stenberg -Daniel Stenberg (20 Feb 2024) +- hsts: Remove single-use single-line function -- docs: dist curl*.1 and install without perl + The hsts_entry() function contains of a single line and is only + used in a single place in the code, so move the allocation into + hsts_create instead to improve code readability. C code usually + don't use the factory abstraction for object creation, and this + small example wasn't following our usual code style. - Drop docs/mk-ca-bundle.1 from the tarball. It can be generated at will. + Closes: #13604 + Reviewed-by: Daniel Stenberg - Closes #12959 - Fixes #12921 - Reported-by: Michael Forney +Viktor Szakats (12 May 2024) -Stefan Eissing (20 Feb 2024) +- lib: bump hash sizes to `size_t` -- OpenSSL QUIC: adapt to v3.3.x + Follow-up to cc907e80a2498c0599253271a6f657f614b52a4e #13502 + Cherry-picked from #13489 + Closes #13601 - - set our idle timeout as transport parameter - - query negotiated idle timeout for connection alive checks - - query number of available bidi streams on a connection - - use write_ex2 with SSL_WRITE_FLAG_CONCLUDE to signal - EOF on last chunk write, so stream close does not - require an additional QUIC packet +- tests: make the unit test result type `CURLcode` - Closes #12933 + Before this patch, the result code was a mixture of `int` and + `CURLcode`. -Ramiro Garcia (19 Feb 2024) + Also adjust casts and fix a couple of minor issues found along the way. -- MANUAL.md: fix typo + Cherry-picked from #13489 + Closes #13600 - Closes #12965 +- appveyor: tidy-ups -Daniel Stenberg (19 Feb 2024) + - delete a duplicate line. + - simplify a `make` call. + - merge two `if` branches. + - reorder autotools options for clarity. + - add `--enable-warnings` where missing (it's also the default.) + - add empty lines to YAML for readability. + - use lowercase install prefix/directory. -- BINDINGS: add mcurl, the python binding + Closes #13598 - Ref: #12956 - Closes #12962 +Daniel Stenberg (12 May 2024) -- mk-ca-bundle.md: cleanups and polish +- docs/cmdline-opts: mention STARTTLS for --ssl and --ssl-reqd - Closes #12958 + ... since users might look for those terms in the manpage. -- spellcheck.yml: remove .1/.3 handling, clean all man page .md files + Closes #13590 - Since we generate all .1 and .3 files from markdown now, we can limit - the spellcheck to the markdown versions only. +- setopt: warn on Curl_set*opt() uses not using the return value - Closes #12960 + And switch the invokes that would "set" NULL to instead just plainly + free the pointer, as those were otherwise the invokes that would ignore + the return code. And possibly confuse static code analyzers. -- libcurl-docs: cleanups + Closes #13591 - CURLMOPT_SOCKETDATA.md: fix typo - CURLMOPT_TIMERDATA.md: fix typo - CURLOPT_COOKIELIST.m: quote strings - CURLOPT_PREREQFUNCTION.md: quote variable names - CURLOPT_TCP_NODELAY.md: rephrased to please spell checker - CURLOPT_WILDCARDMATCH.md: rephrased - libcurl-tutorial.md: use correct option name - curl_global_init_mem.md: quote headers - curl_easy_getinfo.md: use correct symbol names in headers - curl_global_trace.md: quote some headers - curl_ws_meta.md: quote struct field names - libcurl-env.md: quote headers +Orgad Shaneh (12 May 2024) -- cd2nroff: remove backticks from titles +- autotools: delete unused functions -- RELEASE-NOTES: synced + Closes #13605 -Stefan Eissing (18 Feb 2024) +Viktor Szakats (11 May 2024) -- http_chunks: fix the accounting of consumed bytes +- examples: fix/silence `-Wsign-conversion` - Prior to this change chunks were handled correctly although in verbose - mode libcurl could incorrectly warn of "Leftovers after chunking" even - if there were none. + - extend `FD_SET()` hack to all platforms (was only Cygwin). + Warnings may also happen in other envs, e.g. OmniOS. + Ref: https://github.com/libssh2/libssh2/actions/runs/8854199687/job/2431676 + 2831#step:3:2021 - Reported-by: Michael Kaufmann + - tidy-up `CURLcode` vs `int` use. - Fixes https://github.com/curl/curl/issues/12937 - Closes https://github.com/curl/curl/pull/12939 + - cast an unsigned to `long` before passing to `curl_easy_setopt()`. -- file: use xfer buf for file:// transfers + Cherry-picked from #13489 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + Closes #13501 - - For file:// transfers use the multi handle's transfer buffer for - up- and downloads. +Orgad Shaneh (11 May 2024) - Prior to this change a6c9a33 (precedes 8.6.0) changed the file:// - transfers to use a smaller stack based buffer, and that caused a - significant performance decrease in Windows. +- cmake: fix `HAVE_IOCTLSOCKET_FIONBIO` test with gcc 14 - Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086 - Reported-by: edmcln@users.noreply.github.com + The function signature has had u_long flags since ever. This is how it + is defined in the documentation, and implemented in MinGW. - Closes https://github.com/curl/curl/pull/12932 + The code that uses ioctlsocket in nonblock.c also has unsigned long. -Karthikdasari0423 (18 Feb 2024) + Error: + CurlTests.c:275:41: error: passing argument 3 of 'ioctlsocket' from incompati + ble pointer type [-Wincompatible-pointer-types] + 275 | if(0 != ioctlsocket(0, FIONBIO, &flags)) + | ^~~~~~ + | | + | int * + In file included from CurlTests.c:266: + /opt/mxe/usr/i686-w64-mingw32.static/include/winsock2.h:1007:76: note: expect + ed 'u_long *' {aka 'long unsigned int *'} but argument is of type 'int *' + 1007 | WINSOCK_API_LINKAGE int WSAAPI ioctlsocket(SOCKET s,__LONG32 cmd,u_ + long *argp); + | ~~ + ~~~~~~^~~~ -- HTTP3.md: always run nghttp3 submodule init + Closes #13578 - - For consistency change all 'build nghttp3' commands to run submodule - init after cloning, even if the branch does not have submodules. +Jay Satiro (10 May 2024) - Follow-up to 5a4b2f93 and 4f794558. +- ftp: fix build for CURL_DISABLE_VERBOSE_STRINGS - Closes https://github.com/curl/curl/pull/12928 + This is a follow-up to b7c7dffe which changed the FTP state change + verbose debug text (aka infof) to tracing debug text (aka trc). -LeeRiva (18 Feb 2024) + Prior to this change if libcurl was without DEBUGBUILD and built with + CURL_DISABLE_VERBOSE_STRINGS (ie --disable-verbose) the build would + error. -- CURLOPT_POSTQUOTE.md: fix typo + Caught by Circle CI job openssl-no-verbose. - Closes https://github.com/curl/curl/pull/12926 +- lib: clear the easy handle's saved errno before transfer -Evgeny Grin (Karlson2k) (18 Feb 2024) + - Clear data->state.os_errno before transfer. -- checksrc.pl: fix handling .checksrc with CRLF + - Explain the change in behavior in the CURLINFO_OS_ERRNO doc. - - When parsing .checksrc chomp the (CR)LF line ending. + - Add to the CURLINFO_OS_ERRNO doc the list of libcurl network-related + errors that may cause the errno to be saved. - Prior to this change on Windows checksrc.pl would not process the - symbols in .checksrc properly, since many git repos in Windows use auto - crlf to check out files with CRLF line endings. + data->state.os_errno is saved before libcurl returns a network-related + failure such as connection failure. It is accessible to the user via + CURLINFO_OS_ERRNO so they can get more information about the failure. - Closes https://github.com/curl/curl/pull/12924 + Prior to this change it wasn't cleared before transfer, so if a user + retrieved the saved errno it could be from a previous transfer. That is + because an errno is not always saved for network-related errors. -Richard Levitte (18 Feb 2024) + Closes https://github.com/curl/curl/pull/13574 -- cmake: fix install for older CMake versions +Stefan Eissing (10 May 2024) - - Generate the docs install list by using a foreach loop instead of - LIST:TRANSFORM since older CMake can't handle the latter. +- ftp: add tracing support - Reported-by: Dan Fandrich + - add `Curl_trc_feat_ftp` for tracing via trace config + - add macro CURL_TRC_FTP(data, fmt, ...) + - replace DEBUGF(infof()) statements in ftp.c by CURL_TRC_FTP() + - always trace FTP connection state - Fixes https://github.com/curl/curl/issues/12920 - Closes https://github.com/curl/curl/pull/12922 + Closes #13580 -Stefan Eissing (16 Feb 2024) +Daniel Stenberg (10 May 2024) -- vtls: fix tls proxy peer verification +- http: remove redundant check - - When verifying a proxy certificate for an ip address, use the correct - ip family. + Spotted by CodeSonar - Prior to this change the "connection" ip family was used, which was not - necessarily the same. + Closes #13582 - Reported-by: HsiehYuho@users.noreply.github.com +Viktor Szakats (10 May 2024) - Fixes https://github.com/curl/curl/issues/12831 - Closes https://github.com/curl/curl/pull/12931 +- ldap: fix unused variables (seen on OmniOS) -Dan Fandrich (15 Feb 2024) + ``` + ../../lib/ldap.c: In function 'ldap_do': + ../../lib/ldap.c:380:11: error: unused variable 'ldap_ca' [-Werror=unused-v + ariable] + 380 | char *ldap_ca = conn->ssl_config.CAfile; + | ^~~~~~~ + ../../lib/ldap.c:379:9: error: unused variable 'ldap_option' [-Werror=unuse + d-variable] + 379 | int ldap_option; + | ^~~~~~~~~~~ + ``` + Ref: https://github.com/curl/curl/actions/runs/9033564377/job/24824192730#ste + p:3:6059 -- CI: Bump the Circle CI base Ubuntu image to the latest 20.04 + Ref: #13583 + Closes #13588 - The previous ones are going to be removed soon, plus the new ones - include all the fixes since then. +Daniel Stenberg (10 May 2024) -Jay Satiro (13 Feb 2024) +- url: make parse_login_details use memdup0 -- transfer: improve Windows SO_SNDBUF update limit + Also make the user and password arguments mandatory, since all code + paths in libcurl used them anyway. - - Change the 1 second SO_SNDBUF update limit from per transfer to per - connection. + Adapted unit test case 1620 to the new rules. + + Closes #13584 - Prior to this change many transfers over the same connection could cause - many SO_SNDBUF updates made to that connection per second, which was - unnecessary. +Orgad Shaneh (10 May 2024) - Closes https://github.com/curl/curl/pull/12911 +- digest: replace strcpy for empty string with simple assignment -- schannel: fix hang on unexpected server close + Closes #13586 - - Treat TLS connection close (either due to a close_notify from the - server or just closed due to receiving 0) as pending data. +Viktor Szakats (10 May 2024) - This is because in some cases schannel_recv knows the connection is - closed but has to return actual pending data so it can't return 0 or an - error to indicate no more data. In this case schannel_recv must be - called again, which only happens if readwrite_data sees that there is - still pending data. +- autotools: fix `HAVE_IOCTLSOCKET_FIONBIO` test for gcc 14 - Prior to this change if the total size of the body that libcurl expected - to receive from the server was unknown then it was possible under some - network conditions that libcurl would hang waiting to receive more data, - when in fact a close_notify alert indicating no more data would be sent - was already processed. + ``` + conftest.c:152:41: error: passing argument 3 of 'ioctlsocket' from incompatib + le pointer type [-Wincompatible-pointer-types] + 152 | if(0 != ioctlsocket(0, FIONBIO, &flags)) + | ^~~~~~ + | | + | int * + ``` - Fixes https://github.com/curl/curl/issues/12894 - Closes https://github.com/curl/curl/pull/12910 + Reported-by: LigH + Fixes #13579 + Closes #13587 -Daniel Stenberg (10 Feb 2024) +- CI: ignore test 286 on Appveyor gcc 7 build -- KNOWN_BUGS: FTP upload fails if remebered dir is deleted + Disabled earlier for gcc 9 builds. gcc 7 uses the same runner and + prone to similar intermittent failures. - Closes #12181 - Closes #12923 + Follow-up to f1e05a6e6e7225fa09952abb2c935ae1abe44f45 #12106 #12040 + Closes #13575 -Michał Antoniak (10 Feb 2024) +Daniel Stenberg (10 May 2024) -- mbedtls: use mbedtls_ssl_conf_{min|max}_tls_version +- cf-socket: don't try getting local IP without socket - ... instead of the deprecated mbedtls_ssl_conf_{min|max}_version + In cf_tcp_connect(), it might fail and not get a socket assigned to + ctx->sock but set_local_ip() is still called which would make + getsockname() get invoked with a negative file desriptor and fail. - Closes #12905 + By adding this check, set_local_ip() will now instead blank out the + fields correctly. -Dan Fandrich (9 Feb 2024) + Spotted by CodeSonar -- CI: bump to actions/cache@v4 to avoid warning + Closes #13577 -Evgeny Grin (Karlson2k) (9 Feb 2024) +- tool_getparam: remove two redundant conditions -- test1165: improve pattern matching + When getstr() does not return error, it returns a valid pointer. - * Fix excluded digits at the end of the symbols ('CURL_DISABLE_POP3' - was checked as 'CURL_DISABLE_POP') + Spotted by CodeSonar - Closes #12903 + Closes #13576 -Dan Fandrich (9 Feb 2024) +Stefan Eissing (10 May 2024) -- scripts: Fix cijobs.pl for Azure and GHA +- quiche: trust its timeout handling - The spacing in the yaml files changed. + - set the idle timeout transport parameter + in milliseconds as documented by quiche + - do not calculate the idle timeout, rely on + quiche handling it -Daniel Stenberg (9 Feb 2024) + Closes #13581 -- RELEASE-NOTES: synced +Daniel Stenberg (10 May 2024) -- TODO: use pkg-config to find libpsl +- dmaketgz: accept a SOURCE_DATE_EPOCH as an second argument - Closes #12919 + to make it easier to reproduce a tarball -- TODO: avoid nroff + Closes #13573 - Instead of adjusting roffit, skip the nroff step. +- RELEASE-NOTES: synced - Closes #12919 +Stefan Eissing (10 May 2024) -Dan Fandrich (9 Feb 2024) +- h3/ngtcp2: improve error handling -- Revert "CI: run Circle macOS builds on x86 for now" + - identify ngtcp2 and nghttp3 error codes that are fatal + - close quic connection on fatal errors + - refuse further filter operations once connection is closed + - confusion about the nghttp3 API. We should close the QUIC stream on + cancel and not use the nghttp3 calls intended to be invoked when the + QUIC stream was closed by the peer. - This reverts commit 2683de3078eadc86d9b182e7417f4ee75a247e2c. - ARM resources are now available in Circle CI, so run these builds on ARM - again. This platform needs explicit paths set to libpsl and its - dependency icu4c. + Closes #13562 - Follow-up to 2683de30 +Jay Satiro (10 May 2024) - Closes #12635 +- docs: fix some CURLINFO examples -Viktor Szakats (9 Feb 2024) + - improve getinfo result check for example sections: + CURLINFO_ACTIVESOCKET, CURLINFO_LASTSOCKET, CURLINFO_SSL_VERIFYRESULT, + CURLINFO_PROXY_SSL_VERIFYRESULT -- cmake: add warning for using TLS libraries without 1.3 support + - fix getinfo result check for example sections: + CURLINFO_NUM_CONNECTS, CURLINFO_OS_ERRNO - Closes #12900 + - fix verify result check for example sections: + CURLINFO_PROXY_SSL_VERIFYRESULT -Daniel Stenberg (9 Feb 2024) + Bug: https://github.com/curl/curl/discussions/13557#discussion-6625507 + Reported-by: farazrbx@users.noreply.github.com -- configure: add warning for using TLS libraries without 1.3 support + Closes https://github.com/curl/curl/pull/13559 - Closes #12900 +Daniel Stenberg (9 May 2024) -Michał Antoniak (9 Feb 2024) +- KNOWN_BUGS: gssapi library name + version is missing in curl_version_info() -- mbedtls: fix building when MBEDTLS_X509_REMOVE_INFO flag is defined + Closes #13492 + Closes #13570 - Closes #12904 +- krb5: use dynbuf -Stefan Eissing (9 Feb 2024) + Closes #13568 -- ftp: fix socket wait activity in ftp_domore_getsock +- managen: fix the option sort order - - when waiting on the data connection, always add the control socket to - the pollset on state STOP or let the pingpong add the socket according - to its needs. + ... it used to strip off the .d file extension to sort correctly but + ever since the extension changed to .md the operation failed and the + sort got wrong. - Reported-by: Fabian Vogt - Fixes #12901 - Closes #12913 + Follow-up to 2494b8dd5175cee7f2e -Daniel Stenberg (9 Feb 2024) + Closes #13567 -- dist: make sure the http tests are in the tarball +Stefan Eissing (8 May 2024) - Fixes #12914 - Reported-by: Fabian Vogt - Closes #12917 +- GHA: repair the linux-old job -Stefan Eissing (9 Feb 2024) + package libc6_2.28-10+deb10u2_amd64.deb changed to + libc6_2.28-10+deb10u3_amd64.deb -- multi: add xfer_buf to multi handle + Closes #13564 - - can be borrowed by transfer during recv-write operation - - needs to be released before borrowing again - - adjustis size to `data->set.buffer_size` - - used in transfer.c readwrite_data() +Viktor Szakats (8 May 2024) - Closes #12805 +- appveyor: make gcc 6 mingw64 job build-only -Daniel Stenberg (9 Feb 2024) + This job has proven to be the flakiest of all, and it's also the oldest + Windows runner we had tests running on: 'Visual Studio 2015', that is + running on Windows Server 2012 R2: + https://www.appveyor.com/docs/windows-images-software/ -- write-out.md: clarify error handling details + Turn off tests on this job to help stabilizing CI runs. - - it gets used even if the transfer fails + This was also one of the slowest running job amongst the AppVeyor CI ones. - - it does not cause error to be returned even if it fails + Flakiness data: + https://testclutch.curl.se/static/reports/summary.html + Entries: + Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static, no-unity + (curl) [current] + Appveyor / CMake, mingw-w64, gcc 6, Debug, x86, Schannel, Static (curl) [fo + rmer] - Closes #12909 + Closes #13566 -Stefan Eissing (8 Feb 2024) +Stefan Eissing (8 May 2024) -- ftp: do lineend conversions in client writer +- unit2604: use alloc instead of overlong string const - - remove the ftp special handling from sendf.c - - let ftp_do() add a client writer that does - the linened conversions - - change the lineend conversion to no longer - modify the passed buffer, but write smaller - chunks to the next cwriter instead. The - inefficiency of this will be mitigated once - we add output buffering for all client writes. + Closes #13563 - Closes #12878 +Daniel Gustafsson (8 May 2024) -- ftp: tracing improvements +- bufq: remove duplicate word in comment - - trace socketindex for connection filters when not the first - - trace socket fd in tcp - - trace pollset adjusts in vtls + Inspired by 13552. - Closes #12902 + Closes: #13554 + Reviewed-by: Daniel Stenberg -Karthikdasari0423 (8 Feb 2024) +Viktor Szakats (8 May 2024) -- HTTP3.md: adjust the OpenSSL QUIC install instructions +- lib/cf-h1-proxy: silence compiler warnings (gcc 14) + + They came up ealier with gcc 12 (Windows), but apparently gcc 14 is + still reporting them, also under Linux. - tried installing with old steps but failed - tried with newly added setps and able to build - ``` - root@ubuntu:~/curl# ./src/curl -V - /root/curl/src/.libs/curl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSS - L_3.2.0' not found (required by /root/curl/lib/.libs/libcurl.so.4) - root@ubuntu:~/curl# - ``` ``` - root@ubuntu:~/curl# ./src/curl -V - curl 8.6.1-DEV (x86_64-pc-linux-gnu) libcurl/8.6.1-DEV OpenSSL/3.2.0 zlib/1.2 - .11 brotli/1.0.9 libpsl/0.21.0 nghttp3/1.1.0 OpenLDAP/2.5.16 - Release-Date: [unreleased] - Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns - ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp - Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz - NTLM PSL SSL threadsafe TLS-SRP UnixSockets - root@ubuntu:~/curl# + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c: In functi + on 'cf_h1_proxy_close': + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1060:17: w + arning: null pointer dereference [-Wnull-dereference] + 1060 | cf->connected = FALSE; + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:1061:8: wa + rning: null pointer dereference [-Wnull-dereference] + 1061 | if(cf->ctx) { + | ~~^~~~~ + In function 'tunnel_free', + inlined from 'cf_h1_proxy_destroy' at /home/runner/work/curl-for-win/curl + -for-win/curl/lib/cf-h1-proxy.c:1053:3: + /home/runner/work/curl-for-win/curl-for-win/curl/lib/cf-h1-proxy.c:198:27: wa + rning: null pointer dereference [-Wnull-dereference] + 198 | struct h1_tunnel_state *ts = cf->ctx; + | ^~ ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/8985369476/job/2467921 + 9528#step:3:6320 + + Fixes #13237 + Closes #13555 + +Michał Antoniak (8 May 2024) - Closes #12896 +- mbedtls: support TLS 1.3 -Daniel Stenberg (8 Feb 2024) + Closes #13539 -- TODO: align the TOC with the header +Daniel Stenberg (8 May 2024) -- docs: make sure curl.1 is included in dist tarballs +- version: use msnprintf instead of strncpy - Ref: https://github.com/curl/curl/issues/12832#issuecomment-1933271873 + - to ensure a terminating null byte + - to avoid zero-padding the target - Closes #12892 + debug code only -Karthikdasari0423 (8 Feb 2024) + Closes #13549 -- HTTP3.md: remove quiche word in Openssl 3.2 +- curl_path: make Curl_get_pathname use dynbuf - Closes #12893 + ... instead of malloc and memcpy -Daniel Stenberg (7 Feb 2024) + - unit test 2604 verifies Curl_get_pathname() -- curl: when allocating variables, add the name into the struct + Closes #13550 - This saves the name from being an extra separate allocation. +- lib: make protocol handlers store scheme name lowercase - Closes #12891 + - saves a lowercase operation when the "[scheme]_proxy" name is + generated + - appears less "shouting" + - update test 970, 972, 1438 and 1536 -- lib582: remove code causing warning that is never run + Closes #13553 - The previous realloc code in this code could trigger a compiler warning, - but since that code path cannot happen in normal circumstances it now - instead exits with an error message there. +- lib: remove two instances of "only only" messages - Ref: #12887 - Closes #12890 + Fixes #13551 + Reported-by: Lucas Nussbaum + Closes #13552 -Stefan Eissing (7 Feb 2024) +Pavel Pavlov (7 May 2024) -- vtls: revert "receive max buffer" + add test case +- asyn-thread: fix curl_global_cleanup crash in Windows - - add test_05_04 for requests using http/1.0, http/1.1 and h2 against an - Apache resource that does an unclean TLS shutdown. - - revert special workarund in openssl.c for suppressing shutdown errors - on multiplexed connections - - vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53 + - Make sure that asynchronous resolves handled by Winsock are stopped + before WSACleanup is called. - Fixes #12885 - Fixes #12844 + This is implemented by ensuring that when Curl_resolver_kill is called + (eg via multi_done) it will cancel the Winsock asynchronous resolve and + wait for the cancellation to complete. Winsock runs the asynchronous + completion routine immediately when a resolve is canceled. - Closes #12848 + Prior to this change it was possible that during curl_global_cleanup + "a DNS resolver thread created by GetAddrInfoExW did not terminate yet, + however curl is already shutting down, deinitializing Winsock with + WSACleanup() leading to an access violation." -Daniel Stenberg (7 Feb 2024) + Background: -- tests: support setting/using blank content env variables + If libcurl is built with the asynchronous threaded resolver option for + Windows then it resolves in one of two ways. For Windows 8.1 and later, + libcurl resolves by using the Winsock asynchronous resolver which does + its own thread management. For older versions of Windows, libcurl + resolves by creating a separate thread that calls getaddrinfo. This + change only affects the former and it's already handled for the latter. - - test450: remove --config from the keywords - - test2080: change return code - - test428: add --config as a keyword - - test428: disable on Windows due to CI problems + Reported-by: Ch40zz@users.noreply.github.com -- curl: exit on config file parser errors + Fixes https://github.com/curl/curl/issues/13509 + Closes https://github.com/curl/curl/pull/13518 - Like when trying to import an environment variable that does not exist. +Jay Satiro (7 May 2024) - Also fix a bug for reading env variables when there is a default value - set. +- asyn-thread: fix Curl_thread_create result check - Bug: https://curl.se/mail/archive-2024-02/0008.html - Reported-by: Brett Buddin + - Compare to curl_thread_t_null instead of 0 for error. - Add test 462 to verify. + Currently for both supported thread libraries (pthreads and Windows) + curl_thread_t_null is defined as 0. However, the pattern throughout the + code is to check against curl_thread_t_null and not 0 since for + posterity some thread library may not use 0 for error. - Closes #12862 + Closes https://github.com/curl/curl/pull/13542 -Daniel Szmulewicz (7 Feb 2024) +- curl_multibyte: remove access() function wrapper for Windows -- CURLOPT_WRITEFUNCTION.md: typo fix + - Remove curlx_win32_access() which was a wrapper to use access() in + Windows. - The maximum amount of body data that is passed to the write - callback is defined in the curl.h header file + This is a follow-up to 602fc213, one of two commits which removed + access() calls from the codebase and banned use of the function. - Closes #12889 + Closes https://github.com/curl/curl/pull/13529 -Daniel Stenberg (7 Feb 2024) +Daniel Gustafsson (6 May 2024) -- lib: convert Curl_get_line to use dynbuf +- tls: Remove EXAMPLEs from deprecated options - Create the line in a dynbuf. Aborts the reading of the file on - errors. Avoids having to always allocate maximum amount from the - start. Avoids direct malloc. + CURLOPT_EGDSOCKET and CURLOPT_RANDOM_FILE are both completely dead + so remove their example sections since the code there is useless. + There is still a way to inject a random file for OpenSSL older than + 1.1.0 but it's not what the example showed (and it's not even done + with this option) so we refrain from documenting it here. - Closes #12846 + Closes: #13540 + Reviewed-by: Daniel Stenberg -- KNOWN_BUGS: unicode on Windows +- tests: Only require EXAMPLE for non-deprecated options - Closes #11461 - Closes #12231 - Closes #12883 + Manpages which document deprecated CURLOPT_ or CURLINFO_ are not + required to have an EXAMPLE section since they might effectively + be dead no-ops which we don't want to trick users into believing + they can use by copying example code. -- tool_operate: change precedence of server Retry-After time + Closes: #13540 + Reviewed-by: Daniel Stenberg - - When calculating the retry time, no longer allow a server's requested - Retry-After time to take precedence over a longer retry time (either - default algorithmic or user-specified). +Daniel Stenberg (6 May 2024) - Prior to this change the server's Retry-After time took precedence over - curl's retry time in all cases, but that's not always practical for - short Retry-After times depending on how busy the server is. +- EXPERIMENTAL: add graduation requirements for each feature - Bug: https://curl.se/mail/archive-2024-01/0022.html - Reported-by: Dirk Hünniger + Starting now, experimental features should have a set of documentated + requirements of what is needed for the feature to graduate. - Closes https://github.com/curl/curl/pull/12871 + This adds requirements to all existing experiments. -- cmdline-docs: quote and angle bracket cleanup + Closes #13541 - - make sure angle brackets are escaped - - remove a lot of superfluous double quotes - - replace several double quotes with backticks +Ivan (6 May 2024) - To make nicer-looking markdown. +- misc: fix typos, quoting and spelling - Closes #12884 + Fix wording of comments, and misquotings where `' is markdown parsed + where it shouldn't be, and remove a misspelled preprocessor comment + which really isn't needed (and removing it makes it match surrounding + code better). -- badwords: use hostname, not host name + Closes: #13538 + Reviewed-by: Daniel Gustafsson - and username, filename - consistently. Fixed the patterns in - badwords.txt to catch these. +Daniel Gustafsson (6 May 2024) - Closes #12888 +- tests: Mark tftpd timer function as noreturn -Viktor Szakats (6 Feb 2024) + This avoids the below compiler warning: -- cmake: fix function description in comment [ci skip] + tftpd.c:280:1: warning: function 'timer' could be declared with + attribute 'noreturn' [-Wmissing-noreturn] - Closes #12879 + Closes: #13534 + Reviewed-by: Daniel Stenberg -Daniel Stenberg (6 Feb 2024) +- doh: Remove unused function prototype -- header.md: remove backslash, make nicer markdown + Closes: #13536 + Reviewed-by: Daniel Stenberg - - remove a leftover backslash before a dash - - use backticks for "code" strings +Daniel Stenberg (6 May 2024) - Closes #12877 +- doh: cleanups in ECH related functions -- docs: add mk-ca-bundle.1 to dist + - make local_decode_rdata_name use dynbuf instead of calloc + memcpy + - avoid extra memdup in local_decode_rdata_alpn + - no need to if() before free() + - use memdup instead of calloc + memcpy in Curl_doh_decode_httpsrr - ... which also makes it get built. But don't build this or curl-config.1 - if build docs is disabled. + Reviewed-by: Stephen Farrell + Closes #13526 - Closes #12875 +Viktor Szakats (5 May 2024) -Stefan Eissing (6 Feb 2024) +- libssh2: delete redundant feature guard -- https-proxy: use IP address and cert with ip in alt names + Delete `HAVE_LIBSSH2_VERSION` (equivalent to + `LIBSSH2_VERSION_NUM` > 0x010100) guard surrounding + a `LIBSSH2_VERSION_NUM` > 0x010B00 one. - - improve info logging when peer verification fails to indicate - if DNS name or ip address has been tried to match - - add test case for contacting https proxy with ip address - - add pytest env check on loaded credentials and re-issue - when they are no longer valid - - disable proxy ip address test for bearssl, since not supported there + Reviewed-by: Daniel Gustafsson + Closes #13537 - Ref: #12831 - Closes #12838 +Jan Venekamp (5 May 2024) -Jiawen Geng (6 Feb 2024) +- tool_cfgable: free {proxy_}cipher13_list on exit -- docs: add necessary setup for nghttp3 + Author: Jan Venekamp + Reviewed-by: Daniel Gustafsson + Closes: #13531 - Now nghttp3 has submodules - https://github.com/ngtcp2/nghttp3/blob/main/.gitmodules +RainRat (4 May 2024) - Closes #12859 +- doh: Fix typo in comment -Peter Krefting (6 Feb 2024) + Closes: #13504 + Author: RainRat on Github + Reviewed-by: Daniel Stenberg + Reviewed-by: Daniel Gustafsson -- version: allow building with ancient libpsl +Christian Schmitz (4 May 2024) - The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7 - ships with version 0.7.0 which lacks this API. Revert to using the older - versioning API if we detect an old libpsl version. +- dynbuf: Fix returncode on memory error - Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4 - Bug: https://curl.se/mail/archive-2024-02/0004.html - Reported-by: Scott Mutter - Closes #12872 + Curl_dyn_vaddf should return a proper error code in case allocating + memory failed. -Daniel Stenberg (6 Feb 2024) + Closes: #13533 + Author: Christian Schmitz + Reviewed-by: Daniel Gustafsson -- TODO: Support latest rustls +Daniel Stenberg (3 May 2024) - Closes #12737 - Closes #12874 +- RELEASE-NOTES: synced + +Jan Venekamp (2 May 2024) + +- bearssl: use common code for cipher suite lookup -- docs: make curldown do angle brackets like markdown + Take advantage of the Curl_cipher_suite_walk_str() and + Curl_cipher_suite_get_str() functions introduced in commit fba9afeb. - Make sure we use \< and \> in markdown all over so that it renders - correctly, on GitHub and elsewhere. cd2nroff now outputs a warning if it - finds an unescaled angle bracket. + This also fixes CURLOPT_SSL_CIPHER_LIST not working at all for bearssl + due to commit ff74cef5. - Ref: #12854 - Closes #12869 + Closes #13464 -- docs: fix the --disable-docs for autotools +Daniel Stenberg (2 May 2024) - Follow-up to 541321507e386 +- curl.h: change CURL_SSLVERSION_* from enum to defines - Closes #12870 + C++20 and later compilers emit a deprecation warning if values from two + different enums are combined with a bitwise operation the way the + CURL_SSLVERSION_* values were previously created. -- RELEASE-NOTES: synced + Reported-by: Michael Kaufmann + Fixes #13510 + Closes #13511 -- libcurl-security.md: Active FTP passes on the local IP address +- configure: error on missing perl if docs or manual is enabled - Reported-by: Harry Sintonen - Closes #12867 + Fixes #13508 + Reported-by: Harmen Stoppels + Closes #13514 -Stefan Eissing (5 Feb 2024) +- tool_cb_rea: limit rate unpause for -T . uploads -- configure: do not link with nghttp3 unless necessary + To avoid getting stuck in a busy-loop when nothing is read from stdin, + this function now checks the call rate and might enforce a short sleep + when called repeatedly without uploading anything. It is a crude + work-around to avoid a 100% busy CPU. - Fixes #12833 - Closes #12864 - Reported-by: Ryan Carsten Schmidt + Reported-by: magisterquis on hackerone + Fixes #13174 + Closes #13506 -Daniel Stenberg (5 Feb 2024) +Viktor Szakats (1 May 2024) -- THANKS: add Dmitry Tretyakov +- appveyor: enable websockets for VS2017 jobs - ... since I missed to give credit to the report in the fix of #12861 + Follow-up to eb4fe6c6340c3d5b0c347c6e30be004d4f9117d7 #13232 + Closes #13513 -Stefan Eissing (5 Feb 2024) +Daniel Stenberg (30 Apr 2024) -- openssl-quic: check on Windows that socket conv to int is possible +- if2ip: make the buf_size arg a size_t - Fixes #12861 - Closes #12865 + sizes should be size_t -Daniel Stenberg (5 Feb 2024) + Ref: #13489 + Closes #13505 -- tool_cb_hdr: only parse etag + content-disposition for 2xx +- cf-https-connect: use timeouts as unsigned ints - ... and ignore them for other response codes. + To match the type used in 'set.happy_eyeballs_timeout'. - Reported-by: Harry Sintonen - Closes #12866 + Ref: #13489 + Closes #13503 + +- hash: change 'slots' to size_t from int -- md4: include strdup.h for the memdup proto + - an unsigned type makes more sense + - size_t seems suitable + - on 64 bit args, the struct alignment makes the new Curl_hash remain + the same size - Reported-by: Erik Schnetter - Fixes #12849 - Closes #12863 + Closes #13502 -Joel Depooter (5 Feb 2024) +Viktor Szakats (30 Apr 2024) -- docs: add missing slashes to SChannel client certificate documentation +- libssh2: replace `access()` with `stat()` - When setting the CURLOPT_SSLCERT option to a certificate thumprint, it - is required to have a backslash between the "store location", "store - name" and "thumbprint" tokens. These slashes were present in the - previous documentation, but were missed in the transition to markdown - documentation. + Prefer `stat()` to verify the presence of key files. - Closes #12854 + This drops the last uses of `access()` in the codebase, which was + reported to cause issues in some cases. -Stefan Eissing (5 Feb 2024) + Also add `access()` to the list of banned functions in checksrc. -- HTTP/2: write response directly + Ref: https://github.com/curl/curl/pull/13412#issuecomment-2065505415 + Ref: https://github.com/curl/curl/pull/13482#issuecomment-2078980522 + Ref: #13497 + Co-authored-by: Jay Satiro + Closes #13498 - - use the new `Curl_xfer_write_resp()` to write incoming responses - directly to the client - - eliminates `stream->recvbuf` - - memory consumption on parallel transfers minimized +Daniel Stenberg (30 Apr 2024) - Closes #12828 +- multi: remove useless assignment -Daniel Stenberg (5 Feb 2024) + Spotted by CodeSonar -- cookie.md: provide an example sending a fixed cookie + Closes #13500 - Closes #12868 +- RELEASE-NOTES: synced -Lars Kellogg-Stedman (5 Feb 2024) +fuzzard (29 Apr 2024) -- ALTSVC.md: correct a typo +- cmake: FindNGHTTP2 add static lib name to find_library call - The ALPN documentation erroneously referred to a "host number" instead - of a "port number". + Add the static library name, nghttp2_static as a name to search. - Closes #12852 + This provides cmake parity with the winbuild Makefile.vc allowing + the cmake build to find and allow the link to static nghttp2 library. -Boris Verkhovskiy (5 Feb 2024) +Viktor Szakats (29 Apr 2024) -- proxy1.0.md: fix example +- DISTROS: add patch and issues link for curl-for-win - Closes #12856 + curl-for-win sometimes includes curl patches that were already merged in + master, but not yet part of a stable release. -Chris Webb (5 Feb 2024) + Also include the Issues link. Build-specific issues are handled there. -- configure: add --disable-docs flag + Ref: #13493 + Closes #13499 - Building man pages from curldown sources now requires perl. Add a - --disable-docs flag to configure to enable building and installing - without documentation where perl is not available or man pages are not - required. This is selected automatically (with a warning) when perl is - not found by configure. +Daniel Stenberg (29 Apr 2024) - Fixes #12832 - Closes #12857 +- mime: avoid using access() -Faraz Fallahi (5 Feb 2024) + If stat() fails, there is no point in calling access() -- connect.c: fix typo + Also: return error immediately if the stat() fails. - Closes #12858 + Ref: #13482 + Closes #13497 -Daniel Stenberg (1 Feb 2024) +Stefan Eissing (29 Apr 2024) -- sendf: ignore response body to HEAD +- tests: add SNI and peer name checks - and mark the stream for close, but return OK since the response this far - was ok - if headers were received. Partly because this is what curl has - done traditionally. + - connect to DNS names with trailing dot + - connect to DNS names with double trailing dot + - rustls, always give `peer->hostname` and let it + figure out SNI itself + - add SNI tests for ip address and localhost + - document in code and TODO that QUIC with ngtcp2+wolfssl + does not do proper peer verification of the certificate + - mbedtls, skip tests with ip address verification as not + supported by the library - Test 499 verifies. Updates test 689. + Closes #13486 - Reported-by: Sergey Bronnikov - Bug: https://curl.se/mail/lib-2024-02/0000.html - Closes #12842 +Daniel Stenberg (29 Apr 2024) -- ftp: treat a 226 arriving before data as a signal to read data +- curl_getdate.md: document two-digit year handling - For active mode transfers. + Mentioned-by: Paul Gilmartin + Ref: https://curl.se/mail/archive-2024-04/0014.html + Closes #13494 - Due to some interesting timing, curl can sometimes get the 226 (transfer - complete) over the control channel first, before the data connection - signals readability. If this happens, use that as a signal to check the - data connection. +Viktor Szakats (29 Apr 2024) - Additionally, set the socket filter in listen mode *before* the - PORT/EPRT command is issued, to reduce the risk that the little time gap - could interfere. +- cmake: add `BUILD_EXAMPLES` option to build examples - This issue never reproduced for me on Debian and takes several hundred - rounds for me to trigger on my mac. + You can enable it with `-DBUILD_EXAMPLES=ON`. - Reported-by: Stefan Eissing - Fixes #12823 - Closes #12841 + To match autotools' `make examples` feature. + Windows (static) builds not tested. -Patrick Monnerat (1 Feb 2024) + Also enable examples in a pair of CI jobs. -- OS400: avoid using awk in the build scripts + Apply related updates to the macOS CI workflow: + - drop unused `CXX` envs. + - drop no longer needed `-Wno-error=undef -Wno-error=conversion` flags. + - pass `-Wno-deprecated-declarations` to GCC too (for `BUILD_EXAMPLES`). + - document why `-Wno-deprecated-declarations` is necessary. - Awk is a PASE program and its use may cause a failure depending on the - CCSID of the calling script (IBM bug?). + Closes #13491 - For this reason, revert to an sed-only solution to extract the exported - symbols from the header files. +Stefan Eissing (26 Apr 2024) - Closes #12826 +- http3: quiche+ngtcp2 improvements -Jan Macku (1 Feb 2024) + - quiche: error transfers that try to receive on a closed + or draining connection + - ngtcp2: use callback for extending max bidi streams. This + allows more precise calculation of MAX_CONCURRENT as we + only can start a new stream when the server acknowledges + the close - not when we locally have closed it. + - remove a fprintf() from h2-download client to avoid excess + log files on tests timing out. -- docs: remove `mk-ca-bundle.1` from `man_MANS` + Closes #13475 - It was accidentally added in https://github.com/curl/curl/pull/12730 +- vtls: TLS session storage overhaul - Co-authored-by: Lukáš Zaoral - Signed-off-by: Jan Macku + - add session with destructor callback + - remove vtls `session_free` method + - let `Curl_ssl_addsessionid()` take ownership + of session object, freeing it also on failures + - change tls backend use + - test_17, add tests for SSL session resumption - Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b - Closes #12843 + Closes #13386 -Daniel Stenberg (1 Feb 2024) +- multi: multi_wait improvements -- RELEASE-NOTES: synced + - only call `multi_getsock()` once for all transfers + - realloc pollset array on demand + - fold repeated sockets - and bump to 8.6.1 for now + Closes #13150 -- cmdline-docs/Makefile: avoid using a fixed temp file name +Philip Heiduck (25 Apr 2024) - By appending the pid number two different runs at the same time will not - trample over the same file. +- ci: remove microsoft-prod.list - Reported-by: Jon Rumsey - Fixes #12829 - Closes #12839 + This is added by default, and it is often broken, but we don't need + anything from it. -- asyn-thread: use wakeup_close to close the read descriptor + Closes #13473 - Reported-by: Dan Fandrich - Ref: #12834 - Closes #12836 +Evgeny Grin (Karlson2k) (25 Apr 2024) -Stefan Eissing (1 Feb 2024) +- curl_setup.h: detect 'inline' support -- ntml_wb: fix buffer type typo + Closes #13355 - Closes #12825 +Daniel Stenberg (25 Apr 2024) -Daniel Stenberg (1 Feb 2024) +- multi: avoid memory-leak risk -- tool_operate: do not set CURLOPT_QUICK_EXIT in debug builds + 'newurl' is allocated in some conditions and used in a few scenarios, + but there were theoretical combinations in which it would not get freed. + Move the free to happen unconditionally. Never triggered by tests, but + spotted by Coverity. - Since it allows (small) memory leaks that interfere with torture tests - and regular memory-leak checks. + Closes #13471 - Reported-by: Dan Fandrich - Fixes #12834 - Closes #12835 +Johann Sebastian Schicho (25 Apr 2024) -Boris Verkhovskiy (31 Jan 2024) +- sendf: Curl_cwriter_write: remove comment disallowing zero length writes -- form-string.md: correct the example + They are needed to pass CLIENTWRITE_EOS. - Closes #12822 + Closes #13477 -Version 8.6.0 (31 Jan 2024) +Stefan Eissing (25 Apr 2024) -Daniel Stenberg (31 Jan 2024) +- CI: macos fixes for new ARM GHA images -- RELEASE-NOTES: synced + - based on #13478 with additions from #13476 + - make homebrew install path flexible + - fix OpenSSL pkgconfig files libdir + - add path to --with-libssh2 target + - disable gcc securetransport due to linker + errors (missing symbols), probably because + the os version is no longer low enough - curl 8.6.0 + Assisted-by: Viktor Szakats -- THANKS: new contributors from 8.5.0 + Closes #13479 -Jay Satiro (31 Jan 2024) +- content_encoding: ignore duplicate chunked encoding -- cd2nroff: use perl 'strict' and 'warnings' + - ignore duplicate "chunked" transfer-encodings from + a server to accomodate for broken implementations + - add test1482 and test1483 - - Use strict and warnings pragmas. + Reported-by: Mel Zuser + Fixes #13451 + Closes #13461 - - If open() fails then show the reason. +Daniel Stenberg (25 Apr 2024) - - Set STDIN io layer :crlf so that input is properly read on Windows. +- tool: move tool_ftruncate64 to tool_util.c - - When STDIN is used as input, the filename $f is now set to "STDIN". + ... and the prototype to tool_setup.h, to make them both available more + widely and accurately. - Various error messages in single() use $f for the filename and this way - it is not undefined when STDIN. + Follow-up to 00bef95946d3511 - Closes https://github.com/curl/curl/pull/12819 + Fixes #13458 + Closes #13459 -Daniel Stenberg (30 Jan 2024) +Viktor Szakats (24 Apr 2024) -- cd2nroff: fix duplicate output issue +- lib: silence `-Wsign-conversion` in base64, strcase, mprintf - Assisted-by: Jay Satiro - Fixes https://github.com/curl/curl-www/issues/321 - Closes #12818 + Closes #13467 -- lib: error out on multissl + http3 +- CI: retain failure code after `./configure` with Circle CI - Since the QUIC/h3 code has no knowledge or handling of multissl it might - bring unintended consequences if we allow it. + Suggested-by: Dan Fandrich + Follow-up to 43299e93c06b96fea8a8dc9b1c2e49c82bc21801 #13462 + Follow-up to d7332e3e46c3ef401b34e6a1a129eb4dd846c452 #12635 + Closes #13468 - configure, cmake and curl_setup.h all now reject this combination. +Daniel Stenberg (24 Apr 2024) - Assisted-by: Viktor Szakats - Assisted-by: Gisle Vanem - Ref: #12806 - Closes #12807 +- RELEASE-NOTES: synced -Patrick Monnerat (29 Jan 2024) +Jan Venekamp (24 Apr 2024) -- OS400: sync ILE/RPG binding +- mbedTLS: implement CURLOPT_SSL_CIPHER_LIST option - Also do not force git CRLF line endings on *.cmd files for OS400. + Use a lookup list to set the cipher suites, allowing the + ciphers to be set by either openssl or IANA names. - Closes #12815 + To keep the binary size of the lookup list down we compress + each entry in the cipher list down to 2 + 6 bytes using the + C preprocessor. -Viktor Szakats (28 Jan 2024) + Closes #13442 -- build: delete/replace 3 more clang warning pragmas +Viktor Szakats (24 Apr 2024) - - tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma. +- CI: show more failed `config.log` on Circle CI - - whitespace formatting in `mprintf.h`, lib518, lib537. + Show last 1000 lines of `config.log` if `./configure` fails. This was + already done for one job, this patch extends it to all. - - lib518: fix wrong variable in `sizeof()`. + Ref: #13438 + Closes #13462 - - lib518: bump variables to `rlim_t`. - Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469 +Daniel Stenberg (24 Apr 2024) - - lib518: sync error message with lib537 - Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce +- telnet: check return code from fileno() - - lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas - by reworking test code. + and return error if necessary - Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812 - Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 - Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + Spotted by CodeSonar - Reviewed-by: Daniel Stenberg - Closes #12814 + Closes #13457 -Richard Levitte (27 Jan 2024) +Viktor Szakats (24 Apr 2024) -- cmake: freshen up docs/INSTALL.cmake +- tls: fix SecureTransport + BearSSL cmake unity builds - - Turn docs/INSTALL.cmake into a proper markdown file, - docs/INSTALL-CMAKE.md - - Move things around to divide the description into configuration, - building and installing sections - - Mention the more modern cmake options to configure, build and install, - but also retain the older variants as fallbacks + Avoid clashing static function names by namespacing them. - Closes #12772 + Pointed-out-by: Jan Venekamp + Ref: https://github.com/curl/curl/pull/13442#discussion_r1576350700 + Closes #13450 -Viktor Szakats (27 Jan 2024) +Jay Satiro (24 Apr 2024) -- build: delete/replace clang warning pragmas +- dllmain: Call OpenSSL thread cleanup for Windows and Cygwin - - delete redundant warning suppressions for `-Wformat-nonliteral`. - This now relies on `CURL_PRINTF()` and it's theoratically possible - that this macro isn't active but the warning is. We're ignoring this - as a corner-case here. + - Call OPENSSL_thread_stop on thread termination (DLL_THREAD_DETACH) + to prevent a memory leak in case OpenSSL is linked statically. - - replace two pragmas with code changes to avoid the warnings. + - Warn in libcurl-thread.3 that if OpenSSL is linked statically then it + may require thread cleanup. - Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 - Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + OpenSSL may need per-thread cleanup to stop a memory leak. For Windows + and Cygwin if libcurl was built as a DLL then we can do that for the + user by calling OPENSSL_thread_stop on thread termination. However, if + libcurl was built statically then we do not have notification of thread + termination and cannot do that for the user. - Reviewed-by: Daniel Stenberg - Closes #12812 + Also, there are several other unusual cases where it may be necessary + for the user to call OPENSSL_thread_stop, so in the libcurl-thread + warning I added a link to the OpenSSL documentation. -Daniel Stenberg (27 Jan 2024) + Co-authored-by: Viktor Szakats -- RELEASE-NOTES: synced + Reported-by: southernedge@users.noreply.github.com + Reported-by: zmcx16@users.noreply.github.com -- http: only act on 101 responses when they are HTTP/1.1 + Ref: https://www.openssl.org/docs/man3.0/man3/OPENSSL_thread_stop.html#NOTES - For 101 responses claiming to be any other protocol, bail out. This - would previously trigger an assert. + Fixes https://github.com/curl/curl/issues/12327 + Closes https://github.com/curl/curl/pull/12408 - Add test 1704 to verify. +Jan Venekamp (24 Apr 2024) - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184 - Closes #12811 +- rustls: remove incorrect SSLSUPP_TLS13_CIPHERSUITES flag -Scarlett McAllister (27 Jan 2024) + The rustls backend advertises SSLSUPP_TLS13_CIPHERSUITES, but + the code does not actually seem to support it (yet?). Removed + the flag and corrected documentation. -- _VARIABLES.md: add missing 'be' into the sentence + Closes #13452 - Closes #12809 +Stefan Eissing (24 Apr 2024) -Stefan Eissing (27 Jan 2024) +- quiche: expire all active transfers on connection close -- mqtt, remove remaining use of data->state.buffer + - when a connection close is detected, all ongoing transfers + need to expire bc no more POLL events are likely to happen + for them. - Closes #12799 + Fixes #13439 + Reported-by: Jay Satiro + Closes #13447 -Daniel Stenberg (27 Jan 2024) +Dan Fandrich (23 Apr 2024) -- x509asn1: switch from malloc to dynbuf +- tests: fix feature case in test1481 - Closes #12808 + This test was being skipped everywhere because the feature never + matched. -- x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy + Closes #13445 - Closes #12808 +Gusted (23 Apr 2024) -- x509asn1: reduce malloc in Curl_extract_certinfo +- tool_operate: don't truncate the etag save file by default - Using dynbuf + This fixes a regression of 75d79a4486b279100209ddf8c7fdb12955fb66e9. The + code in tool-operate truncated the etag save file, under the assumption + that the file would be written with a new etag value. However since + 75d79a4486b279100209ddf8c7fdb12955fb66e9 that might not be the case + anymore and could result in the file being truncated when --etag-compare + and --etag-save was used and that the etag value matched with what the + server responded. Instead the truncation should not be done when a new + etag value should be written. - Closes #12808 + Test 3204 was added to verify that the file with the etag value doesn't + change the contents when used by --etag-compare and --etage-save and + that value matches with what the server returns on a non 2xx response. -Jay Satiro (27 Jan 2024) + Closes #13432 -- THANKS: add Alexander Bartel and Brennan Kinney +Abdullah Alyan (22 Apr 2024) - They reported and investigated #10259 which was fixed by 7b2d98df. +- tests: enable test 1117 for hyper - Ref: https://github.com/curl/curl/issues/10259 + Closes #13436 -Daniel Stenberg (26 Jan 2024) +Daniel Stenberg (22 Apr 2024) -- krb5: add prototype to silence clang warnings on mvsnprintf() +- sendf: useless assignment in cr_lc_read() - "error: format string is not a string literal" + Spotted by CodeSonar - Follow-up to 09230127589eccc7 which made the warning appear + Closes #13437 - Assisted-by: Viktor Szakats - Closes #12803 +- tool_paramhlp: remove duplicate assign -- x509asn1: remove code for WANT_VERIFYHOST + Spotted by CodeSonar - No code ever sets this anymore since we dropped gskit + Closes #13433 - Follow-up to 78d6232f1f326b9ab4d +- transfer: remove useless assignment - Closes #12804 + in Curl_xfer_recv_resp -- socks: reduce the buffer size to 600 (from 8K) + Spotted by CodeSonar - This is malloc'ed memory and it does not more. Test 742 helps us verify - this. + Closes #13435 - Closes #12789 +- http: acknowledge a returned error code -Stefan Eissing (26 Jan 2024) + ... and do not overwrite it with a new value that could then hide the + problem. -- file+ftp: use stack buffers instead of data->state.buffer + Spotted by CodeSonar - Closes #12789 + Closes #13434 -- vtls: receive max buffer +- tool_operate: init vars unconditionally in post_per_transfer - - do not only receive one TLS record, but try to fill - the passed buffer - - consider <4K remaning space is "filled". + In case of (the unlikely) early return, they could otherwise remain + uninitialized - Closes #12801 + Spotted by CodeSonar -Daniel Stenberg (26 Jan 2024) + Closes #13430 -- docs: do not start lines/sentences with So, But nor And +- RELEASE-NOTES: synced - Closes #12802 +- urlapi: allow setting port number zero -- docs: remove spurious ampersands from markdown + Also set and check errno when strtoul() parsing numbers for better error + checking. - They were leftovers from the nroff conversion. + Updated test 1560 - Follow-up to eefcc1bda4bccd800f5a5 + Closes #13427 - Closes #12800 +- http_aws_sigv4: remove useless assignment -Patrick Monnerat (26 Jan 2024) + This code assigned the variable the same value it already had -- sasl: make login option string override http auth + Spotted by CodeSonar - - Use http authentication mechanisms as a default, not a preset. + Closes #13426 - Consider http authentication options which are mapped to SASL options as - a default (overriding the hardcoded default mask for the protocol) that - is ignored if a login option string is given. +- file: remove useless assignment - Prior to this change, if some HTTP auth options were given, sasl mapped - http authentication options to sasl ones but merged them with the login - options. + This code assigned the variable the same value it already had. - That caused problems with the cli tool that sets the http login option - CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag - maps to more than one sasl mechanisms and the latter cannot be cleared - individually by the login options string. + Spotted by CodeSonar - New test 992 checks this. + Closes #13425 - Fixes https://github.com/curl/curl/issues/10259 - Closes https://github.com/curl/curl/pull/12790 +- test2406: verify -f with HTTP/2 -Stefan Eissing (26 Jan 2024) +Stefan Eissing (19 Apr 2024) -- socks: use own buffer instead of data->state.buffer +- http2 + ngtcp2: pass CURLcode errors from callbacks - Closes #12788 + - errors returned by Curl_xfer_write_resp() and the header variant are + not errors in the protocol. The result needs to be returned on the + next recv() from the protocol filter. -Daniel Stenberg (26 Jan 2024) + - make xfer write errors for response data cause the stream to be + cancelled -- socks: fix generic output string to say SOCKS instead of SOCKS4 + - added pytest test_02_14 and test_02_15 to verify that also for + parallel processing - ... since it was also logged for SOCKS5. + Reported-by: Laramie Leavitt + Fixes #13411 + Closes #13424 - Closes #12797 +Daniel Stenberg (19 Apr 2024) -- test742: test SOCKS5 with max length user, password and hostname +- request: make Curl_req_init return void - Adjusted the socksd server accordingly to allow for configuring that - long user name and password. + Since it could not return error and therefore this change removes dead + code for the caller. - Closes #12797 + Spotted by CodeSonar. -Stefan Eissing (25 Jan 2024) + Closes #13423 -- ssh: use stack scratch buffer for seeks +- multi: remove the unused Curl_preconnect function - - instead of data->state.buffer + The implementation has been removed, no point in keeping it around. - Closes #12794 + Follow-up to 476adfeac019ed -Daniel Stenberg (25 Jan 2024) + Closes #13422 -- krb5: access the response buffer correctly +- Curl_creader_read: init two variables to avoid using them uninited - As the pingpong code no longer uses the download buffer. + Spotted by CodeSonar - Folllow-up to c2d973627bab12ab - Pointed-out-by: Stefan Eissing - Closes #12796 + Closes #13419 -Stefan Eissing (25 Jan 2024) +- http: reject HTTP major version switch mid connection -- mqtt: use stack scratch buffer for recv+publish + A connection that has seen an HTTP major version now refuses any other + major HTTP version in future responses. Previously, a HTTP/1.x + connection would just silently accept HTTP/2 or HTTP/3 in the status + lines as long as it had support for those built-in. It would then just + lead to confusion and badness. - - instead of data->state.buffer + Indirectly Spotted by CodeSonar which identified a duplicate assignment + in this function. - Closes #12792 + Add test 471 to verify -- telnet, use stack scratch buffer for do + Closes #13421 - - instead of data->state.buffer +- mqtt: when Curl_xfer_recv returns error, don't use nread - Closes #12793 + A returned error code makes other return value unreliable, and in this + case potentially uninitialized. On error, do not read other return + values like the nread counter. -- http, use stack scratch buffer + Spotted by CodeSonar - - instead of data->state.buffer + Closes #13418 - Closes #12791 +- ftp: fix socket leak on rare error -- ntlm_wb: do not use data->state.buf any longer + In the function AcceptServerConnect() the newly created socket would + leak if Curl_conn_tcp_accepted_set() returns error. Which basically + should never happen. - Closes #12787 + Spotted by CodeSonar. -- gitignore: the generated `libcurl-symbols.md` + Closes #13417 - Closes #12795 +- urlapi: remove unused flags argument from Curl_url_set_authority -Daniel Stenberg (25 Jan 2024) + The function is only called from a single place (for HTTP/2 server push) + so might as well just assume this fixed option every time. -- tool: fix the listhelp generation command + Closes #13409 - The previous command line to generate the tool_listhelp.c source file - broke with 2494b8dd5175cee7. +- github/ISSUE_TEMPLATE: tweak the commericual support text - Make 'make listhelp' invoked in src/ generate it. Also update the - comment in the file to mention the right procedure. +- github/ISSUE_TEMPLATE: link the GitHub discussions too - Closes #12786 + ... and move the feature request line to the bottom. -- http: check for "Host:" case insensitively +- curl_url_get.md: clarify queries and fragments and CURLU_GET_EMPTY - When checking if the user wants to replace the header, the check should - be case insensitive. + Follow-up to 3eac21d86bc5 - Adding test 461 to verify + Closes #13407 - Found-by: Dan Fandrich - Ref: #12782 - Closes #12784 +Stefan Eissing (18 Apr 2024) -Tatsuhiro Tsujikawa (25 Jan 2024) +- tests: check caddy server version to match test expectations -- configure: add libngtcp2_crypto_boringssl detection + - new caddy servers no longer return 200 on POSTs, but 405 + as they should - If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested, - try to detect libngtcp2_crypto_boringssl. + Closes #13405 - Reported-by: ウさん - Fixes #12724 - Closes #12769 +Daniel Stenberg (18 Apr 2024) -Daniel Stenberg (25 Jan 2024) +- curl_url_set.md: extended -- http: remove comment reference to a removed solution + Closes #13404 - Follow-up to 58974d25d +- urlapi: add CURLU_GET_EMPTY for empty queries and fragments - Closes #12785 + By default the API inhibits empty queries and fragments extracted. + Unless this new flag is set. -Stefan Eissing (25 Jan 2024) + This also makes the behavior more consistent: without it set, zero + length queries and fragments are considered not present in the URL. With + the flag set, they are returned as a zero length strings if they were in + fact present in the URL. -- pytest: Scorecard tracking CPU and RSS + This applies when extracting the individual query and fragment + components and for the full URL. - Closes #12765 + Closes #13396 -Graham Campbell (25 Jan 2024) +- RELEASE-NOTES: synced -- GHA: bump ngtcp2, gnutls, mod_h2, quiche +- lib1560: test with leading zeroes and more IPv4 versions - - ngtcp2 to v1.2.0 - - gnutls to 3.8.3 - - mod_h2 to 2.0.26 - - quiche to 0.20.0 + Inspired by WHATWG URL Spec test inputs - Closes #12778 - Closes #12779 - Closes #12780 - Closes #12781 + Closes #13400 -Daniel Stenberg (25 Jan 2024) +Christian Schmitz (17 Apr 2024) -- ftpserver.pl: send 213 SIZE response without spurious newline +- smtp: result of Curl_bufq_cread was not used -- pingpong: stop using the download buffer + return the result back to the caller. - The pingpong logic now uses its own dynbuf for receiving command - response data. + Closes #13398 - When the "final" response header for a commanad has been received, that - final line is left first in the recvbuf for the protocols to parse at - will. If there is additional data behind the final response line, the - 'overflow' counter is indicate how many bytes. +Daniel Stenberg (17 Apr 2024) - Closes #12757 +- urlapi: fix relative redirects to fragment-only -- gen.pl: remove bold from .IP used for ## + Using the URL API for a redirect URL when the redirected-to string + starts with a hash, ie is only a fragment, the API would produce the + wrong final URL. - Reported-by: Viktor Szakats - Fixes #12776 - Closes #12777 + Adjusted test 1560 to test for several new redirect cases. -Viktor Szakats (24 Jan 2024) + Closes #13394 -- cmake: rework options to enable curl and libcurl docs +Jiwoo Park (17 Apr 2024) - Rework CMake options for building/using curl tool and libcurl manuals. +- url: fix use of an uninitialized variable - - rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning: - to build man page and built-in manual for curl tool. + Closes #13399 - - rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning: - to build man pages for libcurl. +Patrick Monnerat (17 Apr 2024) - - `BUILD_LIBCURL_DOCS` now works without having to enable - `ENABLE_CURL_MANUAL` too. +- os400: sync with latest changes - - drop support for existing CMake-level `USE_MANUAL` option to avoid - confusion. (It used to work with the effect of current - `ENABLE_CURL_MANUAL`, but only by accident.) + - Conversion support for new version info character field rtmp_version. + - New ILE/RPG declarations. - Assisted-by: Richard Levitte - Ref: #12771 - Closes #12773 + Closes #13402 -Daniel Stenberg (24 Jan 2024) +Daniel Stenberg (17 Apr 2024) -- urlapi: remove assert +- ngtcp2: fix macro use - This assert triggers wrongly when CURLU_GUESS_SCHEME and - CURLU_NO_AUTHORITY are both set and the URL is a single path. + macro "H3_STREAM_CTX" requires 2 arguments, but only 1 given - I think this assert has played out its role. It was introduced in a - rather big refactor. + Follow-up to c6655f7029ec5c128561e3ecf1f93db3ed0432a4 - Follow-up to 4cfa5bcc9a + Closes #13401 - Reported-by: promptfuzz_ on hackerone - Closes #12775 +Christian Schmitz (17 Apr 2024) -Patrick Monnerat (24 Jan 2024) +- sendf: fix two typos in comments -- tests: avoid int/size_t conversion size/sign warnings + The parameters are named data, not date. - Closes #12768 + Closes #13393 -Daniel Stenberg (24 Jan 2024) +- lib: silence warnings on comma misuse -- GHA: add a job scanning for "bad words" in markdown + Building curl with -Wcomma, I see warnings about "possible misuse of + comma operator here" and moving fields assignment out of the for() fixes + it. - This means words, phrases or things we have decided not to use - words that - are spelled right according to the dictionary but we want to avoid. In the - name of consistency and better documentation. + Closes #13392 - Closes #12764 +Stefan Eissing (17 Apr 2024) -Viktor Szakats (23 Jan 2024) +- http/2, http/3: decouple stream state from easy handle -- cmake: speed up curldown processing, enable by default + - add `Curl_hash_offt` as hashmap between a `curl_off_t` and + an object. Use this in h2+h3 connection filters to associate + `data->id` with the internal stream state. + - changed implementations of all affected connection filters + - removed `h2_ctx*` and `h3_ctx*` from `struct HTTP` and thus + the easy handle + - solves the problem of attaching "foreign protocol" easy handles + during connection shutdown - - cmake: enable `BUILD_DOCS` by default (this controls converting and - installing `.3` files from `.md` sources) + Test 1616 verifies the new hash functions. - - cmake: speed up generating `.3` files by using a single command per - directory, instead of a single command per file. This reduces external - commands by about a thousand. (There remains some CMake logic kicking - in resulting in 500 -one per file- external `-E touch_nocreate` calls.) + Closes #13204 - - cd2nroff: add ability to process multiple input files. +Daniel Stenberg (17 Apr 2024) - - cd2nroff: add `-k` option to use the source filename to form the - output filename. (instead of the default in-file `Title:` line.) +- ROADMAP: remove completed entries, mention websocket - Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c - Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753 - Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 +- THANKS-filter: name fixes - Closes #12762 +Christian Schmitz (17 Apr 2024) -Richard Levitte (23 Jan 2024) +- winbuild: add ENABLE_WEBSOCKETS option -- docs: install curl.1 with cmake as well + Closes #13232 - Closes #12759 +Daniel Stenberg (17 Apr 2024) -Daniel Stenberg (23 Jan 2024) +- dmaketgz: compacter -- osslq: remove the TLS library from the version output + Removes the need for disabling shellcheck warnings. - Since we only support using a single TLS library at any one time, we - know that the TLS library for QUIC is the same that is also shown for - regular TLS. + Follow-up to d28f74913c2 + Proposed-by: Viktor Szakats + Closes #13391 - Fixes #12763 - Reported-by: Viktor Szakats - Closes #12767 +Dan Fandrich (16 Apr 2024) -Stefan Eissing (23 Jan 2024) +- tests: Fix uninitialized value warning -- CI: remove unnecessary OpenSSL 3 option `enable-tls1_3` + The check for an option must be predicated on options existing at all. - .. and switch OpenSSL 3 libdir from lib64 to lib for consistency. + Follow-up to f7cc9e91 - Closes https://github.com/curl/curl/pull/12758 +Christian Schmitz (17 Apr 2024) -- GHA: bump nghttp2 version to v1.59.0 +- idn: add native AppleIDN (icucore) support for macOS/iOS - - Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version. + I implemented the IDN functions for macOS and iOS using Unicode + libraries coming with macOS and iOS. - Closes https://github.com/curl/curl/pull/12766 + Builds and runs here on macOS 14.2.1. Also verified to load and + run on older macOS version 10.13. -Daniel Stenberg (23 Jan 2024) + Build requires macOS SDK 13 or equivalent. -- RELEASE-NOTES: synced + Set `-DUSE_APPLE_IDN=ON` CMake option to enable it. + With autotools and other build tools, set these manual options: + ``` + CPPFLAGS=-DUSE_APPLE_IDN + LIBS=-licucore + ``` -- docs/cmdline: change to .md for cmdline docs + Completes TODO 1.6. - - switch all invidual files documenting command line options into .md, - as the documentation is now markdown-looking. + TODO: add autotools option and feature-detection. - - made the parser treat 4-space indents as quotes + Refs: #5330 #5371 + Co-authored-by: Viktor Szakats + Closes #13246 - - switch to building the curl.1 manpage using the "mainpage.idx" file, - which lists the files to include to generate it, instead of using the - previous page-footer/headers. Also, those files are now also .md - ones, using the same format. I gave them underscore prefixes to make - them sort separately: - _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md, - _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md, - _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md, - _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md +Stefan Eissing (16 Apr 2024) - - updated test cases accordingly +- http3: extend download abort tests, fixes in ngtcp2 - Closes #12751 + - fix flow handling in ngtcp2 to ACK data on streams + we abort ourself. + - extend test_02_23* cases to also run for h3 + - skip test_02_23* for OpenSSL QUIC as it gets stalled + on progressing the connection -dependabot[bot] (23 Jan 2024) + Closes #13374 -- CI: bump actions/cache from 3 to 4 +Daniel Stenberg (16 Apr 2024) - Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. - - [Release notes](https://github.com/actions/cache/releases) - - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) - - [Commits](https://github.com/actions/cache/compare/v3...v4) +- tests: add -q as first option when invoking curl for tests - --- - updated-dependencies: - - dependency-name: actions/cache - dependency-type: direct:production - update-type: version-update:semver-major - ... + To reduce the risk that the user running the tests has a .curlrc present + that messes things up. - Signed-off-by: dependabot[bot] - Closes #12756 + Support 'option="no-q"' for the tag to switch it off on demand. + Use this new feature in test 433 and 436. -Daniel Stenberg (23 Jan 2024) + Ref: #13284 + Closes #13387 -- openssl: when verifystatus fails, remove session id from cache +- dmaketgz: release tarball generation using docker - To prevent that it gets used in a subsequent transfer that skips the - verifystatus check since that check can't be done when the session id is - reused. + For easier reproducibility. - Reported-by: Hiroki Kurosawa - Closes #12760 + Mention using this script in RELEASE-PROCEDURE -Viktor Szakats (23 Jan 2024) + Closes #13388 -- cmake: add option to disable building docs +Viktor Szakats (16 Apr 2024) -Richard Levitte (23 Jan 2024) +- cmake: update ECH code and minor fixups -- cmake: use curldown to build man pages + - `openssl_check_symbol_exists()` expects a 4th argument now. + Follow-up to edc2702a1fe3a4a5386ffd9aa4f240f0c0197fa2 #13373 - This throws away the previous HTML and PDF producers, to mimic what - Makefile.am does as faithfully as possible. + - minor comment/script touch-ups. + Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 - Closes #12753 + - fix indentation. -Daniel Stenberg (23 Jan 2024) + Closes #13383 -- mksymbolsmanpage.pl: provide references to where the symbol is used +- tests: fix shellcheck issues in `ech_tests.sh` -- docs: introduce "curldown" for libcurl man page format + Add double-quotes where missing. - curldown is this new file format for libcurl man pages. It is markdown - inspired with differences: + Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 + Closes #13382 - - Each file has a set of leading headers with meta-data - - Supports a small subset of markdown - - Uses .md file extensions for editors/IDE/GitHub to treat them nicely - - Generates man pages very similar to the previous ones - - Generates man pages that still convert nicely to HTML on the website - - Detects and highlights mentions of curl symbols automatically (when - their man page section is specified) +- dist: add ECH files to tarball - tools: + Also sort `EXTRA_DIST` list in `tests/Makefile.am` and make it diffable. - - cd2nroff: converts from curldown to nroff man page - - nroff2cd: convert an (old) nroff man page to curldown - - cdall: convert many nroff pages to curldown versions - - cd2cd: verifies and updates a curldown to latest curldown + Follow-up to a362962b7289ec02b412890c9515657cf0ed50ac #11922 + Closes #13381 - This setup generates .3 versions of all the curldown versions at build time. +- openvms: look for `USE_IPV6` in `config.h` (was: `ENABLE_IPV6`) - CI: + The OpenVMS script `config_h.com` is parsing the config header + generated by autotools. Let's make it look for the macro name we now + use universally across the codebase. - Since the documentation is now technically markdown in the eyes of many - things, the CI runs many more tests and checks on this documentation, - including proselint, link checkers and tests that make sure we capitalize the - first letter after a period... + Follow-up to e411c98f702f0fb38dceec95e7507ef15a00d12c #13349 + Closes #13360 - Closes #12730 +daniel-j-h (16 Apr 2024) -Viktor Szakats (22 Jan 2024) +- Dockerfile: for release automation and reproducibility -- libssh2: use `libssh2_session_callback_set2()` with v1.11.1 + Closes #13250 - To avoid a local hack to pass function pointers and to avoid - deprecation warnings when building with libssh2 v1.11.1 or newer: - ``` - lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is depreca - ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- - declarations] - lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is depreca - ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- - declarations] - ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/2072082 - 1100#step:3:4982 +Stefan Eissing (16 Apr 2024) - Ref: https://github.com/libssh2/libssh2/pull/1285 - Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db - 3cf1d4b0b4 - Reviewed-by: Daniel Stenberg - Closes #12754 +- cw-out: improved error handling -Daniel Stenberg (22 Jan 2024) + - remember error encountered in invoking write callback and always fail + afterwards without further invokes -- transfer: make the select_bits_paused condition check both directions + - check behaviour in test_02_17 with h2-pausing client - If there is activity in a direction that is not paused, return false. + Reported-by: Pavel Kropachev + Fixes #13337 + Closes #13340 - Reported-by: Sergey Bronnikov - Bug: https://curl.se/mail/lib-2024-01/0049.html - Closes #12740 +Daniel Stenberg (16 Apr 2024) -Stefan Eissing (22 Jan 2024) +- version: add "ECH" as a feature -- http3: initial support for OpenSSL 3.2 QUIC stack + If available - - HTTP/3 for curl using OpenSSL's own QUIC stack together - with nghttp3 - - configure with `--with-openssl-quic` to enable curl to - build this. This requires the nghttp3 library - - implementation with the following restrictions: - * macOS has to use an unconnected UDP socket due to an - issue in OpenSSL's datagram implementation - See https://github.com/openssl/openssl/issues/23251 - This makes connections to non-reponsive servers hang. - * GET requests will send the indicator that they have - no body in a separate QUIC packet. This may result - in processing delays or Transfer-Encodings on proxied - requests - * uploads that encounter blocks will use 100% cpu as - detection of these flow control issue is not working - (we have not figured out to pry that from OpenSSL). + Follow-up to a362962b7 + Closes #13378 - Closes #12734 +- CURLOPT_ECH: polish -Viktor Szakats (22 Jan 2024) + - remove the pointer to build instructions, it won't work in manpages + - add see-also + - minor white space edits -- cmake: fix `ENABLE_MANUAL` option + Closes #13379 - Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`. +Viktor Szakats (16 Apr 2024) - Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the - option designed to enable building and using the built-in curl manual. - (`USE_MANUAL=ON` option worked for this instead, by accident). +- tidy-up: whitespace [ci skip] - Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409 - Closes #12749 +- mbedtls: fix building with v3 in CMake Unity mode -Mohammadreza Hendiani (19 Jan 2024) + Before this patch the internal feature detection macro + `HAS_MBEDTLS_RESULT_CODE_BASED_FUNCTIONS` was defined in three files, + with an incomplete logic in one of them. In Unity mode that spilled + into another source file and broke the build. -- TODO: update broken link to ratelimit-headers draft + Closes #13377 - Closes #12741 +- cmake: add librtmp/rtmpdump option and detection -Daniel Stenberg (19 Jan 2024) + Add CMake option `USE_LIBRTMP`. Disabled by default. -- cmake: when USE_MANUAL=YES, build the curl.1 man page + This library requires OpenSSL TLS-backend when linked statically. - Fixes KNOWN_BUG 15.4 + Follow-up to 6eb9e65781fa1fd8a0bcfe0715187a3a35f09ae4 #13364 + Closes #13373 - Closes #12742 +Stephen Farrell (16 Apr 2024) -- cmdline-opts/write-out.d: remove spurious double quotes +- TLS: add support for ECH (Encrypted Client Hello) -Stefan Eissing (19 Jan 2024) + An EXPERIMENTAL feature used with CURLOPT_ECH and --ech. -- rtsp: Convert assertion into debug log + Closes #11922 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 +Daniel Stenberg (15 Apr 2024) - - write excess bytes to the client where the standard excess bytes - checks will report any wrongness and fail the transfer +- RELEASE-NOTES: synced - Fixes #12738 - Closes #12739 +- multi: introduce SETUP state for better timeouts -Daniel Stenberg (19 Jan 2024) + Since we can go to the CONNECT state from PENDING, potentially multiple + times for a single transfer, this change introdues a SETUP state that + happens before CONNECT when doing a new transfer. -- headers: remove assert from Curl_headers_push + Now, doing a redirect on a handle goes back to SETUP (not CONNECT like + before) and we initilize the connect timeout etc in SETUP. Previously, + we would do it in CONNECT but that would make it unreliable in cases + where a transfer goes in and out between CONNECT and PENDING multiple + times. - The fuzzer managed to reach the function without a terminating CR or LF - so let's handle it normally. While there, remove the goto. + SETUP is transient, so the handle never actually stays in that state. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839 + Additionally: take care of timeouts of PENDING transfers in + curl_multi_perform() - Closes #12721 + Ref: #13227 + Closes #13371 -- curl_easy_getinfo.3: remove the wrong time value count +Tal Regev (15 Apr 2024) - It said "six" time values but they are eight by now. Remove the mention - of the amount. +- cmake: forward `USE_LIBRTMP` option to C - Closes #12727 + Define in C `USE_LIBRTMP` if user requested it from cmake. -Viktor Szakats (18 Jan 2024) + Closes #13364 -- mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls` +Daniel Stenberg (15 Apr 2024) - - Silence warning in mbedTLS v3.5.1 public headers: - ``` - ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: r - edundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls] - ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: pre - vious declaration of 'psa_set_key_domain_parameters' was here - ``` - Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae - 785a513e157c - Ref: https://github.com/libssh2/libssh2/pull/1226 +- curl_version_info: provide librtmp version - - Fix compiler warnings seen with gcc 9.2.0 + cmake unity: - ``` - ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read': - ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull - -dereference] - 189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &res - ult); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~ - ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write': - ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull - -dereference] - 168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, & - result); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~ - ``` + Ref: https://github.com/curl/curl/pull/13364#issuecomment-2054151942 + Reported-by: talregev on github + Closes #13368 - - delete stray `#else`. +blankie (15 Apr 2024) - Closes #12720 +- docs: clarify CURLOPT_MAXFILESIZE and CURLOPT_MAXFILESIZE_LARGE -Daniel Stenberg (17 Jan 2024) + The bounds of the size parameter were not specified, and nor was it + specified how to disable the maximum file size check. -- docs: cleanup nroff format use + The documentation also incorrectly stated that CURLOPT_MAXFILESIZE + always returns CURLE_OK and that CURLOPT_MAXFILESIZE_LARGE only returns + CURLE_OK or CURLE_UNKNOWN_OPTION. - - remove use of .BI for code snippet - - stop using .br, just do a blank line - - remove use of .PP - - remove use for .sp - - remove backslash in .IP - - use .IP instead of .TP + It also did not mention what the default value is, which is zero. This + commit updates the documentation to make note of all these things. - Closes #12731 + Closes #13372 -Stefan Eissing (17 Jan 2024) +Patrick Monnerat (15 Apr 2024) -- test2307: fix expected failure code after ws refactoring +- OS400: post-shellcheck changes adjustments - Fixes #12722 - Closes #12728 + Build scripts must be executed by the os/400 shell (sh), not bash which + is a PASE program. -Jay Satiro (17 Jan 2024) + Shell function get_make_vars() escaping reworked to match $() subcommand + construct. -- cf-socket: show errno in tcpkeepalive error messages + Follow-up to 8a622baf9e9233241bbe93d6599c99cb46478614 + Closes #13366 - - If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set - then show the errno in the verbose error messages. +Viktor Szakats (15 Apr 2024) - Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652 +- OS400: tidy-up - Closes https://github.com/curl/curl/pull/12726 + Drop/fixup mods trying to make some syntax highlighters happier. -- tool_getparam: stop supporting `@filename` style for --cookie + Follow-up to 8a622baf9e9233241bbe93d6599c99cb46478614 #13309 + Closes #13362 - The `@filename` style was never documented for --cookie - but prior to this change curl would accept it anyway and always treat a - @ prefixed string as a filename. +Daniel Stenberg (15 Apr 2024) - That's a problem if the string also contains a = sign because then it is - documented to be interpreted as a cookie string and not a filename. +- multi: timeout handles even without connection - Example: + When there is a "change" in a multi handle and pending handles are moved + back to the main list to be retested if they can proceed further (for + example a previous transfer completed or a connection has a confirmed + multiplexed state), the timeout check in multi_runsingle() would not + trigger because it required an established connection. - `--cookie @foo=bar` + This could make a pending tranfer go back to pending state even though + it had been "in progress" for a longer time than permitted. By removing + the requirement for an associated connection, the timeout check will be + done proper even for transfers that has not yet been assigned one. - Before: Interpreted as load cookies from filename foo=bar. + Ref #13227 + Reported-by: Rahul Krishna M + Closes #13276 - After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`). +Patrick Monnerat (15 Apr 2024) - Other curl options with a data/filename option-value use the `@filename` - to distinguish filenames which is probably how this happened. The - --cookie option has never been documented that way. +- mprintf: check fputc error rather than matching returned character - Ref: https://curl.se/docs/manpage.html#-b + OS/400 ascii fputc wrapper deviates from the posix standard by the + fact that it returns the ebcdic encoding of the original ascii + character. Testing for a matching value for success will then always + fail. - Closes https://github.com/curl/curl/pull/12645 + This commit replaces the chariacter comparison by an explicit error + return check. -Stefan Eissing (16 Jan 2024) + Follow-up to ef2cf58 + Closes #13367 -- websockets: refactor decode chain +Viktor Szakats (14 Apr 2024) - - use client writer stack for decoding frames - - move websocket protocol handler to ws.c +- ci: add CMake build variation, fixup libssh detection in `linux-old` - Closes #12713 + To test without c-ares and hit `easy_lock.h` on an old system. Use this + new build step to introduce small variations, and also test libssh2. -- websockets: check for negative payload lengths + Also add workaround to existing job to enable libssh. (CMake's generic + auto-detection doesn't seem to work here.): + ``` + CMake Warning at CMakeLists.txt:908 (find_package): + Could not find a package configuration file provided by "libssh" with any + of the following names: - - in en- and decoding, check the websocket frame payload lengths for - negative values (from curl_off_t) and error the operation in that case - - add test 2307 to verify + libsshConfig.cmake + libssh-config.cmake + ``` + Ref: https://github.com/curl/curl/actions/runs/8661316091/job/23750974358#ste + p:5:69 - Closes #12707 + Closes #13361 -Daniel Stenberg (16 Jan 2024) +- lib: merge `ENABLE_QUIC` C macro into `USE_HTTP3` -- docs: mention env vars not used by schannel + Before this patch `lib/curl_setup.h` defined these two macros right + next to each other, then the source code used them interchangeably. - Ref: #12704 + After this patch, `USE_HTTP3` guards all HTTP/3 / QUIC features. + (Like `USE_HTTP2` does for HTTP/2.) `ENABLE_QUIC` is no longer used. - Co-authored-by: Jay Satiro + This patch doesn't change the way HTTP/3 is enabled via autotools + or CMake. Builders who enabled HTTP/3 manually by defining both of + these macros via `CPPFLAGS` can now delete `-DENABLE_QUIC`. - Closes #12711 + Closes #13352 -- tool_operate: make --remove-on-error only remove "real" files +- build: prefer `USE_IPV6` macro internally (was: `ENABLE_IPV6`) - Reported-by: Harry Sintonen - Assisted-by: Dan Fandrich + Before this patch, two macros were used to guard IPv6 features in curl + sources: `ENABLE_IPV6` and `USE_IPV6`. This patch makes the source use + the latter for consistency with other similar switches. - Closes #12710 + `-DENABLE_IPV6` remains accepted for compatibility as a synonym for + `-DUSE_IPV6`, when passed to the compiler. -Jay Wu (16 Jan 2024) + `ENABLE_IPV6` also remains the name of the CMake and `Makefile.vc` + options to control this feature. -- url: don't set default CA paths for Secure Transport backend + Closes #13349 - As the default for this backend is the native CA store. +Dan Fandrich (12 Apr 2024) - Closes #12704 +- DISTROS: mark rolling release distros -Lin Sun (16 Jan 2024) + These are ones that are unlikely to have back-ported curl patches. -- asyn-ares: with modern c-ares, use its default timeout + Closes #13353 - Closes #12703 +Daniel Stenberg (12 Apr 2024) -Daniel Stenberg (15 Jan 2024) +- mbedtls: cut off trailing newlines from debug logs -- tool_operate: stop setting the file comment on Amiga + To avoid double newlines in the output. - - the URL is capped at 80 cols, which ruins it if longer - - it does not strip off URL credentials - - it is done unconditonally, not on --xattr - - we don't have Amiga in the CI which makes fixing it blindly fragile + Reported-by: Gisle Vanem + Fixes #13321 + Closes #13356 - Someone who builds and tests on Amiga can add it back correctly in a - future if there is a desire. +- RELEASE-NOTES: synced - Reported-by: Harry Sintonen - Closes #12709 +Stefan Eissing (12 Apr 2024) -Stefan Eissing (15 Jan 2024) +- CURLINFO_REQUEST_SIZE: fixed, add tests for transfer infos reported -- rtsp: deal with borked server responses + - tests for 'size_request' and other stats reported, for + presence and consistency - - enforce a response body length of 0, if the - response has no Content-lenght. This is according - to the RTSP spec. - - excess bytes in a response body are forwarded to - the client writers which will report and fail the - transfer + Reported-by: Jonatan Vela + Fixes #13269 + Closes #13275 - Follow-up to d7b6ce6 - Fixes #12701 - Closes #12706 +Viktor Szakats (11 Apr 2024) -Daniel Stenberg (14 Jan 2024) +- dist: add files missing from release tarball -- version: show only the libpsl version, not its dependencies + Closes #13346 - The libpsl version output otherwise also includes version number for its - dependencies, like IDN lib, but since libcurl does not use libpsl's IDN - functionality those components are not important. +- ci: parallelize more, tidy up cmake commands (distcheck, macos) - Ref: https://github.com/curl/curl-for-win/issues/63 - Closes #12700 + Also enable `-DCURL_WERROR=ON` in the Linux cmake build test. -Brad Harder (14 Jan 2024) + Closes #13343 -- curl.h: CURLOPT_DNS_SERVERS is only available with c-ares +Toon Claes (11 Apr 2024) - Closes #12695 +- docs: add CURLOPT_NOPROGRESS to CURLOPT_XFERINFOFUNCTION example -Daniel Stenberg (14 Jan 2024) + It's important to set `CURLOPT_NOPROGRESS` to `0` if you want your + transfer callback function, set by `CURLOPT_XFERINFOFUNCTION`, getting + called. To emphasize this to the users, add this to the code example. -- cmdline-opts/gen.pl: error on initital blank line + Closes #13348 - After the "---" separator, there should be no blank line and this script - now errors out if one is detected. +RainRat (11 Apr 2024) - Ref: #12696 - Closes #12698 +- misc: fix typos -- cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper + Closes #13344 - Follow-up to 693cd1679361828a which was incomplete +Colin Leroy-Mira (11 Apr 2024) - Ref #12680 - Closes #12697 +- file: add support for getting basic directory listings -- curl_multi_fdset.3: remove mention of null pointer support + Not supported on Windows (yet) - ... since this funtion has not supported null pointer fd_set arguments since - at least 2006. (That's when I stopped my git blame journey) + Closes #13137 - Fixes #12691 - Reported-by: sfan5 on github - Closes #12692 +Viktor Szakats (11 Apr 2024) -Mark Huang (14 Jan 2024) +- ci: add curl-for-win builds: Linux MUSL, macOS, Windows -- docs/cmdline: remove unnecessary line breaks + Linux MUSL (llvm/clang), macOS Apple clang, Windows (llvm/clang). - Closes #12696 + Configured with HTTP/2 and HTTP/3 and other dependencies (the default + curl-for-win) for a comprehensive build test. -Daniel Stenberg (14 Jan 2024) + ``` + curl 8.8.0-DEV (x86_64-unknown-linux-musl) libcurl/8.8.0-DEV LibreSSL/3.9.1 z + lib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 + ngtcp2/1.4.0 nghttp3/1.2.0 + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss + Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefil + e libz NTLM PSL SSL threadsafe UnixSockets zstd -- transfer: remove warning: Value stored to 'blen' is never read + curl 8.8.0-DEV (x86_64-apple-darwin) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1. + 3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 ngtcp + 2/1.4.0 nghttp3/1.2.0 + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws w + ss + Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefil + e libz NTLM PSL SSL threadsafe UnixSockets zstd - Detected by scan-build + curl 8.8.0-DEV (x86_64-w64-mingw32) libcurl/8.8.0-DEV LibreSSL/3.9.1 zlib/1.3 + .1 brotli/1.1.0 zstd/1.5.6 WinIDN libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.61.0 + ngtcp2/1.4.0 nghttp3/1.2.0 + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws w + ss + Features: alt-svc AsynchDNS brotli HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerb + eros Largefile libz NTLM PSL SPNEGO SSL SSPI threadsafe UnixSockets zstd + ``` - Follow-up from 1cd2f0072f + Limited to x64, because for build testing the additional CPUs don't add + much value compared to the extra build time. They can be enabled easily + if deemed useful. - Closes #12693 + To the extent of curl-for-win configuration options, it's trivial to add + further build combinations. -Stefan Eissing (13 Jan 2024) + Closes #13335 -- lib: replace readwrite with write_resp +- OS400: fix shellcheck warnings in scripts - This clarifies the handling of server responses by folding the code for - the complicated protocols into their protocol handlers. This concerns - mainly HTTP and its bastard sibling RTSP. + - use `$()` instead of backticks, and re-arrange double-quotes inside. + - add missing `|| exit 1` to `cd` calls. (could be dropped by using `set -eu` + .) + - add `-n` to a few `if`s. + - shorten redirections by using `{} >` (as shellcheck recommended). + - silence warnings where variables were detected as unused (SC2034). + - a couple misc updates to silence warnings. + - switch to bash shebang for `-ot` feature. + - split two lines to unbreak syntax highlighting in my editor. (`$(expr \`, ` + $(dirname \`) - The terms "read" and "write" are often used without clear context if - they refer to the connect or the client/application side of a - transfer. This PR uses "read/write" for operations on the client side - and "send/receive" for the connection, e.g. server side. If this is - considered useful, we can revisit renaming of further methods in another - PR. + Also enable CI checks for OS/400 shell scripts. - Curl's protocol handler `readwrite()` method been changed: + Ref: #13307 + Closes #13309 - ```diff - - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, - - const char *buf, size_t blen, - - size_t *pconsumed, bool *readmore); - + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t ble - n, - + bool is_eos, bool *done); - ``` +Stefan Eissing (11 Apr 2024) - The name was changed to clarify that this writes reponse data to the - client side. The parameter changes are: - - * `conn` removed as it always operates on `data->conn` - * `pconsumed` removed as the method needs to handle all data on success - * `readmore` removed as no longer necessary - * `is_eos` as indicator that this is the last call for the transfer - response (end-of-stream). - * `done` TRUE on return iff the transfer response is to be treated as - finished - - This change affects many files only because of updated comments in - handlers that provide no implementation. The real change is that the - HTTP protocol handlers now provide an implementation. - - The HTTP protocol handlers `write_resp()` implementation will get passed - **all** raw data of a server response for the transfer. The HTTP/1.x - formatted status and headers, as well as the undecoded response - body. `Curl_http_write_resp_hds()` is used internally to parse the - response headers and pass them on. This method is public as the RTSP - protocol handler also uses it. - - HTTP/1.1 "chunked" transport encoding is now part of the general - *content encoding* writer stack, just like other encodings. A new flag - `CLIENTWRITE_EOS` was added for the last client write. This allows - writers to verify that they are in a valid end state. The chunked - decoder will check if it indeed has seen the last chunk. - - The general response handling in `transfer.c:466` happens in function - `readwrite_data()`. This mainly operates now like: +- lib: add Curl_xfer_write_resp_hd - ``` - static CURLcode readwrite_data(data, ...) - { - do { - Curl_xfer_recv_resp(data, buf) - ... - Curl_xfer_write_resp(data, buf) - ... - } while(interested); - ... - } - ``` + Add method in protocol handlers to allow writing of a single, + 0-terminated header line. Avoids parsing and copying these lines. - All the response data handling is implemented in - `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` - implementation if available, or does the default behaviour. + Closes #13165 - All raw response data needs to pass through this function. Which also - means that anyone in possession of such data may call - `Curl_xfer_write_resp()`. +- llist: add Curl_llist_append() - Closes #12480 + - use for better readability in all places where the "insert_next" + actually performs an append to the list + - add some tests in unit1300 -Daniel Stenberg (13 Jan 2024) + Closes #13336 -- RELEASE-NOTES: synced +- gnutls: lazy init the trust settings -- TODO: TFTP doesn't convert LF to CRLF for mode=netascii + - delay loading of trust anchors and CRLs after the ClientHello + has been sent off + - add tracing to IO operations + - on IO errors, return the CURLcode of the underlying filter - Closes #12655 - Closes #12690 + Closes #13339 -- gen: do italics/bold for a range of letters, not just single word +Marcel Raad (10 Apr 2024) - Previously it would match only on a sequence of non-space, which made it - miss to highlight for example "public suffix list". +- http_negotiate: fix `CURL_DISABLE_PROXY` build - Updated the recent cookie.d edit from 5da57193b732 to use bold instead - of italics. + `proxyuserpwd` was removed from `dynamically_allocated_data` in commit + f46385d36df. - Closes #12689 + Closes https://github.com/curl/curl/pull/13334 -- docs: describe and highlight super cookies +Viktor Szakats (10 Apr 2024) - Reported-by: Yadhu Krishna M +- quic: fixup duplicate static function name (for cmake unity) - Closes #12687 + Visible in daily curl-for-win builds: + https://github.com/curl/curl-for-win/actions/runs/8621925870 -- configure: when enabling QUIC, check that TLS supports QUIC + ``` + lib/vquic/curl_ngtcp2.c:1916:12: error: redefinition of 'ossl_new_session_cb' + static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid) + ^ + lib/vtls/openssl.c:2978:12: note: previous definition is here + static int ossl_new_session_cb(SSL *ssl, SSL_SESSION *ssl_sessionid) + ^ + ``` + https://github.com/curl/curl-for-win/actions/runs/8621925870/job/23631885439# + step:3:6965 - Most importantly perhaps is when using OpenSSL that the used - build/flavor has the QUIC API: the vanilla OpenSSL does not, only - BoringSSL, libressl, AWS-LC and quictls do. + Follow-up to 3210101088dfa3d6a125d213226b092f2f866722 #13172 + Closes #13332 - Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7f - e57e#r136780413 +- appveyor: make VS2010 job build-only, enable Schannel, fix compiler warnings - Closes #12683 + Tests were consistently flaky for a while. -Stefan Eissing (11 Jan 2024) + Also fix compiler warnings in `CertOpenStore()` calls for old MSVC compilers: + ``` + C:/projects/curl/lib/vtls/schannel.c(688): + warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater s + ize + C:/projects/curl/lib/vtls/schannel_verify.c(642): + warning C4306: 'type cast' : conversion from 'int' to 'LPCSTR' of greater s + ize + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49580310/job/ywu2y44 + kymgc0nif#L106 -- vquic: extract TLS setup into own source + Closes #13330 - - separate ngtcp2 specific parts out - - provide callback during init to allow ngtcp2 to apply its defaults +Daniel Stenberg (10 Apr 2024) - Closes #12678 +- projects: drop MSVC project files for recent versions -Sergey Markelov (11 Jan 2024) + We encourage users to generate visual studio project files using CMake. -- multi: remove total timer reset in file_do() while fetching file:// + We keep project files in git for ancient visual studio versions that + cmake cannot generate files for, but we no longer ship the project files + in the tarballs. - The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts - with resetting the timer that is a start point for further multi states. - If file://, MSTATE_DO calls file_do() that should not reset the total - timer. Otherwise, the total time is always less than the pre-transfer - and the start transfer times. + appveyor: switch VisualStudioSolution job to VC12 (Visual Studio 2013) - Closes #12682 + Co-Authored-by: Viktor Szakats + Co-Authored-by: Jay Satiro -Daniel Stenberg (11 Jan 2024) + Closes #13311 -- http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT +Viktor Szakats (9 Apr 2024) - Extended test 80 to verify this. +- cmake: use namespaced custom target names - Reported-by: Stefan Eissing - Fixes #12680 - Closes #12681 + Rename custom target to namespaced (unique) names to avoid colliding + with 3rd-party projects (e.g. libzip) built together with curl. -- sectransp: do verify_cert without memdup for blobs + Reported-by: hammlee96 on github + Fixes #13324 + Closes #13326 - Since the information is then already stored in memory, this can avoid - an extra set of malloc + free calls. +- appveyor: re-enable OpenSSL 3, bump to 3.2.1 - Closes #12679 + Ref: b62454a875d70f93ab5347c050903596feb45a23 #13266 + Closes #13329 -- hsts: remove assert for zero length domain +Stefan Eissing (9 Apr 2024) - A zero length domain can happen if the HSTS parser is given invalid - input data which is not unheard of and is done by the fuzzer. +- CI: upgrade openssl version to 3.3.0 for openssl-quic - Follow-up from cfe7902111ae547873 + Closes #13328 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661 +Daniel Stenberg (9 Apr 2024) - Closes #12676 +- RELEASE-NOTES: synced -- headers: make sure the trailing newline is not stored + Bump to 8.8.0-DEV - extended test1940 to verify blank header fields too +- curl_multi_waitfds.md: add protocol mention - Bug: https://curl.se/mail/lib-2024-01/0019.html - Reported-by: Dmitry Karpov - Closes #12675 + Follow-up to 02beac6bb6b -- curl_easy_header.3: tiny language fix +Dmitry Karpov (9 Apr 2024) - Closes #12672 +- lib: add curl_multi_waitfds -- examples/range.c: add + New function call, similar to curl_multi_fdset() - Closes #12671 + Closes #13135 -- examples/netrc.c: add +Viktor Szakats (9 Apr 2024) - Closes #12671 +- dist: verify tarball reproducibility in CI -- examples/ipv6.c: new example showing IPv6-only internet transfer + Closes #13327 - Closes #12671 +Stefan Eissing (9 Apr 2024) -- examples/address-scope.c: renamed from ipv6.c +- tests: stabilitze test_02_23* - It shows address scope use really + - h2-download now always opens the output file on first write callback + invocation, if it will pause the transfer or not. + - Checks on output files then does not depend on the amount of data curl + has collected for the first write. - Closes #12671 + Closes #13323 -Stefan Eissing (9 Jan 2024) +- tls: fix compile issues on old-linux CI -- multi: pollset adjust, init with FIRSTSOCKET during connect + Follow-up to 3210101088dfa + Closes #13325 - - `conn->sockfd` is set by `Curl_setup_transfer()`, but that - is called *after* the connection has been established - - use `conn->sock[FIRSTSOCKET]` instead +Viktor Szakats (9 Apr 2024) - Follow-up to a0f94800d507de - Closes #12664 +- dist: add reproducible dir entries to tarballs -Daniel Stenberg (9 Jan 2024) + In the initial implementation of reproducible tarballs, they were + missing directory entries, while .zip archives had them. It meant + that on extracting the tarball, on-disk directory entries got the + current timestamp. -- WEBSOCKET.md: remove dead link + This patch fixes this by including directory entries in the tarball, + with reproducible timestamps. It also moves sorting inside tar, + to ensure reproducible directory entry timestamps on extract + (without the need of `--delay-directory-restore` option, when + extracting with GNU tar. BSD tar got that right by default.) -- CI: spellcheck/appveyor: invoke configure --without-libpsl + GNU tar 1.28 (2014-07-28) introduced `--sort=`. - Follow-up to 2998874bb61ac6 + Ref: https://github.com/curl/curl/pull/13299#discussion_r1555957350 + Follow-up to 860cd5fc2dc8e165fadd2c19a9b7c73b3ae5069d #13299 + Closes #13322 -- cmdline/docs/*.d: switch to using ## instead of .IP +Stefan Eissing (9 Apr 2024) - To make the editing easier. To write and to read. +- tls: use shared init code for TCP+QUIC - Closes #12667 + Closes #13172 -- gen.pl: support ## for doing .IP in table-like lists +Daniel Stenberg (9 Apr 2024) - Warn on use of .RS/.IP/.RE +- .mailmap: update Gisle's preferred email - Closes #12667 +Jan Macku (9 Apr 2024) -Jay Satiro (9 Jan 2024) +- doc: pytest `--repeat` -> `--count` -- cookie.d: Document use of empty string to enable cookie engine + Pytest doesn't have a `--repeat` option, but it does have a `--count` + option. - - Explain that --cookie "" can be used to enable the cookie engine - without reading any initial cookies. + ``` + --count=COUNT Number of times to repeat each test + ``` - As is documented in CURLOPT_COOKIEFILE. + Closes #13218 - Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html +Daniel Stenberg (9 Apr 2024) - Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420 - Reported-by: janko-js@users.noreply.github.com +- src/Makefile.am: access curl.txt using a relative path, not abs - Closes https://github.com/curl/curl/pull/12646 + ... to make it work when mounted using different mount points. Like when + generated/used inside and outside of a docker image. -Daniel Stenberg (9 Jan 2024) + Closes #13320 -- setopt: use memdup0 when cloning COPYPOSTFIELDS +- build: remove MacOSX-Framework script - Closes #12651 + I don't think this is much used these days. -- telnet: use dynbuf instad of malloc for escape buffer + Also remove the libcurl.plist file used (only) by this script - Previously, send_telnet_data() would malloc + free a buffer every time - for escaping IAC codes. Now, it reuses a dynbuf for this purpose. + Closes #13313 - Closes #12652 +- release-tools.sh: store the timestamp and release tag too -- CI: install libpsl or configure --without-libpsl in builds + When maketgz invokes this script to generate the docs/RELEASE-TOOLS.md + file that gets bundled in the release, it now also passes on the exact + timestamp and version number so that those details also get mentioned in + the document. They will help users reproduce an identical tarball. - As a follow-up to the stricted libpsl check in configure + Closes #13319 -- configure: make libpsl detection failure cause error +Viktor Szakats (8 Apr 2024) - To force users to explictily disable it if they really don't want it - used and make it harder to accidentally miss it. +- GHA: disable permissions where missing - --without-libpsl is the option to use if PSL is not wanted. + Reviewed-by: Daniel Stenberg + Closes #13306 - Closes #12661 +Stefan Eissing (8 Apr 2024) -- RELEASE-NOTES: synced +- CI: update component versions -- pop3: replace calloc + memcpy with memdup0 + - ngtcp2: v1.4.0 + - nghttp3: v1.2.0 + - nghttp2: v1.61.0 + - mod_h2: v2.0.27 - ... and make sure to return error on out of memory. + Closes #13316 - Closes #12650 +Jérôme Leclercq (8 Apr 2024) -- lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT +- CMake: check fseeko after detecting HAVE_FILE_OFFSET_BITS - Closes #12658 + Closes #13264 -- mime: use memdup0 instead of malloc + memcpy +Stefan Eissing (8 Apr 2024) - Closes #12649 +- http2: emit RST when client write fails -- tool_getparam: move the --rate logic into set_rate() + - When the writing of response data fails, reset the stream + and do not return a callback error to nghttp2. That would + be a fatal error for the connection and harm other requests. + - add test cases for various abort scenarios -- tool_getparam: switch to an enum for every option + Reported-by: Konstantin Kuzov + Fixes #13292 + Closes #13298 - To make the big switch much easier to read/understand and to make it - easier to add new options. +Kailun Qin (8 Apr 2024) -- tool_getparam: build post data using dynbuf (more) +- mbedtls: call mbedtls_ssl_setup() after RNG callback is set -- tool_getparam: replace malloc + copy by dynbuf for --data + Since mbedTLS v3.6.0, the RNG check added in ssl_conf_check() will fail + if no RNG is provided when calling mbedtls_ssl_setup(). -- tool_getparam: make data_urlencode avoid direct malloc + Therefore, mbedtls_ssl_conf_rng() needs to be called before the SSL + context is passed to mbedtls_ssl_setup(). - use aprintf() instead + Ref: https://github.com/Mbed-TLS/mbedtls/commit/b422cab052b51ec84758638d6783d + 6ba4fc60613 -- tool_getparam: move the --url-query logic into url_query() + Signed-off-by: Kailun Qin + Closes #13314 - This function is not doing post at all so it was always weirdly placed. +Daniel Stenberg (8 Apr 2024) -- tool_getparam: move the --data logic into set_data() +- NTLM_WB: drop support -- tool_getparam: unify the cmdline switch() into a single one + The feature has not worked for months and has been marked as DEPRECATED + for six+ months. - - easier to follow, easier to modify, easier to extend, possibly slightly - faster + Closes #13249 - - each case now has the long option as a comment +- curl_trc: fix build error when lacking verbose messages -- tool_getparam: bsearch cmdline options + Follow-up from 0b28ece657b2273 + Closes #13312 - - the option names are now alpha sorted and lookup is a lot faster +Viktor Szakats (8 Apr 2024) - - use case sensitive matching. It was previously case insensitive, but that - was not documented nor tested. +- contrithanks: honor `CURLWWW` variable - - remove "partial match" feature. It was not documented, not tested and - was always fragile as existing use could break when we add a new - option + Reviewed-by: Daniel Stenberg + Closes #13315 - - lookup short options via a table +- GHA: add shellcheck job and fix warnings, shell tidy-ups - Closes #12631 + Reviewed-by: Daniel Stenberg + Closes #13307 -Gabe (8 Jan 2024) +- dist: do not require Perl in `maketgz` -- COPYING: update copyright year + Perl remains required for the tarball build process. - Closes #12654 + Follow-up to 860cd5fc2dc8e165fadd2c19a9b7c73b3ae5069d #13299 -Stefan Eissing (8 Jan 2024) + Reviewed-by: Daniel Stenberg + Closes #13310 -- url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD +Daniel Stenberg (8 Apr 2024) - Also add more tracing to test 19 +- RELEASE-NOTES: synced - Follow-up to a0f9480 +- docs/cmdline-opts: invoke managen using a relative path - Fixes #12657 - Closes #12659 + ... no need to use an absolute path, that makes the build unncessarily + fail if invoked using a different mount point. managen now takes options + to find the input files. -Daniel Stenberg (8 Jan 2024) + Update test1478 to provide the dir arguments to managen -- connect: remove margin from eyeballer alloc + Closes #13281 - Presumably leftovers from debugging +- GHA: add valgrind to a wolfSSL build - Closes #12647 + Closes #13274 -- ftp: only consider entry path if it has a length +Viktor Szakats (7 Apr 2024) - Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb +- dist: `set -eu`, fix shellcheck, make reproducible and smaller tarballs - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631 + - set bash `-eu` and fix fallouts. + - fix shellcheck warnings. + - set and use `SOURCE_DATE_EPOCH` for reproducibility. + Authored-by: Daniel J. H. + Ref: #13280 + - set `TZ=UTC` and `LC_ALL=C` for reproducibility. + - make file timestamps in tarball/zip reproducible. + - make directory timestamps in zip reproducible. + - make timestamps of tarballs/zip reproducible. + - make file order in tarball/zip reproducible. + - omit extra file metadata from zip for reproducibility. + - use maximum zip compression. + - use POSIX `ustar` tarball format to avoid supply chain vulnerability: + https://seclists.org/oss-sec/2021/q4/0 + - make uid/gid in tarball reproducible. + - omit owner user/group names from tarball for reproducibility and privacy. + - omit current timestamp from .gz header for reproducibility. + - display SHA-256 hashes of produced tarballs/zip. + - fix whitespace. - Avoids a NULL pointer deref. + `.tar.gz` also became smaller in the process: 4,462,311 -> 4,148,249 bytes (8 + .7.1) - Closes #12648 + Requires GNU tar, GNU date, `sha256sum`. -Stefan Eissing (7 Jan 2024) + Reviewed-by: Daniel Stenberg + Ref: #13250 + Closes #13299 -- transfer: adjust_pollset improvements +Gisle Vanem (7 Apr 2024) - - let `multi_getsock()` initialize the pollset in what the - transfer state requires in regards to SEND/RECV - - change connection filters `adjust_pollset()` implementation - to react on the presence of POLLIN/-OUT in the pollset and - no longer check CURL_WANT_SEND/CURL_WANT_RECV - - cf-socket will no longer add POLLIN on its own - - http2 and http/3 filters will only do adjustments if the - passed pollset wants to POLLIN/OUT for the transfer on - the socket. This is similar to the HTTP/2 proxy filter - and works in stacked filters. +- tests/http: fix compiler warning - Closes #12640 + - Init result code variable to fix clang warning that it may be used + uninitialized. -Daniel Stenberg (6 Jan 2024) + Fixes https://github.com/curl/curl/issues/13301 + Closes https://github.com/curl/curl/pull/13304 -- ftp: use memdup0 to store the OS from a SYST 215 response +Stefan Eissing (6 Apr 2024) - avoid malloc + direct buffer fiddle +- vquic: use new curl_int64_t type - Closes #12639 + - add curl_int64_t signed 64-bit type for lib use -- ftp: use dynbuf to store entrypath + - define CURL_PRId64, CURL_PRIu64 format ids - avoid direct malloc + - use curl_int64_t in vquic - Closes #12638 + curl_int64_t signed complements the existing curl_uint64_t unsigned. -Lealem Amedie (6 Jan 2024) + Note that `curl_int64_t` and `int64_t` are assignable from each other + but not identical. Some platforms with 64 long type defint int64_t as + "long long" (staring at macOS) which messes up things like pointers and + format identifiers. -- wolfssl: load certificate *chain* for PEM client certs + Closes https://github.com/curl/curl/pull/13293 - Closes #12634 +Jay Satiro (5 Apr 2024) -Stefan Eissing (4 Jan 2024) +- lib: use multi instead of multi_easy for the active multi -- http: adjust_pollset fix + - Use data->multi and not data->multi_easy to refer to the active multi. - do not add a socket for POLLIN when the transfer does not want to send - (for example is paused). + The easy handle's active multi is always data->multi. - Follow-up to 47f5b1a + This is a follow up to 757dfdf which changed curl so that an easy handle + used with the easy interface and then multi interface cannot have two + different multi handles associated with it at the same time + (data->multi_easy from the easy interface and data->multi from the multi + interface). - Reported-by: bubbleguuum on github - Fixes #12632 - Closes #12633 + Closes https://github.com/curl/curl/pull/12665 -Daniel Stenberg (3 Jan 2024) +Viktor Szakats (5 Apr 2024) -- tool: make parser reject blank arguments if not supported +- tidy-up: whitespace [ci skip] - Already in the getstr() function that clones the input argument. +Daniel Stenberg (5 Apr 2024) - Closes #12620 +- makefile: remove the sorting from the vc-ide action -dependabot[bot] (3 Jan 2024) + This target generates the MSVC project files. This change removes the + extra sorting and instead makes the script use the order of the files as + listed in the variables - which are mostly sorted anyway. -- build(deps): bump github/codeql-action from 2 to 3 + This is an attempt to make the project file generation more easily + reproducible. - Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 - to 3. - - [Release notes](https://github.com/github/codeql-action/releases) - - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - - [Commits](https://github.com/github/codeql-action/compare/v2...v3) + Ref: #13250 + Closes #13294 - --- - updated-dependencies: - - dependency-name: github/codeql-action - dependency-type: direct:production - update-type: version-update:semver-major - ... +Gisle Vanem (5 Apr 2024) - Signed-off-by: dependabot[bot] +- bearssl: fix compiler warnings - Closes #12625 + "variables may be uninitialized when used" -- build(deps): bump actions/checkout from 3 to 4 + Fixes #13290 + Closes #13297 - Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. - - [Release notes](https://github.com/actions/checkout/releases) - - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - - [Commits](https://github.com/actions/checkout/compare/v3...v4) +Daniel Stenberg (5 Apr 2024) - --- - updated-dependencies: - - dependency-name: actions/checkout - dependency-type: direct:production - update-type: version-update:semver-major - ... +- DISTROS: Cygwin updates - Signed-off-by: dependabot[bot] + Brought-by: Brian Inglis + Fixes #13258 + Co-authored-by: Viktor Szakats + Closes #13279 - Closes #12624 +Stefan Eissing (5 Apr 2024) -- build(deps): bump actions/upload-artifact from 3 to 4 +- lib: add trace support for client reads and writes - Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) f - rom 3 to 4. - - [Release notes](https://github.com/actions/upload-artifact/releases) - - [Commits](https://github.com/actions/upload-artifact/compare/v3...v4) + - add `CURL_TRC_READ()` and `CURL_TRC_WRITE()` + - use in generic client writers and readers, as well + as http headers, chunking and websockets - --- - updated-dependencies: - - dependency-name: actions/upload-artifact - dependency-type: direct:production - update-type: version-update:semver-major - ... + Closes #13223 - Signed-off-by: dependabot[bot] +Michał Antoniak (5 Apr 2024) - Closes #12627 +- urldata: remove fields not used depending on used features -- build(deps): bump actions/download-artifact from 3 to 4 + Reduced size of dynamically_allocated_data structure. - Bumps [actions/download-artifact](https://github.com/actions/download-artifac - t) from 3 to 4. - - [Release notes](https://github.com/actions/download-artifact/releases) - - [Commits](https://github.com/actions/download-artifact/compare/v3...v4) + Reduced number of stored values in enum dupstring and enum dupblob. This + affects the reduced array placed in the UserDefined structure. - --- - updated-dependencies: - - dependency-name: actions/download-artifact - dependency-type: direct:production - update-type: version-update:semver-major - ... + Closes #13188 - Signed-off-by: dependabot[bot] +Viktor Szakats (5 Apr 2024) - Closes #12626 +- cmake: enable `-pedantic-errors` for clang when `CURL_WERROR=ON` -Stefan Eissing (3 Jan 2024) + clang doesn't have the issues of GCC and old CMake versions. -- http3/quiche: fix result code on a stream reset + Note: This introduces asymmetry with autotools, which only enables + this for GCC. - - fixes pytest failures in test 07_22 - - aligns CURLcode values on stream reset with ngtcp2 + Reviewed-by: Daniel Stenberg + Closes #13286 - Closes #12629 +- cmake: fix `CURL_WERROR=ON` for old CMake and use it in GHA/linux-old -Daniel Stenberg (2 Jan 2024) + - cmake: fix `-pedantic-errors` for old CMake with `CURL_WERROR=ON` set. -- setopt: clear mimepost when formp is freed + `-pedantic-errors` option throws a warning with GCC (all versions) and + makes `check_symbol_exists()` fail in CMake versions older than + v3.23.0 (2022-03-29), when CMake introduced a workaround: - A precaution to avoid a possibly dangling pointer left behind. + https://gitlab.kitware.com/cmake/cmake/-/issues/13208 + https://gitlab.kitware.com/cmake/cmake/-/commit/eeb45401163d831b8c841ef6eba + 81466b4067b68 + https://gitlab.kitware.com/cmake/cmake/-/commit/1ab7c3cd28b27ca162c4559e102 + 6e5cad1898ade - Reported-by: Thomas Ferguson - Fixes #12608 - Closes #12621 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 -Andy Alt (2 Jan 2024) + - set `CURL_WERROR=ON` for the `linux-old` job in CI. -- CI: Add dependabot.yml + Closes #13282 - This will cause dependabot to open a PR when various actions are - updated, provided that the action maintainer has issued a release. +- lib: use `#error` instead of invalid syntax in `curl_setup_once.h` - Closes #12623 + Reviewed-by: Daniel Stenberg + Closes #13287 -Gisle Vanem (2 Jan 2024) +Daniel Stenberg (5 Apr 2024) -- content_encoding: change return code to typedef'ed enum +- GHA: on macOS remove $HOME/.curlrc - ... to work around a clang ubsan warning. + A recent image upgrade added a $HOME/.curlrc by default using --ipv4. - Fixes #12618 - Closes #12622 + Ref: https://github.com/actions/runner-images/pull/9586 + Fixes #13284 + Closes #13285 -Daniel Stenberg (2 Jan 2024) +Viktor Szakats (4 Apr 2024) -- tool: prepend output_dir in header callback +- cmake: fixup `DEPENDS` filename - When Content-Disposition parsing is used and an output dir is prepended, - make sure to store that new file name correctly so that it can be used - for setting the file timestamp when --remote-time is used. + Fixing: + ``` + make[2]: Circular docs/curl-config.1 <- docs/curl-config.1 dependency dropped + . + make[2]: Circular docs/mk-ca-bundle.1 <- docs/mk-ca-bundle.1 dependency dropp + ed. + ``` + Ref: https://github.com/curl/curl/actions/runs/8559617487/job/23456740844?pr= + 13282#step:6:18 - Extended test 3012 to verify. + Follow-up to 5023ffad2c27d4b916ddb91800f99ecc5d3aad07 #13197 + Closes #13283 - Co-Authored-by: Jay Satiro - Reported-by: hgdagon on github - Fixes #12614 - Closes #12617 +- GHA: enable unity mode for cmake jobs + tidy-ups -- test1254: fix typo in name plus shorten it + Unity mode is not supported by CMake v3.7.2 used in linux-old, but + enable it anyway for consistency and to kick in automatically once + migrating to a newer old Linux in the future. -- RELEASE-NOTES: synced + Also: + - replace `CMAKE_COMPILE_WARNING_AS_ERROR` with `CURL_WERROR`. + - delete default build option `PICKY_COMPILER=ON`. -Viktor Szakats (2 Jan 2024) + Closes #13277 -- schannel: fix `-Warith-conversion` gcc 13 warning +Dan Fandrich (4 Apr 2024) - ``` - lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' - may change the sign of the result [-Warith-conversion] - 1201 | *extension_len = *list_len + - | ^ - ``` +- CI: Add CI build on Debian stretch to test old support - Closes #12616 + This version still has ELTS support and contains some old versions of + key components like cmake to help prevent us from breaking that support. -- asyn-thread: silence `-Wcast-align` warning for Windows + Closes #13029 - Seen with llvm/clang 17: - ``` - lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct - thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align] - 310 | CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlap - ped); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ~~~~ - .../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded fro - m macro 'CONTAINING_RECORD' - 717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(addre - ss) - (ULONG_PTR)(&((type *)0)->field))) - | ^~~~~~~~~~~~~~~~~~~~~~ - ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ``` +Stefan Eissing (4 Apr 2024) - Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482 +- request: paused upload on completed download, assess connection - Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261 - Closes #12615 + A transfer with a completed download that is still uploading needs to + check the connection state when it is PAUSEd, since connection + close/errors would otherwise go unnoticed. -Daniel Stenberg (2 Jan 2024) + Reported-by: Sergey Bronnikov + Fixes #13260 + Closes #13271 -- tool_listhelp: regenerate after recent .d updates +Daniel Stenberg (4 Apr 2024) - Makes it survive test 1478 +- url: do not URL decode proxy crendentials - Closes #12612 + The two options CURLOPT_PROXYUSERNAME and CURLOPT_PROXYPASSWORD set the + actual names as-is, not URL encoded. -- test1478: verify src/tool_listhelp.c + Modified test 503 to use percent-encoded strings in the credential + strings that should be passed on as-is. - Verify that the source file on disk is identical to the output of gen.pl - listhelp, as otherwise they are out of sync and need attention. + Reported-by: Sergey Ogryzkov + Fixes #13265 + Closes #13270 - Closes #12612 +Viktor Szakats (4 Apr 2024) -- testutil: make runtests support %include +- appveyor: enable cmake unity mode by default - Using this instruction, a test case can include the contents of a file - into the test during the preprocessing. + Leave one non-unity cmake job. This makes the jobs finish slightly + quicker, while giving more coverage for unity issues. - Closes #12612 + Before: + https://ci.appveyor.com/project/curlorg/curl/builds/49496977 + https://ci.appveyor.com/project/curlorg/curl/builds/49500372 + After: + https://ci.appveyor.com/project/curlorg/curl/builds/49500338 -- runtests: for mode="text" on , fix newlines on both parts + Also fixup unrelated whitespace. - Closes #12612 + Reviewed-by: Daniel Stenberg + Closes #13217 -Jay Satiro (2 Jan 2024) +Daniel Stenberg (4 Apr 2024) -- quiche: return CURLE_HTTP3 on send to invalid stream +- RELEASE-NOTES: synced - Prior to this change if a send failed on a stream in an invalid state - (according to quiche) and not marked as closed (according to libcurl) - then the send function would return CURLE_SEND_ERROR. +Viktor Szakats (4 Apr 2024) - We already have similar code for ngtcp2 to return CURLE_HTTP3 in this - case. +- cmake: speed up libcurl doc building again - Caught by test test_07_upload.py: test_07_22_upload_parallel_fail. + This time limit the number of files per command to avoid exceeding + limitations of certain OS/shell envs. - Fixes https://github.com/curl/curl/issues/12590 - Closes https://github.com/curl/curl/pull/12597 + Such known env is Windows with the `cmd.exe` shell, which features an + 8K command-line length limit to this day. -Daniel Stenberg (1 Jan 2024) + Allowlisting `UNIX` to have no limit and using a limit of 200 for other + envs to be safe. If there is a way to detect `cmd.exe` and/or we know + which precise envs are sensitive to this, we can tweak these conditions + further. -- cmdline-opts: update availability for the *-ca-native options + Even with the low limit, this patch reduces external commands by 200x, + making builds much faster. - Closes #12613 + Ref: #12762 2620aa930bc73af1e4c70b10e3125b957b96ecfb (initial) + Ref: #13047 f03c85635f35269f1f45b983bf216624f541760a (revert) -Patrick Monnerat (31 Dec 2023) + Reviewed-by: Daniel Stenberg + Closes #13207 -- openldap: fix STARTTLS +- cmake: tidy-up to use `WORKING_DIRECTORY` - It was not working anymore since introduction of connection filters. + Reviewed-by: Daniel Stenberg + Closes #13206 - Also do not attempt to recover from a failing TLS negotiation with - CURLUSESSL_TRY. +- cmake: generate misc manpages and install `mk-ca-bundle.pl` - Closes #12610 + - install `mk-ca-bundle.pl` like autotools does. -Daniel Stenberg (31 Dec 2023) + - generate and install `mk-ca-bundle.1` and `curl-config.1` like + autotools. This fixes tests 1140 and 1173. -- haproxy-clientip.d: document the arg + Reported-by: Dan Fandrich + Fixes #13194 - The arg keyword was missing and therefore not present in the man page. + - add option `BUILD_MISC_DOCS` to control building the above two + manpages. Enabled by default. - Closes #12611 + - appveyor: stop disabling tests 1140 and 1173. -annalee (29 Dec 2023) + Reviewed-by: Daniel Stenberg + Closes #13197 -- configure: fix no default int compile error in ipv6 detection +Fabian Keil (4 Apr 2024) - Closes #12607 +- wolfssl: plug memory leak in wolfssl_connect_step2() -Dan Fandrich (28 Dec 2023) + Fixes: -- CI: Fix use of any-glob-to-all-files in the labeler + test 2034...[simple HTTPS GET with DER public key pinning] + ==61829== 22,610 (3,744 direct, 18,866 indirect) bytes in 1 blocks are d + efinitely lost in loss record 51 of 54 + ==61829== at 0x484BB74: malloc (vg_replace_malloc.c:446) + ==61829== by 0x4B53A80: wolfSSL_Malloc (memory.c:344) + ==61829== by 0x4C1C8E1: wolfSSL_X509_new (x509.c:5326) + ==61829== by 0x4C3977D: d2i_X509orX509REQ (x509.c:3628) + ==61829== by 0x4C1D1F4: wolfSSL_X509_d2i (x509.c:3664) + ==61829== by 0x4C1C37B: wolfSSL_X509_dup (x509.c:13425) + ==61829== by 0x4C197DB: wolfSSL_get_peer_certificate (ssl.c:18765) + ==61829== by 0x33297C: wolfssl_connect_step2 (wolfssl.c:875) + ==61829== by 0x331669: wolfssl_connect_common (wolfssl.c:1287) + ==61829== by 0x3303E9: wolfssl_connect_nonblocking (wolfssl.c:1319) + ==61829== by 0x32FE89: ssl_connect_nonblocking (vtls.c:510) + ==61829== by 0x32DBE5: ssl_cf_connect (vtls.c:1679) + ==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307) + ==61829== by 0x27D9CF: cf_setup_connect (connect.c:1199) + ==61829== by 0x27ABD7: Curl_conn_cf_connect (cfilters.c:307) + ==61829== by 0x283CEA: cf_hc_baller_connect (cf-https-connect.c:135) - Despite its name, this atom acts like one-glob-to-all-files and a - different syntax with braces must be used to get - any-glob-to-all-files semantics. Unfortunately, this makes the file - completely unreadable. + Closes #13272 - Ref: https://github.com/actions/labeler/issues/731 +Viktor Szakats (3 Apr 2024) -Daniel Stenberg (29 Dec 2023) +- appveyor: OpenSSL 3 no longer found by CMake, revert to 1.1.1 -- CURLOPT_AUTOREFERER.3: mention CURLINFO_REFERER + OpenSSL moved directories, and bumped versions in AppVeyor CI. -- CURLINFO_REFERER.3: clarify that it is the *request* header + Downgrading is not an ideal solution, but however trivial the solution + may be, I failed to come with anything that made CMake recognize either + OpenSSL 3.1 or 3.2. - That libcurl itself sent in the most recent request + Possibly caused by: + https://github.com/appveyor/build-images/commit/702e8cdca01f28f6a40687783f493 + c786cebbe2c + https://github.com/appveyor/build-images/pull/149 - Closes #12605 + Closes #13266 -Jay Satiro (28 Dec 2023) +hongfei.li (3 Apr 2024) -- system_win32: fix a function pointer assignment warning +- winbuild: use $(RC) correctly - - Use CURLX_FUNCTION_CAST to suppress a function pointer assignment - warning. + Cloes #13267 - a6bbc87f added lookups of some Windows API functions and then cast them - like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn - about that as breaking strict-aliasing rules so this PR changes those - assignments to use CURLX_FUNCTION_CAST. +Daniel Stenberg (3 Apr 2024) - Bug: https://github.com/curl/curl/pull/12581#issuecomment-1869804317 - Reported-by: Marcel Raad +- dist: remove the curl-config.1 from the tarball - Closes https://github.com/curl/curl/pull/12602 + The markdown file is already there and the .1 file gets generated in the + build. -- verify-examples.pl: fail verification on unescaped backslash + Ref: #13250 + Closes #13268 - - Check that all backslashes in EXAMPLE are properly escaped. +- curl_global_trace.md: shorten the description - eg manpage must always use `\\n` never `\n`. + Closes #13263 - This is because the manpage requires we always double blackslash to show - a single backslash. Prior to this change an erroneous single backslash - would pass through and compile even though it would not show correctly - in the manpage. +- test1901: verify chunked POST from callback with CURLOPT_POSTFIELDSIZE set - Co-authored-by: Daniel Stenberg + Follow-up to 721941aadf4ad - Ref: https://github.com/curl/curl/pull/12588 + Ref: #13257 + Closes #13262 - Closes https://github.com/curl/curl/pull/12589 +Stefan Eissing (2 Apr 2024) -- vtls: fix missing multissl version info +- http: with chunked POST forced, disable length check on read callback - - Fix erroneous buffer copy logic from ff74cef5. + - when an application forces HTTP/1.1 chunked transfer encoding + by setting the corresponding header and instructs curl to use + the CURLOPT_READFUNCTION, disregard any POST length information. + - this establishes backward compatibility with previous curl versions - Prior to this change the MultiSSL version info returned to the user - was empty. + Applications are encouraged to not force "chunked", but rather + set length information for a POST. By setting -1, curl will + auto-select chunked on HTTP/1.1 and work properly on other HTTP + versions. - Closes https://github.com/curl/curl/pull/12599 + Reported-by: Jeff King + Fixes #13229 + Closes #13257 -Daniel Stenberg (27 Dec 2023) +Jay Satiro (1 Apr 2024) -- KNOWN_BUGS: [RTSP] Some methods do not support response bodies +- INSTALL-CMAKE.md: explain `cmake -G ` - Closes #12414 + - Explain that CMake's -G option can be used to specify which build + system to generate files for. -Patrick Monnerat (27 Dec 2023) + Example: cmake ../curl -G "MinGW Makefiles" -- openldap: fix an LDAP crash + Ref: https://github.com/curl/curl/pull/12224#issuecomment-2026813645 - Reported-by: Ozan Cansel - Fixes #12593 - Closes #12600 + Closes https://github.com/curl/curl/pull/13244 -Daniel Stenberg (27 Dec 2023) +Daniel Stenberg (1 Apr 2024) -- getinfo: CURLINFO_QUEUE_TIME_T +- libcurl-opts: mention pipelining less - Returns the time, in microseconds, during which this transfer was held - in a waiting queue before it started "for real". A transfer might be put - in a queue if after getting started, it cannot create a new connection - etc due to set conditions and limits imposed by the application. + libcurl has not supported HTTP pipelining since many years. Remove a few + (more) mentions of the feature. - Ref: #12293 - Closes #12368 + Closes #13254 -- RELEASE-NOTES: synced +Daniel McCarney (31 Mar 2024) -Jay Satiro (26 Dec 2023) +- m4: reposition USE_RUSTLS="yes" for pkg-config -- examples/sendrecv: fix comment line length + It's necessary to set this var to "yes" _after_ AC_DEFINE and AC_SUBST + in order for a later `test` to pass so that `check_for_ca_bundle=1` ends + up being set. This is in turn required for the default CA certificate + bundle to be set when building w/ rustls & pkg-config. - Caught by checksrc. + Reported-by: Matt Jolly + Fixes #13248 + Closes #13251 -Haydar Alaidrus (23 Dec 2023) +Daniel Stenberg (31 Mar 2024) -- CURLOPT_POSTFIELDS.3: fix incorrect C string escape in example +- maketgz: put docs/RELEASE-TOOL.md into the tarball - - Escape inner quotes with two backslashes. + Generated with scripts/release-tools.sh - Two backslashes escapes the backslash for the man page and will show as - a single backslash. + The script lists the exact Debian package names and version numbers for + the tools that are used to generate the tarball. - eg: "{\\"name\\": \\"daniel\\"}" shows as "{\"name\": \"daniel\"}". + Closes #13239 - Closes https://github.com/curl/curl/pull/12588 +- cd2nroff/manage: use UTC when SOURCE_DATE_EPOCH is set -Viktor Szakats (23 Dec 2023) + Make them independent of the TZ setting. Also set a date string like + YYYY-MM-DD to avoid a local month name in the date. -- appveyor: tidy-ups + Reported-by: Carlos Henrique Lima Melara + Fixes #13242 + Closes #13243 - - replace two remaining backslashes with forward slashes. - - tidy up the way we form and pass `TFLAGS`. +- RELEASE-NOTES: synced - Follow-up to 2d4d0c1fd32f5cc3f946c407c8eccd5477b287df #12572 +- docs/MAIL-ETIQUETTE: convert to markdown - Closes #12582 + To render nicer. To get spellchecked. -Stefan Eissing (22 Dec 2023) + Closes #13247 -- transfer: fix upload rate limiting, add test cases +- reuse: add copyright + license info to individual docs/*.md files - - add test cases for rate limiting uploads for all - http versions - - fix transfer loop handling of limits. Signal a re-receive - attempt only on exhausting maxloops without an EAGAIN - - fix `data->state.selectbits` forcing re-receive to also - set re-sending when transfer is doing this. + Instead of use 'docs/*.md' in dep5. For clarity and avoiding a wide- + matching wildcard. - Reported-by: Karthikdasari0423 on github - Fixes #12559 - Closes #12586 + + Remove mention of old files from .reuse/dep5 + + add info to .github/dependabot.yml + + make scripts/copyright.pl warn on non-matching patterns -Daniel Stenberg (22 Dec 2023) + Closes #13245 -- mbedtls: free the entropy when threaded +- test470: warn about unicode quote character read from config file - The entropy_free was never done for threaded builds, causing a small - (fixed) memory leak. + Idea-by: Emanuele Torre - Reported-by: RevaliQaQ on github - Fixes #12584 - Closes #12585 +- test469: verify warning when argument has unicode quote -Stefan Eissing (22 Dec 2023) +- tool_getparam: output warning for leading unicode quote character -- http2: improved on_stream_close/data_done handling + ... in the option argument. - - there seems to be a code path that cleans up easy handles without - triggering DONE or DETACH events to the connection filters. This - would explain wh nghttp2 still holds stream user data - - add GOOD check to easy handle used in on_close_callback to - prevent crashes, ASSERTs in debug builds. - - NULL the stream user data early before submitting RST - - add checks in on_stream_close() to identify UNGOOD easy handles + Typically this is a mistake done when copying example command lines from + online documentation using the wrong quote character. - Reported-by: Hans-Christian Egtvedt - Fixes #10936 - Closes #12562 + Presumably there are also other potential quote characters that might be + used, and this check is done without even knowing that unicode is used! -Daniel Stenberg (22 Dec 2023) + Reported-by: Sanjay Pujare + Fixes #13214 + Closes #13215 -- mprintf: overhaul and bugfixes +- tool: follow-up getenv fix - In a test case using lots of snprintf() calls using many commonly used - %-codes per call, this version is around 30% faster than previous - version. + Remove a double free. Change the IPFS env use to a plain getenv() simply + because coverity gets confused. - It also fixes the #12561 bug which made it not behave correctly when - given unknown %-sequences. Fixing that flaw required a different take on - the problem, which resulted in the new two-arrays model. + Follow-up to 9126b141c9398fe + Closes #13241 - lib557: extended - Verify the #12561 fix and test more printf features +- idn: make Curl_idnconvert_hostname() use Curl_idn_decode() - unit1398: fix test: It used a $ only for one argument, which is not - supported. + In the name of less code duplication - Fixes #12561 - Closes #12563 + Closes #13236 -Viktor Szakats (21 Dec 2023) +- curl-confopts.m4: define CARES_NO_DEPRECATED when c-ares is used -- appveyor: replace PowerShell with bash + parallel autotools + Starting in 1.28.0 c-ares added deprecation warnings for some API calls + libcurl uses. - PowerShell works (after a steep development curve), but one property of - it stuck and kept causing unresolvable usability issues: With - `$ErrorActionPreference=Stop`, it does abort on failures, but shows only - the first line of the error message. In `Continue` mode, it shows the - full error message, but doesn't stop on all errors. Another issue is - PowerShell considering any stderr output as if the command failed (this - has been improved in 7.2 (2021-Nov), but fixed versions aren't running - in CI and will not be for a long time in all test images.) + Closes #13240 - Thus, we're going with bash. +- vquic: use CURL_FORMAT_CURL_OFF_T for 64 bit printf output - Also: - - use `-j2` with autotools tests, making them finish 5-15 minutes per - job faster. - - omit `POSIX_PATH_PREFIX`. - - use `WINDIR`. - - prefer forward slashes. + Reported-by: Keitagit-kun on github + Fixes #13224 + Closes #13231 - Follow-up to: 75078a415d9c769419aed4153d3d525a8eba95af #11999 - Ref: #12444 +- openldap: create ldap URLs correctly for IPv6 addresses - Fixes #12560 - Closes #12572 + Reported-by: Sergio Durigan Junior + Fixes #13228 + Closes #13235 -Pavel Pavlov (21 Dec 2023) +- curl: use curl_getenv instead of the curlx_ version -- asyn-thread: use GetAddrInfoExW on >= Windows 8 + The curlx one was once introduced when we still considered dropping the + libcurl function at some point. To reduce confusion and to make it + easier to understand when curl_free() should be used, use the actual + libcurl function call directly instead. - For doing async DNS resolution instead of starting a thread for each - request. + Closes #13230 - Fixes #12481 - Closes #12482 +Evgeny Grin (Karlson2k) (30 Mar 2024) -Daniel Stenberg (21 Dec 2023) +- curl_sha512_256: do not use workaround for NetBSD when not needed -- strerror: repair get_winsock_error() + Assisted-by: riastradh on github + Assisted-by: Michael Kaufmann + Closes #13225 - It would try to read longer than the provided string and crash. +Matt Jolly (30 Mar 2024) - Follow-up to ff74cef5d4a0cf60106517a1c7384 - Reported-by: calvin2021y on github - Fixes #12578 - Closes #12579 +- m4: fix rustls pkg-config codepath -- CURLOPT_SSH_*_KEYFILE: clarify + The previous pkg-config code would successfully detect rustls but did + not set all appropriate variables and call the right macros to properly + configure cURL. - Closes #12554 + Reported-by: kpcyrd on github + Fixes #13200 + Closes #13202 -ivanfywang (21 Dec 2023) +Daniel McCarney (30 Mar 2024) -- ngtcp2: put h3 at the front of alpn +- deps: update librustls 0.12.0 -> 0.13.0 - Closes #12576 + This commit updates the optional rustls-ffi librustls dependency from + 0.12.0 to 0.13.0. This version is based on the latest available rustls + release (0.23.4). -Daniel Stenberg (21 Dec 2023) + The breaking API changes from 0.12.0 to 0.13.0 are in API surface unused + by curl, so this is an in-place update without any code changes. -- test460: verify a command line using --expand with no argument + The `RUSTLS.md` documentation is updated to reflect the new version in + use, and to clarify that `cbindgen` isn't required to build `librustls` + - it's only used by developers to update the vendored `rustls.h` header + file maintained upstream. - This verifies the fix for #12565 + Closes #13238 -- tool_getparam: do not try to expand without an argument +Daniel Stenberg (28 Mar 2024) - This would lead to a segfault. +- RELEASE-NOTES: synced - Fixes #12565 - Reported-by: Geeknik Labs - Closes #12575 +- tool_xattr: "guess" URL scheme if none is provided -- RELEASE-NOTES: synced + ... when figuring out the source URL to store. - Bumped version to 8.6.0 because of changes + Reported-by: Dagfinn Ilmari Mannsåker + Fixes #13205 + Closes #13221 -- Makefile.am: fix the MSVC project generation +- tool_xattr: in debug builds, act normally if CURL_FAKE_XATTR is not set - It made the vcxproj files not get included in dist tarballs. + Closes #13220 - Regression since 74423b5df4c8117891eb89 (8.5.0) +Stefan Eissing (28 Mar 2024) - Reported-by: iAroc on github - Fixes #12564 - Closes #12567 +- content_encoding: brotli and others, pass through 0-length writes -zengwei2000 (21 Dec 2023) + - curl's transfer handling may write 0-length chunks at the end of the + download with an EOS flag. (HTTP/2 does this commonly) -- altsvc: free 'as' when returning error + - content encoders need to pass-through such a write and not count this + as error in case they are finished decoding - Closes #12570 + Fixes #13209 + Fixes #13212 + Closes #13219 - Signed-off-by: zengwei +Tobias Stoeckmann (28 Mar 2024) -Viktor Szakats (20 Dec 2023) +- libssh2: set length to 0 if strdup failed -- build: fix `-Wconversion`/`-Wsign-conversion` warnings + Internally, libssh2 dereferences the NULL pointer if length is non-zero. + The callback function cannot return the error condition, so at least + prevent subsequent crash. - Fix remaining warnings in examples and tests which are not suppressed - by the pragma in `lib/curl_setup.h`. + Closes #13213 - Silence a toolchain issue causing warnings in `FD_SET()` calls with - older Cygwin/MSYS2 builds. Likely fixed on 2020-08-03 by: - https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=5717262b8ecfed0f7f - ab63e2c09c78991e36f9dd +Daniel Stenberg (28 Mar 2024) - Follow-up to 2dbe75bd7f3c36837aa06fd87a442bdf3fb7faef #12492 +- RELEASE-PROCEDURE: mention an initial working build - Closes #12557 + This is the step that was not done and caused the 8.7.0 mishap (it + lacked the correctly generated hugehelp file). -- build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings + Remove the mention of the copyright script as this is verified by a CI + job these days: the REUSE one. - - enable `-Wsign-conversion` warnings, but also setting them to not - raise errors. - - fix `-Warith-conversion` warnings seen in CI. - These are triggered by `-Wsign-converion` and causing errors unless - explicitly silenced. It makes more sense to fix them, there just a few - of them. - - fix some `-Wsign-conversion` warnings. - - hide `-Wsign-conversion` warnings with a `#pragma`. - - add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build - basis. - - update a CI job to unhide them with the above macro: - https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3 + Closes #13216 - Closes #12492 +Paul Howarth (28 Mar 2024) -- cmake: tidy-up `OtherTests.cmake` +- curl_sha512_255: fix detection of OpenSSL 1.1.1 or later - - make more obvious which detection uses which prep steps. - - merge and streamline conditions. - - these should not alter detection results. + Use the same OPENSSL_VERSION_NUMBER comparison as in lib/vtls/openssl.c. - Also align log output messages from - `Macros.cmake` / `curl_internal_test` with rest of the build. + Closes #13208 - Closes #12551 +Robert Moreton (28 Mar 2024) -- appveyor: switch to out-of-tree builds +- cf-socket: remove references to l_ip, l_port - With cmake and autotools. + Fixes #13210 + Closes #13211 - Closes #12550 +Daniel Stenberg (28 Mar 2024) -Daniel Stenberg (19 Dec 2023) +- openssl: do not set SSL_MODE_RELEASE_BUFFERS -- DEPRECATE.md: mention that NTLM_WB no longer works + While it might save some memory, it causes OpenSSL to instead do a huge + amount of allocations. - Ref: #12479 - Closes #12553 + Ref: #13136 + Closes #13203 -- CURLOPT_SERVER_RESPONSE_TIMEOUT_MS: add +- curl: make --help adapt to the terminal width - Proposed-by: Yifei Kong - Ref: https://curl.se/mail/lib-2023-11/0023.html - Closes #12369 + Instead of assuming and working with 80 colums, try figuring out what + width is actually used. -Viktor Szakats (18 Dec 2023) + Ref: #13141 -- build: more `-Wformat` fixes + Closes #13171 - - memdebug: update to not trigger `-Wformat-nonliteral` warnings. - - imap: mark `imap_sendf()` with `CURL_PRINTF()`. - - tool_msgs: mark static function with `CURL_PRINTF()`. +- RELEASE-NOTES: synced - Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 + and bump to 8.7.2 for now - Closes #12540 +- configure: make --disable-docs imply --disable-manual -- windows: delete redundant headers + Because when the docs is not built, the necesary curl.txt file is not + present so then the manual cannot get built. - `winsock2.h` pulls in `windows.h`. `ws2tcpip.h` pulls in `winsock2.h`. - `winsock2.h` and `ws2tcpip.h` are also pulled by `curl/curl.h`. + Reported-by: Harry Sintonen + Closes #13191 - Keep only those headers that are not already included, or the code under - it uses something from that specific header. +Chris Webb (27 Mar 2024) - Closes #12539 +- cmdline-docs: fix make install with configure --disable-docs -- cmake: prefill/cache `HAVE_STRUCT_SOCKADDR_STORAGE` + make -C docs/cmdline-opts install depends on all-am, which in turn + depends on $(MANS), unconditionally defined to be $(man_MANS). - Also add missing include to `OtherTests.cmake`. It didn't cause an issue - because the parent already included this earlier by chance. + As with CLEANFILES, only add curl.1 to man_MANS when BUILD_DOCS is true + so we don't try to build curl.1 unnecessarily. - Closes #12537 + Closes #13198 -Daniel Stenberg (18 Dec 2023) +Version 8.7.1 (27 Mar 2024) -- runner.pm: fix perl warning when running tests +Daniel Stenberg (27 Mar 2024) - Use of uninitialized value $runner::gdbthis in numeric eq (==) at runner. - pm +- RELEASE-PROCEDURE: remove old release dates, add new pending ones - Follow-up from 3dcf301752a09d9 +Version 8.7.0 (27 Mar 2024) - Closes #12549 +Daniel Stenberg (27 Mar 2024) -- runtests: support -gl. Like -g but for lldb. +- RELEASE-NOTES: synced - Follow-up to 63b5748 + curl 8.7.0 release - Invokes the test case via lldb instead of gdb. Since using gdb is such a - pain on mac, using lldb is sometimes less quirky. +- THANKS: new contributors from the 8.7.0 release - Closes #12547 +- CURLOPT_POSTFIELDS.md: used for MQTT as well -- curl.h: add CURLE_TOO_LARGE + Closes #13189 - A new error code to be used when an internal field grows too large, like - when a dynbuf reaches its maximum. Previously it would return - CURLE_OUT_OF_MEMORY for this, which is highly misleading. +- http: remove stale comment about rewindbeforesend - Ref: #12268 - Closes #12269 + ... because that struct field exists no more. -- CI/circleci: disable MQTT in the HTTP-only build + Follow-up to 14bcea074a782272. - And remove the use of configure options that don't actually exist + Closes #13187 - Closes #12546 +- DISTROS: add document with distro pointers -Yedaya Katsman (18 Dec 2023) + Lots of organizations distribute curl packages to end users. This is a + collection of pointers to where to learn more about curl on and with + each distro. -- tests: respect $TMPDIR when creating unix domain sockets + Assisted-by: Alan Coopersmith + Assisted-by: Andrew Kaster + Assisted-by: Andy Fiddaman + Assisted-by: Arjan van de Ven + Assisted-by: Brian Clemens + Assisted-by: chrysos349 on github + Assisted-by: Dan Fandrich + Assisted-by: Dan McDonald + Assisted-by: Gaelan Steele + Assisted-by: graywolf on github + Assisted-by: Jan Macku + Assisted-by: John Marshall + Assisted-by: Jonathan Perkin + Assisted-by: Kevin Daudt + Assisted-by: Marcus Müller + Assisted-by: Michał Górny + Assisted-by: Outvi V + Assisted-by: Ross Burton + Assisted-by: Sean Molenaar + Assisted-by: Till Wegmüller + Assisted-by: Viktor Szakats + Assisted-by: Winni Neessen - When running on termux, where $TMPDIR isn't /tmp, running the tests - failed, since the server config tried creating sockets in /tmp, without - checking the temp dir config. Use the TMPDIR variable that makes it find - the correct directory everywhere [0] + Closes #13178 - [0] https://perldoc.perl.org/File::Temp#tempfile +Fabian Keil (25 Mar 2024) - Closes #12545 +- wolfSSL: do not call the stub function wolfSSL_BIO_set_init() -Viktor Szakats (17 Dec 2023) + Calling the function isn't necessary and causes the build + to fail when wolfSSL has been compiled with NO_WOLFSSL_STUB: -- ssh: fix namespace of two local macros + Making all in opts + CCLD curl + ld: error: undefined symbol: wolfSSL_BIO_set_init + >>> referenced by wolfssl.c:235 (vtls/wolfssl.c:235) + >>> libcurl_la-wolfssl.o:(wolfssl_bio_cf_create) in archiv + e ../lib/.libs/libcurl.a + cc: error: linker command failed with exit code 1 (use -v to see invocat + ion) + *** Error code 1 - Avoid using the libssh and libssh2 macro namespaces by prefixing - these local macro names with `CURL_`. + Closes #13164 - Follow-up to 413a0fedd02c8c6df1d294534b8c6e306fcca7a2 #12346 +Daniel Stenberg (25 Mar 2024) - Reviewed-by: Daniel Stenberg - Closes #12544 +- cmdline-opts: shorter help texts -- cmake: whitespace tidy-up in `OtherTests.cmake` + In an effort to increase the readability of the "--help all" output on + narrow (80 column) terminals. - Closes #12538 + Co-authored-by: Jay Satiro -Mark Sinkovics (16 Dec 2023) + Closes #13169 -- cmake: fix generation for system name iOS +Matt Jolly (25 Mar 2024) - This PR fixes a problem that happens during CMake configuration when - the `CMAKE_SYSTEM_NAME` set to `iOS` and not `Darwin`. This value is - available (as far as I remember) version 3.14. The final solution - (thanks to @vszakats) is to use `APPLE` which contains all the Apple - platforms https://cmake.org/cmake/help/latest/variable/APPLE.html. +- curl-rustls.m4: add pkg-config support to rustls detection - This issue was found when during vcpkg installation. Running command - `vcpkg install curl:arm64-ios` and `vcpkg install curl:x64-ios` failed - with message: - ``` - CMake Error: try_run() invoked in cross-compiling mode, please set the follow - ing cache variables appropriately: - HAVE_H_ERRNO_ASSIGNABLE_EXITCODE (advanced) - ``` - After this fix, I was able to compile the compile the binary without - any issue. + Based on the existing openssl pkg-config detection, this commit tries to + use pkg-config to find `rustls` then falls back to the current approach + if that fails. - In addition to that fix, this PR also contains an simplification to - check if the platform is not APPLE. + We use the following logic: - Co-authored-by: Viktor Szakats - Closes #12515 + - if no path is provided, just use pkg-config, if it's not there we have + a problem! + - if a path is provided, try pkg-config + + if pkg-config fails, try and find rustls directly -Daniel Stenberg (16 Dec 2023) + Closes #13179 -- RELEASE-NOTES: synced +Mohammadreza Hendiani (25 Mar 2024) -Baruch Siach (16 Dec 2023) +- TODO: update 13.11 with more information -- gnutls: fix build with --disable-verbose + Closes #13173 - infof() parameters must be defined event with --disable-verbose since - commit dac293cfb702 ("lib: apache style infof and trace - macros/functions"). +Daniel Stenberg (23 Mar 2024) - Move also 'ptr' definition under !CURL_DISABLE_VERBOSE_STRINGS. +- docs/libcurl: generate PROTOCOLS from meta-data - Fixes the following build failure: + Remove the PROTOCOLS section from the source files completely and + instead generate them based on the header data in the curldown files. - In file included from ../lib/sendf.h:29, - from vtls/gtls.c:44: - vtls/gtls.c: In function 'Curl_gtls_verifyserver': - vtls/gtls.c:841:34: error: 'version' undeclared (first use in this function); - did you mean 'session'? - 841 | gnutls_protocol_get_name(version), ptr); - | ^~~~~~~ + It also generates TLS backend information for options marked for TLS as + protocol. - Closes #12505 + Closes #13175 -Viktor Szakats (16 Dec 2023) +- CURLMOPT_MAX*: mention what happens if changed mid-transfer -- build: delete unused `HAVE_{GSSHEIMDAL,GSSMIT,HEIMDAL}` + For CURLMOPT_MAXCONNECTS and CURLMOPT_MAX_HOST_CONNECTIONS - Stop setting `HAVE_GSSHEIMDAL`, `HAVE_GSSMIT` and `HAVE_HEIMDAL`. - There was no place in the build system or source code that used them. + Ref: #13158 + Closes #13176 - Reviewed-by: Daniel Stenberg - Closes #12506 +- docs/libcurl: add TLS backend info for all TLS options -- build: remove redundant `CURL_PULL_*` settings + All man pages that are listed to be for TLS now must also specify + exactly what TLS backends the option works for, or use All if they all + work. - These macros were not propagated to the source code from CMake. + cd2nroff makes sure this is done and that the listed backends exist. - autotools set only one of them (`CURL_PULL_SYS_POLL_H`), initially to - address an AIX issue [1]. This later broke when introducing `system.h` - [2] without the logic it enabled. A subsequent fix [3] re-added the - logic, and also enabled it for AIX before its use, directly in - `system.h`. + Closes #13168 - [1] 2012-11-23: 665adcd4b7bcdb7deb638cdc499fbe71f8d777f2 - [2] 2017-03-29: 9506d01ee50d5908138ebad0fd9fbd39b66bd64d #1373 - [3] 2017-08-25: 8a84fcc4b59e8b78d2acc6febf44a43d6bc81b59 #1828 #1833 +- docs/libcurl: cleanups - Reviewed-by: Daniel Stenberg - Closes #12502 + - CURLINFO_TLS_SESSION.md: remove mention of NSS + - CURLINFO_TLS_SSL_PTR.md: remove NSS leftover + - CURLOPT_CAINFO.md: drop mention of backends not supporting this + - CURLOPT_CAPATH.md: wolfSSL also supports this -- system.h: sync mingw `CURL_TYPEOF_CURL_SOCKLEN_T` with other compilers + Closes #13166 - Align mingw with the other Windows compilers and use the `int` type for - `CURL_TYPEOF_CURL_SOCKLEN_T` (and thus for `curl_socklent_t`). This - makes it unnecessary to make a mingw-specific trick and pull all Windows - headers early just for this type definition. This type is specific to - Windows, not to the compiler. mingw-w64's Windows header maps it to - `int` too. +- docs: make each libcurl man specify protocol(s) - With this we also delete all remaining uses of `CURL_PULL_WS2TCPIP_H`. + The mandatory header now has a mandatory list of protocols for which the + manpage is relevant. - [ The official solution is to use `socklen_t` for all Windows compilers. - In this case we may want to update `curl/curl.h` to pull in Windows - headers before `system.h`. ] + Most man pages already has a "PROTOCOLS" section, but this introduces a + stricter way to specify the relevant protocols. - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12501 - -- windows: simplify detecting and using system headers - - - autotools, cmake: assume that if we detect Windows, `windows.h`, - `winsock2.h` and `ws2tcpip.h` do exist. - - lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of - looking for `winsock2.h`. - - autotools: merge 3 Windows check methods into one. - - move Watt-32 and lwIP socket support to `setup-win32.h` from - `config-win32.h`. It opens up using these with all build tools. Also - merge logic with Windows Sockets. - - fix to assume Windows sockets with the mingw32ce toolchain. - Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4 - - cmake: delete unused variable `signature_call_conv` since - eb33ccd5332435fa50f1758e5debb869c6942b7f. - - autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection. - - examples/externalsocket: fix header order. - - cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue` - that wasn't used anymore. - - cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test - `SIZEOF_STRUCT_SOCKADDR_STORAGE`. - - After this patch curl universally uses `_WIN32` to guard - Windows-specific logic. It guards Windows Sockets-specific logic with - `USE_WINSOCK` (this might need further work). - - Reviewed-by: Jay Satiro - Closes #12495 - -- build: enable missing OpenSSF-recommended warnings, with fixes - - https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening - -Guide-for-C-and-C++.html - as of 2023-11-29 [1]. - - Enable new recommended warnings (except `-Wsign-conversion`): - - - enable `-Wformat=2` for clang (in both cmake and autotools). - - add `CURL_PRINTF()` internal attribute and mark functions accepting - printf arguments with it. This is a copy of existing - `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible - with redefinting the `printf` symbol: - https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 - - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for - mingw-w64 and enable it on this platform. - - enable `-Wimplicit-fallthrough`. - - enable `-Wtrampolines`. - - add `-Wsign-conversion` commented with a FIXME. - - cmake: enable `-pedantic-errors` the way we do it with autotools. - Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 - - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format - checks. Previously it was always disabled due to the internal `printf` - macro. - - Fix them: - - - fix bug where an `set_ipv6_v6only()` call was missed in builds with - `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. - - add internal `FALLTHROUGH()` macro. - - replace obsolete fall-through comments with `FALLTHROUGH()`. - - fix fallthrough markups: Delete redundant ones (showing up as - warnings in most cases). Add missing ones. Fix indentation. - - silence `-Wformat-nonliteral` warnings with llvm/clang. - - fix one `-Wformat-nonliteral` warning. - - fix new `-Wformat` and `-Wformat-security` warnings. - - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its - definition to `lib/curl_setup.h` allowing use in `tests/server`. - - lib: fix two wrongly passed string arguments in log outputs. - Co-authored-by: Jay Satiro - - fix new `-Wformat` warnings on mingw-w64. - - [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895b - fc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options- - Hardening-Guide-for-C-and-C%2B%2B.md - - Closes #12489 - -- Makefile.mk: drop Windows support - - And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga. - - We recommend CMake instead. With unity mode it's much faster, and about - the same without. - - Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 - Reviewed-by: Daniel Stenberg - Closes #12224 + cd2nroff verifies that at least one protocol is mentioned (which can be + `*`). -Daniel Stenberg (16 Dec 2023) + This information is not used just yet, but A) the PROTOCOLS section can + now instead get generated and get a unified wording across all manpages + and B) this allows us to more reliably filter/search for protocol + specific manpages/options. -- cmdline-docs: use .IP consistently + Closes #13166 - Remove use of .TP and some .B. The idea is to reduce nroff syntax as - much as possible and to use it consistently. Ultimately, we should be - able to introduce our own easier-to-use-and-read syntax/formatting and - convert on generation time. +Stefan Eissing (21 Mar 2024) - Closes #12535 +- http2, http3: only return CURLE_PARTIAL_FILE when bytes were received -Tatsuhiko Miyagawa (16 Dec 2023) + - should resolve spurious pytest failures when stream were reset + right after response header were received -- http: fix off-by-one error in request method length check + Clsoes #13151 - It should allow one more byte. +- http: separate response parsing from response action - Closes #12534 + - move code that triggers on end-of-response into separate function from + parsing + - simplify some headp/headerlen usage + - add `httpversion` to SingleRequest to indicate the version of the + current response -Daniel Stenberg (15 Dec 2023) + Closes #13134 -- curl: show ipfs and ipns as supported "protocols" +Daniel Stenberg (21 Mar 2024) - They are accepted schemes in URLs passed to curl (the tool, not the - library). +- http2: remove the third (unused) argument from http2_data_done() - Also makes curl-config show the same list. + Closes #13154 - Co-Authored-by: Jay Satiro - Reported-by: Chara White - Bug: https://curl.se/mail/archive-2023-12/0026.html - Closes #12508 +- RELEASE-NOTES: synced -- Revert "urldata: move async resolver state from easy handle to connectdata" +Evgeny Grin (Karlson2k) (21 Mar 2024) - This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198) +- RELEASE-NOTES: corrected - We want the c-ares channel to be held in the easy handle, not per - connection - for performance. + Corrected link for item 118 - Closes #12524 + Closes #13157 -Viktor Szakats (15 Dec 2023) +Daniel Stenberg (19 Mar 2024) -- openssl: re-match LibreSSL deinit with init +- CURLOPT_INTERFACE.md: remove spurious amp, add see-also - Earlier we switched to use modern initialization with LibreSSL v2.7.0 - and up, but did not touch deinitialization [1]. Fix it in this patch. + Closes #13149 - Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611 +Stefan Eissing (19 Mar 2024) - [1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014 +- http: improve response header handling, save cpu cycles - Reported-by: Mike Hommey - Reviewed-by: Daniel Stenberg - Fixes #12525 - Closes #12526 + Saving some cpu cycles in http response header processing: + - pass the length of the header line along + - use string constant sizeof() instead of strlen() + - check line length if prefix is possible + - switch on first header char to limit checks -Daniel Stenberg (14 Dec 2023) + Closes #13143 -- libssh: supress warnings without version check +Daniel Stenberg (19 Mar 2024) - Define unconditionally. +- tool_getparam: accept a blank -w "" - Follow-up from d21bd2190c46ad7fa + Added test 468 to verify. - Closes #12523 + Regression from 07bcae89d5d00 (shipped in 8.6.0) + Reported-by: Thomas Pyle + Fixes #13144 + Closes #13145 -- hostip: return error immediately when Curl_ip2addr() fails +Evgeny Grin (Karlson2k) (18 Mar 2024) - Closes #12522 +- curl_sha512_256: work around a NetBSD bug -Theo (14 Dec 2023) + Based on Michael Kaufmann analysis and suggestion -- libssh: improve the deprecation warning dismissal + Closes #13133 - Previous code was compiler dependant, and dismissed all deprecation warnings - indiscriminately. +Stefan Eissing (18 Mar 2024) - libssh provides a way to disable the deprecation warnings for libssh only, an - d - naturally this is the preferred way. +- http: expect 100 rework - This commit uses that, to prevent the erroneous hiding of potential, unrelate - d - deprecation warnings. + Move all handling of HTTP's `Expect: 100-continue` feature into a client + reader. Add sending flag `KEEP_SEND_TIMED` that triggers transfer + sending on general events like a timer. - Fixes #12519 - Closes #12520 + HTTP installs a `CURL_CR_PROTOCOL` reader when announcing `Expect: + 100-continue`. That reader works as follows: -Daniel Stenberg (14 Dec 2023) + - on first invocation, records time, starts the `EXPIRE_100_TIMEOUT` + timer, disables `KEEP_SEND`, enables `KEEP_SEND_TIMER` and returns 0, + eos=FALSE like a paused upload. -- test1474: removed + - on subsequent invocation it checks if the timer has expired. If so, it + enables `KEEP_SEND` and switches to passing through reads to the + underlying readers. - The test was already somewhat flaky and disabled on several platforms, - and after 1da640abb688 even more unstable. + Transfer handling's `readwrite()` will be invoked when a timer expires + (like `EXPIRE_100_TIMEOUT`) or when data from the server arrives. Seeing + `KEEP_SEND_TIMER`, it will try to upload more data, which triggers + reading from the client readers again. Which then may lead to a new + pausing or cause the upload to start. -- readwrite_data: loop less + Flags and timestamps connected to this have been moved from + `SingleRequest` into the reader's context. - This function is made to loop in order to drain incoming data - faster. Completely removing the loop has a measerably negative impact on - transfer speeds. + Closes #13110 - Downsides with the looping include +- mbedtls: fix pytest for newer versions - - it might call the progress callback much more seldom. Especially if - the write callback is slow. + Fix the expectations in pytest for newer versions of mbedtls - - rate limiting becomes less exact + Closes #13132 - - a single transfer might "starve out" other parallel transfers +Daniel Stenberg (15 Mar 2024) - - QUIC timers for other connections can't be maintained correctly +- ipv6.md: mention IPv4 mapped addresses - The long term fix should be to remove the loop and optimize coming back - to avoid the transfer speed penalty. + Reported-by: Josh Soref + Assisted-by: Jay Satiro + Fixes #13112 + Closes #13131 - This fix lower the max loop count to reduce the starvation problem, and - avoids the loop completely for when rate-limiting is in progress. +Stefan Eissing (15 Mar 2024) - Ref: #12488 - Ref: https://curl.se/mail/lib-2023-12/0012.html - Closes #12504 +- http: revisit http_perhapsrewind() -Stefan Eissing (14 Dec 2023) + - use facilities provided by client readers better + - work also for non-uploading requests like GET/HEAD + - update documentation -- lib: eliminate `conn->cselect_bits` + Closes #13117 - - use `data->state.dselect_bits` everywhere instead - - remove `bool *comeback` parameter as non-zero - `data->state.dselect_bits` will indicate that IO is - incomplete. +- test 1541: verify getinfo values on first header callback - Closes #12512 + Reported-by: chensong1211 on github + Ref: #13125 + Closes #13128 -- connect: refactor `Curl_timeleft()` +- TLS: start shutdown only when peer did not already close - - less local vars, "better" readability - - added documentation + - When curl sees a TCP close from the peer, do not start a TLS shutdown. + TLS shutdown is a handshake and if the peer already closed the + connection, it is not interested in participating. - Closes #12518 + Reported-by: dfdity on github + Assisted-by: Jiří Bok + Assisted-by: Pēteris Caune + Fixes #10290 + Closes #13087 -Dmitry Karpov (14 Dec 2023) +Daniel Stenberg (14 Mar 2024) -- cookie: avoid fopen with empty file name +- RELEASE-NOTES: synced - Closes #12514 +- curl: make --libcurl output better CURLOPT_*SSLVERSION -Viktor Szakats (13 Dec 2023) + The option is really two enums ORed together, so it needs special + attention to make the code output nice. -- tests/server: delete workaround for old-mingw + Added test 1481 to verify. Both the server and the proxy versions. - mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this. + Reported-by: Boris Verkhovskiy + Fixes #13127 + Closes #13129 - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 +- GHA/linux: add sysctl trick to work-around GitHub runner issue - Reviewed-by: Jay Satiro - Closes #12510 + The GitHub image runner update from 20240304.1.0 to 20240310.1 + introduces a problem for clang-14. The issue is caused by + incompatibility between llvm 14 provided in ubuntu-22.04 image and the + much newer kernel configured with high-entropy ASLR. -- cmake: delete obsolete TODOs more [ci skip] + As a work-around, we issue a sysctl command to lower the entropy and get + clang-14 to work again. - - manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288 - - soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023 - - bunch of others that are completed - - `NTLM_WB_ENABLED` is implemented in a basic form, and now also - scheduled for removal, so a TODO at this point isn't useful. + URL: https://github.com/actions/runner-images/issues/9491 - And this 'to-check' item: + Closes #13124 - Q: "The cmake build selected to run gcc with -fPIC on my box while the - plain configure script did not." +- SPONSORS: describe the basics - A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 - and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly - enable PIC for libcurl shared lib. Or when building libcurl for - shared and static lib in a single pass. We do this by default for - Windows or when enabled by the user via `SHARE_LIB_OBJECT`. - Otherwise we don't touch this setting. Meaning the default set by - CMake (if any) or the toolchain is used. On Debian Bookworm, this - means that PIC is disabled for static libs by default. Some platforms - (like macOS), has PIC enabled by default. - autotools supports the double-pass mode only, and in that case - CMake seems to match PIC behaviour now (as tested on Linux with gcc.) + Closes #13119 - Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500 +- GOVERNANCE: document the core team - Reviewed-by: Jay Satiro - Closes #12509 + Closes #13118 -Stefan Eissing (12 Dec 2023) +Jay Satiro (13 Mar 2024) -- CLIENT-WRITERS: design and use documentation +- vquic-tls: fix the error code returned for bad CA file - Closes #12507 + - Return CURLE_SSL_CACERT_BADFILE if wolfSSL encounters a problem + reading the cert file or path. -Viktor Szakats (12 Dec 2023) + This is a follow-up to the parent commit aedbbdf1. -- cmake: delete obsolete TODO items [ci skip] + Reported-by: Karthikdasari0423@users.noreply.github.com - There is always room for improvement, but CMake is up to par now with - autotools, so there is no longer a good reason to keep around these - inline TODO items. + Fixes https://github.com/curl/curl/issues/13115 - Answering one of questions: +Daniel Stenberg (12 Mar 2024) - Q: "The gcc command line use neither -g nor any -O options. As a - developer, I also treasure our configure scripts's --enable-debug - option that sets a long range of "picky" compiler options." +- vquic-tls: return appropirate errors on wolfSSL errors - A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info - and optimization level. E.g.: - - `Release` = `-O3` + no debug info - - `MinSizeRel` = `-Os` + no debug info - - `Debug` = `-O0` + debug info + Reported-by: Dexter Gerig + Closes #13107 - https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-deb - ug-release-relwithdebinfo-and-minsizerel/59314670#59314670 - https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#defaul - t-and-custom-configurations +Viktor Szakats (12 Mar 2024) - For picky warnings we have the `PICKY_COMPILER` options, enabled by - default. +- tidy-up: one comment and EOF newlines - Closes #12500 + Reviewed-by: Daniel Stenberg + Closes #13108 -Stefan Eissing (11 Dec 2023) +Daniel Stenberg (12 Mar 2024) -- CONNECTION-FILTERS: update documentation +- cmdline-opts: language cleanups - Closes #12497 + Use imperative mood consistently for the first sentence describing an + option. -Daniel Stenberg (11 Dec 2023) + "Set this" instead "tell curl to set" or "this sets..." -- lib: reduce use of strncpy + Plus some extra cleanups and rephrasing. - - bearssl: select cipher without buffer copies - - http_aws_sigv4: avoid strncpy, require exact timestamp length - - http_aws_sigv4: use memcpy isntead of strncpy - - openssl: avoid strncpy calls - - schannel: check for 1.3 algos without buffer copies - - strerror: avoid strncpy calls - - telnet: avoid strncpy, return error on too long inputs - - vtls: avoid strncpy in multissl_version() + Closes #13106 - Closes #12499 +- managen: remove space before protocols -- CI/distcheck: run full tests + For options that are listed for specific protocols, the protocols (shown + first within parentheses) are now output without the leading space in the + manpage output. - To be able to detect missing files better, this now runs the full CI - test suite. If done before, it would have detected #12462 before - release. + Closes #13105 - Closes #12503 +Jay Satiro (12 Mar 2024) -- docs: clean up Protocols: for cmdline options +- mbedtls: properly cleanup the thread-shared entropy - ... and some other minor polish. + - Store the state of the thread-shared entropy for global init/cleanup. - Closes #12496 + - Use curl's thread support of mbedtls for all Windows builds instead of + just when the threaded resolver is used via USE_THREADS_WIN32. -- cmdline/gen: fix the sorting of the man page options + Prior to this change on global cleanup curl builds that have curl thread + support for mbedtls freed the entropy (8b1d2298) but failed to mark that + it had been freed, which caused problems on subsequent init + transfer. - They were previously sorted based on the file names, which use a .d - extension, making "data" get placed after "data-binary" etc. Making the - sort ignore the extention fixes the ordering. + Bug: https://github.com/curl/curl/discussions/11919#discussioncomment-8687105 + Reported-by: awesomekosm@users.noreply.github.com - Reported-by: Boris Verkhovskiy - Bug: https://curl.se/mail/archive-2023-12/0014.html - Closes #12494 + Closes https://github.com/curl/curl/pull/13071 -Daniel Gustafsson (9 Dec 2023) +Daniel Stenberg (12 Mar 2024) -- doh: remove unused local variable +- tool_getparam: handle non-existing (out of range) short-options - The nurl variable is no longer used during probing following - a refactoring, so remove. + ... correctly, even when they follow an existing one without a space in + between. - Closes #12491 + Verify with test 467 -Jay Satiro (8 Dec 2023) + Follow-up to 07dd60c05b + Reported-by: Geeknik Labs + Fixes #13101 + Closes #13102 -- build: fix Windows ADDRESS_FAMILY detection +Stefan Eissing (11 Mar 2024) - - Include winsock2.h for Windows ADDRESS_FAMILY detection. +- lib: move 'done' parameter to SingleRequests - Prior to this change cmake detection didn't work because it included - ws2def.h by itself, which is missing needed types from winsock2.h. + A transfer may do several `SingleRequest`s for its success. This happens + regularly for authentication, follows and retries on failed connections. + The "readwrite()" calls and functions connected to those carried a `bool + *done` parameter to indicate that the current `SingleRequest` is over. + This may happen before `upload_done` or `download_done` bits of + `SingleRequest` are set. - Prior to this change autotools detection didn't work because it did not - include any Windows header. + The problem with that is now `write_resp()` protocol handlers are + invoked in places where the `bool *done` cannot be passed up to the + caller. Instead of being a bool in the call chain, it needs to become a + member of `SingleRequest`, reflecting its state. - In both cases libcurl would fall back on unsigned short as the address - family type, which is the same as ADDRESS_FAMILY. + This removes the `bool *done` parameter and adds the `done` bit to + `SingleRequest` instead. It adds `Curl_req_soft_reset()` for using a + `SingleRequest` in a follow up, clearing `done` and other + flags/counters. - Co-authored-by: Viktor Szakats + Closes #13096 - Closes https://github.com/curl/curl/pull/12441 +- request: clarify message when request has been sent off -Daniel Stenberg (8 Dec 2023) + Change the "uploaded and fine" message for requests without a body -- lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding + Reported-by: Karthikdasari0423 on github + Fixes #13093 + Closes #13095 - Since the copy does not stop at a null byte, let's not call it anything - that makes you think it works like the common strndup() function. +Daniel Stenberg (11 Mar 2024) - Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat +- RELEASE-NOTES: synced - Closes #12490 +Stefan Eissing (9 Mar 2024) -- convsrctest.pl: removed: not used, not shipped in tarballs +- lib: keep conn IP information together -- tests: rename tests scripts to the test number + new struct ip_quadruple for holding local/remote addr+port - It is hard to name the scripts sensibly. Lots of them are similarly - named and the name did not tell which test that used them. + - used in data->info and conn and cf-socket.c + - copy back and forth complete struct + - add 'secondary' to conn + - use secondary in reporting success for ftp 2nd connection - The new approach is rather to name them based on the test number that - runs them. Also helps us see which scripts are for individual tests - rather than for general test infra. + Reported-by: DasKutti on github + Fixes #13084 + Closes #13090 - - badsymbols.pl -> test1167.pl - - check-deprecated.pl -> test1222.pl - - check-translatable-options.pl -> test1544.pl - - disable-scan.pl -> test1165.pl - - error-codes.pl -> test1175.pl - - errorcodes.pl -> test1477.pl - - extern-scan.pl -> test1135.pl - - manpage-scan.pl -> test1139.pl - - manpage-syntax.pl -> test1173.pl - - markdown-uppercase.pl -> test1275.pl - - mem-include-scan.pl -> test1132.pl - - nroff-scan.pl -> test1140.pl - - option-check.pl -> test1276.pl - - options-scan.pl -> test971.pl - - symbol-scan.pl -> test1119.pl - - version-scan.pl -> test1177.pl +Daniel Stenberg (8 Mar 2024) - Closes #12487 +- scripts/managen: the new name and home for the manpage generator -Michał Antoniak (8 Dec 2023) + It was previously docs/cmdline-opts/gen.pl -- sendf: fix compiler warning with CURL_DISABLE_HEADERS_API + Closes #13089 - fix MSVC warning C4189: 'htype': local variable is initialized but not - referenced - when CURL_DISABLE_HEADERS_API is defined. +- VULN-DISCLOSURE-POLICY.md: update detail about CVE requests - Closes #12485 + curl is a CNA now -Viktor Szakats (8 Dec 2023) + Closes #13088 -- tidy-up: whitespace +Stefan Eissing (8 Mar 2024) - Closes #12484 +- lib: client reader polish -Stefan Eissing (7 Dec 2023) + - seek_func/seek_client, use transfer values only + - remove copies held in `struct connectdata`, use only + ever `data->set.seek_func` + - resolves possible issues in multiuse connections + - new mime post reader eliminates need to ever overwriting this -- test_02_download: fix paramters to test_02_27 + - websockets, remove empty Curl_ws_done() function - - it is a special client that only ever uses http/2 + Closes #13079 - Closes #12467 +Marcel Raad (8 Mar 2024) -Michał Antoniak (7 Dec 2023) +- lib1598: fix `CURLOPT_POSTFIELDSIZE` usage -- vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY + It requires a `long` argument. - Closes #12459 + Closes https://github.com/curl/curl/pull/13085 -Daniel Stenberg (7 Dec 2023) +Daniel Stenberg (8 Mar 2024) -- lib: strndup/memdup instead of malloc, memcpy and null-terminate +- docs/cmdline-opts: drop the curl.1 from the dist tarball - - bufref: use strndup - - cookie: use strndup - - formdata: use strndup - - ftp: use strndup - - gtls: use aprintf instead of malloc + strcpy * 2 - - http: use strndup - - mbedtls: use strndup - - md4: use memdup - - ntlm: use memdup - - ntlm_sspi: use strndup - - pingpong: use memdup - - rtsp: use strndup instead of malloc, memcpy and null-terminate - - sectransp: use strndup - - socks_gssapi.c: use memdup - - vtls: use dynbuf instead of malloc, snprintf and memcpy - - vtls: use strdup instead of malloc + memcpy - - wolfssh: use strndup + Since it is no longer needed for building tool_hugehelp.c and all the + docs is available in readable markdown format in the tarball, the peeps + that don't want to build the manpage still do good. - Closes #12453 + Removing it also fixes the complexity of out-of-tree builds when the + curl.1 exists in the source tree. -- strdup: remove the memchr check from Curl_strndup +- test1140/1173: extend wildcards to find curl.1 - It makes it possible to clone a binary chunk of data. + ... in its new build path. - Closes #12453 + Also update the test scripts to be more precise in error messages to + help us understand CI errors better. -- ftp: handle the PORT parsing without allocation + Follow-up to f03c85635f35269f1 + Ref: #13029 + Closes #13083 - Also reduces amount of *cpy() calls. +- http2: minor tweaks to optimize two struct sizes - Closes #12456 + - use BIT() instead of bool + - place the struct fields in (roughly) size order -- RELEASE-NOTES: synced + Closes #13082 - Bumped to 8.5.1 +- buildconf.bat: remove outdated groff/nroff use -- url: for disabled protocols, mention if found in redirect + - don't try to generate the real hugehelp file, because it requires + curl.txt which needs a build + - don't attempt to do anything in a c-ares subdirectory - To help users better understand where the URL (and denied scheme) comes - from. Also removed "in libcurl" from the message, since the disabling - can be done by the application. + Follow-up to f03c85635f35269 + Closes #13078 - The error message now says "not supported" or "disabled" depending on - why it was denied: +- http2: memory errors in the push callbacks are fatal - Protocol "hej" not supported - Protocol "http" disabled + Use the correct nghttp2 error code accordingly. - And in redirects: + Closes #13081 - Protocol "hej" not supported (in redirect) - Protocol "http" disabled (in redirect) +Viktor Szakats (7 Mar 2024) - Reported-by: Mauricio Scheffer - Fixes #12465 - Closes #12469 +- mkhelp: rename variable to fix compiler warnings -Stefan Eissing (6 Dec 2023) + ``` + src\tool_operate.c(541,33): warning C4459: declaration of 'm' hides global de + claration [_bld\src\curl.vcxproj] + _bld\src\tool_hugehelp.c(8,27): + see declaration of 'm' + src\tool_paramhlp.c(307,14): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + src\tool_progress.c(118,16): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + src\tool_writeout.c(288,31): warning C4459: declaration of 'm' hides global d + eclaration [_bld\src\curl.vcxproj] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/49348159/job/51ee75c + d2n0wj6lc#L614 -- sectransp_ make TLSCipherNameForNumber() available in non-verbose config + Reviewed-by: Daniel Stenberg + Closes #13077 - Reported-by: Cajus Pollmeier - Closes #12476 - Fixes #12474 +Daniel Stenberg (7 Mar 2024) -YX Hao (6 Dec 2023) +- KNOWN_BUGS: POP3 issue when reading small chunks -- lib: fix variable undeclared error caused by `infof` changes + Closes #12063 - `--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined. - `infof` isn't `Curl_nop_stmt` anymore: dac293c. +- RELEASE-NOTES: synced - Follow-up to dac293c +Robert Moreton (7 Mar 2024) - Closes #12470 +- asyn-ares: fix data race warning -Viktor Szakats (6 Dec 2023) + - Store the c-ares version during global init. -- tidy-up: fix yamllint whitespace issues in labeler.yml + Prior to this change several threads could write the same data to a + static int variable at the same time. Though in practice it's not a + problem ThreadSanitizer may warn. - Follow-up to bda212911457c6fadfbba50be61afc4ca513fa56 #12466 + Reported-by: Nikita Taranov + Assisted-by: Jay Satiro - Reviewed-by: Dan Fandrich - Closes #12475 + Fixes #13065 + Closes #13000 -- tidy-up: fix yamllint whitespace issues +Stefan Eissing (7 Mar 2024) - Closes #12466 +- hyper: implement unpausing via client reader -Chris Sauer (6 Dec 2023) + Just a tidy up to contain 'ifdef' pollution of common + code parts with implementation specifics. -- cmake: fix typo + - remove the ifdef hyper unpausing in easy.c + - add hyper client reader for CURL_CR_PROTOCOL phase + that implements the unpause method for calling + the hyper waker if it is set - Follow-up to aace27b - Closes #12464 + Closes #13075 -Daniel Stenberg (6 Dec 2023) +- ngtcp2: no recvbuf for stream -- dist: add tests/errorcodes.pl to the tarball + - write response data directly to the transfer via + `Curl_xfer_write_resp()` like we do in HTTP/2. - Used by test 1477 + Closes #13073 - Reported-by: Xi Ruoyao - Follow-up to 0ca3a4ec9a7 - Fixes #12462 - Closes #12463 +- docs/cmdline-opts/.gitignore: ignore curl.txt -Dan Fandrich (6 Dec 2023) + Closes #13076 -- github/labeler: update a missed key in the v5 upgrade +Evgeny Grin (Karlson2k) (7 Mar 2024) - Follow-up to ce03fe3ba +- sha512_256: add support for GnuTLS and OpenSSL -Version 8.5.0 (6 Dec 2023) + This is a follow-up for PR #12897. -Daniel Stenberg (6 Dec 2023) + Add support for SHA-512/256 digest calculation by TLS backends. + Currently only OpenSSL and GnuTLS (actually, nettle) support + SHA-512/256. -- RELEASE-NOTES: synced + Closes #13070 - The curl 8.5.0 release. +- digest: add check for hashing error -Dan Fandrich (5 Dec 2023) + Closes #13072 -- github/labeler: switch from the beta to labeler v5 +Viktor Szakats (7 Mar 2024) - Some keys were renamed and the dot option was made default. +- cmake: enable `ENABLE_CURL_MANUAL` by default - Closes #12458 + Meaning `curl.1` and `src/tool_hugehelp.c` are built by default, + and `--manual` in curl tool is also enabled by default. -Daniel Stenberg (5 Dec 2023) + This syncs behaviour with autotools. -- DEPRECATE: remove NTLM_WB in June 2024 + For a reproducible `curl.1`, `SOURCE_DATE_EPOCH` needs to be set + to a consistent date, e.g. the timestamp of `CHANGES`. - Ref: https://curl.se/mail/lib-2023-12/0010.html + A pre-built manual (e.g. the one distributed in the official source + tarball) will be ignored and rebuilt after this patch, unless + explicitly disabling this option. - Closes #12451 + Fixes #13028 + Closes #13069 -Jacob Hoffman-Andrews (4 Dec 2023) +Stefan Eissing (7 Mar 2024) -- rustls: implement connect_blocking +- http2: push headers better cleanup - Closes #11647 + - provide common cleanup method for push headers -Daniel Stenberg (4 Dec 2023) + Closes #13054 -- examples/rtsp-options.c: add +Daniel Stenberg (7 Mar 2024) - Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and - CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. +- GIT-INFO: convert to markdown - Closes #12452 + Closes #13074 -Stefan Eissing (4 Dec 2023) +Richard Levitte (7 Mar 2024) -- ngtcp2: ignore errors on unknown streams +- cmake: fix libcurl.pc and curl-config library specifications - - expecially in is_alive checks on connections, we might - see incoming packets on streams already forgotten and closed, - leading to errors reported by nghttp3. Ignore those. + Letting CMake figure out where libraries are located gives you full + paths. When generating libcurl.pc and curl-config, getting libraries as + full paths is unusual when one expects to get a list of -l. - Closes #12449 + To meet expectations, an effort is made to convert the full paths into + -l, possibly with -L before it. -Daniel Stenberg (4 Dec 2023) + Fixes #6169 + Fixes #12748 + Closes #12930 -- docs: make all examples in all libcurl man pages compile +Daniel Stenberg (7 Mar 2024) - Closes #12448 +- test463: HTTP with -d @file with file containing CR, LF and null byte -- checksrc.pl: support #line instructions +- paramhlp: fix CRLF-stripping files with "-d @file" - makes it identify the correct source file and line + All CR and LF bytes should be stripped, as documented, and all other + bytes are inluded in the data. Starting now, it also excludes null bytes + as they would otherwise also cut the data short. -- GHA/man-examples: verify libcurl man page examples + Reported-by: Simon K + Fixes #13063 + Closes #13064 -- verify-examples.pl: verify that all man page examples compile clean +Viktor Szakats (7 Mar 2024) -- RELEASE-NOTES: synced +- cmake: fix `CURL_WINDOWS_SSPI=ON` with Schannel disabled -Graham Campbell (2 Dec 2023) + Prior to this change `CURL_WINDOWS_SSPI` was accidentally forced `OFF` + when building without the Schannel TLS backend. -- http3: bump ngtcp2 and nghttp3 versions + This in turn may have caused Kerberos, SPNEGO and SSPI features + disappearing even with `CURL_WINDOWS_SSPI=ON` set. - nghttp3 v1.1.0 - ngtcp2 v1.1.0 + This patch fixes it by using the `CURL_USE_SCHANNEL` setting as a + default for `CURL_WINDOWS_SSPI`, but allowing a manual override. - In docs and CI + Also update the option text to better tell its purpose. - Closes #12446 + Thanks-to: Andreas Loew + Reviewed-by: Daniel Stenberg + Ref: #13056 + Closes #13061 -- CI/quiche: use `3.1.4+quic` consistently in CI workflows +Jay Satiro (6 Mar 2024) - Closes #12447 +- KNOWN_BUGS: FTPS server compatibility on Windows with Schannel -Viktor Szakats (2 Dec 2023) + - Remove "2.12 FTPS with Schannel times out file list operation" -- test1545: disable deprecation warnings + - Remove "7.12 FTPS directory listing hangs on Windows with Schannel" - Fixes: - https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr - p6pk#L1205 + - Add "7.12 FTPS server compatibility on Windows with Schannel" - Same with details: - https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil - b6wt#L1263 - ``` - tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 - .0. Use curl_mime_init() [-Werror=deprecated-declarations] - 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", - | ^~~~~~~~~~~~ - [...] - ``` + This change adds a more generic bug description that explains FTPS with + the latest curl and Schannel is not widely used and may have more bugs + than other TLS backends. - Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 + The two removed FTPS Schannel bugs can't be reproduced any longer and + were likely fixed by 24d6c288. - Fixes #12445 - Closes #12444 + Ref: https://github.com/curl/curl/issues/5284 + Ref: https://github.com/curl/curl/issues/9161 + Ref: https://github.com/curl/curl/issues/12894 -Daniel Stenberg (2 Dec 2023) + Closes https://github.com/curl/curl/pull/13032 -- INSTALL: update list of ports and CPU archs +- trace-config.md: remove the mutexed options list -- symbols-in-versions: the CLOSEPOLICY options are deprecated + - Remove the rendered manpage message that says: + "[--trace-config] is mutually exclusive to --trace and -v, --verbose". - The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. + Actually it can be used with either of those options, which are mutually + exclusive to each other but not to --trace-config. -z2_ (1 Dec 2023) + Ref: https://curl.se/docs/manpage.html#--trace-config -- build: fix builds that disable protocols but not digest auth + Closes https://github.com/curl/curl/pull/13031 - - Build base64 functions if digest auth is not disabled. +Daniel Stenberg (6 Mar 2024) - Prior to this change if some protocols were disabled but not digest auth - then a build error would occur due to missing base64 functions. +- mkhelp: simplify the generated hugehelp program - Fixes https://github.com/curl/curl/issues/12440 - Closes https://github.com/curl/curl/pull/12442 + Use a plain array and puts() every line, also allows us to provide the + strings without ending newlines. -Michał Antoniak (1 Dec 2023) + - merge blank lines into the next one as a prefixed newline. + - turn eight consecutive spaces into a tab (since they can only be on the + left side of text) + - the newly generated tool_hugehelp is 3K lines shorter and 50K smaller + - modifies the top logo layout a little by reducing the indent -- connect: reduce number of transportation providers + Closes #13047 - Use only the ones necessary - the ones that are built-in. Saves a few - bytes in the resulting code. +- docs: ascii version of manpage without nroff - Closes #12438 + Create ASCII version of manpage without nroff -David Benjamin (1 Dec 2023) + - build src/tool_hugegelp.c from the ascii manpage + - move the the manpage and the ascii version build to docs/cmdline-opts + - remove all use of nroff from the build process + - should make the build entirely reproducible (by avoiding nroff) -- vtls: consistently use typedef names for OpenSSL structs + - partly reverts 2620aa9 to build libcurl option man pages one by one + in cmake because the appveyor builds got all crazy until I did - The foo_st names don't appear in OpenSSL public API documentation. The - FOO typedefs are more common. This header was already referencing - SSL_CTX via . There is a comment about avoiding - , but OpenSSL actually declares all the typedefs in - , which is already included by (and - every other OpenSSL header), so just use that. Though I've included it - just to be explicit. + The ASCII version of the manpage - (I'm also fairly sure including already triggers the - Schannel conflicts anyway. The comment was probably just out of date.) + - is built with gen.pl, just like the manpage is + - has a right-justified column making the appearance similar to the previous + version + - uses a 4-space indent per level (instead of the old version's 7) + - does not do hyphenation of words (which nroff does) - Closes #12439 + History -Lau (1 Dec 2023) + We first made the curl build use nroff for building the hugehelp file in + December 1998, for curl 5.2. -- libcurl-security.3: fix typo + Closes #13047 - Fixed minimal typo. +Stefan Eissing (6 Mar 2024) - Closes #12437 +- lib: add `void *ctx` to reader/writer instances -Stefan Eissing (1 Dec 2023) + - `struct Curl_cwriter` and `struct Curl_creader` now carry a + `void *ctx` member that points to the instance as allocated. + - using `r->ctx` and `w->ctx` as pointer to the instance specific + struct that has been allocated -- ngtcp2: fix races in stream handling + Reported-by: Rudi Heitbaum + Fixes #13035 + Closes #13059 - - fix cases where ngtcp2 invokes callbacks on streams that - nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND - in these cases as it is normal behaviour. +- http: fix dead code in setting post client reader - Closes #12435 + - postsize was always 0, thus the check's else never happened + after the mime client reader was introduced -Emanuele Torre (1 Dec 2023) + Follow-up to 0ba47146f7ff3d + Closes #13060 -- tool_writeout_json: fix JSON encoding of non-ascii bytes +- http2: fix push discard - char variables if unspecified can be either signed or unsigned depending - on the platform according to the C standard; in most platforms, they are - signed. + - fix logic in discarding a failed pushed stream so that + stream context is properly cleaned up - This meant that the *i<32 waas always true for bytes with the top bit - set. So they were always getting encoded as \uXXXX, and then since they - were also signed negative, they were getting extended with 1s causing - '\xe2' to be expanded to \uffffffe2, for example: + Closes #13055 - $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul - l - \uffffffe2\uffffff80\uffffff9c +- transfer.c: break receive loop in speed limited transfers - I fixed this bug by making the code use explicitly unsigned char* - variables instead of char* variables. + - the change breaks looping in transfer.c receive for transfers that are + speed limited on having gotten *some* bytes. + - the overall speed limit timing is done in multi.c - Test 268 verifies + Reported-by: Dmitry Karpov + Bug: https://curl.se/mail/lib-2024-03/0001.html + Closes #13050 - Reported-by: iconoclasthero - Closes #12434 +- mime: add client reader -Stefan Eissing (1 Dec 2023) + Add `mime` client reader. Encapsulates reading from mime parts, getting + their length, rewinding and unpausing. -- cf-socket: TCP trace output local address used in connect + - remove special mime handling from sendf.c and easy.c + - add general "unpause" method to client readers + - use new reader in http/imap/smtp + - make some mime functions static that are now only used internally - Closes #12427 + In addition: + - remove flag 'forbidchunk' as no longer needed -Jay Satiro (1 Dec 2023) + Closes #13039 -- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation +Daniel Stenberg (5 Mar 2024) - - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it - includes protocol-specific instructions that trigger a transfer. +- RELEASE-NOTES: synced - Prior to this change it explicitly said that it did not include those - instructions in the time, but that is incorrect. +- TODO: remove "build HTTP/3 with OpenSSL and nghttp3 using cmake" - The change is a copy of the fixed explanation already in - CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. + Follow-up to 8e741644a229c37 - Reported-by: eeverettrbx@users.noreply.github.com +Tal Regev (5 Mar 2024) - Fixes https://github.com/curl/curl/issues/12431 - Closes https://github.com/curl/curl/pull/12432 +- cmake: add USE_OPENSSL_QUIC support -Daniel Stenberg (30 Nov 2023) + Closes #13034 -- multi: during ratelimit multi_getsock should return no sockets +Stefan Eissing (5 Mar 2024) - ... as there is nothing to wait for then, it just waits. Otherwise, this - causes much more CPU work and updates than necessary during ratelimit - periods. +- TIMER_STARTTRANSFER: set the same for everyone - Ref: https://curl.se/mail/lib-2023-11/0056.html - Closes #12430 + - set TIMER_STARTTRANSFER on seeing the first response bytes + in the download client writer, not coming from a CONNECT + - initialized the timer the same way for all protocols + - remove explicit setting of TIMER_STARTTRANSFER in file.c + and c-hyper.c -Dmitry Karpov (30 Nov 2023) + Closes #13052 -- transfer: abort pause send when connection is marked for closing +Michael Kaufmann (5 Mar 2024) - This handles cases of some bi-directional "upgrade" scenarios - (i.e. WebSockets) where sending is paused until some "upgrade" handshake - is completed, but server rejects the handshake and closes the - connection. +- http: better error message for HTTP/1.x response without status line - Closes #12428 + If a response without a status line is received, and the connection is + known to use HTTP/1.x (not HTTP/0.9), report the error "Invalid status + line" instead of "Received HTTP/0.9 when not allowed". -Daniel Stenberg (28 Nov 2023) + Closes #13045 -- RELEASE-NOTES: synced +Viktor Szakats (5 Mar 2024) -- openssl: when a session-ID is reused, skip OCSP stapling +- KNOWN_BUGS: fix typo - Fixes #12399 - Reported-by: Alexey Larikov - Closes #12418 + Reviewed-by: Daniel Stenberg + Closes #13051 -- test1545: test doing curl_formadd twice with missing file +Sebastian Neubauer (5 Mar 2024) - Reproduces #12410 - Verifies the fix - Closes #12421 +- smpt: fix starttls -- Curl_http_body: cleanup properly when Curl_getformdata errors + In cases where the connection was fast, curl sometimes failed to open a + connection. This fixes a regression of c2d973627bab12abc5486a3f3. - Reported-by: yushicheng7788 on github - Based-on-work-by: yushicheng7788 on github - Fixes #12410 - Closes #12421 + The regression triggered in these steps: -- test1477: verify that libcurl-errors.3 and public headers are synced + 1. Create an smtp connection + 2. Use STARTTLS + 3. Receive the response + 4. We are inside the loop in `smtp_statemachine`, calling + `smtp_state_starttls_resp` + 5. In the good flow, we exit the loop, re-enter `smtp_statemachine` and + run `smtp_perform_upgrade_tls` at the start of the function. - The script errorcodes.pl extracts all error codes from all headers and - checks that they are all documented, then checks that all documented - error codes are also specified in a header file. + In the bad flow, we stay in the while loop, calling + `Curl_pp_readresp`, which reads part of the TLS handshake and things + go wrong. - Closes #12424 + The reason is that `Curl_pp_moredata` changed behavior and always + returns `true`, so we stay in the loop in `smtp_statemachine`. With a + slow connection `Curl_pp_readresp` cannot read new data and returns + `CURL_AGAIN`, so we leave the loop and re-enter `smtp_statemachine`. -- libcurl-errors.3: sync with current public headers + With a fast connection, `Curl_pp_readresp` reads new data from the tcp + connection, which is part of the TLS handshake. - Closes #12424 + The fix is in `Curl_pp_moredata`, which needs to take the final line + into account and return `false` if only the final line is stored. -Stefan Eissing (28 Nov 2023) + Closes #13048 -- test459: fix for parallel runs +Stefan Eissing (5 Mar 2024) - - change warniing message to work better with varying filename - length. - - adapt test output check to new formatting +- lib: enhance client reader resume + rewind - Follow-up to 97ccc4479f77ba3191c6 - Closes #12423 + - update client reader documentation + - client reader, add rewind capabilities + - tell creader to rewind on next start + - Curl_client_reset() will keep reader for future rewind if requested + - add Curl_client_cleanup() for freeing all resources independent of + rewinds + - add Curl_client_start() to trigger rewinds + - move rewind code from multi.c to sendf.c and make part of + "cr-in"'s implementation + - http, move the "resume_from" handling into the client readers + - the setup of a HTTP request is reshuffled to follow: + * determine method, target, auth negotiation + * install the client reader(s) for the request, including crlf + conversions and "chunked" encoding + * apply ranges to client reader + * concat request headers, upgrades, cookies, etc. + * complete request by determining Content-Length of installed + readers in combination with method + * send + - add methods for client readers to + * return the overall length they will generate (or -1 when unknown) + * return the amount of data on the CLIENT level, so that + expect-100 can decide if it want to apply itself + * set a "resume_from" offset or fail if unsupported + - struct HTTP has become largely empty now + - rename `Client_reader_*` to `Curl_creader_*` -Daniel Stenberg (27 Nov 2023) + Closes #13026 -- tool_cb_prg: make the carriage return fit for wide progress bars +Viktor Szakats (5 Mar 2024) - When the progress bar was made max width (256 columns), the fly() - function attempted to generate its output buffer too long so that the - trailing carriage return would not fit and then the output would show - wrongly. The fly function is called when the expected total transfer is - unknown, which could be one or more progress calls before the actual - progress meter get shown when the expected transfer size is provided. +- openssl-quic: fix BIO leak and Windows warning - This new take also replaces the msnprintf() call with a much simpler - memset() for speed. + Caused by an accidentally duplicated line in + d6825df334def106f735ce7e0c1a2ea87bddffb0. - Reported-by: Tim Hill - Fixes #12407 - Closes #12415 + ``` + .../lib/vquic/curl_osslq.c:1095:30: warning: implicit conversion loses intege + r precision: 'curl_socket_t' (aka 'unsigned long long') to 'int' [-Wshorten-6 + 4-to-32] + 1095 | bio = BIO_new_dgram(ctx->q.sockfd, BIO_NOCLOSE); + | ~~~~~~~~~~~~~ ~~~~~~~^~~~~~ + 1 warning and 2 errors generated. + ``` -- tool_parsecfg: make warning output propose double-quoting + Reviewed-by: Stefan Eissing + Closes #13043 - When the config file parser detects a word that *probably* should be - quoted, mention double-quotes as a possible remedy. +- openssl-quic: fix unity build, casing, indentation - Test 459 verifies. + - rename static functions to avoid duplicate symbols in unity mode. + - windows -> Windows/window in error message and comment. + - fix indentation. - Proposed-by: Jiehong on github - Fixes #12409 - Closes #12412 + Reviewed-by: Stefan Eissing + Closes #13044 -Jay Satiro (26 Nov 2023) +Daniel Stenberg (5 Mar 2024) -- curl.rc: switch out the copyright symbol for plain ASCII +- gen.pl: make the "manpageification" faster - .. like we already do for libcurl.rc. + The function that replaces occurances of "--longoption" with "-Z, + --longoption" etc with the proper highlight applied, no longer loops + over the options. - libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" - warning so it was switched to ascii. + Closes #13041 - Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 +- CONTRIBUTE: update the section on documentation format - Suggested-by: Robert Southee + ... since most of it is markdown now. - Closes https://github.com/curl/curl/pull/12403 + Closes #13046 -Daniel Stenberg (26 Nov 2023) +- smtp: free a temp resource -- conncache: use the closure handle when disconnecting surplus connections + The returned address needs to be freed. - Use the closure handle for disconnecting connection cache entries so - that anything that happens during the disconnect is not stored and - associated with the 'data' handle which already just finished a transfer - and it is important that details from the unrelated disconnect does not - taint meta-data in the data handle. + Follow-up to e3905de8196d67b89df1602feb84c1f993211b20 + Spotted by Coverity - Like storing the response code. + Closes #13038 - This also adjust test 1506. Unfortunately it also removes a key part of - the test that verifies that a connection is closed since when this - output vanishes (because the closure handle is used), we don't know - exactly that the connection actually gets closed in this test... +- _VARIABLES.md: improve the description - Reported-by: ohyeaah on github - Fixes #12367 - Closes #12405 + Closes #13040 -- RELEASE-NOTES: synced +dependabot[bot] (4 Mar 2024) -Stefan Eissing (24 Nov 2023) +- build(deps): bump fsfe/reuse-action from 2 to 3 -- quic: make eyeballers connect retries stop at weird replies + Bumps [fsfe/reuse-action](https://github.com/fsfe/reuse-action) from 2 to 3. + - [Release notes](https://github.com/fsfe/reuse-action/releases) + - [Commits](https://github.com/fsfe/reuse-action/compare/v2...v3) - - when a connect immediately goes into DRAINING state, do - not attempt retries in the QUIC connection filter. Instead, - return CURLE_WEIRD_SERVER_REPLY - - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an - inconclusive answer. When all addresses have been attempted, - rewind the address list once on an inconclusive answer. - - refs #11832 where connects were retried indefinitely until - the overall timeout fired + --- + updated-dependencies: + - dependency-name: fsfe/reuse-action + dependency-type: direct:production + update-type: version-update:semver-major + ... - Closes #12400 + Signed-off-by: dependabot[bot] -Daniel Stenberg (24 Nov 2023) +Stefan Eissing (4 Mar 2024) -- CI: verify libcurl function SYNPOSIS sections +- pytest: adapt to API change - With the .github/scripits/verify-synopsis.pl script + - pytest has changed the signature of the hook pytest_report_header() + for some obscure reason and that change landed in our CI now - Closes #12402 + - remove the changed param that we never used anyway -- docs/libcurl: SYNSOPSIS cleanup + Closes #13037 - - use the correct include file - - make sure they are declared as in the header file - - fix minor nroff syntax mistakes (missing .fi) +Daniel Stenberg (4 Mar 2024) - These are verified by verify-synopsis.pl, which extracts the SYNPOSIS - code and runs it through gcc. +- cookie: if psl fails, reject the cookie - Closes #12402 + A libpsl install without data and no built-in database is now considered + bad enough to reject all cookies since they cannot be checked. It is + somewhat of a user error, but still. -- sendf: fix comment typo + Reported-by: Dan Fandrich + Closes #13033 -- fopen: allocate the dir after fopen +Stefan Eissing (4 Mar 2024) - Move the allocation of the directory name down to after the fopen() call - to allow that shortcut code path to avoid a superfluous malloc+free - cycle. +- lib: further send/upload handling polish - Follow-up to 73b65e94f35311 + - Move all the "upload_done" handling to request.c - Closes #12398 + - add possibility to abort sending of a request + - add `Curl_req_done_sending()` for checks + - transfer.c: readwrite_upload() now clean -Stefan Eissing (24 Nov 2023) + - removing data->state.ulbuf and data->req.upload_fromhere -- transfer: cleanup done+excess handling + - as well as data->req.upload_present + - set data->req.upload_done on having read all from + the client and completely flushed the send buffer - - add `SingleRequest->download_done` as indicator that - all download bytes have been received - - remove `stop_reading` bool from readwrite functions - - move excess body handling into client download writer + - tftp, remove setting of data->req.upload_fromhere - Closes #12371 + - serves no purpose as `upload_present` is not set + and the data itself is directly `sendto()` anyway -Daniel Stenberg (23 Nov 2023) + - smtp, make upload EOB conversion a client reader + - xfer_ulbuf addition -- fopen: create new file using old file's mode + - add xfer_ulbuf for borrowing, similar to xfer_buf + - use in file upload + - use in c-hyper body sending - Because the function renames the temp file to the target name as a last - step, if the file was previously owned by a different user, not ORing - the old mode could otherwise end up creating a file that was no longer - readable by the original owner after save. + - h1-proxy, remove init of data->state.uilbuf that is never used + - smb, add own send_buf instead of using data->state.ulbuf - Reported-by: Loïc Yhuel - Fixes #12299 - Closes #12395 + Closes #13010 -- test1476: require proxy +Daniel Stenberg (4 Mar 2024) - Follow-up from 323df4261c3542 +- RELEASE-NOTES: synced - Closes #12394 +kpcyrd (3 Mar 2024) -- fopen: create short(er) temporary file name +- rustls: fix two warnings related to number types - Only using random letters in the name plus a ".tmp" extension. Not by - appending characters to the final file name. + Reported-by: Gisle Vanem + Follow-up to #12989 + Closes #13017 - Reported-by: Maksymilian Arciemowicz +Stefan Eissing (3 Mar 2024) - Closes #12388 +- bufq: writing into a softlimit queue cannot be partial -Stefan Eissing (23 Nov 2023) + - when unable to obtain a new chunk on a softlimit bufq, + this is an allocation error and needs to be reported as + such. + - writes into a soflimit bufq never must be partial success -- tests: git ignore generated second-hsts.txt file + Reported-by: Dan Fandrich + Fixes #13020 + Closes #13023 - File is generated in test lib1900 +Dan Fandrich (2 Mar 2024) - Follow-up to 7cb03229d9e9c5 +- configure: Don't build shell completions when disabled - Closes #12393 + With the recent changes to completion file building, the files were + built always and only installation was selectively disabled. Now, when + they are disabled they aren't even built, avoiding a build-time error in + environments where it's not possible to run the curl binary that was + just created (e.g. if library paths were not set up correctly). -Viktor Szakats (23 Nov 2023) + Follow-up to 0f7aba83c -- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 + Reported-by: av223119 on github + Fixes #13027 + Closes #13030 - Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to - OpenSSL 1.1.x, and LibreSSL 3.6 or upper. +Jay Satiro (2 Mar 2024) - With the caveat, that "group name" and "type name" are missing from - the log output with these TLS backends. +- cmdline-opts/_EXITCODES: sync with libcurl-errors - Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + - Add error code 100 (CURLE_TOO_LARGE) to the list of error codes that + can be returned by the curl tool. - Reviewed-by: Daniel Stenberg - Closes #12385 + Closes https://github.com/curl/curl/pull/13015 -Daniel Stenberg (23 Nov 2023) +Stefan Eissing (1 Mar 2024) -- urldata: fix typo in comment +- hyper: disable test1598 due to lack of trailer support -- CI: codespell + Follow-up to 50838095 - The list of words to ignore is in the file - .github/scripts/codespell-ignore.txt + Closes #13016 - Closes #12390 +Dan Fandrich (1 Mar 2024) -- lib: fix comment typos +- ftp: Mark a const buffer as const - Five separate ones, found by codespell +- appveyor: Properly skip if only CircleCI is changed - Closes #12390 +- docs: Update minimal binary size in INSTALL.md -- test1476: verify cookie PSL mixed case + Include more options to reduce binary size. -- cookie: lowercase the domain names before PSL checks +- configure: Don't make shell completions without perl - Reported-by: Harry Sintonen + The code that attempted to skip building the shell completions didn't + work properly and tried to build them even if perl wasn't available. + This step, as well as the install step, is now properly skipped without + perl. - Closes #12387 + Follow-up to 89733e2dd -Viktor Szakats (23 Nov 2023) + Closes #13022 -- openssl: fix building with v3 `no-deprecated` + add CI test +RainRat (1 Mar 2024) - - build quictls with `no-deprecated` in CI to have test coverage for - this OpenSSL 3 configuration. +- misc: Fix typos in docs and lib - - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. - The caller code is meant for OpenSSL 3, while these two functions were - only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 - if built with option `no-deprecated`, causing build errors: - ``` - vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ - algorithms'; ISO C99 and later do not support implicit function declaration - s [-Wimplicit-function-declaration] - vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ - digests'; ISO C99 and later do not support implicit function declarations [ - -Wimplicit-function-declaration] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f - ullLog=true#L7667 + This fixes miscellaneous typos and duplicated words in the docs, lib + and test comments and a few user facing errorstrings. - Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 - Reviewed-by: Alex Bozarth + Author: RainRat on Github + Reviewed-by: Daniel Gustafsson + Reviewed-by: Dan Fandrich + Closes: #13019 - - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with - `no-deprecated` quictls 3 builds. - Do it by moving an existing solution for this from `vtls/openssl.c` - to `vtls/openssl.h` and adjusting caller code. - ``` - vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g - et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit - -function-declaration] - ``` - Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s - tep:24:1178 +Dan Fandrich (29 Feb 2024) - - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and - `-Wunused-function` when trying to build curl with NTLM enabled but - without the necessary TLS backend (with DES) support. +- configure: build & install shell completions when enabled - Closes #12384 + The --with-fish-functions-dir and --with-zsh-functions-dir options + currently have no effect on a normal build because the scripts/ directory + where they're used is not built. Add scripts/ to a normal build and + change the completion options to default to off to preserve the existing + behaviour. -- curl.h: delete Symbian OS references + Closes: #12906 - curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 - via #5989. Delete references to it from public headers, because there - is no fresh release to use those headers with. +- github/labeler: improve the match patterns - Reviewed-by: Dan Fandrich - Reviewed-by: Jay Satiro - Closes #12378 +Stefan Eissing (28 Feb 2024) -- windows: use built-in `_WIN32` macro to detect Windows +- tests: add test1598 for POST with trailers - Windows compilers define `_WIN32` automatically. Windows SDK headers - or build env defines `WIN32`, or we have to take care of it. The - agreement seems to be that `_WIN32` is the preferred practice here. - Make the source code rely on that to detect we're building for Windows. + - test POST fields with trailers and chunked encoding - Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for - Windows detection, next to the official `_WIN32`. After this patch it - only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. + Ref: #12938 + Closes #13009 - There is a slight chance these break compatibility with Windows - compilers that fail to define `_WIN32`. I'm not aware of any obsolete - or modern compiler affected, but in case there is one, one possible - solution is to define this macro manually. +Daniel Stenberg (28 Feb 2024) - grepping for `WIN32` remains useful to discover Windows-specific code. +- cmdline-opts/_VERSION: provide %VERSION correctly - Also: + ... so that it does not get included verbatim in the output. Fixes a + regression shipped in 8.6.0. - - extend `checksrc` to ensure we're not using `WIN32` anymore. + Also fix a format mistake in form.md - - apply minor formatting here and there. + Closes #13008 - - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. +Stefan Eissing (28 Feb 2024) - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg +- lib: Curl_read/Curl_write clarifications - Closes #12376 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to + clarify when and at what level they operate + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer + setup of `conn->sockfd` and `conn->writesockfd` on which + connection filter chain to operate. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index + as parameter. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for + naming consistency + - clarify that the special CURLE_AGAIN hangling to return + `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` + and CURLE_AGAIN is returned by all other send() variants. + - fix a bug in websocket `curl_ws_recv()` that mixed up data + when it arrived in more than a single chunk (to be made + into a sperate PR, also) -Stefan Eissing (22 Nov 2023) + Added as documented [in + CLIENT-READER.md](https://github.com/curl/curl/blob/5b1f31dfbab8aef467c419c68 + aa06dc738cb75d4/docs/CLIENT-READERS.md). -- url: ConnectionExists revisited + - old `Curl_buffer_send()` completely replaced by new `Curl_req_send()` + - old `Curl_fillreadbuffer()` replaced with `Curl_client_read()` + - HTTP chunked uploads are now formatted in a client reader added when + needed. + - FTP line-end conversions are done in a client reader added when + needed. + - when sending requests headers, remaining buffer space is filled with + body data for sending in "one go". This is independent of the request + body size. Resolves #12938 as now small and large requests have the + same code path. - - have common pattern of `if not match, continue` - - revert pages long if()s to return early - - move dead connection check to later since it may - be relatively expensive - - check multiuse also when NOT building with NGHTTP2 - - for MULTIUSE bundles, verify that the inspected - connection indeed supports multiplexing when in use - (bundles may contain a mix of connection, afaict) + Changes done to test cases: - Closes #12373 + - test513: now fails before sending request headers as this initial + "client read" triggers the setup fault. Behaves now the same as in + hyper build + - test547, test555, test1620: fix the length check in the lib code to + only fail for reads *smaller* than expected. This was a bug in the + test code that never triggered in the old implementation. -Daniel Stenberg (22 Nov 2023) + Closes #12969 -- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range +Daniel Gustafsson (28 Feb 2024) - ... or use the default value. +- curldown: Fix email address in Copyright - Also clarify the documentation language somewhat. + The curldown conversion accidentally replaced daniel@haxx.se with + just daniel.se. This reverts back to the proper email address in + the curldown docs as well as in a few other stray places where it + was incorrect (while unrelated to curldown). - Closes #12382 + Reviewed-by: Daniel Stenberg + Closes: #12997 -- urldata: make maxconnects a 32 bit value +Daniel Stenberg (28 Feb 2024) - "2^32 idle connections ought to be enough for anybody" +- getparam: make --ftp-ssl work again - Closes #12375 + Follow-up to 9e4e527 which accidentally broke it -- FEATURES: update the URL phrasing + Reported-by: Jordan Brown + Fixes #13006 + Closes #13007 - The URL is length limited since a while back so "no limit" simply is not - true anymore. Mention the URL RFC standard used instead. +- KNOWN_BUGS: IMAPS connection fails with rustls error - Closes #12383 + Closes #10457 -- wolfssh: remove redundant static prototypes +- KNOWN_BUGS: FTPS upload, FileZilla, GnuTLS and close_notify - vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We - rror=redundant-decls] + Closes #11383 - Closes #12381 +- KNOWN_BUGS: Implicit FTPS upload timeout -- setopt: remove superfluous use of ternary expressions + Closes #11720 - Closes #12374 +- KNOWN_BUGS: HTTP/2 prior knowledge over proxy -- mime: store "form escape" as a single bit + Closes #12641 - Closes #12374 +- TODO: build HTTP/3 with OpenSSL and nghttp3 using cmake -- setopt: check CURLOPT_TFTP_BLKSIZE range on set + Closes #12988 - ... instead of later when the transfer is about to happen. +- TODO: Select signature algorithms - Closes #12374 + Closes #12982 -Viktor Szakats (21 Nov 2023) +- examples: use present tense in comments -- build: add more picky warnings and fix them + remove "will" and some other word fixes - Enable more picky compiler warnings. I've found these options in the - nghttp3 project when implementing the CMake quick picky warning - functionality for it [1]. + Closes #13003 - `-Wunused-macros` was too noisy to keep around, but fixed a few issues - it revealed while testing. +- docs: more language cleanups - - autotools: reflect the more precisely-versioned clang warnings. - Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 - - autotools: sync between clang and gcc the way we set `no-multichar`. - - autotools: avoid setting `-Wstrict-aliasing=3` twice. - - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. - It triggers in libtool-generated stub code. + - present tense + - avoid bad words - - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. + Closes #13003 - - lib/curl_setup.h: delete duplicate declaration for `fileno`. - Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d - (1999-12-29). This suggests this may not be needed anymore, but if - it does, we may restore this for those specific (non-Windows) systems. - - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since - c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. - - lib: delete unused macro `isxdigit_ascii` since - f65f750742068f579f4ee6d8539ed9d5f0afcb85. - - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. - - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. - - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` - macro. - - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. - - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. - - lib/setopt: fix `-Wunreachable-code-break`. - - lib/system_win32 and lib/timeval: fix double declarations for - `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. - - lib/warnless: fix double declarations in CMake UNITY mode [5]. - This was due to force-disabling the header guard of `warnless.h` to - to reapply it to source code coming after `warnless.c` in UNITY - builds. This reapplied declarations too, causing the warnings. - Solved by adding a header guard for the lines that actually need - to be reapplied. - - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. - - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant - block. - - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. - - lib/vtls/sectransp: suppress `-Wunreachable-code`. - Detected in `else` branches of dynamic feature checks, with results - known at compile-time, e.g. - ```c - if(SecCertificateCopySubjectSummary) /* -> true */ - ``` - Likely fixable as a separate micro-project, but given SecureTransport - is deprecated anyway, let's just silence these locally. - - src/tool_help: delete duplicate declaration for `helptext`. - - src/tool_xattr: fix `-Wunreachable-code`. - - tests: delete duplicate declaration for `unitfail` [8]. - - tests: delete duplicate declaration for `strncasecompare`. - - tests/libtest: delete duplicate declaration for `gethostname`. - Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 - (2010-08-02). - Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc - If there are still systems around with warnings, we may restore the - prototype, but limited for those systems. - - tests/lib2305: delete duplicate declaration for - `libtest_debug_config`. - - tests/h2-download: fix `-Wunreachable-code-break`. - - [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 - f8bf9f/cmake/PickyWarningsC.cmake - [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui - qla5fj45?fullLog=true#L1675 - [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 - 2331#step:7:72 - https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 - 2331#step:7:109 - [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L204 - [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L218 - [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 - 2331#step:7:290 - [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 - 2331#step:9:1193 - [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 - 2331#step:33:1870 - - Closes #12331 - -Daniel Stenberg (21 Nov 2023) - -- transfer: avoid unreachable expression - - If curl_off_t and size_t have the same size (which is common on modern - 64 bit systems), a condition cannot occur which Coverity pointed - out. Avoid the warning by having the code conditionally only used if - curl_off_t actually is larger. - - Follow-up to 1cd2f0072fa482e25baa2 - - Closes #12370 - -Stefan Eissing (21 Nov 2023) - -- transfer: readwrite improvements - - - changed header/chunk/handler->readwrite prototypes to accept `buf`, - `blen` and a `pconsumed` pointer. They now get the buffer to work on - and report back how many bytes they consumed - - eliminated `k->str` in SingleRequest - - improved excess data handling to properly calculate with any body data - left in the headerb buffer - - eliminated `k->badheader` enum to only be a bool - - Closes #12283 - -Daniel Stenberg (21 Nov 2023) +Daniel Gustafsson (27 Feb 2024) -- RELEASE-NOTES: synced +- setopt: Fix disabling all protocols -Jiří Hruška (21 Nov 2023) + When disabling all protocols without enabling any, the resulting + set of allowed protocols remained the default set. Clearing the + allowed set before inspecting the passed value from --proto make + the set empty even in the errorpath of no protocols enabled. -- transfer: avoid calling the read callback again after EOF + Co-authored-by: Dan Fandrich + Reported-by: Dan Fandrich + Reviewed-by: Daniel Stenberg + Closes: #13004 - Regression since 7f43f3dc5994d01b12 (7.84.0) +Andreas Kiefer (27 Feb 2024) - Bug: https://curl.se/mail/lib-2023-11/0017.html +- fopen: fix narrowing conversion warning on 32-bit Android - Closes #12363 + This was fixed in commit 06dc599405f, but came back in commit + 03cb1ff4d62. -Daniel Stenberg (21 Nov 2023) + When building for 32-bit ARM or x86 Android, `st_mode` is defined as + `unsigned int` instead of `mode_t`, resulting in a + `-Wimplicit-int-conversion` clang warning because `mode_t` is + `unsigned short`. Add a cast to silence the warning, but only for + 32-bit Android builds, because other architectures and platforms are + not affected. -- doh: provide better return code for responses w/o addresses + Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li + bc/include/sys/stat.h#86 + Closes https://github.com/curl/curl/pull/12998 - Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the - response did not contain any addresses. Now it more accurately returns - CURLE_COULDNT_RESOLVE_HOST. +Stefan Eissing (27 Feb 2024) - Reported-by: lRoccoon on github +- lib: Curl_read/Curl_write clarifications - Fixes #12365 - Closes #12366 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to + clarify when and at what level they operate + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer + setup of `conn->sockfd` and `conn->writesockfd` on which + connection filter chain to operate. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index + as parameter. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for + naming consistency + - clarify that the special CURLE_AGAIN hangling to return + `CURLE_OK` with length 0 only applies to `Curl_xfer_send()` + and CURLE_AGAIN is returned by all other send() variants. + - fix a bug in websocket `curl_ws_recv()` that mixed up data + when it arrived in more than a single chunk -Stefan Eissing (21 Nov 2023) + The method for sending not just raw bytes, but bytes that are either + "headers" or "body". The send abstraction stack, to to bottom, now is: -- HTTP/2, HTTP/3: handle detach of onoing transfers + * `Curl_req_send()`: has parameter to indicate amount of header bytes, + buffers all data. + * `Curl_xfer_send()`: knows on which socket index to send, returns + amount of bytes sent. + * `Curl_conn_send()`: called with socket index, returns amount of bytes + sent. - - refs #12356 where a UAF is reported when closing a connection - with a stream whose easy handle was cleaned up already - - handle DETACH events same as DONE events in h2/h3 filters + In addition there is `Curl_req_flush()` for writing out all buffered + bytes. - Fixes #12356 - Reported-by: Paweł Wegner - Closes #12364 + `Curl_req_send()` is active for requests without body, + `Curl_buffer_send()` still being used for others. This is because the + special quirks need to be addressed in future parts: -Viktor Szakats (20 Nov 2023) + * `expect-100` handling + * `Curl_fillreadbuffer()` needs to add directly to the new + `data->req.sendbuf` + * special body handlings, like `chunked` encodings and line end + conversions will be moved into something like a Client Reader. -- autotools: stop setting `-std=gnu89` with `--enable-warnings` + In functions of the pattern `CURLcode xxx_send(..., ssize_t *written)`, + replace the `ssize_t` with a `size_t`. It makes no sense to allow for negativ + e + values as the returned `CURLcode` already specifies error conditions. This + allows easier handling of lengths without casting. - Do not alter the C standard when building with `--enable-warnings` when - building with gcc. + Closes #12964 - On one hand this alters warning results compared to a default build. - On the other, it may produce different binaries, which is unexpected. +Daniel Stenberg (27 Feb 2024) - Also fix new warnings that appeared after removing `-std=gnu89`: +- multi: make add_handle free any multi_easy - - include: fix public curl headers to use the correct printf mask for - `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 - and Visual Studio 2013 and newer. This fixes the printf mask warnings - in examples and tests. E.g. [1] + If the easy handle that is being added to a multi handle has previously + been used for curl_easy_perform(), there is a private multi handle here + that we can kill off. While it flushes some caches etc for the easy + handle would it be used for an easy interface transfer again after being + used in the multi stack, this cleanup simplifies behavior and uses less + memory. - - conncache: fix printf format string [2]. + Closes #12992 - - http2: fix potential null pointer dereference [3]. - (seen on Slackware with gcc 11.) +- docs: use present tense - - libssh: fix printf format string in SFTP code [4]. - Also make MSVC builds compatible with old CRT versions. + avoid "will", detect "will" as a bad word in the CI - - libssh2: fix printf format string in SFTP code for MSVC. - Applying the same fix as for libssh above. + Also line wrapped a bunch of paragraphs - - unit1395: fix `argument is null` and related issues [5]: - - stop calling `strcmp()` with NULL to avoid undefined behaviour. - - fix checking results if some of them were NULL. - - do not pass NULL to printf `%s`. + Closes #13001 - - ci: keep a build job with `-std=gnu89` to continue testing for - C89-compliance. We can apply this to other gcc jobs as needed. - Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 +- CURLOPT_SSL_CTX_FUNCTION.md: no promises of lifetime after return - [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l - ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b - [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 - 2346#step:6:67 - [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 - 2346#step:30:214 - [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 - 2346#step:29:895 - [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 - 2346#step:33:1689 + ... and cleanup other language. - Closes #12346 + Closes #12999 -- autotools: fix/improve gcc and Apple clang version detection +Stefan Eissing (27 Feb 2024) - - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu - may return `n-win32` (also with `-dumpfullversion`). Causing these - errors and failing to enable picky warnings: - ``` - ../configure: line 23845: test: : integer expression expected - ``` - Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 - 3718#step:5:143 +- lib: send rework - Fix that by stripping any dash-suffix and handling a dotless (major-only) - version number by assuming `.0` in that case. + Curl_read/Curl_write clarifications - `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` - Ref: https://github.com/mamedev/mame/pull/9767 + - replace `Curl_read()`, `Curl_write()` and `Curl_nwrite()` to 1clarify + when and at what level they operate - - fix Apple clang version detection for releases between - 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the - version was under-detected as 3.7 llvm/clang equivalent. + - send/recv of transfer related data is now done via + `Curl_xfer_send()/Curl_xfer_recv()` which no longer has + socket/socketindex as parameter. It decides on the transfer setup of + `conn->sockfd` and `conn->writesockfd` on which connection filter + chain to operate. - - fix Apple clang version detection for 'Apple clang version 11.0.0' - and newer where the Apple clang version was detected, instead of its - llvm/clang equivalent. + - send/recv on a specific connection filter chain is done via + `Curl_conn_send()/Curl_conn_recv()` which get the socket index as + parameter. - - display detected clang/gcc/icc compiler version. + - rename `Curl_setup_transfer()` to `Curl_xfer_setup()` for naming + consistency - Via libssh2: - - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d - 89875ad - https://github.com/libssh2/libssh2/pull/1187 - - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 - 42b592e - https://github.com/libssh2/libssh2/pull/1232 + - clarify that the special CURLE_AGAIN handling to return `CURLE_OK` + with length 0 only applies to `Curl_xfer_send()` and CURLE_AGAIN is + returned by all other send() variants. - Closes #12362 + SingleRequest reshuffling -- autotools: delete LCC compiler support bits + - move functions into request.[ch] + - differentiate between reset and free + - add Curl_req_done() to perform last actions + - add a send `bufq` to SingleRequest for future use in keeping upload data - Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 + Closes #12963 - Closes #12357 +Daniel Stenberg (26 Feb 2024) -- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` +- RELEASE-NOTES: synced - - tests: verify CMake `DISABLE` options. +- http_chunks: remove unused 'endptr' variable - Make an exception for 2 CMake-only ones, and one more that's - using a different naming scheme, also in autotools and source. + Closes #12996 - - cmake: add support for `CURL_DISABLE_HEADERS_API`. +Louis Solofrizzo (26 Feb 2024) - Suggested-by: Daniel Stenberg - Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 +- lib: initialize output pointers to NULL before calling strto[ff,l,ul] - Closes #12353 + In order to make MSAN happy: -Jacob Hoffman-Andrews (20 Nov 2023) + ==2200945==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x596f3b3ed246 in curlx_strtoofft [...]/libcurl/src/lib/strtoofft.c:23 + 9:11 + #1 0x596f3b402156 in Curl_httpchunk_read [...]/libcurl/src/lib/http_chunk + s.c:149:12 + #2 0x596f3b348550 in readwrite_data [...]/libcurl/src/lib/transfer.c:607: + 11 + [...] -- hyper: temporarily remove HTTP/2 support + ==2202041==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x5a3fab66a72a in Curl_parse_port [...]/libcurl/src/lib/urlapi.c:547:8 + #1 0x5a3fab650645 in parse_authority [...]/libcurl/src/lib/urlapi.c:796:1 + 2 + #2 0x5a3fab6740f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 + #3 0x5a3fab664fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: + 1342:12 + [...] - The current design of the Hyper integration requires rebuilding the - Hyper clientconn for each request. However, building the clientconn - requires resending the HTTP/2 connection preface, which is incorrect - from a protocol perspective. That in turn causes servers to send GOAWAY - frames, effectively degrading performance to "no connection reuse" in - the best case. It may also be triggering some bugs where requests get - dropped entirely and reconnects take too long. + ==2202320==WARNING: MemorySanitizer: use-of-uninitialized-value + #0 0x569076a0d6b0 in ipv4_normalize [...]/libcurl/src/lib/urlapi.c:683:12 + #1 0x5690769f2820 in parse_authority [...]/libcurl/src/lib/urlapi.c:803:1 + 0 + #2 0x569076a160f6 in parseurl [...]/libcurl/src/lib/urlapi.c:1176:16 + #3 0x569076a06fc5 in parseurl_and_replace [...]/libcurl/src/lib/urlapi.c: + 1342:12 + [...] - This doesn't rule out HTTP/2 support with Hyper, but it may take a - redesign of the Hyper integration in order to make things work. + Signed-off-by: Louis Solofrizzo + Closes #12995 - Closes #12191 +Stefan Eissing (26 Feb 2024) -Jay Satiro (20 Nov 2023) +- lib: move client writer into own source -- schannel: fix unused variable warning + Refactoring of the client writer that passes the data to the + client/application's callback functions. - Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 - Reported-by: Viktor Szakats + - split out into own source cw-out.[ch] from sendf.c - Closes https://github.com/curl/curl/pull/12361 + - move tempwrite and tempcount from data->state into the context of the + client writer -Daniel Stenberg (19 Nov 2023) + - redesign the 3 tempwrite dynbufs as a linked list of dynbufs. On + paused transfers, this allows to "record" interleaved HEADER/BODY + chunks to be "played back" in the same order on unpausing. -- url: find scheme with a "perfect hash" + - keep the overall size limit of all buffered data to DYN_PAUSE_BUFFER. + On exceeding that, return CURLE_TOO_LARGE instead of + CURLE_OUT_OF_MEMORY as before. - Instead of a loop to scan over the potentially 30+ scheme names, this - uses a "perfect hash" table. This works fine because the set of schemes - is known and cannot change in a build. The hash algorithm and table size - is made to only make a single scheme index per table entry. + - add method to be called when a transfer is DONE to allow writing of + any data still buffered - The perfect hash is generated by a separate tool (scripts/schemetable.c) + - when paused, record HEADER writes exactly as they come for later + playback. HEADERs are documented to be written one-by-one. - Closes #12347 + Closes #12898 -- scripts: add schemetable.c +- urldata: move authneg bit from conn to Curl_easy - This tool generates a scheme-matching table. + - from `conn->bits.authneg` to `data->req.authneg` + - this is a property of the request about to be made + and not a property of the connection + - in multiuse connections, transfer could step on each others + toes here potentially. - It iterates over a number of different initial and shift values in order - to find the hash algorithm that needs the smallest possible table. + Closes #12949 - The generated hash function, table and table size then needs to be used - by the url.c:Curl_getn_scheme_handler() function. +- c-hyper: add header collection writer in hyper builds -Stefan Eissing (19 Nov 2023) + Closes #12880 -- vtls/vquic, keep peer name information together +- http: move headers collecting to writer - - add `struct ssl_peer` to keep hostname, dispname and sni - for a filter - - allocate `sni` for use in VTLS backend - - eliminate `Curl_ssl_snihost()` and its use of the download buffer - - use ssl_peer in SSL and QUIC filters + - add a client writer that does "push" response + headers written to the client if the headers api + is enabled + - remove special handling in sendf.c + - needs to be installed very early on connection + setup to catch CONNECT response headers - Closes #12349 + Closes #12880 -Viktor Szakats (18 Nov 2023) +- sendf: Curl_client_write(), make passed in buf const -- build: always revert `#pragma GCC diagnostic` after use +Michał Antoniak (26 Feb 2024) - Before this patch some source files were overriding gcc warning options, - but without restoring them at the end of the file. In CMake UNITY builds - these options spilled over to the remainder of the source code, - effecitvely disabling them for a larger portion of the codebase than - intended. +- lib: remove curl_mimepart object when CURL_DISABLE_MIME - `#pragma clang diagnostic` didn't have such issue in the codebase. + Remove curl_mimepart object from UserDefined structure when + CURL_DISABLE_MIME flag is active. Reduce size of UserDefined structure. - Reviewed-by: Marcel Raad - Closes #12352 + Also remove unreachable code: when CURL_DISABLE_MIME is set, httpreq can + never have HTTPREQ_POST_MIME value and the same goes for the + CURL_DISABLE_FORM_API flag and the HTTPREQ_POST_FORM value -- tidy-up: casing typos, delete unused Windows version aliases + Closes #12948 - - cmake: fix casing of `UnixSockets` to match the rest of the codebase. +kpcyrd (26 Feb 2024) - - curl-compilers.m4: fix casing in a comment. +- rustls: make curl compile with 0.12.0 - - setup-win32: delete unused Windows version constant aliases. + Closes #12989 - Reviewed-by: Marcel Raad - Closes #12351 +Daniel Stenberg (26 Feb 2024) -- keylog: disable if unused +- strtoofft: fix the overflow check - Fully disable keylog code if there is no TLS or QUIC subsystem using it. + ... to not rely on wrapping, since it is an undefined behavior that is + not what always might happen. This is in our private strtoff() parser + function, used only on platforms without a native version. - Closes #12350 + Reported-by: vulnerabilityspotter on hackerone + Closes #12990 -- cmake: add `CURL_DISABLE_BINDLOCAL` option +- libssh/libssh2: return error on too big range - To match similar autotools option. + If trying to get the range 0 - 2^63 and the remote file is 2^63 bytes or + larger. - Default is `ON`. + Fixes #12983 + Closes #12984 - Reviewed-by: Daniel Stenberg - Closes #12345 +Scott Talbert (24 Feb 2024) -- url: fix `-Wzero-length-array` with no protocols +- setopt: fix check for CURLOPT_PROXY_TLSAUTH_TYPE value - Fixes: - ``` - ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ - -Wc2x-extensions] - 178 | static const struct Curl_handler * const protocols[] = { - | ^ - ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length - -array] - ``` + Prior to this change CURLOPT_PROXY_TLSAUTH_TYPE would return + CURLE_BAD_FUNCTION_ARGUMENT on any type other than NULL. Since there is + only one type of TLS auth and it is also the default (SRP) the TLS auth + would work anyway. - Closes #12344 + Closes https://github.com/curl/curl/pull/12981 -- url: fix builds with `CURL_DISABLE_HTTP` +Jay Satiro (24 Feb 2024) - Fixes: - ``` - ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' - 456 | Curl_mime_cleanpart(data->state.formp); - | ~~~~~~~~~~~ ^ - ``` +- mprintf: fix format prefix I32/I64 for windows compilers - Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 + - Support I32 & I64 (eg: %I64d) for all Win32 builds. - Closes #12343 + Prior to this change mprintf support for the I format prefix, which is a + Microsoft extension, was dependent on the compiler used. -- http: fix `-Wunused-parameter` with no auth and no proxy + When Borland compiler support was removed in fd7ef00f the prefix was + then no longer supported for that compiler; however since it's still + possible to build with Borland I'm restoring support for the prefix in + this way. - ``` - lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] - bool proxy) - ^ - ``` + Reported-by: Paweł Witas - Reviewed-by: Marcel Raad - Closes #12338 + Fixes https://github.com/curl/curl/issues/12944 + Closes https://github.com/curl/curl/pull/12950 -Daniel Stenberg (16 Nov 2023) +Daniel Stenberg (23 Feb 2024) -- TODO: Some TLS options are not offered for HTTPS proxies +- cd2nroff: gen: make `\>` in input to render as plain '>' in output - Closes #12286 - Closes #12342 + The same (copy and pasted) fix/mistake as in gen.pl -- RELEASE-NOTES: synced +- gen: make `\>` in input to render as plain '>' in output -- duphandle: make dupset() not return with pointers to old alloced data + Reported-by: Gisle Vanem + Fixes #12977 + Closes #12978 - As the blob pointers are to be duplicated, the function must not return - mid-function with lingering pointers to the old handle's allocated data, - as that would lead to double-free in OOM situations. +Fabrice Fontaine (23 Feb 2024) - Make sure to clear all destination pointers first to avoid this risk. +- configure.ac: find libpsl with pkg-config - Closes #12337 + Find libpsl with pkg-config to avoid static build failures. -Viktor Szakats (16 Nov 2023) + Ref: http://autobuild.buildroot.org/results/1fb15e1a99472c403d0d3b1a688902f32 + e78d002 -- http: fix `-Wunused-variable` compiler warning + Signed-off-by: Fabrice Fontaine + Closes #12947 - Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. +Daniel Stenberg (23 Feb 2024) - E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + - `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + - `CURL_DISABLE_NTLM` on non-Windows. +- BUG-BOUNTY.md: clarify that the curl security team decides - ``` - ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl - e] - CURLcode result = CURLE_OK; - ^ - ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus - ed-but-set-variable] - unsigned long *availp; - ^ - ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse - d-but-set-variable] - struct auth *authp; - ^ - ``` + Closes #12975 - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 +- THANKS: add bug reporter from #740 - Fixes #12228 - Closes #12335 + Ref: https://github.com/curl/curl/issues/740 -Jay Satiro (16 Nov 2023) +Stefan Eissing (22 Feb 2024) -- tool: support bold headers in Windows +- multi: fix multi_sock handling of select_bits - - If virtual terminal processing is enabled in Windows then use ANSI - escape codes Esc[1m and Esc[22m to turn bold on and off. + - OR the event bitmask to data->state.select_bits instead of overwriting + them. They are cleared again on use. - Suggested-by: Gisle Vanem + Reported-by: 5533asdg on github + Fixes #12971 + Closes #12972 - Ref: https://github.com/curl/curl/discussions/11770 +Daniel Stenberg (22 Feb 2024) - Closes https://github.com/curl/curl/pull/12321 +- curlver: bump to 8.7.0 for next release -Viktor Szakats (15 Nov 2023) +- RELEASE-NOTES: synced -- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` +- write-out: add '%{proxy_used}' - Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + - `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, - or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while - compiling due to a missing HMAC declaration. + Returns 1 if the previous transfer used a proxy, otherwise 0. Useful to + for example determine if a `NOPROXY` pattern matched the hostname or + not. - The reason is that HMAC is required by `lib/sha256.c` which publishes - `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when - building for libssh2 v1.8.2 (2019-05-25) or older. + Extended test 970 and 972 - Make sure to compile the HMAC bits for a successful build. +- CURLINFO_USED_PROXY: return bool whether the proxy was used - Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting - them into separate sources isn't practical. + Adds test536 to verify - Fixes: - ``` - [...] - In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc - url_object.dir/Unity/unity_0_c.c:310: - ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s - truct HMAC_params' - 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { - | ^ - ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par - ams' - [...] - ``` + Closes #12719 - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 +- sha512_256: remove the cast macro, minor language/format edits - Fixes #12273 - Closes #12332 + Follow-up to cbe41d151d6a100c -Daniel Stenberg (15 Nov 2023) + Closes #12966 -- duphandle: also free 'outcurl->cookies' in error path +Stefan Eissing (20 Feb 2024) - Fixes memory-leak when OOM mid-function +- DoH: add trace configuration - Use plain free instead of safefree, since the entire struct is - freed below. + - refs #12397 where it is dicussed how to en-/disable verbose output + of DoH operations + - introducing `struct curl_trc_feat` to track a curl feature for + tracing + - adding `data->state.feat` optionally pointing to the feature a + transfer belongs to + - adding trace functions and verbosity checks on features + - using trace feature in DoH code + - documenting `doh` as feature for `--trace-config` - Remove some free calls that is already freed in Curl_freeset() + Closes #12411 - Closes #12329 +- websocket: fix curl_ws_recv() -Viktor Szakats (15 Nov 2023) + - when data arrived in several chunks, the collection into + the passed buffer always started at offset 0, overwriting + the data already there. -- config-win32: set `HAVE_SNPRINTF` for mingw-w64 + adding test_20_07 to verify fix - It's available in all mingw-w64 releases. We already pre-fill this - detection in CMake. + - debug environment var CURL_WS_CHUNK_SIZE can be used to + influence the buffer chunk size used for en-/decoding. - Closes #12325 + Closes #12945 -- sasl: fix `-Wunused-function` compiler warning +Evgeny Grin (Karlson2k) (20 Feb 2024) - In builds with disabled auths. +- digest: support SHA-512/256 - ``` - lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus - ed-function] - static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, - ^ - 1 warning generated. - ``` - Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st - ep:3:3822 + Also fix the tests. New implementation tested with GNU libmicrohttpd. + The new numbers in tests are real SHA-512/256 numbers (not just some + random ;) numbers ). - Reviewed-by: Daniel Stenberg - Closes #12326 +- tests: add SHA-512/256 unit test -- build: picky warning updates +- SHA-512/256: implement hash algorithm - - cmake: sync some picky gcc warnings with autotools. - - cmake, autotools: add `-Wold-style-definition` for clang too. - - cmake: more precise version info for old clang options. - - cmake: use `IN LISTS` syntax in `foreach()`. + Closes #12897 - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #12324 +- curl_setup.h: add curl_uint64_t internal type -Daniel Stenberg (15 Nov 2023) + The unsigned version of curl_off_t basically -- urldata: move cookielist from UserDefined to UrlState +Daniel Stenberg (20 Feb 2024) - 1. Because the value is not strictly set with a setopt option. +- docs: dist curl*.1 and install without perl - 2. Because otherwise when duping a handle when all the set.* fields are - first copied and an error happens (think out of memory mid-function), - the function would easily free the list *before* it was deep-copied, - which could lead to a double-free. + Drop docs/mk-ca-bundle.1 from the tarball. It can be generated at will. - Closes #12323 + Closes #12959 + Fixes #12921 + Reported-by: Michael Forney -Viktor Szakats (14 Nov 2023) +Stefan Eissing (20 Feb 2024) -- autotools: avoid passing `LDFLAGS` twice to libcurl +- OpenSSL QUIC: adapt to v3.3.x - autotools passes `LDFLAGS` automatically linker commands. curl's - `lib/Makefile.am` customizes libcurl linker flags. In that - customization, it added `LDFLAGS` to the custom flags. This resulted in - passing `LDFLAGS` _twice_ to the `libtool` command. + - set our idle timeout as transport parameter + - query negotiated idle timeout for connection alive checks + - query number of available bidi streams on a connection + - use write_ex2 with SSL_WRITE_FLAG_CONCLUDE to signal + EOF on last chunk write, so stream close does not + require an additional QUIC packet - Most of the time this is benign, but some `LDFLAGS` options can break - the build when passed twice. One such example is passing `.o` files, - e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for - MUSL builds. + Closes #12933 - Passing them twice resulted in duplicate symbol errors: - ``` - libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a - arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] - ld.lld-15: error: duplicate symbol: _start - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - [...] - clang: error: linker command failed with exit code 1 (use -v to see invocatio - n) - ``` +Ramiro Garcia (19 Feb 2024) - This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. - The patch was a works-for-me hack that ended up merged in curl: - https://sourceforge.net/p/curl/bugs/1217/#06ef - With the root cause remaining unclear. +- MANUAL.md: fix typo - Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring - `-L` first? This would be unusual and suggests a bug in either the - linker or in `libtool`. + Closes #12965 - The curl build does pass the list of detected libs via its own - `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` - command line, but it's the job of `libtool` to ensure that even - a peculiar linker gets the options in the expected order. Also because - autotools passes `LDFLAGS` last, making it hardly possible to pass - anything after it. +Daniel Stenberg (19 Feb 2024) - Perhaps in the 10 years since this issue, this already got a fix - upstream. +- BINDINGS: add mcurl, the python binding - This patch deletes `LDFLAGS` from our customized libcurl options, - leaving a single copy of them as passed by autotools automatically. + Ref: #12956 + Closes #12962 - Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - Closes #12310 +- mk-ca-bundle.md: cleanups and polish -- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` + Closes #12958 - To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and - curl tool (`CURL_LDFLAGS_BIN`). +- spellcheck.yml: remove .1/.3 handling, clean all man page .md files - This makes it possible to build libcurl and curl with a single - invocation with lib- and tool-specific custom linker flags. + Since we generate all .1 and .3 files from markdown now, we can limit + the spellcheck to the markdown versions only. - Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, - controlling static/shared, incl. requesting a static curl tool (with - `-static-libtool-libs`) while building both shared and static libcurl. + Closes #12960 - curl-for-win uses the above and some more. +- libcurl-docs: cleanups - These options are already supported in `Makefile.mk`. CMake has built-in - variables for this. + CURLMOPT_SOCKETDATA.md: fix typo + CURLMOPT_TIMERDATA.md: fix typo + CURLOPT_COOKIELIST.m: quote strings + CURLOPT_PREREQFUNCTION.md: quote variable names + CURLOPT_TCP_NODELAY.md: rephrased to please spell checker + CURLOPT_WILDCARDMATCH.md: rephrased + libcurl-tutorial.md: use correct option name + curl_global_init_mem.md: quote headers + curl_easy_getinfo.md: use correct symbol names in headers + curl_global_trace.md: quote some headers + curl_ws_meta.md: quote struct field names + libcurl-env.md: quote headers - Closes #12312 +- cd2nroff: remove backticks from titles -Jay Satiro (14 Nov 2023) +- RELEASE-NOTES: synced -- tool_cb_hdr: add an additional parsing check +Stefan Eissing (18 Feb 2024) - - Don't dereference the past-the-end element when parsing the server's - Content-disposition header. +- http_chunks: fix the accounting of consumed bytes - As 'p' is advanced it can point to the past-the-end element and prior - to this change 'p' could be dereferenced in that case. + Prior to this change chunks were handled correctly although in verbose + mode libcurl could incorrectly warn of "Leftovers after chunking" even + if there were none. - Technically the past-the-end element is not out of bounds because dynbuf - (which manages the header line) automatically adds a null terminator to - every buffer and that is not included in the buffer length passed to - the header callback. + Reported-by: Michael Kaufmann - Closes https://github.com/curl/curl/pull/12320 + Fixes https://github.com/curl/curl/issues/12937 + Closes https://github.com/curl/curl/pull/12939 -Philip Heiduck (14 Nov 2023) +- file: use xfer buf for file:// transfers -- .cirrus.yml: freebsd 14 + - For file:// transfers use the multi handle's transfer buffer for + up- and downloads. - ensure curl works on latest freebsd version + Prior to this change a6c9a33 (precedes 8.6.0) changed the file:// + transfers to use a smaller stack based buffer, and that caused a + significant performance decrease in Windows. - Closes #12053 + Bug: https://github.com/curl/curl/issues/12750#issuecomment-1920103086 + Reported-by: edmcln@users.noreply.github.com -Daniel Stenberg (13 Nov 2023) + Closes https://github.com/curl/curl/pull/12932 -- easy: in duphandle, init the cookies for the new handle +Karthikdasari0423 (18 Feb 2024) - ... not the source handle. +- HTTP3.md: always run nghttp3 submodule init - Closes #12318 + - For consistency change all 'build nghttp3' commands to run submodule + init after cloning, even if the branch does not have submodules. -- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set + Follow-up to 5a4b2f93 and 4f794558. - Previously it would unconditionally use the size, which is set to -1 - when strlen is requested. + Closes https://github.com/curl/curl/pull/12928 - Updated test 544 to verify. +LeeRiva (18 Feb 2024) - Closes #12317 +- CURLOPT_POSTQUOTE.md: fix typo -- RELEASE-NOTES: synced + Closes https://github.com/curl/curl/pull/12926 -- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped +Evgeny Grin (Karlson2k) (18 Feb 2024) - Closes #12315 +- checksrc.pl: fix handling .checksrc with CRLF -- urldata: move hstslist from 'set' to 'state' + - When parsing .checksrc chomp the (CR)LF line ending. - To make it work properly with curl_easy_duphandle(). This, because - duphandle duplicates the entire 'UserDefined' struct by plain copy while - 'hstslist' is a linked curl_list of file names. This would lead to a - double-free when the second of the two involved easy handles were - closed. + Prior to this change on Windows checksrc.pl would not process the + symbols in .checksrc properly, since many git repos in Windows use auto + crlf to check out files with CRLF line endings. - Closes #12315 + Closes https://github.com/curl/curl/pull/12924 -- test1900: verify duphandle with HSTS using multiple files +Richard Levitte (18 Feb 2024) - Closes #12315 +- cmake: fix install for older CMake versions -Goro FUJI (13 Nov 2023) + - Generate the docs install list by using a foreach loop instead of + LIST:TRANSFORM since older CMake can't handle the latter. -- http: allow longer HTTP/2 request method names + Reported-by: Dan Fandrich - - Increase the maximum request method name length from 11 to 23. + Fixes https://github.com/curl/curl/issues/12920 + Closes https://github.com/curl/curl/pull/12922 - For HTTP/1.1 and earlier there's not a specific limit in libcurl for - method length except that it is limited by the initial HTTP request - limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same - and there was no specific limit. +Stefan Eissing (16 Feb 2024) - According to Internet Assigned Numbers Authority (IANA) the longest - registered method is UPDATEREDIRECTREF which is 17 characters. +- vtls: fix tls proxy peer verification - Also there are unregistered methods used by some companies that are - longer than 11 characters. + - When verifying a proxy certificate for an ip address, use the correct + ip family. - The limit was originally added by 61f52a97 but not used until fc2f1e54. + Prior to this change the "connection" ip family was used, which was not + necessarily the same. - Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml + Reported-by: HsiehYuho@users.noreply.github.com - Closes https://github.com/curl/curl/pull/12311 + Fixes https://github.com/curl/curl/issues/12831 + Closes https://github.com/curl/curl/pull/12931 -Jay Satiro (12 Nov 2023) +Dan Fandrich (15 Feb 2024) -- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does +- CI: Bump the Circle CI base Ubuntu image to the latest 20.04 - - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB - and CURLOPT_PROXY_CAINFO_BLOB docs. + The previous ones are going to be removed soon, plus the new ones + include all the fixes since then. - All the other _BLOB option docs already have the same explanation. +Jay Satiro (13 Feb 2024) - Closes https://github.com/curl/curl/pull/12277 +- transfer: improve Windows SO_SNDBUF update limit -Viktor Szakats (11 Nov 2023) + - Change the 1 second SO_SNDBUF update limit from per transfer to per + connection. -- tidy-up: dedupe Windows system libs in cmake + Prior to this change many transfers over the same connection could cause + many SO_SNDBUF updates made to that connection per second, which was + unnecessary. - Reviewed-by: Daniel Stenberg - Closes #12307 + Closes https://github.com/curl/curl/pull/12911 -Junho Choi (11 Nov 2023) +- schannel: fix hang on unexpected server close -- ci: test with latest quiche release (0.19.0) + - Treat TLS connection close (either due to a close_notify from the + server or just closed due to receiving 0) as pending data. - Closes #12180 + This is because in some cases schannel_recv knows the connection is + closed but has to return actual pending data so it can't return 0 or an + error to indicate no more data. In this case schannel_recv must be + called again, which only happens if readwrite_data sees that there is + still pending data. -- quiche: use quiche_conn_peer_transport_params() + Prior to this change if the total size of the body that libcurl expected + to receive from the server was unknown then it was possible under some + network conditions that libcurl would hang waiting to receive more data, + when in fact a close_notify alert indicating no more data would be sent + was already processed. - In recent quiche, transport parameter API is separated - with quiche_conn_peer_transport_params(). - (https://github.com/cloudflare/quiche/pull/1575) - It breaks with bulding with latest(post 0.18.0) quiche. + Fixes https://github.com/curl/curl/issues/12894 + Closes https://github.com/curl/curl/pull/12910 - Closes #12180 +Daniel Stenberg (10 Feb 2024) -Daniel Stenberg (11 Nov 2023) +- KNOWN_BUGS: FTP upload fails if remebered dir is deleted -- Makefile: generate the VC 14.20 project files at dist-time + Closes #12181 + Closes #12923 - Follow-up to 28287092cc5a6d6ef8 (#12282) +Michał Antoniak (10 Feb 2024) - Closes #12290 +- mbedtls: use mbedtls_ssl_conf_{min|max}_tls_version -Sam James (11 Nov 2023) + ... instead of the deprecated mbedtls_ssl_conf_{min|max}_version -- misc: fix -Walloc-size warnings + Closes #12905 - GCC 14 introduces a new -Walloc-size included in -Wextra which gives: +Dan Fandrich (9 Feb 2024) - ``` - src/tool_operate.c: In function ‘add_per_transfer’: - src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo - r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] - 213 | p = calloc(sizeof(struct per_transfer), 1); - | ^ - src/var.c: In function ‘addvariable’: - src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type - struct var’ with size ‘32’ [-Walloc-size] - 361 | p = calloc(sizeof(struct var), 1); - | ^ - ``` +- CI: bump to actions/cache@v4 to avoid warning - The calloc prototype is: - ``` - void *calloc(size_t nmemb, size_t size); - ``` +Evgeny Grin (Karlson2k) (9 Feb 2024) - So, just swap the number of members and size arguments to match the - prototype, as we're initialising 1 struct of size `sizeof(struct - ...)`. GCC then sees we're not doing anything wrong. +- test1165: improve pattern matching - Closes #12292 + * Fix excluded digits at the end of the symbols ('CURL_DISABLE_POP3' + was checked as 'CURL_DISABLE_POP') -Mark Gaiser (11 Nov 2023) + Closes #12903 -- IPFS: bugfixes +Dan Fandrich (9 Feb 2024) - - Fixed endianness bug in gateway file parsing - - Use IPFS_PATH in tests where IPFS_DATA was used - - Fixed typos from traling -> trailing - - Fixed broken link in IPFS.md +- scripts: Fix cijobs.pl for Azure and GHA - Follow-up to 859e88f6533f9e + The spacing in the yaml files changed. - Reported-by: Michael Kaufmann - Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 - Closes #12305 +Daniel Stenberg (9 Feb 2024) -Daniel Stenberg (11 Nov 2023) +- RELEASE-NOTES: synced -- VULN-DISCLOSURE-POLIC: remove broken link to hackerone +- TODO: use pkg-config to find libpsl - It should ideally soon not be done from hackerone anyway + Closes #12919 - Closes #12308 +- TODO: avoid nroff -Andrew Kurushin (11 Nov 2023) + Instead of adjusting roffit, skip the nroff step. -- schannel: add CA cache support for files and memory blobs + Closes #12919 - - Support CA bundle and blob caching. +Dan Fandrich (9 Feb 2024) - Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. +- Revert "CI: run Circle macOS builds on x86 for now" - Closes https://github.com/curl/curl/pull/12261 + This reverts commit 2683de3078eadc86d9b182e7417f4ee75a247e2c. + ARM resources are now available in Circle CI, so run these builds on ARM + again. This platform needs explicit paths set to libpsl and its + dependency icu4c. -Daniel Stenberg (10 Nov 2023) + Follow-up to 2683de30 -- RELEASE-NOTES: synced + Closes #12635 -Charlie C (10 Nov 2023) +Viktor Szakats (9 Feb 2024) -- cmake: option to disable install & drop `curlu` target when unused +- cmake: add warning for using TLS libraries without 1.3 support - This patch makes the following changes: - - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. - - Removes the target `curlu` when the option `BUILD_TESTING` is set to - `OFF` - to prevent it from being loaded in Visual Studio. + Closes #12900 - Closes #12287 +Daniel Stenberg (9 Feb 2024) -Kai Pastor (10 Nov 2023) +- configure: add warning for using TLS libraries without 1.3 support -- cmake: fix multiple include of CURL package + Closes #12900 - Fixes errors on second `find_package(CURL)`. This is a frequent case - with transitive dependencies: - ``` - CMake Error at ...: - add_library cannot create ALIAS target "CURL::libcurl" because another - target with the same name already exists. - ``` +Michał Antoniak (9 Feb 2024) - Test to reproduce: - ```cmake - cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher +- mbedtls: fix building when MBEDTLS_X509_REMOVE_INFO flag is defined - project(curl) + Closes #12904 - set(CURL_DIR "example/lib/cmake/CURL/") - find_package(CURL CONFIG REQUIRED) - find_package(CURL CONFIG REQUIRED) # fails +Stefan Eissing (9 Feb 2024) - add_executable(main main.c) - target_link_libraries(main CURL::libcurl) - ``` +- ftp: fix socket wait activity in ftp_domore_getsock - Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes - Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html - Ref: #12300 - Assisted-by: Harry Mallon - Closes #11913 + - when waiting on the data connection, always add the control socket to + the pollset on state STOP or let the pingpong add the socket according + to its needs. -Viktor Szakats (8 Nov 2023) + Reported-by: Fabian Vogt + Fixes #12901 + Closes #12913 -- tidy-up: use `OPENSSL_VERSION_NUMBER` +Daniel Stenberg (9 Feb 2024) - Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. - Before this patch some places used `OPENSSL_VERSION_MAJOR`. +- dist: make sure the http tests are in the tarball - Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't - define any version number in these implementations: BoringSSL, AWS-LC, - LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to - `opensslv.h`. This wasn't causing a deeper problem because the code is - looking for v3, which is only provided by OpenSSL/quictls as of now. + Fixes #12914 + Reported-by: Fabian Vogt + Closes #12917 - According to https://github.com/openssl/openssl/issues/17517, the macro - `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. +Stefan Eissing (9 Feb 2024) - Reviewed-by: Marcel Raad - Closes #12298 +- multi: add xfer_buf to multi handle -Daniel Stenberg (8 Nov 2023) + - can be borrowed by transfer during recv-write operation + - needs to be released before borrowing again + - adjustis size to `data->set.buffer_size` + - used in transfer.c readwrite_data() -- resolve.d: drop a multi use-sentence + Closes #12805 - Since the `multi:` keyword adds that message. +Daniel Stenberg (9 Feb 2024) - Reported-by: 積丹尼 Dan Jacobson - Fixes https://github.com/curl/curl/discussions/12294 - Closes #12295 +- write-out.md: clarify error handling details -- content_encoding: make Curl_all_content_encodings allocless + - it gets used even if the transfer fails - - Fixes a memory leak pointed out by Coverity - - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? - id=63947 - - Avoids unncessary allocations + - it does not cause error to be returned even if it fails - Follow-up ad051e1cbec68b2456a22661b + Closes #12909 - Closes #12289 +Stefan Eissing (8 Feb 2024) -Michael Kaufmann (7 Nov 2023) +- ftp: do lineend conversions in client writer -- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 + - remove the ftp special handling from sendf.c + - let ftp_do() add a client writer that does + the linened conversions + - change the lineend conversion to no longer + modify the passed buffer, but write smaller + chunks to the next cwriter instead. The + inefficiency of this will be mitigated once + we add output buffering for all client writes. - Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), - avoid it and use "http/1.1" instead. + Closes #12878 - This reverts commit df856cb5c9 (#10183). +- ftp: tracing improvements - Fixes #12259 - Closes #12285 + - trace socketindex for connection filters when not the first + - trace socket fd in tcp + - trace pollset adjusts in vtls -Daniel Stenberg (7 Nov 2023) + Closes #12902 -- Makefile.am: drop vc10, vc11 and vc12 projects from dist +Karthikdasari0423 (8 Feb 2024) - They are end of life products. Support for generating them remain in the - repo for a while but this change drops them from distribution. +- HTTP3.md: adjust the OpenSSL QUIC install instructions - Closes #12288 + tried installing with old steps but failed + tried with newly added setps and able to build + ``` + root@ubuntu:~/curl# ./src/curl -V + /root/curl/src/.libs/curl: /lib/x86_64-linux-gnu/libssl.so.3: version `OPENSS + L_3.2.0' not found (required by /root/curl/lib/.libs/libcurl.so.4) + root@ubuntu:~/curl# + ``` + ``` + root@ubuntu:~/curl# ./src/curl -V + curl 8.6.1-DEV (x86_64-pc-linux-gnu) libcurl/8.6.1-DEV OpenSSL/3.2.0 zlib/1.2 + .11 brotli/1.0.9 libpsl/0.21.0 nghttp3/1.1.0 OpenLDAP/2.5.16 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns + ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp + Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz + NTLM PSL SSL threadsafe TLS-SRP UnixSockets + root@ubuntu:~/curl# + ``` -David Suter (7 Nov 2023) + Closes #12896 -- projects: add VC14.20 project files +Daniel Stenberg (8 Feb 2024) - Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. - OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why - this is missing. Updated the templates to produce a VC14.20 project. - Project opens in Visual Studio 2019 as expected. +- TODO: align the TOC with the header - Closes #12282 +- docs: make sure curl.1 is included in dist tarballs -Daniel Stenberg (7 Nov 2023) + Ref: https://github.com/curl/curl/issues/12832#issuecomment-1933271873 -- curl: move IPFS code into src/tool_ipfs.[ch] + Closes #12892 - - convert ensure_trailing into ensure_trailing_slash - - strdup the URL string to own it proper - - use shorter variable names - - combine some expressions - - simplify error handling in ipfs_gateway() - - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached - - ipfs-gateway.d polish and simplification - - shorten ipfs error message + make them "synthetic" +Karthikdasari0423 (8 Feb 2024) - Closes #12281 +- HTTP3.md: remove quiche word in Openssl 3.2 -Viktor Szakats (6 Nov 2023) + Closes #12893 -- build: delete support bits for obsolete Windows compilers +Daniel Stenberg (7 Feb 2024) - - Pelles C: Unclear status, failed to obtain a fresh copy a few months - ago. Possible website is HTTP-only. ~10 years ago I left this compiler - dealing with crashes and other issues with no response on the forum - for years. It has seen some activity in curl back in 2021. - - LCC: Last stable release in September 2002. - - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned - in 2006. - - Borland C++: We dropped Borland C++ support in 2018. - - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 - (or possibly 2008) as a minimum. +- curl: when allocating variables, add the name into the struct - Closes #12222 + This saves the name from being an extra separate allocation. -- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` + Closes #12891 - We use `stdint.h` unconditionally in all places except one. These uses - are imposed by external dependencies / features. nghttp2, quic, wolfSSL - and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that - any of these features make curl require a C99 compiler. (In case of - MSVC, this means Visual Studio 2010 or newer.) +- lib582: remove code causing warning that is never run - This patch changes the single use of `stdint.h` guarded by - `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using - `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used - anywhere else, allowing to delete this feature check as well. + The previous realloc code in this code could trigger a compiler warning, + but since that code path cannot happen in normal circumstances it now + instead exits with an error message there. - Closes #12275 + Ref: #12887 + Closes #12890 -Daniel Stenberg (6 Nov 2023) +Stefan Eissing (7 Feb 2024) -- tool_operate: do not mix memory models +- vtls: revert "receive max buffer" + add test case - Make sure 'inputpath' only points to memory allocated by libcurl so that - curl_free works correctly. + - add test_05_04 for requests using http/1.0, http/1.1 and h2 against an + Apache resource that does an unclean TLS shutdown. + - revert special workarund in openssl.c for suppressing shutdown errors + on multiplexed connections + - vlts.c restore to its state before 9a90c9dd64d2f03601833a70786d485851bd1b53 - Pointed out by Coverity + Fixes #12885 + Fixes #12844 - Follow-up to 859e88f6533f9e1f890 + Closes #12848 - Closes #12280 +Daniel Stenberg (7 Feb 2024) -Stefan Eissing (6 Nov 2023) +- tests: support setting/using blank content env variables -- lib: client writer, part 2, accounting + logging + - test450: remove --config from the keywords + - test2080: change return code + - test428: add --config as a keyword + - test428: disable on Windows due to CI problems - This PR has these changes: +- curl: exit on config file parser errors - Renaming of unencode_* to cwriter, e.g. client writers - - documentation of sendf.h functions - - move max decode stack checks back to content_encoding.c - - define writer phase which was used as order before - - introduce phases for monitoring inbetween decode phases - - offering default implementations for init/write/close + Like when trying to import an environment variable that does not exist. - Add type paramter to client writer's do_write() - - always pass all writes through the writer stack - - writers who only care about BODY data will pass other writes unchanged + Also fix a bug for reading env variables when there is a default value + set. - add RAW and PROTOCOL client writers - - RAW used for Curl_debug() logging of CURLINFO_DATA_IN - - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and - Curl_pgrsSetDownloadCounter() - - remove all updates of data->req.bytecount and calls to - Curl_pgrsSetDownloadCounter() and Curl_debug() from other code - - adjust test457 expected output to no longer see the excess write + Bug: https://curl.se/mail/archive-2024-02/0008.html + Reported-by: Brett Buddin - Closes #12184 + Add test 462 to verify. -Daniel Stenberg (6 Nov 2023) + Closes #12862 -- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw +Daniel Szmulewicz (7 Feb 2024) - Closes #12278 +- CURLOPT_WRITEFUNCTION.md: typo fix -Viktor Szakats (6 Nov 2023) + The maximum amount of body data that is passed to the write + callback is defined in the curl.h header file -- rand: fix build error with autotools + LibreSSL + Closes #12889 - autotools unexpectedly detects `arc4random` because it is also looking - into dependency libs. One dependency, LibreSSL, happens to publish an - `arc4random` function (via its shared lib before v3.7, also via static - lib as of v3.8.2). When trying to use this function in `lib/rand.c`, - its protoype is missing. To fix that, curl included a prototype, but - that used a C99 type without including `stdint.h`, causing: +Daniel Stenberg (7 Feb 2024) - ``` - ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' - 37 | uint32_t arc4random(void); - | ^ - 1 error generated. - ``` +- lib: convert Curl_get_line to use dynbuf - This patch improves this by dropping the local prototype and instead - limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide - their own random source anyway. + Create the line in a dynbuf. Aborts the reading of the file on + errors. Avoids having to always allocate maximum amount from the + start. Avoids direct malloc. - The better fix would be to teach autotools to not link dependency libs - while detecting `arc4random`. + Closes #12846 - LibreSSL publishing a non-namespaced `arc4random` tracked here: - https://github.com/libressl/portable/issues/928 +- KNOWN_BUGS: unicode on Windows - Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 + Closes #11461 + Closes #12231 + Closes #12883 - Reviewed-by: Daniel Stenberg - Fixes #12257 - Closes #12274 +- tool_operate: change precedence of server Retry-After time -Daniel Stenberg (5 Nov 2023) + - When calculating the retry time, no longer allow a server's requested + Retry-After time to take precedence over a longer retry time (either + default algorithmic or user-specified). -- RELEASE-NOTES: synced + Prior to this change the server's Retry-After time took precedence over + curl's retry time in all cases, but that's not always practical for + short Retry-After times depending on how busy the server is. -- strdup: do Curl_strndup without strncpy + Bug: https://curl.se/mail/archive-2024-01/0022.html + Reported-by: Dirk Hünniger - To avoid (false positive) gcc-13 compiler warnings. + Closes https://github.com/curl/curl/pull/12871 - Follow-up to 4855debd8a2c1cb +- cmdline-docs: quote and angle bracket cleanup - Assisted-by: Jay Satiro - Reported-by: Viktor Szakats - Fixes #12258 + - make sure angle brackets are escaped + - remove a lot of superfluous double quotes + - replace several double quotes with backticks -Enno Boland (5 Nov 2023) + To make nicer-looking markdown. -- HTTP: fix empty-body warning + Closes #12884 - This change fixes a compiler warning with gcc-12.2.0 when - `-DCURL_DISABLE_BEARER_AUTH=ON` is used. +- badwords: use hostname, not host name - /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': - /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp - ty body in an 'else' statement [-Wempty-body] - 1147 | ; - | ^ + and username, filename - consistently. Fixed the patterns in + badwords.txt to catch these. - Closes #12262 + Closes #12888 -Daniel Stenberg (5 Nov 2023) +Viktor Szakats (6 Feb 2024) -- openssl: identify the "quictls" backend correctly +- cmake: fix function description in comment [ci skip] - Since vanilla OpenSSL does not support the QUIC API I think it helps - users to identify the correct OpenSSL fork in version output. The best - (crude) way to do that right now seems to be to check if ngtcp2 support - is enabled. + Closes #12879 - Closes #12270 +Daniel Stenberg (6 Feb 2024) -Mark Gaiser (5 Nov 2023) +- header.md: remove backslash, make nicer markdown -- curl: improved IPFS and IPNS URL support + - remove a leftover backslash before a dash + - use backticks for "code" strings - Previously just ipfs:// and ipns:// was supported, which is - too strict for some usecases. + Closes #12877 - This patch allows paths and query arguments to be used too. - Making this work according to normal http semantics: +- docs: add mk-ca-bundle.1 to dist - ipfs:///foo/bar?key=val - ipns:///foo/bar?key=val + ... which also makes it get built. But don't build this or curl-config.1 + if build docs is disabled. - The gateway url support is changed. - It now only supports gateways in the form of: + Closes #12875 - http:///foo/bar - http:// +Stefan Eissing (6 Feb 2024) - Query arguments here are explicitly not allowed and trigger an intended - malformed url error. +- https-proxy: use IP address and cert with ip in alt names - There also was a crash when IPFS_PATH was set with a non trailing - forward slash. This has been fixed. + - improve info logging when peer verification fails to indicate + if DNS name or ip address has been tried to match + - add test case for contacting https proxy with ip address + - add pytest env check on loaded credentials and re-issue + when they are no longer valid + - disable proxy ip address test for bearssl, since not supported there - Lastly, a load of test cases have been added to verify the above. + Ref: #12831 + Closes #12838 - Reported-by: Steven Allen - Fixes #12148 - Closes #12152 +Jiawen Geng (6 Feb 2024) -Harry Mallon (5 Nov 2023) +- docs: add necessary setup for nghttp3 -- docs: KNOWN_BUGS cleanup + Now nghttp3 has submodules + https://github.com/ngtcp2/nghttp3/blob/main/.gitmodules - * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. - Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 + Closes #12859 - * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. - Fixed in #11806 +Peter Krefting (6 Feb 2024) - * Remove mention of aws-sigv4 query empty value problems +- version: allow building with ancient libpsl - * Remove mention of aws-sigv4 missing amz-content-sha256 - Fixed in #9995 + The psl_check_version_number() API was added in libpsl 0.11.0. CentOS 7 + ships with version 0.7.0 which lacks this API. Revert to using the older + versioning API if we detect an old libpsl version. -- http_aws_sigv4: canonicalise valueless query params + Follow-up to 72bd88adde0e8cf6e63644a7d6df1da01a399db4 + Bug: https://curl.se/mail/archive-2024-02/0004.html + Reported-by: Scott Mutter + Closes #12872 - Fixes #8107 - Closes #12244 +Daniel Stenberg (6 Feb 2024) -Michael Kaufmann (4 Nov 2023) +- TODO: Support latest rustls -- docs: preserve the modification date when copying the prebuilt man page + Closes #12737 + Closes #12874 - The previously built man page "curl.1" must be copied with the original - modification date, otherwise the man page is never updated. +- docs: make curldown do angle brackets like markdown - This fixes a bug that has been introduced with commit 2568441cab. + Make sure we use \< and \> in markdown all over so that it renders + correctly, on GitHub and elsewhere. cd2nroff now outputs a warning if it + finds an unescaled angle bracket. - Reviewed-by: Dan Fandrich - Reviewed-by: Daniel Stenberg + Ref: #12854 + Closes #12869 - Closes #12199 +- docs: fix the --disable-docs for autotools -Daniel Stenberg (4 Nov 2023) + Follow-up to 541321507e386 -- docs: remove bold from some man page SYNOPSIS sections + Closes #12870 - In the name of consistency +- RELEASE-NOTES: synced - Closes #12267 +- libcurl-security.md: Active FTP passes on the local IP address -- openssl: two multi pointer checks should probably rather be asserts + Reported-by: Harry Sintonen + Closes #12867 - ... so add the asserts now and consider removing the dynamic checks in a - future. +Stefan Eissing (5 Feb 2024) - Ref: #12261 - Closes #12264 +- configure: do not link with nghttp3 unless necessary -boilingoden (4 Nov 2023) + Fixes #12833 + Closes #12864 + Reported-by: Ryan Carsten Schmidt -- docs: add supported version for the json write-out +Daniel Stenberg (5 Feb 2024) - xref: https://curl.se/changes.html#7_70_0 +- THANKS: add Dmitry Tretyakov - Closes #12266 + ... since I missed to give credit to the report in the fix of #12861 diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index 7910255eda..cb276cd0bd 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -68,6 +68,7 @@ Aleksandar Milivojevic Aleksander Mazur Aleksandr Krotov Aleksey Tulinov +alervd on github Ales Mlakar Ales Novak Alessandro Ghedini @@ -93,6 +94,7 @@ Alex Nichols Alex Potapenko Alex Rousskov Alex Samorukov +Alex Snast Alex Suykov Alex Vinnik Alex Xu @@ -109,6 +111,7 @@ Alexander Krasnostavsky Alexander Lazic Alexander Pepper Alexander Peslyak +Alexander Shtuchkin Alexander Sinditskiy Alexander Traud Alexander V. Tikhonov @@ -209,6 +212,8 @@ Andrés García Andy Alt Andy Cedilnik Andy Fiddaman +Andy Pan +Andy Reitz Andy Serpa Andy Stamp Andy Tsouladze @@ -270,6 +275,7 @@ AtariDreams on github Ates Goral atjg on github Augustus Saunders +Aurélien Pierre Austin Green av223119 on github Avery Fay @@ -340,6 +346,7 @@ Bertrand Demiddelaer Bertrand Simonnet beslick5 on github Bevan Weiss +Bhanu Prakash Bill Doyle Bill Egert Bill Hoffman @@ -404,6 +411,7 @@ Brian Green Brian Inglis Brian J. Murrell Brian Lund +brian m. carlson Brian Nixon Brian Prodoehl Brian R Duffy @@ -497,6 +505,7 @@ Christian Fillion Christian Grothoff Christian Heimes Christian Hesse +Christian Heusel Christian Hägele Christian Krause Christian Kurz @@ -534,6 +543,7 @@ Clifford Wolf Clint Clayton Cloudogu Siebels Clément Notin +CMD cmfrolick on github codesniffer13 on github Cody Jones @@ -714,6 +724,7 @@ Denis Laxalde Denis Ollier Dennis Clarke Dennis Felsing +dependabot[bot] Derek Higgins Derzsi Dániel Desmond O. Chang @@ -768,16 +779,19 @@ Dmitry S. Baikov Dmitry Tretyakov Dmitry Wagin dnivras on github +dogma Dolbneff A.V Domen Kožar Domenico Andreoli Dominick Meglio Dominik Hölzl Dominik Klemba +Dominik Piątkowski Dominik Thalhammer Dominique Leuenberger Don J Olmstead Dongliang Mu +Dorian Craps Doron Behar Doug Kaufman Doug Porter @@ -838,6 +852,7 @@ Eli Schwartz Elia Tufarolo Elliot Killick Elliot Saba +Elliott Balsley Ellis Pritchard Elmira A Semenova Elms @@ -1071,6 +1086,7 @@ Grigory Entin Grisha Levit Guenole Bescon Guido Berhoerster +Guilherme Puida Guillaume Algis Guillaume Arluison guitared on github @@ -1236,6 +1252,7 @@ Jakub Bochenski Jakub Jelen Jakub Wilk Jakub Zakrzewski +James Abbatiello James Atwill James Brown James Bursa @@ -1293,6 +1310,7 @@ Javier Navarro Javier Sixto Jay Austin Jay Dommaschk +Jay Guerette Jay Wu Jayesh A Shah Jaz Fresh @@ -1354,6 +1372,7 @@ Jesse Noller Jesse Tan jethrogb on github jhoyla on github +Jiang Wenjian Jiawen Geng Jie He Jiehong on github @@ -1460,6 +1479,7 @@ Jonatan Lander Jonatan Vela Jonathan Cardoso Machado Jonathan Hseu +Jonathan Matthews Jonathan Moerman Jonathan Nieder Jonathan Perkin @@ -1557,6 +1577,7 @@ Kari Pahula Karl Chen Karl Moerder Karol Pietrzak +kartatz Kartatz on Github Karthikdasari0423 Karthikdasari0423 on github @@ -1566,6 +1587,7 @@ Katie Wang Katsuhiko YOSHIDA Kazuho Oku kchow-FTNT on github +Keerthi Timmaraju Kees Cook Kees Dekker Keitagit-kun on github @@ -1675,6 +1697,7 @@ Lawrence Wagerfield Leah Neukirchen Lealem Amedie Leandro Coutinho +Lee Li LeeRiva Legoff Vincent Lehel Bernadt @@ -1748,6 +1771,7 @@ Luke Amery Luke Call Luke Dashjr Luke Granger-Brown +Luke Hamburg Lukáš Zaoral luminixinc on github Luo Jinghua @@ -1865,6 +1889,7 @@ Martin Jansen Martin Kammerhofer Martin Kepplinger Martin Lemke +Martin Peck Martin Schmatz Martin Skinner Martin Staael @@ -1874,6 +1899,7 @@ Martin V Martin Vejnár Martin Waleczek Martin Ågren +martinevsky Marty Kuhrt Maruko Masaya Suzuki @@ -1915,6 +1941,7 @@ Matthew Whitehead Matthias Bolte Matthias Gatto Matthias Naegler +Matthieu Baerts Mattias Fornander Matus Uzak Maurice Barnum @@ -2051,6 +2078,7 @@ momala454 on github Momoka Yamamoto MonkeybreadSoftware on github moohoorama on github +Morgan Willcock Morten Minde Neergaard Mostyn Bramley-Moore Moti Avrahami @@ -2175,6 +2203,7 @@ Oliver Schindler Oliver Urbann oliverpool on github Olivier Berger +Olivier Bonaventure Olivier Brunel Omar Ramadan omau on github @@ -2429,6 +2458,7 @@ Renaud Lehoux Rene Bernhardt Rene Rebe renovate[bot] +renovate[bot] Reuven Wachtfogel RevaliQaQ on github Reza Arbab @@ -2615,6 +2645,7 @@ Sascha Zengler Satadru Pramanik Satana de Sant'Ana Saul good +saurabhsingh-dev on github Saurav Babu sayrer on github SBKarr on github @@ -2634,6 +2665,7 @@ Sean McArthur Sean Miller Sean Molenaar Sebastiaan van Erk +Sebastian Andersson Sebastian Haglund Sebastian Mundry Sebastian Neubauer @@ -2663,6 +2695,7 @@ Sergio Mijatovic Sergio-IME on github sergio-nsk on github Serj Kalichev +Sertonix SerusDev on github Seshubabu Pasam Seth Mos @@ -2946,6 +2979,7 @@ Tommy Chiang Tommy Odom Tommy Petty Tommy Tam +tomy2105 on github Ton Voon Toni Moreno Tony Kelman @@ -3040,6 +3074,7 @@ Volker Schmid Vsevolod Novikov vshmuk on hackerone vulnerabilityspotter on hackerone +vuonganh1993 on github vvb2060 vvb2060 on github Vyron Tsingaras @@ -3135,6 +3170,7 @@ Yves Arrouye Yves Lejeune YX Hao z2-2z on github +z2_ z2_ on hackerone Zachary Seguin Zdenek Pavlas @@ -3173,5 +3209,7 @@ zzq1015 on github 加藤郁之 南宫雪珊 左潇峰 +李四 梦终无痕 積丹尼 Dan Jacobson +罗朝辉 -- cgit v1.2.3