From b7dfc6fda6f6b461f45a2ce457911bf128160208 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Wed, 11 Oct 2023 18:36:33 +0300 Subject: libcurl: update to 8.4.0 --- libs/libcurl/docs/CHANGES | 11626 +++++++++++++++++++++++--------------------- libs/libcurl/docs/THANKS | 22 +- 2 files changed, 6115 insertions(+), 5533 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 45791b0d92..8d56bf8e28 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,8978 +6,9540 @@ Changelog -Version 8.3.0 (13 Sep 2023) +Version 8.4.0 (11 Oct 2023) -Daniel Stenberg (13 Sep 2023) +Daniel Stenberg (11 Oct 2023) -- RELEASE-NOTES: syn ced +- RELEASE-NOTES: synced - curl 8.3.0 release +- THANKS: add contributors from 8.4.0 -- THANKS: contributors from 8.3.0 +Jay Satiro (11 Oct 2023) -Thorsten Klein (12 Sep 2023) +- socks: return error if hostname too long for remote resolve -- cmake: set SIZEOF_LONG_LONG in curl_config.h + Prior to this change the state machine attempted to change the remote + resolve to a local resolve if the hostname was longer than 255 + characters. Unfortunately that did not work as intended and caused a + security issue. - in order to support 32bit builds regarding wolfssl CTC_SETTINGS + Bug: https://curl.se/docs/CVE-2023-38545.html - Closes #11839 +Stefan Eissing (10 Oct 2023) -Jay Satiro (12 Sep 2023) +- CI: remove slowed-network tests -- curl_ngtcp2: fix error message + - remove these tests as they are currently not reliable in our CI + setups. -- http_aws_sigv4: handle no-value user header entries + curl handles the test cases, but CI sometimes fails on these due to + additional conditions. Rather than mix them in, an additional CI job + will be added in the future that is specific to them. - - Handle user headers in format 'name:' and 'name;' with no value. + Closes https://github.com/curl/curl/pull/12075 - The former is used when the user wants to remove an internal libcurl - header and the latter is used when the user actually wants to send a - no-value header in the format 'name:' (note the semi-colon is converted - by libcurl to a colon). +Jay Satiro (10 Oct 2023) - Prior to this change the AWS header import code did not special case - either of those and the generated AWS SignedHeaders would be incorrect. +- libcurl-env-dbg.3: move debug variables from libcurl-env.3 - Reported-by: apparentorder@users.noreply.github.com + - Move documentation of libcurl environment variables used only in debug + builds from libcurl-env into a separate document libcurl-env-dbg. - Ref: https://curl.se/docs/manpage.html#-H + - Document more debug environment variables. - Fixes https://github.com/curl/curl/issues/11664 - Closes https://github.com/curl/curl/pull/11668 + Previously undocumented or missing a description: -Dan Fandrich (11 Sep 2023) + CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, + CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, + CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, + CURL_TIME. -- CI: run pytest with the -v option + Closes https://github.com/curl/curl/pull/11811 - This lists of the test cases being run so it can be tracked over time. +Dan Fandrich (9 Oct 2023) - Closes #11824 +- test670: increase the test timeout -Daniel Stenberg (11 Sep 2023) + This should make it more immune to loaded servers. -- HTTP3: the msquic backend is not functional + Ref: #11328 - I ask that we do not submit bugs for this backend just yet as we know it - does not fully work. +Stefan Eissing (9 Oct 2023) - Closes #11831 - Closes #11819 +- MQTT: improve receive of ACKs -- aws_sigv4: the query canon code miscounted URL encoded input + - add `mq->recvbuf` to provide buffering of incomplete + ACK responses + - continue ACK reading until sufficient bytes available + - fixes test failures on low network receives - Added some extra ampersands to test 439 to verify "blank" query parts + Closes #12071 - Follow-up to fc76a24c53b08cdf +Viktor Szakats (9 Oct 2023) - Closes #11829 +- quic: fix BoringSSL build -vvb2060 (11 Sep 2023) + Add guard around `SSL_CTX_set_ciphersuites()` use. -- quic: don't set SNI if hostname is an IP address + Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 - We already do this for TLS connections. + Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 - RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in - "HostName". + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes #12067 - Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 +Stefan Eissing (9 Oct 2023) - Fixes https://github.com/curl/curl/issues/11827 - Closes https://github.com/curl/curl/pull/11828 +- test1540: improve reliability -Daniel Stenberg (10 Sep 2023) + - print that bytes have been received on pausing, but not how many -- RELEASE-NOTES: synced + Closes #12069 -Benoit Pierre (10 Sep 2023) +- test2302: improve reliability -- configure: fix `HAVE_TIME_T_UNSIGNED` check + - make result print collected write data, unless + change in meta flags is detected + - will show same result even when data arrives via + several writecb invocations - The syntax was incorrect (need a proper main body), and the test - condition was wrong (resulting in a signed `time_t` detected as - unsigned). + Closes #12068 - Closes #11825 +Daniel Stenberg (9 Oct 2023) -Daniel Stenberg (9 Sep 2023) +- curl_easy_pause: set "in callback" true on exit if true -- THANKS-filter: pszlazak on github + Because it might have called another callback in the mean time that then + set the bit FALSE on exit. -pszlazak (9 Sep 2023) + Reported-by: Jay Satiro + Fixes #12059 + Closes #12061 -- include.d: explain headers not printed with --fail before 7.75.0 +Viktor Szakats (8 Oct 2023) - Prior to 7.75.0 response headers were not printed if -f/--fail was used - and an error was reported by server. This was fixed in ab525c0 - (precedes 7.75.0). +- h3: add support for ngtcp2 with AWS-LC builds - Closes #11822 + ``` + curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- + LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps + mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss + Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi + SSL NTLM SSL threadsafe UnixSockets + ``` -Daniel Stenberg (8 Sep 2023) + Also delete an obsolete GnuTLS TODO and update the header comment in + `FindNGTCP2.cmake`. -- http_aws_sigv4: skip the op if the query pair is zero bytes + Reviewed-by: Daniel Stenberg + Closes #12066 - Follow-up to fc76a24c53b08cdf +- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros - Spotted by OSS-Fuzz + Syncing this up with CMake. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 - Closes #11823 + Source code uses the built-in `OPENSSL_IS_AWSLC` and + `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is + necessary from the build tools. -- cmdline-docs: use present tense, not future + The one use of `HAVE_BORINGSSL` in the source turned out to be no longer + necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 - + some smaller cleanups + autotools detects this anyway for display purposes. + CMake detects this to decide whether to use the BoringSSL-specific + crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection + result just yet (planned in #12066). - Closes #11821 + Ref: #11964 -- cmdline-docs: make sure to phrase it as "added in ...." + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12065 - References to things that were added or changed in a specific version - should be specified as "(added in [version]) for two reasons: +Marc Hoersken (8 Oct 2023) - 1 - consistency +- CI: move distcheck job from Azure Pipelines to GitHub Actions - 2 - to allow gen.pl to strip them out if deemed referring to too old - versions + This will allow for more trigger excludes within Azure Pipelines. - Closes #11821 + Also fixes seemingly broken check with scripts/installcheck.sh. + Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 -Jay Satiro (8 Sep 2023) + Assisted-by: Philip Heiduck + Closes #9532 -- docs: mark --ssl-revoke-best-effort as Schannel specific +Daniel Stenberg (8 Oct 2023) - Closes https://github.com/curl/curl/pull/11760 +- url: fall back to http/https proxy env-variable if ws/wss not set -Nathan Moinvaziri (8 Sep 2023) + Reported-by: Craig Andrews + Fixes #12031 + Closes #12058 -- schannel: fix ordering of cert chain info +Stefan Eissing (8 Oct 2023) - - Use CERT_CONTEXT's pbCertEncoded to determine chain order. +- cf-socket: simulate slow/blocked receives in debug - CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains - end-entity/server certificate in pbCertEncoded. We can use this pointer - to determine the order of certificates when enumerating hCertStore using - CertEnumCertificatesInStore. + add 2 env variables for non-UDP sockets: + 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly + should return EAGAIN + 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket - This change is to help ensure that the ordering of the certificate chain - requested by the user via CURLINFO_CERTINFO has the same ordering on all - versions of Windows. + Closes #12035 - Prior to this change Schannel certificate order was reversed in 8986df80 - but that was later reverted in f540a39b when it was discovered that - Windows 11 22H2 does the reversal on its own. +- http2: refused stream handling for retry - Ref: https://github.com/curl/curl/issues/9706 + - answer HTTP/2 streams refused via a GOAWAY from the server to + respond with CURLE_RECV_ERROR in order to trigger a retry + on another connection - Closes https://github.com/curl/curl/pull/11632 + Reported-by: black-desk on github + Ref #11859 + Closes #12054 -Chris Talbot (8 Sep 2023) +Jay Satiro (8 Oct 2023) -- digest: Use hostname to generate spn instead of realm +- CURLOPT_DEBUGFUNCTION.3: warn about internal handles - In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 + - Warn that the user's debug callback may be called with the handle + parameter set to an internal handle. - digest-uri-value should be serv-type "/" host , where host is: + Without this warning the user may assume that the only handles their + debug callback receives are the easy handles on which they set + CURLOPT_DEBUGFUNCTION. - The DNS host name or IP address for the service requested. The - DNS host name must be the fully-qualified canonical name of the - host. The DNS host name is the preferred form; see notes on server - processing of the digest-uri. + This is a follow-up to f8cee8cc which changed DoH handles to inherit + the debug callback function set in the user's easy handle. As a result + those handles are now passed to the user's debug callback function. - Realm may not be the host, so we must specify the host explicitly. + Closes https://github.com/curl/curl/pull/12034 - Note this change only affects the non-SSPI digest code. The digest code - used by SSPI builds already uses the hostname to generate the spn. +- url: fix typo - Ref: https://github.com/curl/curl/issues/11369 +Daniel Stenberg (8 Oct 2023) - Closes https://github.com/curl/curl/pull/11395 +- test458: verify --expand-output, expanding a file name accepting option -Daniel Stenberg (7 Sep 2023) + Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) -- docs: remove use of the word 'very' +- tool_getparam: accept variable expansion on file names too - It is mostly superfluous. proselint would complain. + Reported-by: PBudmark on github + Fixes #12048 + Closes #12055 - Closes #11818 +- RELEASE-NOTES: synced -- curl_multi_remove_handle.3: clarify what happens with connection +- multi: do CURLM_CALL_MULTI_PERFORM at two more places - Closes #11817 + ... when it does a state transition but there is no particular socket or + timer activity. This was made apparent when commit b5bb84c removed a + superfluous timer expiry. -- RELEASE-NOTES: synced + Reported-by: Dan Fandrich. + Fixes #12033 + Closes #12056 -- test439: verify query canonization for aws-sigv4 +Viktor Szakats (7 Oct 2023) -- tool_operate: make aws-sigv4 not require TLS to be used +- GHA/linux: mbedtls 3.5.0 + minor dep bumps - Maybe not used too often, but we want it for testing and it should work. + Closes #12057 -- http_aws_sigv4: canonicalize the query +Dan Fandrich (7 Oct 2023) - Percent encoding needs to be done using uppercase, and most - non-alphanumerical must be percent-encoded. +- CI: bump OpenLDAP package version on FreeBSD - Fixes #11794 - Reported-by: John Walker - Closes #11806 + The old one is no longer available. -Wyatt O'Day (7 Sep 2023) +Marc Hoersken (7 Oct 2023) -- lib: add ability to disable auths individually +- docs/libcurl/opts/Makefile.inc: add missing manpage files - Both with configure and cmake + Detected with #9532 - Closes #11490 +Dan Fandrich (7 Oct 2023) -Stefan Eissing (7 Sep 2023) +- tests: fix a race condition in ftp server disconnect -- ngtcp2: fix handling of large requests + If a client disconnected and reconnected quickly, before the ftp server + had a chance to respond, the protocol message/ack (ping/pong) sequence + got out of sync, causing messages sent to the old client to be delivered + to the new. A disconnect must now be acknowledged and intermediate + requests thrown out until it is, which ensures that such synchronization + problems can't occur. This problem could affect ftp, pop3, imap and smtp + tests. - - requests >64K are send in parts to the filter - - fix parsing of the request to assemble it correctly - from several sends - - open a QUIC stream only when the complete request has - been collected + Fixes #12002 + Closes #12049 - Closes #11815 +Viktor Szakats (7 Oct 2023) -- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before +- appveyor: bump mingw-w64 job to gcc 13 (was: 8) - - we delay loading the x509 store to shorten the handshake time. - However an application callback installed via CURLOPT_SSL_CTX_FUNCTION - may need to have the store loaded and try to manipulate it. - - load the x509 store before invoking the app callback + This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). + Adding a modern gcc version to the tests. - Fixes #11800 - Reported-by: guoxinvmware on github - Cloes #11805 + (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 + finished in 32, 35, 34 minutes in the 3 test runs so far.) -Daniel Stenberg (7 Sep 2023) + It also adds a modern CMake version and OS env to our mingw-w64 builds. -- krb5: fix "implicit conversion loses integer precision" warnings + Closes #12051 - conversions to/from enum and unsigned chars +David Benjamin (6 Oct 2023) - Closes #11814 +- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR -Stefan Eissing (7 Sep 2023) + While the struct is still public in OpenSSL, there is a (somewhat + inconvenient) accessor. Use it to remain compatible if it becomes opaque + in the future. -- pytest: improvements + Closes #12038 - - set CURL_CI for pytest runs in CI environments - - exclude timing sensitive tests from CI runs - - for failed results, list only the log and stat of - the failed transfer +Daniel Stenberg (6 Oct 2023) - - fix type in http.c comment +- curl_easy_pause.3: mention it works within callbacks - Closes #11812 + Reported-by: Maxim Dzhura + Bug: https://curl.se/mail/lib-2023-10/0010.html + Closes #12046 -- CI: move on to ngtcp2 v0.19.1 +- curl_easy_pause.3: mention h2/h3 buffering - Closes #11809 + Asked-by: Maxim Dzhura + Ref: https://curl.se/mail/lib-2023-10/0011.html -Dan Fandrich (5 Sep 2023) + Closes #12045 -- CI: run Circle macOS builds on x86 for now +Viktor Szakats (6 Oct 2023) - The ARM machines aren't ready for us and requesting them now causes - warnings e-mails to be sent to some PR pushers. +- cmake: re-add missed C89 headers for specific detections - Ref: #11771 + We removed C89 `setjmp.h` and `signal.h` detections and excluded them + from the global header list we use when detecting functions [1]. Then + missed to re-add these headers to the specific functions which need + them to be detected [2]. Fix this omission in this patch. -Viktor Szakats (5 Sep 2023) + [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 + [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 -- http3: adjust cast for ngtcp2 v0.19.0 + Closes #12043 - ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` - an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. +Daniel Stenberg (6 Oct 2023) - Fixes: - ``` - ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in - teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned - char') [-Wimplicit-int-conversion] - pi.ecn = (uint32_t)ecn; - ~ ^~~~~~~~~~~~~ - ``` +- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE - Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our - docs and CI. + Since there is nothing to wait for there. Avoids the test 1233 hang + reported in #12033. - Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 - 782894a3 - Ref: https://github.com/ngtcp2/ngtcp2/pull/877 - Closes #11798 + Reported-by: Dan Fandrich + Closes #12042 -Stefan Eissing (5 Sep 2023) +Dan Fandrich (5 Oct 2023) -- http: fix sending of large requests +- test1903: actually verify the cookies after the test - - refs #11342 where errors with git https interactions - were observed - - problem was caused by 1st sends of size larger than 64KB - which resulted in later retries of 64KB only - - limit sending of 1st block to 64KB - - adjust h2/h3 filters to cope with parsing the HTTP/1.1 - formatted request in chunks + The test otherwise could do just about anything (except leak memory in + debug mode) and its bad behaviour wouldn't be detected. Now, check the + resulting cookie file to ensure the cookies are still there. - - introducing Curl_nwrite() as companion to Curl_write() - for the many cases where the sockindex is already known + Closes #12041 - Fixes #11342 (again) - Closes #11803 +- test: add missing s -- pytest: fix check for slow_network skips to only apply when intended + The tests will otherwise fail if curl has them disabled. - Closes #11801 +- test1906: set a lower timeout since it's hit on Windows -Daniel Stenberg (5 Sep 2023) + msys2 builds actually hit the connect timeout in normal operation, so + lower the timeout from 5 minutes to 5 seconds to reduce test time. -- curl_url_get/set.3: add missing semicolon in SYNOPSIS + Ref: #11328 + Closes #12036 -- CURLOPT_URL.3: explain curl_url_set() uses the same parser +Daniel Stenberg (5 Oct 2023) -- CURLOPT_URL.3: add two URL API calls in the see-also section +- RELEASE-NOTES: synced -Dan Fandrich (4 Sep 2023) +Jay Satiro (5 Oct 2023) -- CI: add a 32-bit i686 Linux build +- idn: fix WinIDN null ptr deref on bad host - This is done by cross-compiling under regular x86_64 Linux. Since the - kernel offers backwards compatibility, the binaries can be tested as - normal. + - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from + UTF-8 to UTF-16. - Closes #11799 + Prior to this change a failed conversion erroneously returned CURLE_OK + which meant 'decoded' pointer (what would normally point to the + punycode) would not be written to, remain NULL and be dereferenced + causing an access violation. -- tests: fix a type warning on 32-bit x86 + Closes https://github.com/curl/curl/pull/11983 -Viktor Szakats (4 Sep 2023) +Dan Fandrich (4 Oct 2023) -- tests: delete stray `.orig` file +- tests: close the shell used to start sshd - Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f - Closes #11797 + This shell isn't needed once sshd starts, so use "exec" so it doesn't + stick around. -Daniel Stenberg (4 Sep 2023) + Closes #12032 -- RELEASE-NOTES: synced +Daniel Stenberg (4 Oct 2023) -Viktor Szakats (4 Sep 2023) +- base64: also build for curl -- lib: silence compiler warning in inet_ntop6 + Since the tool itself now uses the base64 code using the curlx way, it + needs to build also when the tool needs it. Starting now, the tool build + defines BULDING_CURL to allow lib-side code to use it. - ``` - ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her - e [-Wcomma] - cur.base = i, cur.len = 1; - ^ - ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin - g - cur.base = i, cur.len = 1; - ^~~~~~~~~~~~ - (void)( ) - ``` + Follow-up to 2e160c9c6525 - Closes #11790 + Closes #12010 -Daniel Stenberg (4 Sep 2023) +Eduard Strehlau (4 Oct 2023) -- transfer: also stop the sending on closed connection +- tests: Fix zombie processes left behind by FTP tests. - Previously this cleared the receiving bit only but in some cases it is - also still sending (like a request-body) when disconnected and neither - direction can continue then. + ftpserver.pl correctly cleans up spawned server processes, + but forgets to wait for the shell used to spawn them. + This is barely noticeable during a normal testrun, + but causes process exhaustion and test failure + during a complete torture run of the FTP tests. - Fixes #11769 - Reported-by: Oleg Jukovec - Closes #11795 + Fixes #12018 + Closes #12020 -John Bampton (4 Sep 2023) +Dan Fandrich (4 Oct 2023) -- docs: change `sub-domain` to `subdomain` +- github/labeler: improve labeler matches - https://en.wikipedia.org/wiki/Subdomain +- test574: add a timeout to the test - Closes #11793 + This one hangs occasionally, so this will speed up a test run and allow + logs to be seen when it does. -Stefan Eissing (4 Sep 2023) + Closes #12025 -- multi: more efficient pollfd count for poll +- tests: propagate errors in libtests - - do not use separate pollfds for sockets that have POLLIN+POLLOUT + Use the test macros to automatically propagate some errors, and check + and log others while running the tests. This can help in debugging + exactly why a test has failed. - Closes #11792 +- tests: set --expect100-timeout to improve test reliability -- http2: polish things around POST + On an overloaded server, the default 1 second timeout can go by without + the test server having a chance to respond with the expected headers, + causing tests to fail. Increase the 1 second timeout to 99 seconds so + this failure mode is no longer a problem on test 1129. Some other tests + already set a high value, but make them consistently 99 seconds so if + something goes wrong the test is stalled for less time. - - added test cases for various code paths - - fixed handling of blocked write when stream had - been closed inbetween attempts - - re-enabled DEBUGASSERT on send with smaller data size + Ref: #11328 - - in debug builds, environment variables can be set to simulate a slow - network when sending data. cf-socket.c and vquic.c support - * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be - answered with a EAGAIN. TCP/UNIX sockets. - This is chosen randomly. - * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written - to the network. TCP/UNIX sockets. - Example: 80 means a send with 1000 bytes would only send 800 - This is applied to every send. - * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be - answered with EAGAIN. QUIC only. - This is chosen randomly. +- CI: ignore the "flaky" and "timing-dependent" test results in CMake - Closes #11756 + This was already done for automake builds but CMake builds were missed. + Test 1086 actually causes the test harness to crash with: -Daniel Stenberg (4 Sep 2023) + Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec + ts/curl/tests/ftpserver.pl line 527 -- docs: add curl_global_trace to some SEE ALSO sections + Rather than fix it now, this change leaves test 1086 entirely skipped on + those builds that show this problem. - Closes #11791 + Follow-up to 589dca761 -- os400: fix checksrc nits + Ref: #11865 - Closes #11789 +Viktor Szakats (4 Oct 2023) -Nicholas Nethercote (3 Sep 2023) +- cmake: improve OpenLDAP builds -- hyper: remove `hyptransfer->endtask` + - cmake: detect OpenLDAP based on function `ldap_init_fd`. + autotools does this. autotools also publishes this detection result + in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source + doesn't use this value. (it might need to be remove-listed in + `scripts/cmp-config.pl` for future OpenLDAP test builds.) + This also deletes existing self-declaration method via the + CMake-specific `CURL_USE_OPENLDAP` configuration. - `Curl_hyper_stream` needs to distinguish between two kinds of - `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and - (b) background tasks that hyper produces. It does this by recording the - address of any `foreach` task in `hyptransfer->endtask` before pushing - it into the executor, and then comparing that against the address of - tasks later polled out of the executor. + - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. + Like autotools does. This fixes a long list of these warnings: + ``` + /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i + s not defined, evaluates to 0 [-Wundef] + ``` - This works right now, but there is no guarantee from hyper that the - addresses are stable. `hyper_executor_push` says "The executor takes - ownership of the task, which should not be accessed again unless - returned back to the user with `hyper_executor_poll`". That wording is a - bit ambiguous but with my Rust programmer's hat on I read it as meaning - the task returned with `hyper_executor_poll` may be conceptually the - same as a task that was pushed, but that there are no other guarantees - and comparing addresses is a bad idea. + - cmake: delete LDAP TODO comment no longer relevant. - This commit instead uses `hyper_task_set_userdata` to mark the `foreach` - task with a `USERDATA_RESP_BODY` value which can then be checked for, - removing the need for `hyptransfer->endtask`. This makes the code look - more like that hyper C API examples, which use userdata for every task - and never look at task addresses. + Also: - Closes #11779 + - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature + detection functions. -Dave Cottlehuber (3 Sep 2023) + Ref: #11964 (effort to sync cmake detections with autotools) -- ws: fix spelling mistakes in examples and tests + Closes #12024 - Closes #11784 +- cmake: fix unity builds for more build combinations -Daniel Stenberg (3 Sep 2023) + By using unique static function/variable names in source files + implementing these interfaces. -- tool_filetime: make -z work with file dates before 1970 + - OpenLDAP combined with any SSH backend. - Fixes #11785 - Reported-by: Harry Sintonen - Closes #11786 + - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. -Dan Fandrich (1 Sep 2023) + Closes #12027 -- build: fix portability of mancheck and checksrc targets +Daniel Stenberg (4 Oct 2023) - At least FreeBSD preserves cwd across makefile lines, so rules - consisting of more than one "cd X; do_something" must be explicitly run - in a subshell to avoid this. This problem caused the Cirrus FreeBSD - build to fail when parallel make jobs were enabled. +- tests: remove leading spaces from some tags -- CI: adjust labeler match patterns for new & obsolete files + The threee tags ``, `` and `` were frequently used + with a leading space that this removes. The reason this habbit is so + widespread in testcases is probably that they have been copy and pasted. -- configure: trust pkg-config when it's used for zlib + Hence, fixing them all now might curb this practice from now on. - The library flags retrieved from pkg-config were later thrown out and - harded-coded, which negates the whole reason to use pkg-config. - Also, previously, the assumption was made that --libs-only-l and - --libs-only-L are the full decomposition of --libs, which is untrue and - would not allow linking against a static zlib. The new approach is - better in that it uses --libs, although only if --libs-only-l returns - nothing. + Closes #12028 - Bug: https://curl.se/mail/lib-2023-08/0081.html - Reported-by: Randall - Closes #11778 +Viktor Szakats (4 Oct 2023) -Stefan Eissing (1 Sep 2023) +- GHA: bump actions/checkout -- CI/ngtcp2: clear wolfssl for when cache is ignored + Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 + Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 - Closes #11783 + Closes #12023 -Daniel Stenberg (1 Sep 2023) +- spelling: fix codespell 2.2.6 typos -- RELEASE-NOTES: synced + Closes #12019 -Nicholas Nethercote (1 Sep 2023) +Daniel Stenberg (3 Oct 2023) -- hyper: fix a progress upload counter bug +- GHA: add workflow to compare configure vs cmake outputs - `Curl_pgrsSetUploadCounter` should be a passed a total count, not an - increment. + Uses scripts/cmp-config.pl two compare two curl_config.h files, + presumbly generated with configure and cmake. It displays the + differences and filters out a lot of known lines we ignore. - This changes the failing diff for test 579 with hyper from this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 29 out of 0[LF] - ``` - to this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 40 out of 0[LF] - ``` - Presumably a step in the right direction. + The script also shows the matches that were *not* used. Possibly + subjects for removal. - Closes #11780 + Closes #11964 -Daniel Stenberg (1 Sep 2023) +- appveyor: enable test 571 -- awssiv4: avoid freeing the date pointer on error + Follow-up from 8a940fd55c175f7 / #12013 - Since it was not allocated, don't free it even if it was wrong syntax + Closes #12017 - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 +Viktor Szakats (3 Oct 2023) - Follow-up to b137634ba3adb +- build: alpha-sort source files for lib and src - Closes #11782 + Closes #12014 -Stefan Eissing (1 Sep 2023) +- cmake: delete old `HAVE_LDAP_URL_PARSE` logic -- CI: ngtcp2-linux: use separate caches for tls libraries + Left there by accident after adding proper detection for this. - allow ever changing master for wolfssl + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - Closes #11766 + Ref: #11964 (effort to sync cmake detections with autotools) -- replace `master` as wolfssl-version with recent commit + Closes #12015 -- wolfssl, use master again in CI +Stefan Eissing (3 Oct 2023) - - with the shared session update fix landed in master, it - is time to use that in our CI again +- tests: increase lib571 timeout from 3s to 30s -Nicholas Nethercote (31 Aug 2023) + - 3s is too short for our CI, making this test fail occasionally + - test usually experiences no delay run locally, so 30s wont hurt -- tests: fix formatting errors in `FILEFORMAT.md`. + Closes #12013 - Without the surrounding backticks, these tags get swallowed when the - markdown is rendered. +Viktor Szakats (3 Oct 2023) - Closes #11777 +- cmake: fix unity with Windows Unicode + TrackMemory -Viktor Szakats (31 Aug 2023) + Found the root cause of the startup crash in unity builds with Unicode + and TrackMemory enabled at the same time. -- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` + We must make sure that the `memdebug.h` header doesn't apply to + `lib/curl_multibyte.c` (as even noted in a comment there.) In unity + builds all headers apply to all sources, including `curl_multibyte.c`. + This probably resulted in an infinite loop on startup. - Allow overriding the default TLS backend via a CMake setting. + Exclude this source from unity compilation with TrackMemory enabled, + in both libcurl and curl tool. Enable unity mode for a debug Unicode + CI job to keep it tested. Also delete the earlier workaround that + fully disabled unity for affected builds. - E.g.: - `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` + Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, - schannel, secure-transport, wolfssl + Closes #11928 - The passed string is baked into the curl/libcurl binaries. - The value is case-insensitive. +- cmake: disable unity mode with Windows Unicode + TrackMemory - We added a similar option to autotools in 2017 via - c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. + "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, + aka `-DCURLDEBUG`). - TODO: Convert to lowercase to improve reproducibility. + There is an issue with memory tracking and Unicode when built in "unity" + mode, which results in the curl tool crashing right on startup, even + without any command-line option. Interestingly this doesn't happen under + WINE (at least on the system I tested this on), but consistenly happens + on real Windows machines. Crash is 0xC0000374 heap corruption. Both + shared and static curl executables are affected. - Closes #11774 + This limitation probably won't hit too many people, but it remains + a TODO to find and fix the root cause and drop this workaround. -- sectransp: fix compiler warnings + Example builds and runs: + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd + 7iwj#L313 (static) + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby + qu9c#L317 (shared) - https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# - step:3:11046 - ``` - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 - 4: warning: unused variable 'success' [-Wunused-variable] - OSStatus success; - ^ - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 - 4: warning: unused parameter 'sha256len' [-Wunused-parameter] - size_t sha256len) - ^ - ``` + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Closes #11773 + Ref: #11928 + Closes #12005 -- tidy-up: mostly whitespace nits +- cmake: tidy-up `NOT_NEED_LBER_H` detection - - delete completed TODO from `./CMakeLists.txt`. - - convert a C++ comment to C89 in `./CMake/CurlTests.c`. - - delete duplicate EOLs from EOF. - - add missing EOL at EOF. - - delete whitespace at EOL (except from expected test results). - - convert tabs to spaces. - - convert CRLF EOLs to LF in GHA yaml. - - text casing fixes in `./CMakeLists.txt`. - - fix a codespell typo in `packages/OS400/initscript.sh`. + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - Closes #11772 +- appveyor: rewrite batch in PowerShell + CI improvements -Dan Fandrich (31 Aug 2023) + 1. Rewrite in PowerShell: -- CI: remove Windows builds from Cirrus, without replacement + - rewrite MS-DOS batch build script in PowerShell. + - move some bash operations into native PowerShell. + - fixups for PowerShell insisting on failure when a command outputs + something to stderr. + - fix to actually run `curl -V` after every build. + (and exclude ARM64 builds.) + - also say why we skipped `curl -V` if we had to skip. + - fix CMake warnings about unused configuration variables, by adapting + these dynamically for build cases. + - dedupe OpenSSL path into a variable. + - disable `test1451` failing with a warning anyway due to missing python + impacket. (after trying and failing to install impacket) + PowerShell promotes these warnings to errors by PowerShell. We can also + suppress they wholesale if they start causing issues in the future, + like we already to with `autoreconf` and `./configure`. - If we don't do this, all coverage on Cirrus will cease in a few days. By - removing the Windows builds, the FreeBSD one should still continue - as before. The Windows builds will need be moved to another service to - maintain test coverage. + PowerShell is better than MS-DOS batches, so the hope is this makes it + easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't + supported inline by AppVeyor on Windows build machines, but we are okay + to keep it in an external script, so it's also an option. - Closes #11771 + 2. CI improvements: -- CI: switch macOS ARM build from Cirrus to Circle CI + - enable tests for a "unity" build job. + - speed-up CI initialization by using shallow clones of the curl repo. + - speed-up CMake MSVC jobs with `TrackFileAccess=false`. + - enable parallelism in `VisualStudioSolution` builds. + - display CMake version before builds. + - always show the CPU in job names. + - tell which jobs are build-only in job names. + - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. + - add `config.log` (autotools) to dumped logs (need to enable manually). - Cirrus is drastically reducing their free tier on Sept. 1, so they will - no longer perform all these builds for us. All but one build has been - moved, with the LibreSSL one being dropped because of linking problems - on Circle. + 3. Style: - One important note about this change is that Circle CI is currently - directing all these builds to x86_64 hardware, despite them requesting - ARM. This is because ARM nodes are scheduled to be available on the - free tier only in December. This reduces our architectural diversity - until then but it should automatically come back once those machines are - enabled. + - use single-quotes in YAML like we do in other CI YAML files. + It also allows to drop quoting characters and lighter to write/read. + (keep double quotes for PowerShell strings needing expansion.) -- CI: use the right variable for BSD make + Closes #11999 - BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel - builds before. +- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows -- CI: drop the FreeBSD 12.X build + - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. + Before this patch we set it based it on the presence of `stricmp`, + which correctly enabled it on e.g. Windows, but was inaccurate for + other platforms. - Cirrus' new free tier won't let us have many builds, so drop the - nonessential ones. The FreeBSD 13.X build will still give us the most - relevant FreeBSD coverage. + - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and + LDAPS is not explicitly disabled. This mimics autotools behaviour. + Previously we set it only for Windows LDAP. After this fix, LDAPS is + correctly enabled in default macOS builds. -- CI: move the Alpine build from Cirrus to GHA + - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) + to avoid deprecation warnings for LDAP API. - Cirrus is reducing their free tier to next to nothing, so we must move - builds elsewhere. + - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. + This doesn't make much sense, but let's do it to sync behaviour with + autotools. -Stefan Eissing (30 Aug 2023) + - fix benign typo in variable name. -- test_07_upload.py: fix test_07_34 curl args + Ref: #11964 (effort to sync cmake detections with autotools) - - Pass correct filename to --data-binary. + Closes #12006 - Prior to this change --data-binary was passed an incorrect filename due - to a missing separator in the arguments list. Since aacbeae7 curl will - error on incorrect filenames for POST. +- autotools: restore `HAVE_IOCTL_*` detections - Fixes https://github.com/curl/curl/issues/11761 - Closes https://github.com/curl/curl/pull/11763 + This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in + 4d73854462f30948acab12984b611e9e33ee41e6 and + c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the + `HAVE_IOCTL` result it generated was unused in the source. But, + I did miss the fact that this had two dependent checks: + `CURL_CHECK_FUNC_IOCTL_FIONBIO`, + `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: + `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. -Nicholas Nethercote (30 Aug 2023) + Regression from 4d73854462f30948acab12984b611e9e33ee41e6 -- tests: document which tests fail due to hyper's lack of trailer support. + Ref: #11964 (effort to sync cmake detections with autotools) - Closes #11762 + Closes #12008 -- docs: removing "pausing transfers" from HYPER.md. +Daniel Stenberg (2 Oct 2023) - It's a reference to #8600, which was fixed by #9070. +- RELEASE-PROCEDURE.md: updated coming release dates - Closes #11764 +- RELEASE-NOTES: synced -Patrick Monnerat (30 Aug 2023) +Viktor Szakats (1 Oct 2023) -- os400: handle CURL_TEMP_PRINTF() while building bind source +- cmake: pre-cache `HAVE_POLL_FINE` on Windows - Closes #11547 + Windows doesn't support `poll()`, so we can safely skip checking for + fine poll. -- os400: build test servers + Closes #12003 - Also fix a non-compliant main prototype in disabled.c. +- gha: bump actions to latest versions - Closes #11547 + - actions@checkout@v4 (from v3 and v2) -- tests: fix compilation error for os400 + - fsfe/reuse-action@v2 (from v1) - OS400 uses BSD 4.3 setsockopt() prototype by default: this does not - define parameter as const, resulting in an error if actual parameter is - const. Remove the const keyword from the actual parameter cast: this - works in all conditions, even if the formal parameter uses it. + Closes #12000 - Closes #11547 +Stefan Eissing (30 Sep 2023) -- os400: make programs and command name configurable +- h2: testcase and fix for pausing h2 streams - Closes #11547 + - refs #11982 where it was noted that paused transfers may + close successfully without delivering the complete data + - made sample poc into tests/http/client/h2-pausing.c and + added test_02_27 to reproduce -- os400: move build configuration parameters to a separate script + Closes #11989 + Fixes #11982 + Reported-by: Harry Sintonen - They can then easily be overriden in a script named "config400.override" - that is not part of the distribution. +Viktor Szakats (30 Sep 2023) - Closes #11547 +- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value -- os400: implement CLI tool + Before this patch CMake builds accepted any value and it was used at + runtime as-is. This patch make sure that the selected default backend + is also enabled in the build. It also enforces a full lowercase value. - This is provided as a QADRT (ascii) program, a link to it in the IFS and - a minimal CL command. + This improves reproducibility and brings CMake in sync with autotools + which already worked like described above. - Closes #11547 + Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 -Matthias Gatto (30 Aug 2023) + Closes #11998 -- lib: fix aws-sigv4 having date header twice in some cases +- autotools: adjust `CURL_CA_PATH` value to CMake - When the user was providing the header X-XXX-Date, the header was - re-added during signature computation, and we had it twice in the - request. + autotools was using the same value as CMake, but with an ending + slash. Delete the ending slash to match configurations. - Reported-by: apparentorder@users.noreply.github.com + Ref: #11964 (effort to sync cmake detections with autotools) - Signed-off-by: Matthias Gatto + Closes #11997 - Fixes: https://github.com/curl/curl/issues/11738 - Closes: https://github.com/curl/curl/pull/11754 +- cmake: detect `sys/wait.h` and `netinet/udp.h` -Jay Satiro (30 Aug 2023) + Ref: #11964 (effort to sync cmake detections with autotools) -- multi: remove 'processing: ' debug message + Closes #11996 - - Remove debug message added by e024d566. +Daniel Stenberg (30 Sep 2023) - Closes https://github.com/curl/curl/pull/11759 +- lib: provide and use Curl_hexencode -- ftp: fix temp write of ipv6 address + Generates a lower case ASCII hex output from a binary input. - - During the check to differentiate between a port and IPv6 address - without brackets, write the binary IPv6 address to an in6_addr. + Closes #11990 - Prior to this change the binary IPv6 address was erroneously written to - a sockaddr_in6 'sa6' when it should have been written to its in6_addr - member 'sin6_addr'. There's no fallout because no members of 'sa6' are - accessed before it is later overwritten. +- configure: check for the capath by default - Closes https://github.com/curl/curl/pull/11747 + ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS + L -- tool: change some fopen failures from warnings to errors + cmake: synced - - Error on missing input file for --data, --data-binary, - --data-urlencode, --header, --variable, --write-out. + Assisted-by: Viktor Szakats + Closes #11987 - Prior to this change if a user of the curl tool specified an input file - for one of the above options and that file could not be opened then it - would be treated as zero length data instead of an error. For example, a - POST using `--data @filenametypo` would cause a zero length POST which - is probably not what the user intended. +- wolfssl: ignore errors in CA path - Closes https://github.com/curl/curl/pull/11677 + The default wolfSSL_CTX_load_verify_locations() function is quite picky + with the certificates it loads and will for example return error if just + one of the certs has expired. -- hostip: fix typo + With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it + behaves more similar to what OpenSSL does by default. -Davide Masserut (29 Aug 2023) + Even the set of default certs on my Debian unstable has several expired + ones. -- tool: avoid including leading spaces in the Location hyperlink + Assisted-by: Juliusz Sosinowicz + Assisted-by: Michael Osipov - Co-authored-by: Dan Fandrich + Closes #11987 - Closes #11735 +- create-dirs.d: clarify it also uses --output-dirs -Daniel Stenberg (29 Aug 2023) + Reported-by: Robert Simpson + Fixes #11991 + Closes #11995 -- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline +Viktor Szakats (30 Sep 2023) - Closes #11757 +- appveyor: fix yamlint issues, indent -- connect: stop halving the remaining timeout when less than 600 ms left + Also: + - use double quotes in all batch if statements. - When curl wants to connect to a host, it always has a TIMEOUT. The - maximum time it is allowed to spend until a connect is confirmed. + Closes #11994 - curl will try to connect to each of the IP adresses returned for the - host. Two loops, one for each IP family. +- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` - During the connect loop, while curl has more than one IP address left to - try within a single address family, curl has traditionally allowed (time - left/2) for *this* connect attempt. This, to not get stuck on the - initial addresses in case the timeout but still allow later addresses to - get attempted. + Based on existing autotools logic. - This has the downside that when users set a very short timeout and the - host has a large number of IP addresses, the effective result might be - that every attempt gets a little too short time. + Ref: #11964 (effort to sync cmake detections with autotools) - This change stop doing the divided-by-two if the total time left is - below a threshold. This threshold is 600 milliseconds. + Closes #11981 - Closes #11693 +- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` -- asyn-ares: reduce timeout to 2000ms + Based on existing autotools logic. - When UDP packets get lost this makes for slightly faster retries. This - lower timeout is used by @c-ares itself by default starting next - release. + autotools checks for old versions of the allowlisted target OSes and + disables this feature when seeing them. In CMake we assume we're running + on newer systems and enable regardless of OS version. - Closes #11753 + autotools always runs all 3 probes for non-fast-tracked systems and + enables this feature if any one of them was successful. To save + configuration time, CMake stops at the first successful check. -John Bampton (29 Aug 2023) + OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD + system. I haven't double-checked if this is correct, but looks odd. -- misc: remove duplicate words + Ref: #11964 (effort to sync cmake detections with autotools) - Closes #11740 + Closes #11979 -Daniel Stenberg (29 Aug 2023) +- cmake: fix `HAVE_WRITABLE_ARGV` detection -- RELEASE-NOTES: synced + Move detection before the creation of detection results in + `curl_config.h`. -- wolfSSL: avoid the OpenSSL compat API when not needed + Ref: #11964 (effort to sync cmake detections with autotools) - ... and instead call wolfSSL functions directly. + Closes #11978 - Closes #11752 +- appveyor: minor improvements -Viktor Szakats (28 Aug 2023) + - run `curl -V` after builds to see if they run and with what features. + Except for one job where a CRT DLL is missing. And ARM64 which should + fail, but is silently not launched instead. -- lib: fix null ptr derefs and uninitialized vars (h2/h3) + - copy libcurl DLL next to curl tool and tests binaries in shared mode. + This makes it possible to run the tests. (We don't run tests after + these builds yet.) - Fixing compiler warnings with gcc 13.2.0 in unity builds. + - list the DLLs and EXEs present after the builds. - Assisted-by: Jay Satiro - Assisted-by: Stefan Eissing - Closes #11739 + - add `DEBUG` variable for CMake builds to allow disabling it, for + testing non-debug builds. (currently enabled for all) -Jay Satiro (28 Aug 2023) + - add commented lines that dump CMake configuration logs for debugging + build/auto-detection issues. -- secureserver.pl: fix stunnel version parsing + - add gcc version to jobs where missing. - - Allow the stunnel minor-version version part to be zero. + - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 + to the build mix. - Prior to this change with the stunnel version scheme of . - if either part was 0 then version parsing would fail, causing - secureserver.pl to fail with error "No stunnel", causing tests that use - the SSL protocol to be skipped. As a practical matter this bug can only - be caused by a minor-version part of 0, since the major-version part is - always greater than 0. + - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. - Closes https://github.com/curl/curl/pull/11722 + - delete a duplicate backslash. -- secureserver.pl: fix stunnel path quoting + Closes #11976 - - Store the stunnel path in the private variable $stunnel unquoted and - instead quote it in the command strings. +- configure: replace adhoc domain with `localhost` in tests - Prior to this change the quoted stunnel path was passed to perl's file - operators which cannot handle quoted paths. For example: + Reviewed-by: Daniel Stenberg + Closes #11988 - $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; - if(-x $stunnel or -x "$stunnel") - # false even if path exists and is executable +- tidy-up: use more example domains - Our other test scripts written in perl, unlike this one, use servers.pm - which has a global $stunnel variable with the path stored unquoted and - therefore those scripts don't have this problem. + Also make use of the example TLD: + https://en.wikipedia.org/wiki/.example - Closes https://github.com/curl/curl/pull/11721 + Reviewed-by: Daniel Stenberg + Closes #11992 -Daniel Stenberg (28 Aug 2023) +Dan Fandrich (29 Sep 2023) -- altsvc: accept and parse IPv6 addresses in response headers +- runtests: display the test status if tests appear hung - Store numerical IPv6 addresses in the alt-svc file with the brackets - present. + It sometimes happens that a test hangs during a test run and never + returns. The test harness will wait indefinitely for the results and on + CI servers the CI job will eventually be killed after an hour or two. + At the end of a test run, if results haven't come in within a couple of + minutes, display the status of all test runners and what tests they're + running to help in debugging the problem. - Verify with test 437 and 438 + This feature is really only kick in with parallel testing enabled, which + is fine because without parallel testing it's usually easy to tell what + test has hung. - Fixes #11737 - Reported-by: oliverpool on github - Closes #11743 + Closes #11980 -- libtest: use curl_free() to free libcurl allocated data +- github/labeler: remove workaround for labeler - In several test programs. These mistakes are not detected or a problem - as long as memdebug.h is included, as that provides the debug wrappers - for all memory functions in the same style libcurl internals do it, - which makes curl_free and free effectively the same call. + This was added due to what seemed to be a bug regarding the sync-labels: + config option, but it looks like it wasn't necessary. - Reported-by: Nicholas Nethercote - Closes #11746 + Follow-up to b2b0534e7 -Jay Satiro (28 Aug 2023) +Viktor Szakats (29 Sep 2023) -- disable.d: explain --disable not implemented prior to 7.50.0 +- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] - Option -q/--disable was added in 5.0 but only -q was actually - implemented. Later --disable was implemented in e200034 (precedes - 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). +Daniel Stenberg (29 Sep 2023) - Reported-by: pszlazak@users.noreply.github.com +- docs: replace made up domains with example.com - Fixes https://github.com/curl/curl/issues/11710 - Closes #11712 + in FAQ and MANUAL.md -Nicholas Nethercote (28 Aug 2023) + - example.com was made for this purpose. -- hyper: fix ownership problems + - reduces the risk that one of those domains suddenly start hosting + something nasty and we provide links to them - Some of these changes come from comparing `Curl_http` and - `start_CONNECT`, which are similar, and adding things to them that are - present in one and missing in another. + Closes #11986 - The most important changes: - - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the - happy path. - - In `start_CONNECT`, add a missing `hyper_request_free` on the error - path. - - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. - - In `bodysend`, remove an unnecessary `hyper_body_free` on a different - error path that would cause a double-free. - https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html - says of `hyper_request_set_body`: "This takes ownership of the - hyper_body *, you must not use it or free it after setting it on the - request." This is true even if `hyper_request_set_body` returns an - error; I confirmed this by looking at the hyper source code. +Michael Osipov (29 Sep 2023) - Other changes are minor but make things slightly nicer. +- acinclude.m4: Document proper system truststore on FreeBSD - Closes #11745 + The default system truststore on FreeBSD has been /etc/ssl/certs for many + years now. It is managed canonically through certctl(8) and contains hashed + symlinks for OpenSSL and other TLS providers. + The previous ones require security/ca_root_nss which might not be installed o + r + will not contain any custom CA certificates. -Daniel Stenberg (28 Aug 2023) + Closes #11985 -- multi.h: the 'revents' field of curl_waitfd is supported +Daniel Stenberg (29 Sep 2023) - Since 6d30f8ebed34e7276 +- FAQ: How do I upgrade curl.exe in Windows? - Reported-by: Nicolás Ojeda Bär - Ref: #11748 - Closes #11749 + This is a growing question, better answer it here to get somewhere to + point users to. -Gerome Fournier (27 Aug 2023) + Closes #11984 -- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() +Viktor Szakats (28 Sep 2023) - Closes #11742 +- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC -Daniel Stenberg (27 Aug 2023) + `basename` is present in mingw-w64, missing from MSVC. Pre-cache + accordingly to make configure faster. -- docs: mention critical files in same directories as curl saves + Notice that `basename` has a bug so we later disable it even with + mingw-w64: + https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li + b/curl_setup.h#L820-L825 - ... cannot be fully protected. Don't do it. + Closes #11974 - Co-authored-by: Jay Satiro - Reported-by: Harry Sintonen - Fixes #11530 - Closes #11701 +Daniel Stenberg (28 Sep 2023) -John Hawthorn (26 Aug 2023) +- cmake: add missing checks -- OpenSSL: clear error queue after SSL_shutdown + - check for arc4random. To make rand.c use it accordingly. + - check for fcntl + - fix fseek detection + - add SIZEOF_CURL_SOCKET_T + - fix USE_UNIX_SOCKETS + - define HAVE_SNPRINTF to 1 + - check for fnmatch + - check for sched_yield + - remove HAVE_GETPPID duplicate from curl_config.h + - add HAVE_SENDMSG - We've seen errors left in the OpenSSL error queue (specifically, - "shutdown while in init") by adding some logging it revealed that the - source was this file. + Ref: #11964 - Since we call SSL_read and SSL_shutdown here, but don't check the return - code for an error, we should clear the OpenSSL error queue in case one - was raised. + Co-authored-by: Viktor Szakats + Closes #11973 - This didn't affect curl because we call ERR_clear_error before every - write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when - libcurl is used in a process with other OpenSSL users, they may detect - an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their - own. +- configure: remove unused checks - Co-authored-by: Satana de Sant'Ana + - for sys/uio.h + - for fork + - for connect - Closes #11736 + Ref: #11964 -Alexander Kanavin (25 Aug 2023) + Closes #11973 -- tests: update cookie expiry dates to far in the future +- lib: remove TIME_WITH_SYS_TIME - This allows testing Y2038 with system time set to after that, so that - actual Y2038 issues can be exposed, and not masked by expiry errors. + It is not used in any code anywhere. - Fixes #11576 - Closes #11610 + Ref: #11964 + Closes #11975 -John Bampton (25 Aug 2023) +- docs: update curl man page references -- misc: fix spelling + Detected by the manpage-syntax update - Closes #11733 + Closes #11963 -Daniel Stenberg (25 Aug 2023) +- manpage-syntax: verify curl man page references -- cmdline-opts/page-header: clarify stronger that !opt == URL + 1. References to curl symbols are now checked that they indeed exist as + man pages. This for \f references as well as the names referenced in the + SEE ALSO section. - Everything provided on the command line that is not an option (or an - argument to an option) is treated as a URL. + Allowlist curl.1 since it is not always built in builds - Closes #11734 + 2. References to curl symbols that lack section now causes warning, since tha + t + will prevent them from getting linked properly -- tests/runner: fix %else handling + 3. Check for "bare" references to curl functions and warn, they should be + references - Getting the show state proper for %else and %endif did not properly work - in nested cases. + Closes #11963 - Follow-up to 3d089c41ea9 +- cmake: add check for suseconds_t - Closes #11731 + And fix the HAVE_LONGLONG define -Nicholas Nethercote (25 Aug 2023) + Ref: #11964 + Closes #11977 -- docs: Remove mention of #10803 from `KNOWN_BUGS`. +Viktor Szakats (28 Sep 2023) - Because the leaks have been fixed. +- tidy-up: whitespace fixes -- c-hyper: fix another memory leak in `Curl_http`. + Closes #11972 - There is a `hyper_clientconn_free` call on the happy path, but not one - on the error path. This commit adds one. +- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS - Fixes the second memory leak reported by Valgrind in #10803. + With new option `CURL_DISABLE_SRP=ON` to force-disable it. + To match existing option and detection logic in autotools. - Fixes #10803 - Closes #11729 + Also: + - fix detecting GnuTLS. + We assume `nettle` as a GnuTLS dependency. + - add CMake GnuTLS CI job. + - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) + TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. + - fix compiler warning when building with GnuTLS and disabled TLS-SRP. + - fix comment typos, whitespace. -- c-hyper: fix a memory leak in `Curl_http`. + Ref: #11964 - A request created with `hyper_request_new` must be consumed by either - `hyper_clientconn_send` or `hyper_request_free`. + Closes #11967 - This is not terrifically clear from the hyper docs -- - `hyper_request_free` is documented only with "Free an HTTP request if - not going to send it on a client" -- but a perusal of the hyper code - confirms it. +- tool: use our own stderr variable - This commit adds a `hyper_request_free` to the `error:` path in - `Curl_http` so that the request is consumed when an error occurs after - the request is created but before it is sent. + Earlier this year we changed our own stderr variable to use the standard + name `stderr` (to avoid bugs where someone is using `stderr` instead of + the curl-tool specific variable). This solution needed to override the + standard `stderr` symbol via the preprocessor. This in turn didn't play + well with unity builds and caused curl tool to crash or stay silent due + to an uninitialized stderr. This was a hard to find issue, fixed by + manually breaking out one file from the unity sources. - Fixes the first memory leak reported by Valgrind in #10803. + To avoid two these two tricks, this patch implements a different + solution: Restore using our own local variable for our stderr output and + leave `stderr` as-is. To avoid using `stderr` by mistake, add a + `checksrc` rule (based on logic we already used in lib for `strerror`) + that detects any `stderr` use in `src` and points to using our own + variable instead: `tool_stderr`. - Closes #11729 + Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e + Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db -Daniel Stenberg (25 Aug 2023) + Closes #11958 -- RELEASE-NOTES: synced +Loïc Yhuel (28 Sep 2023) -John Bampton (25 Aug 2023) +- connect: only start the happy eyeballs timer when needed -- misc: spellfixes + The timeout is only used when there is a second address family, for the + delayed eyeballer. - Closes #11730 + Closes #11939 -Daniel Stenberg (25 Aug 2023) +Daniel Stenberg (28 Sep 2023) -- tests: add support for nested %if conditions +- tool_operate: free 'gateway' correctly - Provides more flexiblity to test cases. + Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. - Also warn and bail out if there is an '%else' or %endif' without a - preceeding '%if'. + Also removed repeated wording in IPFS related error messages. - Ref: #11610 - Closes #11728 + Closes #11969 -- time-cond.d: mention what happens on a missing file +Stefan Eissing (28 Sep 2023) - Closes #11727 +- lib: move handling of `data->req.writer_stack` into Curl_client_write() -Christian Hesse (24 Aug 2023) + - move definitions from content_encoding.h to sendf.h + - move create/cleanup/add code into sendf.c + - installed content_encoding writers will always be called + on Curl_client_write(CLIENTWRITE_BODY) + - Curl_client_cleanup() frees writers and tempbuffers from + paused transfers, irregardless of protocol -- docs/cmdline-opts: match the current output + Closes #11908 - The release date has been added in output, reflect that in documentation. +Loïc Yhuel (28 Sep 2023) - Closes #11723 +- multi: round the timeout up to prevent early wakeups -Daniel Stenberg (24 Aug 2023) + Curl_timediff rounds down to the millisecond, so curl_multi_perform can + be called too early, then we get a timeout of 0 and call it again. -- lib: minor comment corrections + The code already handled the case of timeouts which expired less than + 1ms in the future. By rounding up, we make sure we will never ask the + platform to wake up too early. -- docs: rewrite to present tense + Closes #11938 - ... instead of using future tense. +Daniel Stenberg (28 Sep 2023) - + numerous cleanups and improvements - + stick to "reuse" not "re-use" - + fewer contractions +- RELEASE-NOTES: spell out that IPFS is via gateway - Closes #11713 +- RELEASE-NOTES: synced -- urlapi: setting a blank URL ("") is not an ok URL +- tool_operate: avoid strlen() -1 on zero length content from file - Test it in 1560 - Fixes #11714 - Reported-by: ad0p on github - Closes #11715 + Follow-up to 65b563a96a226649ba12cb1e -- spelling: use 'reuse' not 're-use' in code and elsewhere + Closes #11959 - Unify the spelling as both versions were previously used intermittently +- tool_operate: fix memory mixups - Closes #11717 + Switch to plain getenv() from curl_getenv() to avoid the allocation and + having to keep track of which free() or curl_free() that need to be + used. -Michael Osipov (23 Aug 2023) + Coverity found issues and a memory leak. -- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC + Follow-up to 65b563a96a226649ba12cb1e - HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the - default one. Use "long long" in 32 bit mode and just "long" in 64 bit - mode. + Closes #11959 - Closes #11718 +Viktor Szakats (27 Sep 2023) -Dan Fandrich (22 Aug 2023) +- curl-functions.m4: fixup recent bad edits -- tests: don't call HTTP errors OK in test cases + Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Some HTTP errors codes were accompanied by the text OK, which causes - some cognitive dissonance when reading them. + Closes #11966 -- http: close the connection after a late 417 is received +Daniel Stenberg (27 Sep 2023) - In this situation, only part of the data has been sent before aborting - so the connection is no longer usable. +- curl-functions.m4: fix include line - Assisted-by: Jay Satiro - Fixes #11678 - Closes #11679 + This made the getaddrinfo detection fail, but we did not spot it in the + CI because it graciously falled back to using legacy functions instead! -- runtests: slightly increase the longest log file displayed + Follow-up to 96c29900bcec (#11940) - The new limit provides enough space for a 64 KiB data block to be logged - in a trace file, plus a few lines at the start and end for context. This - happens to be the amount of data sent at a time in a PUT request. + Closes #11965 -- tests: add delay command to the HTTP server +- inet_ntop: add typecast to silence Coverity - This adds a delay after client connect. + CID 1024653: Integer handling issues (SIGN_EXTENSION) -Daniel Stenberg (22 Aug 2023) + Suspicious implicit sign extension: "src[i]" with type "unsigned char + const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to + type "int" (32 bits, signed), then sign-extended to type "unsigned long" + (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than + 0x7FFFFFFF, the upper bits of the result will all be 1. -- cirrus: install everthing with pkg, avoid pip + 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); - Assisted-by: Sevan Janiyan + The value will not be greater than 0x7FFFFFFF so this still cannot + happen. - Closes #11711 + Also, switch to ints here instead of longs. The values stored are 16 bit + so at least no need to use 64 bit variables. Also, longs are 32 bit on + some platforms so this logic still needs to work with 32 bits. -- curl_url*.3: update function descriptions + Closes #11960 - - expand and clarify several descriptions - - avoid using future tense all over +- docs: adapt SEE ALSO sections to new requirements - Closes #11708 + To please manpage-syntax.pl used by test 1173 -- RELEASE-NOTES: synced + Closes #11957 -Stefan Eissing (21 Aug 2023) +- manpage-syntax.pl: verify SEE ALSO syntax -- CI/cirrus: disable python install on FreeBSD + - Enforce a single reference per .BR line + - Skip the quotes around the section number for example (3) + - Insist on trailing commas on all lines except the last + - Error on comma on the last SEE ALSO entry - - python cryptography package does not build build FreeBSD - - install just mentions "error" - - this gets the build and the main test suite going again + - List the entries alpha-sorted, not enforced just recommended - Closes #11705 + Closes #11957 -- test2600: fix flakiness on low cpu +- connect: expire the timeout when trying next - - refs #11355 where failures to to low cpu resources in CI - are reported - - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations - to test cases - - trigger Curl_expire() in test filter to allow re-checks before - the usual 1second interval + ... so that it gets called again immediately and can continue trying + addresses to connect to. Otherwise it might unnecessarily wait for a + while there. - Closes #11690 + Fixes #11920 + Reported-by: Loïc Yhuel + Closes #11935 -Maksim Sciepanienka (20 Aug 2023) +- http: remove wrong comment for http_should_fail -- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf + Reported-by: Christian Schmitz + Ref: #11936 + Closes #11941 - Closes #11698 +Dan Fandrich (26 Sep 2023) -Daniel Stenberg (20 Aug 2023) +- tool_setopt: remove unused function tool_setopt_flags -- test687/688: two more basic --xattr tests + This function is identical to tool_setopt_bitmask except that it treats + the argument as unsigned. - Closes #11697 + Closes #11943 -- cmdline-opts/docs: mentioned the negative option part +Viktor Szakats (26 Sep 2023) - ... for --no-alpn and --no-buffer in the same style done for other --no- - options: +- cmake: add feature checks for `memrchr` and `getifaddrs` - "Note that this is the negated option name documented." + - `HAVE_MEMRCHR` for `memrchr`. + - `HAVE_GETIFADDRS` for `getifaddrs`. + This was present in `lib/curl_config.h.cmake` but missed the detection + logic. - Closes #11695 + To match existing autotools feature checks. -Emanuele Torre (19 Aug 2023) + Closes #11954 -- tool/var: also error when expansion result starts with NUL +- cmake: move global headers to specific checks - Expansions whose output starts with NUL were being expanded to the empty - string, and not being recognised as values that contain a NUL byte, and - should error. + Before this patch we added standard headers unconditionally to the + global list of headers used for feature checks. This is unnecessary + and also doesn't help CMake 'Generate' performance. This patch moves + these headers to each feature check where they are actually needed. + Stop using `stddef.h`, as it seems unnecessary. - Closes #11694 + I've used autotools' `m4/curl-functions.m4` to figure out these + dependencies. -Daniel Stenberg (19 Aug 2023) + Also delete checking for the C89 standard header `time.h`, that I + missed in the earlier commit. -- tests: add 'large-time' as a testable feature + Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - This allows test cases to require this feature to run and to be used in - %if conditions. + Closes #11951 - Large here means larger than 32 bits. Ie does not suffer from y2038. +- src/mkhelp: make generated code pass `checksrc` - Closes #11696 + Closes #11955 -- tests/Makefile: add check-translatable-options.pl to tarball +- tests: show which curl tool `runtests.pl` is using - Used in test 1544 + To help debugging when there is issue finding or running it. - Follow-up to ae806395abc8c + Closes #11953 -- gen.pl: fix a long version generation mistake +- CI/azure: make `MAKEFLAGS` global to parallelize all jobs - Too excessive escaping made the parsing not find the correct long names - later and instead add "wrong" links. + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi + th -j3) - Follow-up to 439ff2052e219 + Closes #11952 - Reported-by: Lukas Tribus - Fixes #11688 - Closes #11689 +- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 -- lib: move mimepost data from ->req.p.http to ->state + Also delete an accidental variable reference. - When the legacy CURLOPT_HTTPPOST option is used, it gets converted into - the modem mimpost struct at first use. This data is (now) kept for the - entire transfer and not only per single HTTP request. This re-enables - rewind in the beginning of the second request instead of in end of the - first, as brought by 1b39731. + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 - The request struct is per-request data only. + Closes #11945 - Extend test 650 to verify. +Daniel Stenberg (26 Sep 2023) - Fixes #11680 - Reported-by: yushicheng7788 on github - Closes #11682 +- docs: add see also curl_multi_get_handles to some man pages -Patrick Monnerat (17 Aug 2023) + Assisted-by: Jay Satiro -- os400: do not check translatable options at build time + Closes #11942 - Now that there is a test for this, the build time check is not needed - anymore. +Viktor Szakats (26 Sep 2023) - Closes #11650 +- cmake: assume `_fseeki64` and no `fseeko` on Windows -- test1554: check translatable string options in OS400 wrapper + `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and + at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. - This test runs a perl script that checks all string options are properly - translated by the OS400 character code conversion wrapper. It also - verifies these options are listed in alphanumeric order in the wrapper - switch statement. + `fseeko` is not present in any of these. - Closes #11650 + (mingw-w64 1.0 also offers `fseeko64`.) -Daniel Stenberg (17 Aug 2023) + [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 -- unit3200: skip testing if function is not present + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 - Fake a successful run since we have no easy mechanism to skip this test - for this advanced condition. + Closes #11950 -- unit2600: fix build warning if built without verbose messages +- build: delete checks for C89 standard headers -- test1608: make it build and get skipped without shuffle DNS support + Delete checks and guards for standard C89 headers and assume these are + available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, + `stddef.h`, `signal.h`. -- lib: --disable-bindlocal builds curl without local binding support + Some of these we already used unconditionally, some others we only used + for feature checks. -- test1304: build and skip without netrc support + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i + n CMake) -- lib: build fixups when built with most things disabled + Closes #11940 - Closes #11687 +Stefan Eissing (26 Sep 2023) -- workflows/macos.yml: disable zstd and alt-svc in the http-only build +- multiif.h: remove Curl_multi_dump declaration - Closes #11683 + Follow-up to d850eea2 which removed the Curl_multi_dump definition. -Stefan Eissing (17 Aug 2023) + Closes https://github.com/curl/curl/pull/11946 -- bearssl: handshake fix, provide proper get_select_socks() implementation +Jay Satiro (26 Sep 2023) - - bring bearssl handshake times down from +200ms down to other TLS backends - - vtls: improve generic get_select_socks() implementation - - tests: provide Apache with a suitable ssl session cache +- config-win32: define HAVE__FSEEKI64 - Closes #11675 + Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that + calls _fseeki64 if it is available. -- tests: TLS session sharing test + Closes https://github.com/curl/curl/pull/11944 - - test TLS session sharing with special test client - - expect failure with wolfSSL - - disable flaky wolfSSL test_02_07b +- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER - Closes #11675 + - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place + even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. -Daniel Stenberg (17 Aug 2023) + The behavior is verified by test2048. -- CURLOPT_*TIMEOUT*: extend and clarify + Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 + Reported-by: claudiusaiz@users.noreply.github.com - Closes #11686 + Bug: https://github.com/curl/curl/discussions/11910 + Reported-by: Hakan Sunay Halil -- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails + Closes https://github.com/curl/curl/pull/11930 - And document it. Only return out of memory when it actually is a memory - problem. +Stefan Eissing (26 Sep 2023) - Pointed-out-by: Jacob Mealey - Closes #11674 +- openssl: improve ssl shutdown handling -Mathew Benson (17 Aug 2023) + - If SSL shutdown is not finished then make an additional call to + SSL_read to gather additional tracing. -- cmake: add GnuTLS option + - Fix http2 and h2-proxy filters to forward do_close() calls to the next + filter. - - Option to use GNUTLS was missing. Hence was not able to use GNUTLS - with ngtcp2 for http3. + For example h2 and SSL shutdown before and after this change: - Closes #11685 + Before: -Daniel Stenberg (16 Aug 2023) + Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> + ssl_cf_destroy -- RELEASE-NOTES: synced + After: -- http: remove the p_pragma struct field + Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> + ssl_cf_close - unused since 40e8b4e52 (2008) + Note that currently the tracing does not show output on the connection + closure handle. Refer to discussion in #11878. - Closes #11681 + Ref: https://github.com/curl/curl/discussions/11878 -Jay Satiro (16 Aug 2023) + Closes https://github.com/curl/curl/pull/11858 -- CURLINFO_CERTINFO.3: better explain curl_certinfo struct +Loïc Yhuel (26 Sep 2023) - Closes https://github.com/curl/curl/pull/11666 +- multi: fix small timeouts -- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation + Since Curl_timediff rounds down to the millisecond, timeouts which + expire in less than 1ms are considered as outdated and removed from the + list. We can use Curl_timediff_us instead, big timeouts could saturate + but this is not an issue. - - Remove the out-of-date SSL backend list supported by - CURLOPT_SSL_CTX_FUNCTION. + Closes #11937 - It makes more sense to just refer to that document instead of having - a separate list that has to be kept in sync. +Viktor Szakats (25 Sep 2023) - Closes https://github.com/curl/curl/pull/11665 +- cmake: fix stderr initialization in unity builds -- write-out.d: clarify %{time_starttransfer} + Before this patch, in certain build configurations the curl tool may + not have displayed anything (debug, macOS), or crashed at startup + (debug, Windows). - sync it up with CURLINFO_STARTTRANSFER_TIME_T + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 + Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db -Daniel Stenberg (15 Aug 2023) + Closes #11929 -- transfer: don't set TIMER_STARTTRANSFER on first send +- cmake: fix missing `zlib.h` when compiling `libcurltool` - The time stamp is for measuring the first *received* byte + Came up while testing debug/testing build for Windows. I'm not sure why + it didn't come up in earlier tests with similar config. + `tool_hugehelp.c` might indeed require `zlib.h` and without linking + `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected + dependency headers to the compiler command. - Fixes #11669 - Reported-by: JazJas on github - Closes #11670 + ``` + [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang + --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 + 6_64 + -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO + DE + -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib + -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li + ne-argument + -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static + -libgcc + -lucrt [...] -O3 -DNDEBUG -municode -MD + -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d + -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x + 64/src/tool_hugehelp.c + .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f + ile not found + 6 | #include + | ^~~~~~~~ + ``` -trrui-huawei (15 Aug 2023) + Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 -- quiche: enable quiche to handle timeout events + Closes #11927 - In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` - interface for the application to invoke upon timer - expiration. Therefore, invoking the `on_timeout` function of the - Connection is crucial to ensure seamless functionality of quiche with - timeout events. +- cmake: fix duplicate symbols when linking tests - Closes #11654 + The linker resolves this automatically in non-unity builds. In unity + builds the linker cannot drop a single object with the duplicates, + resulting in these errors. The root issue is that we started including + certain objects both via both libcurlu and libcurltool libs. -- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s + Regression from 39e7c22bb459c2e818f079984989a26a09741860 - Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. + Windows errors: + ``` + [ 3%] Linking C executable unit1303.exe + [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul + tibyte.c.obj + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch + ar': + C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve + rt_UTF8_to_wchar' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:44: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT + F8': + C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve + rt_wchar_to_UTF8' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:66: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': + C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 + _open' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:92: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': + C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 + 2_fopen' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:120: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': + [...] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 + aa4ehny5q#L247 -Daniel Stenberg (15 Aug 2023) + macOS errors: + ``` + [ 56%] Linking C executable unit1302 + duplicate symbol '_curlx_sotouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_sitouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_uztosz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + [...] + ``` + with config: + ``` + -DCMAKE_UNITY_BUILD=ON \ + -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ + -DBUILD_SHARED_LIBS=ON \ + -DBUILD_STATIC_LIBS=OFF + ``` -- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang + Closes #11926 - Closes #9580 +- cmake: lib `CURL_STATICLIB` fixes (Windows) -- imap: add a check for failing strdup() + - always define `CURL_STATICLIB` when building libcurl for Windows. -- imap: remove the only sscanf() call in the IMAP code + This disables `__declspec(dllexport)` for exported libcurl symbols. + In normal mode (hide symbols) these exported symbols are specified + via `libcurl.def`. When not hiding symbols, all symbols are exported + by default. - Avoids the use of a stack buffer. + Regression from 1199308dbc902c52be67fc805c72dd2582520d30 - Closes #11673 + Fixes #11844 -- imap: use a dynbuf in imap_atom + - fix to omit `libcurl.def` when not hiding private symbols. - Avoid a calculation + malloc. Build the output in a dynbuf. + Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 - Closes #11672 + - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also + omitting `libcurl.def` in this case, and exporting all symbols + instead. This ensures that a shared curl tool can access all debug + functions which are not normally exported from libcurl DLL. -Marin Hannache (14 Aug 2023) + - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" + target. -- http: do not require a user name when using CURLAUTH_NEGOTIATE + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 - In order to get Negotiate (SPNEGO) authentication to work in HTTP you - used to be required to provide a (fake) user name (this concerned both - curl and the lib) because the code wrongly only considered - authentication if there was a user name provided, as in: + - delete duplicate `BUILDING_LIBCURL` definitions. - curl -u : --negotiate https://example.com/ + - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. - This commit leverages the `struct auth` want member to figure out if the - user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of - setting a user name both in curl and the lib. + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 - Signed-off-by: Marin Hannache - Reported-by: Enrico Scholz - Fixes https://sourceforge.net/p/curl/bugs/440/ - Fixes #1161 - Closes #9047 + Closes #11914 -Viktor Szakats (13 Aug 2023) +Daniel Stenberg (25 Sep 2023) -- build: streamline non-UWP wincrypt detections +- RELEASE-NOTES: synced - - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build - and disable our non-UWP-compatible use the Windows crypto API. This - allows to drop two dynamic feature checks. +Dan Fandrich (25 Sep 2023) - `WINDOWS_STORE` is true when invoking CMake with - `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. +- tests: fix log directory path in IPFS tests - Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html + Hard-coding the log directory name fails with parallel tests. - - with autotools, drop the separate feature check for `wincrypt.h`. On - one hand this header has been present for long (even Borland C 5.5 had - it from year 2000), on the other we used the check result solely to - enable another check for certain crypto functions. This fails anyway - with the header not present. We save one dynamic feature check at the - configure stage. + Follow-up to 65b563a96 - Reviewed-by: Marcel Raad - Closes #11657 + Ref: #8805 -Nicholas Nethercote (13 Aug 2023) +Daniel Stenberg (25 Sep 2023) -- docs/HYPER.md: update hyper build instructions +- curl_multi_get_handles: get easy handles from a multi handle - Nightly Rust and `-Z unstable-options` are not needed. + Closes #11750 - The instructions here now match the hyper docs exactly: - https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 - 6f88 +Stefan Eissing (25 Sep 2023) - Closes #11662 +- http: h1/h2 proxy unification -Daniel Stenberg (13 Aug 2023) + - use shared code for setting up the CONNECT request + when tunneling, used in HTTP/1.x and HTTP/2 proxying + - eliminate use of Curl_buffer_send() and other manipulations + of `data->req` or `data->state.ulbuf` -- RELEASE-NOTES: synced + Closes #11808 -- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name +Natanael Copa (25 Sep 2023) - Asssisted-by: Jay Satiro - Closes #11655 +- lib: use wrapper for curl_mime_data fseek callback -- spellcheck: adapt to backslashed minuses + fseek uses long offset which does not match with curl_off_t. This leads + to undefined behavior when calling the callback and caused failure on + arm 32 bit. - As the curl.1 has more backslashed minus, the cleanup sed lines xneed to - adapt. + Use a wrapper to solve this and use fseeko which uses off_t instead of + long. - Adjusted some docs slighly. + Thanks to the nice people at Libera IRC #musl for helping finding this + out. - Follow-up to 439ff2052e + Fixes #11882 + Fixes #11900 + Closes #11918 - Closes #11663 +- configure: sort AC_CHECK_FUNCS -- gen: escape more minus + No functional changes. - Detected since it was still hard to search for option names using dashes - in the middle in the man page. +Daniel Stenberg (25 Sep 2023) - Closes #11660 +- warnless: remove unused functions -- cookie-jar.d: enphasize that this option is ONLY writing cookies + Previously put there for use with the intel compiler - Reported-by: Dan Jacobson - Tweaked-by: Jay Satiro - Ref: #11642 - Closes #11661 + Closes #11932 -Nicholas Nethercote (11 Aug 2023) +- GHA/linux: run singleuse to detect single-use global functions -- docs/HYPER.md: document a workaround for a link error + Use --unit for configure --enable-debug builds - Closes #11653 + Closes #11932 -Jay Satiro (11 Aug 2023) +- singleuse: add scan for use in other source codes -- schannel: verify hostname independent of verify cert + This should reduce false-positive to almost zero. Checks for presence in + unit tests if --unit is specified, which is intended for debug builds + where unit testing is enabled. - Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off - and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the - hostname in schannel code. + Closes #11932 - This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and - verify hostname". We discussed a fix several years ago in #3285 but it - went stale. +- multi: remove Curl_multi_dump - Assisted-by: Daniel Stenberg + A debug-only function that is basically never used. Removed to ease the + use of the singleuse script to detect non-static functions not used + outside the file where it is defined. - Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html - Reported-by: Martin Galvan + Closes #11931 - Ref: https://github.com/curl/curl/pull/3285 +Viktor Szakats (24 Sep 2023) - Fixes https://github.com/curl/curl/issues/3284 - Closes https://github.com/curl/curl/pull/10056 +- tests: fix compiler warnings -Daniel Stenberg (11 Aug 2023) + Seen with llvm 17 on Windows x64. -- curl_quiche: remove superfluous NULL check + ``` + .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration + for non-static variable 'logdir' [-Wmissing-variable-declarations] + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration + for non-static variable 'loglockfile' [-Wmissing-variable-declarations] + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat + ion for non-static variable 'logdir' [-Wmissing-variable-declarations] + 43 | const char *logdir = "log"; + | ^ + .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl + e is not intended to be used outside of this translation unit + 43 | const char *logdir = "log"; + | ^ + .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator + here [-Wcomma] + 350 | ++d, ++s; + | ^ + .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa + rning + 350 | ++d, ++s; + | ^~~ + | (void)( ) + ``` - 'stream' is always non-NULL at this point + ``` + .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; + | ~ ^ ~~~~~~~~~~~~~ + 1 warning generated. - Pointed out by Coverity + .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison + 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] + 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. - Closes #11656 + .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > + 2147483647 is always false [-Wtautological-type-limit-compare] + 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + ``` -- curl/urlapi.h: tiny typo + Seen on macOS Intel: + ``` + .../curl/tests/server/sws.c:440:64: warning: field precision should have type + 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] + msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" + , + ~~^~ + 1 warning generated. + ``` -- github/labeler: make HYPER.md set Hyper and not TLS + Closes #11925 -- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 +Jay Satiro (24 Sep 2023) - 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need - to specify version changes for earlier releases in the generated output. +- url: fix netrc info message - This ups the limit from the previous 7.30.0 (Apr 12 2013) + - Fix netrc info message to use the generic ".netrc" filename if the + user did not specify a netrc location. - This hides roughly 35 "added in" mentions. + - Update --netrc doc to add that recent versions of curl on Windows + prefer .netrc over _netrc. - Closes #11651 + Before: + * Couldn't find host google.com in the (nil) file; using defaults -Jay Satiro (10 Aug 2023) + After: + * Couldn't find host google.com in the .netrc file; using defaults -- bug_report: require reporters to specify curl and os versions + Closes https://github.com/curl/curl/pull/11904 - - Change curl version and os sections from single-line input to - multi-line textarea. +Dan Fandrich (23 Sep 2023) - - Require curl version and os sections to be filled out before report - can be submitted. +- wolfssh: do cleanup in Curl_ssh_cleanup - Closes https://github.com/curl/curl/pull/11636 + Closes: #11921 -Daniel Stenberg (9 Aug 2023) +Daniel Stenberg (24 Sep 2023) -- gen.pl: replace all single quotes with aq +- tool_listhelp: regenerated - - this prevents man from using a unicode sequence for them - - which then allows search to work properly + Polished the --ipfs-gateway description - Closes #11645 + Fixed the --trace-config description -Viktor Szakats (9 Aug 2023) + The script also fixed some other small mistakes -- cmake: fix to use variable for the curl namespace + Closes #11923 - Replace (wrong) literal with a variable to specify the curl - namespace. +Viktor Szakats (23 Sep 2023) - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 +- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) - Reported-by: balikalina on Github - Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 - 20d30#r123923098 - Closes #11629 + Also fix to export all symbols in Windows debug builds, making + `-debug-dyn` builds work with `-DCURL_STATICLIB` set. -- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms + Ref: https://github.com/curl/curl/pull/11914 (same for CMake) - 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing - libcurl objects for shared and static targets. + Closes #11924 - The above automatically enabled for Windows builds, with an option to - disable with `SHARE_LIB_OBJECT=OFF`. +Daniel Stenberg (23 Sep 2023) - This patch extend this feature to all platforms as a manual option. - You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects - are built in PIC mode, meaning the static lib will also have PIC code. +- quic: set ciphers/curves the same way regular TLS does - [EXPERIMENTAL] + for OpenSSL/BoringSSL - Closes #11627 + Fixes #11796 + Reported-by: Karthikdasari0423 on github + Assisted-by: Jay Satiro + Closes #11836 -- cmake: assume `wldap32` availability on Windows +- test457: verify --max-filesize with chunked encoding - This system library first shipped with Windows ME, available as an extra - install for some older releases (according to [1]). The import library - was present already in old MinGW 3.4.2 (year 2007). +- lib: let the max filesize option stop too big transfers too - Drop the feature check and its associated `HAVE_WLDAP32` variable. + Previously it would only stop them from getting started if the size is + known to be too big then. - To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` - CMake option, like before. + Update the libcurl and curl docs accordingly. - [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html + Fixes #11810 + Reported-by: Elliot Killick + Assisted-by: Jay Satiro + Closes #11820 - Reviewed-by: Jay Satiro - Closes #11624 +Viktor Szakats (23 Sep 2023) -Daniel Stenberg (9 Aug 2023) +- mingw: delete support for legacy mingw.org toolchain -- page-header: move up a URL paragraph from GLOBBING to URL + Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: + https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ + Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. + It supported the x86 CPU only and used a old Windows API header and + implib set, often causing issues. It also misses most modern Windows + features, offering old versions of both binutils and gcc (no llvm/clang + support). It was last updated 2 years ago. -- variable.d: output the function names table style + curl now relies on toolchains based on the mingw-w64 project: + https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ + https://www.msys2.org/ https://github.com/msys2/msys2 + https://github.com/mstorsjo/llvm-mingw + (Also available via Linux and macOS package managers.) - Also correct the url function name in the header + Closes #11625 - Closes #11641 +Mark Gaiser (23 Sep 2023) -- haproxy-clientip.d: remove backticks +- curl: add support for the IPFS protocols: - This is not markdown + - ipfs:// + - ipns:// - Follow-up to 0a75964d0d94a4 + This allows you tu use ipfs in curl like: + curl ipfs:// + and + curl ipns:// - Closes #11639 + For more information consult the readme at: + https://curl.se/docs/ipfs.html -- RELEASE-NOTES: synced + Closes #8805 -- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens +Daniel Stenberg (23 Sep 2023) - Reported-by: FC Stegerman - Fixes #11635 - Closes #11637 +- bufq: remove Curl_bufq_skip_and_shift (unused) -- cmdline-opts/page-header: reorder, clean up + Closes #11915 - - removed some unnecessary blurb to focus - - moved up the more important URL details - - put "globbing" into its own subtitle and moved down a little - - mention the online man page in the version section +- scripts/singleuse.pl: add curl_global_trace - Closes #11638 +Viktor Szakats (22 Sep 2023) -- c-hyper: adjust the hyper to curlcode conversion +- cmake: fix unity symbol collisions in h2 builds - Closes #11621 + Regression from 331b89a319d0067fa1e6441719307cfef9c7960f -- test2306: make it use a persistent connection + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #11912 - + enable verbose already from the start +Daniel Stenberg (22 Sep 2023) - Closes #11621 +- RELEASE-NOTES: synced -eppesuig (8 Aug 2023) +Dan Fandrich (21 Sep 2023) -- list-only.d: mention SFTP as supported protocol +- github/labeler: improve the match patterns - Closes #11628 + This includes new rules for setting the appleOS and logging labels and + matches on some example files. Also, enable dot mode for wildcard + matches in the .github directory. -Daniel Stenberg (8 Aug 2023) +Daniel Stenberg (21 Sep 2023) -- request.d: use .TP for protocol "labels" +- upload-file.d: describe the file name slash/backslash handling - To render the section nicer in man page. + Closes #11911 - Closes #11630 +Jakub Jelen (21 Sep 2023) -- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP +- libssh: cap SFTP packet size sent - ... as documented. + Due to libssh limitations - Update test 3201 and 3202 accordingly. + Signed-off-by: Jakub Jelen - Reported-by: Markus Sommer - Fixes #11619 - Closes #11626 + Closes #11804 -- page-footer: QLOGDIR works with ngtcp2 and quiche +Daniel Stenberg (21 Sep 2023) - It previously said "both" backends which is confusing as we currently - have three... +- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 - Closes #11631 + Closes #11905 -Stefan Eissing (8 Aug 2023) +- mailmap: unify Michael Osipov under a single email -- http3: quiche, handshake optimization, trace cleanup +Ted Lyngmo (21 Sep 2023) - - load x509 store after clienthello - - cleanup of tracing +- docs: use CURLSSLBACKEND_NONE - Closes #11618 + [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in + documentation and examples. -Daniel Stenberg (8 Aug 2023) + Signed-off-by: Ted Lyngmo -- ngtcp2: remove dead code + Closes #11909 - 'result' is always zero (CURLE_OK) at this point +Dan Fandrich (21 Sep 2023) - Detected by Coverity +- github/labeler: give the sync-labels config item a default value - Closes #11622 + This shouldn't be necessary and is likely a bug with this beta version + of the labeller. -Viktor Szakats (8 Aug 2023) + Also, fix the negative matches for the documentation label. -- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` + Follow-up to dd12b452a + Closes #11907 - OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the - popular forks (yet). Use the macro itself to detect its presence, - replacing the hard-wired fork-specific conditions. +- github/labeler: fix up more the labeler config format - This way the feature will enable automatically when forks implement it, - while also shorter and possibly requiring less future maintenance. + The new version didn't like the workaround we had for a bug in the + previous labeler version, and it should no longer be needed. - Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 + Follow-up to dd12b452a + Closes #11906 - Reviewed-by: Jay Satiro - Closes #11617 +- github/labeler: fix indenting to try to appease labeller -- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 + Follow-up to dd12b452a - LibreSSL 3.4.1 (2021-10-14) added support for - `SSL_CTX_set_ciphersuites`. +Jay Satiro (21 Sep 2023) - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt +- libssh2: fix error message on failed pubkey-from-file - Reviewed-by: Jay Satiro - Closes #11616 + - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error + message "SSH public key authentication failed: Reason unknown (-1)". -- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 + When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a + generic error and therefore doesn't set an error message. AFAICT that is + not documented behavior. - LibreSSL 3.5.0 (2022-02-24) added support for - `SSL_CTX_set_keylog_callback`. + Prior to this change libcurl retrieved the last set error message which + would be from a previous function failing. That resulted in misleading + auth failed error messages in verbose mode. - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt + Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 + Reported-by: consulion@users.noreply.github.com - Reviewed-by: Jay Satiro - Closes #11615 + Closes https://github.com/curl/curl/pull/11881 -- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks +Stefan Eissing (21 Sep 2023) - - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is - also not used by curl, but it is by its optional dependency `librtmp`. - Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This - library has been available since the early days of Windows. +- pytest: exclude test_03_goaway in CI runs due to timing dependency - - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present - since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and - other existing logic already assumed this lib being present, so delete - the check and replace the detection variable with `WIN32` and always - add `ws2_32` on Windows. + Closes #11860 - Closes #11612 +- lib: disambiguate Curl_client_write flag semantics -Daniel Gustafsson (8 Aug 2023) + - use CLIENTWRITE_BODY *only* when data is actually body data + - add CLIENTWRITE_INFO for meta data that is *not* a HEADER + - debug assertions that BODY/INFO/HEADER is not used mixed + - move `data->set.include_header` check into Curl_client_write + so protocol handlers no longer have to care + - add special in FTP for `data->set.include_header` for historic, + backward compatible reasons + - move unpausing of client writes from easy.c to sendf.c, so that + code is in one place and can forward flags correctly -- crypto: ensure crypto initialization works + Closes #11885 - Make sure that context initialization during hash setup works to avoid - going forward with the risk of a null pointer dereference. +Patrick Monnerat (21 Sep 2023) - Reported-by: Philippe Antoine on HackerOne - Assisted-by: Jay Satiro - Assisted-by: Daniel Stenberg +- tftpd: always use curl's own tftp.h - Closes #11614 + Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects + and reports a stringop-overread warning: -Viktor Szakats (7 Aug 2023) + tftpd.c: In function ‘write_behind.isra’: + tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f + rom a region of size 0 [-Wstringop-overread] + 485 | return write(test->ofile, writebuf, count); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + In file included from tftpd.c:71: + /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 + 58 | char tu_data[0]; /* data or error stri + ng */ + | ^~~~~~~ -- openssl: switch to modern init for LibreSSL 2.7.0+ + This occurs because writebuf points to this field and the latter + cannot be considered as being of dynamic length because it is not + the last field in the structure. Thus it is bound to its declared + size. - LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, - `OPENSSL_init_ssl()` function and deprecated the old, manual init - method, as seen in OpenSSL 1.1.0. Switch to the modern method when - available. + This commit always uses curl's own version of tftp.h where the + target field is last in its structure, effectively avoiding the + warning. - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt + As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for + arpa/tftp.h are removed. - Reviewed-by: Daniel Stenberg - Closes #11611 + Closes #11897 -Daniel Stenberg (7 Aug 2023) +Dan Fandrich (20 Sep 2023) -- gskit: remove +- test1474: make precheck more robust on non-Solaris systems - We remove support for building curl with gskit. + If uname -r returns something odd, perl could return an error code and + the test would be erroneously skipped. The qx// syntax avoid this. - - This is a niche TLS library, only running on some IBM systems - - no regular curl contributors use this backend - - no CI builds use or verify this backend - - gskit, or the curl adaption for it, lacks many modern TLS features - making it an inferior solution - - build breakages in this code take weeks or more to get detected - - fixing gskit code is mostly done "flying blind" + Followup to 08f9b2148 - This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has - been mentioned on the curl-library mailing list. +- github/labeler: switch to the 5 beta version - It could be brought back, this is not a ban. Given proper effort and - will, gskit support is welcome back into the curl TLS backend family. + This version adds an important feature that will allow more PRs to be + labelled. Rather than being limited to labeling PRs with files that + match a single glob, it can now label them if multiple changed files + match any one of a number of globs. - Closes #11460 +Daniel Stenberg (20 Sep 2023) -- RELEASE-NOTES: synced +- lib: enable hmac for digest as well -Dan Fandrich (7 Aug 2023) + Previously a build that disabled NTLM and aws-sigv4 would fail to build + since the hmac was disabled, but it is also needed for digest auth. -- THANKS-filter: add a name typo + Follow-up to e92edfbef64448ef -Stefan Eissing (7 Aug 2023) + Fixes #11890 + Reported-by: Aleksander Mazur + Closes #11896 -- http3/ngtcp2: shorten handshake, trace cleanup +- idn: if idn2_check_version returns NULL, return error - - shorten handshake timing by delayed x509 store load (OpenSSL) - as we do for HTTP/2 - - cleanup of trace output, align with HTTP/2 output + ... this avoids a NULL dereference for this unusual case. - Closes #11609 + Reported-by: s0urc3_ on hackerone + Closes #11898 -Daniel Stenberg (7 Aug 2023) +- http: fix CURL_DISABLE_BEARER_AUTH breakage -- headers: accept leading whitespaces on first response header + When bearer auth was disabled, the if/else logic got wrong and caused + problems. - This is a bad header fold but since the popular browsers accept this - violation, so does curl now. Unless built with hyper. + Follow-up to e92edfbef64448ef461 + Fixes #11892 + Reported-by: Aleksander Mazur + Closes #11895 - Add test 1473 to verify and adjust test 2306. +Michael Osipov (20 Sep 2023) - Reported-by: junsik on github - Fixes #11605 - Closes #11607 +- wolfssl: allow capath with CURLOPT_CAINFO_BLOB -- include/curl/mprintf.h: add __attribute__ for the prototypes + Remain consistent with OpenSSL. While CAfile is nulled as documented + with CURLOPT_CAINFO_BLOB, CApath remains intact. - - if gcc or clang is used - - if __STDC_VERSION__ >= 199901L, which means greater than C90 - - if not using mingw - - if CURL_NO_FMT_CHECKS is not defined + Closes #11886 - Closes #11589 +- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c -- tests: fix bad printf format flags in test code + Closes #11886 -- tests: fix header scan tools for attribute edits in mprintf.h +Dan Fandrich (19 Sep 2023) -- cf-socket: log successful interface bind +- test1474: disable test on NetBSD, OpenBSD and Solaris 10 - When the setsockopt SO_BINDTODEVICE operation succeeds, output that in - the verbose output. + These kernels only send a fraction of the requested amount of the first + large block, invalidating the assumptions of the test and causing it to + fail. - Ref: #11599 - Closes #11608 + Assisted-by: Christian Weisgerber + Ref: https://curl.se/mail/lib-2023-09/0021.html + Closes #11888 -- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled +Ryan Schmidt (20 Sep 2023) - Ref: #11457 - Closes #11606 +- cmake, configure: also link with CoreServices -- CURLOPT_SSL_VERIFYPEER.3: add two more see also options + When linking with CoreFoundation, also link with CoreServices which is + apparently required to avoid an NSInvalidArgumentException in software + linking with libcurl on macOS Sonoma 14 and later. - CURLINFO_CAINFO and CURLINFO_CAPATH + Fixes #11893 + Closes #11894 - Closes #11603 +Marc Hoersken (19 Sep 2023) -- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice +- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket - Closes #11007 + These dependencies are now already included in the Docker image. -Graham Campbell (6 Aug 2023) + Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 + b41d15606e97f38cf312c1ce56 -- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 + Closes #11889 - Closes #11585 +Daniel Stenberg (19 Sep 2023) -Daniel Stenberg (6 Aug 2023) +- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files -- TODO: add *5* entries for aws-sigv4 + Ref: #11883 + Reported-by: Michael Osipov + Closes #11884 - Closes #7559 - Closes #8107 - Closes #8810 - Closes #9717 - Closes #10129 +- RELEASE-NOTES: synced -- TODO: LDAP Certificate-Based Authentication +- test3103: CURLOPT_COOKIELIST test - Closes #9641 +- cookie: set ->running in cookie_init even if data is NULL -Stefan Eissing (6 Aug 2023) + This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) -- http2: cleanup trace messages + Test 3103 verifies. - - more compact format with bracketed stream id - - all frames traced in and out + Fixes #11875 + Reported-by: wangp on github + Closes #11876 - Closes #11592 +- test498: total header size for all redirects is larger than accepted -Daniel Stenberg (6 Aug 2023) +- http: use per-request counter to check too large headers -- tests/tftpd+mqttd: make variables static to silence picky warnings + Not the counter that accumulates all headers over all redirects. - Closes #11594 + Follow-up to 3ee79c1674fd6 -- docs/cmdline: remove repeated working for negotiate + ntlm + Do a second check for 20 times the limit for the accumulated size for + all headers. - The extra wording is added automatically by the gen.pl tool + Fixes #11871 + Reported-by: Joshix-1 on github + Closes #11872 - Closes #11597 +Jay Satiro (18 Sep 2023) -- docs/cmdline: add small "warning" to verbose options +- THANKS: add Eric Murphy - "Note that verbose output of curl activities and network traffic might - contain sensitive data, including user names, credentials or secret data - content. Be aware and be careful when sharing trace logs with others." + He reported #11850 (quiche build error) but I forgot to add a + 'reported-by' entry in the fix 267e14f1. - Closes #11596 +Daniel Stenberg (18 Sep 2023) -- RELEASE-NOTES: synced +- h2-proxy: remove left-over mistake in drain_tunnel() -- pingpong: don't use *bump_headersize + Left-over from 331b89a319 - We use that for HTTP(S) only. + Reported-by: 南宫雪珊 - Follow-up to 3ee79c1674fd6 + Closes https://github.com/curl/curl/pull/11877 - Closes #11590 +vvb2060 (18 Sep 2023) -- urldata: remove spurious parenthesis to unbreak no-proxy build +- lib: failf/infof compiler warnings - Follow-up to e12b39e13382 + Closes #11874 - Closes #11591 +Daniel Stenberg (17 Sep 2023) -- easy: don't call Curl_trc_opt() in disabled-verbose builds +- rand: fix 'alnum': array is too small to include a terminating null character - Follow-up to e12b39e133822c6a0 + It was that small on purpose, but this change now adds the null byte to + avoid the error. - Closes #11588 + Follow-up to 3aa3cc9b052353b1 -- http: use %u for printfing int + Reported-by: Dan Fandrich + Ref: #11838 + Closes #11870 - Follow-up to 3ee79c1674fd6f99e8efca5 +Mathias Fuchs (16 Sep 2023) - Closes #11587 +- cmake: fix the help text to the static build option in CMakeLists.txt -Goro FUJI (3 Aug 2023) + Closes #11843 -- vquic: show stringified messages for errno +John Haugabook (16 Sep 2023) - Closes #11584 +- MANUAL.md: change domain to example.com -Stefan Eissing (3 Aug 2023) + Closes #11866 -- trace: make tracing available in non-debug builds +Daniel Stenberg (16 Sep 2023) - Add --trace-config to curl +- doh: inherit DEBUGFUNCTION/DATA - Add curl_global_trace() to libcurl + When creating new transfers for doing DoH, they now inherit the debug + settings from the initiating transfer, so that the application can + redirect and handle the verbose output correctly even for the DoH + transfers. - Closes #11421 + Reported-by: calvin2021y on github + Fixes #11864 + Closes #11869 -Daniel Stenberg (3 Aug 2023) +Dan Fandrich (16 Sep 2023) -- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" +- http_aws_sigv4: fix sorting with empty parts - See also https://github.com/curl/curl/pull/7507 + When comparing with an empty part, the non-empty one is always + considered greater-than. Previously, the two would be considered equal + which would randomly place empty parts amongst non-empty ones. This + showed as a test 439 failure on Solaris as it uses a different + implementation of qsort() that compares parts differently. -- TODO: add "WebSocket read callback" + Fixes #11855 + Closes #11868 - remove "Upgrade to websockets" as we already have this +- CI: ignore the "flaky" and "timing-dependent" test results - Closes #11402 + CI builds will now run these tests, but will ignore the results if they + fail. The relevant tests are ones that are sensitive to timing or + have edge conditions that make them more likely to fail on CI servers, + which are often heavily overloaded and slow. -- test497: verify rejecting too large incoming headers + This change only adds two additional tests to be ignored, since the + others already had the flaky keyword. -- http: return error when receiving too large header set + Closes #11865 - To avoid abuse. The limit is set to 300 KB for the accumulated size of - all received HTTP headers for a single response. Incomplete research - suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to - 1MB. +- runtests: eliminate a warning on old perl versions - Closes #11582 + The warning "Use of implicit split to @_ is deprecated" showed between + perl versions about 5.8 through 5.11. -Stefan Eissing (3 Aug 2023) +- tests: log the test result code after each libtest -- http2: upgrade tests and add fix for non-existing stream + This makes it easier to determine the test status. Also, capitalize + FAILURE and ABORT messages in log lines to make them easier to spot. - - check in h2 filter recv that stream actually exists - and return error if not - - add test for parallel, extreme h2 upgrades that fail if - connections get reused before fully switched - - add h2 upgrade upload test just for completeness +Harry Sintonen (16 Sep 2023) - Closes #11563 +- misc: better random strings -Viktor Szakats (3 Aug 2023) + Generate alphanumerical random strings. -- tests: ensure `libcurl.def` contains all exports + Prior this change curl used to create random hex strings. This was + mostly okay, but having alphanumerical random strings is better: The + strings have more entropy in the same space. - Add `test1279` to verify that `libcurl.def` lists all exported API - functions found in libcurl headers. + The MIME multipart boundary used to be mere 64-bits of randomness due + to being 16 hex chars. With these changes the boundary is 22 + alphanumerical chars, or little over 130 bits of randomness. - Also: + Closes #11838 - - extend test suite XML `stdout` tag with the `loadfile` attribute. +Daniel Stenberg (15 Sep 2023) - - fix `tests/extern-scan.pl` and `test1135` to include websocket API. +- cookie: reduce variable scope, add const - - use all headers (sorted) in `test1135` instead of a manual list. +- cookie: do not store the expire or max-age strings - - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. + Convert it to an expire time at once and save memory. - - add `libcurl.def` to the auto-labeler GHA task. + Closes #11862 - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 +- cookie: remove unnecessary struct fields - Closes #11570 + Plus: reduce the hash table size from 256 to 63. It seems unlikely to + make much of a speed difference for most use cases but saves 1.5KB of + data per instance. -Daniel Stenberg (2 Aug 2023) + Closes #11862 -- url: change default value for CURLOPT_MAXREDIRS to 30 +- RELEASE-NOTES: synced - It was previously unlimited by default, but that's not a sensible - default. While changing this has a remote risk of breaking an existing - use case, I figure it is more likely to actually save users from loops. + Bumped to 8.4.0, the next presumed version - Closes #11581 +Dan Fandrich (14 Sep 2023) -- lib: fix a few *printf() flag mistakes +- test2600: remove special case handling for USE_ALARM_TIMEOUT - Reported-by: Gisle Vanem - Ref: #11574 - Closes #11579 + This was originally added to handle platforms that supported only 1 + second granularity in connect timeouts, but after some recent changes + the test currently permafails on several Windows platforms. -Samuel Chiang (2 Aug 2023) + The need for this special-case was removed in commit 8627416, which + increased the connect timeout in all cases to well above 1 second. -- openssl: make aws-lc version support OCSP + Fixes #11767 + Closes #11849 - And bump version in CI +Daniel Stenberg (14 Sep 2023) - Closes #11568 +- SECURITY-PROCESS.md. call it vulnerability disclosure policy -Daniel Stenberg (2 Aug 2023) + SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md -- tool: make the length argument an int for printf()-.* flags + This a name commonly used for a document like this. This name helps + users find it. - Closes #11578 + Closes #11852 -- tool_operate: fix memory leak when SSL_CERT_DIR is used +Junho Choi (14 Sep 2023) - Detected by Coverity +- quiche: fix build error with --with-ca-fallback - Follow-up to 29bce9857a12b6cfa726a5 + - Fix build error when curl is built with --with-quiche + and --with-ca-fallback. - Closes #11577 + - Add --with-ca-fallback to the quiche CI job. -- tool/var: free memory on OOM + Fixes https://github.com/curl/curl/issues/11850 + Closes https://github.com/curl/curl/pull/11847 - Coverity detected this memory leak in OOM situation +Jay Satiro (14 Sep 2023) - Follow-up to 2e160c9c652504e +- escape: replace Curl_isunreserved with ISUNRESERVED - Closes #11575 + - Use the ALLCAPS version of the macro so that it is clear a macro is + being called that evaluates the variable multiple times. -Viktor Szakats (2 Aug 2023) + - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates + a variable multiple times. -- gha: bump libressl and mbedtls versions + This is a follow-up to 291d225a which changed Curl_isunreserved into an + alias macro for ISUNRESERVED. The problem is the former is not easily + identified as a macro by the caller, which could lead to a bug. - Closes #11573 + For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but + Curl_isunreserved(*foo++) is not even though they both are the same. -Jay Satiro (2 Aug 2023) + Closes https://github.com/curl/curl/pull/11846 -- schannel: fix user-set legacy algorithms in Windows 10 & 11 +Dan Fandrich (13 Sep 2023) - - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then - use the SCHANNEL_CRED legacy structure to pass the list to Schannel. +- tests: increase the default server logs lock timeout - - If the user set both a legacy algorithm list and a TLS 1.3 cipher list - then abort. + This timeout is used to wait for the server to finish writing its logs + before checking them against the expected values. An overloaded machine + could take more than the two seconds previously allocated, so increase + the timeout to 5 seconds. - Although MS doesn't document it, Schannel will not negotiate TLS 1.3 - when SCHANNEL_CRED is used. That means setting a legacy algorithm list - limits the user to earlier versions of TLS. + Ref: #11328 + Closes #11834 - Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would - ignore legacy algorithms in Windows 10 1809 and later. +- tests: increase TEST_HANG_TIMEOUT in two tests - Reported-by: zhihaoy@users.noreply.github.com + These tests had a 5 second timeout compared to 60 seconds for all other + tests. Make these consistent with the others for more reliability on + heavily-loaded machines. - Fixes https://github.com/curl/curl/pull/10741 - Closes https://github.com/curl/curl/pull/10746 + Ref: #11328 -Daniel Stenberg (2 Aug 2023) +- test1056: disable on Windows -- variable.d: setting a variable again overwrites it + This test relies on the IPv6 scope field being ignored when connecting to + ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit + dodgy, but it works on all our test platforms except Windows. This + test was disabled manually on all Windows CI builds already, so instead + add an incompatible feature and precheck so it's skipped on Windows + everywhere automatically. - Reported-by: Niall McGee - Bug: https://twitter.com/niallmcgee/status/1686523075423322113 - Closes #11571 +- test587: add a slight delay after test -Jay Satiro (2 Aug 2023) + This test is designed to connect to the server, then immediately send a + few bytes and disconnect. In some situations, such as on a loaded + server, this doesn't give the server enough time to write its lock file + before its existence is checked. The test harness then fails to find the + server's input log file (because it hasn't been written yet) and fails + the test. By adding a short delay after the test, the HTTP server has + enough time to write its lock file which gives itself more time to write + its remaining files. -- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting + Ref: #11328 - - Re-wrap CURLSSLOPT_ALLOW_BEAST description. +- tests: stop overriding the lock timeout -Daniel Stenberg (2 Aug 2023) + These tests reduce the server lock wait timeout which can increase + flakiness on loaded machines. Since this is merely an optimization, + eliminate them in favour of reliability. -- RELEASE-NOTES: synced + Ref: #11328 -- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set +- tests: add some --expect100-timeout to reduce timing dependencies - Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not - used, thus unnecessarily asking for addresses that will not be used. + These tests can fail when the test machine is so slow that the test HTTP + server didn't get a chance to complete before the client's one second + 100-continue timeout triggered. Increase that 1 second to 999 seconds so + this situation doesn't happen. - Reported-by: Joseph Tharayil - Fixes #11564 - Closes #11565 + Ref: #11328 -- docs: link to the website versions instead of markdowns +- test661: return from test early in case of curl error - ... to make the links work when the markdown is converted to webpages on - https://curl.se +- tests: add the timing-dependent keyword on several tests - Reported-by: Maurício Meneghini Fauth - Fixes https://github.com/curl/curl-www/issues/272 - Closes #11569 + These are ones likely to fail on heavily-loaded machines that alter the + normal test timing. Most of these tests already had the flaky keyword + since this condition makes them more likely to fail on CI. -Viktor Szakats (1 Aug 2023) +- test1592: greatly increase the maximum test timeout -- cmake: cache more config and delete unused ones + It was too short to be reliable on heavily loaded CI machines, and + as a fail-safe only, it didn't need to be short. - - cache more Windows config results for faster initialization. + Ref: #11328 - - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. +- test: minor test cleanups - - delete dead references to `sys/utsname.h`. + Remove an obsolete block of code in tests 2032 & 576. + Add a comment in test 1474. - Closes #11551 +- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts -- egd: delete feature detection and related source code + This gives more of a margin for error when running on overloaded CI + servers. - EGD is Entropy Gathering Daemon, a socket-based entropy source supported - by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it - a while ago. + Ref: #11328 - Its detection in CMake was broken all along because OpenSSL libs were - not linked at the point of feature check. +- tests: improve SLOWDOWN test reliability by reducing sent data - Delete detection from both cmake and autotools, along with the related - source snippet, and the `--with-egd-socket=` `./configure` option. + These tests are run in SLOWDOWN mode which adds a 10 msec delay after + each character output, which means it takes at least 1.6 seconds (and + 320 kernel calls) just to get through the long welcome banner. On an + overloaded system, this can end up taking much more than 1.6 seconds, + and even more than the 7 or 16 second curl timeout that the tests rely + on, causing them to fail. Reducing the size of the welcome banner drops + the total number of characters sent before the transfer starts by more + than half, which reduces the opportunity for test-breaking slowdowns by + the same amount. - Closes #11556 + Ref: #11328 -Stefan Eissing (1 Aug 2023) +- test650: fix an end tag typo -- tests: fix h3 server check and parallel instances +Jay Satiro (13 Sep 2023) - - fix check for availability of nghttpx server - - add `tcp` frontend config for same port as quic, as - without this, port 3000 is bound which clashes for parallel - testing +- tool_cb_wrt: fix debug assertion - Closes #11553 + - Fix off-by-one out-of-bounds array index in Windows debug assertion. -Daniel Stenberg (1 Aug 2023) + Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 + Reported-by: Gisle Vanem -- docs/cmdline-opts: spellfixes, typos and polish +Daniel Stenberg (13 Sep 2023) - To make them accepted by the spell checker +- ctype: add ISUNRESERVED() - Closes #11562 + ... and make Curl_isunreserved() use that macro instead of providing a + separate funtion for the purpose. -- CI/spellcheck: build curl.1 and spellcheck it + Closes #11840 - Added acceptable words +Version 8.3.0 (13 Sep 2023) - Closes #11562 +Daniel Stenberg (13 Sep 2023) -Alexander Jaeger (1 Aug 2023) +- RELEASE-NOTES: syn ced -- misc: fix various typos + curl 8.3.0 release - Closes #11561 +- THANKS: contributors from 8.3.0 -Daniel Stenberg (1 Aug 2023) +Thorsten Klein (12 Sep 2023) -- http2: avoid too early connection re-use/multiplexing +- cmake: set SIZEOF_LONG_LONG in curl_config.h - HTTP/1 connections that are upgraded to HTTP/2 should not be picked up - for reuse and multiplexing by other handles until the 101 switching - process is completed. + in order to support 32bit builds regarding wolfssl CTC_SETTINGS - Lots-of-debgging-by: Stefan Eissing - Reported-by: Richard W.M. Jones - Bug: https://curl.se/mail/lib-2023-07/0045.html - Closes #11557 + Closes #11839 -- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" +Jay Satiro (12 Sep 2023) - This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. +- curl_ngtcp2: fix error message - It's a user error for supplying incomplete information to the build system. +- http_aws_sigv4: handle no-value user header entries - Reported-by: Ryan Schmidt - Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 + - Handle user headers in format 'name:' and 'name;' with no value. -Viktor Szakats (1 Aug 2023) + The former is used when the user wants to remove an internal libcurl + header and the latter is used when the user actually wants to send a + no-value header in the format 'name:' (note the semi-colon is converted + by libcurl to a colon). -- cmake: add support for single libcurl compilation pass + Prior to this change the AWS header import code did not special case + either of those and the generated AWS SignedHeaders would be incorrect. - Before this patch CMake builds used two separate compilation passes to - build the shared and static libcurl respectively. This patch allows to - reduce that to a single pass if the target platform and build settings - allow it. + Reported-by: apparentorder@users.noreply.github.com - This reduces CMake build times when building both static and shared - libcurl at the same time, making these dual builds an almost zero-cost - option. + Ref: https://curl.se/docs/manpage.html#-H - Enable this feature for Windows builds, where the difference between the - two passes was the use of `__declspec(dllexport)` attribute for exported - API functions for the shared builds. This patch replaces this method - with the use of `libcurl.def` at DLL link time. + Fixes https://github.com/curl/curl/issues/11664 + Closes https://github.com/curl/curl/pull/11668 - Also update `Makefile.mk` to use `libcurl.def` to export libcurl API - symbols on Windows. This simplifies (or fixes) this build method (e.g. - in curl-for-win, which generated a `libcurl.def` from `.h` files using - an elaborate set of transformations). +Dan Fandrich (11 Sep 2023) - `libcurl.def` has the maintenance cost of keeping the list of public - libcurl API symbols up-to-date. This list seldom changes, so the cost - is low. +- CI: run pytest with the -v option - Closes #11546 + This lists of the test cases being run so it can be tracked over time. -- cmake: detect `SSL_set0_wbio` in OpenSSL + Closes #11824 - Present in OpenSSL 1.1.0 and BoringSSL. - Missing from LibreSSL 3.8.0. +Daniel Stenberg (11 Sep 2023) - Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 +- HTTP3: the msquic backend is not functional - While here, also fix `RAND_egd()` detection which was broken, likely all - along. This feature is probably broken with CMake builds and also - requires a sufficiently obsolete OpenSSL version, so this part of the - update was not tested. + I ask that we do not submit bugs for this backend just yet as we know it + does not fully work. - Closes #11555 + Closes #11831 + Closes #11819 -- cmake: fixup H2 duplicate symbols for unity builds +- aws_sigv4: the query canon code miscounted URL encoded input - Closes #11550 + Added some extra ampersands to test 439 to verify "blank" query parts -Pablo Busse (1 Aug 2023) + Follow-up to fc76a24c53b08cdf -- openssl: Support async cert verify callback + Closes #11829 - - Update the OpenSSL connect state machine to handle - SSL_ERROR_WANT_RETRY_VERIFY. +vvb2060 (11 Sep 2023) - This allows libcurl users that are using custom certificate validation - to suspend processing while waiting for external I/O during certificate - validation. +- quic: don't set SNI if hostname is an IP address - Closes https://github.com/curl/curl/pull/11499 + We already do this for TLS connections. -Jay Satiro (1 Aug 2023) + RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in + "HostName". -- tool_cb_wrt: fix invalid unicode for windows console + Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 - - Suppress an incomplete UTF-8 sequence at the end of the buffer. + Fixes https://github.com/curl/curl/issues/11827 + Closes https://github.com/curl/curl/pull/11828 - - Attempt to reconstruct incomplete UTF-8 sequence from prior call(s) - in current call. +Daniel Stenberg (10 Sep 2023) - Prior to this change, in Windows console UTF-8 sequences split between - two or more calls to the write callback would cause invalid "replacement - characters" U+FFFD to be printed instead of the actual Unicode - character. This is because in Windows only UTF-16 encoded characters are - printed to the console, therefore we convert the UTF-8 contents to - UTF-16, which cannot be done with partial UTF-8 sequences. +- RELEASE-NOTES: synced - Reported-by: Maksim Arhipov - - Fixes https://github.com/curl/curl/issues/9841 - Closes https://github.com/curl/curl/pull/10890 - -Daniel Stenberg (1 Aug 2023) +Benoit Pierre (10 Sep 2023) -- sectransp: prevent CFRelease() of NULL +- configure: fix `HAVE_TIME_T_UNSIGNED` check - When SecCertificateCopyCommonName() returns NULL, the common_name - pointer remains set to NULL which apparently when calling CFRelease() on - (sometimes?) crashes. + The syntax was incorrect (need a proper main body), and the test + condition was wrong (resulting in a signed `time_t` detected as + unsigned). - Reported-by: Guillaume Algis - Fixes #9194 - Closes #11554 + Closes #11825 -Jay Satiro (1 Aug 2023) +Daniel Stenberg (9 Sep 2023) -- vtls: clarify "ALPN: offers" message +- THANKS-filter: pszlazak on github - Before: - * ALPN: offers h2,http/1.1 +pszlazak (9 Sep 2023) - After: - * ALPN: curl offers h2,http/1.1 +- include.d: explain headers not printed with --fail before 7.75.0 - Bug: https://curl.se/mail/lib-2023-07/0041.html - Reported-by: Richard W.M. Jones - Closes #11544 + Prior to 7.75.0 response headers were not printed if -f/--fail was used + and an error was reported by server. This was fixed in ab525c0 + (precedes 7.75.0). -Daniel Stenberg (1 Aug 2023) + Closes #11822 -- urlapi: make sure zoneid is also duplicated in curl_url_dup +Daniel Stenberg (8 Sep 2023) - Add several curl_url_dup() tests to the general lib1560 test. +- http_aws_sigv4: skip the op if the query pair is zero bytes - Reported-by: Rutger Broekhoff - Bug: https://curl.se/mail/lib-2023-07/0047.html - Closes #11549 + Follow-up to fc76a24c53b08cdf -Sergey (1 Aug 2023) + Spotted by OSS-Fuzz -- urlapi: fix heap buffer overflow + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 + Closes #11823 - `u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-ter - minator. +- cmdline-docs: use present tense, not future - ``` - ==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 a - t pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4 - READ of size 78 at 0x04d48c75 thread T0 - #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\c - ompiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840 - #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_p - arty\curl\lib\strdup.c:97 - #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_part - y\curl\lib\urlapi.c:1297 - #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\clien - t\third_party\curl\lib\urlapi.c:1342 - #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_ - party\curl\lib\urlapi.c:1790 - #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client - \third_party\curl\lib\url.c:1768 - #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_p - arty\curl\lib\url.c:3403 - #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_ - party\curl\lib\url.c:3888 - #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\thi - rd_party\curl\lib\multi.c:1982 - #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\ - third_party\curl\lib\multi.c:2756 - ``` + + some smaller cleanups - Closes #11560 + Closes #11821 -Daniel Stenberg (31 Jul 2023) +- cmdline-docs: make sure to phrase it as "added in ...." -- curl: make %output{} in -w specify a file to write to + References to things that were added or changed in a specific version + should be specified as "(added in [version]) for two reasons: - It can be used multiple times. Use %output{>>name} to append. + 1 - consistency - Add docs. Test 990 and 991 verify. + 2 - to allow gen.pl to strip them out if deemed referring to too old + versions - Idea: #11400 - Suggested-by: ed0d2b2ce19451f2 - Closes #11416 + Closes #11821 -- RELEASE-NOTES: synced +Jay Satiro (8 Sep 2023) -- tool: add "variable" support +- docs: mark --ssl-revoke-best-effort as Schannel specific - Add support for command line variables. Set variables with --variable - name=content or --variable name@file (where "file" can be stdin if set - to a single dash (-)). + Closes https://github.com/curl/curl/pull/11760 - Variable content is expanded in option parameters using "{{name}}" - (without the quotes) if the option name is prefixed with - "--expand-". This gets the contents of the variable "name" inserted, or - a blank if the name does not exist as a variable. Insert "{{" verbatim - in the string by prefixing it with a backslash, like "\\{{". +Nathan Moinvaziri (8 Sep 2023) - Import an environment variable with --variable %name. It makes curl exit - with an error if the environment variable is not set. It can also rather - get a default value if the variable does not exist, using =content or - @file like shown above. +- schannel: fix ordering of cert chain info - Example: get the USER environment variable into the URL: + - Use CERT_CONTEXT's pbCertEncoded to determine chain order. - --variable %USER - --expand-url = "https://example.com/api/{{USER}}/method" + CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains + end-entity/server certificate in pbCertEncoded. We can use this pointer + to determine the order of certificates when enumerating hCertStore using + CertEnumCertificatesInStore. - When expanding variables, curl supports a set of functions that can make - the variable contents more convenient to use. It can trim leading and - trailing white space with "trim", output the contents as a JSON quoted - string with "json", URL encode it with "url" and base 64 encode it with - "b64". To apply functions to a variable expansion, add them colon - separated to the right side of the variable. They are then performed in - a left to right order. + This change is to help ensure that the ordering of the certificate chain + requested by the user via CURLINFO_CERTINFO has the same ordering on all + versions of Windows. - Example: get the contents of a file called $HOME/.secret into a variable - called "fix". Make sure that the content is trimmed and percent-encoded - sent as POST data: + Prior to this change Schannel certificate order was reversed in 8986df80 + but that was later reverted in f540a39b when it was discovered that + Windows 11 22H2 does the reversal on its own. - --variable %HOME=/home/default - --expand-variable fix@{{HOME}}/.secret - --expand-data "{{fix:trim:url}}" - https://example.com/ + Ref: https://github.com/curl/curl/issues/9706 - Documented. Many new test cases. + Closes https://github.com/curl/curl/pull/11632 - Co-brainstormed-by: Emanuele Torre - Assisted-by: Jat Satiro - Closes #11346 +Chris Talbot (8 Sep 2023) -- KNOWN_BUGS: cygwin: make install installs curl-config.1 twice +- digest: Use hostname to generate spn instead of realm - Closes #8839 + In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 -- KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14 + digest-uri-value should be serv-type "/" host , where host is: - Closes #11215 + The DNS host name or IP address for the service requested. The + DNS host name must be the fully-qualified canonical name of the + host. The DNS host name is the preferred form; see notes on server + processing of the digest-uri. -- KNOWN_BUGS: cmake outputs: no version information available + Realm may not be the host, so we must specify the host explicitly. - Closes #11158 + Note this change only affects the non-SSPI digest code. The digest code + used by SSPI builds already uses the hostname to generate the spn. -- KNOWN_BUGS: APOP authentication fails on POP3 + Ref: https://github.com/curl/curl/issues/11369 - Closes #10073 + Closes https://github.com/curl/curl/pull/11395 -- KNOWN_BUGS: hyper is slow +Daniel Stenberg (7 Sep 2023) - Closes #11203 +- docs: remove use of the word 'very' -Patrick Monnerat (31 Jul 2023) + It is mostly superfluous. proselint would complain. -- configure, cmake, lib: more form api deprecation + Closes #11818 - Introduce a --enable-form-api configure option to control its inclusion - in builds. The condition name defined for it is CURL_DISABLE_FORM_API. +- curl_multi_remove_handle.3: clarify what happens with connection - Form api code is dependent of MIME: configure and CMake handle this - dependency automatically: CMake by making it a dependent option - explicitly, configure by inheriting the MIME value by default and - rejecting explicit incompatible values. + Closes #11817 - "form-api" is now a new hidden test feature. +- RELEASE-NOTES: synced - Update libcurl modules to respect this option and adjust tests - accordingly. +- test439: verify query canonization for aws-sigv4 - Closes #9621 +- tool_operate: make aws-sigv4 not require TLS to be used -Daniel Stenberg (31 Jul 2023) + Maybe not used too often, but we want it for testing and it should work. -- mailmap: add Derzsi Dániel +- http_aws_sigv4: canonicalize the query -Derzsi Dániel (31 Jul 2023) + Percent encoding needs to be done using uppercase, and most + non-alphanumerical must be percent-encoded. -- wolfssl: support loading system CA certificates + Fixes #11794 + Reported-by: John Walker + Closes #11806 - Closes #11452 +Wyatt O'Day (7 Sep 2023) -Viktor Szakats (30 Jul 2023) +- lib: add ability to disable auths individually -- nss: delete more NSS references + Both with configure and cmake - Fix the distcheck CI failure and delete more NSS references. + Closes #11490 - Follow-up to 7c8bae0d9c9b2dfeeb008b9a316117d7b9675175 +Stefan Eissing (7 Sep 2023) - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Closes #11548 +- ngtcp2: fix handling of large requests -Daniel Stenberg (29 Jul 2023) + - requests >64K are send in parts to the filter + - fix parsing of the request to assemble it correctly + from several sends + - open a QUIC stream only when the complete request has + been collected -- nss: remove support for this TLS library + Closes #11815 - Closes #11459 +- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before -Ryan Schmidt (29 Jul 2023) + - we delay loading the x509 store to shorten the handshake time. + However an application callback installed via CURLOPT_SSL_CTX_FUNCTION + may need to have the store loaded and try to manipulate it. + - load the x509 store before invoking the app callback -- macOS: fix target detection more + Fixes #11800 + Reported-by: guoxinvmware on github + Cloes #11805 - Now SCDynamicStoreCopyProxies is called (and the required frameworks are - linked in) on all versions of macOS and only on macOS. Fixes crash due - to undefined symbol when built with the macOS 10.11 SDK or earlier. +Daniel Stenberg (7 Sep 2023) - CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and - is now only defined when SCDynamicStoreCopyProxies will actually be - called. Previously, it was defined when ENABLE_IPV6 was not defined but - SCDynamicStoreCopyProxies is not called in that case. +- krb5: fix "implicit conversion loses integer precision" warnings - TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only - when dynamic targets are enabled. TARGET_OS_MAC is always defined but - means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS. - TARGET_OS_IPHONE means any Darwin OS other than macOS. + conversions to/from enum and unsigned chars - Follow-up to c73b2f82 + Closes #11814 - Fixes #11502 - Closes #11516 +Stefan Eissing (7 Sep 2023) -Daniel Stenberg (29 Jul 2023) +- pytest: improvements -- tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR + - set CURL_CI for pytest runs in CI environments + - exclude timing sensitive tests from CI runs + - for failed results, list only the log and stat of + the failed transfer - ... used at once. + - fix type in http.c comment - Reported-by: Gabriel Corona - Fixes #11325 - Closes #11531 + Closes #11812 -Thomas M. DuBuisson (29 Jul 2023) +- CI: move on to ngtcp2 v0.19.1 -- CI: remove Lift's configuration + Closes #11809 - The Lift tool is being retired. Their site reads: +Dan Fandrich (5 Sep 2023) - "Sonatype Lift will be retiring on Sep 12, 2023, with its analysis - stopping on Aug 12, 2023." +- CI: run Circle macOS builds on x86 for now - Closes #11541 + The ARM machines aren't ready for us and requesting them now causes + warnings e-mails to be sent to some PR pushers. -Nathan Moinvaziri (29 Jul 2023) + Ref: #11771 -- Revert "schannel: reverse the order of certinfo insertions" +Viktor Szakats (5 Sep 2023) - This reverts commit 8986df802db9b5338d9d50a54232ebae4dbcf6dd. +- http3: adjust cast for ngtcp2 v0.19.0 - Windows does not guarantee a particular certificate ordering, even - though TLS may have its own ordering/relationship guarantees. Recent - versions of Windows 11 reversed the ordering of ceritifcates returned by - CertEnumCertificatesInStore, therefore this commit no longer works as - initially intended. libcurl makes no guarantees about certificate - ordering if the operating system can't. + ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` + an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. - Ref: https://github.com/curl/curl/issues/9706 + Fixes: + ``` + ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in + teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned + char') [-Wimplicit-int-conversion] + pi.ecn = (uint32_t)ecn; + ~ ^~~~~~~~~~~~~ + ``` - Closes https://github.com/curl/curl/pull/11536 + Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our + docs and CI. -wangzhikun (29 Jul 2023) + Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 + 782894a3 + Ref: https://github.com/ngtcp2/ngtcp2/pull/877 + Closes #11798 -- winbuild: improve check for static zlib +Stefan Eissing (5 Sep 2023) - - Check for zlib static library name zlibstatic.lib. +- http: fix sending of large requests - zlib's static library has a different name depending on how it was - built. zlibstatic.lib is output by cmake. zlibstat.lib is output by - their pre-generated Visual Studio project files (in the contrib - directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall - instead of cdecl if you end up exporting the zlib functions). + - refs #11342 where errors with git https interactions + were observed + - problem was caused by 1st sends of size larger than 64KB + which resulted in later retries of 64KB only + - limit sending of 1st block to 64KB + - adjust h2/h3 filters to cope with parsing the HTTP/1.1 + formatted request in chunks - Prior to this change the makefile only checked for the latter. + - introducing Curl_nwrite() as companion to Curl_write() + for the many cases where the sockindex is already known - Closes https://github.com/curl/curl/pull/11521 + Fixes #11342 (again) + Closes #11803 -Daniel Stenberg (29 Jul 2023) +- pytest: fix check for slow_network skips to only apply when intended -- configure: use the pkg-config --libs-only-l flag for libssh2 + Closes #11801 - ... instead of --libs, as that one also returns -L flags. +Daniel Stenberg (5 Sep 2023) - Reported-by: Wilhelm von Thiele - Fixes #11538 - Closes #11539 +- curl_url_get/set.3: add missing semicolon in SYNOPSIS -Viktor Szakats (29 Jul 2023) +- CURLOPT_URL.3: explain curl_url_set() uses the same parser -- cmake: support building static and shared libcurl in one go +- CURLOPT_URL.3: add two URL API calls in the see-also section - This patch adds the ability to build a static and shared libcurl library - in a single build session. It also adds an option to select which one to - use when building the curl executable. +Dan Fandrich (4 Sep 2023) - New build options: - - `BUILD_STATIC_LIBS`. Default: `OFF`. - Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`. - - `BUILD_STATIC_CURL`. Default: `OFF`. - Requires `BUILD_STATIC_LIBS` enabled. - Enabled automatically if building static libcurl only. - - `STATIC_LIB_SUFFIX`. Default: empty. - - `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide - with static lib name (typically with MSVC) in Windows builds. - Otherwise empty. +- CI: add a 32-bit i686 Linux build - Also: + This is done by cross-compiling under regular x86_64 Linux. Since the + kernel offers backwards compatibility, the binaries can be tested as + normal. - - Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass - it directly to the compiler. This also allows to delete a condition - from `tests/server/CMakeLists.txt`. + Closes #11799 - - Complete a TODO by following the logic used in autotools (also for - `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of - `libcurl.pc` for _static-only_ curl builds. +- tests: fix a type warning on 32-bit x86 - - Convert an existing CI test to build both shared and static libcurl. +Viktor Szakats (4 Sep 2023) - Closes #11505 +- tests: delete stray `.orig` file -Stefan Eissing (28 Jul 2023) + Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f + Closes #11797 -- CI/awslc: add cache for build awslc library +Daniel Stenberg (4 Sep 2023) - Closes #11535 +- RELEASE-NOTES: synced -- GHA/linux.yml: add caching +Viktor Szakats (4 Sep 2023) - Closes #11532 +- lib: silence compiler warning in inet_ntop6 -Daniel Stenberg (27 Jul 2023) + ``` + ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her + e [-Wcomma] + cur.base = i, cur.len = 1; + ^ + ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin + g + cur.base = i, cur.len = 1; + ^~~~~~~~~~~~ + (void)( ) + ``` -- RELEASE-NOTES: synced + Closes #11790 - Bump working version to 8.3.0 +Daniel Stenberg (4 Sep 2023) -- url: remove infof() output for "still name resolving" +- transfer: also stop the sending on closed connection - The message does not help and might get spewed a lot during times. + Previously this cleared the receiving bit only but in some cases it is + also still sending (like a request-body) when disconnected and neither + direction can continue then. - Reported-by: yushicheng7788 on github - Fixes #11394 - Closes #11529 + Fixes #11769 + Reported-by: Oleg Jukovec + Closes #11795 -- KNOWN_BUGS: cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!" +John Bampton (4 Sep 2023) - Closes #11244 +- docs: change `sub-domain` to `subdomain` -Stefan Eissing (27 Jul 2023) + https://en.wikipedia.org/wiki/Subdomain -- CI: quiche updates + Closes #11793 - - remove quiche from standard `linux` workflow - - add mod_h2 caching to quiche workflow - - rename quiche to quiche-linux - - move version definitions into env section +Stefan Eissing (4 Sep 2023) - Closes #11528 +- multi: more efficient pollfd count for poll -- http2: disable asssertion blocking OSSFuzz testing + - do not use separate pollfds for sockets that have POLLIN+POLLOUT - - not clear how this triggers and it blocks OSSFuzz testing other - things. Since we handle the case with an error return, disabling the - assertion for now seems the best way forward. + Closes #11792 - Fixes #11500 - Closes #11519 +- http2: polish things around POST -- http2: fix in h2 proxy tunnel: progress in ingress on sending + - added test cases for various code paths + - fixed handling of blocked write when stream had + been closed inbetween attempts + - re-enabled DEBUGASSERT on send with smaller data size - - depending on what is tunneled, the proxy may never get invoked for - receiving data explicitly. Not progressing ingress may lead to stalls - due to missed WINDOW_UPDATEs. + - in debug builds, environment variables can be set to simulate a slow + network when sending data. cf-socket.c and vquic.c support + * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be + answered with a EAGAIN. TCP/UNIX sockets. + This is chosen randomly. + * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written + to the network. TCP/UNIX sockets. + Example: 80 means a send with 1000 bytes would only send 800 + This is applied to every send. + * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be + answered with EAGAIN. QUIC only. + This is chosen randomly. - CI: - - add a chache for building mod_h2 + Closes #11756 - Closes #11527 +Daniel Stenberg (4 Sep 2023) -- CI ngtcp2+quictls: use nghttpx cache as in quiche build +- docs: add curl_global_trace to some SEE ALSO sections -Jay Satiro (27 Jul 2023) + Closes #11791 -- bearssl: don't load CA certs when peer verification is disabled +- os400: fix checksrc nits - We already do this for other SSL backends. + Closes #11789 - Bug: https://github.com/curl/curl/pull/11457#issuecomment-1644587473 - Reported-by: kyled-dell@users.noreply.github.com +Nicholas Nethercote (3 Sep 2023) - Closes https://github.com/curl/curl/pull/11497 +- hyper: remove `hyptransfer->endtask` -Daniel Stenberg (26 Jul 2023) + `Curl_hyper_stream` needs to distinguish between two kinds of + `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and + (b) background tasks that hyper produces. It does this by recording the + address of any `foreach` task in `hyptransfer->endtask` before pushing + it into the executor, and then comparing that against the address of + tasks later polled out of the executor. -- easy: remove #ifdefs to make code easier on the eye + This works right now, but there is no guarantee from hyper that the + addresses are stable. `hyper_executor_push` says "The executor takes + ownership of the task, which should not be accessed again unless + returned back to the user with `hyper_executor_poll`". That wording is a + bit ambiguous but with my Rust programmer's hat on I read it as meaning + the task returned with `hyper_executor_poll` may be conceptually the + same as a task that was pushed, but that there are no other guarantees + and comparing addresses is a bad idea. - Closes #11525 + This commit instead uses `hyper_task_set_userdata` to mark the `foreach` + task with a `USERDATA_RESP_BODY` value which can then be checked for, + removing the need for `hyptransfer->endtask`. This makes the code look + more like that hyper C API examples, which use userdata for every task + and never look at task addresses. -Stefan Eissing (26 Jul 2023) + Closes #11779 -- GHA: adding quiche workflow +Dave Cottlehuber (3 Sep 2023) - - adding separate quiche workflow to also build nghttpx server for testing +- ws: fix spelling mistakes in examples and tests - Closes #11517 + Closes #11784 -Version 8.2.1 (26 Jul 2023) +Daniel Stenberg (3 Sep 2023) -Daniel Stenberg (26 Jul 2023) +- tool_filetime: make -z work with file dates before 1970 -- RELEASE-NOTES: synced + Fixes #11785 + Reported-by: Harry Sintonen + Closes #11786 - curl 8.2.1 release +Dan Fandrich (1 Sep 2023) -- THANKS: add contributors from 8.2.1 +- build: fix portability of mancheck and checksrc targets -- docs: provide more see also for cipher options + At least FreeBSD preserves cwd across makefile lines, so rules + consisting of more than one "cd X; do_something" must be explicitly run + in a subshell to avoid this. This problem caused the Cirrus FreeBSD + build to fail when parallel make jobs were enabled. - More cross references. Hide nroff errors. +- CI: adjust labeler match patterns for new & obsolete files - Closes #11513 +- configure: trust pkg-config when it's used for zlib -- docs: mark two TLS options for TLS, not SSL + The library flags retrieved from pkg-config were later thrown out and + harded-coded, which negates the whole reason to use pkg-config. + Also, previously, the assumption was made that --libs-only-l and + --libs-only-L are the full decomposition of --libs, which is untrue and + would not allow linking against a static zlib. The new approach is + better in that it uses --libs, although only if --libs-only-l returns + nothing. - Closes #11514 + Bug: https://curl.se/mail/lib-2023-08/0081.html + Reported-by: Randall + Closes #11778 -Brad Harder (25 Jul 2023) +Stefan Eissing (1 Sep 2023) -- curl_multi_wait.3: fix arg quoting to doc macro .BR +- CI/ngtcp2: clear wolfssl for when cache is ignored - Closes #11511 + Closes #11783 -Daniel Stenberg (24 Jul 2023) +Daniel Stenberg (1 Sep 2023) - RELEASE-NOTES: synced -Viktor Szakats (24 Jul 2023) +Nicholas Nethercote (1 Sep 2023) -- cmake: update ngtcp2 detection +- hyper: fix a progress upload counter bug - Replace `OpenSSL` with `quictls` to follow the same change - in the v0.17.0 ngtcp2 release. + `Curl_pgrsSetUploadCounter` should be a passed a total count, not an + increment. - Follow-up to e0093b4b732f6495b0fb1cd6747cbfedcdcf63ed + This changes the failing diff for test 579 with hyper from this: + ``` + Progress callback called with UL 0 out of 0[LF] + -Progress callback called with UL 8 out of 0[LF] + -Progress callback called with UL 16 out of 0[LF] + -Progress callback called with UL 26 out of 0[LF] + -Progress callback called with UL 61 out of 0[LF] + -Progress callback called with UL 66 out of 0[LF] + +Progress callback called with UL 29 out of 0[LF] + ``` + to this: + ``` + Progress callback called with UL 0 out of 0[LF] + -Progress callback called with UL 8 out of 0[LF] + -Progress callback called with UL 16 out of 0[LF] + -Progress callback called with UL 26 out of 0[LF] + -Progress callback called with UL 61 out of 0[LF] + -Progress callback called with UL 66 out of 0[LF] + +Progress callback called with UL 40 out of 0[LF] + ``` + Presumably a step in the right direction. - Closes #11508 + Closes #11780 -Stefan Eissing (24 Jul 2023) +Daniel Stenberg (1 Sep 2023) -- http: VLH, very large header test and fixes +- awssiv4: avoid freeing the date pointer on error - - adding tests using very large passwords in auth - - fixes general http sending to treat h3 like h2, and - not like http1.1 - - eliminate H2_HEADER max definitions and use the commmon - DYN_HTTP_REQUEST everywhere, different limits do not help - - fix http2 handling of requests denied by nghttp2 on send - to immediately report the refused stream + Since it was not allocated, don't free it even if it was wrong syntax - Closes #11509 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 -Andrei Rybak (23 Jul 2023) + Follow-up to b137634ba3adb -- CONTRIBUTE: drop mention of copyright year ranges + Closes #11782 - Year ranges in copyrights were dropped in commits [1] and [2]. - Verification of year ranges in copyrights was dropped from script - 'scripts/copyright.pl' in commit [3]. However, the corresponding - passages in file 'docs/CONTRIBUTE.md' weren't updated. +Stefan Eissing (1 Sep 2023) - Drop mentions of copyright year ranges from 'docs/CONTRIBUTE.md'. +- CI: ngtcp2-linux: use separate caches for tls libraries - [1] 2bc1d775f (copyright: update all copyright lines and remove year - ranges, 2023-01-02) - [2] c46761bd8 (tests/http: remove year ranges from copyrights, - 2023-03-14) - [3] 0e293bacb (copyright.pl: cease doing year verifications, 2023-01-28) + allow ever changing master for wolfssl - Closes #11504 + Closes #11766 -- CONTRIBUTE: fix syntax in commit message description +- replace `master` as wolfssl-version with recent commit - File 'docs/CONTRIBUTE.md' includes a description of how one should write - commit messages in the curl project. Different possible parts of the - message are enclosed in square brackets. One exception is the section - describing how the curl project doesn't use "Signed-off-by" commit - trailers [1], which is enclosed in an opening curly brace paired with a - closing square bracket. +- wolfssl, use master again in CI - Fix the enclosing square brackets in description of "Signed-off-by" - trailers in commit messages in file 'docs/CONTRIBUTE.md'. + - with the shared session update fix landed in master, it + is time to use that in our CI again - [1] See description of option '--signoff' in Git documentation: - https://git-scm.com/docs/git-commit +Nicholas Nethercote (31 Aug 2023) - Closes #11504 +- tests: fix formatting errors in `FILEFORMAT.md`. -Daniel Stenberg (23 Jul 2023) + Without the surrounding backticks, these tags get swallowed when the + markdown is rendered. -- src/mkhelp: strip off escape sequences + Closes #11777 - At some point the nroff command stopped stripping off escape sequences, - so then this script needs to do the job instead. +Viktor Szakats (31 Aug 2023) - Reported-by: VictorVG on github - Fixes #11501 - Closes #11503 +- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` -- KNOWN_BUGS: building for old macOS fails with gcc + Allow overriding the default TLS backend via a CMake setting. - Closes #11441 + E.g.: + `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` -Jacob Hoffman-Andrews (22 Jul 2023) + Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, + schannel, secure-transport, wolfssl -- rustls: update rustls-ffi 0.10.0 + The passed string is baked into the curl/libcurl binaries. + The value is case-insensitive. - This brings in version 0.21.0 of the upstream rustls implementation, - which notable includes support for IP address certificates. + We added a similar option to autotools in 2017 via + c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. - Closes #10865 + TODO: Convert to lowercase to improve reproducibility. -Brad Harder (22 Jul 2023) + Closes #11774 -- websocket: rename arguments/variables to match docs +- sectransp: fix compiler warnings - Pedantry/semantic-alignment between functions, docs, comments with - respect to websocket protocol code; No functional change intended. + https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# + step:3:11046 + ``` + /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 + 4: warning: unused variable 'success' [-Wunused-variable] + OSStatus success; + ^ + /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 + 4: warning: unused parameter 'sha256len' [-Wunused-parameter] + size_t sha256len) + ^ + ``` - * "totalsize", "framesize" becomes "fragsize" (we deal in frame fragments). + Closes #11773 - * "sendflags" becomes "flags" +- tidy-up: mostly whitespace nits - * use canonical CURL *handle + - delete completed TODO from `./CMakeLists.txt`. + - convert a C++ comment to C89 in `./CMake/CurlTests.c`. + - delete duplicate EOLs from EOF. + - add missing EOL at EOF. + - delete whitespace at EOL (except from expected test results). + - convert tabs to spaces. + - convert CRLF EOLs to LF in GHA yaml. + - text casing fixes in `./CMakeLists.txt`. + - fix a codespell typo in `packages/OS400/initscript.sh`. - Closes #11493 + Closes #11772 -Jan Macku (21 Jul 2023) +Dan Fandrich (31 Aug 2023) -- bug_report: use issue forms instead of markdown template +- CI: remove Windows builds from Cirrus, without replacement - Issue forms allow you to define web-like input forms using YAML - syntax. It allows you to guide the reporter to get the required - information. + If we don't do this, all coverage on Cirrus will cease in a few days. By + removing the Windows builds, the FreeBSD one should still continue + as before. The Windows builds will need be moved to another service to + maintain test coverage. - Signed-off-by: Jan Macku - Closes #11474 + Closes #11771 -Daniel Stenberg (21 Jul 2023) +- CI: switch macOS ARM build from Cirrus to Circle CI -- TODO: Obey Retry-After in redirects + Cirrus is drastically reducing their free tier on Sept. 1, so they will + no longer perform all these builds for us. All but one build has been + moved, with the LibreSSL one being dropped because of linking problems + on Circle. - (remove "Set custom client ip when using haproxy protocol" which was - shipped in 8.2.0) + One important note about this change is that Circle CI is currently + directing all these builds to x86_64 hardware, despite them requesting + ARM. This is because ARM nodes are scheduled to be available on the + free tier only in December. This reduces our architectural diversity + until then but it should automatically come back once those machines are + enabled. - Mentioned-by: Yair Lenga - Closes #11447 +- CI: use the right variable for BSD make -- RELEASE-NOTES: synced + BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel + builds before. -Oliver Roberts (21 Jul 2023) +- CI: drop the FreeBSD 12.X build -- amissl: fix AmiSSL v5 detection + Cirrus' new free tier won't let us have many builds, so drop the + nonessential ones. The FreeBSD 13.X build will still give us the most + relevant FreeBSD coverage. - Due to changes in the AmiSSL SDK, the detection needed adjusting. +- CI: move the Alpine build from Cirrus to GHA - Closes #11477 + Cirrus is reducing their free tier to next to nothing, so we must move + builds elsewhere. -Alois Klink (21 Jul 2023) +Stefan Eissing (30 Aug 2023) -- unittest/makefile: remove unneeded unit1621_LDADD +- test_07_upload.py: fix test_07_34 curl args - The `unit1621_LDADD` variable has the exact same value as the `LDADD` - flag in `Makefile.am`, except without `@LDFLAGS@ @LIBCURL_LIBS@`. + - Pass correct filename to --data-binary. - This was originally added by [98e6629][], but I can't see any reason - why it exists, so we should remove it to clean things up. + Prior to this change --data-binary was passed an incorrect filename due + to a missing separator in the arguments list. Since aacbeae7 curl will + error on incorrect filenames for POST. - [98e6629]: https://github.com/curl/curl/commit/98e6629154044e4ab1ee7cff8351c7 - ebcb131e88 + Fixes https://github.com/curl/curl/issues/11761 + Closes https://github.com/curl/curl/pull/11763 - Closes #11494 +Nicholas Nethercote (30 Aug 2023) -- unittest/makefile: remove unneeded unit1394_LDADD +- tests: document which tests fail due to hyper's lack of trailer support. - These custom `unit1394_LDADD` and similar automake overrides are no - longer neded. They were originally added by added by [8dac7be][] for - metalink support, but are no longer after [265b14d][] removed metalink. + Closes #11762 - [8dac7be]: https://github.com/curl/curl/commit/8dac7be438512a8725d3c71e9139bd - fdcac1ed8c - [265b14d]: https://github.com/curl/curl/commit/265b14d6b37c4298bd5556fabcbc37 - d36f911693 +- docs: removing "pausing transfers" from HYPER.md. - Closes #11494 + It's a reference to #8600, which was fixed by #9070. -- cmake: add `libcurlu`/`libcurltool` for unit tests + Closes #11764 - Add a `libcurlu`/`libcurltool` static library that is compiled only for - unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not - built by default, they're only built if unit tests are built. +Patrick Monnerat (30 Aug 2023) - These libraries allow us to compile every unit test with CMake. +- os400: handle CURL_TEMP_PRINTF() while building bind source - Closes #11446 + Closes #11547 -Daniel Stenberg (21 Jul 2023) +- os400: build test servers -- test979: test -u with redirect to (the same) absolute host + Also fix a non-compliant main prototype in disabled.c. - Verifies #11492 + Closes #11547 -- transfer: do not clear the credentials on redirect to absolute URL +- tests: fix compilation error for os400 - Makes test 979 work. Regression shipped in 8.2.0 from commit - dd4d1a26959f63a2c + OS400 uses BSD 4.3 setsockopt() prototype by default: this does not + define parameter as const, resulting in an error if actual parameter is + const. Remove the const keyword from the actual parameter cast: this + works in all conditions, even if the formal parameter uses it. - Fixes #11486 - Reported-by: Cloudogu Siebels - Closes #11492 + Closes #11547 -Jon Rumsey (20 Jul 2023) +- os400: make programs and command name configurable -- os400: correct EXPECTED_STRING_LASTZEROTERMINATED + Closes #11547 - Correct EXPECTED_STRING_LASTZEROTERMINATED to account for - CURLOPT_HAPROXY_CLIENT_IP which requires EBCDIC to ASCII conversion when - passed into curl_easy_setopt(). +- os400: move build configuration parameters to a separate script - Closes #11476 + They can then easily be overriden in a script named "config400.override" + that is not part of the distribution. -Oliver Roberts (20 Jul 2023) + Closes #11547 -- amissl: add missing signal.h include +- os400: implement CLI tool - In some environments, signal.h is already included, but not in others - which cause compilation to fail, so explictly include it. + This is provided as a QADRT (ascii) program, a link to it in the IFS and + a minimal CL command. - Closes #11478 + Closes #11547 -- amigaos: fix sys/mbuf.h m_len macro clash +Matthias Gatto (30 Aug 2023) - The updated Curl_http_req_make and Curl_http_req_make2 functions spawned - a parameter called m_len. The AmigaOS networking headers, derived from - NetBSD, contain "#define m_len m_hdr.mh_len" which clashes with - this. Since we do not actually use mbuf, force the include file to be - ignored, removing the clash. +- lib: fix aws-sigv4 having date header twice in some cases - Closes #11479 + When the user was providing the header X-XXX-Date, the header was + re-added during signature computation, and we had it twice in the + request. -Daniel Stenberg (20 Jul 2023) + Reported-by: apparentorder@users.noreply.github.com -- socks: print ipv6 address within brackets + Signed-off-by: Matthias Gatto - Fixes #11483 - Closes #11484 + Fixes: https://github.com/curl/curl/issues/11738 + Closes: https://github.com/curl/curl/pull/11754 -Christian Schmitz (20 Jul 2023) +Jay Satiro (30 Aug 2023) -- libcurl-errors.3: add CURLUE_OK +- multi: remove 'processing: ' debug message - Closes #11488 + - Remove debug message added by e024d566. -Oliver Roberts (20 Jul 2023) + Closes https://github.com/curl/curl/pull/11759 -- cfilters: rename close/connect functions to avoid clashes +- ftp: fix temp write of ipv6 address - Rename `close` and `connect` in `struct Curl_cftype` for - consistency and to avoid clashes with macros of the same name - (the standard AmigaOS networking connect() function is implemented - via a macro). + - During the check to differentiate between a port and IPv6 address + without brackets, write the binary IPv6 address to an in6_addr. - Closes #11491 + Prior to this change the binary IPv6 address was erroneously written to + a sockaddr_in6 'sa6' when it should have been written to its in6_addr + member 'sin6_addr'. There's no fallout because no members of 'sa6' are + accessed before it is later overwritten. -Stefan Eissing (20 Jul 2023) + Closes https://github.com/curl/curl/pull/11747 -- http2: fix regression on upload EOF handling +- tool: change some fopen failures from warnings to errors - - a regression introduced by c9ec85121110d7cbbbed2990024222c8f5b8afe5 - where optimization of small POST bodies leads to a new code path - for such uploads that did not trigger the "done sending" event - - add triggering this event for early "upload_done" situations + - Error on missing input file for --data, --data-binary, + --data-urlencode, --header, --variable, --write-out. - Fixes #11485 - Closes #11487 - Reported-by: Aleksander Mazur + Prior to this change if a user of the curl tool specified an input file + for one of the above options and that file could not be opened then it + would be treated as zero length data instead of an error. For example, a + POST using `--data @filenametypo` would cause a zero length POST which + is probably not what the user intended. -Daniel Stenberg (19 Jul 2023) + Closes https://github.com/curl/curl/pull/11677 -- configure: check for nghttp2_session_get_stream_local_window_size +- hostip: fix typo - The http2 code uses it now. Introduced in nghttp2 1.15.0 (Sep 2016) +Davide Masserut (29 Aug 2023) - Fixes #11470 - Reported-by: Paul Howarth - Closes #11473 +- tool: avoid including leading spaces in the Location hyperlink -Stefan Eissing (19 Jul 2023) + Co-authored-by: Dan Fandrich -- quiche: fix segfault and other things + Closes #11735 - - refs #11449 where a segfault is reported when IP Eyeballing did - not immediately connect but made several attempts - - The transfer initiating the eyeballing was initialized too early, - leadding to references to the filter instance that was then - replaced in the subsequent eyeball attempts. That led to a use - after free in the buffer handling for the transfer - - transfers are initiated now more lazy (like in the ngtcp2 filter), - when the stream is actually opened - - suppress reporting on quiche event errors for "other" transfers - than the current one to not fail a transfer due to faults in - another one. - - revert recent return value handling for quiche_h3_recv_body() - to not indicate an error but an EAGAIN situation. We wish quiche - would document what functions return. +Daniel Stenberg (29 Aug 2023) - Fixes #11449 - Closes #11469 - Reported-by: ウさん +- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline -Daniel Stenberg (19 Jul 2023) + Closes #11757 -- hostip: return IPv6 first for localhost resolves +- connect: stop halving the remaining timeout when less than 600 ms left - Fixes #11465 - Reported-by: Chilledheart on github - Closes #11466 + When curl wants to connect to a host, it always has a TIMEOUT. The + maximum time it is allowed to spend until a connect is confirmed. -Harry Sintonen (19 Jul 2023) + curl will try to connect to each of the IP adresses returned for the + host. Two loops, one for each IP family. -- tool: fix tool_seek_cb build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T + During the connect loop, while curl has more than one IP address left to + try within a single address family, curl has traditionally allowed (time + left/2) for *this* connect attempt. This, to not get stuck on the + initial addresses in case the timeout but still allow later addresses to + get attempted. - - a variable was renamed, and some use of it wasn't. this fixes the - build. + This has the downside that when users set a very short timeout and the + host has a large number of IP addresses, the effective result might be + that every attempt gets a little too short time. - Closes #11468 + This change stop doing the divided-by-two if the total time left is + below a threshold. This threshold is 600 milliseconds. -Stefan Eissing (19 Jul 2023) + Closes #11693 -- quiche: fix lookup of transfer at multi +- asyn-ares: reduce timeout to 2000ms - - refs #11449 where weirdness in quiche multi connection tranfers was - observed - - fixes lookup of transfer for a quiche event to take the connection - into account - - formerly, a transfer with the same stream_id, but on another connection - could be found + When UDP packets get lost this makes for slightly faster retries. This + lower timeout is used by @c-ares itself by default starting next + release. - Closes #11462 + Closes #11753 -Daniel Stenberg (19 Jul 2023) +John Bampton (29 Aug 2023) + +- misc: remove duplicate words + + Closes #11740 + +Daniel Stenberg (29 Aug 2023) - RELEASE-NOTES: synced - bump to 8.2.1 +- wolfSSL: avoid the OpenSSL compat API when not needed -John Haugabook (19 Jul 2023) + ... and instead call wolfSSL functions directly. -- ciphers.d: put URL in first column + Closes #11752 - This makes the URL turn into a link properly when "webified". +Viktor Szakats (28 Aug 2023) - Fixes https://github.com/curl/curl-www/issues/270 - Closes #11464 +- lib: fix null ptr derefs and uninitialized vars (h2/h3) -Version 8.2.0 (19 Jul 2023) + Fixing compiler warnings with gcc 13.2.0 in unity builds. -Daniel Stenberg (19 Jul 2023) + Assisted-by: Jay Satiro + Assisted-by: Stefan Eissing + Closes #11739 -- RELEASE-NOTES: synced +Jay Satiro (28 Aug 2023) - 8.2.0 release +- secureserver.pl: fix stunnel version parsing -- THANKS-filter: strip out "GitHub" + - Allow the stunnel minor-version version part to be zero. -- THANKS: add contributors from 8.2.0 + Prior to this change with the stunnel version scheme of . + if either part was 0 then version parsing would fail, causing + secureserver.pl to fail with error "No stunnel", causing tests that use + the SSL protocol to be skipped. As a practical matter this bug can only + be caused by a minor-version part of 0, since the major-version part is + always greater than 0. -- RELEASE-PROCEDURE.md: adjust the release dates + Closes https://github.com/curl/curl/pull/11722 -Stefan Eissing (17 Jul 2023) +- secureserver.pl: fix stunnel path quoting -- quiche: fix defects found in latest coverity report + - Store the stunnel path in the private variable $stunnel unquoted and + instead quote it in the command strings. - Closes #11455 + Prior to this change the quoted stunnel path was passed to perl's file + operators which cannot handle quoted paths. For example: -Daniel Stenberg (17 Jul 2023) + $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; + if(-x $stunnel or -x "$stunnel") + # false even if path exists and is executable -- quiche: avoid NULL deref in debug logging + Our other test scripts written in perl, unlike this one, use servers.pm + which has a global $stunnel variable with the path stored unquoted and + therefore those scripts don't have this problem. - Coverity reported "Dereference after null check" + Closes https://github.com/curl/curl/pull/11721 - If stream is NULL and the function exits, the logging must not deref it. +Daniel Stenberg (28 Aug 2023) - Closes #11454 +- altsvc: accept and parse IPv6 addresses in response headers -Stefan Eissing (17 Jul 2023) + Store numerical IPv6 addresses in the alt-svc file with the brackets + present. -- http2: treat initial SETTINGS as a WINDOW_UPDATE + Verify with test 437 and 438 - - refs #11426 where spurious stalls on large POST requests - are reported - - the issue seems to involve the following - * first stream on connection adds up to 64KB of POST - data, which is the max default HTTP/2 stream window size - transfer is set to HOLD - * initial SETTINGS from server arrive, enlarging the stream - window. But no WINDOW_UPDATE is received. - * curl stalls - - the fix un-HOLDs a stream on receiving SETTINGS, not - relying on a WINDOW_UPDATE from lazy servers + Fixes #11737 + Reported-by: oliverpool on github + Closes #11743 - Closes #11450 +- libtest: use curl_free() to free libcurl allocated data -Daniel Stenberg (17 Jul 2023) + In several test programs. These mistakes are not detected or a problem + as long as memdebug.h is included, as that provides the debug wrappers + for all memory functions in the same style libcurl internals do it, + which makes curl_free and free effectively the same call. -- ngtcp2: assigning timeout, but value is overwritten before used + Reported-by: Nicholas Nethercote + Closes #11746 - Reported by Coverity +Jay Satiro (28 Aug 2023) - Closes #11453 +- disable.d: explain --disable not implemented prior to 7.50.0 -- krb5: add typecast to please Coverity + Option -q/--disable was added in 5.0 but only -q was actually + implemented. Later --disable was implemented in e200034 (precedes + 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). -Derzsi Dániel (16 Jul 2023) + Reported-by: pszlazak@users.noreply.github.com -- wolfssl: support setting CA certificates as blob + Fixes https://github.com/curl/curl/issues/11710 + Closes #11712 - Closes #11445 +Nicholas Nethercote (28 Aug 2023) -- wolfssl: detect when TLS 1.2 support is not built into wolfssl +- hyper: fix ownership problems - Closes #11444 + Some of these changes come from comparing `Curl_http` and + `start_CONNECT`, which are similar, and adding things to them that are + present in one and missing in another. -Graham Campbell (15 Jul 2023) + The most important changes: + - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the + happy path. + - In `start_CONNECT`, add a missing `hyper_request_free` on the error + path. + - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. + - In `bodysend`, remove an unnecessary `hyper_body_free` on a different + error path that would cause a double-free. + https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html + says of `hyper_request_set_body`: "This takes ownership of the + hyper_body *, you must not use it or free it after setting it on the + request." This is true even if `hyper_request_set_body` returns an + error; I confirmed this by looking at the hyper source code. -- CI: bump nghttp2 from 1.55.0 to 1.55.1 + Other changes are minor but make things slightly nicer. - Closes #11442 + Closes #11745 -Daniel Stenberg (15 Jul 2023) +Daniel Stenberg (28 Aug 2023) -- curl: return error when asked to use an unsupported HTTP version +- multi.h: the 'revents' field of curl_waitfd is supported - When one of the following options are used but the libcurl in use does - not support it: + Since 6d30f8ebed34e7276 - --http2 - --http2-prior-knowledge - --proxy-http2 + Reported-by: Nicolás Ojeda Bär + Ref: #11748 + Closes #11749 - Closes #11440 +Gerome Fournier (27 Aug 2023) -Chris Paulson-Ellis (14 Jul 2023) +- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() -- cf-socket: don't bypass fclosesocket callback if cancelled before connect + Closes #11742 - After upgrading to 8.1.2 from 7.84.0, I found that sockets were being - closed without calling the fclosesocket callback if a request was - cancelled after the associated socket was created, but before the socket - was connected. This lead to an imbalance of fopensocket & fclosesocket - callbacks, causing problems with a custom event loop integration using - the multi-API. +Daniel Stenberg (27 Aug 2023) - This was caused by cf_socket_close() calling sclose() directly instead - of calling socket_close() if the socket was not active. For regular TCP - client connections, the socket is activated by cf_socket_active(), which - is only called when the socket completes the connect. +- docs: mention critical files in same directories as curl saves - As far as I can tell, this issue has existed since 7.88.0. That is, - since the code in question was introduced by: - commit 71b7e0161032927cdfb4e75ea40f65b8898b3956 - Author: Stefan Eissing - Date: Fri Dec 30 09:14:55 2022 +0100 + ... cannot be fully protected. Don't do it. - lib: connect/h2/h3 refactor + Co-authored-by: Jay Satiro + Reported-by: Harry Sintonen + Fixes #11530 + Closes #11701 - Closes #11439 +John Hawthorn (26 Aug 2023) -Daniel Stenberg (13 Jul 2023) +- OpenSSL: clear error queue after SSL_shutdown -- tool_parsecfg: accept line lengths up to 10M + We've seen errors left in the OpenSSL error queue (specifically, + "shutdown while in init") by adding some logging it revealed that the + source was this file. - Bumped from 100K set in 47dd957daff9 + Since we call SSL_read and SSL_shutdown here, but don't check the return + code for an error, we should clear the OpenSSL error queue in case one + was raised. - Reported-by: Antoine du Hamel - Fixes #11431 - Closes #11435 + This didn't affect curl because we call ERR_clear_error before every + write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when + libcurl is used in a process with other OpenSSL users, they may detect + an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their + own. -Stefan Eissing (13 Jul 2023) + Co-authored-by: Satana de Sant'Ana -- CI: brew fix for openssl in default path + Closes #11736 - If brew install/update links openssl into /usr/local, it will be found - before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of - that by unlinking the keg. +Alexander Kanavin (25 Aug 2023) - Fixes #11413 - Closes #11436 +- tests: update cookie expiry dates to far in the future -Daniel Stenberg (13 Jul 2023) + This allows testing Y2038 with system time set to after that, so that + actual Y2038 issues can be exposed, and not masked by expiry errors. -- RELEASE-NOTES: synced + Fixes #11576 + Closes #11610 -Ondřej Koláček (13 Jul 2023) +John Bampton (25 Aug 2023) -- sectransp: fix EOF handling +- misc: fix spelling - Regression since the large refactor from 2022 + Closes #11733 - Closes #11427 +Daniel Stenberg (25 Aug 2023) -Daniel Stenberg (13 Jul 2023) +- cmdline-opts/page-header: clarify stronger that !opt == URL -- checksrc: quote the file name to work with "funny" letters + Everything provided on the command line that is not an option (or an + argument to an option) is treated as a URL. - Closes #11437 + Closes #11734 -Karthikdasari0423 (13 Jul 2023) +- tests/runner: fix %else handling -- HTTP3.md: ngtcp2 updated to v0.17.0 and nghttp3 to v0.13.0 + Getting the show state proper for %else and %endif did not properly work + in nested cases. - Follow-up to e0093b4b732f6 + Follow-up to 3d089c41ea9 - Closes #11433 + Closes #11731 -Daniel Stenberg (13 Jul 2023) +Nicholas Nethercote (25 Aug 2023) -- CURLOPT_MIMEPOST.3: clarify what setting to NULL means +- docs: Remove mention of #10803 from `KNOWN_BUGS`. - Follow-up to e08382a208d4e480 + Because the leaks have been fixed. - Closes #11430 +- c-hyper: fix another memory leak in `Curl_http`. -Tatsuhiro Tsujikawa (12 Jul 2023) + There is a `hyper_clientconn_free` call on the happy path, but not one + on the error path. This commit adds one. -- ngtcp2: build with 0.17.0 and nghttp3 0.13.0 + Fixes the second memory leak reported by Valgrind in #10803. - - ngtcp2_crypto_openssl was renamed to ngtcp2_crypto_quictls. + Fixes #10803 + Closes #11729 - Closes #11428 +- c-hyper: fix a memory leak in `Curl_http`. -- CI: Bump ngtcp2, nghttp3, and nghttp2 + A request created with `hyper_request_new` must be consumed by either + `hyper_clientconn_send` or `hyper_request_free`. - Closes #11428 + This is not terrifically clear from the hyper docs -- + `hyper_request_free` is documented only with "Free an HTTP request if + not going to send it on a client" -- but a perusal of the hyper code + confirms it. -James Fuller (11 Jul 2023) + This commit adds a `hyper_request_free` to the `error:` path in + `Curl_http` so that the request is consumed when an error occurs after + the request is created but before it is sent. -- example/maxconnects: set maxconnect example + Fixes the first memory leak reported by Valgrind in #10803. - Closes #11343 + Closes #11729 -Pontakorn Prasertsuk (11 Jul 2023) +Daniel Stenberg (25 Aug 2023) -- http2: send HEADER & DATA together if possible +- RELEASE-NOTES: synced - Closes #11420 +John Bampton (25 Aug 2023) -Daniel Stenberg (11 Jul 2023) +- misc: spellfixes -- CI: use wolfSSL 5.6.3 in builds + Closes #11730 - No using master anymore +Daniel Stenberg (25 Aug 2023) - Closes #11424 +- tests: add support for nested %if conditions -SaltyMilk (11 Jul 2023) + Provides more flexiblity to test cases. -- fopen: optimize + Also warn and bail out if there is an '%else' or %endif' without a + preceeding '%if'. - Closes #11419 + Ref: #11610 + Closes #11728 -Daniel Stenberg (11 Jul 2023) +- time-cond.d: mention what happens on a missing file -- cmake: make use of snprintf + Closes #11727 - Follow-up to 935b1bd4544a23a91d68 +Christian Hesse (24 Aug 2023) - Closes #11423 +- docs/cmdline-opts: match the current output -Stefan Eissing (11 Jul 2023) + The release date has been added in output, reflect that in documentation. -- macOS: fix taget detection + Closes #11723 - - TARGET_OS_OSX is not always defined on macOS - - this leads to missing symbol Curl_macos_init() - - TargetConditionals.h seems to define these only when - dynamic targets are enabled (somewhere?) - - this PR fixes that on my macOS 13.4.1 - - I have no clue why CI builds worked without it +Daniel Stenberg (24 Aug 2023) - Follow-up to c7308592fb8ba213fc2c1 - Closes #11417 +- lib: minor comment corrections -Stan Hu (9 Jul 2023) +- docs: rewrite to present tense -- hostip.c: Move macOS-specific calls into global init call + ... instead of using future tense. - https://github.com/curl/curl/pull/7121 introduced a macOS system call - to `SCDynamicStoreCopyProxies`, which is invoked every time an IP - address needs to be resolved. + + numerous cleanups and improvements + + stick to "reuse" not "re-use" + + fewer contractions - However, this system call is not thread-safe, and macOS will kill the - process if the system call is run first in a fork. To make it possible - for the parent process to call this once and prevent the crash, only - invoke this system call in the global initialization routine. + Closes #11713 - In addition, this change is beneficial because it: +- urlapi: setting a blank URL ("") is not an ok URL - 1. Avoids extra macOS system calls for every IP lookup. - 2. Consolidates macOS-specific initialization in a separate file. + Test it in 1560 + Fixes #11714 + Reported-by: ad0p on github + Closes #11715 - Fixes #11252 - Closes #11254 +- spelling: use 'reuse' not 're-use' in code and elsewhere -Daniel Stenberg (9 Jul 2023) + Unify the spelling as both versions were previously used intermittently -- docs: use a space after RFC when spelling out RFC numbers + Closes #11717 - Closes #11382 +Michael Osipov (23 Aug 2023) -Margu (9 Jul 2023) +- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC -- imap-append.c: update to make it more likely to work + HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the + default one. Use "long long" in 32 bit mode and just "long" in 64 bit + mode. - Fixes #10300 - Closes #11397 + Closes #11718 -Emanuele Torre (9 Jul 2023) +Dan Fandrich (22 Aug 2023) -- tool_writeout_json: fix encoding of control characters +- tests: don't call HTTP errors OK in test cases - Control characters without a special escape sequence e.g. %00 or %06 - were being encoded as "u0006" instead of "\u0006". + Some HTTP errors codes were accompanied by the text OK, which causes + some cognitive dissonance when reading them. - Ref: https://github.com/curl/trurl/pull/214#discussion_r1257487858 - Closes #11414 +- http: close the connection after a late 417 is received -Stefan Eissing (9 Jul 2023) + In this situation, only part of the data has been sent before aborting + so the connection is no longer usable. -- http3/ngtcp2: upload EAGAIN handling + Assisted-by: Jay Satiro + Fixes #11678 + Closes #11679 - - refs #11389 where IDLE timeouts on upload are reported - - reword ngtcp2 expiry handling to apply to both send+recv - calls into the filter - - EAGAIN uploads similar to the recent changes in HTTP/2, e.g. - report success only when send data was ACKed. - - HOLD sending of EAGAINed uploads to avoid cpu busy loops - - rename internal function for consistency with HTTP/2 - implementation +- runtests: slightly increase the longest log file displayed - Fixes #11389 - Closes #11390 + The new limit provides enough space for a 64 KiB data block to be logged + in a trace file, plus a few lines at the start and end for context. This + happens to be the amount of data sent at a time in a PUT request. -Brian Nixon (9 Jul 2023) +- tests: add delay command to the HTTP server -- tool_easysrc.h: correct `easysrc_perform` for `CURL_DISABLE_LIBCURL_OPTION` + This adds a delay after client connect. - Closes #11398 +Daniel Stenberg (22 Aug 2023) -Daniel Stenberg (9 Jul 2023) +- cirrus: install everthing with pkg, avoid pip -- RELEASE-NOTES: synced + Assisted-by: Sevan Janiyan -- transfer: clear credentials when redirecting to absolute URL + Closes #11711 - Make sure the user and password for the second request is taken from the - redirected-to URL. +- curl_url*.3: update function descriptions - Add test case 899 to verify. + - expand and clarify several descriptions + - avoid using future tense all over - Reported-by: James Lucas - Fixes #11410 - Closes #11412 + Closes #11708 -Stefan Eissing (8 Jul 2023) +- RELEASE-NOTES: synced -- hyper: fix EOF handling on input +Stefan Eissing (21 Aug 2023) - We ran out of disc space due to an infinite loop with debug logging +- CI/cirrus: disable python install on FreeBSD - Fixes #11377 - Closes #11385 - Reported-by: Dan Fandrich + - python cryptography package does not build build FreeBSD + - install just mentions "error" + - this gets the build and the main test suite going again -- http2: raise header limitations above and beyond + Closes #11705 - - not quite to infinity - - rewrote the implementation of our internal HTTP/1.x request - parsing to work with very large lines using dynbufs. - - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which - is also the limit of curl's general HTTP request processing. +- test2600: fix flakiness on low cpu - Fixes #11405 - Closes #11407 + - refs #11355 where failures to to low cpu resources in CI + are reported + - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations + to test cases + - trigger Curl_expire() in test filter to allow re-checks before + the usual 1second interval -Juan Cruz Viotti (8 Jul 2023) + Closes #11690 -- curl_easy_nextheader.3: add missing open parenthesis examples +Maksim Sciepanienka (20 Aug 2023) - Closes #11409 - Signed-off-by: Juan Cruz Viotti +- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf -Dan Fandrich (7 Jul 2023) + Closes #11698 -- CI: enable verbose test output on pytest +Daniel Stenberg (20 Aug 2023) - This shows individual pass/fail status on tests and makes this output - consistent with other jobs' pytest invocations. +- test687/688: two more basic --xattr tests -Stefan Eissing (28 Jun 2023) + Closes #11697 -- http2: fix crash in handling stream weights +- cmdline-opts/docs: mentioned the negative option part - - Delay the priority handling until the stream has been opened. + ... for --no-alpn and --no-buffer in the same style done for other --no- + options: - - Add test2404 to reproduce and verify. + "Note that this is the negated option name documented." - Weights may change "on the run", which is why there are checks in - general egress handling. These must not trigger when the stream has not - been opened yet. + Closes #11695 - Reported-by: jbgoog@users.noreply.github.com +Emanuele Torre (19 Aug 2023) - Fixes https://github.com/curl/curl/issues/11379 - Closes https://github.com/curl/curl/pull/11384 +- tool/var: also error when expansion result starts with NUL -- tests/http: Add mod_h2 directive `H2ProxyRequests` + Expansions whose output starts with NUL were being expanded to the empty + string, and not being recognised as values that contain a NUL byte, and + should error. - master of mod_h2 now requires H2ProxyRequests directives for forward - proxying with HTTP/2 to work. + Closes #11694 - Ref: https://github.com/icing/mod_h2/commit/3897a7086 +Daniel Stenberg (19 Aug 2023) - Closes https://github.com/curl/curl/pull/11392 +- tests: add 'large-time' as a testable feature -Dan Fandrich (28 Jun 2023) + This allows test cases to require this feature to run and to be used in + %if conditions. -- CI: make Appveyor job names unique + Large here means larger than 32 bits. Ie does not suffer from y2038. - Two otherwise identical mingw-w64 jobs now have their differing compiler - versions mentioned in their names. + Closes #11696 -Sheshadri.V (25 Jun 2023) +- tests/Makefile: add check-translatable-options.pl to tarball -- curl.h: include for vxworks + Used in test 1544 - Closes #11356 + Follow-up to ae806395abc8c -Dan Fandrich (24 Jun 2023) +- gen.pl: fix a long version generation mistake -- CI: enable parallel make in more builds + Too excessive escaping made the parsing not find the correct long names + later and instead add "wrong" links. - Most CI services provide at least two cores, so enable parallel make - jobs to take advantage of that for builds. Some dependencies aren't safe - to build in parallel so leave those as-is. Also, rename a few - workflows to eliminate duplicate names and provide a better idea what - they're about. + Follow-up to 439ff2052e219 -- CI: don't install impacket if tests are not run + Reported-by: Lukas Tribus + Fixes #11688 + Closes #11689 - It just wastes time and bandwidth and isn't even used. +- lib: move mimepost data from ->req.p.http to ->state -divinity76 (24 Jun 2023) + When the legacy CURLOPT_HTTPPOST option is used, it gets converted into + the modem mimpost struct at first use. This data is (now) kept for the + entire transfer and not only per single HTTP request. This re-enables + rewind in the beginning of the second request instead of in end of the + first, as brought by 1b39731. -- configure: the --without forms of the options are also gone + The request struct is per-request data only. - --without-darwin-ssl and --without-metalink + Extend test 650 to verify. - Closes #11378 + Fixes #11680 + Reported-by: yushicheng7788 on github + Closes #11682 -Daniel Stenberg (23 Jun 2023) +Patrick Monnerat (17 Aug 2023) -- configure: add check for ldap_init_fd +- os400: do not check translatable options at build time - ... as otherwise the configure script will say it is OpenLDAP in the - summary, but not set the USE_OPENLDAP define, therefor not using the - intended OpenLDAP code paths. + Now that there is a test for this, the build time check is not needed + anymore. - Regression since 4d7385446 (7.85.0) - Fixes #11372 - Closes #11374 - Reported-by: vlkl-sap on github + Closes #11650 -Michał Petryka (23 Jun 2023) +- test1554: check translatable string options in OS400 wrapper -- cmake: stop CMake from quietly ignoring missing Brotli + This test runs a perl script that checks all string options are properly + translated by the OS400 character code conversion wrapper. It also + verifies these options are listed in alphanumeric order in the wrapper + switch statement. - The CMake project was set to `QUIET` for Brotli instead of - `REQUIRED`. This makes builds unexpectedly ignore missing Brotli even - when `CURL_BROTLI` is enabled. + Closes #11650 - Closes #11376 +Daniel Stenberg (17 Aug 2023) -Emanuele Torre (22 Jun 2023) +- unit3200: skip testing if function is not present -- docs: add more .IP after .RE to fix indentation of generate paragraphs + Fake a successful run since we have no easy mechanism to skip this test + for this advanced condition. - follow-up from 099f41e097c030077b8ec078f2c2d4038d31353b +- unit2600: fix build warning if built without verbose messages - I just thought of checking all the other files with .RE, and I found 6 - other files that were missing .IP at the end. +- test1608: make it build and get skipped without shuffle DNS support - Closes #11375 +- lib: --disable-bindlocal builds curl without local binding support -Stefan Eissing (22 Jun 2023) +- test1304: build and skip without netrc support -- http2: h2 and h2-PROXY connection alive check fixes +- lib: build fixups when built with most things disabled - - fix HTTP/2 check to not declare a connection dead when - the read attempt results in EAGAIN - - add H2-PROXY alive check as for HTTP/2 that was missing - and is needed - - add attach/detach around Curl_conn_is_alive() and remove - these in filter methods - - add checks for number of connections used in some test_10 - proxy tunneling tests + Closes #11687 - Closes #11368 +- workflows/macos.yml: disable zstd and alt-svc in the http-only build -- http2: error stream resets with code CURLE_HTTP2_STREAM + Closes #11683 - - refs #11357, where it was reported that HTTP/1.1 downgrades - no longer works - - fixed with suggested change - - added test_05_03 and a new handler in the curltest module - to reproduce that downgrades work +Stefan Eissing (17 Aug 2023) - Fixes #11357 - Closes #11362 - Reported-by: Jay Satiro +- bearssl: handshake fix, provide proper get_select_socks() implementation -Daniel Stenberg (22 Jun 2023) + - bring bearssl handshake times down from +200ms down to other TLS backends + - vtls: improve generic get_select_socks() implementation + - tests: provide Apache with a suitable ssl session cache -- connect-timeout.d: mention that the DNS lookup is included + Closes #11675 - Closes #11370 +- tests: TLS session sharing test -Emanuele Torre (22 Jun 2023) + - test TLS session sharing with special test client + - expect failure with wolfSSL + - disable flaky wolfSSL test_02_07b -- quote.d: fix indentation of generated paragraphs + Closes #11675 - quote.d was missing a .IP at the end which caused the paragraphs - generated for See-also, Multi, and Example to not be indented correctly. +Daniel Stenberg (17 Aug 2023) - I also remove a redundant "This option can be used multiple times.", and - replaced .IP "item" with .TP .B "item" to make more clear which lines - are part of the list of commands and which aren't. +- CURLOPT_*TIMEOUT*: extend and clarify - Closes #11371 + Closes #11686 -Paul Wise (22 Jun 2023) +- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails -- checksrc: modernise perl file open + And document it. Only return out of memory when it actually is a memory + problem. - Use regular variables and separate file open modes from filenames. + Pointed-out-by: Jacob Mealey + Closes #11674 - Suggested by perlcritic +Mathew Benson (17 Aug 2023) - Copied from https://github.com/curl/trurl/commit/f2784a9240f47ee28a845 +- cmake: add GnuTLS option - Closes #11358 + - Option to use GNUTLS was missing. Hence was not able to use GNUTLS + with ngtcp2 for http3. -Dan Fandrich (21 Jun 2023) + Closes #11685 -- runtests: work around a perl without SIGUSR1 +Daniel Stenberg (16 Aug 2023) - At least msys2 perl v5.32.1 doesn't seem to define this signal. Since - this signal is only used for debugging, just ignore if setting it fails. +- RELEASE-NOTES: synced - Reported-by: Marcel Raad - Fixes #11350 - Closes #11366 +- http: remove the p_pragma struct field -- runtests: include missing valgrind package + unused since 40e8b4e52 (2008) - use valgrind was missing which caused torture tests with valgrind - enabled to fail. + Closes #11681 - Reported-by: Daniel Stenberg - Fixes #11364 - Closes #11365 +Jay Satiro (16 Aug 2023) -- runtests: use more consistent failure lines +- CURLINFO_CERTINFO.3: better explain curl_certinfo struct - After a test failure log a consistent log message to make it easier to - parse the log file. Also, log a consistent message with "ignored" for - failures that cause the test to be not considered at all. These should - perhaps be counted in the skipped category, but this commit does not - change that behaviour. + Closes https://github.com/curl/curl/pull/11666 -- runtests: consistently write the test check summary block +- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation - The memory check character was erroneously omitted if the memory - checking file was not available for some reason, making the block of - characters an inconsistent length. + - Remove the out-of-date SSL backend list supported by + CURLOPT_SSL_CTX_FUNCTION. -- test2600: fix the description + It makes more sense to just refer to that document instead of having + a separate list that has to be kept in sync. - It looks like it was cut-and-pasted. + Closes https://github.com/curl/curl/pull/11665 - Closes #11354 +- write-out.d: clarify %{time_starttransfer} -Daniel Stenberg (21 Jun 2023) + sync it up with CURLINFO_STARTTRANSFER_TIME_T -- TODO: "Support HTTP/2 for HTTP(S) proxies" *done* +Daniel Stenberg (15 Aug 2023) -humbleacolyte (21 Jun 2023) +- transfer: don't set TIMER_STARTTRANSFER on first send -- cf-socket: move ctx declaration under HAVE_GETPEERNAME + The time stamp is for measuring the first *received* byte - Closes #11352 + Fixes #11669 + Reported-by: JazJas on github + Closes #11670 -Daniel Stenberg (20 Jun 2023) +trrui-huawei (15 Aug 2023) -- RELEASE-NOTES: synced +- quiche: enable quiche to handle timeout events -- example/connect-to: show CURLOPT_CONNECT_TO + In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` + interface for the application to invoke upon timer + expiration. Therefore, invoking the `on_timeout` function of the + Connection is crucial to ensure seamless functionality of quiche with + timeout events. - Closes #11340 + Closes #11654 -Stefan Eissing (20 Jun 2023) +- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s -- hyper: unslow + Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. - - refs #11203 where hyper was reported as being slow - - fixes hyper_executor_poll to loop until it is out of - tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issue - s/3237 - - added a fix in hyper io handling for detecting EAGAIN - - added some debug logs to see IO results - - pytest http/1.1 test cases pass - - pytest h2 test cases fail on connection reuse. HTTP/2 - connection reuse does not seem to work. Hyper submits - a request on a reused connection, curl's IO works and - thereafter hyper declares `Hyper: [1] operation was canceled: connection cl - osed` - on stderr without any error being logged before. +Daniel Stenberg (15 Aug 2023) - Fixes #11203 - Reported-by: Gisle Vanem - Advised-by: Sean McArthur - Closes #11344 +- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang -- HTTP/2: upload handling fixes + Closes #9580 - - fixes #11242 where 100% CPU on uploads was reported - - fixes possible stalls on last part of a request body when - that information could not be fully send on the connection - due to an EAGAIN - - applies the same EGAIN handling to HTTP/2 proxying +- imap: add a check for failing strdup() - Reported-by: Sergey Alirzaev - Fixed #11242 - Closes #11342 +- imap: remove the only sscanf() call in the IMAP code -Daniel Stenberg (20 Jun 2023) + Avoids the use of a stack buffer. -- example/opensslthreadlock: remove + Closes #11673 - This shows how to setup OpenSSL mutex callbacks, but this is not - necessary since OpenSSL 1.1.0 - meaning that no currently supported - OpenSSL version requires this anymore +- imap: use a dynbuf in imap_atom - Closes #11341 + Avoid a calculation + malloc. Build the output in a dynbuf. -Dan Fandrich (19 Jun 2023) + Closes #11672 -- libtest: display the times after a test timeout error +Marin Hannache (14 Aug 2023) - This is to help with test failure debugging. +- http: do not require a user name when using CURLAUTH_NEGOTIATE - Ref: #11328 - Closes #11329 + In order to get Negotiate (SPNEGO) authentication to work in HTTP you + used to be required to provide a (fake) user name (this concerned both + curl and the lib) because the code wrongly only considered + authentication if there was a user name provided, as in: -- test2600: bump a test timeout + curl -u : --negotiate https://example.com/ - Case 1 failed at least once on GHA by going 30 msec too long. + This commit leverages the `struct auth` want member to figure out if the + user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of + setting a user name both in curl and the lib. - Ref: #11328 + Signed-off-by: Marin Hannache + Reported-by: Enrico Scholz + Fixes https://sourceforge.net/p/curl/bugs/440/ + Fixes #1161 + Closes #9047 -- runtests: better detect and handle pipe errors in the controller +Viktor Szakats (13 Aug 2023) - Errors reading and writing to the pipes are now better detected and - propagated up to the main test loop so it can be cleanly shut down. Such - errors are usually due to a runner dying so it doesn't make much sense - to try to continue the test run. +- build: streamline non-UWP wincrypt detections -- runtests: cleanly abort the runner if the controller dies + - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build + and disable our non-UWP-compatible use the Windows crypto API. This + allows to drop two dynamic feature checks. - If the controller dies unexpectedly, have the runner stop its servers - and exit cleanly. Otherwise, the orphaned servers will stay running in - the background. + `WINDOWS_STORE` is true when invoking CMake with + `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. -- runtests: improve error logging + Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html - Give more information about test harness error conditions to help figure - out what might be wrong. Print some internal test state when SIGUSR1 is - sent to runtests.pl. + - with autotools, drop the separate feature check for `wincrypt.h`. On + one hand this header has been present for long (even Borland C 5.5 had + it from year 2000), on the other we used the check result solely to + enable another check for certain crypto functions. This fails anyway + with the header not present. We save one dynamic feature check at the + configure stage. - Ref: #11328 + Reviewed-by: Marcel Raad + Closes #11657 -- runtests: better handle ^C during slow tests +Nicholas Nethercote (13 Aug 2023) - Since the SIGINT handler now just sets a flag that must be checked in the - main controller loop, make sure that runs periodically. Rather than - blocking on a response from a test runner near the end of the test run, - add a short timeout to allow it. +- docs/HYPER.md: update hyper build instructions -- runtests: rename server command file + Nightly Rust and `-Z unstable-options` are not needed. - The name ftpserver.cmd was historical and has been used for more than - ftp for many years now. Rename it to plain server.cmd to reduce - confusion. + The instructions here now match the hyper docs exactly: + https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 + 6f88 -- tests: improve reliability of TFTP tests + Closes #11662 - Stop checking the timeout used by the client under test (for most - tests). The timeout will change if the TFTP test server is slow (such as - happens on an overprovisioned CI server) because the client will retry - and reduce its timeout, and the actual value is not important for most - tests. +Daniel Stenberg (13 Aug 2023) - test285 is changed a different way, by increasing the connect timeout. - This improves test coverage by allowing the changed timeout value to be - checked, but improves reliability with a carefully-chosen timeout that - not only allows twice the time to respond as before, but also allows - several retries before the client will change its timeout value. +- RELEASE-NOTES: synced - Ref: #11328 +- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name -Daniel Stenberg (19 Jun 2023) + Asssisted-by: Jay Satiro + Closes #11655 -- cf-socket: skip getpeername()/getsockname for TFTP +- spellcheck: adapt to backslashed minuses - Since the socket is not connected then the call fails. When the call - fails, failf() is called to write an error message that is then - surviving and is returned when the *real* error occurs later. The - earlier, incorrect, error therefore hides the actual error message. + As the curl.1 has more backslashed minus, the cleanup sed lines xneed to + adapt. - This could be seen in stderr for test 1007 + Adjusted some docs slighly. - Test 1007 has now been extended to verify the stderr message. + Follow-up to 439ff2052e - Closes #11332 + Closes #11663 -- example/crawler: make it use a few more options +- gen: escape more minus - For show, but reasonable + Detected since it was still hard to search for option names using dashes + in the middle in the man page. -- libcurl-ws.3: mention raw mode + Closes #11660 - Closes #11339 +- cookie-jar.d: enphasize that this option is ONLY writing cookies -- example/default-scheme: set the default scheme for schemeless URLs + Reported-by: Dan Jacobson + Tweaked-by: Jay Satiro + Ref: #11642 + Closes #11661 - Closes #11338 +Nicholas Nethercote (11 Aug 2023) -- example/hsts-preload: show one way to HSTS preload +- docs/HYPER.md: document a workaround for a link error - Closes #11337 + Closes #11653 -- examples/http-options: show how to send "OPTIONS *" +Jay Satiro (11 Aug 2023) - With CURLOPT_REQUEST_TARGET. +- schannel: verify hostname independent of verify cert - Also add use of CURLOPT_QUICK_EXIT to show. + Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off + and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the + hostname in schannel code. - Closes #11333 + This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and + verify hostname". We discussed a fix several years ago in #3285 but it + went stale. -- examples: make use of CURLOPT_(REDIR_|)PROTOCOLS_STR + Assisted-by: Daniel Stenberg - To show how to use them + Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html + Reported-by: Martin Galvan - Closes #11334 + Ref: https://github.com/curl/curl/pull/3285 -- examples/smtp-mime: use CURLOPT_MAIL_RCPT_ALLOWFAILS + Fixes https://github.com/curl/curl/issues/3284 + Closes https://github.com/curl/curl/pull/10056 - For show +Daniel Stenberg (11 Aug 2023) - Closes #11335 +- curl_quiche: remove superfluous NULL check -- http: rectify the outgoing Cookie: header field size check + 'stream' is always non-NULL at this point - Previously it would count the size of the entire outgoing request and - not just the size of only the Cookie: header field - which was the - intention. + Pointed out by Coverity - This could make the check be off by several hundred bytes in some cases. + Closes #11656 - Closes #11331 +- curl/urlapi.h: tiny typo -Jay Satiro (17 Jun 2023) +- github/labeler: make HYPER.md set Hyper and not TLS -- lib: fix some format specifiers +- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 - - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some - curl_off_t variables. + 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need + to specify version changes for earlier releases in the generated output. - - Use %zu where %zd was erroneously used for some size_t variables. + This ups the limit from the previous 7.30.0 (Apr 12 2013) - Prior to this change some of the Windows CI tests were failing because - in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t. - When %zd was used for some curl_off_t variables then only the lower - 32-bits was read and the upper 32-bits would be read for part or all of - the next specifier. + This hides roughly 35 "added in" mentions. - Fixes https://github.com/curl/curl/issues/11327 - Closes https://github.com/curl/curl/pull/11321 + Closes #11651 -Marcel Raad (16 Jun 2023) +Jay Satiro (10 Aug 2023) -- test427: add `cookies` feature and keyword +- bug_report: require reporters to specify curl and os versions - This test doesn't work with `--disable-cookies`. + - Change curl version and os sections from single-line input to + multi-line textarea. - Closes https://github.com/curl/curl/pull/11320 + - Require curl version and os sections to be filled out before report + can be submitted. -Chris Talbot (15 Jun 2023) + Closes https://github.com/curl/curl/pull/11636 -- imap: Provide method to disable SASL if it is advertised +Daniel Stenberg (9 Aug 2023) - - Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext - LOGIN over SASL auth. +- gen.pl: replace all single quotes with aq - Prior to this change there was no method to be able to fall back to - LOGIN if an IMAP server advertises SASL capabilities. However, this may - be desirable for e.g. a misconfigured server. + - this prevents man from using a unicode sequence for them + - which then allows search to work properly - Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2 + Closes #11645 - ";AUTH=" looks to be the correct way to specify what - authenication method to use, regardless of SASL or not. +Viktor Szakats (9 Aug 2023) - Closes https://github.com/curl/curl/pull/10041 +- cmake: fix to use variable for the curl namespace -Daniel Stenberg (15 Jun 2023) + Replace (wrong) literal with a variable to specify the curl + namespace. -- RELEASE-NOTES: synced + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 -- examples/multi-debugcallback.c: avoid the bool typedef + Reported-by: balikalina on Github + Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 + 20d30#r123923098 + Closes #11629 - Apparently this cannot be done in c23 +- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms - Reported-by: Cristian Rodríguez - Fixes #11299 - Closes #11319 + 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing + libcurl objects for shared and static targets. -- docs/libcurl/libcurl.3: cleanups and improvements + The above automatically enabled for Windows builds, with an option to + disable with `SHARE_LIB_OBJECT=OFF`. - Closes #11317 + This patch extend this feature to all platforms as a manual option. + You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects + are built in PIC mode, meaning the static lib will also have PIC code. -- libcurl-ws.3: fix typo + [EXPERIMENTAL] -- curl_ws_*.3: enhance + Closes #11627 - - all: SEE ALSO the libcurl-ws man page - - send: add example and return value information - - meta: mention that the returned data is read-only +- cmake: assume `wldap32` availability on Windows - Closes #11318 + This system library first shipped with Windows ME, available as an extra + install for some older releases (according to [1]). The import library + was present already in old MinGW 3.4.2 (year 2007). -- docs/libcurl/libcurl-ws.3: see also CURLOPT_WS_OPTIONS + Drop the feature check and its associated `HAVE_WLDAP32` variable. -- docs/libcurl/libcurl-ws.3: minor polish + To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` + CMake option, like before. -- libcurl-ws.3. WebSocket API overview + [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html - Closes #11314 + Reviewed-by: Jay Satiro + Closes #11624 -- libcurl-url.3: also mention CURLUPART_ZONEID +Daniel Stenberg (9 Aug 2023) - ... and sort the two part-using lists alphabetically +- page-header: move up a URL paragraph from GLOBBING to URL -Marcel Raad (14 Jun 2023) +- variable.d: output the function names table style -- fopen: fix conversion warning on 32-bit Android + Also correct the url function name in the header - When building for 32-bit ARM or x86 Android, `st_mode` is defined as - `unsigned int` instead of `mode_t`, resulting in a - -Wimplicit-int-conversion clang warning because `mode_t` is - `unsigned short`. Add a cast to silence the warning. + Closes #11641 - Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li - bc/include/sys/stat.h#86 - Closes https://github.com/curl/curl/pull/11313 +- haproxy-clientip.d: remove backticks -- http2: fix variable type + This is not markdown - `max_recv_speed` is `curl_off_t`, so using `size_t` might result in - -Wconversion GCC warnings for 32-bit `size_t`. Visible in the NetBSD - ARM autobuilds. + Follow-up to 0a75964d0d94a4 - Closes https://github.com/curl/curl/pull/11312 + Closes #11639 -Daniel Stenberg (13 Jun 2023) +- RELEASE-NOTES: synced -- vtls: fix potentially uninitialized local variable warnings +- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens - Follow-up from a4a5e438ae533c + Reported-by: FC Stegerman + Fixes #11635 + Closes #11637 - Closes #11310 +- cmdline-opts/page-header: reorder, clean up -- timeval: use CLOCK_MONOTONIC_RAW if available + - removed some unnecessary blurb to focus + - moved up the more important URL details + - put "globbing" into its own subtitle and moved down a little + - mention the online man page in the version section - Reported-by: Harry Sintonen - Ref: #11288 - Closes #11291 + Closes #11638 -Stefan Eissing (12 Jun 2023) +- c-hyper: adjust the hyper to curlcode conversion -- tool: add curl command line option `--trace-ids` + Closes #11621 - - added and documented --trace-ids to prepend (after the timestamp) - the transfer and connection identifiers to each verbose log line - - format is [n-m] with `n` being the transfer id and `m` being the - connection id. In case there is not valid connection id, print 'x'. - - Log calls with a handle that has no transfer id yet, are written - without any ids. +- test2306: make it use a persistent connection - Closes #11185 + + enable verbose already from the start -- lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID + Closes #11621 - - add an `id` long to Curl_easy, -1 on init - - once added to a multi (or its own multi), it gets - a non-negative number assigned by the connection cache - - `id` is unique among all transfers using the same - cache until reaching LONG_MAX where it will wrap - around. So, not unique eternally. - - CURLINFO_CONN_ID returns the connection id attached to - data or, if none present, data->state.lastconnect_id - - variables and type declared in tool for write out +eppesuig (8 Aug 2023) - Closes #11185 +- list-only.d: mention SFTP as supported protocol -Daniel Stenberg (12 Jun 2023) + Closes #11628 -- CURLOPT_INFILESIZE.3: mention -1 triggers chunked +Daniel Stenberg (8 Aug 2023) - Ref: #11300 - Closes #11304 +- request.d: use .TP for protocol "labels" -Philip Heiduck (12 Jun 2023) + To render the section nicer in man page. -- CI: openssl-3.0.9+quic + Closes #11630 - Closes #11296 +- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP -Karthikdasari0423 (12 Jun 2023) + ... as documented. -- HTTP3.md: update openssl version + Update test 3201 and 3202 accordingly. - Closes #11297 + Reported-by: Markus Sommer + Fixes #11619 + Closes #11626 -Daniel Stenberg (12 Jun 2023) +- page-footer: QLOGDIR works with ngtcp2 and quiche -- vtls: avoid memory leak if sha256 call fails + It previously said "both" backends which is confusing as we currently + have three... - ... in the pinned public key handling function. + Closes #11631 - Reported-by: lizhuang0630 on github - Fixes #11306 - Closes #11307 +Stefan Eissing (8 Aug 2023) -- examples/ipv6: disable on win32 +- http3: quiche, handshake optimization, trace cleanup - I can't make if_nametoindex() work there + - load x509 store after clienthello + - cleanup of tracing - Follow-up to c23dc42f3997acf23 + Closes #11618 - Closes #11305 +Daniel Stenberg (8 Aug 2023) -- tool_operate: allow cookie lines up to 8200 bytes +- ngtcp2: remove dead code - Since this option might set multiple cookies in the same line, it does - not make total sense to cap this at 4096 bytes, which is the limit for a - single cookie name or value. + 'result' is always zero (CURLE_OK) at this point - Closes #11303 + Detected by Coverity -- test427: verify sending more cookies than fit in a 8190 bytes line + Closes #11622 - curl will then only populate the header with cookies that fit, dropping - ones that otherwise would have been sent +Viktor Szakats (8 Aug 2023) - Ref: https://curl.se/mail/lib-2023-06/0020.html +- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` - Closes #11303 + OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the + popular forks (yet). Use the macro itself to detect its presence, + replacing the hard-wired fork-specific conditions. -- testutil: allow multiple %-operators on the same line + This way the feature will enable automatically when forks implement it, + while also shorter and possibly requiring less future maintenance. - Closes #11303 + Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 -Oleg Jukovec (12 Jun 2023) + Reviewed-by: Jay Satiro + Closes #11617 -- docs: update CURLOPT_UPLOAD.3 +- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 - The behavior of CURLOPT_UPLOAD differs from what is described in the - documentation. The option automatically adds the 'Transfer-Encoding: - chunked' header if the upload size is unknown. + LibreSSL 3.4.1 (2021-10-14) added support for + `SSL_CTX_set_ciphersuites`. - Closes #11300 + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt -Daniel Stenberg (12 Jun 2023) + Reviewed-by: Jay Satiro + Closes #11616 -- RELEASE-NOTES: synced +- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 -- CURLOPT_AWS_SIGV4.3: remove unused variable from example + LibreSSL 3.5.0 (2022-02-24) added support for + `SSL_CTX_set_keylog_callback`. - Closes #11302 + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt -- examples/https.c: use CURLOPT_CA_CACHE_TIMEOUT + Reviewed-by: Jay Satiro + Closes #11615 - for demonstration purposes +- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks - Closes #11290 + - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is + also not used by curl, but it is by its optional dependency `librtmp`. + Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This + library has been available since the early days of Windows. -- example/ipv6: feature CURLOPT_ADDRESS_SCOPE in use + - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present + since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and + other existing logic already assumed this lib being present, so delete + the check and replace the detection variable with `WIN32` and always + add `ws2_32` on Windows. - Closes #11282 + Closes #11612 -Karthikdasari0423 (10 Jun 2023) +Daniel Gustafsson (8 Aug 2023) -- docs: Update HTTP3.md for newer ngtcp2 and nghttp3 +- crypto: ensure crypto initialization works - Follow-up to fb9b9b58 + Make sure that context initialization during hash setup works to avoid + going forward with the risk of a null pointer dereference. - Ref: #11184 - Closes #11295 + Reported-by: Philippe Antoine on HackerOne + Assisted-by: Jay Satiro + Assisted-by: Daniel Stenberg -Dan Fandrich (10 Jun 2023) + Closes #11614 -- docs: update the supported ngtcp2 and nghttp3 versions +Viktor Szakats (7 Aug 2023) - Follow-up to cae9d10b +- openssl: switch to modern init for LibreSSL 2.7.0+ - Ref: #11184 - Closes #11294 + LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, + `OPENSSL_init_ssl()` function and deprecated the old, manual init + method, as seen in OpenSSL 1.1.0. Switch to the modern method when + available. -- tests: fix error messages & handling around sockets + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt - The wrong error code was checked on Windows on UNIX socket failures, - which could have caused all UNIX sockets to be reported as having - errored and the tests therefore skipped. Also, a useless error message - was displayed on socket errors in many test servers on Windows because - strerror() doesn't work on WinSock error codes; perror() is overridden - there to work on all errors and is used instead. + Reviewed-by: Daniel Stenberg + Closes #11611 - Ref #11258 - Closes #11265 +Daniel Stenberg (7 Aug 2023) -Daniel Stenberg (9 Jun 2023) +- gskit: remove -- CURLOPT_SSH_PRIVATE_KEYFILE.3: expand on the file search + We remove support for building curl with gskit. - Reported-by: atjg on github - Ref: #11287 - Closes #11289 + - This is a niche TLS library, only running on some IBM systems + - no regular curl contributors use this backend + - no CI builds use or verify this backend + - gskit, or the curl adaption for it, lacks many modern TLS features + making it an inferior solution + - build breakages in this code take weeks or more to get detected + - fixing gskit code is mostly done "flying blind" -Stefan Eissing (9 Jun 2023) + This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has + been mentioned on the curl-library mailing list. -- ngtcp2: use ever increasing timestamp in io + It could be brought back, this is not a ban. Given proper effort and + will, gskit support is welcome back into the curl TLS backend family. - - ngtcp2 v0.16.0 asserts that timestamps passed to its function - will only ever increase. - - Use a context shared between ingress/egress operations that - uses a shared timestamp, regularly updated during calls. + Closes #11460 - Closes #11288 +- RELEASE-NOTES: synced -Daniel Stenberg (9 Jun 2023) +Dan Fandrich (7 Aug 2023) -- GHA: use nghttp2 1.54.0 for the ngtcp2 jobs +- THANKS-filter: add a name typo -Philip Heiduck (9 Jun 2023) +Stefan Eissing (7 Aug 2023) -- GHA: ngtcp2: use 0.16.0 and nghttp3 0.12.0 +- http3/ngtcp2: shorten handshake, trace cleanup -Daniel Stenberg (9 Jun 2023) + - shorten handshake timing by delayed x509 store load (OpenSSL) + as we do for HTTP/2 + - cleanup of trace output, align with HTTP/2 output -- ngtcp2: build with 0.16.0 and nghttp3 0.12.0 + Closes #11609 - - moved to qlog_write - - crypto => encryption - - CRYPTO => ENCRYPTION - - removed "_is_" - - ngtcp2_conn_shutdown_stream_read and - ngtcp2_conn_shutdown_stream_write got flag arguments - - the nghttp3_callbacks struct got a recv_settings callback +Daniel Stenberg (7 Aug 2023) - Closes #11184 +- headers: accept leading whitespaces on first response header -- example/http2-download: set CURLOPT_BUFFERSIZE + This is a bad header fold but since the popular browsers accept this + violation, so does curl now. Unless built with hyper. - Primarily because no other example sets it, and remove the disabling of - the certificate check because we should not recommend that. + Add test 1473 to verify and adjust test 2306. - Closes #11284 + Reported-by: junsik on github + Fixes #11605 + Closes #11607 -- example/crawler: also set CURLOPT_AUTOREFERER +- include/curl/mprintf.h: add __attribute__ for the prototypes - Could make sense, and it was not used in any example before. + - if gcc or clang is used + - if __STDC_VERSION__ >= 199901L, which means greater than C90 + - if not using mingw + - if CURL_NO_FMT_CHECKS is not defined - Closes #11283 + Closes #11589 -Wyatt OʼDay (9 Jun 2023) +- tests: fix bad printf format flags in test code -- tls13-ciphers.d: include Schannel +- tests: fix header scan tools for attribute edits in mprintf.h - Closes #11271 +- cf-socket: log successful interface bind -Daniel Stenberg (9 Jun 2023) + When the setsockopt SO_BINDTODEVICE operation succeeds, output that in + the verbose output. -- curl_pushheader_byname/bynum.3: document in their own man pages + Ref: #11599 + Closes #11608 - These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was - introduced but always lived a life in the shadows, embedded in the - CURLMOPT_PUSHFUNCTION man page. Until now. +- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled - It makes better sense and gives more visibility to document them in - their own stand-alone man pages. + Ref: #11457 + Closes #11606 - Closes #11286 +- CURLOPT_SSL_VERIFYPEER.3: add two more see also options -- curl_mprintf.3: minor fix of the example + CURLINFO_CAINFO and CURLINFO_CAPATH -- curl_url_set: enforce the max string length check for all parts + Closes #11603 - Update the docs and test 1559 accordingly +- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice - Closes #11273 + Closes #11007 -- examples/ftpuploadresume.c: add use of CURLOPT_ACCEPTTIMEOUT_MS +Graham Campbell (6 Aug 2023) - For show +- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 - Closes #11277 + Closes #11585 -- examples/unixsocket.c: example using CURLOPT_UNIX_SOCKET_PATH +Daniel Stenberg (6 Aug 2023) - and alternatively CURLOPT_ABSTRACT_UNIX_SOCKET +- TODO: add *5* entries for aws-sigv4 - Closes #11276 + Closes #7559 + Closes #8107 + Closes #8810 + Closes #9717 + Closes #10129 -Anssi Kolehmainen (8 Jun 2023) +- TODO: LDAP Certificate-Based Authentication -- docs: fix missing parameter names in examples + Closes #9641 - Closes #11278 +Stefan Eissing (6 Aug 2023) -Daniel Stenberg (8 Jun 2023) +- http2: cleanup trace messages -- urlapi: have *set(PATH) prepend a slash if one is missing + - more compact format with bracketed stream id + - all frames traced in and out - Previously the code would just do that for the path when extracting the - full URL, which made a subsequent curl_url_get() of the path to - (unexpectedly) still return it without the leading path. + Closes #11592 - Amend lib1560 to verify this. Clarify the curl_url_set() docs about it. +Daniel Stenberg (6 Aug 2023) - Bug: https://curl.se/mail/lib-2023-06/0015.html - Closes #11272 - Reported-by: Pedro Henrique +- tests/tftpd+mqttd: make variables static to silence picky warnings -Dan Fandrich (7 Jun 2023) + Closes #11594 -- runtests; give each server a unique log lock file +- docs/cmdline: remove repeated working for negotiate + ntlm - Logs are written by several servers and all of them must be finished - writing before the test results can be determined. This means each - server must have its own lock file rather than sharing a single one, - which is how it was done up to now. Previously, the first server to - complete a test would clear the lock before the other server was done, - which caused flaky tests. + The extra wording is added automatically by the gen.pl tool - Lock files are now all found in their own directory, so counting locks - equals counting the files in that directory. The result is that the - proxy logs are now reliably written which actually changes the expected - output for two tests. + Closes #11597 - Fixes #11231 - Closes #11259 +- docs/cmdline: add small "warning" to verbose options -- runtests: make test file directories in log/N + "Note that verbose output of curl activities and network traffic might + contain sensitive data, including user names, credentials or secret data + content. Be aware and be careful when sharing trace logs with others." - Test files in subdirectories were not created after parallel test log - directories were moved down a level due to a now-bad comparison. + Closes #11596 - Follow-up to 92d7dd39 +- RELEASE-NOTES: synced - Ref #11264 - Closes #11267 +- pingpong: don't use *bump_headersize -Daniel Stenberg (7 Jun 2023) + We use that for HTTP(S) only. -- ws: make the curl_ws_meta() return pointer a const + Follow-up to 3ee79c1674fd6 - The returned info is read-only for the user. + Closes #11590 - Closes #11261 +- urldata: remove spurious parenthesis to unbreak no-proxy build -- RELEASE-NOTES: synced + Follow-up to e12b39e13382 -- runtests: move parallel log dirs from logN to log/N + Closes #11591 - Having several hundreds of them in there gets annoying. +- easy: don't call Curl_trc_opt() in disabled-verbose builds - Closes #11264 + Follow-up to e12b39e133822c6a0 -Dan Fandrich (7 Jun 2023) + Closes #11588 -- test447: move the test file into %LOGDIR +- http: use %u for printfing int -Viktor Szakats (7 Jun 2023) + Follow-up to 3ee79c1674fd6f99e8efca5 -- cmake: add support for "unity" builds + Closes #11587 - Aka "jumbo" or "amalgamation" builds. It means to compile all sources - per target as a single C source. This is experimental. +Goro FUJI (3 Aug 2023) - You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. - It requires CMake 3.16 or newer. +- vquic: show stringified messages for errno - It makes builds (much) faster, allows for better optimizations and tends - to promote less ambiguous code. + Closes #11584 - Also add a new AppVeyor CI job and convert an existing one to use - "unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job. +Stefan Eissing (3 Aug 2023) - Fix related issues: - - add missing include guard to `easy_lock.h`. - - rename static variables and functions (and a macro) with names reused - across sources, or shadowed by local variables. - - add an `#undef` after use. - - add a missing `#undef` before use. - - move internal definitions from `ftp.h` to `ftp.c`. - - `curl_memory.h` fixes to make it work when included repeatedly. - - stop building/linking curlx bits twice for a static-mode curl tool. - These caused doubly defined symbols in unity builds. - - silence missing extern declarations compiler warning for ` _CRT_glob`. - - fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`. - - fix colliding static symbols in debug mode: `debugtime()` and - `statename`. - - rename `ssl_backend_data` structure to unique names for each - TLS-backend, along with the `ssl_connect_data` struct member - referencing them. This required adding casts for each access. - - add workaround for missing `[P]UNICODE_STRING` types in certain Windows - builds when compiling `lib/ldap.c`. To support "unity" builds, we had - to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows - `schannel.h` option) _globally_. This caused an indirect inclusion of - Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled - as well. This requires `[P]UNICODE_STRING` types, which is apperantly - not defined automatically (as seen with both MSVS and mingw-w64). - This patch includes `` to fix it. - Ref: https://github.com/curl/curl/runs/13987772013 - Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&vie - w=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f - 38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c - - tweak unity builds to compile `lib/memdebug.c` separately in memory - trace builds to avoid PP confusion. - - force-disable unity for test programs. - - do not compile and link libcurl sources to libtests _twice_ when libcurl - is built in static mode. +- trace: make tracing available in non-debug builds - KNOWN ISSUES: - - running tests with unity builds may fail in cases. - - some build configurations/env may not compile in unity mode. E.g.: - https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfu - auwl8q#L250 + Add --trace-config to curl - Ref: https://github.com/libssh2/libssh2/issues/1034 - Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html - Ref: https://en.wikipedia.org/wiki/Unity_build + Add curl_global_trace() to libcurl - Closes #11095 + Closes #11421 -Daniel Stenberg (7 Jun 2023) +Daniel Stenberg (3 Aug 2023) -- examples/websocket.c: websocket example using CONNECT_ONLY +- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" - Closes #11262 + See also https://github.com/curl/curl/pull/7507 -- websocket-cb: example doing WebSocket download using callback +- TODO: add "WebSocket read callback" - Very basic + remove "Upgrade to websockets" as we already have this - Closes #11260 + Closes #11402 -- test/.gitignore: ignore log* +- test497: verify rejecting too large incoming headers -Dan Fandrich (5 Jun 2023) +- http: return error when receiving too large header set -- runtests: document the -j parallel testing option + To avoid abuse. The limit is set to 300 KB for the accumulated size of + all received HTTP headers for a single response. Incomplete research + suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to + 1MB. - Reported-by: Daniel Stenberg - Ref: #10818 - Closes #11255 + Closes #11582 -- runtests: create multiple test runners when requested +Stefan Eissing (3 Aug 2023) - Parallel testing is enabled by using a nonzero value for the -j option - to runtests.pl. Performant values seem to be about 7*num CPU cores, or - 1.3*num CPU cores if Valgrind is in use. +- http2: upgrade tests and add fix for non-existing stream - Flaky tests due to improper log locking (bug #11231) are exacerbated - while parallel testing, so it is not enabled by default yet. + - check in h2 filter recv that stream actually exists + and return error if not + - add test for parallel, extreme h2 upgrades that fail if + connections get reused before fully switched + - add h2 upgrade upload test just for completeness - Fixes #10818 - Closes #11246 + Closes #11563 -- runtests: handle repeating tests in multiprocess mode +Viktor Szakats (3 Aug 2023) - Such as what happens with the --repeat option. Some functions are - changed to pass the runner ID instead of relying on the non-unique test - number. +- tests: ensure `libcurl.def` contains all exports - Ref: #10818 + Add `test1279` to verify that `libcurl.def` lists all exported API + functions found in libcurl headers. -- runtests: buffer logmsg while running singletest() + Also: - This allows all messages relating to a single test case to be displayed - together at the end of the test. + - extend test suite XML `stdout` tag with the `loadfile` attribute. - Ref: #10818 + - fix `tests/extern-scan.pl` and `test1135` to include websocket API. -- runtests: call initserverconfig() in the runner + - use all headers (sorted) in `test1135` instead of a manual list. - This must be done so variables pick up the runner's unique $LOGDIR. + - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. - Ref: #10818 + - add `libcurl.def` to the auto-labeler GHA task. -- runtests: use a per-runner random seed + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 - Each runner needs a unique random seed to reduce the chance of port - number collisions. The new scheme uses a consistent per-runner source of - randomness which results in deterministic behaviour, as it did before. + Closes #11570 - Ref: #10818 +Daniel Stenberg (2 Aug 2023) -- runtests: complete main test loop refactor for multiple runners +- url: change default value for CURLOPT_MAXREDIRS to 30 - The main test loop is now able to handle multiple runners, or no - additional runner processes at all. At most one process is still - created, however. + It was previously unlimited by default, but that's not a sensible + default. While changing this has a remote risk of breaking an existing + use case, I figure it is more likely to actually save users from loops. - Ref: #10818 + Closes #11581 -- runtests: prepare main test loop for multiple runners +- lib: fix a few *printf() flag mistakes - Some variables are expanded to arrays and hashes so that multiple - runners can be used for running tests. + Reported-by: Gisle Vanem + Ref: #11574 + Closes #11579 - Ref: #10818 +Samuel Chiang (2 Aug 2023) -Stefan Eissing (5 Jun 2023) +- openssl: make aws-lc version support OCSP -- bufq: make write/pass methods more robust + And bump version in CI - - related to #11242 where curl enters busy loop when - sending http2 data to the server + Closes #11568 - Closes #11247 +Daniel Stenberg (2 Aug 2023) -Boris Verkhovskiy (5 Jun 2023) +- tool: make the length argument an int for printf()-.* flags -- tool_getparam: fix comment + Closes #11578 - Closes #11253 +- tool_operate: fix memory leak when SSL_CERT_DIR is used -Raito Bezarius (5 Jun 2023) + Detected by Coverity -- haproxy: add --haproxy-clientip flag to spoof client IPs + Follow-up to 29bce9857a12b6cfa726a5 - CURLOPT_HAPROXY_CLIENT_IP in the library + Closes #11577 - Closes #10779 +- tool/var: free memory on OOM -Daniel Stenberg (5 Jun 2023) + Coverity detected this memory leak in OOM situation -- curl: add --ca-native and --proxy-ca-native + Follow-up to 2e160c9c652504e - These are two boolean options to ask curl to use the native OS's CA - store when verifying TLS servers. For peers and for proxies - respectively. + Closes #11575 - They currently only have an effect for curl on Windows when built to use - OpenSSL for TLS. +Viktor Szakats (2 Aug 2023) - Closes #11049 +- gha: bump libressl and mbedtls versions -Viktor Szakats (5 Jun 2023) + Closes #11573 -- build: drop unused/redundant `HAVE_WINLDAP_H` +Jay Satiro (2 Aug 2023) - Sources did not use it. Autotools used it when checking for the - `winldap` library, which is redundant. +- schannel: fix user-set legacy algorithms in Windows 10 & 11 - With CMake, detection was broken: - ``` - Run Build Command(s):/usr/local/Cellar/cmake/3.26.3/bin/cmake -E env VERBOSE= - 1 /usr/bin/make -f Makefile cmTC_2d8fe/fast && /Library/Developer/CommandLine - Tools/usr/bin/make -f CMakeFiles/cmTC_2d8fe.dir/build.make CMakeFiles/cmTC_2 - d8fe.dir/build - Building C object CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj - /usr/local/opt/llvm/bin/clang --target=x86_64-w64-mingw32 --sysroot=/usr/loca - l/opt/mingw-w64/toolchain-x86_64 -D_WINSOCKAPI_="" -I/my/quictls/x64-ucrt/usr - /include -I/my/zlib/x64-ucrt/usr/include -I/my/brotli/x64-ucrt/usr/include -W - no-unused-command-line-argument -D_UCRT -DCURL_HIDDEN_SYMBOLS -DHAVE_SSL_SE - T0_WBIO -DHAS_ALPN -DNGHTTP2_STATICLIB -DNGHTTP3_STATICLIB -DNGTCP2_STATICLIB - -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static-libgcc -lucrt -Wextra -Wall -p - edantic -Wbad-function-cast -Wconversion -Winline -Wmissing-declarations -Wmi - ssing-prototypes -Wnested-externs -Wno-long-long -Wno-multichar -Wpointer-ari - th -Wshadow -Wsign-compare -Wundef -Wunused -Wwrite-strings -Wcast-align -Wde - claration-after-statement -Wempty-body -Wendif-labels -Wfloat-equal -Wignored - -qualifiers -Wno-format-nonliteral -Wno-sign-conversion -Wno-system-headers - - Wstrict-prototypes -Wtype-limits -Wvla -Wshift-sign-overflow -Wshorten-64-to- - 32 -Wdouble-promotion -Wenum-conversion -Wunused-const-variable -Wcomma -Wmis - sing-variable-declarations -Wassign-enum -Wextra-semi-stmt -MD -MT CMakeFile - s/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -MF CMakeFiles/cmTC_2d8fe.dir/HAVE_WINL - DAP_H.c.obj.d -o CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -c /my/curl/b - ld-cmake-llvm-x64-shared/CMakeFiles/CMakeScratch/TryCompile-3JP6dR/HAVE_WINLD - AP_H.c - In file included from /my/curl/bld-cmake-llvm-x64-shared/CMakeFiles/CMakeScra - tch/TryCompile-3JP6dR/HAVE_WINLDAP_H.c:2: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/winldap.h:17: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/schnlsp.h:9: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/schannel.h:10: - /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt - .h:5041:254: error: unknown type name 'PSYSTEMTIME' - WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ - NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, - DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER - pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT - ENSIONS pExtensions); - - - - ^ - /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt - .h:5041:278: error: unknown type name 'PSYSTEMTIME' - WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ - NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, - DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER - pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT - ENSIONS pExtensions); - - - - ^ - 2 errors generated. - make[1]: *** [CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj] Error 1 - make: *** [cmTC_2d8fe/fast] Error 2 - exitCode: 2 - ``` + - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then + use the SCHANNEL_CRED legacy structure to pass the list to Schannel. - Cherry-picked from #11095 88e4a21ff70ccef391cf99c8165281ff81374503 - Reviewed-by: Daniel Stenberg - Closes #11245 + - If the user set both a legacy algorithm list and a TLS 1.3 cipher list + then abort. -Daniel Stenberg (5 Jun 2023) + Although MS doesn't document it, Schannel will not negotiate TLS 1.3 + when SCHANNEL_CRED is used. That means setting a legacy algorithm list + limits the user to earlier versions of TLS. -- urlapi: scheme starts with alpha + Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would + ignore legacy algorithms in Windows 10 1809 and later. - Add multiple tests to lib1560 to verify + Reported-by: zhihaoy@users.noreply.github.com - Fixes #11249 - Reported-by: ad0p on github - Closes #11250 + Fixes https://github.com/curl/curl/pull/10741 + Closes https://github.com/curl/curl/pull/10746 -- RELEASE-NOTES: synced +Daniel Stenberg (2 Aug 2023) -- CURLOPT_MAIL_RCPT_ALLOWFAILS: replace CURLOPT_MAIL_RCPT_ALLLOWFAILS +- variable.d: setting a variable again overwrites it - Deprecate the name using three Ls and prefer the name with two. + Reported-by: Niall McGee + Bug: https://twitter.com/niallmcgee/status/1686523075423322113 + Closes #11571 - Replaces #10047 - Closes #11218 +Jay Satiro (2 Aug 2023) -- tests/servers: generate temp names in /tmp for unix domain sockets +- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting - ... instead of putting them in the regular pid directories because - systems generally have strict length requirements for the path name to - be shorter than 107 bytes and we easily hit that boundary otherwise. + - Re-wrap CURLSSLOPT_ALLOW_BEAST description. - The new concept generates two random names: one for the socks daemon and - one for http. +Daniel Stenberg (2 Aug 2023) - Reported-by: Andy Fiddaman - Fixes #11152 - Closes #11166 +- RELEASE-NOTES: synced -Stefan Eissing (2 Jun 2023) +- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set -- http2: better support for --limit-rate + Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not + used, thus unnecessarily asking for addresses that will not be used. - - leave transfer loop when --limit-rate is in effect and has - been received - - adjust stream window size to --limit-rate plus some slack - to make the server observe the pacing we want - - add test case to confirm behaviour + Reported-by: Joseph Tharayil + Fixes #11564 + Closes #11565 - Closes #11115 +- docs: link to the website versions instead of markdowns -- curl_log: evaluate log statement only when transfer is verbose + ... to make the links work when the markdown is converted to webpages on + https://curl.se - Closes #11238 + Reported-by: Maurício Meneghini Fauth + Fixes https://github.com/curl/curl-www/issues/272 + Closes #11569 -Daniel Stenberg (2 Jun 2023) +Viktor Szakats (1 Aug 2023) -- libssh2: provide error message when setting host key type fails +- cmake: cache more config and delete unused ones - Ref: https://curl.se/mail/archive-2023-06/0001.html + - cache more Windows config results for faster initialization. - Closes #11240 + - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. -Igor Todorovski (2 Jun 2023) + - delete dead references to `sys/utsname.h`. -- system.h: remove __IBMC__/__IBMCPP__ guards and apply to all z/OS compiles + Closes #11551 - Closes #11241 +- egd: delete feature detection and related source code -Daniel Stenberg (2 Jun 2023) + EGD is Entropy Gathering Daemon, a socket-based entropy source supported + by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it + a while ago. -- docs/SECURITY-PROCESS.md: link to example of previous critical flaw + Its detection in CMake was broken all along because OpenSSL libs were + not linked at the point of feature check. -Mark Seuffert (2 Jun 2023) + Delete detection from both cmake and autotools, along with the related + source snippet, and the `--with-egd-socket=` `./configure` option. -- README.md: updated link to opencollective + Closes #11556 - Closes #11232 +Stefan Eissing (1 Aug 2023) -Daniel Stenberg (1 Jun 2023) +- tests: fix h3 server check and parallel instances -- libssh2: use custom memory functions + - fix check for availability of nghttpx server + - add `tcp` frontend config for same port as quic, as + without this, port 3000 is bound which clashes for parallel + testing - Because of how libssh2_userauth_keyboard_interactive_ex() works: the - libcurl callback allocates memory that is later free()d by libssh2, we - must set the custom memory functions. + Closes #11553 - Reverts 8b5f100db388ee60118c08aa28 +Daniel Stenberg (1 Aug 2023) - Ref: https://github.com/libssh2/libssh2/issues/1078 - Closes #11235 +- docs/cmdline-opts: spellfixes, typos and polish -- test447: test PUTting a file that grows + To make them accepted by the spell checker - ... and have curl trim the end when it reaches the expected total amount - of bytes instead of over-sending. + Closes #11562 - Reported-by: JustAnotherArchivist on github - Closes #11223 +- CI/spellcheck: build curl.1 and spellcheck it -- curl: count uploaded data to stop at the originally given size + Added acceptable words - Closes #11223 - Fixes #11222 - Reported-by: JustAnotherArchivist on github + Closes #11562 -- tool: remove exclamation marks from error/warning messages +Alexander Jaeger (1 Aug 2023) -- tool: use errorf() for error output +- misc: fix various typos - Convert a number of fprintf() calls. + Closes #11561 -- tool: remove newlines from all helpf/notef/warnf/errorf calls +Daniel Stenberg (1 Aug 2023) - Make voutf() always add one. +- http2: avoid too early connection re-use/multiplexing - Closes #11226 + HTTP/1 connections that are upgraded to HTTP/2 should not be picked up + for reuse and multiplexing by other handles until the 101 switching + process is completed. -- tests/servers.pm: pick unused port number with a server socket + Lots-of-debgging-by: Stefan Eissing + Reported-by: Richard W.M. Jones + Bug: https://curl.se/mail/lib-2023-07/0045.html + Closes #11557 - This change replaces the previous method of picking a port number at - random to try to start servers on, then retrying up to ten times with - new random numbers each time, with a function that creates a server - socket on port zero, thereby getting a suitable random port set by the - kernel. That server socket is then closed and that port number is used - to setup the actual test server on. +- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" - There is a risk that *another* server can be started on the machine in - the time gap, but the server verification feature will detect that. + This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. - Closes #11220 + It's a user error for supplying incomplete information to the build system. -- RELEASE-NOTES: synced + Reported-by: Ryan Schmidt + Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 - bump to 8.2.0 +Viktor Szakats (1 Aug 2023) -Alejandro R. Sedeño (31 May 2023) +- cmake: add support for single libcurl compilation pass -- configure: fix run-compiler for old /bin/sh + Before this patch CMake builds used two separate compilation passes to + build the shared and static libcurl respectively. This patch allows to + reduce that to a single pass if the target platform and build settings + allow it. - If you try to assign and export on the same line on some older /bin/sh - implementations, it complains: + This reduces CMake build times when building both static and shared + libcurl at the same time, making these dual builds an almost zero-cost + option. - ``` - $ export "NAME=value" - NAME=value: is not an identifier - ``` + Enable this feature for Windows builds, where the difference between the + two passes was the use of `__declspec(dllexport)` attribute for exported + API functions for the shared builds. This patch replaces this method + with the use of `libcurl.def` at DLL link time. - This commit rewrites run-compiler's assignments and exports to work with - old /bin/sh, splitting assignment and export into two separate - statements, and only quote the value. So now we have: + Also update `Makefile.mk` to use `libcurl.def` to export libcurl API + symbols on Windows. This simplifies (or fixes) this build method (e.g. + in curl-for-win, which generated a `libcurl.def` from `.h` files using + an elaborate set of transformations). - ``` - NAME="value" - export NAME - ``` + `libcurl.def` has the maintenance cost of keeping the list of public + libcurl API symbols up-to-date. This list seldom changes, so the cost + is low. - While we're here, make the same change to the two supporting - assign+export lines preceeding the script to be consistent with how - exports work throughout the rest of configure.ac. + Closes #11546 - Closes #11228 +- cmake: detect `SSL_set0_wbio` in OpenSSL -Philip Heiduck (31 May 2023) + Present in OpenSSL 1.1.0 and BoringSSL. + Missing from LibreSSL 3.8.0. -- circleci: install impacket & wolfssl 5.6.0 + Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 - Closes #11221 + While here, also fix `RAND_egd()` detection which was broken, likely all + along. This feature is probably broken with CMake builds and also + requires a sufficiently obsolete OpenSSL version, so this part of the + update was not tested. -Daniel Stenberg (31 May 2023) + Closes #11555 -- tool_urlglob: use curl_off_t instead of longs +- cmake: fixup H2 duplicate symbols for unity builds - To handle more globs better (especially on Windows) + Closes #11550 - Closes #11224 +Pablo Busse (1 Aug 2023) -Dan Fandrich (30 May 2023) +- openssl: Support async cert verify callback -- scripts: Fix GHA matrix job detection in cijobs.pl + - Update the OpenSSL connect state machine to handle + SSL_ERROR_WANT_RETRY_VERIFY. - The parsing is pretty brittle and it broke detecting some jobs at some - point. Also, detect if Windows is used in GHA. + This allows libcurl users that are using custom certificate validation + to suspend processing while waiting for external I/O during certificate + validation. -- runtests: abort test run after failure without -a + Closes https://github.com/curl/curl/pull/11499 - This was broken in a recent refactor and test runs would not stop. +Jay Satiro (1 Aug 2023) - Follow-up to d4a1b5b6 +- tool_cb_wrt: fix invalid unicode for windows console - Reported-by: Daniel Stenberg - Fixes #11225 - Closes #11227 + - Suppress an incomplete UTF-8 sequence at the end of the buffer. -Version 8.1.2 (30 May 2023) + - Attempt to reconstruct incomplete UTF-8 sequence from prior call(s) + in current call. -Daniel Stenberg (30 May 2023) + Prior to this change, in Windows console UTF-8 sequences split between + two or more calls to the write callback would cause invalid "replacement + characters" U+FFFD to be printed instead of the actual Unicode + character. This is because in Windows only UTF-16 encoded characters are + printed to the console, therefore we convert the UTF-8 contents to + UTF-16, which cannot be done with partial UTF-8 sequences. -- RELEASE-NOTES: synced + Reported-by: Maksim Arhipov - 8.1.2 release + Fixes https://github.com/curl/curl/issues/9841 + Closes https://github.com/curl/curl/pull/10890 -- THANKS: contributors from 8.1.2 +Daniel Stenberg (1 Aug 2023) -- lib1560: verify more scheme guessing +- sectransp: prevent CFRelease() of NULL - - on 2nd level domains - - on names without dots + When SecCertificateCopyCommonName() returns NULL, the common_name + pointer remains set to NULL which apparently when calling CFRelease() on + (sometimes?) crashes. - As mentioned in #11161, "imap.com" will be guessed IMAP + Reported-by: Guillaume Algis + Fixes #9194 + Closes #11554 - Closes #11219 +Jay Satiro (1 Aug 2023) -- page-header: minor wording polish in the URL segment +- vtls: clarify "ALPN: offers" message - Closes #11217 + Before: + * ALPN: offers h2,http/1.1 -- page-header: mention curl version and how to figure out current release + After: + * ALPN: curl offers h2,http/1.1 - Closes #11216 + Bug: https://curl.se/mail/lib-2023-07/0041.html + Reported-by: Richard W.M. Jones + Closes #11544 -- RELEASE-NOTES: synced +Daniel Stenberg (1 Aug 2023) -- configure: without pkg-config and no custom path, use -lnghttp2 +- urlapi: make sure zoneid is also duplicated in curl_url_dup - Reported-by: correctmost on github - Fixes #11186 - Closes #11210 + Add several curl_url_dup() tests to the general lib1560 test. -Stefan Eissing (28 May 2023) + Reported-by: Rutger Broekhoff + Bug: https://curl.se/mail/lib-2023-07/0047.html + Closes #11549 -- curl: cache the --trace-time value for a second +Sergey (1 Aug 2023) - - caches HH:MM:SS computed and reuses it for logging during - the same second. - - common function for plain log line start formatting +- urlapi: fix heap buffer overflow - Closes #11211 + `u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-ter + minator. -Kev Jackson (28 May 2023) + ``` + ==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 a + t pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4 + READ of size 78 at 0x04d48c75 thread T0 + #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\c + ompiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840 + #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_p + arty\curl\lib\strdup.c:97 + #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_part + y\curl\lib\urlapi.c:1297 + #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\clien + t\third_party\curl\lib\urlapi.c:1342 + #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_ + party\curl\lib\urlapi.c:1790 + #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client + \third_party\curl\lib\url.c:1768 + #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_p + arty\curl\lib\url.c:3403 + #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_ + party\curl\lib\url.c:3888 + #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\thi + rd_party\curl\lib\multi.c:1982 + #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\ + third_party\curl\lib\multi.c:2756 + ``` -- libcurl.m4: remove trailing 'dnl' that causes this to break autoconf + Closes #11560 - Closes #11212 +Daniel Stenberg (31 Jul 2023) -Stefan Eissing (26 May 2023) +- curl: make %output{} in -w specify a file to write to -- http3: send EOF indicator early as possible + It can be used multiple times. Use %output{>>name} to append. - - ngtcp2 and quiche implementations relied on the DONE_SEND event - to forward the EOF for uploads to the libraries. This often - result in a last 0 length EOF data. Tracking the amount of - data left to upload allows EOF indication earlier. - - refs #11205 where CloudFlare DoH servers did not like to - receive the initial upload DATA without EOF and returned - a 400 Bad Request + Add docs. Test 990 and 991 verify. - Reported-by: Sergey Fionov - Fixes #11205 - Closes #11207 + Idea: #11400 + Suggested-by: ed0d2b2ce19451f2 + Closes #11416 -Daniel Stenberg (26 May 2023) +- RELEASE-NOTES: synced -- scripts/contri*sh: no longer grep -v ' ' +- tool: add "variable" support - Originally these scripts filtered out names that have no space so that - they better avoid nick names not intended for credits. Such names are - not too commonly used, plus we now give credit even to those. + Add support for command line variables. Set variables with --variable + name=content or --variable name@file (where "file" can be stdin if set + to a single dash (-)). - Additionally: non-latin names, like Asian, don't have spaces at all so - they were also filtered out and had to be manually added which made it - an error-prone operation where Asian names eventually easily fell off by - mistake. + Variable content is expanded in option parameters using "{{name}}" + (without the quotes) if the option name is prefixed with + "--expand-". This gets the contents of the variable "name" inserted, or + a blank if the name does not exist as a variable. Insert "{{" verbatim + in the string by prefixing it with a backslash, like "\\{{". - Closes #11206 + Import an environment variable with --variable %name. It makes curl exit + with an error if the environment variable is not set. It can also rather + get a default value if the variable does not exist, using =content or + @file like shown above. -- cf-socket: restore Curl_sock_assign_addr() + Example: get the USER environment variable into the URL: - Regression since it was not private. Also used by msh3.c + --variable %USER + --expand-url = "https://example.com/api/{{USER}}/method" - Follow-up to 8e85764b7bd7f05f5 - Reported-by: Gisle Vanem - Fixes #11202 - Closes #11204 + When expanding variables, curl supports a set of functions that can make + the variable contents more convenient to use. It can trim leading and + trailing white space with "trim", output the contents as a JSON quoted + string with "json", URL encode it with "url" and base 64 encode it with + "b64". To apply functions to a variable expansion, add them colon + separated to the right side of the variable. They are then performed in + a left to right order. -- RELEASE-NOTES: synced + Example: get the contents of a file called $HOME/.secret into a variable + called "fix". Make sure that the content is trimmed and percent-encoded + sent as POST data: - Taken down to 8.1.2 now for pending patch release + --variable %HOME=/home/default + --expand-variable fix@{{HOME}}/.secret + --expand-data "{{fix:trim:url}}" + https://example.com/ -- libssh: when keyboard-interactive auth fails, try password + Documented. Many new test cases. - The state machine had a mistake in that it would not carry on to that - next step. + Co-brainstormed-by: Emanuele Torre + Assisted-by: Jat Satiro + Closes #11346 - This also adds a verbose output what methods that are available from the - server and renames the macros that change to the next auth methods to - try. +- KNOWN_BUGS: cygwin: make install installs curl-config.1 twice - Reported-by: 左潇峰 - Fixes #11196 - Closes #11197 + Closes #8839 -Emanuele Torre (25 May 2023) +- KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14 -- configure: fix build with arbitrary CC and LD_LIBRARY_PATH + Closes #11215 - Since ./configure and processes that inherit its environment variables - are the only callers of the run-compiler script, we can just save the - current value of the LD_LIBRARY_PATH and CC variables to another pair of - environment variables, and make run-compiler a static script that - simply restores CC and LD_LIBRARY_PATH to the saved value, and before - running the compiler. +- KNOWN_BUGS: cmake outputs: no version information available - This avoids having to inject the values of the variables in the script, - possibly causing problems if they contains spaces, quotes, and other - special characters. + Closes #11158 - Also add exports in the script just in case LD_LIBRARY_PATH and CC are - not already in the environment. +- KNOWN_BUGS: APOP authentication fails on POP3 - follow-up from 471dab2 + Closes #10073 - Closes #11182 +- KNOWN_BUGS: hyper is slow -Daniel Stenberg (25 May 2023) + Closes #11203 -- urlapi: remove superfluous host name check +Patrick Monnerat (31 Jul 2023) - ... as it is checked later more proper. +- configure, cmake, lib: more form api deprecation - Closes #11195 + Introduce a --enable-form-api configure option to control its inclusion + in builds. The condition name defined for it is CURL_DISABLE_FORM_API. -Stefan Eissing (25 May 2023) + Form api code is dependent of MIME: configure and CMake handle this + dependency automatically: CMake by making it a dependent option + explicitly, configure by inheriting the MIME value by default and + rejecting explicit incompatible values. -- http2: fix EOF handling on uploads with auth negotiation + "form-api" is now a new hidden test feature. - - doing a POST with `--digest` does an override on the initial request - with `Content-Length: 0`, but the http2 filter was unaware of that - and expected the originally request body. It did therefore not - send a final DATA frame with EOF flag to the server. - - The fix overrides any initial notion of post size when the `done_send` - event is triggered by the transfer loop, leading to the EOF that - is necessary. - - refs #11194. The fault did not happen in testing, as Apache httpd - never tries to read the request body of the initial request, - sends the 401 reply and closes the stream. The server used in the - reported issue however tried to read the EOF and timed out on the - request. + Update libcurl modules to respect this option and adjust tests + accordingly. - Reported-by: Aleksander Mazur - Fixes #11194 - Cloes #11200 + Closes #9621 -Daniel Stenberg (23 May 2023) +Daniel Stenberg (31 Jul 2023) -- RELEASE-NOTES: synced +- mailmap: add Derzsi Dániel - bump to 8.2.0 +Derzsi Dániel (31 Jul 2023) -- lib: remove unused functions, make single-use static +- wolfssl: support loading system CA certificates - Closes #11174 + Closes #11452 -- scripts/singleuse.pl: add more API calls +Viktor Szakats (30 Jul 2023) -Christian Hesse (23 May 2023) +- nss: delete more NSS references -- configure: quote the assignments for run-compiler + Fix the distcheck CI failure and delete more NSS references. - Building for multilib failed, as the compiler command contains an - extra argument. That needs quoting. + Follow-up to 7c8bae0d9c9b2dfeeb008b9a316117d7b9675175 - Regression from b78ca50cb3dda361f9c1 + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Closes #11548 - Fixes #11179 - Closes #11180 +Daniel Stenberg (29 Jul 2023) -Daniel Stenberg (23 May 2023) +- nss: remove support for this TLS library -- misc: fix spelling mistakes + Closes #11459 - Reported-by: musvaage on github - Fixes #11171 - Closes #11172 +Ryan Schmidt (29 Jul 2023) -Version 8.1.1 (23 May 2023) +- macOS: fix target detection more -Daniel Stenberg (23 May 2023) + Now SCDynamicStoreCopyProxies is called (and the required frameworks are + linked in) on all versions of macOS and only on macOS. Fixes crash due + to undefined symbol when built with the macOS 10.11 SDK or earlier. -- RELEASE-NOTES: synced + CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and + is now only defined when SCDynamicStoreCopyProxies will actually be + called. Previously, it was defined when ENABLE_IPV6 was not defined but + SCDynamicStoreCopyProxies is not called in that case. - curl 8.1.1 + TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only + when dynamic targets are enabled. TARGET_OS_MAC is always defined but + means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS. + TARGET_OS_IPHONE means any Darwin OS other than macOS. -- THANKS: contributors from the 8.1.1 release + Follow-up to c73b2f82 -Dan Fandrich (22 May 2023) + Fixes #11502 + Closes #11516 -- docs: fix fuzzing documentation link +Daniel Stenberg (29 Jul 2023) - Follow-up to 4c712a1b +- tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR -- CI: add an Alpine build with MUSL + ... used at once. - MUSL is another libc implementation which has its own unique issues - worth testing. + Reported-by: Gabriel Corona + Fixes #11325 + Closes #11531 - Ref: #11140 - Closes #11178 +Thomas M. DuBuisson (29 Jul 2023) -- runtests: add a missing \n at the end of a log message +- CI: remove Lift's configuration -correctmost on github (22 May 2023) + The Lift tool is being retired. Their site reads: -- SECURITY-PROCESS.md: link security advisory doc and fix typo + "Sonatype Lift will be retiring on Sep 12, 2023, with its analysis + stopping on Aug 12, 2023." - Closes #11177 + Closes #11541 -Daniel Stenberg (22 May 2023) +Nathan Moinvaziri (29 Jul 2023) -- TODO: build curl with Windows Unicode support +- Revert "schannel: reverse the order of certinfo insertions" - Closes #7229 + This reverts commit 8986df802db9b5338d9d50a54232ebae4dbcf6dd. -- KNOWN_BUGS: hyper memory-leaks + Windows does not guarantee a particular certificate ordering, even + though TLS may have its own ordering/relationship guarantees. Recent + versions of Windows 11 reversed the ordering of ceritifcates returned by + CertEnumCertificatesInStore, therefore this commit no longer works as + initially intended. libcurl makes no guarantees about certificate + ordering if the operating system can't. - Closes #10803 + Ref: https://github.com/curl/curl/issues/9706 -Stefan Eissing (22 May 2023) + Closes https://github.com/curl/curl/pull/11536 -- http/2: unstick uploads +wangzhikun (29 Jul 2023) - - refs #11157 and #11175 where uploads get stuck or lead to RST streams - - fixes our h2 send behaviour to continue sending in the nghttp2 session - as long as it wants to. This will empty our send buffer as long as - the remote stream/connection window allows. - - in case the window is exhausted, the data remaining in the send buffer - will wait for a WINDOW_UPDATE from the server. Which is a socket event - that engages our transfer loop again - - the problem in the issue was that we did not exhaust the window, but - left data in the sendbuffer and no further socket events did happen. - The server was just waiting for us to send more. - - relatedly, there was an issue fixed that closing a stream with KEEP_HOLD - set kept the transfer from shutting down - as it should have - leading - to a timeout. +- winbuild: improve check for static zlib - Closes #11176 + - Check for zlib static library name zlibstatic.lib. -Daniel Stenberg (21 May 2023) + zlib's static library has a different name depending on how it was + built. zlibstatic.lib is output by cmake. zlibstat.lib is output by + their pre-generated Visual Studio project files (in the contrib + directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall + instead of cdecl if you end up exporting the zlib functions). -- workflows/macos: add a job using gcc + debug + secure transport + Prior to this change the makefile only checked for the latter. -Jay Satiro (21 May 2023) + Closes https://github.com/curl/curl/pull/11521 -- lib: fix conversion warnings with gcc on macOS +Daniel Stenberg (29 Jul 2023) -Daniel Stenberg (21 May 2023) +- configure: use the pkg-config --libs-only-l flag for libssh2 -- sectransp.c: make the code c89 compatible + ... instead of --libs, as that one also returns -L flags. - Follow-up to dd2bb485521c2ec713001b3a + Reported-by: Wilhelm von Thiele + Fixes #11538 + Closes #11539 - Reported-by: FeignClaims on github - Fixes #11155 - Closes #11159 +Viktor Szakats (29 Jul 2023) -Emanuele Torre (21 May 2023) +- cmake: support building static and shared libcurl in one go -- Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirect - s" + This patch adds the ability to build a static and shared libcurl library + in a single build session. It also adds an option to select which one to + use when building the curl executable. - This reverts commit df6c2f7b544f1f35f2a3e0be11f345affeb6fe9c. - (It only keep the test case that checks redirection to an absolute URL - without hostname and CURLU_NO_AUTHORITY). + New build options: + - `BUILD_STATIC_LIBS`. Default: `OFF`. + Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`. + - `BUILD_STATIC_CURL`. Default: `OFF`. + Requires `BUILD_STATIC_LIBS` enabled. + Enabled automatically if building static libcurl only. + - `STATIC_LIB_SUFFIX`. Default: empty. + - `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide + with static lib name (typically with MSVC) in Windows builds. + Otherwise empty. - I originally wanted to make CURLU_ALLOW_SPACE accept spaces in the - hostname only because I thought - curl_url_set(CURLUPART_URL, CURLU_ALLOW_SPACE) was already accepting - them, and they were only not being accepted in the hostname when - curl_url_set(CURLUPART_URL) was used for a redirection. + Also: - That is not actually the case, urlapi never accepted hostnames with - spaces, and a hostname with a space in it never makes sense. - I probably misread the output of my original test when I they were - normally accepted when using CURLU_ALLOW_SPACE, and not redirecting. + - Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass + it directly to the compiler. This also allows to delete a condition + from `tests/server/CMakeLists.txt`. - Some other URL parsers seems to allow space in the host part of the URL, - e.g. both python3's urllib.parse module, and Chromium's javascript URL - object allow spaces (chromium percent escapes the spaces with %20), - (they also both ignore TABs, and other whitespace characters), but those - URLs with spaces in the hostname are useless, neither python3's requests - module nor Chromium's window.location can actually use them. + - Complete a TODO by following the logic used in autotools (also for + `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of + `libcurl.pc` for _static-only_ curl builds. - There is no reason to add support for URLs with spaces in the host, - since it was not a inconsistency bug; let's revert that patch before it - makes it into release. Sorry about that. + - Convert an existing CI test to build both shared and static libcurl. - I also reverted the extra check for CURLU_NO_AUTHORITY since that does - not seem to be necessary, CURLU_NO_AUTHORITY already worked for - redirects. + Closes #11505 - Closes #11169 +Stefan Eissing (28 Jul 2023) -Dan Fandrich (20 May 2023) +- CI/awslc: add cache for build awslc library -- runtests: use the correct fd after select + Closes #11535 - The code was using the wrong fd when determining which runner was ready - with a response. +- GHA/linux.yml: add caching - Ref: #10818 - Closes #11160 + Closes #11532 -- test425: fix the log directory for the upload +Daniel Stenberg (27 Jul 2023) - This must be %LOGDIR to let it work with parallel tests. +- RELEASE-NOTES: synced - Ref: #10969 + Bump working version to 8.3.0 -- runtests: handle interrupted reads from IPC pipes +- url: remove infof() output for "still name resolving" - These can be interrupted by signals, especially SIGINT to shut down, and - must be restarted so the IPC call arrives correctly. If the read just - returns an error instead, the IPC calling state will go out of sync and - a proper shutdown won't happen. + The message does not help and might get spewed a lot during times. - Ref: #10818 + Reported-by: yushicheng7788 on github + Fixes #11394 + Closes #11529 -Stefan Eissing (20 May 2023) +- KNOWN_BUGS: cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!" -- http2: upload improvements + Closes #11244 - Make send buffer smaller to have progress and "upload done" reporting - closer to reality. Fix handling of send "drain" condition to no longer - trigger once the transfer loop reports it is done sending. Also do not - trigger the send "drain" on RST streams. +Stefan Eissing (27 Jul 2023) - Background: - - a upload stall was reported in #11157 that timed out - - test_07_33a reproduces a problem with such a stall if the - server 404s the request and RSTs the stream. - - test_07_33b verifies a successful PUT, using the parameters - from #11157 and checks success +- CI: quiche updates - Ref: #11157 - Closes #11165 + - remove quiche from standard `linux` workflow + - add mod_h2 caching to quiche workflow + - rename quiche to quiche-linux + - move version definitions into env section -- http2: increase stream window size to 10 MB + Closes #11528 - Reported-by: pandada8 on github +- http2: disable asssertion blocking OSSFuzz testing - Fixes #11162 - Closes #11167 + - not clear how this triggers and it blocks OSSFuzz testing other + things. Since we handle the case with an error return, disabling the + assertion for now seems the best way forward. -Daniel Stenberg (20 May 2023) + Fixes #11500 + Closes #11519 -- lib: rename struct 'http_req' to 'httpreq' +- http2: fix in h2 proxy tunnel: progress in ingress on sending - Because FreeBSD 14 kidnapped the name. - Ref: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271526 + - depending on what is tunneled, the proxy may never get invoked for + receiving data explicitly. Not progressing ingress may lead to stalls + due to missed WINDOW_UPDATEs. - Fixes #11163 - Closes #11164 + CI: + - add a chache for building mod_h2 -Viktor Szakats (20 May 2023) + Closes #11527 -- cmake: avoid `list(PREPEND)` for compatibility +- CI ngtcp2+quictls: use nghttpx cache as in quiche build - `list(PREPEND)` requires CMake v3.15, our minimum is v3.7. +Jay Satiro (27 Jul 2023) - Ref: https://cmake.org/cmake/help/latest/command/list.html#prepend +- bearssl: don't load CA certs when peer verification is disabled - Regression from 1e3319a167d2f32d295603167486e9e88af9bb4e + We already do this for other SSL backends. - Reported-by: Keitagit-kun on Github - Fixes #11141 - Closes #11144 + Bug: https://github.com/curl/curl/pull/11457#issuecomment-1644587473 + Reported-by: kyled-dell@users.noreply.github.com -Daniel Stenberg (19 May 2023) + Closes https://github.com/curl/curl/pull/11497 -- RELEASE-NOTES: synced +Daniel Stenberg (26 Jul 2023) -Stefan Eissing (19 May 2023) +- easy: remove #ifdefs to make code easier on the eye -- ngtcp2: proper handling of uint64_t when adjusting send buffer + Closes #11525 - Fixes #11149 - Closes #11153 +Stefan Eissing (26 Jul 2023) -- ngtcp2: fix compiler warning about possible null-deref +- GHA: adding quiche workflow - - compiler analyzer did not include the call context for this - static function where the condition had already been checked. - - eleminating the problem by making stream a call parameter + - adding separate quiche workflow to also build nghttpx server for testing - Fixes #11147 - Closes #11151 + Closes #11517 -Emanuele Torre (19 May 2023) +Version 8.2.1 (26 Jul 2023) -- docs: document that curl_url_cleanup(NULL) is a safe no-op +Daniel Stenberg (26 Jul 2023) - This has always been the case, but it was not documented. +- RELEASE-NOTES: synced - The paragraph was copied verbatim from curl_easy_cleanup.3 + curl 8.2.1 release - Closes #11150 +- THANKS: add contributors from 8.2.1 -Antoine Pitrou (19 May 2023) +- docs: provide more see also for cipher options -- select: avoid returning an error on EINTR from select() or poll() + More cross references. Hide nroff errors. - This was already done for the poll() and select() calls - made directly from Curl_poll(), but was missed in - Curl_wait_ms(), which is called when there are no fds - to wait on. + Closes #11513 - Fixes #11135 - Closes #11143 +- docs: mark two TLS options for TLS, not SSL -Daniel Stenberg (19 May 2023) + Closes #11514 -- vquic.c: make recvfrom_packets static, avoid compiler warning +Brad Harder (25 Jul 2023) - warning: no previous prototype for 'recvfrom_packets' +- curl_multi_wait.3: fix arg quoting to doc macro .BR - Reported-by: Keitagit-kun on github - Fixes #11146 - Closes #11148 + Closes #11511 -- urlapi: allow numerical parts in the host name +Daniel Stenberg (24 Jul 2023) - It can only be an IPv4 address if all parts are all digits and no more than - four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be - passed through as a host name. +- RELEASE-NOTES: synced - Regression from 17a15d88467 shipped in 8.1.0 +Viktor Szakats (24 Jul 2023) - Extended test 1560 accordingly. +- cmake: update ngtcp2 detection - Reported-by: Pavel Kalyugin - Fixes #11129 - Closes #11131 + Replace `OpenSSL` with `quictls` to follow the same change + in the v0.17.0 ngtcp2 release. -Emilio Cobos Álvarez (19 May 2023) + Follow-up to e0093b4b732f6495b0fb1cd6747cbfedcdcf63ed -- http2: double http request parser max line length + Closes #11508 - This works around #11138, by doubling the limit, and should be a - relatively safe fix. +Stefan Eissing (24 Jul 2023) - Ideally the buffer would grow as needed and there would be no need for a - limit? But that might be follow-up material. +- http: VLH, very large header test and fixes - Fixes #11138 - Closes #11139 + - adding tests using very large passwords in auth + - fixes general http sending to treat h3 like h2, and + not like http1.1 + - eliminate H2_HEADER max definitions and use the commmon + DYN_HTTP_REQUEST everywhere, different limits do not help + - fix http2 handling of requests denied by nghttp2 on send + to immediately report the refused stream -Emanuele Torre (18 May 2023) + Closes #11509 -- configure: fix --help alignment +Andrei Rybak (23 Jul 2023) - AC_ARG_ENABLE seems to only trim off whitespace from the start and end - of its help-string argument, while prepending two spaces of indentation - to all lines. +- CONTRIBUTE: drop mention of copyright year ranges - This means that the two spaces of indentation between the --enable-rtsp - and the --disable-rtsp line were not removed causing ./configure --help - to print: + Year ranges in copyrights were dropped in commits [1] and [2]. + Verification of year ranges in copyrights was dropped from script + 'scripts/copyright.pl' in commit [3]. However, the corresponding + passages in file 'docs/CONTRIBUTE.md' weren't updated. - Optional Features: - [...] - --enable-rtsp Enable RTSP support - --disable-rtsp Disable RTSP support + Drop mentions of copyright year ranges from 'docs/CONTRIBUTE.md'. - I removed the indentation to fix the issue, now it prints: + [1] 2bc1d775f (copyright: update all copyright lines and remove year + ranges, 2023-01-02) + [2] c46761bd8 (tests/http: remove year ranges from copyrights, + 2023-03-14) + [3] 0e293bacb (copyright.pl: cease doing year verifications, 2023-01-28) - Optional Features: - [...] - --enable-rtsp Enable RTSP support - --disable-rtsp Disable RTSP support + Closes #11504 - The --enable-hsts and --disable-hsts lines had the same problems, and - have been fixed too. +- CONTRIBUTE: fix syntax in commit message description - Closes #11142 + File 'docs/CONTRIBUTE.md' includes a description of how one should write + commit messages in the curl project. Different possible parts of the + message are enclosed in square brackets. One exception is the section + describing how the curl project doesn't use "Signed-off-by" commit + trailers [1], which is enclosed in an opening curly brace paired with a + closing square bracket. -Deal(一线灵) (18 May 2023) + Fix the enclosing square brackets in description of "Signed-off-by" + trailers in commit messages in file 'docs/CONTRIBUTE.md'. -- cmake: repair cross compiling + [1] See description of option '--signoff' in Git documentation: + https://git-scm.com/docs/git-commit - It cannot *run* code for testing purposes when cross-compiling. + Closes #11504 - Closes #11130 +Daniel Stenberg (23 Jul 2023) -Daniel Stenberg (18 May 2023) +- src/mkhelp: strip off escape sequences -- configure: generate a script to run the compiler + At some point the nroff command stopped stripping off escape sequences, + so then this script needs to do the job instead. - in the CURL_RUN_IFELSE macro, with LD_LIBRARY_PATH set to the value of - the configure invoke, and not the value that might be used later, - intended for the execution of the output the compiler ouputs. + Reported-by: VictorVG on github + Fixes #11501 + Closes #11503 - For example when the compiler uses the same library (like libz) that - configure checks for. +- KNOWN_BUGS: building for old macOS fails with gcc - Reported-by: Jonas Bülow - Fixes #11114 - Closes #11120 + Closes #11441 -Stefan Eissing (18 May 2023) +Jacob Hoffman-Andrews (22 Jul 2023) -- cf-socket: completely remove the disabled USE_RECV_BEFORE_SEND_WORKAROUND +- rustls: update rustls-ffi 0.10.0 - Closes #11118 + This brings in version 0.21.0 of the upstream rustls implementation, + which notable includes support for IP address certificates. -Emanuele Torre (18 May 2023) + Closes #10865 -- urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects +Brad Harder (22 Jul 2023) - curl_url_set(uh, CURLUPART_URL, redirurl, flags) was not respecing - CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl - when redirecting to an absolute URL. +- websocket: rename arguments/variables to match docs - Closes #11136 + Pedantry/semantic-alignment between functions, docs, comments with + respect to websocket protocol code; No functional change intended. -Colin Cross (18 May 2023) + * "totalsize", "framesize" becomes "fragsize" (we deal in frame fragments). -- hostip: move easy_lock.h include above curl_memory.h + * "sendflags" becomes "flags" - Similar to #9561, move easy_lock.h above curl_memory.h to fix building - against musl libc. + * use canonical CURL *handle - Closes #11140 + Closes #11493 -Hind Montassif (18 May 2023) +Jan Macku (21 Jul 2023) -- curl_easy_getinfo: clarify on return data types +- bug_report: use issue forms instead of markdown template - Closes #11126 + Issue forms allow you to define web-like input forms using YAML + syntax. It allows you to guide the reporter to get the required + information. -Emanuele Torre (18 May 2023) + Signed-off-by: Jan Macku + Closes #11474 -- checksrc: disallow spaces before labels +Daniel Stenberg (21 Jul 2023) - Out of 415 labels throughout the code base, 86 of those labels were - not at the start of the line. Which means labels always at the start of - the line is the favoured style overall with 329 instances. +- TODO: Obey Retry-After in redirects - Out of the 86 labels not at the start of the line: - * 75 were indented with the same indentation level of the following line - * 8 were indented with exactly one space - * 2 were indented with one fewer indentation level then the following - line - * 1 was indented with the indentation level of the following line minus - three space (probably unintentional) + (remove "Set custom client ip when using haproxy protocol" which was + shipped in 8.2.0) - Co-Authored-By: Viktor Szakats + Mentioned-by: Yair Lenga + Closes #11447 - Closes #11134 +- RELEASE-NOTES: synced -Daniel Stenberg (18 May 2023) +Oliver Roberts (21 Jul 2023) -- cookie: update the comment on cookie length and size limits +- amissl: fix AmiSSL v5 detection - To refer to the proper cookie RFC and the upcoming RFC refresh. + Due to changes in the AmiSSL SDK, the detection needed adjusting. - Closes #11127 + Closes #11477 -- url: provide better error message when URLs fail to parse +Alois Klink (21 Jul 2023) - By providing the URL API error message into the error message. +- unittest/makefile: remove unneeded unit1621_LDADD - Ref: #11129 - Closes #11137 + The `unit1621_LDADD` variable has the exact same value as the `LDADD` + flag in `Makefile.am`, except without `@LDFLAGS@ @LIBCURL_LIBS@`. -- RELEASE-NOTES: synced + This was originally added by [98e6629][], but I can't see any reason + why it exists, so we should remove it to clean things up. - bumped to 8.1.1 + [98e6629]: https://github.com/curl/curl/commit/98e6629154044e4ab1ee7cff8351c7 + ebcb131e88 -Jon Rumsey (18 May 2023) + Closes #11494 -- os400: update chkstrings.c +- unittest/makefile: remove unneeded unit1394_LDADD - Compensate changes for recent changes to urldata.h to reclassify - STRING_AWS_SIGV4. + These custom `unit1394_LDADD` and similar automake overrides are no + longer neded. They were originally added by added by [8dac7be][] for + metalink support, but are no longer after [265b14d][] removed metalink. - Fixes #11132 - Closes #11133 + [8dac7be]: https://github.com/curl/curl/commit/8dac7be438512a8725d3c71e9139bd + fdcac1ed8c + [265b14d]: https://github.com/curl/curl/commit/265b14d6b37c4298bd5556fabcbc37 + d36f911693 -Version 8.1.0 (17 May 2023) + Closes #11494 -Daniel Stenberg (17 May 2023) +- cmake: add `libcurlu`/`libcurltool` for unit tests -- RELEASE-NOTES: synced + Add a `libcurlu`/`libcurltool` static library that is compiled only for + unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not + built by default, they're only built if unit tests are built. -- THANKS: contributors from the 8.1.0 release + These libraries allow us to compile every unit test with CMake. -- hostip: include easy_lock.h before using GLOBAL_INIT_IS_THREADSAFE + Closes #11446 - Since that header file is the only place that define can be defined. +Daniel Stenberg (21 Jul 2023) - Reported-by: Marc Deslauriers +- test979: test -u with redirect to (the same) absolute host - Follow-up to 13718030ad4b3209 + Verifies #11492 - Closes #11121 +- transfer: do not clear the credentials on redirect to absolute URL -Thomas Taylor (16 May 2023) + Makes test 979 work. Regression shipped in 8.2.0 from commit + dd4d1a26959f63a2c -- aws-sigv4.d: fix region identifier in example + Fixes #11486 + Reported-by: Cloudogu Siebels + Closes #11492 - Closes #11117 +Jon Rumsey (20 Jul 2023) -Philip Heiduck (15 May 2023) +- os400: correct EXPECTED_STRING_LASTZEROTERMINATED -- mlc_config.json: remove this linkcheck CI job config file + Correct EXPECTED_STRING_LASTZEROTERMINATED to account for + CURLOPT_HAPROXY_CLIENT_IP which requires EBCDIC to ASCII conversion when + passed into curl_easy_setopt(). - Closes #11113 + Closes #11476 -Daniel Silverstone (15 May 2023) +Oliver Roberts (20 Jul 2023) -- ssh: Add support for libssh2 read timeout +- amissl: add missing signal.h include - Hook the new (1.11.0 or newer) libssh2 support for setting a read timeout - into the SERVER_RESPONSE_TIMEOUT option. With this done, clients can use - the standard curl response timeout setting to also control the time that - libssh2 will wait for packets from a slow server. This is necessary to - enable use of very slow SFTP servers. + In some environments, signal.h is already included, but not in others + which cause compilation to fail, so explictly include it. - Signed-off-by: Daniel Silverstone + Closes #11478 - Closes #10965 +- amigaos: fix sys/mbuf.h m_len macro clash -Osama Albahrani (14 May 2023) + The updated Curl_http_req_make and Curl_http_req_make2 functions spawned + a parameter called m_len. The AmigaOS networking headers, derived from + NetBSD, contain "#define m_len m_hdr.mh_len" which clashes with + this. Since we do not actually use mbuf, force the include file to be + ignored, removing the clash. -- GIT-INFO: add --with-openssl + Closes #11479 - Closes #11110 +Daniel Stenberg (20 Jul 2023) -Daniel Stenberg (13 May 2023) +- socks: print ipv6 address within brackets -- RELEASE-NOTES: synced + Fixes #11483 + Closes #11484 -Marcel Raad (13 May 2023) +Christian Schmitz (20 Jul 2023) -- md(4|5): don't use deprecated iOS functions +- libcurl-errors.3: add CURLUE_OK - They are marked as deprecated in iOS 13.0, which might result in - warnings-as-errors. + Closes #11488 - Also, use `*_MIN_REQUIRED` instead of `*_MIN_ALLOWED`, which seems to - be what's currently used. +Oliver Roberts (20 Jul 2023) - Bug: https://github.com/curl/curl/issues/11098 - Closes https://github.com/curl/curl/pull/11102 +- cfilters: rename close/connect functions to avoid clashes -- md4: only build when used + Rename `close` and `connect` in `struct Curl_cftype` for + consistency and to avoid clashes with macros of the same name + (the standard AmigaOS networking connect() function is implemented + via a macro). - Its only usage in curl_ntlm_core.c is guarded by `USE_CURL_NTLM_CORE`, - so let's use this here too. + Closes #11491 - Ref: https://github.com/curl/curl/issues/11098 - Closes https://github.com/curl/curl/pull/11102 +Stefan Eissing (20 Jul 2023) -Vítor Galvão (12 May 2023) +- http2: fix regression on upload EOF handling -- write-out.d: Use response_code in example + - a regression introduced by c9ec85121110d7cbbbed2990024222c8f5b8afe5 + where optimization of small POST bodies leads to a new code path + for such uploads that did not trigger the "done sending" event + - add triggering this event for early "upload_done" situations - Closes #11107 + Fixes #11485 + Closes #11487 + Reported-by: Aleksander Mazur -Shohei Maeda (12 May 2023) +Daniel Stenberg (19 Jul 2023) -- url: fix null dispname for --connect-to option +- configure: check for nghttp2_session_get_stream_local_window_size - Closes #11106 + The http2 code uses it now. Introduced in nghttp2 1.15.0 (Sep 2016) -Daniel Stenberg (12 May 2023) + Fixes #11470 + Reported-by: Paul Howarth + Closes #11473 -- test2306: verify getting a second response with folded headers +Stefan Eissing (19 Jul 2023) - Reproduces the isue #11101 and verifies the fix. +- quiche: fix segfault and other things - Verifies a17b2a503f + - refs #11449 where a segfault is reported when IP Eyeballing did + not immediately connect but made several attempts + - The transfer initiating the eyeballing was initialized too early, + leadding to references to the filter instance that was then + replaced in the subsequent eyeball attempts. That led to a use + after free in the buffer handling for the transfer + - transfers are initiated now more lazy (like in the ngtcp2 filter), + when the stream is actually opened + - suppress reporting on quiche event errors for "other" transfers + than the current one to not fail a transfer due to faults in + another one. + - revert recent return value handling for quiche_h3_recv_body() + to not indicate an error but an EAGAIN situation. We wish quiche + would document what functions return. -- headers: clear (possibly) lingering pointer in init + Fixes #11449 + Closes #11469 + Reported-by: ウさん - The "prevhead" pointer is used for the headers storage but was not - cleared correctly in init, which made it possible to act up when a - handle is reused. +Daniel Stenberg (19 Jul 2023) - Reported-by: Steve Herrell - Fixes #11101 - Closes #11103 +- hostip: return IPv6 first for localhost resolves -- RELEASE-NOTES: synced + Fixes #11465 + Reported-by: Chilledheart on github + Closes #11466 -- ngtcp2: use 0.15.0 +Harry Sintonen (19 Jul 2023) - - nghttp3 0.11.0 - - nghttp2 1.53.0 +- tool: fix tool_seek_cb build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T - Adapt to new API calls + - a variable was renamed, and some use of it wasn't. this fixes the + build. - Closes #11031 + Closes #11468 -Jay Satiro (10 May 2023) +Stefan Eissing (19 Jul 2023) -- openssl: fix indent +- quiche: fix lookup of transfer at multi -Daniel Stenberg (10 May 2023) + - refs #11449 where weirdness in quiche multi connection tranfers was + observed + - fixes lookup of transfer for a quiche event to take the connection + into account + - formerly, a transfer with the same stream_id, but on another connection + could be found -- CURLOPT_DNS_CACHE_TIMEOUT.3: fix spelling + Closes #11462 - Follow-up to 9ed7d56e044f5aa1b29 +Daniel Stenberg (19 Jul 2023) - Closes #11096 +- RELEASE-NOTES: synced -- hostip: use time_t for storing oldest DNS entry + bump to 8.2.1 - Theoretically, the oldest time could overflow an int. In practice that - won't happen, but let's do this to please analyzers. +John Haugabook (19 Jul 2023) - Follow-up to 9ed7d56e044f5aa1b2928ccde6245d0 +- ciphers.d: put URL in first column - Pointed out by Coverity. - Closes #11094 + This makes the URL turn into a link properly when "webified". -- http: free the url before storing a new copy + Fixes https://github.com/curl/curl-www/issues/270 + Closes #11464 - To avoid a memory-leak. +Version 8.2.0 (19 Jul 2023) - Reported-by: Hiroki Kurosawa +Daniel Stenberg (19 Jul 2023) - Closes #11093 +- RELEASE-NOTES: synced -- compressed.d: clarify the words on "not notifying headers" + 8.2.0 release - Reported-by: Dylan Anthony - Fixes #11091 - Closes #11092 +- THANKS-filter: strip out "GitHub" -- libssh2: free fingerprint better +- THANKS: add contributors from 8.2.0 - Reported-by: Wei Chong Tan - Closes #11088 +- RELEASE-PROCEDURE.md: adjust the release dates -- CURLOPT_IPRESOLVE.3: clarify that this for host names, not IP addresses +Stefan Eissing (17 Jul 2023) - Reported-by: Harry Sintonen - Closes #11087 +- quiche: fix defects found in latest coverity report -- hostip: enforce a maximum DNS cache size independent of timeout value + Closes #11455 - To reduce the damage an application can cause if using -1 or other - ridiculous timeout values and letting the cache live long times. +Daniel Stenberg (17 Jul 2023) - The maximum number of entries in the DNS cache is now totally - arbitrarily and hard-coded set to 29999. +- quiche: avoid NULL deref in debug logging - Closes #11084 + Coverity reported "Dereference after null check" -- hostip: store dns timeout as 'int' + If stream is NULL and the function exits, the logging must not deref it. - ... because it set and held as an 'int' elsewhere and can never be - larger. + Closes #11454 -- RELEASE-NOTES: synced +Stefan Eissing (17 Jul 2023) -- tool_operate: refuse (--data or --form) and --continue-at combo +- http2: treat initial SETTINGS as a WINDOW_UPDATE - libcurl assumes that a --continue-at resumption is done to continue an - upload using the read callback and neither --data nor --form use - that and thus won't do what the user wants. Whatever the user wants - with this strange combination. + - refs #11426 where spurious stalls on large POST requests + are reported + - the issue seems to involve the following + * first stream on connection adds up to 64KB of POST + data, which is the max default HTTP/2 stream window size + transfer is set to HOLD + * initial SETTINGS from server arrive, enlarging the stream + window. But no WINDOW_UPDATE is received. + * curl stalls + - the fix un-HOLDs a stream on receiving SETTINGS, not + relying on a WINDOW_UPDATE from lazy servers - Add test 426 to verify. + Closes #11450 - Reported-by: Smackd0wn on github - Fixes #11081 - Closes #11083 +Daniel Stenberg (17 Jul 2023) -- transfer: refuse POSTFIELDS + RESUME_FROM combo +- ngtcp2: assigning timeout, but value is overwritten before used - The code assumes that such a resume is wanting to continue an upload - using the read callback, and since POSTFIELDS is done without callback - libcurl will just misbehave. + Reported by Coverity - This combo will make the transfer fail with CURLE_BAD_FUNCTION_ARGUMENT - with an explanation in the error message. + Closes #11453 - Reported-by: Smackd0wn on github - Fixes #11081 - Closes #11083 +- krb5: add typecast to please Coverity -- ipv4.d/ipv6.d: they are "mutex", not "boolean" +Derzsi Dániel (16 Jul 2023) - ... which for example means they do not have --no-* versions. +- wolfssl: support setting CA certificates as blob - Reported-by: Harry Sintonen - Fixes #11085 - Closes #11086 + Closes #11445 -- docs/SECURITY-ADVISORY.md: how to write a curl security advisory +- wolfssl: detect when TLS 1.2 support is not built into wolfssl - Closes #11080 + Closes #11444 -nobedee on github (5 May 2023) +Graham Campbell (15 Jul 2023) -- MANUAL.md: add dict example for looking up a single definition +- CI: bump nghttp2 from 1.55.0 to 1.55.1 - Closes #11077 + Closes #11442 -Dan Fandrich (5 May 2023) +Daniel Stenberg (15 Jul 2023) -- runtests: fix -c option when run with valgrind +- curl: return error when asked to use an unsupported HTTP version - The curl binary argument wasn't being quoted properly. This seems to - have broken at some point after quoting was added in commit 606b29fe. + When one of the following options are used but the libcurl in use does + not support it: - Reported-by: Daniel Stenberg - Ref: #11073 - Fixes #11074 - Closes #11076 + --http2 + --http2-prior-knowledge + --proxy-http2 -- runtests: support creating more than one runner process + Closes #11440 - The controller currently only creates and uses one, but more are now - possible. +Chris Paulson-Ellis (14 Jul 2023) - Ref: #10818 +- cf-socket: don't bypass fclosesocket callback if cancelled before connect -- runtests: spawn a new process for the test runner + After upgrading to 8.1.2 from 7.84.0, I found that sockets were being + closed without calling the fclosesocket callback if a request was + cancelled after the associated socket was created, but before the socket + was connected. This lead to an imbalance of fopensocket & fclosesocket + callbacks, causing problems with a custom event loop integration using + the multi-API. - When the -j option is given, a new process is spawned in which the test - programs are run and from which test servers are started. Only one - process can be started at once, but this is sufficient to test that the - infrastructure can isolate those functions in a new task. There should - be no visible difference between the two modes at the moment. + This was caused by cf_socket_close() calling sclose() directly instead + of calling socket_close() if the socket was not active. For regular TCP + client connections, the socket is activated by cf_socket_active(), which + is only called when the socket completes the connect. - Ref: #10818 - Closes #11064 + As far as I can tell, this issue has existed since 7.88.0. That is, + since the code in question was introduced by: + commit 71b7e0161032927cdfb4e75ea40f65b8898b3956 + Author: Stefan Eissing + Date: Fri Dec 30 09:14:55 2022 +0100 -- runtests: turn singletest() into a state machine + lib: connect/h2/h3 refactor - This allows it to run in a non-blocking manner. + Closes #11439 - Ref: #10818 +Daniel Stenberg (13 Jul 2023) -- runtests: change runner interface to be asynchronous +- tool_parsecfg: accept line lengths up to 10M - Program arguments are marshalled and then written to the end of a pipe - which is later read from and the arguments unmarshalled before the - desired function is called normally. The function return values are - then marshalled and written into another pipe when is later read from - and unmarshalled before being returned to the caller. + Bumped from 100K set in 47dd957daff9 - The implementation is currently blocking but can be made non-blocking - without any changes to the API. This allows calling multiple runners - without blocking in the future. + Reported-by: Antoine du Hamel + Fixes #11431 + Closes #11435 - Ref: #10818 +Stefan Eissing (13 Jul 2023) -- runtests: call citest_finishtest in singletest +- CI: brew fix for openssl in default path - This is where citest_starttest is called. + If brew install/update links openssl into /usr/local, it will be found + before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of + that by unlinking the keg. - Ref: #10818 + Fixes #11413 + Closes #11436 -- runtests: add a runner initialization function +Daniel Stenberg (13 Jul 2023) - This sets up the runner environment to start running tests. +- RELEASE-NOTES: synced - Ref: #10818 +Ondřej Koláček (13 Jul 2023) -- runtests: remove directory from server filename variables +- sectransp: fix EOF handling - There will soon be multiple log directories so the paths will no longer - be static in runtests.pl. Also, get rid of $SERVER2IN which was not - used. + Regression since the large refactor from 2022 - Ref: #10818 + Closes #11427 -- runtests: reduce package exports after refactoring +Daniel Stenberg (13 Jul 2023) - Some recent refactoring made these export no longer necessary. Also, - stop displaying the Unix socket paths at startup since there will soon - be many of them and they're not that interesting. +- checksrc: quote the file name to work with "funny" letters - Ref: #10818 + Closes #11437 -- runtests: use a function to obtain $LOGDIR for a test +Karthikdasari0423 (13 Jul 2023) - This will no longer be static soon. +- HTTP3.md: ngtcp2 updated to v0.17.0 and nghttp3 to v0.13.0 - Ref: #10818 + Follow-up to e0093b4b732f6 -Jay Satiro (5 May 2023) + Closes #11433 -- tool_cb_hdr: Fix 'Location:' formatting for early VTE terminals +Daniel Stenberg (13 Jul 2023) - - Disable hyperlink formatting for the 'Location:' header value in VTE - 0.48.1 and earlier, since it is buggy in some of those versions. +- CURLOPT_MIMEPOST.3: clarify what setting to NULL means - Prior to this change those terminals may show the location header value - as gibberish or show it twice. + Follow-up to e08382a208d4e480 - Ref: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#backw - ard-compatibility + Closes #11430 - Fixes https://github.com/curl/curl/issues/10428 - Closes https://github.com/curl/curl/pull/11071 +Tatsuhiro Tsujikawa (12 Jul 2023) -François Michel (3 May 2023) +- ngtcp2: build with 0.17.0 and nghttp3 0.13.0 -- quiche: disable pacing while pacing is not actually performed + - ngtcp2_crypto_openssl was renamed to ngtcp2_crypto_quictls. - Closes #11068 + Closes #11428 -Daniel Stenberg (2 May 2023) +- CI: Bump ngtcp2, nghttp3, and nghttp2 -- easy_cleanup: require a "good" handle to act + Closes #11428 - By insisting that the passed in handle is "good" (the magic number is - intact), this can limit the potential damage if a bad pointer is passed - in. Like when this function is called twice on the same handle pointer. +James Fuller (11 Jul 2023) - Ref: #10964 - Closes #11061 +- example/maxconnects: set maxconnect example -Andreas Falkenhahn (1 May 2023) + Closes #11343 -- amiga: Fix CA certificate paths for AmiSSL and MorphOS +Pontakorn Prasertsuk (11 Jul 2023) - AmiSSL stores certificates in `AmiSSL:Certs` and MorphOS stores them in - `MOSSYS:Data/SSL/curl-ca-bundle.crt`. +- http2: send HEADER & DATA together if possible - Closes https://github.com/curl/curl/pull/11059 + Closes #11420 -Daniel Stenberg (30 Apr 2023) +Daniel Stenberg (11 Jul 2023) -- http2: (void)-mark when we explicitly ignore the return code +- CI: use wolfSSL 5.6.3 in builds - When h2_progress_egress() is called. Pointed out by Coverity. + No using master anymore - Closes #11057 + Closes #11424 -- checksrc: find bad indentation in conditions without open brace +SaltyMilk (11 Jul 2023) - If the previous line starts with if/while/for AND ends with a closed - parenthesis and there's an equal number of open and closed parentheses - on that line, verify that this line is indented $indent more steps, if - not a cpp line. +- fopen: optimize - Also adjust the fall-out from this fix. + Closes #11419 - Closes #11054 +Daniel Stenberg (11 Jul 2023) -Diogo Teles Sant'Anna (28 Apr 2023) +- cmake: make use of snprintf -- CI: Set minimal permissions on workflow ngtcp2-quictls.yml + Follow-up to 935b1bd4544a23a91d68 - Signed-off-by: Diogo Teles Sant'Anna + Closes #11423 - Closes #11055 +Stefan Eissing (11 Jul 2023) -Dan Fandrich (28 Apr 2023) +- macOS: fix taget detection -- CI: use another glob syntax for matching files on Appveyor + - TARGET_OS_OSX is not always defined on macOS + - this leads to missing symbol Curl_macos_init() + - TargetConditionals.h seems to define these only when + dynamic targets are enabled (somewhere?) + - this PR fixes that on my macOS 13.4.1 + - I have no clue why CI builds worked without it - The previous globbing syntax was not matching files recursively in - directories, so try appending a /* to more closely match the examples at - https://www.appveyor.com/docs/how-to/filtering-commits/ + Follow-up to c7308592fb8ba213fc2c1 + Closes #11417 -Daniel Stenberg (28 Apr 2023) +Stan Hu (9 Jul 2023) -- multi: add multi-ignore logic to multi_socket_action +- hostip.c: Move macOS-specific calls into global init call - The multi-ignore logic that was previously applied to - curl_multi_perform() (#10750) is here applied to the loop within - curl_multi_socket_action() to make it use the same optimization: most - handles have the same signal-ignore option state so this drastically - reduces the number of ignore/unignore calls per libcurl function invoke. + https://github.com/curl/curl/pull/7121 introduced a macOS system call + to `SCDynamicStoreCopyProxies`, which is invoked every time an IP + address needs to be resolved. - Follow-up to bc90308328afb8 + However, this system call is not thread-safe, and macOS will kill the + process if the system call is run first in a fork. To make it possible + for the parent process to call this once and prevent the crash, only + invoke this system call in the global initialization routine. - Closes #11045 + In addition, this change is beneficial because it: -Stefan Eissing (28 Apr 2023) + 1. Avoids extra macOS system calls for every IP lookup. + 2. Consolidates macOS-specific initialization in a separate file. -- http2: do flow window accounting for cancelled streams + Fixes #11252 + Closes #11254 - - nghttp2 does not free connection level window flow for - aborted streams - - when closing transfers, make sure that any buffered - response data is "given back" to the flow control window - - add tests test_02_22 and test_02_23 to reproduce +Daniel Stenberg (9 Jul 2023) - Closes #11052 +- docs: use a space after RFC when spelling out RFC numbers -- pingpong: fix compiler warning "assigning an enum to unsigned char" + Closes #11382 - Closes #11050 +Margu (9 Jul 2023) -Daniel Stenberg (28 Apr 2023) +- imap-append.c: update to make it more likely to work -- configure: fix detection of apxs (for httpd) + Fixes #10300 + Closes #11397 - The condition check was turned the wrong way around! +Emanuele Torre (9 Jul 2023) - Closes #11051 +- tool_writeout_json: fix encoding of control characters -Viktor Szakats (28 Apr 2023) + Control characters without a special escape sequence e.g. %00 or %06 + were being encoded as "u0006" instead of "\u0006". -- ci: `-Wno-vla` no longer necessary + Ref: https://github.com/curl/trurl/pull/214#discussion_r1257487858 + Closes #11414 - We handle this issue in the source now. +Stefan Eissing (9 Jul 2023) - Follow-up to b725fe1944b45406676ea3aff333ae3085a848d9 +- http3/ngtcp2: upload EAGAIN handling - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Closes #11048 + - refs #11389 where IDLE timeouts on upload are reported + - reword ngtcp2 expiry handling to apply to both send+recv + calls into the filter + - EAGAIN uploads similar to the recent changes in HTTP/2, e.g. + report success only when send data was ACKed. + - HOLD sending of EAGAINed uploads to avoid cpu busy loops + - rename internal function for consistency with HTTP/2 + implementation -Marcel Raad (28 Apr 2023) + Fixes #11389 + Closes #11390 -- tests/http: make curl_setup.h the first include +Brian Nixon (9 Jul 2023) - This is required for the macros there to take effect for system - libraries. Specifically, including the system libraries first led to - warnings about `_FILE_OFFSET_BITS` being redefined in curl_config.h on - the Solaris autobuilds for ws-data.c and ws-pingpong.c. - Also make the curl includes come first for the other source files here - for consistency. +- tool_easysrc.h: correct `easysrc_perform` for `CURL_DISABLE_LIBCURL_OPTION` - Closes https://github.com/curl/curl/pull/11046 + Closes #11398 -Emanuele Torre (27 Apr 2023) +Daniel Stenberg (9 Jul 2023) -- checksrc: check for spaces before the colon of switch labels +- RELEASE-NOTES: synced - Closes #11047 +- transfer: clear credentials when redirecting to absolute URL -Daniel Stenberg (27 Apr 2023) + Make sure the user and password for the second request is taken from the + redirected-to URL. -- RELEASE-NOTES: synced + Add test case 899 to verify. -- libssh: tell it to use SFTP non-blocking + Reported-by: James Lucas + Fixes #11410 + Closes #11412 - Reported-by: Andreas Huebner - Fixes #11020 - Closes #11039 +Stefan Eissing (8 Jul 2023) -Stefan Eissing (27 Apr 2023) +- hyper: fix EOF handling on input -- http2: enlarge the connection window + We ran out of disc space due to an infinite loop with debug logging - - fixes stalled connections + Fixes #11377 + Closes #11385 + Reported-by: Dan Fandrich - - Make the connection window large enough, so that there is - some room left should 99/100 streams be PAUSED by the application +- http2: raise header limitations above and beyond - Reported-by: Paweł Wegner - Fixes #10988 - Closes #11043 + - not quite to infinity + - rewrote the implementation of our internal HTTP/1.x request + parsing to work with very large lines using dynbufs. + - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which + is also the limit of curl's general HTTP request processing. -Daniel Stenberg (27 Apr 2023) + Fixes #11405 + Closes #11407 -- checksrc: fix SPACEBEFOREPAREN for conditions starting with "*" +Juan Cruz Viotti (8 Jul 2023) - The open paren check wants to warn for spaces before open parenthesis - for if/while/for but also for any function call. In order to avoid - catching function pointer declarations, the logic allows a space if the - first character after the open parenthesis is an asterisk. +- curl_easy_nextheader.3: add missing open parenthesis examples - I also spotted what we did not include "switch" in the check but we should. + Closes #11409 + Signed-off-by: Juan Cruz Viotti - This check is a little lame, but we reduce this problem by not allowing - that space for if/while/for/switch. +Dan Fandrich (7 Jul 2023) - Reported-by: Emanuele Torre - Closes #11044 +- CI: enable verbose test output on pytest -- docs: minor polish + This shows individual pass/fail status on tests and makes this output + consistent with other jobs' pytest invocations. - - "an HTTP*" (not "a") - - remove a few contractions - - remove a spurious "a" - - reduce use of "I" in texts +Stefan Eissing (28 Jun 2023) - Closes #11040 +- http2: fix crash in handling stream weights -- ws: fix CONT opcode check + - Delay the priority handling until the stream has been opened. - Detected by Coverity. Follow-up to 930c00c259 + - Add test2404 to reproduce and verify. - Closes #11037 + Weights may change "on the run", which is why there are checks in + general egress handling. These must not trigger when the stream has not + been opened yet. -Dan Fandrich (27 Apr 2023) + Reported-by: jbgoog@users.noreply.github.com -- CI: switch the awslc builds to build out-of-tree + Fixes https://github.com/curl/curl/issues/11379 + Closes https://github.com/curl/curl/pull/11384 - This is a common configuration that should be tested to avoid - regressions. The awsls cmake build was already out-of-tree so the - automake build now joins it. +- tests/http: Add mod_h2 directive `H2ProxyRequests` - Ref: #11006 + master of mod_h2 now requires H2ProxyRequests directives for forward + proxying with HTTP/2 to work. -- tests/http: fix out-of-tree builds + Ref: https://github.com/icing/mod_h2/commit/3897a7086 - Add both lib/ directories (src & build) to the search path so - curl_setup.h and its dependencies can be found. + Closes https://github.com/curl/curl/pull/11392 - Followup-to acd82c8b +Dan Fandrich (28 Jun 2023) - Ref: #11006 - Closes #11036 +- CI: make Appveyor job names unique -Daniel Stenberg (27 Apr 2023) + Two otherwise identical mingw-w64 jobs now have their differing compiler + versions mentioned in their names. -- urlapi: make internal function start with Curl_ +Sheshadri.V (25 Jun 2023) - Curl_url_set_authority() it is. +- curl.h: include for vxworks - Follow-up to acd82c8bfd + Closes #11356 - Closes #11035 +Dan Fandrich (24 Jun 2023) -YX Hao (26 Apr 2023) +- CI: enable parallel make in more builds -- cf-socket: turn off IPV6_V6ONLY on Windows if it is supported + Most CI services provide at least two cores, so enable parallel make + jobs to take advantage of that for builds. Some dependencies aren't safe + to build in parallel so leave those as-is. Also, rename a few + workflows to eliminate duplicate names and provide a better idea what + they're about. - IPV6_V6ONLY refs: - https://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses - https://github.com/golang/go/blob/master/src/net/ipsock_posix.go - https://en.wikipedia.org/wiki/Unix-like - https://learn.microsoft.com/en-us/windows/win32/winsock/ipproto-ipv6-socket-o - ptions +- CI: don't install impacket if tests are not run - default value refs: - https://datatracker.ietf.org/doc/html/rfc3493#section-5.3 - https://www.kernel.org/doc/html/latest/networking/ip-sysctl.html#proc-sys-net - -ipv6-variables + It just wastes time and bandwidth and isn't even used. - Closes #10975 +divinity76 (24 Jun 2023) -Daniel Stenberg (26 Apr 2023) +- configure: the --without forms of the options are also gone -- urldata: shrink *select_bits int => unsigned char + --without-darwin-ssl and --without-metalink - - dselect_bits - - cselect_bits + Closes #11378 - ... are using less than 8 bits. Changed types and moved them towards - the end of the structs to fit better. +Daniel Stenberg (23 Jun 2023) - Closes #11025 +- configure: add check for ldap_init_fd -Stefan Eissing (26 Apr 2023) + ... as otherwise the configure script will say it is OpenLDAP in the + summary, but not set the USE_OPENLDAP define, therefor not using the + intended OpenLDAP code paths. -- tests/http: more tests with specific clients + Regression since 4d7385446 (7.85.0) + Fixes #11372 + Closes #11374 + Reported-by: vlkl-sap on github - - Makefile support for building test specific clients in tests/http/clients - - auto-make of clients when invoking pytest - - added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush - - added test_02_21 for lib based downloads and pausing/unpausing transfers +Michał Petryka (23 Jun 2023) - curl url parser: - - added internal method `curl_url_set_authority()` for setting the - authority part of a url (used for PUSH_PROMISE) +- cmake: stop CMake from quietly ignoring missing Brotli - http2: - - made logging of PUSH_PROMISE handling nicer + The CMake project was set to `QUIET` for Brotli instead of + `REQUIRED`. This makes builds unexpectedly ignore missing Brotli even + when `CURL_BROTLI` is enabled. - Placing python test requirements in requirements.txt files - - separate files to base test suite and http tests since use - and module lists differ - - using the files in the gh workflows + Closes #11376 - websocket test cases, fixes for we and bufq - - bufq: account for spare chunks in space calculation - - bufq: reset chunks that are skipped empty - - ws: correctly encode frames with 126 bytes payload - - ws: update frame meta information on first call of collect - callback that fills user buffer - - test client ws-data: some test/reporting improvements +Emanuele Torre (22 Jun 2023) - Closes #11006 +- docs: add more .IP after .RE to fix indentation of generate paragraphs -Jay Satiro (26 Apr 2023) + follow-up from 099f41e097c030077b8ec078f2c2d4038d31353b -- libssh2: fix crash in keyboard callback + I just thought of checking all the other files with .RE, and I found 6 + other files that were missing .IP at the end. - - Always set the libssh2 'abstract' user-pointer to the libcurl easy - handle associated with the ssh session, so it is always passed to the - ssh keyboard callback. + Closes #11375 - Prior to this change and since 8b5f100 (precedes curl 8.0.0), if libcurl - was built without CURL_DEBUG then it could crash during the ssh auth - phase due to a null dereference in the ssh keyboard callback. +Stefan Eissing (22 Jun 2023) - Reported-by: Andreas Falkenhahn +- http2: h2 and h2-PROXY connection alive check fixes - Fixes https://github.com/curl/curl/pull/11024 - Closes https://github.com/curl/curl/pull/11026 + - fix HTTP/2 check to not declare a connection dead when + the read attempt results in EAGAIN + - add H2-PROXY alive check as for HTTP/2 that was missing + and is needed + - add attach/detach around Curl_conn_is_alive() and remove + these in filter methods + - add checks for number of connections used in some test_10 + proxy tunneling tests -Daniel Stenberg (26 Apr 2023) + Closes #11368 -- docs: clarify that more backends have HTTPS proxy support +- http2: error stream resets with code CURLE_HTTP2_STREAM - Closes #11033 + - refs #11357, where it was reported that HTTP/1.1 downgrades + no longer works + - fixed with suggested change + - added test_05_03 and a new handler in the curltest module + to reproduce that downgrades work -- KNOWN_BUGS: remove two not-bugs + Fixes #11357 + Closes #11362 + Reported-by: Jay Satiro - - 11.7 signal-based resolver timeouts +Daniel Stenberg (22 Jun 2023) - Not considered a bug anymore but just implementation details. People - should avoid using timeouts with the synchronous name resolver. +- connect-timeout.d: mention that the DNS lookup is included - - 11.16 libcurl uses renames instead of locking for atomic operations + Closes #11370 - Not a bug, just a description of how it works +Emanuele Torre (22 Jun 2023) - Closes #11032 +- quote.d: fix indentation of generated paragraphs -Harry Sintonen (26 Apr 2023) + quote.d was missing a .IP at the end which caused the paragraphs + generated for See-also, Multi, and Example to not be indented correctly. -- hostip: add locks around use of global buffer for alarm() + I also remove a redundant "This option can be used multiple times.", and + replaced .IP "item" with .TP .B "item" to make more clear which lines + are part of the list of commands and which aren't. - When building with the sync name resolver and timeout ability we now - require thread-safety to be present to enable it. + Closes #11371 - Closes #11030 +Paul Wise (22 Jun 2023) -Daniel Stenberg (26 Apr 2023) +- checksrc: modernise perl file open -- curl_path: bring back support for SFTP path ending in /~ + Use regular variables and separate file open modes from filenames. - libcurl used to do a directory listing for this case (even though the - documentation says a URL needs to end in a slash for this), but - 4e2b52b5f7a3 modified the behavior. + Suggested by perlcritic - This change brings back a directory listing for SFTP paths that are - specified exactly as /~ in the URL. + Copied from https://github.com/curl/trurl/commit/f2784a9240f47ee28a845 - Reported-by: Pavel Mayorov - Fixes #11001 - Closes #11023 + Closes #11358 -Emanuele Torre (26 Apr 2023) +Dan Fandrich (21 Jun 2023) -- docs/libcurl/curl_*escape.3: rename "url" argument to "input"/"string" +- runtests: work around a perl without SIGUSR1 - Also reword the DESCRIPTION section to mention "input"/"string" argument - in bold. + At least msys2 perl v5.32.1 doesn't seem to define this signal. Since + this signal is only used for debugging, just ignore if setting it fails. - Closes #11027 + Reported-by: Marcel Raad + Fixes #11350 + Closes #11366 -- docs/libcurl: minor cleanups +- runtests: include missing valgrind package - I was reading curl_unescape(3) and I noticed that there was an extra - space after the open parenthesis in the SYNOPSIS; I removed the extra - space. + use valgrind was missing which caused torture tests with valgrind + enabled to fail. - I also ran a few grep -r commands to find and remove extra spaces - after '(' in other files, and to find and replace uses of `T*' instead - of `T *'. Some of the instances of `T*` where unnecessary casts that I - removed. + Reported-by: Daniel Stenberg + Fixes #11364 + Closes #11365 - I also fixed a comment that was misaligned in CURLMOPT_SOCKETFUNCTION.3. +- runtests: use more consistent failure lines - And I fixed some formatting inconsistencies: in curl_unescape(3), all - function parameter were mentioned with bold text except length, that was - mentioned as 'length'; and, in curl_easy_unescape(3), all parameters - were mentioned in bold text except url that was italicised. Now they are - all mentioned in bold. - Documentation is not very consistent in how function parameter are - formatted: many pages italicise them, and others display them in bold - text; but I think it makes sense to at least be consistent with - formatting within the same page. + After a test failure log a consistent log message to make it easier to + parse the log file. Also, log a consistent message with "ignored" for + failures that cause the test to be not considered at all. These should + perhaps be counted in the skipped category, but this commit does not + change that behaviour. - Closes #11027 +- runtests: consistently write the test check summary block -Daniel Stenberg (26 Apr 2023) + The memory check character was erroneously omitted if the memory + checking file was not available for some reason, making the block of + characters an inconsistent length. -- man pages: simplify the .TH sections +- test2600: fix the description - - remove the version numbers - - simplify the texts + It looks like it was cut-and-pasted. - The date and version number will be put there for releases when maketgz - runs the updatemanpages.pl script. + Closes #11354 - Closes #11029 +Daniel Stenberg (21 Jun 2023) -- hostcheck: fix host name wildcard checking +- TODO: "Support HTTP/2 for HTTP(S) proxies" *done* - The leftmost "label" of the host name can now only match against single - '*'. Like the browsers have worked for a long time. +humbleacolyte (21 Jun 2023) - - extended unit test 1397 for this - - move some SOURCE variables from unit/Makefile.am to unit/Makefile.inc +- cf-socket: move ctx declaration under HAVE_GETPEERNAME - Reported-by: Hiroki Kurosawa - Closes #11018 + Closes #11352 -Dan Fandrich (25 Apr 2023) +Daniel Stenberg (20 Jun 2023) -- smbserver: remove temporary files before exit +- RELEASE-NOTES: synced - Each execution of test 1451 would leave a file in /tmp before. Since - Windows can't delete a file while it's open, all the temporary file - names are stored and deleted on exit. +- example/connect-to: show CURLOPT_CONNECT_TO - Closes #10990 + Closes #11340 -Stefan Eissing (25 Apr 2023) +Stefan Eissing (20 Jun 2023) -- Websocket en-/decoding +- hyper: unslow - - state is fully kept at connection, since curl_ws_send() and - curl_ws_rec() have lifetime beyond usual transfers - - no more limit on frame sizes + - refs #11203 where hyper was reported as being slow + - fixes hyper_executor_poll to loop until it is out of + tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issue + s/3237 + - added a fix in hyper io handling for detecting EAGAIN + - added some debug logs to see IO results + - pytest http/1.1 test cases pass + - pytest h2 test cases fail on connection reuse. HTTP/2 + connection reuse does not seem to work. Hyper submits + a request on a reused connection, curl's IO works and + thereafter hyper declares `Hyper: [1] operation was canceled: connection cl + osed` + on stderr without any error being logged before. - Reported-by: simplerobot on github - Fixes #10962 - Closes #10999 + Fixes #11203 + Reported-by: Gisle Vanem + Advised-by: Sean McArthur + Closes #11344 -Patrick Monnerat (25 Apr 2023) +- HTTP/2: upload handling fixes -- urldata: copy CURLOPT_AWS_SIGV4 value on handle duplication + - fixes #11242 where 100% CPU on uploads was reported + - fixes possible stalls on last part of a request body when + that information could not be fully send on the connection + due to an EAGAIN + - applies the same EGAIN handling to HTTP/2 proxying - Prior to this change STRING_AWS_SIGV4 (CURLOPT_AWS_SIGV4) was wrongly - marked as binary data that could not be duplicated. + Reported-by: Sergey Alirzaev + Fixed #11242 + Closes #11342 - Without this fix, this option's value is not copied upon calling - curl_easy_duphandle(). +Daniel Stenberg (20 Jun 2023) - Closes https://github.com/curl/curl/pull/11021 +- example/opensslthreadlock: remove -Stefan Eissing (25 Apr 2023) + This shows how to setup OpenSSL mutex callbacks, but this is not + necessary since OpenSSL 1.1.0 - meaning that no currently supported + OpenSSL version requires this anymore -- http3: expire unpaused transfers in all HTTP/3 backends + Closes #11341 - Closes #11005 +Dan Fandrich (19 Jun 2023) -- http2: always EXPIRE_RUN_NOW unpaused http/2 transfers +- libtest: display the times after a test timeout error - - just increasing the http/2 flow window does not necessarily - make a server send new data. It may already have exhausted - the window before + This is to help with test failure debugging. - Closes #11005 + Ref: #11328 + Closes #11329 -- http2: pass `stream` to http2_handle_stream_close to avoid NULL checks +- test2600: bump a test timeout - Closes #11005 + Case 1 failed at least once on GHA by going 30 msec too long. -- h2/h3: replace `state.drain` counter with `state.dselect_bits` + Ref: #11328 - - `drain` was used by http/2 and http/3 implementations to indicate - that the transfer requires send/recv independant from its socket - poll state. Intended as a counter, it was used as bool flag only. - - a similar mechanism exists on `connectdata->cselect_bits` where - specific protocols can indicate something similar, only for the - whole connection. - - `cselect_bits` are cleard in transfer.c on use and, importantly, - also set when the transfer loop expended its `maxloops` tries. - `drain` was not cleared by transfer and the http2/3 implementations - had to take care of that. - - `dselect_bits` is cleared *and* set by the transfer loop. http2/3 - does no longer clear it, only set when new events happen. +- runtests: better detect and handle pipe errors in the controller - This change unifies the handling of socket poll overrides, extending - `cselect_bits` by a easy handle specific value and a common treatment in - transfers. + Errors reading and writing to the pipes are now better detected and + propagated up to the main test loop so it can be cleanly shut down. Such + errors are usually due to a runner dying so it doesn't make much sense + to try to continue the test run. - Closes #11005 +- runtests: cleanly abort the runner if the controller dies -Daniel Stenberg (25 Apr 2023) + If the controller dies unexpectedly, have the runner stop its servers + and exit cleanly. Otherwise, the orphaned servers will stay running in + the background. -- socketpair: verify with a random value +- runtests: improve error logging - ... instead of using the curl time struct, since it would use a few - uninitialized bytes and the sanitizers would complain. This is a neater - approach I think. + Give more information about test harness error conditions to help figure + out what might be wrong. Print some internal test state when SIGUSR1 is + sent to runtests.pl. - Reported-by: Boris Kuschel - Fixes #10993 - Closes #11015 + Ref: #11328 -Stefan Eissing (25 Apr 2023) +- runtests: better handle ^C during slow tests -- HTTP3: document the ngtcp2/nghttp3 versions to use for building curl + Since the SIGINT handler now just sets a flag that must be checked in the + main controller loop, make sure that runs periodically. Rather than + blocking on a response from a test runner near the end of the test run, + add a short timeout to allow it. - - refs #11011 to clarify this for people building curl themselves +- runtests: rename server command file - Closes #11019 + The name ftpserver.cmd was historical and has been used for more than + ftp for many years now. Rename it to plain server.cmd to reduce + confusion. -Daniel Stenberg (25 Apr 2023) +- tests: improve reliability of TFTP tests -- lib: unify the upload/method handling + Stop checking the timeout used by the client under test (for most + tests). The timeout will change if the TFTP test server is slow (such as + happens on an overprovisioned CI server) because the client will retry + and reduce its timeout, and the actual value is not important for most + tests. - By making sure we set state.upload based on the set.method value and not - independently as set.upload, we reduce confusion and mixup risks, both - internally and externally. - - Closes #11017 - -- RELEASE-NOTES: synced + test285 is changed a different way, by increasing the connect timeout. + This improves test coverage by allowing the changed timeout value to be + checked, but improves reliability with a carefully-chosen timeout that + not only allows twice the time to respond as before, but also allows + several retries before the client will change its timeout value. -Dan Fandrich (24 Apr 2023) + Ref: #11328 -- CI: don't run CI jobs if only another CI was changed +Daniel Stenberg (19 Jun 2023) - A few paths were missed in the last commit, as well as a job added since - then. +- cf-socket: skip getpeername()/getsockname for TFTP - Followup-to 395b9175 + Since the socket is not connected then the call fails. When the call + fails, failf() is called to write an error message that is then + surviving and is returned when the *real* error occurs later. The + earlier, incorrect, error therefore hides the actual error message. -- CI: adjust labeler match patterns + This could be seen in stderr for test 1007 -- runtests: support buffering log messages in runner & servers + Test 1007 has now been extended to verify the stderr message. - Log messages generated with logmsg can now be buffered and returned from - the runner as a return value. This will be needed with parallel testing - to allow all messages for one test to be displayed together instead of - interspersed with messages of multiple tests. Buffering can be disabled - by setting a logging callback function with setlogfunc, which is - currently being done to preserve existing logging behaviour for now. + Closes #11332 - Some additional output is generated in verbose and debugprotocol modes, - which don't always use logmsg. These modes also impact some servers - which generate extra messages. No attempt is made to buffer everything - if these modes are enabled. +- example/crawler: make it use a few more options - Ref: #10818 - Closes #11016 + For show, but reasonable -- runtests: more consistently use logmsg in server control code +- libcurl-ws.3: mention raw mode - Also, display an error when sshversioninfo returns one. + Closes #11339 - Ref: #10818 +- example/default-scheme: set the default scheme for schemeless URLs -- runtests: create runner functions for clearlocks and stopservers + Closes #11338 - runtests.pl now uses runner for all server actions beyond the initial - variable configuration. +- example/hsts-preload: show one way to HSTS preload - Ref: #10818 + Closes #11337 -- runtests: tightened servers package exports +- examples/http-options: show how to send "OPTIONS *" - The defaults are intended for runtests.pl, whereas runner.pm needs to - explicitly specify them. + With CURLOPT_REQUEST_TARGET. -- runtests: display logs on server failure in singletest() + Also add use of CURLOPT_QUICK_EXIT to show. - This is closer to the place where logs are displayed on test failure. - Also, only display these logs if -p is given, which is the same flag - that controls display of test failure logs. Some server log files - need to be deleted later so that they stay around long enough to be - displayed on failure. + Closes #11333 - Ref: #10818 +- examples: make use of CURLOPT_(REDIR_|)PROTOCOLS_STR -- runtests: turn a print into a logmsg + To show how to use them - Also enable another couple of useful messages in verbose mode. + Closes #11334 - Ref: #10818 +- examples/smtp-mime: use CURLOPT_MAIL_RCPT_ALLOWFAILS -Daniel Stenberg (24 Apr 2023) + For show -- http: store the password in the correct variable + Closes #11335 - Typo from fc2f1e547a4a, detected by Coverity (because there's dead code - due to this). +- http: rectify the outgoing Cookie: header field size check - Closes #11002 + Previously it would count the size of the entire outgoing request and + not just the size of only the Cookie: header field - which was the + intention. -Stefan Eissing (24 Apr 2023) + This could make the check be off by several hundred bytes in some cases. -- HTTP3/quiche: terminate h1 response header when no body is sent + Closes #11331 - - fixes a failure in test2501 where a response without body was missing - the final empty line +Jay Satiro (17 Jun 2023) - Closes #11003 +- lib: fix some format specifiers -Dan Fandrich (22 Apr 2023) + - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some + curl_off_t variables. -- runtests: move showdiff into runtests.pl + - Use %zu where %zd was erroneously used for some size_t variables. - It's not used anywhere else. + Prior to this change some of the Windows CI tests were failing because + in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t. + When %zd was used for some curl_off_t variables then only the lower + 32-bits was read and the upper 32-bits would be read for part or all of + the next specifier. -- devtest: add a new script for testing the test harness + Fixes https://github.com/curl/curl/issues/11327 + Closes https://github.com/curl/curl/pull/11321 - This is currently useful for starting a test server on its own without - an associated test, which can be used for interactive curl testing or - for validating parts of the test harness itself. More commands can be - added to perform additional functions in the future. +Marcel Raad (16 Jun 2023) - Ref: #10818 - Closes #11008 +- test427: add `cookies` feature and keyword -- runtests: refactor the main test loop into two + This test doesn't work with `--disable-cookies`. - The test loop now has an initial loop that first runs through all - possible tests to build a set of those to attempt on this run based on - features and keywords and only then goes through that new list to run - them. This actually makes it three loops through all tests cases, as - there is an existing loop that gathers possible test numbers from the - test files on disk. + Closes https://github.com/curl/curl/pull/11320 - This has two minor effects on the output: all the tests that will be - skipped are displayed at the start (instead of being interspersed with - other tests) and the -l option no longer shows a count of tests at the - end or a (misleading) statement that tests have run successfully. The - skipped tests are also omitted from the test results sent to AppVeyor - and Azure in CI builds. +Chris Talbot (15 Jun 2023) - Another effect is a reduction in the amount of work considered part of - the "Test definition reading and preparation time" reported with -r - making those figures slightly lower than before. +- imap: Provide method to disable SASL if it is advertised - Ref: #10818 + - Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext + LOGIN over SASL auth. -- runtests: track only the current test timings in runner.pm + Prior to this change there was no method to be able to fall back to + LOGIN if an IMAP server advertises SASL capabilities. However, this may + be desirable for e.g. a misconfigured server. - This avoids passing these data through through global variables, which - soon won't be possible. + Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2 - Ref: #10818 + ";AUTH=" looks to be the correct way to specify what + authenication method to use, regardless of SASL or not. -- runtests: skip test preprocessing when doing -l + Closes https://github.com/curl/curl/pull/10041 - This speeds up the output tremendously by avoiding unnecessary work. +Daniel Stenberg (15 Jun 2023) -- runtests: simplify value returned regarding use of valgrind +- RELEASE-NOTES: synced - As a side effect this will now also show in verbose mode that valgrind - is being skipped on tests that explicitly disable it, such as 600. +- examples/multi-debugcallback.c: avoid the bool typedef - Ref: #10818 + Apparently this cannot be done in c23 -- runtests: fix quoting in Appveyor and Azure test integration + Reported-by: Cristian Rodríguez + Fixes #11299 + Closes #11319 - Test 1442's name was not quoted correctly so wasn't registered in - Appveyor and it had the wrong name in Azure. The JSON string quotes were - also invalid, even though both servers happened to accept it regardless. +- docs/libcurl/libcurl.3: cleanups and improvements - Closes #11010 + Closes #11317 -Daniel Stenberg (19 Apr 2023) +- libcurl-ws.3: fix typo -- RELEASE-NOTES: synced +- curl_ws_*.3: enhance -Dan Fandrich (18 Apr 2023) + - all: SEE ALSO the libcurl-ws man page + - send: add example and return value information + - meta: mention that the returned data is read-only -- runtests: spread out the port numbers used by servers + Closes #11318 - The server ports are chosen randomly for each server, but the random - ranges chosen were inconsistently-sized and overlapping. Now, they are - spread out more so at least the first random port chosen for each server - is guaranteed to not also be chosen by another server. The starting port - numbers are also raised to put them in the Ephemeral Port range—not the - range defined by RFC 6335 but the one used by Linux, which starts lower - and gives us more room to work with. +- docs/libcurl/libcurl-ws.3: see also CURLOPT_WS_OPTIONS - Reported-by: Daniel Stenberg +- docs/libcurl/libcurl-ws.3: minor polish -- runtests: fix problems on failure +- libcurl-ws.3. WebSocket API overview - The verify time must be set in this case, like all cases. An error - message needs to be displayed as well. + Closes #11314 -- runtests: fix perl warning when is wrong +- libcurl-url.3: also mention CURLUPART_ZONEID -- runtests: don't try to stop stunnel before trying again + ... and sort the two part-using lists alphabetically - Calling stopserver() before retrying stunnel due to an error would stop - the dependent server (such as HTTP) meaning stunnel would have nothing - to talk to when it came up. Don't try to force a stop when it didn't - actually start. Also, don't mark the server as bad for future use when - it starts up on a retry. +Marcel Raad (14 Jun 2023) - Reported-by: eaglegai at github - Tested-by: eaglegai at github - Fixes #10976 +- fopen: fix conversion warning on 32-bit Android -- runtests: don't accidentally randomly choose the same port + When building for 32-bit ARM or x86 Android, `st_mode` is defined as + `unsigned int` instead of `mode_t`, resulting in a + -Wimplicit-int-conversion clang warning because `mode_t` is + `unsigned short`. Add a cast to silence the warning. - If a server couldn't be started on a port, a new one is randomly chosen - and the server is tried again. Avoid accidentally using a - randomly-chosen 0 port offset by adding 1 to the random number. + Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li + bc/include/sys/stat.h#86 + Closes https://github.com/curl/curl/pull/11313 - Found-by: Daniel Stenberg +- http2: fix variable type -- runtests: don't attempt to use a port we know is in use + `max_recv_speed` is `curl_off_t`, so using `size_t` might result in + -Wconversion GCC warnings for 32-bit `size_t`. Visible in the NetBSD + ARM autobuilds. - This reduces the startup time when there is a known conflict on the - random port chosen for a server. This was already done for stunnel, but - now it's done for all servers. + Closes https://github.com/curl/curl/pull/11312 -- http-server: fix server name in a log message +Daniel Stenberg (13 Jun 2023) - This changed when the file was renamed in commit cbf57176 +- vtls: fix potentially uninitialized local variable warnings -- runtests: refactor into more packages + Follow-up from a4a5e438ae533c - testutil.pm now contains a few miscellaneous functions that are used in - several places but have no better place to live. subvariables moves to - servers.pm since most variables that it substitutes relate to servers, - so this is the most appropriate place. Rename a few functions for better - naming consistency. + Closes #11310 - Ref: #10818 - Closes #10995 +- timeval: use CLOCK_MONOTONIC_RAW if available -- runtests: call timestampskippedevents() in singletest + Reported-by: Harry Sintonen + Ref: #11288 + Closes #11291 - ..rather than by the runner +Stefan Eissing (12 Jun 2023) -- runtests: assume a newer Valgrind by default +- tool: add curl command line option `--trace-ids` - The tests for an older Valgrind version should probably just be deleted, - given that they're testing for an 18-year-old version. + - added and documented --trace-ids to prepend (after the timestamp) + the transfer and connection identifiers to each verbose log line + - format is [n-m] with `n` being the transfer id and `m` being the + connection id. In case there is not valid connection id, print 'x'. + - Log calls with a handle that has no transfer id yet, are written + without any ids. + + Closes #11185 + +- lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID + + - add an `id` long to Curl_easy, -1 on init + - once added to a multi (or its own multi), it gets + a non-negative number assigned by the connection cache + - `id` is unique among all transfers using the same + cache until reaching LONG_MAX where it will wrap + around. So, not unique eternally. + - CURLINFO_CONN_ID returns the connection id attached to + data or, if none present, data->state.lastconnect_id + - variables and type declared in tool for write out + + Closes #11185 + +Daniel Stenberg (12 Jun 2023) + +- CURLOPT_INFILESIZE.3: mention -1 triggers chunked + + Ref: #11300 + Closes #11304 + +Philip Heiduck (12 Jun 2023) + +- CI: openssl-3.0.9+quic + + Closes #11296 + +Karthikdasari0423 (12 Jun 2023) + +- HTTP3.md: update openssl version + + Closes #11297 + +Daniel Stenberg (12 Jun 2023) + +- vtls: avoid memory leak if sha256 call fails + + ... in the pinned public key handling function. + + Reported-by: lizhuang0630 on github + Fixes #11306 + Closes #11307 + +- examples/ipv6: disable on win32 + + I can't make if_nametoindex() work there + + Follow-up to c23dc42f3997acf23 + + Closes #11305 + +- tool_operate: allow cookie lines up to 8200 bytes + + Since this option might set multiple cookies in the same line, it does + not make total sense to cap this at 4096 bytes, which is the limit for a + single cookie name or value. + + Closes #11303 + +- test427: verify sending more cookies than fit in a 8190 bytes line + + curl will then only populate the header with cookies that fit, dropping + ones that otherwise would have been sent + + Ref: https://curl.se/mail/lib-2023-06/0020.html + + Closes #11303 + +- testutil: allow multiple %-operators on the same line + + Closes #11303 + +Oleg Jukovec (12 Jun 2023) + +- docs: update CURLOPT_UPLOAD.3 + + The behavior of CURLOPT_UPLOAD differs from what is described in the + documentation. The option automatically adds the 'Transfer-Encoding: + chunked' header if the upload size is unknown. + + Closes #11300 + +Daniel Stenberg (12 Jun 2023) + +- RELEASE-NOTES: synced + +- CURLOPT_AWS_SIGV4.3: remove unused variable from example + + Closes #11302 + +- examples/https.c: use CURLOPT_CA_CACHE_TIMEOUT + + for demonstration purposes + + Closes #11290 + +- example/ipv6: feature CURLOPT_ADDRESS_SCOPE in use + + Closes #11282 + +Karthikdasari0423 (10 Jun 2023) + +- docs: Update HTTP3.md for newer ngtcp2 and nghttp3 + + Follow-up to fb9b9b58 + + Ref: #11184 + Closes #11295 + +Dan Fandrich (10 Jun 2023) + +- docs: update the supported ngtcp2 and nghttp3 versions + + Follow-up to cae9d10b + + Ref: #11184 + Closes #11294 + +- tests: fix error messages & handling around sockets + + The wrong error code was checked on Windows on UNIX socket failures, + which could have caused all UNIX sockets to be reported as having + errored and the tests therefore skipped. Also, a useless error message + was displayed on socket errors in many test servers on Windows because + strerror() doesn't work on WinSock error codes; perror() is overridden + there to work on all errors and is used instead. + + Ref #11258 + Closes #11265 + +Daniel Stenberg (9 Jun 2023) + +- CURLOPT_SSH_PRIVATE_KEYFILE.3: expand on the file search + + Reported-by: atjg on github + Ref: #11287 + Closes #11289 + +Stefan Eissing (9 Jun 2023) + +- ngtcp2: use ever increasing timestamp in io + + - ngtcp2 v0.16.0 asserts that timestamps passed to its function + will only ever increase. + - Use a context shared between ingress/egress operations that + uses a shared timestamp, regularly updated during calls. + + Closes #11288 + +Daniel Stenberg (9 Jun 2023) + +- GHA: use nghttp2 1.54.0 for the ngtcp2 jobs + +Philip Heiduck (9 Jun 2023) + +- GHA: ngtcp2: use 0.16.0 and nghttp3 0.12.0 + +Daniel Stenberg (9 Jun 2023) + +- ngtcp2: build with 0.16.0 and nghttp3 0.12.0 + + - moved to qlog_write + - crypto => encryption + - CRYPTO => ENCRYPTION + - removed "_is_" + - ngtcp2_conn_shutdown_stream_read and + ngtcp2_conn_shutdown_stream_write got flag arguments + - the nghttp3_callbacks struct got a recv_settings callback + + Closes #11184 + +- example/http2-download: set CURLOPT_BUFFERSIZE + + Primarily because no other example sets it, and remove the disabling of + the certificate check because we should not recommend that. + + Closes #11284 + +- example/crawler: also set CURLOPT_AUTOREFERER + + Could make sense, and it was not used in any example before. + + Closes #11283 + +Wyatt OʼDay (9 Jun 2023) + +- tls13-ciphers.d: include Schannel + + Closes #11271 + +Daniel Stenberg (9 Jun 2023) + +- curl_pushheader_byname/bynum.3: document in their own man pages + + These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was + introduced but always lived a life in the shadows, embedded in the + CURLMOPT_PUSHFUNCTION man page. Until now. + + It makes better sense and gives more visibility to document them in + their own stand-alone man pages. + + Closes #11286 + +- curl_mprintf.3: minor fix of the example + +- curl_url_set: enforce the max string length check for all parts + + Update the docs and test 1559 accordingly + + Closes #11273 + +- examples/ftpuploadresume.c: add use of CURLOPT_ACCEPTTIMEOUT_MS + + For show + + Closes #11277 + +- examples/unixsocket.c: example using CURLOPT_UNIX_SOCKET_PATH + + and alternatively CURLOPT_ABSTRACT_UNIX_SOCKET + + Closes #11276 + +Anssi Kolehmainen (8 Jun 2023) + +- docs: fix missing parameter names in examples + + Closes #11278 + +Daniel Stenberg (8 Jun 2023) + +- urlapi: have *set(PATH) prepend a slash if one is missing + + Previously the code would just do that for the path when extracting the + full URL, which made a subsequent curl_url_get() of the path to + (unexpectedly) still return it without the leading path. + + Amend lib1560 to verify this. Clarify the curl_url_set() docs about it. + + Bug: https://curl.se/mail/lib-2023-06/0015.html + Closes #11272 + Reported-by: Pedro Henrique + +Dan Fandrich (7 Jun 2023) + +- runtests; give each server a unique log lock file + + Logs are written by several servers and all of them must be finished + writing before the test results can be determined. This means each + server must have its own lock file rather than sharing a single one, + which is how it was done up to now. Previously, the first server to + complete a test would clear the lock before the other server was done, + which caused flaky tests. + + Lock files are now all found in their own directory, so counting locks + equals counting the files in that directory. The result is that the + proxy logs are now reliably written which actually changes the expected + output for two tests. + + Fixes #11231 + Closes #11259 + +- runtests: make test file directories in log/N + + Test files in subdirectories were not created after parallel test log + directories were moved down a level due to a now-bad comparison. + + Follow-up to 92d7dd39 + + Ref #11264 + Closes #11267 + +Daniel Stenberg (7 Jun 2023) + +- ws: make the curl_ws_meta() return pointer a const + + The returned info is read-only for the user. + + Closes #11261 + +- RELEASE-NOTES: synced + +- runtests: move parallel log dirs from logN to log/N + + Having several hundreds of them in there gets annoying. + + Closes #11264 + +Dan Fandrich (7 Jun 2023) + +- test447: move the test file into %LOGDIR + +Viktor Szakats (7 Jun 2023) + +- cmake: add support for "unity" builds + + Aka "jumbo" or "amalgamation" builds. It means to compile all sources + per target as a single C source. This is experimental. + + You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. + It requires CMake 3.16 or newer. + + It makes builds (much) faster, allows for better optimizations and tends + to promote less ambiguous code. + + Also add a new AppVeyor CI job and convert an existing one to use + "unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job. + + Fix related issues: + - add missing include guard to `easy_lock.h`. + - rename static variables and functions (and a macro) with names reused + across sources, or shadowed by local variables. + - add an `#undef` after use. + - add a missing `#undef` before use. + - move internal definitions from `ftp.h` to `ftp.c`. + - `curl_memory.h` fixes to make it work when included repeatedly. + - stop building/linking curlx bits twice for a static-mode curl tool. + These caused doubly defined symbols in unity builds. + - silence missing extern declarations compiler warning for ` _CRT_glob`. + - fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`. + - fix colliding static symbols in debug mode: `debugtime()` and + `statename`. + - rename `ssl_backend_data` structure to unique names for each + TLS-backend, along with the `ssl_connect_data` struct member + referencing them. This required adding casts for each access. + - add workaround for missing `[P]UNICODE_STRING` types in certain Windows + builds when compiling `lib/ldap.c`. To support "unity" builds, we had + to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows + `schannel.h` option) _globally_. This caused an indirect inclusion of + Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled + as well. This requires `[P]UNICODE_STRING` types, which is apperantly + not defined automatically (as seen with both MSVS and mingw-w64). + This patch includes `` to fix it. + Ref: https://github.com/curl/curl/runs/13987772013 + Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&vie + w=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f + 38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c + - tweak unity builds to compile `lib/memdebug.c` separately in memory + trace builds to avoid PP confusion. + - force-disable unity for test programs. + - do not compile and link libcurl sources to libtests _twice_ when libcurl + is built in static mode. + + KNOWN ISSUES: + - running tests with unity builds may fail in cases. + - some build configurations/env may not compile in unity mode. E.g.: + https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfu + auwl8q#L250 -- runtests: refactor test runner code into runner.pm + Ref: https://github.com/libssh2/libssh2/issues/1034 + Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html + Ref: https://en.wikipedia.org/wiki/Unity_build - This is code that is directly responsible for running a single test. - This will eventually run in a separate process as part of the parallel - testing project. + Closes #11095 + +Daniel Stenberg (7 Jun 2023) + +- examples/websocket.c: websocket example using CONNECT_ONLY + Closes #11262 + +- websocket-cb: example doing WebSocket download using callback + + Very basic + + Closes #11260 + +- test/.gitignore: ignore log* + +Dan Fandrich (5 Jun 2023) + +- runtests: document the -j parallel testing option + + Reported-by: Daniel Stenberg Ref: #10818 + Closes #11255 -- runtests: skip unneeded work if test won't be running +- runtests: create multiple test runners when requested - This speeds up tests by avoiding unnecessary processing. + Parallel testing is enabled by using a nonzero value for the -j option + to runtests.pl. Performant values seem to be about 7*num CPU cores, or + 1.3*num CPU cores if Valgrind is in use. + + Flaky tests due to improper log locking (bug #11231) are exacerbated + while parallel testing, so it is not enabled by default yet. + + Fixes #10818 + Closes #11246 + +- runtests: handle repeating tests in multiprocess mode + + Such as what happens with the --repeat option. Some functions are + changed to pass the runner ID instead of relying on the non-unique test + number. Ref: #10818 -- runtests: factor out singletest_postcheck +- runtests: buffer logmsg while running singletest() - This will eventually need to be part of the test runner. + This allows all messages relating to a single test case to be displayed + together at the end of the test. Ref: #10818 -- test303: kill server after test +- runtests: call initserverconfig() in the runner - Otherwise, an HTTP test closely following this one with a tight time - constraint (e.g. 672) could fail because the test server stays sitting - with the wait command for a while. + This must be done so variables pick up the runner's unique $LOGDIR. -Patrick Monnerat (18 Apr 2023) + Ref: #10818 -- OS400: provide ILE/RPG usage examples +- runtests: use a per-runner random seed - Closes https://github.com/curl/curl/pull/10994 + Each runner needs a unique random seed to reduce the chance of port + number collisions. The new scheme uses a consistent per-runner source of + randomness which results in deterministic behaviour, as it did before. -- OS400: improve vararg emulation + Ref: #10818 - - Use V7R4 RPG procedure overloading to improve vararg emulation. +- runtests: complete main test loop refactor for multiple runners - From OS400 V7R4 and above, ILE/RPG implements a limited procedure - overloading feature that can be used to improve curl's typed - implementation of varargs procedures. This commit applies it to - curl_easy_setopt(), curl_multi_setopt(), curl_share_setopt() and - curl_easy_getinfo(). + The main test loop is now able to handle multiple runners, or no + additional runner processes at all. At most one process is still + created, however. - Closes https://github.com/curl/curl/pull/10994 + Ref: #10818 -- OS400: fix and complete ILE/RPG binding +- runtests: prepare main test loop for multiple runners - - Fix wrong definitions of CURL_ZERO_TERNINATED, curl_mime_data() and - curl_mime_data_ccsid(). + Some variables are expanded to arrays and hashes so that multiple + runners can be used for running tests. - - Add recent definitions, in particular blob, header API and WebSockets - API. + Ref: #10818 - - Support for CURLVERSION_ELEVENTH. +Stefan Eissing (5 Jun 2023) - - New functions for EBCDIC support. +- bufq: make write/pass methods more robust - Reflect these changes in README.OS400. + - related to #11242 where curl enters busy loop when + sending http2 data to the server - Closes https://github.com/curl/curl/pull/10994 + Closes #11247 -- OS400: implement EBCDIC support for recent features +Boris Verkhovskiy (5 Jun 2023) - - Support CURLVERSION_ELEVENTH. +- tool_getparam: fix comment - - New function curl_url_strerror_ccsid(). + Closes #11253 - - curl_easy_setopt_ccsid() supports blobs and 3 recent string options. +Raito Bezarius (5 Jun 2023) - - New function curl_easy_header_ccsid(). +- haproxy: add --haproxy-clientip flag to spoof client IPs - - New generic latin1<-->ccsid conversion functions curl_from_ccsid() and - curl_to_ccsid() for user convenience. + CURLOPT_HAPROXY_CLIENT_IP in the library - - README.OS400 updated accordingly. + Closes #10779 - - Removed a leftover QsoSSL support identifier. +Daniel Stenberg (5 Jun 2023) - Closes https://github.com/curl/curl/pull/10994 +- curl: add --ca-native and --proxy-ca-native -- OS400: rework build scripts + These are two boolean options to ask curl to use the native OS's CA + store when verifying TLS servers. For peers and for proxies + respectively. - - Rename shell function "system" to "CLcommand" to avoid confusion with - built-in command. + They currently only have an effect for curl on Windows when built to use + OpenSSL for TLS. - - Reformat scripts. Fix some indentations. Avoid lines > 80 characters - where possible. + Closes #11049 - - Support ASCII runtime development files in a user-defined directory - path. +Viktor Szakats (5 Jun 2023) - - FIX SONAME detection. +- build: drop unused/redundant `HAVE_WINLDAP_H` - - Drop form API test program compilation (does not exist anymore). + Sources did not use it. Autotools used it when checking for the + `winldap` library, which is redundant. + + With CMake, detection was broken: + ``` + Run Build Command(s):/usr/local/Cellar/cmake/3.26.3/bin/cmake -E env VERBOSE= + 1 /usr/bin/make -f Makefile cmTC_2d8fe/fast && /Library/Developer/CommandLine + Tools/usr/bin/make -f CMakeFiles/cmTC_2d8fe.dir/build.make CMakeFiles/cmTC_2 + d8fe.dir/build + Building C object CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj + /usr/local/opt/llvm/bin/clang --target=x86_64-w64-mingw32 --sysroot=/usr/loca + l/opt/mingw-w64/toolchain-x86_64 -D_WINSOCKAPI_="" -I/my/quictls/x64-ucrt/usr + /include -I/my/zlib/x64-ucrt/usr/include -I/my/brotli/x64-ucrt/usr/include -W + no-unused-command-line-argument -D_UCRT -DCURL_HIDDEN_SYMBOLS -DHAVE_SSL_SE + T0_WBIO -DHAS_ALPN -DNGHTTP2_STATICLIB -DNGHTTP3_STATICLIB -DNGTCP2_STATICLIB + -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static-libgcc -lucrt -Wextra -Wall -p + edantic -Wbad-function-cast -Wconversion -Winline -Wmissing-declarations -Wmi + ssing-prototypes -Wnested-externs -Wno-long-long -Wno-multichar -Wpointer-ari + th -Wshadow -Wsign-compare -Wundef -Wunused -Wwrite-strings -Wcast-align -Wde + claration-after-statement -Wempty-body -Wendif-labels -Wfloat-equal -Wignored + -qualifiers -Wno-format-nonliteral -Wno-sign-conversion -Wno-system-headers - + Wstrict-prototypes -Wtype-limits -Wvla -Wshift-sign-overflow -Wshorten-64-to- + 32 -Wdouble-promotion -Wenum-conversion -Wunused-const-variable -Wcomma -Wmis + sing-variable-declarations -Wassign-enum -Wextra-semi-stmt -MD -MT CMakeFile + s/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -MF CMakeFiles/cmTC_2d8fe.dir/HAVE_WINL + DAP_H.c.obj.d -o CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -c /my/curl/b + ld-cmake-llvm-x64-shared/CMakeFiles/CMakeScratch/TryCompile-3JP6dR/HAVE_WINLD + AP_H.c + In file included from /my/curl/bld-cmake-llvm-x64-shared/CMakeFiles/CMakeScra + tch/TryCompile-3JP6dR/HAVE_WINLDAP_H.c:2: + In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi + ngw32/include/winldap.h:17: + In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi + ngw32/include/schnlsp.h:9: + In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi + ngw32/include/schannel.h:10: + /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt + .h:5041:254: error: unknown type name 'PSYSTEMTIME' + WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ + NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, + DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER + pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT + ENSIONS pExtensions); + + + + ^ + /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt + .h:5041:278: error: unknown type name 'PSYSTEMTIME' + WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ + NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, + DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER + pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT + ENSIONS pExtensions); + + + + ^ + 2 errors generated. + make[1]: *** [CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj] Error 1 + make: *** [cmTC_2d8fe/fast] Error 2 + exitCode: 2 + ``` + + Cherry-picked from #11095 88e4a21ff70ccef391cf99c8165281ff81374503 + Reviewed-by: Daniel Stenberg + Closes #11245 + +Daniel Stenberg (5 Jun 2023) + +- urlapi: scheme starts with alpha + + Add multiple tests to lib1560 to verify + + Fixes #11249 + Reported-by: ad0p on github + Closes #11250 + +- RELEASE-NOTES: synced + +- CURLOPT_MAIL_RCPT_ALLOWFAILS: replace CURLOPT_MAIL_RCPT_ALLLOWFAILS + + Deprecate the name using three Ls and prefer the name with two. + + Replaces #10047 + Closes #11218 + +- tests/servers: generate temp names in /tmp for unix domain sockets + + ... instead of putting them in the regular pid directories because + systems generally have strict length requirements for the path name to + be shorter than 107 bytes and we easily hit that boundary otherwise. + + The new concept generates two random names: one for the socks daemon and + one for http. + + Reported-by: Andy Fiddaman + Fixes #11152 + Closes #11166 + +Stefan Eissing (2 Jun 2023) + +- http2: better support for --limit-rate + + - leave transfer loop when --limit-rate is in effect and has + been received + - adjust stream window size to --limit-rate plus some slack + to make the server observe the pacing we want + - add test case to confirm behaviour + + Closes #11115 + +- curl_log: evaluate log statement only when transfer is verbose + + Closes #11238 + +Daniel Stenberg (2 Jun 2023) + +- libssh2: provide error message when setting host key type fails - Closes https://github.com/curl/curl/pull/10994 + Ref: https://curl.se/mail/archive-2023-06/0001.html -Sevan Janiyan (18 Apr 2023) + Closes #11240 -- tests/sshserver.pl: Define AddressFamily earlier +Igor Todorovski (2 Jun 2023) - As the comment states "Address family must be specified before ListenAddress" - , otherwise the tests fail to run - `"failed starting SSH server" 52 times (582, 583, 600, 601, 602, 603, 604, 60 - 5, 606 and 43 more)` +- system.h: remove __IBMC__/__IBMCPP__ guards and apply to all z/OS compiles - Closes #10983 + Closes #11241 -Stefan Eissing (18 Apr 2023) +Daniel Stenberg (2 Jun 2023) -- quiche: Enable IDLE egress handling +- docs/SECURITY-PROCESS.md: link to example of previous critical flaw - Follow-up to 544abeea which added the handling but wrongly left it - commented out. +Mark Seuffert (2 Jun 2023) - Closes https://github.com/curl/curl/pull/11000 +- README.md: updated link to opencollective -Daniel Stenberg (18 Apr 2023) + Closes #11232 -- docs/examples/protofeats.c: Outputs all protocols and features +Daniel Stenberg (1 Jun 2023) - Showing off one way to get to char pointer arrays of info returned by - curl_version_info() +- libssh2: use custom memory functions - Closes #10991 + Because of how libssh2_userauth_keyboard_interactive_ex() works: the + libcurl callback allocates memory that is later free()d by libssh2, we + must set the custom memory functions. -- tests/keywords.pl: remove + Reverts 8b5f100db388ee60118c08aa28 - This script does not work since the introduction of the test - preprocessing. If we need this functionality, it probably needs to be - moved into the runtests tool or similar. + Ref: https://github.com/libssh2/libssh2/issues/1078 + Closes #11235 - Reported-by: Dan Fandrich - Fixes #10895 - Closes #10987 +- test447: test PUTting a file that grows -Stefan Eissing (17 Apr 2023) + ... and have curl trim the end when it reaches the expected total amount + of bytes instead of over-sending. -- http2: support HTTP/2 to forward proxies, non-tunneling + Reported-by: JustAnotherArchivist on github + Closes #11223 - - with `--proxy-http2` allow h2 ALPN negotiation to - forward proxies - - applies to http: requests against a https: proxy only, - as https: requests will auto-tunnel - - adding a HTTP/1 request parser in http1.c - - removed h2h3.c - - using new request parser in nghttp2 and all h3 backends - - adding test 2603 for request parser - - adding h2 proxy test cases to test_10_* +- curl: count uploaded data to stop at the originally given size - scorecard.py: request scoring accidentally always run curl - with '-v'. Removed that, expect double numbers. + Closes #11223 + Fixes #11222 + Reported-by: JustAnotherArchivist on github - labeller: added http1.* and h2-proxy sources to detection +- tool: remove exclamation marks from error/warning messages - Closes #10967 +- tool: use errorf() for error output -Daniel Stenberg (17 Apr 2023) + Convert a number of fprintf() calls. -- curl_easy_unescape.3: rename the argument +- tool: remove newlines from all helpf/notef/warnf/errorf calls - and highlight it appropriately in the text. + Make voutf() always add one. - Closes #10979 + Closes #11226 -Viktor Szakats (17 Apr 2023) +- tests/servers.pm: pick unused port number with a server socket -- autotools: sync up clang picky warnings with cmake + This change replaces the previous method of picking a port number at + random to try to start servers on, then retrying up to ten times with + new random numbers each time, with a function that creates a server + socket on port zero, thereby getting a suitable random port set by the + kernel. That server socket is then closed and that port number is used + to setup the actual test server on. - Bringing missing options over from CMake. + There is a risk that *another* server can be started on the machine in + the time gap, but the server verification feature will detect that. - Move around existing `-Wno-pointer-bool-conversion` option to come - _after_ `-Wconversion`. + Closes #11220 - Reviewed-by: Marcel Raad - Closes #10974 +- RELEASE-NOTES: synced -Daniel Stenberg (17 Apr 2023) + bump to 8.2.0 -- tests/libtest/lib1900.c: remove +Alejandro R. Sedeño (31 May 2023) - This file was left behind when the rest of the test was previously removed. +- configure: fix run-compiler for old /bin/sh - Follow-up to e50a877df74f + If you try to assign and export on the same line on some older /bin/sh + implementations, it complains: -- src/tool_operhlp.c: fix value stored to 'uerr' is never read + ``` + $ export "NAME=value" + NAME=value: is not an identifier + ``` - Ref: https://github.com/curl/curl/pull/10974#issuecomment-1510461343 - Reported-by: Viktor Szakats - Closes #10982 + This commit rewrites run-compiler's assignments and exports to work with + old /bin/sh, splitting assignment and export into two separate + statements, and only quote the value. So now we have: -Viktor Szakats (16 Apr 2023) + ``` + NAME="value" + export NAME + ``` -- cmake: speed up and extend picky clang/gcc options + While we're here, make the same change to the two supporting + assign+export lines preceeding the script to be consistent with how + exports work throughout the rest of configure.ac. - Extend existing picky compiler options with ones missing compared to - autotools builds. Also sync options between clang and gcc. + Closes #11228 - Redesign the way we enable these options to avoid the slow option - detection almost completely. +Philip Heiduck (31 May 2023) - This reduces the number of detections from 35 to zero for clang and - 3 for gcc, even after adding a bunch of new options. +- circleci: install impacket & wolfssl 5.6.0 - clang 3.0 (2011-11-29) and gcc 2.95 (1999-07-31) now required. + Closes #11221 - Also show enabled picky options. +Daniel Stenberg (31 May 2023) - Ref: https://github.com/libssh2/libssh2/pull/952 +- tool_urlglob: use curl_off_t instead of longs - Reviewed-by: Daniel Stenberg - Closes #10973 + To handle more globs better (especially on Windows) -Andreas Falkenhahn (16 Apr 2023) + Closes #11224 -- nbtlm: use semicolons instead of commas for (void) args +Dan Fandrich (30 May 2023) - Closes #10978 +- scripts: Fix GHA matrix job detection in cijobs.pl -Daniel Stenberg (15 Apr 2023) + The parsing is pretty brittle and it broke detecting some jobs at some + point. Also, detect if Windows is used in GHA. -- multi: free up more data earleier in DONE +- runtests: abort test run after failure without -a - Before checking for more users of the connection and possibly bailing - out. + This was broken in a recent refactor and test runs would not stop. - Fixes #10971 - Reported-by: Paweł Wegner - Closes #10972 + Follow-up to d4a1b5b6 -- RELEASE-NOTES: synced + Reported-by: Daniel Stenberg + Fixes #11225 + Closes #11227 -- curl: do NOT append file name to path for upload when there's a query +Version 8.1.2 (30 May 2023) - Added test 425 to verify. +Daniel Stenberg (30 May 2023) - Reported-by: Dirk Rosenkranz - Bug: https://curl.se/mail/archive-2023-04/0008.html - Closes #10969 +- RELEASE-NOTES: synced -- libcurl-thread.3: improved name resolver wording + 8.1.2 release - And make better .SH sections +- THANKS: contributors from 8.1.2 - Closes #10966 +- lib1560: verify more scheme guessing -Colman Mbuya (14 Apr 2023) + - on 2nd level domains + - on names without dots -- CURLOPT_PROXY_SSL_VERIFYPEER.3: fix minor grammar mistake + As mentioned in #11161, "imap.com" will be guessed IMAP - Closes #10968 + Closes #11219 -Daniel Stenberg (14 Apr 2023) +- page-header: minor wording polish in the URL segment -- curl: add --proxy-http2 + Closes #11217 - For trying HTTP/2 with an HTTPS proxy. +- page-header: mention curl version and how to figure out current release - Closes #10926 + Closes #11216 -- KNOWN_BUGS: remove fixed or outdated issues, move non-bugs +- RELEASE-NOTES: synced - - remove h3 issues believed to be fixed +- configure: without pkg-config and no custom path, use -lnghttp2 - - make the flaky CI issue be generic and not Windows specific + Reported-by: correctmost on github + Fixes #11186 + Closes #11210 - - "TLS session cache does not work with TFO" now documented +Stefan Eissing (28 May 2023) - This is now a documented restriction and not a bug. TFO in general is - rarely used and has other problems, making it a low-priotity thing to - work on. +- curl: cache the --trace-time value for a second - - remove "Renegotiate from server may cause hang for OpenSSL backend" + - caches HH:MM:SS computed and reuses it for logging during + the same second. + - common function for plain log line start formatting - This is an OpenSSL issue, not a curl one. Even if it taints curl. + Closes #11211 - - rm "make distclean loops forever" +Kev Jackson (28 May 2023) - - rm "configure finding libs in wrong directory" +- libcurl.m4: remove trailing 'dnl' that causes this to break autoconf - Added a section to docs/INSTALL.md about it. + Closes #11212 - - "A shared connection cache is not thread-safe" +Stefan Eissing (26 May 2023) - Moved over to TODO and expanded for other sharing improvements we - could do +- http3: send EOF indicator early as possible - - rm "CURLOPT_OPENSOCKETPAIRFUNCTION is missing" + - ngtcp2 and quiche implementations relied on the DONE_SEND event + to forward the EOF for uploads to the libraries. This often + result in a last 0 length EOF data. Tracking the amount of + data left to upload allows EOF indication earlier. + - refs #11205 where CloudFlare DoH servers did not like to + receive the initial upload DATA without EOF and returned + a 400 Bad Request - - rm "Blocking socket operations in non-blocking API" + Reported-by: Sergey Fionov + Fixes #11205 + Closes #11207 - Already listed as a TODO +Daniel Stenberg (26 May 2023) - - rm "curl compiled on OSX 10.13 failed to run on OSX 10.10" +- scripts/contri*sh: no longer grep -v ' ' - Water under the bridge. No one cares about this anymore. + Originally these scripts filtered out names that have no space so that + they better avoid nick names not intended for credits. Such names are + not too commonly used, plus we now give credit even to those. - - rm "build on Linux links libcurl to libdl" + Additionally: non-latin names, like Asian, don't have spaces at all so + they were also filtered out and had to be manually added which made it + an error-prone operation where Asian names eventually easily fell off by + mistake. - Verified to not be true (anymore). + Closes #11206 - - rm "libpsl is not supported" +- cf-socket: restore Curl_sock_assign_addr() - The cmake build supports it since cafb356e19cda22 + Regression since it was not private. Also used by msh3.c - Closes #10963 + Follow-up to 8e85764b7bd7f05f5 + Reported-by: Gisle Vanem + Fixes #11202 + Closes #11204 -- url: fix PVS nits +- RELEASE-NOTES: synced - - expression 'hostptr' is always true - - a part of conditional expression is always true: proxypasswd - - expression 'proxyuser' is always true - - avoid multiple Curl_now() calls in allocate_conn + Taken down to 8.1.2 now for pending patch release - Ref: #10929 - Closes #10959 +- libssh: when keyboard-interactive auth fails, try password -- bufq: simplify since expression is always true + The state machine had a mistake in that it would not carry on to that + next step. - The check for 'len' is already done so it will remain true until - updated. Pointed out by PVS. + This also adds a verbose output what methods that are available from the + server and renames the macros that change to the next auth methods to + try. - Ref: #10929 - Closes #10958 + Reported-by: 左潇峰 + Fixes #11196 + Closes #11197 -- hash: fix assigning same value +Emanuele Torre (25 May 2023) - Pointed out by PVS +- configure: fix build with arbitrary CC and LD_LIBRARY_PATH - Ref: #10929 - Closes #10956 + Since ./configure and processes that inherit its environment variables + are the only callers of the run-compiler script, we can just save the + current value of the LD_LIBRARY_PATH and CC variables to another pair of + environment variables, and make run-compiler a static script that + simply restores CC and LD_LIBRARY_PATH to the saved value, and before + running the compiler. -- cookie: address PVS nits + This avoids having to inject the values of the variables in the script, + possibly causing problems if they contains spaces, quotes, and other + special characters. - - avoid assigning the same value again - - remove superfluous check of co->domain - - reduce variable scope for namep/valuep + Also add exports in the script just in case LD_LIBRARY_PATH and CC are + not already in the environment. - Ref: #10929 - Closes #10954 + follow-up from 471dab2 -Stefan Eissing (14 Apr 2023) + Closes #11182 -- cf-socket: Disable socket receive buffer by default +Daniel Stenberg (25 May 2023) - - Disable socket receive buffer unless USE_RECV_BEFORE_SEND_WORKAROUND - is in place. +- urlapi: remove superfluous host name check - While we would like to use the receive buffer, we have stalls in - parallel transfers where not all buffered data is consumed and no socket - events happen. + ... as it is checked later more proper. - Note USE_RECV_BEFORE_SEND_WORKAROUND is a Windows sockets workaround - that has been disabled by default since b4b6e4f1, due to other bugs. + Closes #11195 - Closes https://github.com/curl/curl/pull/10961 +Stefan Eissing (25 May 2023) -- cf-h2-proxy: fix processing ingress to stop too early +- http2: fix EOF handling on uploads with auth negotiation - - progress ingress stopped too early, causing data - from the underlying filters to not be processed and - report that no tunnel data was available - - this lead to "hangers" where no socket activity was - seen but data rested in buffers + - doing a POST with `--digest` does an override on the initial request + with `Content-Length: 0`, but the http2 filter was unaware of that + and expected the originally request body. It did therefore not + send a final DATA frame with EOF flag to the server. + - The fix overrides any initial notion of post size when the `done_send` + event is triggered by the transfer loop, leading to the EOF that + is necessary. + - refs #11194. The fault did not happen in testing, as Apache httpd + never tries to read the request body of the initial request, + sends the 401 reply and closes the stream. The server used in the + reported issue however tried to read the EOF and timed out on the + request. - Closes #10952 + Reported-by: Aleksander Mazur + Fixes #11194 + Cloes #11200 -- http3: check stream_ctx more thoroughly in all backends +Daniel Stenberg (23 May 2023) - - callbacks and filter methods might be invoked at unexpected - times, e.g. when the transfer's stream_ctx has not been initialized - yet or, more likely, has already been taken down. - - check for existance of stream_ctx in such places and return - an error or silently succeed the call. +- RELEASE-NOTES: synced - Closes #10951 + bump to 8.2.0 -Daniel Stenberg (13 Apr 2023) +- lib: remove unused functions, make single-use static -- ftp: fix 'portsock' variable was assigned the same value + Closes #11174 - Pointed out by PVS +- scripts/singleuse.pl: add more API calls - Ref: #10929 - Closes #10955 +Christian Hesse (23 May 2023) -- ftp: remove dead code +- configure: quote the assignments for run-compiler - This condition can never be true here since it is handled already 28 - lines above. + Building for multilib failed, as the compiler command contains an + extra argument. That needs quoting. - Pointed out by PVS. + Regression from b78ca50cb3dda361f9c1 - Ref: #10929 - Closes #10957 + Fixes #11179 + Closes #11180 -- cf-h1-proxy: skip an extra NULL assign +Daniel Stenberg (23 May 2023) - and use Curl_safefree() once to save another NULL assign. Found by PVS. +- misc: fix spelling mistakes - Ref. #10929 - Closes #10953 + Reported-by: musvaage on github + Fixes #11171 + Closes #11172 -Philip Heiduck (13 Apr 2023) +Version 8.1.1 (23 May 2023) -- GHA: suppress git clone output +Daniel Stenberg (23 May 2023) - Follow-up: https://github.com/curl/curl/commit/8203aa6ed405ec832d2c62f18dfda2 - 93f89a23f9 +- RELEASE-NOTES: synced - Closes #10949 + curl 8.1.1 -Stefan Eissing (13 Apr 2023) +- THANKS: contributors from the 8.1.1 release -- cf-socket: remove dead code discovered by PVS +Dan Fandrich (22 May 2023) - Closes #10960 +- docs: fix fuzzing documentation link -Daniel Stenberg (13 Apr 2023) + Follow-up to 4c712a1b -- http: skip a double NULL assign +- CI: add an Alpine build with MUSL - and also use a local variable to shorten the long names and increase - readability in the function. Pointed out by PVS. + MUSL is another libc implementation which has its own unique issues + worth testing. - Ref: #10929 - Closes #10950 + Ref: #11140 + Closes #11178 -- mime: skip NULL assigns after Curl_safefree() +- runtests: add a missing \n at the end of a log message - Pointed out by PVS. +correctmost on github (22 May 2023) - Ref: #10929 - Closes #10947 +- SECURITY-PROCESS.md: link security advisory doc and fix typo -- rtsp: skip NULL assigns after Curl_safefree() + Closes #11177 - ... since this is a macro that assigns NULL itself. Pointed out by PVS. +Daniel Stenberg (22 May 2023) - Ref: #10929 - Closes #10946 +- TODO: build curl with Windows Unicode support -- smb: remove double assign + Closes #7229 - The same value is assigned the same value already a few lines above. - Pointed out by PVS. +- KNOWN_BUGS: hyper memory-leaks - Ref: #10929 - Closes #10945 + Closes #10803 -- transfer: skip extra assign +Stefan Eissing (22 May 2023) - The 'result' variable already contains CURLE_OK at this point, no use in - setting it again. Pointed out by PVS. +- http/2: unstick uploads - Ref: #10929 - Closes #10944 + - refs #11157 and #11175 where uploads get stuck or lead to RST streams + - fixes our h2 send behaviour to continue sending in the nghttp2 session + as long as it wants to. This will empty our send buffer as long as + the remote stream/connection window allows. + - in case the window is exhausted, the data remaining in the send buffer + will wait for a WINDOW_UPDATE from the server. Which is a socket event + that engages our transfer loop again + - the problem in the issue was that we did not exhaust the window, but + left data in the sendbuffer and no further socket events did happen. + The server was just waiting for us to send more. + - relatedly, there was an issue fixed that closing a stream with KEEP_HOLD + set kept the transfer from shutting down - as it should have - leading + to a timeout. -- urlapi: skip a pointless assign + Closes #11176 - It stores a null byte after already having confirmed there is a null - byte there. Detected by PVS. +Daniel Stenberg (21 May 2023) - Ref: #10929 - Closes #10943 +- workflows/macos: add a job using gcc + debug + secure transport -Philip Heiduck (13 Apr 2023) +Jay Satiro (21 May 2023) -- GHA: suppress git clone output +- lib: fix conversion warnings with gcc on macOS - Closes #10939 +Daniel Stenberg (21 May 2023) -Stefan Eissing (13 Apr 2023) +- sectransp.c: make the code c89 compatible -- tests: make test_12_01 a bit more forgiving on connection counts + Follow-up to dd2bb485521c2ec713001b3a -- cf-socket: add socket recv buffering for most tcp cases + Reported-by: FeignClaims on github + Fixes #11155 + Closes #11159 - - use bufq as recv buffer, also for Windows pre-receive handling - - catch small reads followed by larger ones in a single socket - call. A common pattern on TLS connections. +Emanuele Torre (21 May 2023) - Closes #10787 +- Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirect + s" -Daniel Stenberg (13 Apr 2023) + This reverts commit df6c2f7b544f1f35f2a3e0be11f345affeb6fe9c. + (It only keep the test case that checks redirection to an absolute URL + without hostname and CURLU_NO_AUTHORITY). -- urlapi: cleanups + I originally wanted to make CURLU_ALLOW_SPACE accept spaces in the + hostname only because I thought + curl_url_set(CURLUPART_URL, CURLU_ALLOW_SPACE) was already accepting + them, and they were only not being accepted in the hostname when + curl_url_set(CURLUPART_URL) was used for a redirection. - - move host checks together - - simplify the scheme parser loop and the end of host name parser - - avoid itermediate buffer storing in multiple places - - reduce scope for several variables - - skip the Curl_dyn_tail() call for speed - - detect IPv6 earlier and skip extra checks for such hosts - - normalize directly in dynbuf instead of itermediate buffer - - split out the IPv6 parser into its own funciton - - call the IPv6 parser directly for ipv6 addresses - - remove (unused) special treatment of % in host names - - junkscan() once in the beginning instead of scattered - - make junkscan return error code - - remove unused query management from dedotdotify() - - make Curl_parse_login_details use memchr - - more use of memchr() instead of strchr() and less strlen() calls - - make junkscan check and return the URL length + That is not actually the case, urlapi never accepted hostnames with + spaces, and a hostname with a space in it never makes sense. + I probably misread the output of my original test when I they were + normally accepted when using CURLU_ALLOW_SPACE, and not redirecting. - An optimized build runs one of my benchmark URL parsing programs ~41% - faster using this branch. (compared against the shipped 7.88.1 library - in Debian) + Some other URL parsers seems to allow space in the host part of the URL, + e.g. both python3's urllib.parse module, and Chromium's javascript URL + object allow spaces (chromium percent escapes the spaces with %20), + (they also both ignore TABs, and other whitespace characters), but those + URLs with spaces in the hostname are useless, neither python3's requests + module nor Chromium's window.location can actually use them. - Closes #10935 + There is no reason to add support for URLs with spaces in the host, + since it was not a inconsistency bug; let's revert that patch before it + makes it into release. Sorry about that. -Josh McCullough (13 Apr 2023) + I also reverted the extra check for CURLU_NO_AUTHORITY since that does + not seem to be necessary, CURLU_NO_AUTHORITY already worked for + redirects. -- http2: fix typo in infof() call + Closes #11169 - Closes #10940 +Dan Fandrich (20 May 2023) -Daniel Stenberg (12 Apr 2023) +- runtests: use the correct fd after select -- noproxy: pointer to local array 'hostip' is stored outside scope + The code was using the wrong fd when determining which runner was ready + with a response. - Ref: #10929 - Closes #10933 + Ref: #10818 + Closes #11160 -Stefan Eissing (12 Apr 2023) +- test425: fix the log directory for the upload -- connect: fix https connection setup to treat ssl_mode correctly + This must be %LOGDIR to let it work with parallel tests. - - for HTTPS protocol, a disabled ssl should never be acceptables. + Ref: #10969 - Closes #10934 +- runtests: handle interrupted reads from IPC pipes -Douglas R. Reno (12 Apr 2023) + These can be interrupted by signals, especially SIGINT to shut down, and + must be restarted so the IPC call arrives correctly. If the read just + returns an error instead, the IPC calling state will go out of sync and + a proper shutdown won't happen. -- CMakeLists.txt: fix typo for Haiku detection + Ref: #10818 - Closes #10937 +Stefan Eissing (20 May 2023) -Dan Fandrich (11 Apr 2023) +- http2: upload improvements -- pathhelp: use the cached $use_cygpath when available + Make send buffer smaller to have progress and "upload done" reporting + closer to reality. Fix handling of send "drain" condition to no longer + trigger once the transfer loop reports it is done sending. Also do not + trigger the send "drain" on RST streams. -- runtests: eliminate unneeded variable + Background: + - a upload stall was reported in #11157 that timed out + - test_07_33a reproduces a problem with such a stall if the + server 404s the request and RSTs the stream. + - test_07_33b verifies a successful PUT, using the parameters + from #11157 and checks success -- runtests: make the # of server start attempts a constant + Ref: #11157 + Closes #11165 -- runtests: on startup failure call displaylogs only in serverfortest +- http2: increase stream window size to 10 MB - This reduces the number of calls spread throughout the code. + Reported-by: pandada8 on github - Ref: #10818 - Closes #10919 + Fixes #11162 + Closes #11167 -- runtests: return an error code with startservers() +Daniel Stenberg (20 May 2023) - The code indicates the kind of failure encountered in starting a server, - which can be used by the caller to tailor the user experience. +- lib: rename struct 'http_req' to 'httpreq' - Ref: #10818 + Because FreeBSD 14 kidnapped the name. + Ref: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271526 -- runtests: abort early if runpingpongserver is given a bad server type + Fixes #11163 + Closes #11164 -- runtests: don't use the SMB server verification time as reference +Viktor Szakats (20 May 2023) - %FTPTIME2 and %FTPTIME3 should be set by the FTP server only, for - consistency. +- cmake: avoid `list(PREPEND)` for compatibility -- tests: factor out the test server management code + `list(PREPEND)` requires CMake v3.15, our minimum is v3.7. - This now lives in servers.pm with some configuration variables moved to - globalconfig.pm + Ref: https://cmake.org/cmake/help/latest/command/list.html#prepend - Ref: #10818 + Regression from 1e3319a167d2f32d295603167486e9e88af9bb4e -- runtests: remove an inappropriate use of runclientoutput + Reported-by: Keitagit-kun on Github + Fixes #11141 + Closes #11144 - This function is intended for running client code, not servers. +Daniel Stenberg (19 May 2023) -- runtests: only add $LIBDIR to the path for checktestcmd +- RELEASE-NOTES: synced - Since checkcmd is for finding servers, there will never be anything in - this directory of interest to them. +Stefan Eissing (19 May 2023) - Ref: #10818 +- ngtcp2: proper handling of uint64_t when adjusting send buffer -- tests: log sshserver.pl messages to a file + Fixes #11149 + Closes #11153 - The logmsg messages were thrown away before, so they are now available - for debugging. +- ngtcp2: fix compiler warning about possible null-deref -- runtests: also show DISABLED tests with -l + - compiler analyzer did not include the call context for this + static function where the condition had already been checked. + - eleminating the problem by making stream a call parameter - Other reasons for skipping tests are ignored for -l, so being explicitly - disabled should be too. + Fixes #11147 + Closes #11151 -- runtests: move the UNIX sockets into $PIDDIR +Emanuele Torre (19 May 2023) - These were missed when the other server files were moved there. +- docs: document that curl_url_cleanup(NULL) is a safe no-op - Follow-up to 70d2fca2 + This has always been the case, but it was not documented. - Ref: #10818 + The paragraph was copied verbatim from curl_easy_cleanup.3 -- tests: tighten up perl exports + Closes #11150 - This reduces namespace pollution a little. +Antoine Pitrou (19 May 2023) - Ref: #10818 +- select: avoid returning an error on EINTR from select() or poll() -- tests: turn perl modules into full packages + This was already done for the poll() and select() calls + made directly from Curl_poll(), but was missed in + Curl_wait_ms(), which is called when there are no fds + to wait on. - This helps enforce more modularization and encapsulation. Enable and fix - warnings on a few packages. Also, rename ftp.pm to processhelp.pm since - there's really nothing ftp-specific in it. + Fixes #11135 + Closes #11143 - Ref: #10818 +Daniel Stenberg (19 May 2023) -Daniel Stenberg (11 Apr 2023) +- vquic.c: make recvfrom_packets static, avoid compiler warning -- multi: remove a few superfluous assigns + warning: no previous prototype for 'recvfrom_packets' - PVS found these "The 'rc' variable was assigned the same value." cases. + Reported-by: Keitagit-kun on github + Fixes #11146 + Closes #11148 - Ref: #10929 - Closes #10932 +- urlapi: allow numerical parts in the host name -- schannel: add clarifying comment + It can only be an IPv4 address if all parts are all digits and no more than + four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be + passed through as a host name. - Explaining how the PVS warning in #10929 is wrong: Dereferencing of the - null pointer 'backend->cred' might take place. + Regression from 17a15d88467 shipped in 8.1.0 - Closes #10931 + Extended test 1560 accordingly. -- cookie: clarify that init with data set to NULL reads no file + Reported-by: Pavel Kalyugin + Fixes #11129 + Closes #11131 - ... and make Curl_cookie_add() require 'data' being set proper with an - assert. +Emilio Cobos Álvarez (19 May 2023) - The function has not worked with a NULL data for quite some time so this - just corrects the code and comment. +- http2: double http request parser max line length - This is a different take than the proposed fixed in #10927 + This works around #11138, by doubling the limit, and should be a + relatively safe fix. - Reported-by: Kvarec Lezki - Ref: #10929 - Closes #10930 + Ideally the buffer would grow as needed and there would be no need for a + limit? But that might be follow-up material. -Kvarec Lezki (11 Apr 2023) + Fixes #11138 + Closes #11139 -- vtls: remove int typecast for sizeof() +Emanuele Torre (18 May 2023) - V220 Suspicious sequence of types castings: memsize -> 32-bit integer -> - memsize. The value being cast: 'sizeof - (buf->data)'. curl\lib\vtls\vtls.c 2025 +- configure: fix --help alignment - https://pvs-studio.com/en/docs/warnings/v220/ + AC_ARG_ENABLE seems to only trim off whitespace from the start and end + of its help-string argument, while prepending two spaces of indentation + to all lines. - Closes #10928 + This means that the two spaces of indentation between the --enable-rtsp + and the --disable-rtsp line were not removed causing ./configure --help + to print: -Stefan Eissing (11 Apr 2023) + Optional Features: + [...] + --enable-rtsp Enable RTSP support + --disable-rtsp Disable RTSP support -- http2: fix copynpaste error reported by coverity + I removed the indentation to fix the issue, now it prints: - - move all code handling HTTP/2 frames for a particular - stream into a separate function to keep from confusing - the call `data` with the stream `data`. + Optional Features: + [...] + --enable-rtsp Enable RTSP support + --disable-rtsp Disable RTSP support - Closes #10924 + The --enable-hsts and --disable-hsts lines had the same problems, and + have been fixed too. -Dan Fandrich (11 Apr 2023) + Closes #11142 -- tests: log a too-long Unix socket path in sws and socksd +Deal(一线灵) (18 May 2023) - Ref: #10919 +- cmake: repair cross compiling -Daniel Stenberg (11 Apr 2023) + It cannot *run* code for testing purposes when cross-compiling. -- gen.pl: error on duplicated See-Also fields + Closes #11130 - Updated http2.d accordingly. +Daniel Stenberg (18 May 2023) - Closes #10925 +- configure: generate a script to run the compiler -- http2: avoid possible null pointer dereference + in the CURL_RUN_IFELSE macro, with LD_LIBRARY_PATH set to the value of + the configure invoke, and not the value that might be used later, + intended for the execution of the output the compiler ouputs. - Reported-by: Dan Fandrich - Fixes #10920 - Closes #10923 + For example when the compiler uses the same library (like libz) that + configure checks for. -- lib1560: verify that more bad host names are rejected + Reported-by: Jonas Bülow + Fixes #11114 + Closes #11120 - when setting the hostname component of a URL +Stefan Eissing (18 May 2023) - Closes #10922 +- cf-socket: completely remove the disabled USE_RECV_BEFORE_SEND_WORKAROUND -- curl_url_set.3: mention that users can set content rather freely + Closes #11118 + +Emanuele Torre (18 May 2023) - ... which then might render bad URLs if you extract a URL later. +- urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects - Closes #10921 + curl_url_set(uh, CURLUPART_URL, redirurl, flags) was not respecing + CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl + when redirecting to an absolute URL. -Dan Fandrich (10 Apr 2023) + Closes #11136 -- CI: retry failed downloads of aws-lc +Colin Cross (18 May 2023) - Don't fail the build in case of a temporary server problem. +- hostip: move easy_lock.h include above curl_memory.h -- test1169: fix so it works properly everywhere + Similar to #9561, move easy_lock.h above curl_memory.h to fix building + against musl libc. - - Use an absolute path for the -L option since the module isn't in the - perl path - - Create the needed test file in a section; isn't - intended for this - - Fix the test number in the file name, which was wrong + Closes #11140 - Follow-up to f754990a +Hind Montassif (18 May 2023) - Ref: #10818 - Fixes #10889 - Closes #10917 +- curl_easy_getinfo: clarify on return data types -- tests: stop using strndup(), which isn't portable + Closes #11126 - It's not available on Solaris 10, for example. Since this is just test - code that doesn't need to use an optimized system version, replace it - with the implementation copied from tool_cb_hdr.c. +Emanuele Torre (18 May 2023) -- runtests: fix an incorrect comment about the ld_preload feature +- checksrc: disallow spaces before labels - Follow-up to 1f631864 + Out of 415 labels throughout the code base, 86 of those labels were + not at the start of the line. Which means labels always at the start of + the line is the favoured style overall with 329 instances. - Ref: #10818 + Out of the 86 labels not at the start of the line: + * 75 were indented with the same indentation level of the following line + * 8 were indented with exactly one space + * 2 were indented with one fewer indentation level then the following + line + * 1 was indented with the indentation level of the following line minus + three space (probably unintentional) -Daniel Stenberg (9 Apr 2023) + Co-Authored-By: Viktor Szakats -- urlapi: prevent setting invalid schemes with *url_set() + Closes #11134 - A typical mistake would be to try to set "https://" - including the - separator - this is now rejected as that would then lead to - url_get(... URL...) would get an invalid URL extracted. +Daniel Stenberg (18 May 2023) - Extended test 1560 to verify. +- cookie: update the comment on cookie length and size limits - Closes #10911 + To refer to the proper cookie RFC and the upcoming RFC refresh. -Biswapriyo Nath (9 Apr 2023) + Closes #11127 -- http2: remove unused Curl_http2_strerror function declaration +- url: provide better error message when URLs fail to parse - Curl_http2_strerror was renamed to http2_strerror in - 05b100aee247bb9bec8e9a1b0 and then http2_strerror was removed in - 5808a0d0f5ea0399d4a2a2 + By providing the URL API error message into the error message. - This also fixes the following compiler error + Ref: #11129 + Closes #11137 - lib/http2.h:41:33: error: unknown type name 'uint32_t' - lib/http2.h:1:1: note: 'uint32_t' is defined in header '' +- RELEASE-NOTES: synced - Closes #10912 + bumped to 8.1.1 -Daniel Stenberg (8 Apr 2023) +Jon Rumsey (18 May 2023) -- RELEASE-NOTES: synced +- os400: update chkstrings.c -SuperIlu on github (8 Apr 2023) + Compensate changes for recent changes to urldata.h to reclassify + STRING_AWS_SIGV4. -- config-dos.h: fix SIZEOF_CURL_OFF_T for MS-DOS/DJGPP + Fixes #11132 + Closes #11133 - Fixes #10905 - Closes #10910 +Version 8.1.0 (17 May 2023) -Daniel Stenberg (8 Apr 2023) +Daniel Stenberg (17 May 2023) -- lib: remove CURLX_NO_MEMORY_CALLBACKS +- RELEASE-NOTES: synced - The only user of this define was 'chkdecimalpoint' - a special purpose - test tool that was built but not used anymore (since 17c18fbc3 - Apr - 2020). +- THANKS: contributors from the 8.1.0 release - Closes #10908 +- hostip: include easy_lock.h before using GLOBAL_INIT_IS_THREADSAFE -- CURLPROXY_HTTPS2: for HTTPS proxy that may speak HTTP/2 + Since that header file is the only place that define can be defined. - Setting this proxy type allows curl to negotiate and use HTTP/2 with - HTTPS proxies. + Reported-by: Marc Deslauriers - Closes #10900 + Follow-up to 13718030ad4b3209 -Ali Khodkar (8 Apr 2023) + Closes #11121 -- write-out.d: add missing periods +Thomas Taylor (16 May 2023) - Closes #10897 +- aws-sigv4.d: fix region identifier in example -Daniel Stenberg (7 Apr 2023) + Closes #11117 -- http2: remove check for !data after it was already dereferenced +Philip Heiduck (15 May 2023) - Pointed out by Coverity +- mlc_config.json: remove this linkcheck CI job config file - Closes #10906 + Closes #11113 -- http_proxy: provide missing arg to infof() call +Daniel Silverstone (15 May 2023) - Pointed out by Coverity +- ssh: Add support for libssh2 read timeout - Closes #10904 + Hook the new (1.11.0 or newer) libssh2 support for setting a read timeout + into the SERVER_RESPONSE_TIMEOUT option. With this done, clients can use + the standard curl response timeout setting to also control the time that + libssh2 will wait for packets from a slow server. This is necessary to + enable use of very slow SFTP servers. -- content_encoding: only do tranfer-encoding compression if asked to + Signed-off-by: Daniel Silverstone - To reduce surprises. Update test 387 and 418 accordingly. + Closes #10965 - Closes #10899 +Osama Albahrani (14 May 2023) -- sws: comparison of unsigned expression < 0 is always false +- GIT-INFO: add --with-openssl - Follow-up to 356dd0b73a75ed6d5 + Closes #11110 - Closes #10903 +Daniel Stenberg (13 May 2023) -- lib/cmake: add HAVE_WRITABLE_ARGV check +- RELEASE-NOTES: synced - Assisted-by: Jakub Zakrzewski - Closes #10896 +Marcel Raad (13 May 2023) -- configure: don't set HAVE_WRITABLE_ARGV on Windows +- md(4|5): don't use deprecated iOS functions - Ref: #10888 - Closes #10896 + They are marked as deprecated in iOS 13.0, which might result in + warnings-as-errors. -- vtls: fix build error when proxy-disabled + Also, use `*_MIN_REQUIRED` instead of `*_MIN_ALLOWED`, which seems to + be what's currently used. - Closes #10901 + Bug: https://github.com/curl/curl/issues/11098 + Closes https://github.com/curl/curl/pull/11102 -Stefan Eissing (6 Apr 2023) +- md4: only build when used -- tests: increase sws timeout for more robust testing + Its only usage in curl_ntlm_core.c is guarded by `USE_CURL_NTLM_CORE`, + so let's use this here too. - - for https CONNECT forwarding, this was fixed at 5 seconds - which led to spurious CI test failures - - add --keepalive parameter to sws to control this - - let httpserver use 30 seconds + Ref: https://github.com/curl/curl/issues/11098 + Closes https://github.com/curl/curl/pull/11102 - Closes #10898 +Vítor Galvão (12 May 2023) -- http2: move HTTP/2 stream vars into local context +- write-out.d: Use response_code in example - - remove NGHTTP2 members of `struct HTTP` - - add `void *h2_ctx` to `struct HTTP` - - add `void *h3_ctx` to `struct HTTP` - - separate h2/h3 pointers are needed for eyeballing - - manage local stream_ctx in http implementations + Closes #11107 - Closes #10877 +Shohei Maeda (12 May 2023) -- proxy: http2 proxy tunnel implementation +- url: fix null dispname for --connect-to option - - currently only on debug build and when env variable - CURL_PROXY_TUNNEL_H2 is present. - - will ALPN negotiate with the proxy server and switch - tunnel filter based on the protocol negotiated. - - http/1.1 tunnel code moved into cf-h1-proxy.[ch] - - http/2 tunnel code implemented in cf-h2-proxy.[ch] - - tunnel start and ALPN set remains in http_proxy.c - - moving all haproxy related code into cf-haproxy.[ch] + Closes #11106 - VTLS changes - - SSL filters rely solely on the "alpn" specification they - are created with and no longer check conn->bits.tls_enable_alpn. - - checks on which ALPN specification to use (or none at all) are - done in vtls.c when creating the filter. +Daniel Stenberg (12 May 2023) - Testing - - added a nghttpx forward proxy to the pytest setup that - speaks HTTP/2 and forwards all requests to the Apache httpd - forward proxy server. - - extending test coverage in test_10 cases - - adding proxy tests for direct/tunnel h1/h2 use of basic auth. - - adding test for http/1.1 and h2 proxy tunneling to pytest +- test2306: verify getting a second response with folded headers - Closes #10780 + Reproduces the isue #11101 and verifies the fix. -- vtls and h2 improvements + Verifies a17b2a503f - - eliminate receive loop in vtls to fill buffer. This may - lead to partial reads of data which is counter productive - - let http2 instead loop smarter to process pending network - data without transfer switches +- headers: clear (possibly) lingering pointer in init - scorecard improvements - - do not start caddy when only httpd is requested - - allow curl -v to stderr file on --curl-verbose + The "prevhead" pointer is used for the headers storage but was not + cleared correctly in init, which made it possible to act up when a + handle is reused. - Closes #10891 + Reported-by: Steve Herrell + Fixes #11101 + Closes #11103 -Daniel Stenberg (6 Apr 2023) +- RELEASE-NOTES: synced -- tests: 1078 1288 1297 use valid IPv4 addresses +- ngtcp2: use 0.15.0 - With the enhanced URL parser, these tests failed because of their bad - IPv4 use. + - nghttp3 0.11.0 + - nghttp2 1.53.0 -- urlapi: detect and error on illegal IPv4 addresses + Adapt to new API calls - Using bad numbers in an IPv4 numerical address now returns - CURLUE_BAD_HOSTNAME. + Closes #11031 - I noticed while working on trurl and it was originally reported here: - https://github.com/curl/trurl/issues/78 +Jay Satiro (10 May 2023) - Updated test 1560 accordingly. +- openssl: fix indent - Closes #10894 +Daniel Stenberg (10 May 2023) -- RELEASE-NOTES: synced +- CURLOPT_DNS_CACHE_TIMEOUT.3: fix spelling -- urlapi: URL encoding for the URL missed the fragment + Follow-up to 9ed7d56e044f5aa1b29 - Meaning that it would wrongly still store the fragment using spaces - instead of %20 if allowing space while also asking for URL encoding. + Closes #11096 - Discovered when playing with trurl. +- hostip: use time_t for storing oldest DNS entry - Added test to lib1560 to verify the fix. + Theoretically, the oldest time could overflow an int. In practice that + won't happen, but let's do this to please analyzers. - Closes #10887 + Follow-up to 9ed7d56e044f5aa1b2928ccde6245d0 -- rtsp: convert mallocs to dynbuf for RTP buffering + Pointed out by Coverity. + Closes #11094 - Closes #10786 +- http: free the url before storing a new copy -- tool_writeout: add URL component variables + To avoid a memory-leak. - Output specific components from the used URL. The following variables - are added for this purpose: + Reported-by: Hiroki Kurosawa - url.scheme, url.user, url.password, url.options, url.host, url.port, - url.path, url.query, url.fragment, url.zoneid + Closes #11093 - Add the following for outputting parts of the "effective URL": +- compressed.d: clarify the words on "not notifying headers" - urle.scheme, urle.user, urle.password, urle.options, urle.host, urle.port, - urle.path, urle.query, urle.fragment, urle.zoneid + Reported-by: Dylan Anthony + Fixes #11091 + Closes #11092 - Added test 423 and 424 to verify. +- libssh2: free fingerprint better - Closes #10853 + Reported-by: Wei Chong Tan + Closes #11088 -Stefan Eissing (4 Apr 2023) +- CURLOPT_IPRESOLVE.3: clarify that this for host names, not IP addresses -- tests/http: improved httpd detection + Reported-by: Harry Sintonen + Closes #11087 - - better error messages when not found/complete - - handling of `--without-test-httpd` +- hostip: enforce a maximum DNS cache size independent of timeout value - Reported-by: kwind on github - Fixes #10879 - Closes #10883 + To reduce the damage an application can cause if using -1 or other + ridiculous timeout values and letting the cache live long times. -Daniel Stenberg (4 Apr 2023) + The maximum number of entries in the DNS cache is now totally + arbitrarily and hard-coded set to 29999. -- configure: make quiche require quiche_conn_send_ack_eliciting + Closes #11084 - curl now requires quiche version >= 1.17.1 to be used and this function - was added in this version and makes a convenient check. +- hostip: store dns timeout as 'int' - This requirement is because this is the lowest quiche version that - supports peer-initiated key updates correctly. + ... because it set and held as an 'int' elsewhere and can never be + larger. - Closes #10886 +- RELEASE-NOTES: synced -Dan Fandrich (1 Apr 2023) +- tool_operate: refuse (--data or --form) and --continue-at combo -- unit tests: use the unit test infrastructure better + libcurl assumes that a --continue-at resumption is done to continue an + upload using the read callback and neither --data nor --form use + that and thus won't do what the user wants. Whatever the user wants + with this strange combination. - Allow UNITTEST_STOP to return the error code, use the fail & abort - macros to indicate test failure and return success instead of fail if - the unit test can't test anything because of missing features at - compile-time. A couple of tests could never fail because they were - overriding the failure return code. + Add test 426 to verify. -- runtests: strip EOL on precheck output on Windows, too + Reported-by: Smackd0wn on github + Fixes #11081 + Closes #11083 - Precheck failures would show on two lines in the test summary output - otherwise. +- transfer: refuse POSTFIELDS + RESUME_FROM combo -- tests: move server config files under the pid dir + The code assumes that such a resume is wanting to continue an upload + using the read callback, and since POSTFIELDS is done without callback + libcurl will just misbehave. - These files are generated by the test servers and must therefore be - found in the log directory to make them available to only those servers - once multiple test runners are executing in parallel. They must also not - be deleted with the log files, so they are stored in the pidfile - directory. + This combo will make the transfer fail with CURLE_BAD_FUNCTION_ARGUMENT + with an explanation in the error message. - Ref: #10818 - Closes #10875 + Reported-by: Smackd0wn on github + Fixes #11081 + Closes #11083 -- runtests: use the ssh key filenames from the sshhelp package +- ipv4.d/ipv6.d: they are "mutex", not "boolean" -- tests: move pidfiles and portfiles under the log directory + ... which for example means they do not have --no-* versions. - This is to segregate all files written by a test process into a single - root to allow for future parallel testing. + Reported-by: Harry Sintonen + Fixes #11085 + Closes #11086 - Ref: #10818 - Closes #10874 +- docs/SECURITY-ADVISORY.md: how to write a curl security advisory -- runtests: minor code cleanups + Closes #11080 -- runtests: call processexists() and pidfromfile() +nobedee on github (5 May 2023) - rather than duplicating the logic in several places. +- MANUAL.md: add dict example for looking up a single definition -Viktor Szakats (31 Mar 2023) + Closes #11077 -- cmake: do not add zlib headers for openssl +Dan Fandrich (5 May 2023) - Logic copied earlier from wolfSSL. wolfSSL requires zlib headers for its - public headers. OpenSSL does not, so stop adding zlib headers for it. +- runtests: fix -c option when run with valgrind - Follow-up to 1e3319a167d2f32d295603167486e9e88af9bb4e + The curl binary argument wasn't being quoted properly. This seems to + have broken at some point after quoting was added in commit 606b29fe. - Closes #10878 + Reported-by: Daniel Stenberg + Ref: #11073 + Fixes #11074 + Closes #11076 -Stefan Eissing (31 Mar 2023) +- runtests: support creating more than one runner process -- rustls: fix error in recv handling + The controller currently only creates and uses one, but more are now + possible. - - when rustls is told to recieve more TLS data and its internal - plaintext buffers are full, it returns an IOERROR - - avoid receiving TLS data while plaintext is not read empty + Ref: #10818 - pytest: - - increase curl run timeout when invoking pytest with higher verbosity +- runtests: spawn a new process for the test runner - Closes #10876 + When the -j option is given, a new process is spawned in which the test + programs are run and from which test servers are started. Only one + process can be started at once, but this is sufficient to test that the + infrastructure can isolate those functions in a new task. There should + be no visible difference between the two modes at the moment. -- http3: improvements across backends + Ref: #10818 + Closes #11064 - - ngtcp2: using bufq for recv stream data - - internal stream_ctx instead of `struct HTTP` members - for quiche, ngtcp2 and msh3 - - no more QUIC related members in `struct HTTP` - - experimental use of recvmmsg(), disabled by default - - testing on my old debian box shows no throughput improvements. - - leaving it in, but disabled, for future revisit - - vquic: common UDP receive code for ngtcp2 and quiche - - vquic: common UDP send code for ngtcp2 and quiche - - added pytest skips for known msh3 failures - - fix unit2601 to survive torture testing - - quiche: using latest `master` from quiche and enabling large download - tests, now that key change is supported - - fixing test_07_21 where retry handling of starting a stream - was faulty - - msh3: use bufq for recv buffering headers and data - - msh3: replace fprintf debug logging with LOG_CF where possible - - msh3: force QUIC expire timers on recv/send to have more than - 1 request per second served +- runtests: turn singletest() into a state machine - Closes #10772 + This allows it to run in a non-blocking manner. -Dan Fandrich (30 Mar 2023) + Ref: #10818 -- test1471/2: add http as a required feature +- runtests: change runner interface to be asynchronous - curl bails out early with a different error message if http support is - compiled out. + Program arguments are marshalled and then written to the end of a pipe + which is later read from and the arguments unmarshalled before the + desired function is called normally. The function return values are + then marshalled and written into another pipe when is later read from + and unmarshalled before being returned to the caller. - Ref: #10705 + The implementation is currently blocking but can be made non-blocking + without any changes to the API. This allows calling multiple runners + without blocking in the future. -- tests: limit return code of unit tests and lib tests + Ref: #10818 - Values greater than 125 have special meanings, so cap it there. Unit - tests and lib tests use the number of failures as the return code, so a - large number of failures (such as test 2601 as a torture test) can - exceed this causing the test to be erroneously reported as having - failed. +- runtests: call citest_finishtest in singletest - Ref: #10720 + This is where citest_starttest is called. -- test1960: point to the correct path for the precheck tool + Ref: #10818 - Otherwise, it might find the binary in .libs which can cause it to use - the system libcurl which can fail. This error is only visible by - noticing that the test is skipped. +- runtests: add a runner initialization function - Follow-up to e4dfe6fc + This sets up the runner environment to start running tests. - Ref: #10651 + Ref: #10818 -- tests: use the proper %LOGDIR path on two tests +- runtests: remove directory from server filename variables - Follow-up to e7a021e1 + There will soon be multiple log directories so the paths will no longer + be static in runtests.pl. Also, get rid of $SERVER2IN which was not + used. Ref: #10818 -Daniel Stenberg (30 Mar 2023) - -- rtsp: fix Value stored to 'skip_size' is never read - - Pointed out by scan-build +- runtests: reduce package exports after refactoring - Follow-up to 6c6306f3008f2c9b20a64 + Some recent refactoring made these export no longer necessary. Also, + stop displaying the Unix socket paths at startup since there will soon + be many of them and they're not that interesting. - Closes #10872 + Ref: #10818 -Stefan Eissing (30 Mar 2023) +- runtests: use a function to obtain $LOGDIR for a test -- tests/http: relax connection check in test_07_02 + This will no longer be static soon. - Only 1 connection will be used when curl is slow, happens when - address-sanitized in CI, for example + Ref: #10818 - Closes #10865 +Jay Satiro (5 May 2023) -- http2: flow control and buffer improvements +- tool_cb_hdr: Fix 'Location:' formatting for early VTE terminals - - use bufq for send/receive of network data - - usd bufq for send/receive of stream data - - use HTTP/2 flow control with no-auto updates to control the - amount of data we are buffering for a stream - HTTP/2 stream window set to 128K after local tests, defined - code constant for now - - elminiating PAUSEing nghttp2 processing when receiving data - since a stream can now take in all DATA nghttp2 forwards + - Disable hyperlink formatting for the 'Location:' header value in VTE + 0.48.1 and earlier, since it is buggy in some of those versions. - Improved scorecard and adjuste http2 stream window sizes - - scorecard improved output formatting and options default - - scorecard now also benchmarks small requests / second + Prior to this change those terminals may show the location header value + as gibberish or show it twice. - Closes #10771 + Ref: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#backw + ard-compatibility -Dan Fandrich (30 Mar 2023) + Fixes https://github.com/curl/curl/issues/10428 + Closes https://github.com/curl/curl/pull/11071 -- runtests: show error message if file can't be written +François Michel (3 May 2023) -- tests: fix remaining servers to run with a dynamic log directory +- quiche: disable pacing while pacing is not actually performed - This final commit in the series is sufficient to allow the tests succeed - if $LOGDIR is changed in runtests.pl. + Closes #11068 - Ref: #10818 - Closes #10866 +Daniel Stenberg (2 May 2023) -- tests: fix fake_ntlm to run with a dynamic log directory +- easy_cleanup: require a "good" handle to act - Ref: #10818 + By insisting that the passed in handle is "good" (the magic number is + intact), this can limit the potential damage if a bad pointer is passed + in. Like when this function is called twice on the same handle pointer. -- tests: fix http servers to run with a dynamic log directory + Ref: #10964 + Closes #11061 - Ref: #10818 +Andreas Falkenhahn (1 May 2023) -- tests: fix ftpserver to run with a dynamic log directory +- amiga: Fix CA certificate paths for AmiSSL and MorphOS - Ref: #10818 + AmiSSL stores certificates in `AmiSSL:Certs` and MorphOS stores them in + `MOSSYS:Data/SSL/curl-ca-bundle.crt`. -- tests: fix C servers to run with a dynamic log directory + Closes https://github.com/curl/curl/pull/11059 - Ref: #10818 +Daniel Stenberg (30 Apr 2023) -- tests: fix lib tests to run with a dynamic log directory +- http2: (void)-mark when we explicitly ignore the return code - Ref: #10818 + When h2_progress_egress() is called. Pointed out by Coverity. -- tests: fix unit tests to run with a dynamic log directory + Closes #11057 - Ref: #10818 +- checksrc: find bad indentation in conditions without open brace -- tests: use %LOGDIR to refer to the log directory + If the previous line starts with if/while/for AND ends with a closed + parenthesis and there's an equal number of open and closed parentheses + on that line, verify that this line is indented $indent more steps, if + not a cpp line. - This will allow it be set dynamically. + Also adjust the fall-out from this fix. - Ref: #10818 + Closes #11054 -- runtests: track verification time even if no files to compare +Diogo Teles Sant'Anna (28 Apr 2023) -- getpart: better handle case of file not found +- CI: Set minimal permissions on workflow ngtcp2-quictls.yml -- testcurl: bump version date + Signed-off-by: Diogo Teles Sant'Anna - It hadn't been updated in 9 years; it's time. + Closes #11055 -- tests: switch to 3-argument open in test suite +Dan Fandrich (28 Apr 2023) - The perl 2-argument open has been considered not-quite-deprecated since - the 3-argument form was introduced almost a quarter century ago. +- CI: use another glob syntax for matching files on Appveyor -- tests: silence some Perl::Critic warnings in test suite + The previous globbing syntax was not matching files recursively in + directories, so try appending a /* to more closely match the examples at + https://www.appveyor.com/docs/how-to/filtering-commits/ - Not all warnings are fixed; many are as much stylistic suggestions than - anything and IMHO don't do much to actually improve the code. +Daniel Stenberg (28 Apr 2023) - Ref: #10818 - Closes #10861 +- multi: add multi-ignore logic to multi_socket_action -- docs: bump the minimum perl version to 5.6 + The multi-ignore logic that was previously applied to + curl_multi_perform() (#10750) is here applied to the loop within + curl_multi_socket_action() to make it use the same optimization: most + handles have the same signal-ignore option state so this drastically + reduces the number of ignore/unignore calls per libcurl function invoke. - It's actually been this way since at least 2012 (when a 3-argument open - was added to runtests.pl). Given the lack of complaints in the interim, - it's safe to call this 23 year old perl version the minimum. + Follow-up to bc90308328afb8 -- runtests: memoize the getpart* subroutines to speed up access + Closes #11045 - The refactored code calls these functions with the same arguments more - often, so this prevents redundant test case file parsing. +Stefan Eissing (28 Apr 2023) - Approved-by: Daniel Stenberg - Ref: #10818 - Closes #10833 +- http2: do flow window accounting for cancelled streams -- runtests: remove duplicated feature variables + - nghttp2 does not free connection level window flow for + aborted streams + - when closing transfers, make sure that any buffered + response data is "given back" to the flow control window + - add tests test_02_22 and test_02_23 to reproduce - Use the feature map stored in the hash table instead. Most of the - variables were only used only once, to set the value in the hash table. + Closes #11052 - Ref: #10818 +- pingpong: fix compiler warning "assigning an enum to unsigned char" -- runtests: also ignore test file problems when ignoring results + Closes #11050 - This simplifies error handling in the test verification code and makes - it more consistent. +Daniel Stenberg (28 Apr 2023) - Ref: #10818 +- configure: fix detection of apxs (for httpd) -- runtests: more refactoring for clarity + The condition check was turned the wrong way around! - Ref: #10818 + Closes #11051 -- runtests: don't start servers if -l is given +Viktor Szakats (28 Apr 2023) -- runtests: fix typos +- ci: `-Wno-vla` no longer necessary -- runtests: refactor singletest() into separate functions + We handle this issue in the source now. - This takes it from a 1200 line behemoth into something more manageable. - The content and order of the functions is taken almost directly from - singletest() so the diff sans whitespace is quite short. + Follow-up to b725fe1944b45406676ea3aff333ae3085a848d9 - Ref: #10818 + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Closes #11048 -- runtests: refactor singletest() into distinct sections +Marcel Raad (28 Apr 2023) - Namely: - - Verify that this test case should be run - - Start the servers needed to run this test case - - Check that test environment is fine to run this test case - - Prepare the test environment to run this test case - - Run the test command - - Clean up after test command - - Verify test succeeded +- tests/http: make curl_setup.h the first include - Ref: #10818 + This is required for the macros there to take effect for system + libraries. Specifically, including the system libraries first led to + warnings about `_FILE_OFFSET_BITS` being redefined in curl_config.h on + the Solaris autobuilds for ws-data.c and ws-pingpong.c. + Also make the curl includes come first for the other source files here + for consistency. -- runtests: stop copying a few arrays where not needed + Closes https://github.com/curl/curl/pull/11046 - Unlike some other languages that just copy a pointer, perl copies the - entire array contents which takes time for a large array. +Emanuele Torre (27 Apr 2023) - Ref: #10818 +- checksrc: check for spaces before the colon of switch labels -- runtests: reduce redundant calls to getpart/getpartattr + Closes #11047 - These functions scan through the entire test file every time to find the - right section, so they can be slow for large test files. +Daniel Stenberg (27 Apr 2023) - Ref: #10818 +- RELEASE-NOTES: synced -- tests: document that the unittest keyword is special +- libssh: tell it to use SFTP non-blocking - Also, add other features that were missing. + Reported-by: Andreas Huebner + Fixes #11020 + Closes #11039 -Stefan Eissing (30 Mar 2023) +Stefan Eissing (27 Apr 2023) -- docs: add documentation for bufq +- http2: enlarge the connection window - Closes #10869 + - fixes stalled connections -Daniel Stenberg (30 Mar 2023) + - Make the connection window large enough, so that there is + some room left should 99/100 streams be PAUSED by the application -- RELEASE-NOTES: synced + Reported-by: Paweł Wegner + Fixes #10988 + Closes #11043 -Matt Jolly (30 Mar 2023) +Daniel Stenberg (27 Apr 2023) -- hostip: refuse to resolve the .onion TLD +- checksrc: fix SPACEBEFOREPAREN for conditions starting with "*" - RFC 7686 states that: + The open paren check wants to warn for spaces before open parenthesis + for if/while/for but also for any function call. In order to avoid + catching function pointer declarations, the logic allows a space if the + first character after the open parenthesis is an asterisk. - > Applications that do not implement the Tor - > protocol SHOULD generate an error upon the use of .onion and - > SHOULD NOT perform a DNS lookup. + I also spotted what we did not include "switch" in the check but we should. - Let's do that. + This check is a little lame, but we reduce this problem by not allowing + that space for if/while/for/switch. - https://www.rfc-editor.org/rfc/rfc7686#section-2 + Reported-by: Emanuele Torre + Closes #11044 - Add test 1471 and 1472 to verify +- docs: minor polish - Fixes #543 - Closes #10705 + - "an HTTP*" (not "a") + - remove a few contractions + - remove a spurious "a" + - reduce use of "I" in texts -Philip Heiduck (30 Mar 2023) + Closes #11040 -- GHA: update ngtcp2-*.yml to v0.10.0 +- ws: fix CONT opcode check - Closes #10612 + Detected by Coverity. Follow-up to 930c00c259 -Stefan Eissing (30 Mar 2023) + Closes #11037 -- tests/http: fix log formatting on wrong exit code +Dan Fandrich (27 Apr 2023) - Closes #10868 +- CI: switch the awslc builds to build out-of-tree -Daniel Stenberg (30 Mar 2023) + This is a common configuration that should be tested to avoid + regressions. The awsls cmake build was already out-of-tree so the + automake build now joins it. -- spellcheck.words: unify the AWS-LC spelling + Ref: #11006 - Follow-up to 34ef4fab22d93 +- tests/http: fix out-of-tree builds - Closes #10867 + Add both lib/ directories (src & build) to the search path so + curl_setup.h and its dependencies can be found. -Jim King (30 Mar 2023) + Followup-to acd82c8b -- openssl: interop with AWS-LC + Ref: #11006 + Closes #11036 - * Configure changes to detect AWS-LC - * CMakeLists.txt changes to detect AWS-LC - * Compile-time branches needed to support AWS-LC - * Correctly set OSSL_VERSION and report AWS-LC release number - * GitHub Actions script to build with autoconf and cmake against AWS-LC +Daniel Stenberg (27 Apr 2023) - AWS-LC is a BoringSSL/OpenSSL derivative - For more information see https://github.com/awslabs/aws-lc/ +- urlapi: make internal function start with Curl_ - Closes #10320 + Curl_url_set_authority() it is. -Viktor Szakats (30 Mar 2023) + Follow-up to acd82c8bfd -- cmake: picky-linker fixes for openssl, ZLIB, H3 and more + Closes #11035 - - fix HTTP/3 support detection with OpenSSL/quictls built with ZLIB. - (Requires curl be built with ZLIB option also.) +YX Hao (26 Apr 2023) - - fix HTTP/3 support detection with OpenSSL/quictls/LibreSSL and `ld` - linker on Windows. +- cf-socket: turn off IPV6_V6ONLY on Windows if it is supported - - fix HTTP/3 support detection with wolfSSL to automatically add - `ws2_32` to the lib list on Windows. For all linkers. + IPV6_V6ONLY refs: + https://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses + https://github.com/golang/go/blob/master/src/net/ipsock_posix.go + https://en.wikipedia.org/wiki/Unix-like + https://learn.microsoft.com/en-us/windows/win32/winsock/ipproto-ipv6-socket-o + ptions - - reposition ZLIB (and other compression) detection _after_ TLS - detection, but before calling HTTP/3-support detection via - `CheckQuicSupportInOpenSSL`. + default value refs: + https://datatracker.ietf.org/doc/html/rfc3493#section-5.3 + https://www.kernel.org/doc/html/latest/networking/ip-sysctl.html#proc-sys-net + -ipv6-variables - May be a regression from ebef55a61df0094b9790710a42f63c48e7de3c13 - May fix #10832 (Reported-by: Micah Snyder) + Closes #10975 - This also seems to fix an odd case, where OpenSSL/quictls is correctly - detected, but its header path is not set while compiling, breaking - build at `src/curl_ntlm_core.c`. Reason for this remains undiscovered. +Daniel Stenberg (26 Apr 2023) - - satisfy "picky" linkers such as `ld` with MinGW, that are highly - sensitive to lib order, by also adding brotli to the beginning of the - lib list. +- urldata: shrink *select_bits int => unsigned char - - satisfy "picky" linkers by adding certain Windows systems libs to - the lib list for OpenSSL/LibreSSL. (Might need additional ones for - other forks, such as `pthread` for BoringSSL.) + - dselect_bits + - cselect_bits - Note: It'd make sense to _always_ add `ws2_32`, `crypt32` (except - Windows App targets perhaps?), `bcrypt` (except old-mingw!) on Windows - at this point. They are almost always required, and if some aren't, - they are ignored by the linker with no effect on final binaries. + ... are using less than 8 bits. Changed types and moved them towards + the end of the structs to fit better. - Closes #10857 + Closes #11025 -Stefan Eissing (30 Mar 2023) +Stefan Eissing (26 Apr 2023) -- vlts: use full buffer size when receiving data if possible +- tests/http: more tests with specific clients - SSL backends like OpenSSL/wolfSSL and other return the content of one - TLS record on read, but usually there are more available. + - Makefile support for building test specific clients in tests/http/clients + - auto-make of clients when invoking pytest + - added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush + - added test_02_21 for lib based downloads and pausing/unpausing transfers - Change the vtls cfilter recv() function to fill the given buffer until a - read would block. + curl url parser: + - added internal method `curl_url_set_authority()` for setting the + authority part of a url (used for PUSH_PROMISE) - Closes #10736 + http2: + - made logging of PUSH_PROMISE handling nicer -dengjfzh on github (30 Mar 2023) + Placing python test requirements in requirements.txt files + - separate files to base test suite and http tests since use + and module lists differ + - using the files in the gh workflows -- rtsp: skip malformed RTSP interleaved frame data + websocket test cases, fixes for we and bufq + - bufq: account for spare chunks in space calculation + - bufq: reset chunks that are skipped empty + - ws: correctly encode frames with 126 bytes payload + - ws: update frame meta information on first call of collect + callback that fills user buffer + - test client ws-data: some test/reporting improvements - Some IP cameras send malformed RTSP interleaved frames sometimes, which - can cause curl_easy_perform return 1 (CURLE_UNSUPPORTED_PROTOCOL). This - change attempts to skip clearly incorrect RTSP interleaving frame data. + Closes #11006 - Closes #10808 +Jay Satiro (26 Apr 2023) -Stefan Eissing (30 Mar 2023) +- libssh2: fix crash in keyboard callback -- lib: add `bufq` and `dynhds` + - Always set the libssh2 'abstract' user-pointer to the libcurl easy + handle associated with the ssh session, so it is always passed to the + ssh keyboard callback. - Adding `bufq`: - - at init() time configured to hold up to `n` chunks of `m` bytes each. - - various methods for reading from and writing to it. - - `peek` support to get access to buffered data without copy - - `pass` support to allow buffer flushing on write if it becomes full - - use case: IO buffers for dynamic reads and writes that do not blow up - - distinct from `dynbuf` in that: - - it maintains a read position - - writes on a full bufq return CURLE_AGAIN instead of nuking itself - - Init options: - - SOFT_LIMIT: allow writes into a full bufq - - NO_SPARES: free empty chunks right away - - a `bufc_pool` that can keep a number of spare chunks to - be shared between different `bufq` instances + Prior to this change and since 8b5f100 (precedes curl 8.0.0), if libcurl + was built without CURL_DEBUG then it could crash during the ssh auth + phase due to a null dereference in the ssh keyboard callback. - Adding `dynhds`: - - a straightforward list of name+value pairs as used for HTTP headers - - headers can be appended dynamically - - headers can be removed again - - headers can be replaced - - headers can be looked up - - http/1.1 formatting into a `dynbuf` - - configured at init() with limits on header counts and total string - sizes - - use case: pass a HTTP request or response around without being version - specific - - express a HTTP request without a curl easy handle (used in h2 proxy - tunnels) - - future extension possibilities: - - conversions of `dynhds` to nghttp2/nghttp3 name+value arrays + Reported-by: Andreas Falkenhahn - Closes #10720 + Fixes https://github.com/curl/curl/pull/11024 + Closes https://github.com/curl/curl/pull/11026 -- pytest: improvements for suitable curl and error output +Daniel Stenberg (26 Apr 2023) - - will check built curl for http and https support and - skip all tests if not there - - will dump stdout/stderr/trace output on errored responses +- docs: clarify that more backends have HTTPS proxy support - Closes #10829 + Closes #11033 -Daniel Stenberg (29 Mar 2023) +- KNOWN_BUGS: remove two not-bugs -- lib: use correct printf flags for sockets and timediffs + - 11.7 signal-based resolver timeouts - Introduces CURL_FORMAT_SOCKET_T for outputting socket numbers. + Not considered a bug anymore but just implementation details. People + should avoid using timeouts with the synchronous name resolver. - Fixes #10737 - Reported-by: Gisle Vanem - Closes #10855 + - 11.16 libcurl uses renames instead of locking for atomic operations -- telnet: make MSVC ignore warning for assignment within conditional + Not a bug, just a description of how it works - Follow-up to d92a5007b60e0af7d + Closes #11032 - Closes #10859 +Harry Sintonen (26 Apr 2023) -- ws: handle reads before EAGAIN better +- hostip: add locks around use of global buffer for alarm() - Reported-by: simplerobot on github - Fixes #10831 - Closes #10856 + When building with the sync name resolver and timeout ability we now + require thread-safety to be present to enable it. -- test1592: add flaky keyword + Closes #11030 - Closes #10860 +Daniel Stenberg (26 Apr 2023) -Frank Gevaerts (28 Mar 2023) +- curl_path: bring back support for SFTP path ending in /~ -- lib/sha256.c: typo fix in comment (duplicated "is available") + libcurl used to do a directory listing for this case (even though the + documentation says a URL needs to end in a slash for this), but + 4e2b52b5f7a3 modified the behavior. - Closes #10851 + This change brings back a directory listing for SFTP paths that are + specified exactly as /~ in the URL. -Arne Soete (28 Mar 2023) + Reported-by: Pavel Mayorov + Fixes #11001 + Closes #11023 -- tests: update tests/httpd references to tests/http +Emanuele Torre (26 Apr 2023) - tests/httpd was renamed to tests/http in #10654. This patch updates some - references in the README +- docs/libcurl/curl_*escape.3: rename "url" argument to "input"/"string" - Closes #10854 + Also reword the DESCRIPTION section to mention "input"/"string" argument + in bold. -Kamil Dudka (28 Mar 2023) + Closes #11027 -- telnet: simplify the implementation of str_is_nonascii() +- docs/libcurl: minor cleanups - There is no need to traverse the string twice. + I was reading curl_unescape(3) and I noticed that there was an extra + space after the open parenthesis in the SYNOPSIS; I removed the extra + space. - Closes #10852 + I also ran a few grep -r commands to find and remove extra spaces + after '(' in other files, and to find and replace uses of `T*' instead + of `T *'. Some of the instances of `T*` where unnecessary casts that I + removed. -Frank Gevaerts (28 Mar 2023) + I also fixed a comment that was misaligned in CURLMOPT_SOCKETFUNCTION.3. -- curl_easy_getinfo.3: typo fix (duplicated "from the") + And I fixed some formatting inconsistencies: in curl_unescape(3), all + function parameter were mentioned with bold text except length, that was + mentioned as 'length'; and, in curl_easy_unescape(3), all parameters + were mentioned in bold text except url that was italicised. Now they are + all mentioned in bold. + Documentation is not very consistent in how function parameter are + formatted: many pages italicise them, and others display them in bold + text; but I think it makes sense to at least be consistent with + formatting within the same page. - Closes #10850 + Closes #11027 -Philip Heiduck (28 Mar 2023) +Daniel Stenberg (26 Apr 2023) -- wolfssl.yml: bump to version 5.6.0 +- man pages: simplify the .TH sections - Closes #10843 + - remove the version numbers + - simplify the texts -Daniel Stenberg (28 Mar 2023) + The date and version number will be put there for releases when maketgz + runs the updatemanpages.pl script. -- RELEASE-NOTES: synced + Closes #11029 -Ronan Pigott (28 Mar 2023) +- hostcheck: fix host name wildcard checking -- docs/cmdline-opts: document the dotless config path + The leftmost "label" of the host name can now only match against single + '*'. Like the browsers have worked for a long time. - The real xdg config path is $XDG_CONFIG_HOME/curlrc, without the dot. - The dotless name seems preferable, so let's match the documentation to - the behavior. + - extended unit test 1397 for this + - move some SOURCE variables from unit/Makefile.am to unit/Makefile.inc - Closes #10849 + Reported-by: Hiroki Kurosawa + Closes #11018 -Daniel Stenberg (28 Mar 2023) +Dan Fandrich (25 Apr 2023) -- HTTP-COOKIES.md: mention the #HttpOnly_ prefix +- smbserver: remove temporary files before exit - Fixes #10847 - Reported-by: Harry Sintonen - Closes #10848 + Each execution of test 1451 would leave a file in /tmp before. Since + Windows can't delete a file while it's open, all the temporary file + names are stored and deleted on exit. -- dynbuf: never allocate larger than "toobig" + Closes #10990 - As dynbufs always have a fixed maximum size which they are not allowed - to grow larger than, making sure that it never allocates a larger buffer - makes sure the buffer does not allocate memory that will never be used. +Stefan Eissing (25 Apr 2023) - Closes #10845 +- Websocket en-/decoding -- ftplistparser: replace realloc with dynbuf + - state is fully kept at connection, since curl_ws_send() and + curl_ws_rec() have lifetime beyond usual transfers + - no more limit on frame sizes - Closes #10844 + Reported-by: simplerobot on github + Fixes #10962 + Closes #10999 -- ftplistparser: use ISDIGIT() +Patrick Monnerat (25 Apr 2023) - Closes #10844 +- urldata: copy CURLOPT_AWS_SIGV4 value on handle duplication -- ftplistparser: move out private data from public struct + Prior to this change STRING_AWS_SIGV4 (CURLOPT_AWS_SIGV4) was wrongly + marked as binary data that could not be duplicated. - The public 'curl_fileinfo' struct contained three fields that are for - internal purposes only. This change makes them unused in the public - struct. + Without this fix, this option's value is not copied upon calling + curl_easy_duphandle(). - The new private struct fields are also renamed to make this separation - more obvious internally. + Closes https://github.com/curl/curl/pull/11021 - Closes #10844 +Stefan Eissing (25 Apr 2023) -- openssl: fix indents - white space edits only +- http3: expire unpaused transfers in all HTTP/3 backends - Closes #10840 + Closes #11005 -- url: remove call to Curl_llist_destroy in Curl_close +- http2: always EXPIRE_RUN_NOW unpaused http/2 transfers - A list that is created with a NULL "destructor" does not need to be - destroyed. Not calling it is faster than calling it. + - just increasing the http/2 flow window does not necessarily + make a server send new data. It may already have exhausted + the window before - Closes #10846 + Closes #11005 -- multi: remove PENDING + MSGSENT handles from the main linked list +- http2: pass `stream` to http2_handle_stream_close to avoid NULL checks - As they are not driving transfers or any socket activity, the main loop - does not need to iterate over these handles. A performance improvement. + Closes #11005 - They are instead only held in their own separate lists. +- h2/h3: replace `state.drain` counter with `state.dselect_bits` - 'data->multi' is kept a pointer to the multi handle as long as the easy - handle is actually part of it even when the handle is moved to the - pending/msgsent lists. It needs to know which multi handle it belongs - to, if for example curl_easy_cleanup() is called before the handle is - removed from the multi handle. + - `drain` was used by http/2 and http/3 implementations to indicate + that the transfer requires send/recv independant from its socket + poll state. Intended as a counter, it was used as bool flag only. + - a similar mechanism exists on `connectdata->cselect_bits` where + specific protocols can indicate something similar, only for the + whole connection. + - `cselect_bits` are cleard in transfer.c on use and, importantly, + also set when the transfer loop expended its `maxloops` tries. + `drain` was not cleared by transfer and the http2/3 implementations + had to take care of that. + - `dselect_bits` is cleared *and* set by the transfer loop. http2/3 + does no longer clear it, only set when new events happen. - Alll 'data->multi' pointers of handles still part of the multi handle - gets cleared by curl_multi_cleanup() which "orphans" all previously - attached easy handles. + This change unifies the handling of socket poll overrides, extending + `cselect_bits` by a easy handle specific value and a common treatment in + transfers. - This is take 2. The first version was reverted for the 8.0.1 release. + Closes #11005 - Assisted-by: Stefan Eissing - Closes #10801 +Daniel Stenberg (25 Apr 2023) -Stefan Eissing (26 Mar 2023) +- socketpair: verify with a random value -- tests/http: add timeout to running curl in test cases + ... instead of using the curl time struct, since it would use a few + uninitialized bytes and the sanitizers would complain. This is a neater + approach I think. - - we had a CI case once where `curl` seemingly did not - return and it was hard to guess what happened. - - make curl execution in test cases time out after 60 seconds + Reported-by: Boris Kuschel + Fixes #10993 + Closes #11015 - Closes #10783 +Stefan Eissing (25 Apr 2023) -Daniel Stenberg (26 Mar 2023) +- HTTP3: document the ngtcp2/nghttp3 versions to use for building curl -- RELEASE-PROCEDURE: update to new schedule + - refs #11011 to clarify this for people building curl themselves - Ref: https://curl.se/mail/lib-2023-03/0062.html + Closes #11019 - Assisted-by: Andy Alt - Assisted-by: Dan Frandrich +Daniel Stenberg (25 Apr 2023) - Closes #10827 +- lib: unify the upload/method handling -Patrick Monnerat (26 Mar 2023) + By making sure we set state.upload based on the set.method value and not + independently as set.upload, we reduce confusion and mixup risks, both + internally and externally. -- doc: curl_mime_init() strong easy handle binding has been relaxed in 7.87.0 + Closes #11017 - Reported-by: Chloe Kudryavtsev - Fixes #10834 - Closes #10835 +- RELEASE-NOTES: synced -Jay Satiro (25 Mar 2023) +Dan Fandrich (24 Apr 2023) -- CURLOPT_WRITEFUNCTION.3: fix typo +- CI: don't run CI jobs if only another CI was changed - Reported-by: Osaila@users.noreply.github.com + A few paths were missed in the last commit, as well as a job added since + then. - Fixes https://github.com/curl/curl/issues/10839 + Followup-to 395b9175 -Dan Fandrich (24 Mar 2023) +- CI: adjust labeler match patterns -- CI: skip some more builds when possible +- runtests: support buffering log messages in runner & servers - When a commit only contains tests, documentation, or cmake files, skip - those builds that aren't affected by those. + Log messages generated with logmsg can now be buffered and returned from + the runner as a return value. This will be needed with parallel testing + to allow all messages for one test to be displayed together instead of + interspersed with messages of multiple tests. Buffering can be disabled + by setting a logging callback function with setlogfunc, which is + currently being done to preserve existing logging behaviour for now. - The file filters available on the CI services don't seem to allow - skipping individual jobs, only the entire workflow, so we can't get any - more fine-grained than this. + Some additional output is generated in verbose and debugprotocol modes, + which don't always use logmsg. These modes also impact some servers + which generate extra messages. No attempt is made to buffer everything + if these modes are enabled. -- CI: add and adjust labeler match patterns + Ref: #10818 + Closes #11016 - Allow cmdline tool alongside other labels. +- runtests: more consistently use logmsg in server control code -Kai Pastor (25 Mar 2023) + Also, display an error when sshversioninfo returns one. -- CMake: make config version 8 compatible with 7 + Ref: #10818 - Reviewed-by: Jakub Zakrzewski - Closes #10819 +- runtests: create runner functions for clearlocks and stopservers -Daniel Stenberg (24 Mar 2023) + runtests.pl now uses runner for all server actions beyond the initial + variable configuration. -- RELEASE-NOTES: synced + Ref: #10818 - Bumped version-in-progress to 8.1.0 +- runtests: tightened servers package exports -- GHA: add a memory-sanitizer job + The defaults are intended for runtests.pl, whereas runner.pm needs to + explicitly specify them. - Closes #10815 +- runtests: display logs on server failure in singletest() -Dan Fandrich (23 Mar 2023) + This is closer to the place where logs are displayed on test failure. + Also, only display these logs if -p is given, which is the same flag + that controls display of test failure logs. Some server log files + need to be deleted later so that they stay around long enough to be + displayed on failure. -- CI: fix brew retries on GHA + Ref: #10818 - The fix in the previous commit was complete for Cirrus but accidentally - left off a part for GHA. +- runtests: turn a print into a logmsg - Follow-up to c2b7249d + Also enable another couple of useful messages in verbose mode. -- CI: skip Azure for more commits which change only GHA + Ref: #10818 -Daniel Stenberg (23 Mar 2023) +Daniel Stenberg (24 Apr 2023) -- cmake: set SONAME for SunOS too +- http: store the password in the correct variable - Provided-by: Brian Lund + Typo from fc2f1e547a4a, detected by Coverity (because there's dead code + due to this). - Closes #10816 + Closes #11002 -Stefan Eissing (23 Mar 2023) +Stefan Eissing (24 Apr 2023) -- ngtcp2: adjust config and code checks for ngtcp2 without nghttp3 +- HTTP3/quiche: terminate h1 response header when no body is sent - - make configure show on HTTP3 feature that both ngtcp2 and nghttp3 - are in play - - define ENABLE_QUIC only when USE_NGTCP2 and USE_NGHTTP3 are defined - - add USE_NGHTTP3 in the ngtcp2 implementation + - fixes a failure in test2501 where a response without body was missing + the final empty line - Fixes #10793 - Closes #10821 + Closes #11003 -Daniel Stenberg (23 Mar 2023) +Dan Fandrich (22 Apr 2023) -- data.d: emphasize no conversion +- runtests: move showdiff into runtests.pl - When asking curl to send a POST, curl does not encode or change the data. + It's not used anywhere else. - Ref: #10820 - Closes #10823 +- devtest: add a new script for testing the test harness -- server/getpart: clear the buffer before load + This is currently useful for starting a test server on its own without + an associated test, which can be used for interactive curl testing or + for validating parts of the test harness itself. More commands can be + added to perform additional functions in the future. - Fixes msan warnings: + Ref: #10818 + Closes #11008 - ==54195==WARNING: MemorySanitizer: use-of-uninitialized-value - #0 0x55ece35e57cb in line_length /home/runner/work/curl/curl/tests/server - /getpart.c:111:25 - #1 0x55ece35e3b83 in readline /home/runner/work/curl/curl/tests/server/ge - tpart.c:164:24 - #2 0x55ece35e0269 in getpart /home/runner/work/curl/curl/tests/server/get - part.c:347:18 - #3 0x55ece36180b6 in parse_servercmd /home/runner/work/curl/curl/tests/se - rver/sws.c:283:13 +- runtests: refactor the main test loop into two - Closes #10822 + The test loop now has an initial loop that first runs through all + possible tests to build a set of those to attempt on this run based on + features and keywords and only then goes through that new list to run + them. This actually makes it three loops through all tests cases, as + there is an existing loop that gathers possible test numbers from the + test files on disk. -- ntlm: clear lm and nt response buffers before use + This has two minor effects on the output: all the tests that will be + skipped are displayed at the start (instead of being interspersed with + other tests) and the -l option no longer shows a count of tests at the + end or a (misleading) statement that tests have run successfully. The + skipped tests are also omitted from the test results sent to AppVeyor + and Azure in CI builds. - To avoid the risk of MemorySanitizer: use-of-uninitialized-value + Another effect is a reduction in the amount of work considered part of + the "Test definition reading and preparation time" reported with -r + making those figures slightly lower than before. - Closes #10814 + Ref: #10818 -- digest: clear target buffer +- runtests: track only the current test timings in runner.pm - Closes #10814 + This avoids passing these data through through global variables, which + soon won't be possible. -Douglas R. Reno (22 Mar 2023) + Ref: #10818 -- cmake: bring in the network library on Haiku. +- runtests: skip test preprocessing when doing -l - When cross-compiling for Haiku, the networking library needs to be - brought in. Without this, an unknown type of "Error" is reported in - lib/curl_setup_once.h. + This speeds up the output tremendously by avoiding unnecessary work. - This is also needed when using CMake natively on Haiku to build libcurl. +- runtests: simplify value returned regarding use of valgrind - Fixes #10296 - Closes #10792 + As a side effect this will now also show in verbose mode that valgrind + is being skipped on tests that explicitly disable it, such as 600. -Daniel Stenberg (22 Mar 2023) + Ref: #10818 -- runtests: die if curl version can be found +- runtests: fix quoting in Appveyor and Azure test integration - Closes #10813 + Test 1442's name was not quoted correctly so wasn't registered in + Appveyor and it had the wrong name in Azure. The JSON string quotes were + also invalid, even though both servers happened to accept it regardless. -Stefan Eissing (22 Mar 2023) + Closes #11010 -- multi: add handle asserts in DEBUG builds +Daniel Stenberg (19 Apr 2023) - For GOOD_EASY_HANDLE and GOOD_MULTI_HANDLE checks +- RELEASE-NOTES: synced - - allow NULL pointers to "just" return an error as before - - fail hard on nun-NULL pointers that no longer show the MAGICs +Dan Fandrich (18 Apr 2023) - Closes #10812 +- runtests: spread out the port numbers used by servers -Jon Rumsey (22 Mar 2023) + The server ports are chosen randomly for each server, but the random + ranges chosen were inconsistently-sized and overlapping. Now, they are + spread out more so at least the first random port chosen for each server + is guaranteed to not also be chosen by another server. The starting port + numbers are also raised to put them in the Ephemeral Port range—not the + range defined by RFC 6335 but the one used by Linux, which starts lower + and gives us more room to work with. -- gskit: various compile errors in OS400 + Reported-by: Daniel Stenberg - Various compile failures in gskit.c; +- runtests: fix problems on failure - - pipe_ssloverssl() needs Curl_easy data parameter for - Curl_conn_cf_get_socket(cf, data) - - key_passwd is in ssl_config, not conn_config - - close_on() has 2 parameters, not 4 - - getsockopt() needs to call Curl_conn_cf_get_socket(), not - cxn->sock[FIRSTSOCKET] + The verify time must be set in this case, like all cases. An error + message needs to be displayed as well. - Fixes #10799 - Closes #10800 +- runtests: fix perl warning when is wrong -Daniel Stenberg (22 Mar 2023) +- runtests: don't try to stop stunnel before trying again -- tool_operate: pass a long as CURLOPT_HEADEROPT argument + Calling stopserver() before retrying stunnel due to an error would stop + the dependent server (such as HTTP) meaning stunnel would have nothing + to talk to when it came up. Don't try to force a stop when it didn't + actually start. Also, don't mark the server as bad for future use when + it starts up on a retry. - Closes #10798 + Reported-by: eaglegai at github + Tested-by: eaglegai at github + Fixes #10976 -- GHA: run all linux test jobs with valgrind +- runtests: don't accidentally randomly choose the same port - Closes #10798 + If a server couldn't be started on a port, a new one is randomly chosen + and the server is tried again. Avoid accidentally using a + randomly-chosen 0 port offset by adding 1 to the random number. -- GHA-linux: add an address-sanitizer build + Found-by: Daniel Stenberg - Closes #10810 +- runtests: don't attempt to use a port we know is in use -Version 8.0.1 (20 Mar 2023) + This reduces the startup time when there is a known conflict on the + random port chosen for a server. This was already done for stunnel, but + now it's done for all servers. -Daniel Stenberg (20 Mar 2023) +- http-server: fix server name in a log message -- RELEASE-NOTES: synced + This changed when the file was renamed in commit cbf57176 - curl 8.0.1 +- runtests: refactor into more packages -- Revert "multi: remove PENDING + MSGSENT handles from the main linked list" + testutil.pm now contains a few miscellaneous functions that are used in + several places but have no better place to live. subvariables moves to + servers.pm since most variables that it substitutes relate to servers, + so this is the most appropriate place. Rename a few functions for better + naming consistency. - This reverts commit f6d6f3ce01e377932f1ce7c24ee34d45a36950b8. + Ref: #10818 + Closes #10995 - The commits caused issues in the 8.0.0 release. Needs a retake. +- runtests: call timestampskippedevents() in singletest - Reported-by: Kamil Dudka - Closes #10795 + ..rather than by the runner -- include/curl/curlver.h: bump to 8.0.1 +- runtests: assume a newer Valgrind by default -Version 8.0.0 (20 Mar 2023) + The tests for an older Valgrind version should probably just be deleted, + given that they're testing for an 18-year-old version. -Daniel Stenberg (20 Mar 2023) +- runtests: refactor test runner code into runner.pm -- RELEASE-NOTES: synced + This is code that is directly responsible for running a single test. + This will eventually run in a separate process as part of the parallel + testing project. - The curl 8.0.0 release + Ref: #10818 -- THANKS: from the 8.0.0 release +- runtests: skip unneeded work if test won't be running -- scripts/delta: fix "ambiguous argument" when used in branches + This speeds up tests by avoiding unnecessary processing. -- SECURITY-PROCESS.md: Busy-loops are not security problems + Ref: #10818 - Closes #10790 +- runtests: factor out singletest_postcheck -Stefan Eissing (17 Mar 2023) + This will eventually need to be part of the test runner. -- tests/http: do not save files for downloads in scorecard testing + Ref: #10818 - Closes #10788 +- test303: kill server after test -Daniel Stenberg (17 Mar 2023) + Otherwise, an HTTP test closely following this one with a tight time + constraint (e.g. 672) could fail because the test server stays sitting + with the wait command for a while. -- cf-socket: use port 80 when resolving name for local bind +Patrick Monnerat (18 Apr 2023) - It turns out c-ares returns an error when asked to resolve a host name with - ares_getaddrinfo using port number 0. +- OS400: provide ILE/RPG usage examples - Reported as a c-ares bug here: https://github.com/c-ares/c-ares/issues/517 + Closes https://github.com/curl/curl/pull/10994 - The work-around is to simply use port 80 instead, as the number typically doe - s - not make a difference and a non-zero number works for c-ares. +- OS400: improve vararg emulation - Fixes #10759 - Reported-by: Matt Jolly - Closes #10789 + - Use V7R4 RPG procedure overloading to improve vararg emulation. -- curl.h: require gcc 12.1 for the deprecation magic + From OS400 V7R4 and above, ILE/RPG implements a limited procedure + overloading feature that can be used to improve curl's typed + implementation of varargs procedures. This commit applies it to + curl_easy_setopt(), curl_multi_setopt(), curl_share_setopt() and + curl_easy_getinfo(). - Reported-by: kchow-FTNT on github - Fixes #10726 - Closes #10784 + Closes https://github.com/curl/curl/pull/10994 -- Revert "rtsp: use dynbuf instead of custom reallocs" +- OS400: fix and complete ILE/RPG binding - This reverts commit 1b9ea3239d22147e00d8 because of OSS-fuzz reports. - I'll do another take after the pending release. + - Fix wrong definitions of CURL_ZERO_TERNINATED, curl_mime_data() and + curl_mime_data_ccsid(). - Closes #10785 + - Add recent definitions, in particular blob, header API and WebSockets + API. -- test422: verify --next used without a prior URL + - Support for CURLVERSION_ELEVENTH. - Closes #10782 + - New functions for EBCDIC support. -- tool_getparam: error if --next is used without a prior URL + Reflect these changes in README.OS400. - Reported-by: 積丹尼 Dan Jacobson - Ref: https://github.com/curl/curl/pull/10769#discussion_r1137895629 + Closes https://github.com/curl/curl/pull/10994 - Closes #10782 +- OS400: implement EBCDIC support for recent features -- libssh: use dynbuf instead of realloc + - Support CURLVERSION_ELEVENTH. - When building lines to show for SFTP directory listings. + - New function curl_url_strerror_ccsid(). - Closes #10778 + - curl_easy_setopt_ccsid() supports blobs and 3 recent string options. -- lib2305: deal with CURLE_AGAIN + - New function curl_easy_header_ccsid(). - The test does a slightly ugly busy-loop for this case but should be - managable due to it likely being a very short moment. + - New generic latin1<-->ccsid conversion functions curl_from_ccsid() and + curl_to_ccsid() for user convenience. - Mention CURLE_AGAIN in curl_ws_recv.3 + - README.OS400 updated accordingly. - Fixes #10760 - Reported-by: Jay Satiro - Closes #10781 + - Removed a leftover QsoSSL support identifier. -- rtsp: use dynbuf instead of custom reallocs + Closes https://github.com/curl/curl/pull/10994 - For the RTP buffering. +- OS400: rework build scripts - Closes #10776 + - Rename shell function "system" to "CLcommand" to avoid confusion with + built-in command. -- libssh2: remove unused variable from libssh2's struct + - Reformat scripts. Fix some indentations. Avoid lines > 80 characters + where possible. - Closes #10777 + - Support ASCII runtime development files in a user-defined directory + path. -- RELEASE-NOTES: synced + - FIX SONAME detection. -- multi: remove PENDING + MSGSENT handles from the main linked list + - Drop form API test program compilation (does not exist anymore). - As they are not driving transfers or any socket activity, the main loop - does not need to iterate over these handles. A performance improvement. + Closes https://github.com/curl/curl/pull/10994 - They are instead only held in their own separate lists. +Sevan Janiyan (18 Apr 2023) - Assisted-by: Stefan Eissing - Ref: #10743 - Closes #10762 +- tests/sshserver.pl: Define AddressFamily earlier -- multi: turn link/unlinking easy handles into dedicated functions + As the comment states "Address family must be specified before ListenAddress" + , otherwise the tests fail to run + `"failed starting SSH server" 52 times (582, 583, 600, 601, 602, 603, 604, 60 + 5, 606 and 43 more)` -- http_aws_sigv4: fix scan-build "value stored to 'ret' is never read" + Closes #10983 - Follow-up to 495d09810aa9a +Stefan Eissing (18 Apr 2023) - Closes #10766 +- quiche: Enable IDLE egress handling -- lib: skip Curl_llist_destroy calls + Follow-up to 544abeea which added the handling but wrongly left it + commented out. - Linked lists themselves do not carry any allocations, so for the lists - that do not have have a set destructor we can just skip the - Curl_llist_destroy() call and save CPU time. + Closes https://github.com/curl/curl/pull/11000 - Closes #10764 +Daniel Stenberg (18 Apr 2023) -- lib643: LIB644 is never defined, this is dead code +- docs/examples/protofeats.c: Outputs all protocols and features - Closes #10765 + Showing off one way to get to char pointer arrays of info returned by + curl_version_info() -- libtest/Makefile.inc: remove superfluous variables + Closes #10991 - Rely on the defaults when possible. +- tests/keywords.pl: remove - Closes #10765 + This script does not work since the introduction of the test + preprocessing. If we need this functionality, it probably needs to be + moved into the runtests tool or similar. -- tests/http: remove year ranges from copyrights + Reported-by: Dan Fandrich + Fixes #10895 + Closes #10987 - Closes #10763 +Stefan Eissing (17 Apr 2023) -Casey Bodley (14 Mar 2023) +- http2: support HTTP/2 to forward proxies, non-tunneling -- aws_sigv4: fall back to UNSIGNED-PAYLOAD for sign_as_s3 + - with `--proxy-http2` allow h2 ALPN negotiation to + forward proxies + - applies to http: requests against a https: proxy only, + as https: requests will auto-tunnel + - adding a HTTP/1 request parser in http1.c + - removed h2h3.c + - using new request parser in nghttp2 and all h3 backends + - adding test 2603 for request parser + - adding h2 proxy test cases to test_10_* - all s3 requests default to UNSIGNED-PAYLOAD and add the required - x-amz-content-sha256 header. this allows CURLAUTH_AWS_SIGV4 to correctly - sign s3 requests to amazon with no additional configuration + scorecard.py: request scoring accidentally always run curl + with '-v'. Removed that, expect double numbers. - Signed-off-by: Casey Bodley + labeller: added http1.* and h2-proxy sources to detection - Closes #9995 + Closes #10967 -Viktor Szakats (14 Mar 2023) +Daniel Stenberg (17 Apr 2023) -- wolfssl: add quic/ngtcp2 detection in cmake, and fix builds +- curl_easy_unescape.3: rename the argument - - add QUIC/ngtcp2 detection in CMake with wolfSSL. + and highlight it appropriately in the text. - Because wolfSSL uses zlib if available, move compression detection - before TLS detection. (OpenSSL might also need this in the future.) + Closes #10979 - - wolfSSL 5.5.0 started using C99 types in its `quic.h` header, but it - doesn't #include the necessary C99 header itself, breaking builds - (unless another dependency pulled it by chance.) Add local workaround - for it. For this to work with all build tools, we had to fix our - header detection first. Ref: #10745 +Viktor Szakats (17 Apr 2023) - Ref: https://github.com/curl/curl-for-win/commit/6ad5f6ecc15620c15625fc4434 - 76b3a1ecef4f3f +- autotools: sync up clang picky warnings with cmake - Closes #10739 + Bringing missing options over from CMake. -Stefan Eissing (14 Mar 2023) + Move around existing `-Wno-pointer-bool-conversion` option to come + _after_ `-Wconversion`. -- secure-transport: fix recv return code handling + Reviewed-by: Marcel Raad + Closes #10974 - Return code handling of recv calls were not always correct when an error - occured or the connection was closed. +Daniel Stenberg (17 Apr 2023) - Closes #10717 +- tests/libtest/lib1900.c: remove -- http2: Use KEEP_SEND_HOLD for flow control in HTTP/2 + This file was left behind when the rest of the test was previously removed. - - use the defined, but so far not used, KEEP_SEND_HOLD bit for flow - control based suspend of sending in transfers. + Follow-up to e50a877df74f - Prior to this change KEEP_SEND_PAUSE bit was used instead, but that can - interfere with pausing streams from the user side via curl_easy_pause. +- src/tool_operhlp.c: fix value stored to 'uerr' is never read - Fixes https://github.com/curl/curl/issues/10751 - Closes https://github.com/curl/curl/pull/10753 + Ref: https://github.com/curl/curl/pull/10974#issuecomment-1510461343 + Reported-by: Viktor Szakats + Closes #10982 -Dan Fandrich (13 Mar 2023) +Viktor Szakats (16 Apr 2023) -- tests: fix control code that hid some text in runtests.1 +- cmake: speed up and extend picky clang/gcc options -- tests: sync option lists in runtests.pl & its man page + Extend existing picky compiler options with ones missing compared to + autotools builds. Also sync options between clang and gcc. -Daniel Stenberg (13 Mar 2023) + Redesign the way we enable these options to avoid the slow option + detection almost completely. -- multi: make multi_perform ignore/unignore signals less often + This reduces the number of detections from 35 to zero for clang and + 3 for gcc, even after adding a bunch of new options. - For improved performance + clang 3.0 (2011-11-29) and gcc 2.95 (1999-07-31) now required. - Reported-by: Jerome St-Louis - Ref: #10743 - Closes #10750 + Also show enabled picky options. -Viktor Szakats (13 Mar 2023) + Ref: https://github.com/libssh2/libssh2/pull/952 -- cmake: delete unused HAVE__STRTOI64 + Reviewed-by: Daniel Stenberg + Closes #10973 - Also delete obsolete surrounding comments. +Andreas Falkenhahn (16 Apr 2023) - Reviewed-by: Daniel Stenberg - Closes #10756 +- nbtlm: use semicolons instead of commas for (void) args -- CI: fix copyright header + Closes #10978 - Follow-up to 395b9175b7422d699fa93643973295c106cdf147 +Daniel Stenberg (15 Apr 2023) -Daniel Stenberg (13 Mar 2023) +- multi: free up more data earleier in DONE -- RELEASE-PROCEDURE.md: update coming release dates + Before checking for more users of the connection and possibly bailing + out. -Stefan Eissing (13 Mar 2023) + Fixes #10971 + Reported-by: Paweł Wegner + Closes #10972 -- tests/http: add pytest to GHA and improve tests +- RELEASE-NOTES: synced - - added to: ngtcp2-quictls, ngtcp2-gnutls and the linux varians - quiche, bearssl, libressl, mbedtls, openssl3, rustls - - added disabled in ngtcp2-wolfssl due to weird SSL_connect() errors - not reproducable locally +- curl: do NOT append file name to path for upload when there's a query - Improvements on pytest: + Added test 425 to verify. - - handling of systems with nghttpx in $PATH - - configure will seach $PATH got nghttpx used in pytest - - pytest fixes for managing nghttpx without h3 support - - ngtcp2-wolfssl: use a fully enabled wolfssl build + Reported-by: Dirk Rosenkranz + Bug: https://curl.se/mail/archive-2023-04/0008.html + Closes #10969 - - lower parallel count for http/1.1 tests, since we do not - want to test excessive connections. - - check built curl for HTTPS-proxy support in proxy tests - - bearssl does not like one of our critical cert extensions, making - it non-critical now - - bearssl is too slow for test_12, skipping - - making sure we do h3 tests only when curl and server support is there +- libcurl-thread.3: improved name resolver wording - Closes #10699 + And make better .SH sections -Marcel Raad (13 Mar 2023) + Closes #10966 -- tool_operate: silence unused parameter warning +Colman Mbuya (14 Apr 2023) - `global` is only used in the `my_setopt` macro version without - `CURL_DISABLE_LIBCURL_OPTION` since commit 4774decf10a. +- CURLOPT_PROXY_SSL_VERIFYPEER.3: fix minor grammar mistake - Closes https://github.com/curl/curl/pull/10752 + Closes #10968 -Viktor Szakats (13 Mar 2023) +Daniel Stenberg (14 Apr 2023) -- build: fix stdint/inttypes detection with non-autotools +- curl: add --proxy-http2 - Fix `stdint.h` and `inttypes.h` detection with non-autotools builds on - Windows. (autotools already auto-detected them accurately.) + For trying HTTP/2 with an HTTPS proxy. - `lib/config-win32.h` builds (e.g. `Makefile.mk`): - - set `HAVE_STDINT_H` where supported. - - set `HAVE_INTTYPES_H` for MinGW. + Closes #10926 - CMake: - - auto-detect them on Windows. (They were both force-disabled.) - - delete unused `CURL_PULL_STDINT_H`. - - delete unused `CURL_PULL_INTTYPES_H`. - - stop detecting `HAVE_STDINT_H` twice. - Present since the initial CMake commit: 4c5307b45655ba75ab066564afdc0c111a8 - b9291 +- KNOWN_BUGS: remove fixed or outdated issues, move non-bugs - curl doesn't use these C99 headers, we need them now to workaround - broken wolfSSL builds. Ref: #10739 + - remove h3 issues believed to be fixed - Once that clears up, we can delete these detections and macros (unless - we want to keep them for future us.) + - make the flaky CI issue be generic and not Windows specific - Reviewed-by: Daniel Stenberg - Closes #10745 + - "TLS session cache does not work with TFO" now documented -Daniel Stenberg (13 Mar 2023) + This is now a documented restriction and not a bug. TFO in general is + rarely used and has other problems, making it a low-priotity thing to + work on. -- RELEASE-NOTES: synced + - remove "Renegotiate from server may cause hang for OpenSSL backend" -- ftp: add more conditions for connection reuse + This is an OpenSSL issue, not a curl one. Even if it taints curl. - Reported-by: Harry Sintonen - Closes #10730 + - rm "make distclean loops forever" -Dan Fandrich (12 Mar 2023) + - rm "configure finding libs in wrong directory" -- tests: make first.c the same for both lib tests and unit tests + Added a section to docs/INSTALL.md about it. - The only difference used to be global variable used in unittest tests. - After cb7ed5a removed individual flag overrides for the unittests, first.c - was no longer recompiled for unit tests to include the flag, so whether it - worked or gave a link error depended on whether it was compiled in - libtest or unittest first. This way also speeds up the build by - eliminating 40 identical compile invocations. + - "A shared connection cache is not thread-safe" - Fixes #10749 + Moved over to TODO and expanded for other sharing improvements we + could do -- tests: use AM_CPPFILES to modify flags in unit tests + - rm "CURLOPT_OPENSOCKETPAIRFUNCTION is missing" - Using CPPFLAGS sometimes caused odd compile issues when building tests - with parallel make and AM_CPPFILES is the right flag, anyway. + - rm "Blocking socket operations in non-blocking API" - Follow-up to cb7ed5a + Already listed as a TODO - Ref #10749 + - rm "curl compiled on OSX 10.13 failed to run on OSX 10.10" -Viktor Szakats (13 Mar 2023) + Water under the bridge. No one cares about this anymore. -- Makefile.mk: fix -g option in debug mode [ci skip] + - rm "build on Linux links libcurl to libdl" - Add it to `CFLAGS` (was: `LDFLAGS`). + Verified to not be true (anymore). - Closes #10747 + - rm "libpsl is not supported" -Jay Satiro (12 Mar 2023) + The cmake build supports it since cafb356e19cda22 -- tool: improve --stderr handling + Closes #10963 - - freopen stderr with the user-specified file (--stderr file) instead of - using a separate 'errors' stream. +- url: fix PVS nits - - In tool_setup.h override stdio.h's stderr macro as global variable - tool_stderr. + - expression 'hostptr' is always true + - a part of conditional expression is always true: proxypasswd + - expression 'proxyuser' is always true + - avoid multiple Curl_now() calls in allocate_conn - Both freopen and overriding the stderr macro are necessary because if - the user-specified filename is "-" then stdout is assigned to - tool_stderr and no freopen takes place. See the PR for more information. + Ref: #10929 + Closes #10959 - Ref: https://github.com/curl/curl/issues/10491 +- bufq: simplify since expression is always true - Closes https://github.com/curl/curl/pull/10673 + The check for 'len' is already done so it will remain true until + updated. Pointed out by PVS. -Dan Fandrich (11 Mar 2023) + Ref: #10929 + Closes #10958 -- CI: don't run CI jobs if only another CI was changed +- hash: fix assigning same value - Also skip builds on non-Windows platforms when only Windows build files - have changed. + Pointed out by PVS - This should reduce the number of useless builds and the associated - waiting time and chance of spurious failures, freeing resources for - new PRs. + Ref: #10929 + Closes #10956 - Closes #10742 +- cookie: address PVS nits -- http: don't send 100-continue for short PUT requests + - avoid assigning the same value again + - remove superfluous check of co->domain + - reduce variable scope for namep/valuep - This is already how curl is documented to behave in Everything curl, but - in actuality only short POSTs skip this. This should knock 30 seconds - off a full run of the test suite since the 100-continue timeout will no - longer be hit. + Ref: #10929 + Closes #10954 - Closes #10740 +Stefan Eissing (14 Apr 2023) -- tests: add DELAY keyword to more tests using waits +- cf-socket: Disable socket receive buffer by default -- tests: hack to build most unit tests under cmake + - Disable socket receive buffer unless USE_RECV_BEFORE_SEND_WORKAROUND + is in place. - These are only built when a libcurl static library is available, since - we're not building a special libcurlu library yet and these tests rely - on private symbols that aren't available in the shared library. A few - unit tests do require libcurlu, so those are not built. + While we would like to use the receive buffer, we have stalls in + parallel transfers where not all buffered data is consumed and no socket + events happen. - Closes #10722 + Note USE_RECV_BEFORE_SEND_WORKAROUND is a Windows sockets workaround + that has been disabled by default since b4b6e4f1, due to other bugs. -- tests: fix MSVC unreachable code warnings in unit tests + Closes https://github.com/curl/curl/pull/10961 - Switch unit1654 to use the proper test macros as well. +- cf-h2-proxy: fix processing ingress to stop too early -- tests: make CPPFLAGS common to all unit tests + - progress ingress stopped too early, causing data + from the underlying filters to not be processed and + report that no tunnel data was available + - this lead to "hangers" where no socket activity was + seen but data rested in buffers - There's no need to specify them individually. + Closes #10952 -- tests: keep cmake unit tests names in sync +- http3: check stream_ctx more thoroughly in all backends - Put only the test names into Makefile.inc so they can be used by both - cmake and automake. This will prevent the list of tests from becoming - out of date when they are also built under cmake. + - callbacks and filter methods might be invoked at unexpected + times, e.g. when the transfer's stream_ctx has not been initialized + yet or, more likely, has already been taken down. + - check for existance of stream_ctx in such places and return + an error or silently succeed the call. -Viktor Szakats (11 Mar 2023) + Closes #10951 -- src: silence wmain() warning for all build methods +Daniel Stenberg (13 Apr 2023) - llvm/clang and gcc doesn't recognize the wmain() function in Unicode - Windows builds: +- ftp: fix 'portsock' variable was assigned the same value - llvm/clang: - ``` - ../../src/tool_main.c:239:5: warning: no previous prototype for function 'wma - in' [-Wmissing-prototypes] - int wmain(int argc, wchar_t *argv[]) - ^ - 1 warning generated. - ``` + Pointed out by PVS - gcc: - ``` - ../../src/tool_main.c:239:5: warning: no previous prototype for 'wmain' [-Wmi - ssing-prototypes] - 239 | int wmain(int argc, wchar_t *argv[]) - | ^~~~~ - ``` + Ref: #10929 + Closes #10955 - Before this patch, we already silenced it with CMake. This patch moves - the silencing to the source, so that it applies to all build tools. +- ftp: remove dead code - Bug: https://github.com/curl/curl/issues/7229#issuecomment-1464806651 + This condition can never be true here since it is handled already 28 + lines above. - Reviewed-by: Marcel Raad - Closes #10744 + Pointed out by PVS. -Dan Fandrich (10 Mar 2023) + Ref: #10929 + Closes #10957 -- CI: fix retrying on brew failures +- cf-h1-proxy: skip an extra NULL assign - The previous attempt didn't consider that the shell would exit - immediately after the false statement in the retry case. + and use Curl_safefree() once to save another NULL assign. Found by PVS. - Follow-up to dc141a37 + Ref. #10929 + Closes #10953 -Stefan Eissing (10 Mar 2023) +Philip Heiduck (13 Apr 2023) -- http2: fix error handling during parallel operations +- GHA: suppress git clone output - RST and connection close were not handled correctly during parallel - transfers, leading to aborted response bodies being reported complete. + Follow-up: https://github.com/curl/curl/commit/8203aa6ed405ec832d2c62f18dfda2 + 93f89a23f9 - Closes #10715 + Closes #10949 -Daniel Stenberg (10 Mar 2023) +Stefan Eissing (13 Apr 2023) -- url: only reuse connections with same GSS delegation +- cf-socket: remove dead code discovered by PVS - Reported-by: Harry Sintonen - Closes #10731 + Closes #10960 -Viktor Szakats (10 Mar 2023) +Daniel Stenberg (13 Apr 2023) -- lib: silence clang/gcc -Wvla warnings in brotli headers +- http: skip a double NULL assign - brotli v1.0.0 throughout current latest v1.0.9 and latest master [1] - trigger this warning. + and also use a local variable to shorten the long names and increase + readability in the function. Pointed out by PVS. - It happened with CMake and GNU Make. autotools builds avoid it with - the `convert -I options to -isystem` macro. + Ref: #10929 + Closes #10950 - llvm/clang: - ``` - In file included from ./curl/lib/content_encoding.c:36: - ./brotli/x64-ucrt/usr/include/brotli/decode.h:204:34: warning: variable lengt - h array used [-Wvla] - const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], - ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - ./brotli/x64-ucrt/usr/include/brotli/port.h:253:34: note: expanded from macro - 'BROTLI_ARRAY_PARAM' - ^~~~~~ - In file included from ./curl/lib/content_encoding.c:36: - ./brotli/x64-ucrt/usr/include/brotli/decode.h:206:48: warning: variable lengt - h array used [-Wvla] - uint8_t decoded_buffer[BROTLI_ARRAY_PARAM(*decoded_size)]); - ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~ - ./brotli/x64-ucrt/usr/include/brotli/port.h:253:35: note: expanded from macro - 'BROTLI_ARRAY_PARAM' - ~^~~~~ - ``` +- mime: skip NULL assigns after Curl_safefree() - gcc: - ``` - In file included from ./curl/lib/content_encoding.c:36: - ./brotli/x64-ucrt/usr/include/brotli/decode.h:204:5: warning: ISO C90 forbids - variable length array 'encoded_buffer' [-Wvla] - 204 | const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], - | ^~~~~ - ./brotli/x64-ucrt/usr/include/brotli/decode.h:206:5: warning: ISO C90 forbids - variable length array 'decoded_buffer' [-Wvla] - 206 | uint8_t decoded_buffer[BROTLI_ARRAY_PARAM(*decoded_size)]); - | ^~~~~~~ - ``` + Pointed out by PVS. - [1] https://github.com/google/brotli/commit/ed1995b6bda19244070ab5d331111f16f - 67c8054 + Ref: #10929 + Closes #10947 - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #10738 +- rtsp: skip NULL assigns after Curl_safefree() -Daniel Stenberg (10 Mar 2023) + ... since this is a macro that assigns NULL itself. Pointed out by PVS. -- curl_path: create the new path with dynbuf + Ref: #10929 + Closes #10946 - Closes #10729 +- smb: remove double assign -- url: remove dummy protocol handler + The same value is assigned the same value already a few lines above. + Pointed out by PVS. - Just two added checks were needed saves a whole handler struct. + Ref: #10929 + Closes #10945 - Closes #10727 +- transfer: skip extra assign -Dan Fandrich (10 Mar 2023) + The 'result' variable already contains CURLE_OK at this point, no use in + setting it again. Pointed out by PVS. -- CI: retry a failed brew update too, not just brew install + Ref: #10929 + Closes #10944 - Also, make sure an eventual failure ends up returning a failure code so - the job stops. +- urlapi: skip a pointless assign -Daniel Stenberg (10 Mar 2023) + It stores a null byte after already having confirmed there is a null + byte there. Detected by PVS. -- url: fix the SSH connection reuse check + Ref: #10929 + Closes #10943 - Reported-by: Harry Sintonen - Closes #10735 +Philip Heiduck (13 Apr 2023) -- CURLOPT_PROXY.3: curl+NSS does not handle HTTPS over unix domain socket +- GHA: suppress git clone output - It results in error "NSS error -5985 (PR_ADDRESS_NOT_SUPPORTED_ERROR)" + Closes #10939 - Disabled test 1470 for NSS builds and documented the restriction. +Stefan Eissing (13 Apr 2023) - Reported-by: Dan Fandrich - Fixes #10723 - Closes #10734 +- tests: make test_12_01 a bit more forgiving on connection counts -- CURLSHOPT_SHARE.3: HSTS sharing is not thread-safe +- cf-socket: add socket recv buffering for most tcp cases - Reported-by: Hiroki Kurosawa - Closes #10732 + - use bufq as recv buffer, also for Windows pre-receive handling + - catch small reads followed by larger ones in a single socket + call. A common pattern on TLS connections. -- telnet: only accept option arguments in ascii + Closes #10787 - To avoid embedded telnet negotiation commands etc. +Daniel Stenberg (13 Apr 2023) - Reported-by: Harry Sintonen - Closes #10728 +- urlapi: cleanups -- test1903: test use of COOKIEFILE - reset - COOKIEFILE + - move host checks together + - simplify the scheme parser loop and the end of host name parser + - avoid itermediate buffer storing in multiple places + - reduce scope for several variables + - skip the Curl_dyn_tail() call for speed + - detect IPv6 earlier and skip extra checks for such hosts + - normalize directly in dynbuf instead of itermediate buffer + - split out the IPv6 parser into its own funciton + - call the IPv6 parser directly for ipv6 addresses + - remove (unused) special treatment of % in host names + - junkscan() once in the beginning instead of scattered + - make junkscan return error code + - remove unused query management from dedotdotify() + - make Curl_parse_login_details use memchr + - more use of memchr() instead of strchr() and less strlen() calls + - make junkscan check and return the URL length - This also tests for the memory leak bug fixed by parent commit b559ef6f. + An optimized build runs one of my benchmark URL parsing programs ~41% + faster using this branch. (compared against the shipped 7.88.1 library + in Debian) - Ref: #10694 + Closes #10935 - Closes https://github.com/curl/curl/pull/10712 +Josh McCullough (13 Apr 2023) -Jay Satiro (10 Mar 2023) +- http2: fix typo in infof() call -- url: fix cookielist memleak when curl_easy_reset + Closes #10940 - - Free set.cookelist in Curl_freeset instead of Curl_close. +Daniel Stenberg (12 Apr 2023) - Prior to this change the cookielist linked list wasn't freed by - curl_easy_reset which calls Curl_freeset to free all set. +- noproxy: pointer to local array 'hostip' is stored outside scope - Bug: https://github.com/curl/curl/issues/10694#issuecomment-1458619157 - Reported-by: Sergey Ryabinin + Ref: #10929 + Closes #10933 - Closes https://github.com/curl/curl/pull/10709 +Stefan Eissing (12 Apr 2023) -Dan Fandrich (10 Mar 2023) +- connect: fix https connection setup to treat ssl_mode correctly -- tests: fix some keywords and unused sections + - for HTTPS protocol, a disabled ssl should never be acceptables. -- tests: fix test1301 to call the right binary + Closes #10934 - It was refactored in commit 480ac6e5 but this step was missed. +Douglas R. Reno (12 Apr 2023) -- tests: add timeout, SLOWDOWN and DELAY keywords to tests +- CMakeLists.txt: fix typo for Haiku detection - These are tests that are testing timing and end up being quite slow. + Closes #10937 -Daniel Stenberg (10 Mar 2023) +Dan Fandrich (11 Apr 2023) -- RELEASE-NOTES: synced +- pathhelp: use the cached $use_cygpath when available -Stefan Eissing (10 Mar 2023) +- runtests: eliminate unneeded variable -- wolfSSL: ressurect the BIO `io_result` +- runtests: make the # of server start attempts a constant - In pytest'ing the situation occored that wolfSSL reported an - IO error when the underlying BIO operation was returning an - CURLE_AGAIN condition. +- runtests: on startup failure call displaylogs only in serverfortest - Readding the `io_result` filter context member to detect such - situations. + This reduces the number of calls spread throughout the code. - Also, making sure that the returned CURLcode is initialized - on all recv operations outcome. + Ref: #10818 + Closes #10919 - Closes #10716 +- runtests: return an error code with startservers() -- gssapi: align global `gss_OID_desc` vars to silence ld warnings on macOS vent - ura + The code indicates the kind of failure encountered in starting a server, + which can be used by the caller to tailor the user experience. - Refs #9975 which first reported this. + Ref: #10818 - Closes #10718 +- runtests: abort early if runpingpongserver is given a bad server type -Daniel Stenberg (10 Mar 2023) +- runtests: don't use the SMB server verification time as reference -- libssh2: only set the memory callbacks when debugging + %FTPTIME2 and %FTPTIME3 should be set by the FTP server only, for + consistency. - This makes us debug libssh2 less and libcurl more when for example - running torture tests that otherwise will spend a lot of time in libssh2 - functions. +- tests: factor out the test server management code - We leave libssh2 to test libssh2. + This now lives in servers.pm with some configuration variables moved to + globalconfig.pm - Closes #10721 + Ref: #10818 -- docs/SECURITY-PROCESS.md: updates +- runtests: remove an inappropriate use of runclientoutput - - allow Low+Medium issues to be managed through plain PRs - - update the bug-bounty part to reflect current reality + This function is intended for running client code, not servers. - Closes #10719 +- runtests: only add $LIBDIR to the path for checktestcmd -Dan Fandrich (9 Mar 2023) + Since checkcmd is for finding servers, there will never be anything in + this directory of interest to them. -- tests: fix tag markup issues in some tests + Ref: #10818 -Marcel Raad (9 Mar 2023) +- tests: log sshserver.pl messages to a file -- tests: add `cookies` features + The logmsg messages were thrown away before, so they are now available + for debugging. - These tests don't work with `--disable-cookies`. +- runtests: also show DISABLED tests with -l - Closes https://github.com/curl/curl/pull/10713 + Other reasons for skipping tests are ignored for -l, so being explicitly + disabled should be too. -- test420: add cookies keyword +- runtests: move the UNIX sockets into $PIDDIR - It fails with `--disable-cookies`. + These were missed when the other server files were moved there. - Closes https://github.com/curl/curl/pull/10713 + Follow-up to 70d2fca2 -Dan Fandrich (8 Mar 2023) + Ref: #10818 -- CI: Add more labeler match patterns +- tests: tighten up perl exports - Also, add the CI, tests or libcurl API tags in conjunction with any - others that might also apply. + This reduces namespace pollution a little. -Andy Alt (9 Mar 2023) + Ref: #10818 -- GHA: minor improvements to spellcheck +- tests: turn perl modules into full packages - Closes #10640 + This helps enforce more modularization and encapsulation. Enable and fix + warnings on a few packages. Also, rename ftp.pm to processhelp.pm since + there's really nothing ftp-specific in it. -Daniel Stenberg (9 Mar 2023) + Ref: #10818 -- test1671: fix after fix +Daniel Stenberg (11 Apr 2023) -- test421: -w %{header_json} test with multiple same header names +- multi: remove a few superfluous assigns - To reproduce the issue in #10704 + PVS found these "The 'rc' variable was assigned the same value." cases. -- tool_writeout_json. fix the output for duplicate header names + Ref: #10929 + Closes #10932 - Header entries with index != 0 are handled at the index 0 level so they - should then be skipped when iterated over. +- schannel: add clarifying comment - Reported-by: Boris Okunskiy - Fixes #10704 - Closes #10707 + Explaining how the PVS warning in #10929 is wrong: Dereferencing of the + null pointer 'backend->cred' might take place. -- headers: make curl_easy_header and nextheader return different buffers + Closes #10931 - By letting curl_easy_header() and curl_easy_nextheader() store the - header data in their own struct storage when they return a pointer to - it, it makes it possible for applications to use them both in a loop. - Like the curl tool does. +- cookie: clarify that init with data set to NULL reads no file - Reported-by: Boris Okunskiy - Fixes #10704 - Closes #10707 + ... and make Curl_cookie_add() require 'data' being set proper with an + assert. -rcombs (8 Mar 2023) + The function has not worked with a NULL data for quite some time so this + just corrects the code and comment. -- urlapi: take const args in _dup and _get functions + This is a different take than the proposed fixed in #10927 - Closes #10708 + Reported-by: Kvarec Lezki + Ref: #10929 + Closes #10930 -- urlapi: avoid mutating internals in getter routine +Kvarec Lezki (11 Apr 2023) - This was not intended. +- vtls: remove int typecast for sizeof() - Closes #10708 + V220 Suspicious sequence of types castings: memsize -> 32-bit integer -> + memsize. The value being cast: 'sizeof + (buf->data)'. curl\lib\vtls\vtls.c 2025 -Daniel Stenberg (8 Mar 2023) + https://pvs-studio.com/en/docs/warnings/v220/ -- urlapi: '%' is illegal in host names + Closes #10928 - Update test 1560 to verify +Stefan Eissing (11 Apr 2023) - Ref: #10708 - Closes #10711 +- http2: fix copynpaste error reported by coverity -- ftp: make the 'ftpauth' a more normal 'char *'-array + - move all code handling HTTP/2 frames for a particular + stream into a separate function to keep from confusing + the call `data` with the stream `data`. - Closes #10703 + Closes #10924 -Evgeny Grin (Karlson2k) (8 Mar 2023) +Dan Fandrich (11 Apr 2023) -- doc: fix compiler warning in libcurl.m4 +- tests: log a too-long Unix socket path in sws and socksd - Current test for curl_free() may produce warnings with strict compiler - flags or even with default compiler flags with upcoming versions. - These warning could turned into errors by -Werror or similar flags. - Such warnings/errors are avoided by this patch. + Ref: #10919 - Closes #10710 +Daniel Stenberg (11 Apr 2023) -Viktor Szakats (8 Mar 2023) +- gen.pl: error on duplicated See-Also fields -- misc: fix typos + Updated http2.d accordingly. - Closes #10706 + Closes #10925 -Stefan Eissing (7 Mar 2023) +- http2: avoid possible null pointer dereference -- ftp: active mode with SSL, add the damn filter + Reported-by: Dan Fandrich + Fixes #10920 + Closes #10923 - - since 7.87.0 we lost adding the SSL filter for an active - FTP connection that uses SSL. This leads to hangers and timeouts - as reported in #10666. +- lib1560: verify that more bad host names are rejected - Reported-by: SandakovMM on github - Fixes #10666 - Closes #10669 + when setting the hostname component of a URL -Daniel Stenberg (7 Mar 2023) + Closes #10922 -- docs: extend the URL API descriptions +- curl_url_set.3: mention that users can set content rather freely - Closes #10701 + ... which then might render bad URLs if you extract a URL later. -Stefan Eissing (7 Mar 2023) + Closes #10921 -- url: fix logic in connection reuse to deny reuse on "unclean" connections +Dan Fandrich (10 Apr 2023) - - add parameter to `conn_is_alive()` cfilter method that returns - if there is input data waiting on the connection - - refrain from re-using connnection from the cache that have - input pending - - adapt http/2 and http/3 alive checks to digest pending input - to check the connection state - - remove check_cxn method from openssl as that was just doing - what the socket filter now does. - - add tests for connection reuse with special server configs +- CI: retry failed downloads of aws-lc - Closes #10690 + Don't fail the build in case of a temporary server problem. -Daniel Stenberg (6 Mar 2023) +- test1169: fix so it works properly everywhere -- x509asn1: use plain %x, not %lx, when the arg is an int + - Use an absolute path for the -L option since the module isn't in the + perl path + - Create the needed test file in a section; isn't + intended for this + - Fix the test number in the file name, which was wrong - Pointed out by Coverity. + Follow-up to f754990a - Closes #10689 + Ref: #10818 + Fixes #10889 + Closes #10917 -Stefan Eissing (6 Mar 2023) +- tests: stop using strndup(), which isn't portable -- http2: fix handling of RST and GOAWAY to recognize partial transfers + It's not available on Solaris 10, for example. Since this is just test + code that doesn't need to use an optimized system version, replace it + with the implementation copied from tool_cb_hdr.c. - - a reset transfer (HTTP/2 RST) did not always lead to the proper - error message on receiving its response, leading to wrong reports - of a successful transfer - - test_05_02 was able to trigger this condition with increased transfer - count. The simulated response errors did not carry a 'Content-Length' - so only proper RST handling could detect the abort - - When doing such transfers in parallel, a connection could enter the - state where - a) it had been closed (GOAWAY received) - b) the RST had not been "seen" for the transfer yet - or c) the GOAWAY announced an error and the last successful - stream id was not checked against ongoing transfers +- runtests: fix an incorrect comment about the ld_preload feature - Closes #10693 + Follow-up to 1f631864 -- tests: use dynamic ports numbers in pytest suite + Ref: #10818 - - necessary ports are bound at start of test suite and then - given to server fixtures for use. - - this make parallel use of pytest (in separate directories), - practically safe for use as OS tend to not reuse such port numbers - for a while +Daniel Stenberg (9 Apr 2023) - Closes #10692 +- urlapi: prevent setting invalid schemes with *url_set() -- connect: fix time_connect and time_appconnect timer statistics + A typical mistake would be to try to set "https://" - including the + separator - this is now rejected as that would then lead to + url_get(... URL...) would get an invalid URL extracted. - - time_connect was not updated when the overall connection failed, - e.g. when SSL verification was unsuccessful, refs #10670 - - rework gather those values to interrogate involved filters, - also from all eyeballing attempts, to report the maximum of - those values. - - added 3 test cases in test_06 to check reported values on - successful, partially failed and totally failed connections. + Extended test 1560 to verify. - Reported-by: Master Inspire - Fixes #10670 - Closes #10671 + Closes #10911 -Daniel Stenberg (6 Mar 2023) +Biswapriyo Nath (9 Apr 2023) -- test1905: update output cookie order +- http2: remove unused Curl_http2_strerror function declaration - After the #10685 update + Curl_http2_strerror was renamed to http2_strerror in + 05b100aee247bb9bec8e9a1b0 and then http2_strerror was removed in + 5808a0d0f5ea0399d4a2a2 -- test420: verify expiring cookies + This also fixes the following compiler error - Cookies that are loaded fine from a jar but then are expired in headers. + lib/http2.h:41:33: error: unknown type name 'uint32_t' + lib/http2.h:1:1: note: 'uint32_t' is defined in header '' -- cookie: don't load cookies again when flushing + Closes #10912 - Reported-by: Sergio Mijatovic - Fixes #10677 - Closes #10685 +Daniel Stenberg (8 Apr 2023) - RELEASE-NOTES: synced -Andy Alt (6 Mar 2023) - -- docs: note '--data-urlencode' option - - Closes #10687 +SuperIlu on github (8 Apr 2023) -Daniel Stenberg (6 Mar 2023) +- config-dos.h: fix SIZEOF_CURL_OFF_T for MS-DOS/DJGPP -- DEPRECATE: the original legacy mingw version 1 + Fixes #10905 + Closes #10910 - Remove completely in September 2023 +Daniel Stenberg (8 Apr 2023) - Closes #10667 +- lib: remove CURLX_NO_MEMORY_CALLBACKS -Harry Sintonen (6 Mar 2023) + The only user of this define was 'chkdecimalpoint' - a special purpose + test tool that was built but not used anymore (since 17c18fbc3 - Apr + 2020). -- rand: use arc4random as fallback when available + Closes #10908 - Normally curl uses cryptographically strong random provided by the - selected SSL backend. If compiled without SSL support, a naive built-in - function was used instead. +- CURLPROXY_HTTPS2: for HTTPS proxy that may speak HTTP/2 - Generally this was okay, but it will result in some downsides for non- - SSL builds, such as predictable temporary file names. + Setting this proxy type allows curl to negotiate and use HTTP/2 with + HTTPS proxies. - This change ensures that arc4random will be used instead, if available. + Closes #10900 - Closes #10672 +Ali Khodkar (8 Apr 2023) -Grisha Levit (6 Mar 2023) +- write-out.d: add missing periods -- tool: dump headers even if file is write-only + Closes #10897 - The fixes in #10079 brought a (seemingly unrelated) change of open mode - from `wb`/`ab` to `wb+`/`ab+` for the headerfile. This makes it no - longer possible to write the header file to e.g. a pipe, like: +Daniel Stenberg (7 Apr 2023) - curl -D >(grep ...) file:///dev/null +- http2: remove check for !data after it was already dereferenced - Which presently results in `Warning: Failed to open /dev/fd/63` + Pointed out by Coverity - See #10079 - Closes #10675 + Closes #10906 diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index 08983c38a6..b8e754e4ac 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -338,6 +338,7 @@ Bjoern Sikora Bjorn Augustsson Bjorn Reese Björn Stenberg +black-desk on github Blaise Potard Blake Burkhart bnfp on github @@ -404,6 +405,7 @@ Bylon2 on github Byrial Jensen Caleb Raitto Calvin Buckley +calvin2021y on github Cameron Blomquist Cameron Cawley Cameron Kaiser @@ -490,6 +492,7 @@ civodul on github Claes Jakobsson Clarence Gardner Claudio Neves +claudiusaiz on github clbr on github Clemens Gruber Cliff Crosland @@ -513,6 +516,7 @@ Colin Watson Colm Buckley Colman Mbuya Constantine Sapuntzakis +consulion on github coralw on github correctmost on github Cory Benfield @@ -552,7 +556,6 @@ Dan Becker Dan Cristian Dan Donahue Dan Fandrich -Dan Frandrich Dan Johnson Dan Kenigsberg Dan Locks @@ -770,6 +773,7 @@ Edin Kadribasic Edmond Yu Edoardo Lolletti Eduard Bloch +Eduard Strehlau Edward Kimmel Edward Rudd Edward Sheldrake @@ -784,6 +788,7 @@ elelel on github elephoenix on github Eli Schwartz Elia Tufarolo +Elliot Killick Elliot Saba Ellis Pritchard Elmira A Semenova @@ -814,6 +819,7 @@ Eric Lavigne Eric Lubin Eric Melville Eric Mertens +Eric Murphy Eric Musser Eric Rautman Eric Rescorla @@ -1024,6 +1030,7 @@ h1zzz on github H3RSKO on github Hagai Auro Haibo Huang +Hakan Sunay Halil Hamish Mackenzie hamstergene on github Han Han @@ -1153,6 +1160,7 @@ Jaime Fullaondo jakirkham on github Jakob Hirsch Jakub Bochenski +Jakub Jelen Jakub Wilk Jakub Zakrzewski James Atwill @@ -1391,6 +1399,7 @@ Josh Kapell Josh McCullough Josh Soref joshhe on github +Joshix-1 on github Joshua Kwan Joshua Root Joshua Swink @@ -1417,6 +1426,7 @@ Julian Z Julien Chaffraix Julien Nabet Julien Royer +Juliusz Sosinowicz Jun Tseng Jun-ichiro itojun Hagino Jun-ya Kato @@ -1554,6 +1564,7 @@ Larry Lin Larry Stefani Larry Stone Lars Buitinck +Lars Francke Lars Gustafsson Lars J. Aas Lars Johannesen @@ -1612,6 +1623,7 @@ Loganaden Velvindron Loic Dachary Loren Kirkby Lorenzo Miniero +Loïc Yhuel Luan Cestari Luca Altea Luca Boccassi @@ -1764,6 +1776,7 @@ MasterInQuestion on github Mateusz Loskot Mathew Benson Mathias Axelsson +Mathias Fuchs Mathias Gumz Mathieu Carbonneaux Mathieu Legare @@ -1806,6 +1819,7 @@ Max Mehl Max Peal Max Savenkov Max Zettlmeißl +Maxim Dzhura Maxim Ivanov Maxim Perenesenko Maxim Prohorov @@ -1940,6 +1954,7 @@ Nach M. S. Nagai H Nao Yonashiro naost3rn on github +Natanael Copa Nate Prewitt Nathan Coulter Nathan Moinvaziri @@ -2127,6 +2142,7 @@ Pawel A. Gajda Pawel Kierski Paweł Kowalski Paweł Wegner +PBudmark on github Pedro Henrique Pedro Larroy Pedro Monreal @@ -2356,6 +2372,7 @@ Robert Olson Robert Prag Robert Ronto Robert Schumann +Robert Simpson Robert Weaver Robert Wruck Robin A. Meade @@ -2422,6 +2439,7 @@ Ryuichi KAWAMATA rzrymiak on github Rémy Léone S. Moonesamy +s0urc3_ on hackerone Sai Ram Kunala Salah-Eddin Shaban Saleem Abdulrasool @@ -2861,6 +2879,7 @@ vvb2060 on github Vyron Tsingaras Vítor Galvão W. Mark Kubacki +w0x42 on hackerone Waldek Kozba Walter J. Mack wangzhikun @@ -2977,6 +2996,7 @@ zzq1015 on github ウさん 不确定 加藤郁之 +南宫雪珊 左潇峰 梦终无痕 積丹尼 Dan Jacobson -- cgit v1.2.3