From 2ae0fc7299825cc561197d3f23c90e52ae8db58b Mon Sep 17 00:00:00 2001 From: dartraiden Date: Sun, 10 Dec 2023 18:22:53 +0300 Subject: libcurl: update to 8.5.0 --- libs/libcurl/docs/CHANGES | 12691 +++++++++++++++++++++++--------------------- libs/libcurl/docs/THANKS | 42 + 2 files changed, 6802 insertions(+), 5931 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 8d56bf8e28..85fa4522fb 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,9540 +6,10369 @@ Changelog -Version 8.4.0 (11 Oct 2023) +Version 8.5.0 (6 Dec 2023) -Daniel Stenberg (11 Oct 2023) +Daniel Stenberg (6 Dec 2023) - RELEASE-NOTES: synced -- THANKS: add contributors from 8.4.0 - -Jay Satiro (11 Oct 2023) + The curl 8.5.0 release. -- socks: return error if hostname too long for remote resolve +Dan Fandrich (5 Dec 2023) - Prior to this change the state machine attempted to change the remote - resolve to a local resolve if the hostname was longer than 255 - characters. Unfortunately that did not work as intended and caused a - security issue. +- github/labeler: switch from the beta to labeler v5 - Bug: https://curl.se/docs/CVE-2023-38545.html + Some keys were renamed and the dot option was made default. -Stefan Eissing (10 Oct 2023) + Closes #12458 -- CI: remove slowed-network tests +Daniel Stenberg (5 Dec 2023) - - remove these tests as they are currently not reliable in our CI - setups. +- DEPRECATE: remove NTLM_WB in June 2024 - curl handles the test cases, but CI sometimes fails on these due to - additional conditions. Rather than mix them in, an additional CI job - will be added in the future that is specific to them. + Ref: https://curl.se/mail/lib-2023-12/0010.html - Closes https://github.com/curl/curl/pull/12075 + Closes #12451 -Jay Satiro (10 Oct 2023) +Jacob Hoffman-Andrews (4 Dec 2023) -- libcurl-env-dbg.3: move debug variables from libcurl-env.3 +- rustls: implement connect_blocking - - Move documentation of libcurl environment variables used only in debug - builds from libcurl-env into a separate document libcurl-env-dbg. + Closes #11647 - - Document more debug environment variables. +Daniel Stenberg (4 Dec 2023) - Previously undocumented or missing a description: +- examples/rtsp-options.c: add - CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, - CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, - CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, - CURL_TIME. + Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and + CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. - Closes https://github.com/curl/curl/pull/11811 + Closes #12452 -Dan Fandrich (9 Oct 2023) +Stefan Eissing (4 Dec 2023) -- test670: increase the test timeout +- ngtcp2: ignore errors on unknown streams - This should make it more immune to loaded servers. + - expecially in is_alive checks on connections, we might + see incoming packets on streams already forgotten and closed, + leading to errors reported by nghttp3. Ignore those. - Ref: #11328 + Closes #12449 -Stefan Eissing (9 Oct 2023) +Daniel Stenberg (4 Dec 2023) -- MQTT: improve receive of ACKs +- docs: make all examples in all libcurl man pages compile - - add `mq->recvbuf` to provide buffering of incomplete - ACK responses - - continue ACK reading until sufficient bytes available - - fixes test failures on low network receives + Closes #12448 - Closes #12071 +- checksrc.pl: support #line instructions -Viktor Szakats (9 Oct 2023) + makes it identify the correct source file and line -- quic: fix BoringSSL build +- GHA/man-examples: verify libcurl man page examples - Add guard around `SSL_CTX_set_ciphersuites()` use. +- verify-examples.pl: verify that all man page examples compile clean - Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 +- RELEASE-NOTES: synced - Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 +Graham Campbell (2 Dec 2023) - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes #12067 +- http3: bump ngtcp2 and nghttp3 versions -Stefan Eissing (9 Oct 2023) + nghttp3 v1.1.0 + ngtcp2 v1.1.0 -- test1540: improve reliability + In docs and CI - - print that bytes have been received on pausing, but not how many + Closes #12446 - Closes #12069 +- CI/quiche: use `3.1.4+quic` consistently in CI workflows -- test2302: improve reliability + Closes #12447 - - make result print collected write data, unless - change in meta flags is detected - - will show same result even when data arrives via - several writecb invocations +Viktor Szakats (2 Dec 2023) - Closes #12068 +- test1545: disable deprecation warnings -Daniel Stenberg (9 Oct 2023) + Fixes: + https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr + p6pk#L1205 -- curl_easy_pause: set "in callback" true on exit if true + Same with details: + https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil + b6wt#L1263 + ``` + tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 + .0. Use curl_mime_init() [-Werror=deprecated-declarations] + 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", + | ^~~~~~~~~~~~ + [...] + ``` - Because it might have called another callback in the mean time that then - set the bit FALSE on exit. + Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 - Reported-by: Jay Satiro - Fixes #12059 - Closes #12061 + Fixes #12445 + Closes #12444 -Viktor Szakats (8 Oct 2023) +Daniel Stenberg (2 Dec 2023) -- h3: add support for ngtcp2 with AWS-LC builds +- INSTALL: update list of ports and CPU archs - ``` - curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- - LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 - Release-Date: [unreleased] - Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps - mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss - Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi - SSL NTLM SSL threadsafe UnixSockets - ``` +- symbols-in-versions: the CLOSEPOLICY options are deprecated - Also delete an obsolete GnuTLS TODO and update the header comment in - `FindNGTCP2.cmake`. + The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. - Reviewed-by: Daniel Stenberg - Closes #12066 +z2_ (1 Dec 2023) -- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros +- build: fix builds that disable protocols but not digest auth - Syncing this up with CMake. + - Build base64 functions if digest auth is not disabled. - Source code uses the built-in `OPENSSL_IS_AWSLC` and - `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is - necessary from the build tools. + Prior to this change if some protocols were disabled but not digest auth + then a build error would occur due to missing base64 functions. - The one use of `HAVE_BORINGSSL` in the source turned out to be no longer - necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 + Fixes https://github.com/curl/curl/issues/12440 + Closes https://github.com/curl/curl/pull/12442 - autotools detects this anyway for display purposes. - CMake detects this to decide whether to use the BoringSSL-specific - crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection - result just yet (planned in #12066). +Michał Antoniak (1 Dec 2023) - Ref: #11964 +- connect: reduce number of transportation providers - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12065 + Use only the ones necessary - the ones that are built-in. Saves a few + bytes in the resulting code. -Marc Hoersken (8 Oct 2023) + Closes #12438 -- CI: move distcheck job from Azure Pipelines to GitHub Actions +David Benjamin (1 Dec 2023) - This will allow for more trigger excludes within Azure Pipelines. +- vtls: consistently use typedef names for OpenSSL structs - Also fixes seemingly broken check with scripts/installcheck.sh. - Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 + The foo_st names don't appear in OpenSSL public API documentation. The + FOO typedefs are more common. This header was already referencing + SSL_CTX via . There is a comment about avoiding + , but OpenSSL actually declares all the typedefs in + , which is already included by (and + every other OpenSSL header), so just use that. Though I've included it + just to be explicit. - Assisted-by: Philip Heiduck - Closes #9532 + (I'm also fairly sure including already triggers the + Schannel conflicts anyway. The comment was probably just out of date.) -Daniel Stenberg (8 Oct 2023) + Closes #12439 -- url: fall back to http/https proxy env-variable if ws/wss not set +Lau (1 Dec 2023) - Reported-by: Craig Andrews - Fixes #12031 - Closes #12058 +- libcurl-security.3: fix typo -Stefan Eissing (8 Oct 2023) + Fixed minimal typo. -- cf-socket: simulate slow/blocked receives in debug + Closes #12437 - add 2 env variables for non-UDP sockets: - 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly - should return EAGAIN - 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket +Stefan Eissing (1 Dec 2023) - Closes #12035 +- ngtcp2: fix races in stream handling -- http2: refused stream handling for retry + - fix cases where ngtcp2 invokes callbacks on streams that + nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND + in these cases as it is normal behaviour. - - answer HTTP/2 streams refused via a GOAWAY from the server to - respond with CURLE_RECV_ERROR in order to trigger a retry - on another connection + Closes #12435 - Reported-by: black-desk on github - Ref #11859 - Closes #12054 +Emanuele Torre (1 Dec 2023) -Jay Satiro (8 Oct 2023) +- tool_writeout_json: fix JSON encoding of non-ascii bytes -- CURLOPT_DEBUGFUNCTION.3: warn about internal handles + char variables if unspecified can be either signed or unsigned depending + on the platform according to the C standard; in most platforms, they are + signed. - - Warn that the user's debug callback may be called with the handle - parameter set to an internal handle. + This meant that the *i<32 waas always true for bytes with the top bit + set. So they were always getting encoded as \uXXXX, and then since they + were also signed negative, they were getting extended with 1s causing + '\xe2' to be expanded to \uffffffe2, for example: - Without this warning the user may assume that the only handles their - debug callback receives are the easy handles on which they set - CURLOPT_DEBUGFUNCTION. + $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul + l + \uffffffe2\uffffff80\uffffff9c - This is a follow-up to f8cee8cc which changed DoH handles to inherit - the debug callback function set in the user's easy handle. As a result - those handles are now passed to the user's debug callback function. + I fixed this bug by making the code use explicitly unsigned char* + variables instead of char* variables. - Closes https://github.com/curl/curl/pull/12034 + Test 268 verifies -- url: fix typo + Reported-by: iconoclasthero + Closes #12434 -Daniel Stenberg (8 Oct 2023) +Stefan Eissing (1 Dec 2023) -- test458: verify --expand-output, expanding a file name accepting option +- cf-socket: TCP trace output local address used in connect - Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) + Closes #12427 -- tool_getparam: accept variable expansion on file names too +Jay Satiro (1 Dec 2023) - Reported-by: PBudmark on github - Fixes #12048 - Closes #12055 +- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation -- RELEASE-NOTES: synced + - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it + includes protocol-specific instructions that trigger a transfer. -- multi: do CURLM_CALL_MULTI_PERFORM at two more places + Prior to this change it explicitly said that it did not include those + instructions in the time, but that is incorrect. - ... when it does a state transition but there is no particular socket or - timer activity. This was made apparent when commit b5bb84c removed a - superfluous timer expiry. + The change is a copy of the fixed explanation already in + CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. - Reported-by: Dan Fandrich. - Fixes #12033 - Closes #12056 + Reported-by: eeverettrbx@users.noreply.github.com -Viktor Szakats (7 Oct 2023) + Fixes https://github.com/curl/curl/issues/12431 + Closes https://github.com/curl/curl/pull/12432 -- GHA/linux: mbedtls 3.5.0 + minor dep bumps +Daniel Stenberg (30 Nov 2023) - Closes #12057 +- multi: during ratelimit multi_getsock should return no sockets -Dan Fandrich (7 Oct 2023) + ... as there is nothing to wait for then, it just waits. Otherwise, this + causes much more CPU work and updates than necessary during ratelimit + periods. -- CI: bump OpenLDAP package version on FreeBSD + Ref: https://curl.se/mail/lib-2023-11/0056.html + Closes #12430 - The old one is no longer available. +Dmitry Karpov (30 Nov 2023) -Marc Hoersken (7 Oct 2023) +- transfer: abort pause send when connection is marked for closing -- docs/libcurl/opts/Makefile.inc: add missing manpage files + This handles cases of some bi-directional "upgrade" scenarios + (i.e. WebSockets) where sending is paused until some "upgrade" handshake + is completed, but server rejects the handshake and closes the + connection. - Detected with #9532 + Closes #12428 -Dan Fandrich (7 Oct 2023) +Daniel Stenberg (28 Nov 2023) -- tests: fix a race condition in ftp server disconnect +- RELEASE-NOTES: synced - If a client disconnected and reconnected quickly, before the ftp server - had a chance to respond, the protocol message/ack (ping/pong) sequence - got out of sync, causing messages sent to the old client to be delivered - to the new. A disconnect must now be acknowledged and intermediate - requests thrown out until it is, which ensures that such synchronization - problems can't occur. This problem could affect ftp, pop3, imap and smtp - tests. +- openssl: when a session-ID is reused, skip OCSP stapling - Fixes #12002 - Closes #12049 + Fixes #12399 + Reported-by: Alexey Larikov + Closes #12418 -Viktor Szakats (7 Oct 2023) +- test1545: test doing curl_formadd twice with missing file -- appveyor: bump mingw-w64 job to gcc 13 (was: 8) + Reproduces #12410 + Verifies the fix + Closes #12421 - This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). - Adding a modern gcc version to the tests. +- Curl_http_body: cleanup properly when Curl_getformdata errors - (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 - finished in 32, 35, 34 minutes in the 3 test runs so far.) + Reported-by: yushicheng7788 on github + Based-on-work-by: yushicheng7788 on github + Fixes #12410 + Closes #12421 - It also adds a modern CMake version and OS env to our mingw-w64 builds. +- test1477: verify that libcurl-errors.3 and public headers are synced - Closes #12051 + The script errorcodes.pl extracts all error codes from all headers and + checks that they are all documented, then checks that all documented + error codes are also specified in a header file. -David Benjamin (6 Oct 2023) + Closes #12424 -- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR +- libcurl-errors.3: sync with current public headers - While the struct is still public in OpenSSL, there is a (somewhat - inconvenient) accessor. Use it to remain compatible if it becomes opaque - in the future. + Closes #12424 - Closes #12038 +Stefan Eissing (28 Nov 2023) -Daniel Stenberg (6 Oct 2023) +- test459: fix for parallel runs -- curl_easy_pause.3: mention it works within callbacks + - change warniing message to work better with varying filename + length. + - adapt test output check to new formatting - Reported-by: Maxim Dzhura - Bug: https://curl.se/mail/lib-2023-10/0010.html - Closes #12046 + Follow-up to 97ccc4479f77ba3191c6 + Closes #12423 -- curl_easy_pause.3: mention h2/h3 buffering +Daniel Stenberg (27 Nov 2023) - Asked-by: Maxim Dzhura - Ref: https://curl.se/mail/lib-2023-10/0011.html +- tool_cb_prg: make the carriage return fit for wide progress bars - Closes #12045 + When the progress bar was made max width (256 columns), the fly() + function attempted to generate its output buffer too long so that the + trailing carriage return would not fit and then the output would show + wrongly. The fly function is called when the expected total transfer is + unknown, which could be one or more progress calls before the actual + progress meter get shown when the expected transfer size is provided. -Viktor Szakats (6 Oct 2023) + This new take also replaces the msnprintf() call with a much simpler + memset() for speed. -- cmake: re-add missed C89 headers for specific detections + Reported-by: Tim Hill + Fixes #12407 + Closes #12415 - We removed C89 `setjmp.h` and `signal.h` detections and excluded them - from the global header list we use when detecting functions [1]. Then - missed to re-add these headers to the specific functions which need - them to be detected [2]. Fix this omission in this patch. +- tool_parsecfg: make warning output propose double-quoting - [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 - [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + When the config file parser detects a word that *probably* should be + quoted, mention double-quotes as a possible remedy. - Closes #12043 + Test 459 verifies. -Daniel Stenberg (6 Oct 2023) + Proposed-by: Jiehong on github + Fixes #12409 + Closes #12412 -- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE +Jay Satiro (26 Nov 2023) - Since there is nothing to wait for there. Avoids the test 1233 hang - reported in #12033. +- curl.rc: switch out the copyright symbol for plain ASCII - Reported-by: Dan Fandrich - Closes #12042 + .. like we already do for libcurl.rc. -Dan Fandrich (5 Oct 2023) + libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" + warning so it was switched to ascii. -- test1903: actually verify the cookies after the test + Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 - The test otherwise could do just about anything (except leak memory in - debug mode) and its bad behaviour wouldn't be detected. Now, check the - resulting cookie file to ensure the cookies are still there. + Suggested-by: Robert Southee - Closes #12041 + Closes https://github.com/curl/curl/pull/12403 -- test: add missing s +Daniel Stenberg (26 Nov 2023) - The tests will otherwise fail if curl has them disabled. +- conncache: use the closure handle when disconnecting surplus connections -- test1906: set a lower timeout since it's hit on Windows + Use the closure handle for disconnecting connection cache entries so + that anything that happens during the disconnect is not stored and + associated with the 'data' handle which already just finished a transfer + and it is important that details from the unrelated disconnect does not + taint meta-data in the data handle. - msys2 builds actually hit the connect timeout in normal operation, so - lower the timeout from 5 minutes to 5 seconds to reduce test time. + Like storing the response code. - Ref: #11328 - Closes #12036 + This also adjust test 1506. Unfortunately it also removes a key part of + the test that verifies that a connection is closed since when this + output vanishes (because the closure handle is used), we don't know + exactly that the connection actually gets closed in this test... -Daniel Stenberg (5 Oct 2023) + Reported-by: ohyeaah on github + Fixes #12367 + Closes #12405 - RELEASE-NOTES: synced -Jay Satiro (5 Oct 2023) +Stefan Eissing (24 Nov 2023) -- idn: fix WinIDN null ptr deref on bad host +- quic: make eyeballers connect retries stop at weird replies - - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from - UTF-8 to UTF-16. + - when a connect immediately goes into DRAINING state, do + not attempt retries in the QUIC connection filter. Instead, + return CURLE_WEIRD_SERVER_REPLY + - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an + inconclusive answer. When all addresses have been attempted, + rewind the address list once on an inconclusive answer. + - refs #11832 where connects were retried indefinitely until + the overall timeout fired - Prior to this change a failed conversion erroneously returned CURLE_OK - which meant 'decoded' pointer (what would normally point to the - punycode) would not be written to, remain NULL and be dereferenced - causing an access violation. + Closes #12400 - Closes https://github.com/curl/curl/pull/11983 +Daniel Stenberg (24 Nov 2023) -Dan Fandrich (4 Oct 2023) +- CI: verify libcurl function SYNPOSIS sections -- tests: close the shell used to start sshd + With the .github/scripits/verify-synopsis.pl script - This shell isn't needed once sshd starts, so use "exec" so it doesn't - stick around. + Closes #12402 - Closes #12032 +- docs/libcurl: SYNSOPSIS cleanup -Daniel Stenberg (4 Oct 2023) + - use the correct include file + - make sure they are declared as in the header file + - fix minor nroff syntax mistakes (missing .fi) -- base64: also build for curl + These are verified by verify-synopsis.pl, which extracts the SYNPOSIS + code and runs it through gcc. - Since the tool itself now uses the base64 code using the curlx way, it - needs to build also when the tool needs it. Starting now, the tool build - defines BULDING_CURL to allow lib-side code to use it. + Closes #12402 - Follow-up to 2e160c9c6525 +- sendf: fix comment typo - Closes #12010 +- fopen: allocate the dir after fopen -Eduard Strehlau (4 Oct 2023) + Move the allocation of the directory name down to after the fopen() call + to allow that shortcut code path to avoid a superfluous malloc+free + cycle. -- tests: Fix zombie processes left behind by FTP tests. + Follow-up to 73b65e94f35311 - ftpserver.pl correctly cleans up spawned server processes, - but forgets to wait for the shell used to spawn them. - This is barely noticeable during a normal testrun, - but causes process exhaustion and test failure - during a complete torture run of the FTP tests. + Closes #12398 - Fixes #12018 - Closes #12020 +Stefan Eissing (24 Nov 2023) -Dan Fandrich (4 Oct 2023) +- transfer: cleanup done+excess handling -- github/labeler: improve labeler matches + - add `SingleRequest->download_done` as indicator that + all download bytes have been received + - remove `stop_reading` bool from readwrite functions + - move excess body handling into client download writer -- test574: add a timeout to the test + Closes #12371 - This one hangs occasionally, so this will speed up a test run and allow - logs to be seen when it does. +Daniel Stenberg (23 Nov 2023) - Closes #12025 +- fopen: create new file using old file's mode -- tests: propagate errors in libtests + Because the function renames the temp file to the target name as a last + step, if the file was previously owned by a different user, not ORing + the old mode could otherwise end up creating a file that was no longer + readable by the original owner after save. - Use the test macros to automatically propagate some errors, and check - and log others while running the tests. This can help in debugging - exactly why a test has failed. + Reported-by: Loïc Yhuel + Fixes #12299 + Closes #12395 -- tests: set --expect100-timeout to improve test reliability +- test1476: require proxy - On an overloaded server, the default 1 second timeout can go by without - the test server having a chance to respond with the expected headers, - causing tests to fail. Increase the 1 second timeout to 99 seconds so - this failure mode is no longer a problem on test 1129. Some other tests - already set a high value, but make them consistently 99 seconds so if - something goes wrong the test is stalled for less time. + Follow-up from 323df4261c3542 - Ref: #11328 + Closes #12394 -- CI: ignore the "flaky" and "timing-dependent" test results in CMake +- fopen: create short(er) temporary file name - This was already done for automake builds but CMake builds were missed. - Test 1086 actually causes the test harness to crash with: + Only using random letters in the name plus a ".tmp" extension. Not by + appending characters to the final file name. - Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec - ts/curl/tests/ftpserver.pl line 527 + Reported-by: Maksymilian Arciemowicz - Rather than fix it now, this change leaves test 1086 entirely skipped on - those builds that show this problem. + Closes #12388 - Follow-up to 589dca761 +Stefan Eissing (23 Nov 2023) - Ref: #11865 +- tests: git ignore generated second-hsts.txt file -Viktor Szakats (4 Oct 2023) + File is generated in test lib1900 -- cmake: improve OpenLDAP builds + Follow-up to 7cb03229d9e9c5 - - cmake: detect OpenLDAP based on function `ldap_init_fd`. - autotools does this. autotools also publishes this detection result - in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source - doesn't use this value. (it might need to be remove-listed in - `scripts/cmp-config.pl` for future OpenLDAP test builds.) - This also deletes existing self-declaration method via the - CMake-specific `CURL_USE_OPENLDAP` configuration. + Closes #12393 - - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. - Like autotools does. This fixes a long list of these warnings: - ``` - /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i - s not defined, evaluates to 0 [-Wundef] - ``` +Viktor Szakats (23 Nov 2023) - - cmake: delete LDAP TODO comment no longer relevant. +- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 - Also: + Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to + OpenSSL 1.1.x, and LibreSSL 3.6 or upper. - - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature - detection functions. + With the caveat, that "group name" and "type name" are missing from + the log output with these TLS backends. - Ref: #11964 (effort to sync cmake detections with autotools) + Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Closes #12024 + Reviewed-by: Daniel Stenberg + Closes #12385 -- cmake: fix unity builds for more build combinations +Daniel Stenberg (23 Nov 2023) - By using unique static function/variable names in source files - implementing these interfaces. +- urldata: fix typo in comment - - OpenLDAP combined with any SSH backend. +- CI: codespell - - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. + The list of words to ignore is in the file + .github/scripts/codespell-ignore.txt - Closes #12027 + Closes #12390 -Daniel Stenberg (4 Oct 2023) +- lib: fix comment typos -- tests: remove leading spaces from some tags + Five separate ones, found by codespell - The threee tags ``, `` and `` were frequently used - with a leading space that this removes. The reason this habbit is so - widespread in testcases is probably that they have been copy and pasted. + Closes #12390 - Hence, fixing them all now might curb this practice from now on. +- test1476: verify cookie PSL mixed case - Closes #12028 +- cookie: lowercase the domain names before PSL checks -Viktor Szakats (4 Oct 2023) + Reported-by: Harry Sintonen -- GHA: bump actions/checkout + Closes #12387 - Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 - Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 +Viktor Szakats (23 Nov 2023) - Closes #12023 +- openssl: fix building with v3 `no-deprecated` + add CI test -- spelling: fix codespell 2.2.6 typos + - build quictls with `no-deprecated` in CI to have test coverage for + this OpenSSL 3 configuration. - Closes #12019 + - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. + The caller code is meant for OpenSSL 3, while these two functions were + only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 + if built with option `no-deprecated`, causing build errors: + ``` + vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ + algorithms'; ISO C99 and later do not support implicit function declaration + s [-Wimplicit-function-declaration] + vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ + digests'; ISO C99 and later do not support implicit function declarations [ + -Wimplicit-function-declaration] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f + ullLog=true#L7667 -Daniel Stenberg (3 Oct 2023) + Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 + Reviewed-by: Alex Bozarth -- GHA: add workflow to compare configure vs cmake outputs + - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with + `no-deprecated` quictls 3 builds. + Do it by moving an existing solution for this from `vtls/openssl.c` + to `vtls/openssl.h` and adjusting caller code. + ``` + vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g + et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit + -function-declaration] + ``` + Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s + tep:24:1178 - Uses scripts/cmp-config.pl two compare two curl_config.h files, - presumbly generated with configure and cmake. It displays the - differences and filters out a lot of known lines we ignore. + - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and + `-Wunused-function` when trying to build curl with NTLM enabled but + without the necessary TLS backend (with DES) support. - The script also shows the matches that were *not* used. Possibly - subjects for removal. + Closes #12384 - Closes #11964 +- curl.h: delete Symbian OS references -- appveyor: enable test 571 + curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 + via #5989. Delete references to it from public headers, because there + is no fresh release to use those headers with. - Follow-up from 8a940fd55c175f7 / #12013 + Reviewed-by: Dan Fandrich + Reviewed-by: Jay Satiro + Closes #12378 - Closes #12017 +- windows: use built-in `_WIN32` macro to detect Windows -Viktor Szakats (3 Oct 2023) + Windows compilers define `_WIN32` automatically. Windows SDK headers + or build env defines `WIN32`, or we have to take care of it. The + agreement seems to be that `_WIN32` is the preferred practice here. + Make the source code rely on that to detect we're building for Windows. -- build: alpha-sort source files for lib and src + Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for + Windows detection, next to the official `_WIN32`. After this patch it + only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. - Closes #12014 + There is a slight chance these break compatibility with Windows + compilers that fail to define `_WIN32`. I'm not aware of any obsolete + or modern compiler affected, but in case there is one, one possible + solution is to define this macro manually. -- cmake: delete old `HAVE_LDAP_URL_PARSE` logic + grepping for `WIN32` remains useful to discover Windows-specific code. - Left there by accident after adding proper detection for this. + Also: - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 + - extend `checksrc` to ensure we're not using `WIN32` anymore. - Ref: #11964 (effort to sync cmake detections with autotools) + - apply minor formatting here and there. - Closes #12015 + - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. -Stefan Eissing (3 Oct 2023) + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg -- tests: increase lib571 timeout from 3s to 30s + Closes #12376 - - 3s is too short for our CI, making this test fail occasionally - - test usually experiences no delay run locally, so 30s wont hurt +Stefan Eissing (22 Nov 2023) - Closes #12013 +- url: ConnectionExists revisited -Viktor Szakats (3 Oct 2023) + - have common pattern of `if not match, continue` + - revert pages long if()s to return early + - move dead connection check to later since it may + be relatively expensive + - check multiuse also when NOT building with NGHTTP2 + - for MULTIUSE bundles, verify that the inspected + connection indeed supports multiplexing when in use + (bundles may contain a mix of connection, afaict) -- cmake: fix unity with Windows Unicode + TrackMemory + Closes #12373 - Found the root cause of the startup crash in unity builds with Unicode - and TrackMemory enabled at the same time. +Daniel Stenberg (22 Nov 2023) - We must make sure that the `memdebug.h` header doesn't apply to - `lib/curl_multibyte.c` (as even noted in a comment there.) In unity - builds all headers apply to all sources, including `curl_multibyte.c`. - This probably resulted in an infinite loop on startup. +- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range - Exclude this source from unity compilation with TrackMemory enabled, - in both libcurl and curl tool. Enable unity mode for a debug Unicode - CI job to keep it tested. Also delete the earlier workaround that - fully disabled unity for affected builds. + ... or use the default value. - Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 + Also clarify the documentation language somewhat. - Closes #11928 + Closes #12382 -- cmake: disable unity mode with Windows Unicode + TrackMemory +- urldata: make maxconnects a 32 bit value - "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, - aka `-DCURLDEBUG`). + "2^32 idle connections ought to be enough for anybody" - There is an issue with memory tracking and Unicode when built in "unity" - mode, which results in the curl tool crashing right on startup, even - without any command-line option. Interestingly this doesn't happen under - WINE (at least on the system I tested this on), but consistenly happens - on real Windows machines. Crash is 0xC0000374 heap corruption. Both - shared and static curl executables are affected. + Closes #12375 - This limitation probably won't hit too many people, but it remains - a TODO to find and fix the root cause and drop this workaround. +- FEATURES: update the URL phrasing - Example builds and runs: - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd - 7iwj#L313 (static) - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby - qu9c#L317 (shared) + The URL is length limited since a while back so "no limit" simply is not + true anymore. Mention the URL RFC standard used instead. - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 + Closes #12383 - Ref: #11928 - Closes #12005 +- wolfssh: remove redundant static prototypes -- cmake: tidy-up `NOT_NEED_LBER_H` detection + vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We + rror=redundant-decls] - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 + Closes #12381 -- appveyor: rewrite batch in PowerShell + CI improvements +- setopt: remove superfluous use of ternary expressions - 1. Rewrite in PowerShell: + Closes #12374 - - rewrite MS-DOS batch build script in PowerShell. - - move some bash operations into native PowerShell. - - fixups for PowerShell insisting on failure when a command outputs - something to stderr. - - fix to actually run `curl -V` after every build. - (and exclude ARM64 builds.) - - also say why we skipped `curl -V` if we had to skip. - - fix CMake warnings about unused configuration variables, by adapting - these dynamically for build cases. - - dedupe OpenSSL path into a variable. - - disable `test1451` failing with a warning anyway due to missing python - impacket. (after trying and failing to install impacket) - PowerShell promotes these warnings to errors by PowerShell. We can also - suppress they wholesale if they start causing issues in the future, - like we already to with `autoreconf` and `./configure`. +- mime: store "form escape" as a single bit - PowerShell is better than MS-DOS batches, so the hope is this makes it - easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't - supported inline by AppVeyor on Windows build machines, but we are okay - to keep it in an external script, so it's also an option. + Closes #12374 - 2. CI improvements: +- setopt: check CURLOPT_TFTP_BLKSIZE range on set - - enable tests for a "unity" build job. - - speed-up CI initialization by using shallow clones of the curl repo. - - speed-up CMake MSVC jobs with `TrackFileAccess=false`. - - enable parallelism in `VisualStudioSolution` builds. - - display CMake version before builds. - - always show the CPU in job names. - - tell which jobs are build-only in job names. - - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. - - add `config.log` (autotools) to dumped logs (need to enable manually). + ... instead of later when the transfer is about to happen. - 3. Style: + Closes #12374 - - use single-quotes in YAML like we do in other CI YAML files. - It also allows to drop quoting characters and lighter to write/read. - (keep double quotes for PowerShell strings needing expansion.) +Viktor Szakats (21 Nov 2023) - Closes #11999 +- build: add more picky warnings and fix them -- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows + Enable more picky compiler warnings. I've found these options in the + nghttp3 project when implementing the CMake quick picky warning + functionality for it [1]. - - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. - Before this patch we set it based it on the presence of `stricmp`, - which correctly enabled it on e.g. Windows, but was inaccurate for - other platforms. + `-Wunused-macros` was too noisy to keep around, but fixed a few issues + it revealed while testing. - - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and - LDAPS is not explicitly disabled. This mimics autotools behaviour. - Previously we set it only for Windows LDAP. After this fix, LDAPS is - correctly enabled in default macOS builds. + - autotools: reflect the more precisely-versioned clang warnings. + Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 + - autotools: sync between clang and gcc the way we set `no-multichar`. + - autotools: avoid setting `-Wstrict-aliasing=3` twice. + - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. + It triggers in libtool-generated stub code. - - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) - to avoid deprecation warnings for LDAP API. + - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. - - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. - This doesn't make much sense, but let's do it to sync behaviour with - autotools. + - lib/curl_setup.h: delete duplicate declaration for `fileno`. + Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d + (1999-12-29). This suggests this may not be needed anymore, but if + it does, we may restore this for those specific (non-Windows) systems. + - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since + c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. + - lib: delete unused macro `isxdigit_ascii` since + f65f750742068f579f4ee6d8539ed9d5f0afcb85. + - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. + - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. + - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` + macro. + - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. + - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. + - lib/setopt: fix `-Wunreachable-code-break`. + - lib/system_win32 and lib/timeval: fix double declarations for + `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. + - lib/warnless: fix double declarations in CMake UNITY mode [5]. + This was due to force-disabling the header guard of `warnless.h` to + to reapply it to source code coming after `warnless.c` in UNITY + builds. This reapplied declarations too, causing the warnings. + Solved by adding a header guard for the lines that actually need + to be reapplied. + - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. + - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant + block. + - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. + - lib/vtls/sectransp: suppress `-Wunreachable-code`. + Detected in `else` branches of dynamic feature checks, with results + known at compile-time, e.g. + ```c + if(SecCertificateCopySubjectSummary) /* -> true */ + ``` + Likely fixable as a separate micro-project, but given SecureTransport + is deprecated anyway, let's just silence these locally. + - src/tool_help: delete duplicate declaration for `helptext`. + - src/tool_xattr: fix `-Wunreachable-code`. + - tests: delete duplicate declaration for `unitfail` [8]. + - tests: delete duplicate declaration for `strncasecompare`. + - tests/libtest: delete duplicate declaration for `gethostname`. + Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 + (2010-08-02). + Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc + If there are still systems around with warnings, we may restore the + prototype, but limited for those systems. + - tests/lib2305: delete duplicate declaration for + `libtest_debug_config`. + - tests/h2-download: fix `-Wunreachable-code-break`. + + [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 + f8bf9f/cmake/PickyWarningsC.cmake + [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui + qla5fj45?fullLog=true#L1675 + [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 + 2331#step:7:72 + https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 + 2331#step:7:109 + [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L204 + [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L218 + [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 + 2331#step:7:290 + [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 + 2331#step:9:1193 + [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 + 2331#step:33:1870 + + Closes #12331 + +Daniel Stenberg (21 Nov 2023) + +- transfer: avoid unreachable expression + + If curl_off_t and size_t have the same size (which is common on modern + 64 bit systems), a condition cannot occur which Coverity pointed + out. Avoid the warning by having the code conditionally only used if + curl_off_t actually is larger. + + Follow-up to 1cd2f0072fa482e25baa2 + + Closes #12370 + +Stefan Eissing (21 Nov 2023) + +- transfer: readwrite improvements + + - changed header/chunk/handler->readwrite prototypes to accept `buf`, + `blen` and a `pconsumed` pointer. They now get the buffer to work on + and report back how many bytes they consumed + - eliminated `k->str` in SingleRequest + - improved excess data handling to properly calculate with any body data + left in the headerb buffer + - eliminated `k->badheader` enum to only be a bool + + Closes #12283 + +Daniel Stenberg (21 Nov 2023) - - fix benign typo in variable name. +- RELEASE-NOTES: synced - Ref: #11964 (effort to sync cmake detections with autotools) +Jiří Hruška (21 Nov 2023) - Closes #12006 +- transfer: avoid calling the read callback again after EOF -- autotools: restore `HAVE_IOCTL_*` detections + Regression since 7f43f3dc5994d01b12 (7.84.0) - This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in - 4d73854462f30948acab12984b611e9e33ee41e6 and - c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the - `HAVE_IOCTL` result it generated was unused in the source. But, - I did miss the fact that this had two dependent checks: - `CURL_CHECK_FUNC_IOCTL_FIONBIO`, - `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: - `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. + Bug: https://curl.se/mail/lib-2023-11/0017.html - Regression from 4d73854462f30948acab12984b611e9e33ee41e6 + Closes #12363 - Ref: #11964 (effort to sync cmake detections with autotools) +Daniel Stenberg (21 Nov 2023) - Closes #12008 +- doh: provide better return code for responses w/o addresses -Daniel Stenberg (2 Oct 2023) + Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the + response did not contain any addresses. Now it more accurately returns + CURLE_COULDNT_RESOLVE_HOST. -- RELEASE-PROCEDURE.md: updated coming release dates + Reported-by: lRoccoon on github -- RELEASE-NOTES: synced + Fixes #12365 + Closes #12366 -Viktor Szakats (1 Oct 2023) +Stefan Eissing (21 Nov 2023) -- cmake: pre-cache `HAVE_POLL_FINE` on Windows +- HTTP/2, HTTP/3: handle detach of onoing transfers - Windows doesn't support `poll()`, so we can safely skip checking for - fine poll. + - refs #12356 where a UAF is reported when closing a connection + with a stream whose easy handle was cleaned up already + - handle DETACH events same as DONE events in h2/h3 filters - Closes #12003 + Fixes #12356 + Reported-by: Paweł Wegner + Closes #12364 -- gha: bump actions to latest versions +Viktor Szakats (20 Nov 2023) - - actions@checkout@v4 (from v3 and v2) +- autotools: stop setting `-std=gnu89` with `--enable-warnings` - - fsfe/reuse-action@v2 (from v1) + Do not alter the C standard when building with `--enable-warnings` when + building with gcc. - Closes #12000 + On one hand this alters warning results compared to a default build. + On the other, it may produce different binaries, which is unexpected. -Stefan Eissing (30 Sep 2023) + Also fix new warnings that appeared after removing `-std=gnu89`: -- h2: testcase and fix for pausing h2 streams + - include: fix public curl headers to use the correct printf mask for + `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 + and Visual Studio 2013 and newer. This fixes the printf mask warnings + in examples and tests. E.g. [1] - - refs #11982 where it was noted that paused transfers may - close successfully without delivering the complete data - - made sample poc into tests/http/client/h2-pausing.c and - added test_02_27 to reproduce + - conncache: fix printf format string [2]. - Closes #11989 - Fixes #11982 - Reported-by: Harry Sintonen + - http2: fix potential null pointer dereference [3]. + (seen on Slackware with gcc 11.) -Viktor Szakats (30 Sep 2023) + - libssh: fix printf format string in SFTP code [4]. + Also make MSVC builds compatible with old CRT versions. -- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value + - libssh2: fix printf format string in SFTP code for MSVC. + Applying the same fix as for libssh above. - Before this patch CMake builds accepted any value and it was used at - runtime as-is. This patch make sure that the selected default backend - is also enabled in the build. It also enforces a full lowercase value. + - unit1395: fix `argument is null` and related issues [5]: + - stop calling `strcmp()` with NULL to avoid undefined behaviour. + - fix checking results if some of them were NULL. + - do not pass NULL to printf `%s`. - This improves reproducibility and brings CMake in sync with autotools - which already worked like described above. + - ci: keep a build job with `-std=gnu89` to continue testing for + C89-compliance. We can apply this to other gcc jobs as needed. + Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 - Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 + [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l + ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b + [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 + 2346#step:6:67 + [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 + 2346#step:30:214 + [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 + 2346#step:29:895 + [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 + 2346#step:33:1689 - Closes #11998 + Closes #12346 -- autotools: adjust `CURL_CA_PATH` value to CMake +- autotools: fix/improve gcc and Apple clang version detection - autotools was using the same value as CMake, but with an ending - slash. Delete the ending slash to match configurations. + - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu + may return `n-win32` (also with `-dumpfullversion`). Causing these + errors and failing to enable picky warnings: + ``` + ../configure: line 23845: test: : integer expression expected + ``` + Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 + 3718#step:5:143 - Ref: #11964 (effort to sync cmake detections with autotools) + Fix that by stripping any dash-suffix and handling a dotless (major-only) + version number by assuming `.0` in that case. - Closes #11997 + `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` + Ref: https://github.com/mamedev/mame/pull/9767 -- cmake: detect `sys/wait.h` and `netinet/udp.h` + - fix Apple clang version detection for releases between + 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the + version was under-detected as 3.7 llvm/clang equivalent. - Ref: #11964 (effort to sync cmake detections with autotools) + - fix Apple clang version detection for 'Apple clang version 11.0.0' + and newer where the Apple clang version was detected, instead of its + llvm/clang equivalent. - Closes #11996 + - display detected clang/gcc/icc compiler version. -Daniel Stenberg (30 Sep 2023) + Via libssh2: + - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d + 89875ad + https://github.com/libssh2/libssh2/pull/1187 + - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 + 42b592e + https://github.com/libssh2/libssh2/pull/1232 -- lib: provide and use Curl_hexencode + Closes #12362 - Generates a lower case ASCII hex output from a binary input. +- autotools: delete LCC compiler support bits - Closes #11990 + Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 -- configure: check for the capath by default + Closes #12357 - ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS - L +- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` - cmake: synced - - Assisted-by: Viktor Szakats - Closes #11987 + - tests: verify CMake `DISABLE` options. -- wolfssl: ignore errors in CA path + Make an exception for 2 CMake-only ones, and one more that's + using a different naming scheme, also in autotools and source. - The default wolfSSL_CTX_load_verify_locations() function is quite picky - with the certificates it loads and will for example return error if just - one of the certs has expired. + - cmake: add support for `CURL_DISABLE_HEADERS_API`. - With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it - behaves more similar to what OpenSSL does by default. + Suggested-by: Daniel Stenberg + Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 - Even the set of default certs on my Debian unstable has several expired - ones. + Closes #12353 - Assisted-by: Juliusz Sosinowicz - Assisted-by: Michael Osipov +Jacob Hoffman-Andrews (20 Nov 2023) - Closes #11987 +- hyper: temporarily remove HTTP/2 support -- create-dirs.d: clarify it also uses --output-dirs + The current design of the Hyper integration requires rebuilding the + Hyper clientconn for each request. However, building the clientconn + requires resending the HTTP/2 connection preface, which is incorrect + from a protocol perspective. That in turn causes servers to send GOAWAY + frames, effectively degrading performance to "no connection reuse" in + the best case. It may also be triggering some bugs where requests get + dropped entirely and reconnects take too long. - Reported-by: Robert Simpson - Fixes #11991 - Closes #11995 + This doesn't rule out HTTP/2 support with Hyper, but it may take a + redesign of the Hyper integration in order to make things work. -Viktor Szakats (30 Sep 2023) + Closes #12191 -- appveyor: fix yamlint issues, indent +Jay Satiro (20 Nov 2023) - Also: - - use double quotes in all batch if statements. +- schannel: fix unused variable warning - Closes #11994 + Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 + Reported-by: Viktor Szakats -- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` + Closes https://github.com/curl/curl/pull/12361 - Based on existing autotools logic. +Daniel Stenberg (19 Nov 2023) - Ref: #11964 (effort to sync cmake detections with autotools) +- url: find scheme with a "perfect hash" - Closes #11981 + Instead of a loop to scan over the potentially 30+ scheme names, this + uses a "perfect hash" table. This works fine because the set of schemes + is known and cannot change in a build. The hash algorithm and table size + is made to only make a single scheme index per table entry. -- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` + The perfect hash is generated by a separate tool (scripts/schemetable.c) - Based on existing autotools logic. + Closes #12347 - autotools checks for old versions of the allowlisted target OSes and - disables this feature when seeing them. In CMake we assume we're running - on newer systems and enable regardless of OS version. +- scripts: add schemetable.c - autotools always runs all 3 probes for non-fast-tracked systems and - enables this feature if any one of them was successful. To save - configuration time, CMake stops at the first successful check. + This tool generates a scheme-matching table. - OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD - system. I haven't double-checked if this is correct, but looks odd. + It iterates over a number of different initial and shift values in order + to find the hash algorithm that needs the smallest possible table. - Ref: #11964 (effort to sync cmake detections with autotools) + The generated hash function, table and table size then needs to be used + by the url.c:Curl_getn_scheme_handler() function. - Closes #11979 +Stefan Eissing (19 Nov 2023) -- cmake: fix `HAVE_WRITABLE_ARGV` detection +- vtls/vquic, keep peer name information together - Move detection before the creation of detection results in - `curl_config.h`. + - add `struct ssl_peer` to keep hostname, dispname and sni + for a filter + - allocate `sni` for use in VTLS backend + - eliminate `Curl_ssl_snihost()` and its use of the download buffer + - use ssl_peer in SSL and QUIC filters - Ref: #11964 (effort to sync cmake detections with autotools) + Closes #12349 - Closes #11978 +Viktor Szakats (18 Nov 2023) -- appveyor: minor improvements +- build: always revert `#pragma GCC diagnostic` after use - - run `curl -V` after builds to see if they run and with what features. - Except for one job where a CRT DLL is missing. And ARM64 which should - fail, but is silently not launched instead. + Before this patch some source files were overriding gcc warning options, + but without restoring them at the end of the file. In CMake UNITY builds + these options spilled over to the remainder of the source code, + effecitvely disabling them for a larger portion of the codebase than + intended. - - copy libcurl DLL next to curl tool and tests binaries in shared mode. - This makes it possible to run the tests. (We don't run tests after - these builds yet.) + `#pragma clang diagnostic` didn't have such issue in the codebase. - - list the DLLs and EXEs present after the builds. + Reviewed-by: Marcel Raad + Closes #12352 - - add `DEBUG` variable for CMake builds to allow disabling it, for - testing non-debug builds. (currently enabled for all) +- tidy-up: casing typos, delete unused Windows version aliases - - add commented lines that dump CMake configuration logs for debugging - build/auto-detection issues. + - cmake: fix casing of `UnixSockets` to match the rest of the codebase. - - add gcc version to jobs where missing. + - curl-compilers.m4: fix casing in a comment. - - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 - to the build mix. + - setup-win32: delete unused Windows version constant aliases. - - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. + Reviewed-by: Marcel Raad + Closes #12351 - - delete a duplicate backslash. +- keylog: disable if unused - Closes #11976 + Fully disable keylog code if there is no TLS or QUIC subsystem using it. -- configure: replace adhoc domain with `localhost` in tests + Closes #12350 - Reviewed-by: Daniel Stenberg - Closes #11988 +- cmake: add `CURL_DISABLE_BINDLOCAL` option -- tidy-up: use more example domains + To match similar autotools option. - Also make use of the example TLD: - https://en.wikipedia.org/wiki/.example + Default is `ON`. Reviewed-by: Daniel Stenberg - Closes #11992 + Closes #12345 -Dan Fandrich (29 Sep 2023) +- url: fix `-Wzero-length-array` with no protocols -- runtests: display the test status if tests appear hung + Fixes: + ``` + ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ + -Wc2x-extensions] + 178 | static const struct Curl_handler * const protocols[] = { + | ^ + ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length + -array] + ``` - It sometimes happens that a test hangs during a test run and never - returns. The test harness will wait indefinitely for the results and on - CI servers the CI job will eventually be killed after an hour or two. - At the end of a test run, if results haven't come in within a couple of - minutes, display the status of all test runners and what tests they're - running to help in debugging the problem. + Closes #12344 - This feature is really only kick in with parallel testing enabled, which - is fine because without parallel testing it's usually easy to tell what - test has hung. +- url: fix builds with `CURL_DISABLE_HTTP` - Closes #11980 + Fixes: + ``` + ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' + 456 | Curl_mime_cleanpart(data->state.formp); + | ~~~~~~~~~~~ ^ + ``` -- github/labeler: remove workaround for labeler + Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 - This was added due to what seemed to be a bug regarding the sync-labels: - config option, but it looks like it wasn't necessary. + Closes #12343 - Follow-up to b2b0534e7 +- http: fix `-Wunused-parameter` with no auth and no proxy -Viktor Szakats (29 Sep 2023) + ``` + lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] + bool proxy) + ^ + ``` -- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] + Reviewed-by: Marcel Raad + Closes #12338 -Daniel Stenberg (29 Sep 2023) +Daniel Stenberg (16 Nov 2023) -- docs: replace made up domains with example.com +- TODO: Some TLS options are not offered for HTTPS proxies - in FAQ and MANUAL.md + Closes #12286 + Closes #12342 - - example.com was made for this purpose. +- RELEASE-NOTES: synced - - reduces the risk that one of those domains suddenly start hosting - something nasty and we provide links to them +- duphandle: make dupset() not return with pointers to old alloced data - Closes #11986 + As the blob pointers are to be duplicated, the function must not return + mid-function with lingering pointers to the old handle's allocated data, + as that would lead to double-free in OOM situations. -Michael Osipov (29 Sep 2023) + Make sure to clear all destination pointers first to avoid this risk. -- acinclude.m4: Document proper system truststore on FreeBSD + Closes #12337 - The default system truststore on FreeBSD has been /etc/ssl/certs for many - years now. It is managed canonically through certctl(8) and contains hashed - symlinks for OpenSSL and other TLS providers. - The previous ones require security/ca_root_nss which might not be installed o - r - will not contain any custom CA certificates. +Viktor Szakats (16 Nov 2023) - Closes #11985 +- http: fix `-Wunused-variable` compiler warning -Daniel Stenberg (29 Sep 2023) + Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. -- FAQ: How do I upgrade curl.exe in Windows? + E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + + `CURL_DISABLE_NTLM` on non-Windows. - This is a growing question, better answer it here to get somewhere to - point users to. + ``` + ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl + e] + CURLcode result = CURLE_OK; + ^ + ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus + ed-but-set-variable] + unsigned long *availp; + ^ + ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse + d-but-set-variable] + struct auth *authp; + ^ + ``` - Closes #11984 + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 -Viktor Szakats (28 Sep 2023) + Fixes #12228 + Closes #12335 -- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC +Jay Satiro (16 Nov 2023) - `basename` is present in mingw-w64, missing from MSVC. Pre-cache - accordingly to make configure faster. +- tool: support bold headers in Windows - Notice that `basename` has a bug so we later disable it even with - mingw-w64: - https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li - b/curl_setup.h#L820-L825 + - If virtual terminal processing is enabled in Windows then use ANSI + escape codes Esc[1m and Esc[22m to turn bold on and off. - Closes #11974 + Suggested-by: Gisle Vanem -Daniel Stenberg (28 Sep 2023) + Ref: https://github.com/curl/curl/discussions/11770 -- cmake: add missing checks + Closes https://github.com/curl/curl/pull/12321 - - check for arc4random. To make rand.c use it accordingly. - - check for fcntl - - fix fseek detection - - add SIZEOF_CURL_SOCKET_T - - fix USE_UNIX_SOCKETS - - define HAVE_SNPRINTF to 1 - - check for fnmatch - - check for sched_yield - - remove HAVE_GETPPID duplicate from curl_config.h - - add HAVE_SENDMSG +Viktor Szakats (15 Nov 2023) - Ref: #11964 +- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` - Co-authored-by: Viktor Szakats - Closes #11973 + Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + + `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, + or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while + compiling due to a missing HMAC declaration. -- configure: remove unused checks + The reason is that HMAC is required by `lib/sha256.c` which publishes + `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when + building for libssh2 v1.8.2 (2019-05-25) or older. - - for sys/uio.h - - for fork - - for connect + Make sure to compile the HMAC bits for a successful build. - Ref: #11964 + Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting + them into separate sources isn't practical. - Closes #11973 + Fixes: + ``` + [...] + In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc + url_object.dir/Unity/unity_0_c.c:310: + ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s + truct HMAC_params' + 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { + | ^ + ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par + ams' + [...] + ``` -- lib: remove TIME_WITH_SYS_TIME + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 - It is not used in any code anywhere. + Fixes #12273 + Closes #12332 - Ref: #11964 - Closes #11975 +Daniel Stenberg (15 Nov 2023) -- docs: update curl man page references +- duphandle: also free 'outcurl->cookies' in error path - Detected by the manpage-syntax update + Fixes memory-leak when OOM mid-function - Closes #11963 + Use plain free instead of safefree, since the entire struct is + freed below. -- manpage-syntax: verify curl man page references + Remove some free calls that is already freed in Curl_freeset() - 1. References to curl symbols are now checked that they indeed exist as - man pages. This for \f references as well as the names referenced in the - SEE ALSO section. + Closes #12329 - Allowlist curl.1 since it is not always built in builds +Viktor Szakats (15 Nov 2023) - 2. References to curl symbols that lack section now causes warning, since tha - t - will prevent them from getting linked properly +- config-win32: set `HAVE_SNPRINTF` for mingw-w64 - 3. Check for "bare" references to curl functions and warn, they should be - references + It's available in all mingw-w64 releases. We already pre-fill this + detection in CMake. - Closes #11963 + Closes #12325 -- cmake: add check for suseconds_t +- sasl: fix `-Wunused-function` compiler warning - And fix the HAVE_LONGLONG define + In builds with disabled auths. - Ref: #11964 - Closes #11977 + ``` + lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus + ed-function] + static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, + ^ + 1 warning generated. + ``` + Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st + ep:3:3822 -Viktor Szakats (28 Sep 2023) + Reviewed-by: Daniel Stenberg + Closes #12326 -- tidy-up: whitespace fixes +- build: picky warning updates - Closes #11972 + - cmake: sync some picky gcc warnings with autotools. + - cmake, autotools: add `-Wold-style-definition` for clang too. + - cmake: more precise version info for old clang options. + - cmake: use `IN LISTS` syntax in `foreach()`. -- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Closes #12324 - With new option `CURL_DISABLE_SRP=ON` to force-disable it. - To match existing option and detection logic in autotools. +Daniel Stenberg (15 Nov 2023) - Also: - - fix detecting GnuTLS. - We assume `nettle` as a GnuTLS dependency. - - add CMake GnuTLS CI job. - - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) - TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. - - fix compiler warning when building with GnuTLS and disabled TLS-SRP. - - fix comment typos, whitespace. +- urldata: move cookielist from UserDefined to UrlState - Ref: #11964 + 1. Because the value is not strictly set with a setopt option. - Closes #11967 + 2. Because otherwise when duping a handle when all the set.* fields are + first copied and an error happens (think out of memory mid-function), + the function would easily free the list *before* it was deep-copied, + which could lead to a double-free. -- tool: use our own stderr variable + Closes #12323 - Earlier this year we changed our own stderr variable to use the standard - name `stderr` (to avoid bugs where someone is using `stderr` instead of - the curl-tool specific variable). This solution needed to override the - standard `stderr` symbol via the preprocessor. This in turn didn't play - well with unity builds and caused curl tool to crash or stay silent due - to an uninitialized stderr. This was a hard to find issue, fixed by - manually breaking out one file from the unity sources. +Viktor Szakats (14 Nov 2023) - To avoid two these two tricks, this patch implements a different - solution: Restore using our own local variable for our stderr output and - leave `stderr` as-is. To avoid using `stderr` by mistake, add a - `checksrc` rule (based on logic we already used in lib for `strerror`) - that detects any `stderr` use in `src` and points to using our own - variable instead: `tool_stderr`. +- autotools: avoid passing `LDFLAGS` twice to libcurl - Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e - Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db + autotools passes `LDFLAGS` automatically linker commands. curl's + `lib/Makefile.am` customizes libcurl linker flags. In that + customization, it added `LDFLAGS` to the custom flags. This resulted in + passing `LDFLAGS` _twice_ to the `libtool` command. - Closes #11958 + Most of the time this is benign, but some `LDFLAGS` options can break + the build when passed twice. One such example is passing `.o` files, + e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for + MUSL builds. -Loïc Yhuel (28 Sep 2023) + Passing them twice resulted in duplicate symbol errors: + ``` + libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a + arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] + ld.lld-15: error: duplicate symbol: _start + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + [...] + clang: error: linker command failed with exit code 1 (use -v to see invocatio + n) + ``` -- connect: only start the happy eyeballs timer when needed + This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. + The patch was a works-for-me hack that ended up merged in curl: + https://sourceforge.net/p/curl/bugs/1217/#06ef + With the root cause remaining unclear. - The timeout is only used when there is a second address family, for the - delayed eyeballer. + Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring + `-L` first? This would be unusual and suggests a bug in either the + linker or in `libtool`. - Closes #11939 + The curl build does pass the list of detected libs via its own + `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` + command line, but it's the job of `libtool` to ensure that even + a peculiar linker gets the options in the expected order. Also because + autotools passes `LDFLAGS` last, making it hardly possible to pass + anything after it. -Daniel Stenberg (28 Sep 2023) + Perhaps in the 10 years since this issue, this already got a fix + upstream. -- tool_operate: free 'gateway' correctly + This patch deletes `LDFLAGS` from our customized libcurl options, + leaving a single copy of them as passed by autotools automatically. - Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. + Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + Closes #12310 - Also removed repeated wording in IPFS related error messages. +- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` - Closes #11969 + To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and + curl tool (`CURL_LDFLAGS_BIN`). -Stefan Eissing (28 Sep 2023) + This makes it possible to build libcurl and curl with a single + invocation with lib- and tool-specific custom linker flags. -- lib: move handling of `data->req.writer_stack` into Curl_client_write() + Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, + controlling static/shared, incl. requesting a static curl tool (with + `-static-libtool-libs`) while building both shared and static libcurl. - - move definitions from content_encoding.h to sendf.h - - move create/cleanup/add code into sendf.c - - installed content_encoding writers will always be called - on Curl_client_write(CLIENTWRITE_BODY) - - Curl_client_cleanup() frees writers and tempbuffers from - paused transfers, irregardless of protocol + curl-for-win uses the above and some more. - Closes #11908 + These options are already supported in `Makefile.mk`. CMake has built-in + variables for this. -Loïc Yhuel (28 Sep 2023) + Closes #12312 -- multi: round the timeout up to prevent early wakeups +Jay Satiro (14 Nov 2023) - Curl_timediff rounds down to the millisecond, so curl_multi_perform can - be called too early, then we get a timeout of 0 and call it again. +- tool_cb_hdr: add an additional parsing check - The code already handled the case of timeouts which expired less than - 1ms in the future. By rounding up, we make sure we will never ask the - platform to wake up too early. + - Don't dereference the past-the-end element when parsing the server's + Content-disposition header. - Closes #11938 + As 'p' is advanced it can point to the past-the-end element and prior + to this change 'p' could be dereferenced in that case. -Daniel Stenberg (28 Sep 2023) + Technically the past-the-end element is not out of bounds because dynbuf + (which manages the header line) automatically adds a null terminator to + every buffer and that is not included in the buffer length passed to + the header callback. -- RELEASE-NOTES: spell out that IPFS is via gateway + Closes https://github.com/curl/curl/pull/12320 -- RELEASE-NOTES: synced +Philip Heiduck (14 Nov 2023) -- tool_operate: avoid strlen() -1 on zero length content from file +- .cirrus.yml: freebsd 14 - Follow-up to 65b563a96a226649ba12cb1e + ensure curl works on latest freebsd version - Closes #11959 + Closes #12053 -- tool_operate: fix memory mixups +Daniel Stenberg (13 Nov 2023) - Switch to plain getenv() from curl_getenv() to avoid the allocation and - having to keep track of which free() or curl_free() that need to be - used. +- easy: in duphandle, init the cookies for the new handle - Coverity found issues and a memory leak. + ... not the source handle. - Follow-up to 65b563a96a226649ba12cb1e + Closes #12318 - Closes #11959 +- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set -Viktor Szakats (27 Sep 2023) + Previously it would unconditionally use the size, which is set to -1 + when strlen is requested. -- curl-functions.m4: fixup recent bad edits + Updated test 544 to verify. - Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + Closes #12317 - Closes #11966 +- RELEASE-NOTES: synced -Daniel Stenberg (27 Sep 2023) +- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped -- curl-functions.m4: fix include line + Closes #12315 - This made the getaddrinfo detection fail, but we did not spot it in the - CI because it graciously falled back to using legacy functions instead! +- urldata: move hstslist from 'set' to 'state' - Follow-up to 96c29900bcec (#11940) + To make it work properly with curl_easy_duphandle(). This, because + duphandle duplicates the entire 'UserDefined' struct by plain copy while + 'hstslist' is a linked curl_list of file names. This would lead to a + double-free when the second of the two involved easy handles were + closed. - Closes #11965 + Closes #12315 -- inet_ntop: add typecast to silence Coverity +- test1900: verify duphandle with HSTS using multiple files - CID 1024653: Integer handling issues (SIGN_EXTENSION) + Closes #12315 - Suspicious implicit sign extension: "src[i]" with type "unsigned char - const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to - type "int" (32 bits, signed), then sign-extended to type "unsigned long" - (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than - 0x7FFFFFFF, the upper bits of the result will all be 1. +Goro FUJI (13 Nov 2023) - 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); +- http: allow longer HTTP/2 request method names - The value will not be greater than 0x7FFFFFFF so this still cannot - happen. + - Increase the maximum request method name length from 11 to 23. - Also, switch to ints here instead of longs. The values stored are 16 bit - so at least no need to use 64 bit variables. Also, longs are 32 bit on - some platforms so this logic still needs to work with 32 bits. + For HTTP/1.1 and earlier there's not a specific limit in libcurl for + method length except that it is limited by the initial HTTP request + limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same + and there was no specific limit. - Closes #11960 + According to Internet Assigned Numbers Authority (IANA) the longest + registered method is UPDATEREDIRECTREF which is 17 characters. -- docs: adapt SEE ALSO sections to new requirements + Also there are unregistered methods used by some companies that are + longer than 11 characters. - To please manpage-syntax.pl used by test 1173 + The limit was originally added by 61f52a97 but not used until fc2f1e54. - Closes #11957 + Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml -- manpage-syntax.pl: verify SEE ALSO syntax + Closes https://github.com/curl/curl/pull/12311 - - Enforce a single reference per .BR line - - Skip the quotes around the section number for example (3) - - Insist on trailing commas on all lines except the last - - Error on comma on the last SEE ALSO entry +Jay Satiro (12 Nov 2023) - - List the entries alpha-sorted, not enforced just recommended +- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does - Closes #11957 + - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB + and CURLOPT_PROXY_CAINFO_BLOB docs. -- connect: expire the timeout when trying next + All the other _BLOB option docs already have the same explanation. - ... so that it gets called again immediately and can continue trying - addresses to connect to. Otherwise it might unnecessarily wait for a - while there. + Closes https://github.com/curl/curl/pull/12277 - Fixes #11920 - Reported-by: Loïc Yhuel - Closes #11935 +Viktor Szakats (11 Nov 2023) -- http: remove wrong comment for http_should_fail +- tidy-up: dedupe Windows system libs in cmake - Reported-by: Christian Schmitz - Ref: #11936 - Closes #11941 + Reviewed-by: Daniel Stenberg + Closes #12307 -Dan Fandrich (26 Sep 2023) +Junho Choi (11 Nov 2023) -- tool_setopt: remove unused function tool_setopt_flags +- ci: test with latest quiche release (0.19.0) - This function is identical to tool_setopt_bitmask except that it treats - the argument as unsigned. + Closes #12180 - Closes #11943 +- quiche: use quiche_conn_peer_transport_params() -Viktor Szakats (26 Sep 2023) + In recent quiche, transport parameter API is separated + with quiche_conn_peer_transport_params(). + (https://github.com/cloudflare/quiche/pull/1575) + It breaks with bulding with latest(post 0.18.0) quiche. -- cmake: add feature checks for `memrchr` and `getifaddrs` + Closes #12180 - - `HAVE_MEMRCHR` for `memrchr`. - - `HAVE_GETIFADDRS` for `getifaddrs`. - This was present in `lib/curl_config.h.cmake` but missed the detection - logic. +Daniel Stenberg (11 Nov 2023) - To match existing autotools feature checks. +- Makefile: generate the VC 14.20 project files at dist-time - Closes #11954 + Follow-up to 28287092cc5a6d6ef8 (#12282) -- cmake: move global headers to specific checks + Closes #12290 - Before this patch we added standard headers unconditionally to the - global list of headers used for feature checks. This is unnecessary - and also doesn't help CMake 'Generate' performance. This patch moves - these headers to each feature check where they are actually needed. - Stop using `stddef.h`, as it seems unnecessary. +Sam James (11 Nov 2023) - I've used autotools' `m4/curl-functions.m4` to figure out these - dependencies. +- misc: fix -Walloc-size warnings - Also delete checking for the C89 standard header `time.h`, that I - missed in the earlier commit. + GCC 14 introduces a new -Walloc-size included in -Wextra which gives: - Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + ``` + src/tool_operate.c: In function ‘add_per_transfer’: + src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo + r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] + 213 | p = calloc(sizeof(struct per_transfer), 1); + | ^ + src/var.c: In function ‘addvariable’: + src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type + struct var’ with size ‘32’ [-Walloc-size] + 361 | p = calloc(sizeof(struct var), 1); + | ^ + ``` - Closes #11951 + The calloc prototype is: + ``` + void *calloc(size_t nmemb, size_t size); + ``` -- src/mkhelp: make generated code pass `checksrc` + So, just swap the number of members and size arguments to match the + prototype, as we're initialising 1 struct of size `sizeof(struct + ...)`. GCC then sees we're not doing anything wrong. - Closes #11955 + Closes #12292 -- tests: show which curl tool `runtests.pl` is using +Mark Gaiser (11 Nov 2023) - To help debugging when there is issue finding or running it. +- IPFS: bugfixes - Closes #11953 + - Fixed endianness bug in gateway file parsing + - Use IPFS_PATH in tests where IPFS_DATA was used + - Fixed typos from traling -> trailing + - Fixed broken link in IPFS.md -- CI/azure: make `MAKEFLAGS` global to parallelize all jobs + Follow-up to 859e88f6533f9e - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi - th -j3) + Reported-by: Michael Kaufmann + Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 + Closes #12305 - Closes #11952 +Daniel Stenberg (11 Nov 2023) -- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 +- VULN-DISCLOSURE-POLIC: remove broken link to hackerone - Also delete an accidental variable reference. + It should ideally soon not be done from hackerone anyway - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 + Closes #12308 - Closes #11945 +Andrew Kurushin (11 Nov 2023) -Daniel Stenberg (26 Sep 2023) +- schannel: add CA cache support for files and memory blobs -- docs: add see also curl_multi_get_handles to some man pages + - Support CA bundle and blob caching. - Assisted-by: Jay Satiro + Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. - Closes #11942 + Closes https://github.com/curl/curl/pull/12261 -Viktor Szakats (26 Sep 2023) +Daniel Stenberg (10 Nov 2023) -- cmake: assume `_fseeki64` and no `fseeko` on Windows +- RELEASE-NOTES: synced - `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and - at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. +Charlie C (10 Nov 2023) - `fseeko` is not present in any of these. +- cmake: option to disable install & drop `curlu` target when unused - (mingw-w64 1.0 also offers `fseeko64`.) + This patch makes the following changes: + - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. + - Removes the target `curlu` when the option `BUILD_TESTING` is set to + `OFF` - to prevent it from being loaded in Visual Studio. - [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 + Closes #12287 - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 +Kai Pastor (10 Nov 2023) - Closes #11950 +- cmake: fix multiple include of CURL package -- build: delete checks for C89 standard headers + Fixes errors on second `find_package(CURL)`. This is a frequent case + with transitive dependencies: + ``` + CMake Error at ...: + add_library cannot create ALIAS target "CURL::libcurl" because another + target with the same name already exists. + ``` - Delete checks and guards for standard C89 headers and assume these are - available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, - `stddef.h`, `signal.h`. + Test to reproduce: + ```cmake + cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher - Some of these we already used unconditionally, some others we only used - for feature checks. + project(curl) - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i - n CMake) + set(CURL_DIR "example/lib/cmake/CURL/") + find_package(CURL CONFIG REQUIRED) + find_package(CURL CONFIG REQUIRED) # fails - Closes #11940 + add_executable(main main.c) + target_link_libraries(main CURL::libcurl) + ``` -Stefan Eissing (26 Sep 2023) + Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes + Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html + Ref: #12300 + Assisted-by: Harry Mallon + Closes #11913 -- multiif.h: remove Curl_multi_dump declaration +Viktor Szakats (8 Nov 2023) - Follow-up to d850eea2 which removed the Curl_multi_dump definition. +- tidy-up: use `OPENSSL_VERSION_NUMBER` - Closes https://github.com/curl/curl/pull/11946 + Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. + Before this patch some places used `OPENSSL_VERSION_MAJOR`. -Jay Satiro (26 Sep 2023) + Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't + define any version number in these implementations: BoringSSL, AWS-LC, + LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to + `opensslv.h`. This wasn't causing a deeper problem because the code is + looking for v3, which is only provided by OpenSSL/quictls as of now. -- config-win32: define HAVE__FSEEKI64 + According to https://github.com/openssl/openssl/issues/17517, the macro + `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. - Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that - calls _fseeki64 if it is available. + Reviewed-by: Marcel Raad + Closes #12298 - Closes https://github.com/curl/curl/pull/11944 +Daniel Stenberg (8 Nov 2023) -- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER +- resolve.d: drop a multi use-sentence - - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place - even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. + Since the `multi:` keyword adds that message. - The behavior is verified by test2048. + Reported-by: 積丹尼 Dan Jacobson + Fixes https://github.com/curl/curl/discussions/12294 + Closes #12295 - Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 - Reported-by: claudiusaiz@users.noreply.github.com +- content_encoding: make Curl_all_content_encodings allocless - Bug: https://github.com/curl/curl/discussions/11910 - Reported-by: Hakan Sunay Halil + - Fixes a memory leak pointed out by Coverity + - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? + id=63947 + - Avoids unncessary allocations - Closes https://github.com/curl/curl/pull/11930 + Follow-up ad051e1cbec68b2456a22661b -Stefan Eissing (26 Sep 2023) + Closes #12289 -- openssl: improve ssl shutdown handling +Michael Kaufmann (7 Nov 2023) - - If SSL shutdown is not finished then make an additional call to - SSL_read to gather additional tracing. +- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 - - Fix http2 and h2-proxy filters to forward do_close() calls to the next - filter. + Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), + avoid it and use "http/1.1" instead. - For example h2 and SSL shutdown before and after this change: + This reverts commit df856cb5c9 (#10183). - Before: + Fixes #12259 + Closes #12285 - Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> - ssl_cf_destroy +Daniel Stenberg (7 Nov 2023) - After: +- Makefile.am: drop vc10, vc11 and vc12 projects from dist - Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> - ssl_cf_close + They are end of life products. Support for generating them remain in the + repo for a while but this change drops them from distribution. - Note that currently the tracing does not show output on the connection - closure handle. Refer to discussion in #11878. + Closes #12288 - Ref: https://github.com/curl/curl/discussions/11878 +David Suter (7 Nov 2023) - Closes https://github.com/curl/curl/pull/11858 +- projects: add VC14.20 project files -Loïc Yhuel (26 Sep 2023) + Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. + OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why + this is missing. Updated the templates to produce a VC14.20 project. + Project opens in Visual Studio 2019 as expected. -- multi: fix small timeouts + Closes #12282 - Since Curl_timediff rounds down to the millisecond, timeouts which - expire in less than 1ms are considered as outdated and removed from the - list. We can use Curl_timediff_us instead, big timeouts could saturate - but this is not an issue. +Daniel Stenberg (7 Nov 2023) - Closes #11937 +- curl: move IPFS code into src/tool_ipfs.[ch] -Viktor Szakats (25 Sep 2023) + - convert ensure_trailing into ensure_trailing_slash + - strdup the URL string to own it proper + - use shorter variable names + - combine some expressions + - simplify error handling in ipfs_gateway() + - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached + - ipfs-gateway.d polish and simplification + - shorten ipfs error message + make them "synthetic" -- cmake: fix stderr initialization in unity builds + Closes #12281 - Before this patch, in certain build configurations the curl tool may - not have displayed anything (debug, macOS), or crashed at startup - (debug, Windows). +Viktor Szakats (6 Nov 2023) - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 - Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db +- build: delete support bits for obsolete Windows compilers - Closes #11929 + - Pelles C: Unclear status, failed to obtain a fresh copy a few months + ago. Possible website is HTTP-only. ~10 years ago I left this compiler + dealing with crashes and other issues with no response on the forum + for years. It has seen some activity in curl back in 2021. + - LCC: Last stable release in September 2002. + - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned + in 2006. + - Borland C++: We dropped Borland C++ support in 2018. + - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 + (or possibly 2008) as a minimum. -- cmake: fix missing `zlib.h` when compiling `libcurltool` + Closes #12222 - Came up while testing debug/testing build for Windows. I'm not sure why - it didn't come up in earlier tests with similar config. - `tool_hugehelp.c` might indeed require `zlib.h` and without linking - `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected - dependency headers to the compiler command. +- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` - ``` - [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang - --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 - 6_64 - -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO - DE - -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib - -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li - ne-argument - -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static - -libgcc - -lucrt [...] -O3 -DNDEBUG -municode -MD - -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d - -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x - 64/src/tool_hugehelp.c - .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f - ile not found - 6 | #include - | ^~~~~~~~ - ``` + We use `stdint.h` unconditionally in all places except one. These uses + are imposed by external dependencies / features. nghttp2, quic, wolfSSL + and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that + any of these features make curl require a C99 compiler. (In case of + MSVC, this means Visual Studio 2010 or newer.) - Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 + This patch changes the single use of `stdint.h` guarded by + `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using + `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used + anywhere else, allowing to delete this feature check as well. - Closes #11927 + Closes #12275 -- cmake: fix duplicate symbols when linking tests +Daniel Stenberg (6 Nov 2023) - The linker resolves this automatically in non-unity builds. In unity - builds the linker cannot drop a single object with the duplicates, - resulting in these errors. The root issue is that we started including - certain objects both via both libcurlu and libcurltool libs. +- tool_operate: do not mix memory models - Regression from 39e7c22bb459c2e818f079984989a26a09741860 + Make sure 'inputpath' only points to memory allocated by libcurl so that + curl_free works correctly. + + Pointed out by Coverity + + Follow-up to 859e88f6533f9e1f890 + + Closes #12280 + +Stefan Eissing (6 Nov 2023) + +- lib: client writer, part 2, accounting + logging + + This PR has these changes: + + Renaming of unencode_* to cwriter, e.g. client writers + - documentation of sendf.h functions + - move max decode stack checks back to content_encoding.c + - define writer phase which was used as order before + - introduce phases for monitoring inbetween decode phases + - offering default implementations for init/write/close + + Add type paramter to client writer's do_write() + - always pass all writes through the writer stack + - writers who only care about BODY data will pass other writes unchanged + + add RAW and PROTOCOL client writers + - RAW used for Curl_debug() logging of CURLINFO_DATA_IN + - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and + Curl_pgrsSetDownloadCounter() + - remove all updates of data->req.bytecount and calls to + Curl_pgrsSetDownloadCounter() and Curl_debug() from other code + - adjust test457 expected output to no longer see the excess write + + Closes #12184 + +Daniel Stenberg (6 Nov 2023) + +- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw + + Closes #12278 + +Viktor Szakats (6 Nov 2023) + +- rand: fix build error with autotools + LibreSSL + + autotools unexpectedly detects `arc4random` because it is also looking + into dependency libs. One dependency, LibreSSL, happens to publish an + `arc4random` function (via its shared lib before v3.7, also via static + lib as of v3.8.2). When trying to use this function in `lib/rand.c`, + its protoype is missing. To fix that, curl included a prototype, but + that used a C99 type without including `stdint.h`, causing: - Windows errors: ``` - [ 3%] Linking C executable unit1303.exe - [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul - tibyte.c.obj - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch - ar': - C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve - rt_UTF8_to_wchar' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:44: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT - F8': - C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve - rt_wchar_to_UTF8' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:66: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': - C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 - _open' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:92: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': - C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 - 2_fopen' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:120: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': - [...] + ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' + 37 | uint32_t arc4random(void); + | ^ + 1 error generated. ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 - aa4ehny5q#L247 - macOS errors: - ``` - [ 56%] Linking C executable unit1302 - duplicate symbol '_curlx_sotouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_sitouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_uztosz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - [...] - ``` - with config: - ``` - -DCMAKE_UNITY_BUILD=ON \ - -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ - -DBUILD_SHARED_LIBS=ON \ - -DBUILD_STATIC_LIBS=OFF - ``` + This patch improves this by dropping the local prototype and instead + limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide + their own random source anyway. - Closes #11926 + The better fix would be to teach autotools to not link dependency libs + while detecting `arc4random`. -- cmake: lib `CURL_STATICLIB` fixes (Windows) + LibreSSL publishing a non-namespaced `arc4random` tracked here: + https://github.com/libressl/portable/issues/928 - - always define `CURL_STATICLIB` when building libcurl for Windows. + Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 - This disables `__declspec(dllexport)` for exported libcurl symbols. - In normal mode (hide symbols) these exported symbols are specified - via `libcurl.def`. When not hiding symbols, all symbols are exported - by default. + Reviewed-by: Daniel Stenberg + Fixes #12257 + Closes #12274 - Regression from 1199308dbc902c52be67fc805c72dd2582520d30 +Daniel Stenberg (5 Nov 2023) - Fixes #11844 +- RELEASE-NOTES: synced - - fix to omit `libcurl.def` when not hiding private symbols. +- strdup: do Curl_strndup without strncpy - Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 + To avoid (false positive) gcc-13 compiler warnings. - - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also - omitting `libcurl.def` in this case, and exporting all symbols - instead. This ensures that a shared curl tool can access all debug - functions which are not normally exported from libcurl DLL. + Follow-up to 4855debd8a2c1cb - - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" - target. + Assisted-by: Jay Satiro + Reported-by: Viktor Szakats + Fixes #12258 - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 +Enno Boland (5 Nov 2023) - - delete duplicate `BUILDING_LIBCURL` definitions. +- HTTP: fix empty-body warning - - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. + This change fixes a compiler warning with gcc-12.2.0 when + `-DCURL_DISABLE_BEARER_AUTH=ON` is used. - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 + /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': + /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp + ty body in an 'else' statement [-Wempty-body] + 1147 | ; + | ^ - Closes #11914 + Closes #12262 -Daniel Stenberg (25 Sep 2023) +Daniel Stenberg (5 Nov 2023) -- RELEASE-NOTES: synced +- openssl: identify the "quictls" backend correctly -Dan Fandrich (25 Sep 2023) + Since vanilla OpenSSL does not support the QUIC API I think it helps + users to identify the correct OpenSSL fork in version output. The best + (crude) way to do that right now seems to be to check if ngtcp2 support + is enabled. -- tests: fix log directory path in IPFS tests + Closes #12270 - Hard-coding the log directory name fails with parallel tests. +Mark Gaiser (5 Nov 2023) - Follow-up to 65b563a96 +- curl: improved IPFS and IPNS URL support - Ref: #8805 + Previously just ipfs:// and ipns:// was supported, which is + too strict for some usecases. -Daniel Stenberg (25 Sep 2023) + This patch allows paths and query arguments to be used too. + Making this work according to normal http semantics: -- curl_multi_get_handles: get easy handles from a multi handle + ipfs:///foo/bar?key=val + ipns:///foo/bar?key=val - Closes #11750 + The gateway url support is changed. + It now only supports gateways in the form of: -Stefan Eissing (25 Sep 2023) + http:///foo/bar + http:// -- http: h1/h2 proxy unification + Query arguments here are explicitly not allowed and trigger an intended + malformed url error. - - use shared code for setting up the CONNECT request - when tunneling, used in HTTP/1.x and HTTP/2 proxying - - eliminate use of Curl_buffer_send() and other manipulations - of `data->req` or `data->state.ulbuf` + There also was a crash when IPFS_PATH was set with a non trailing + forward slash. This has been fixed. - Closes #11808 + Lastly, a load of test cases have been added to verify the above. -Natanael Copa (25 Sep 2023) + Reported-by: Steven Allen + Fixes #12148 + Closes #12152 -- lib: use wrapper for curl_mime_data fseek callback +Harry Mallon (5 Nov 2023) - fseek uses long offset which does not match with curl_off_t. This leads - to undefined behavior when calling the callback and caused failure on - arm 32 bit. +- docs: KNOWN_BUGS cleanup - Use a wrapper to solve this and use fseeko which uses off_t instead of - long. + * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. + Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 - Thanks to the nice people at Libera IRC #musl for helping finding this - out. + * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. + Fixed in #11806 - Fixes #11882 - Fixes #11900 - Closes #11918 + * Remove mention of aws-sigv4 query empty value problems -- configure: sort AC_CHECK_FUNCS + * Remove mention of aws-sigv4 missing amz-content-sha256 + Fixed in #9995 - No functional changes. +- http_aws_sigv4: canonicalise valueless query params -Daniel Stenberg (25 Sep 2023) + Fixes #8107 + Closes #12244 -- warnless: remove unused functions +Michael Kaufmann (4 Nov 2023) - Previously put there for use with the intel compiler +- docs: preserve the modification date when copying the prebuilt man page - Closes #11932 + The previously built man page "curl.1" must be copied with the original + modification date, otherwise the man page is never updated. -- GHA/linux: run singleuse to detect single-use global functions + This fixes a bug that has been introduced with commit 2568441cab. - Use --unit for configure --enable-debug builds + Reviewed-by: Dan Fandrich + Reviewed-by: Daniel Stenberg - Closes #11932 + Closes #12199 -- singleuse: add scan for use in other source codes +Daniel Stenberg (4 Nov 2023) - This should reduce false-positive to almost zero. Checks for presence in - unit tests if --unit is specified, which is intended for debug builds - where unit testing is enabled. +- docs: remove bold from some man page SYNOPSIS sections - Closes #11932 + In the name of consistency -- multi: remove Curl_multi_dump + Closes #12267 - A debug-only function that is basically never used. Removed to ease the - use of the singleuse script to detect non-static functions not used - outside the file where it is defined. +- openssl: two multi pointer checks should probably rather be asserts - Closes #11931 + ... so add the asserts now and consider removing the dynamic checks in a + future. -Viktor Szakats (24 Sep 2023) + Ref: #12261 + Closes #12264 -- tests: fix compiler warnings +boilingoden (4 Nov 2023) - Seen with llvm 17 on Windows x64. +- docs: add supported version for the json write-out - ``` - .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration - for non-static variable 'logdir' [-Wmissing-variable-declarations] - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration - for non-static variable 'loglockfile' [-Wmissing-variable-declarations] - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat - ion for non-static variable 'logdir' [-Wmissing-variable-declarations] - 43 | const char *logdir = "log"; - | ^ - .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl - e is not intended to be used outside of this translation unit - 43 | const char *logdir = "log"; - | ^ - .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator - here [-Wcomma] - 350 | ++d, ++s; - | ^ - .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa - rning - 350 | ++d, ++s; - | ^~~ - | (void)( ) - ``` + xref: https://curl.se/changes.html#7_70_0 - ``` - .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; - | ~ ^ ~~~~~~~~~~~~~ - 1 warning generated. + Closes #12266 - .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison - 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] - 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. +Viktor Szakats (3 Nov 2023) - .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > - 2147483647 is always false [-Wtautological-type-limit-compare] - 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - ``` +- appveyor: make VS2008-built curl tool runnable - Seen on macOS Intel: - ``` - .../curl/tests/server/sws.c:440:64: warning: field precision should have type - 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] - msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" - , - ~~^~ - 1 warning generated. - ``` + By linking the CRT statically. This avoids the error about missing + runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`. - Closes #11925 + Closes #12263 -Jay Satiro (24 Sep 2023) +Stefan Eissing (3 Nov 2023) -- url: fix netrc info message +- url: proxy ssl connection reuse fix - - Fix netrc info message to use the generic ".netrc" filename if the - user did not specify a netrc location. + - tunnel https proxy used for http: transfers does + no check if proxy-ssl configuration matches + - test cases added, test_10_12 fails on 8.4.0 - - Update --netrc doc to add that recent versions of curl on Windows - prefer .netrc over _netrc. + Closes #12255 + +Jay Satiro (3 Nov 2023) + +- curl_sspi: support more revocation error names in error messages + + - Add these revocation errors to sspi error list: + CRYPT_E_NO_REVOCATION_DLL, CRYPT_E_NO_REVOCATION_CHECK, + CRYPT_E_REVOCATION_OFFLINE and CRYPT_E_NOT_IN_REVOCATION_DATABASE. + + Prior to this change those error codes were not matched to their macro + name and instead shown as "unknown error". Before: - * Couldn't find host google.com in the (nil) file; using defaults + + schannel: next InitializeSecurityContext failed: + Unknown error (0x80092013) - The revocation function was + unable to check revocation because the revocation server was offline. After: - * Couldn't find host google.com in the .netrc file; using defaults - Closes https://github.com/curl/curl/pull/11904 + schannel: next InitializeSecurityContext failed: + CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was + unable to check revocation because the revocation server was offline. -Dan Fandrich (23 Sep 2023) + Bug: https://github.com/curl/curl/issues/12239 + Reported-by: Niracler Li -- wolfssh: do cleanup in Curl_ssh_cleanup + Closes https://github.com/curl/curl/pull/12241 - Closes: #11921 +- strdup: don't allow Curl_strndup to read past a null terminator -Daniel Stenberg (24 Sep 2023) + - Use malloc + strncpy instead of Curl_memdup to dupe the string before + null terminating it. -- tool_listhelp: regenerated + Prior to this change if Curl_strndup was passed a length longer than + the allocated string then it could copy out of bounds. - Polished the --ipfs-gateway description + This change is for posterity. Curl_strndup was added in the parent + commit and currently none of the calls to it pass a length that would + cause it to read past the allocated length of the input. - Fixed the --trace-config description + Follow-up to d3b3ba35. - The script also fixed some other small mistakes + Closes https://github.com/curl/curl/pull/12254 - Closes #11923 +Daniel Stenberg (2 Nov 2023) -Viktor Szakats (23 Sep 2023) +- lib: add and use Curl_strndup() -- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) + The Curl_strndup() function is similar to memdup(), but copies 'n' bytes + then adds a terminating null byte ('\0'). - Also fix to export all symbols in Windows debug builds, making - `-debug-dyn` builds work with `-DCURL_STATICLIB` set. + Closes #12251 - Ref: https://github.com/curl/curl/pull/11914 (same for CMake) +- CURPOST_POSTFIELDS.3: add CURLOPT_COPYPOSTFIELDS in SEE ALSO - Closes #11924 +Stefan Eissing (2 Nov 2023) -Daniel Stenberg (23 Sep 2023) +- pytest: use lower count in repeat tests -- quic: set ciphers/curves the same way regular TLS does + - lower large iteration counts in some tests somewhat for + the same coverage with less duration - for OpenSSL/BoringSSL + Closes #12248 - Fixes #11796 - Reported-by: Karthikdasari0423 on github - Assisted-by: Jay Satiro - Closes #11836 +Daniel Stenberg (2 Nov 2023) -- test457: verify --max-filesize with chunked encoding +- RELEASE-NOTES: synced -- lib: let the max filesize option stop too big transfers too +- docs: clarify that curl passes on input unfiltered - Previously it would only stop them from getting started if the size is - known to be too big then. + ... for several options. - Update the libcurl and curl docs accordingly. + Reported-by: Ophir Lojkine - Fixes #11810 - Reported-by: Elliot Killick - Assisted-by: Jay Satiro - Closes #11820 + Closes #12249 -Viktor Szakats (23 Sep 2023) +- urlapi: when URL encoding the fragment, pass in the right length -- mingw: delete support for legacy mingw.org toolchain + A benign bug because it would only add an extra null terminator. - Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: - https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ - Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. - It supported the x86 CPU only and used a old Windows API header and - implib set, often causing issues. It also misses most modern Windows - features, offering old versions of both binutils and gcc (no llvm/clang - support). It was last updated 2 years ago. + Made lib1560 get a test that runs this code. - curl now relies on toolchains based on the mingw-w64 project: - https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ - https://www.msys2.org/ https://github.com/msys2/msys2 - https://github.com/mstorsjo/llvm-mingw - (Also available via Linux and macOS package managers.) + Closes #12250 - Closes #11625 +Stefan Eissing (2 Nov 2023) -Mark Gaiser (23 Sep 2023) +- vtls: late clone of connection ssl config -- curl: add support for the IPFS protocols: + - perform connection cache matching against `data->set.ssl.primary` + and proxy counterpart + - fully clone connection ssl config only when connection is used - - ipfs:// - - ipns:// + Closes #12237 - This allows you tu use ipfs in curl like: - curl ipfs:// - and - curl ipns:// +- msh3: error when built with CURL_DISABLE_SOCKETPAIR set - For more information consult the readme at: - https://curl.se/docs/ipfs.html + Reported-by: Gisle Vanem + Closes #12252 + Fixes #12213 - Closes #8805 +Daniel Stenberg (2 Nov 2023) -Daniel Stenberg (23 Sep 2023) +- hsts: skip single-dot hostname -- bufq: remove Curl_bufq_skip_and_shift (unused) + Reported-by: Maksymilian Arciemowicz - Closes #11915 + Closes #12247 -- scripts/singleuse.pl: add curl_global_trace +- vtls: fix build without proxy -Viktor Szakats (22 Sep 2023) + Follow-up to bf0e278a3c54bc7fee7360da17c -- cmake: fix unity symbol collisions in h2 builds + closes #12243 - Regression from 331b89a319d0067fa1e6441719307cfef9c7960f +- docs/example/keepalive.c: show TCP keep-alive options - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #11912 + Closes #12242 -Daniel Stenberg (22 Sep 2023) +- lib1560: verify appending blank URL encoded query string -- RELEASE-NOTES: synced +- urlapi: skip appending NULL pointer query -Dan Fandrich (21 Sep 2023) + Reported-by: kirbyn17 on hackerone -- github/labeler: improve the match patterns + Closes #12240 - This includes new rules for setting the appleOS and logging labels and - matches on some example files. Also, enable dot mode for wildcard - matches in the .github directory. +- lib1560: verify setting host to "" with and without URL encode -Daniel Stenberg (21 Sep 2023) +- urlapi: avoid null deref if setting blank host to url encode -- upload-file.d: describe the file name slash/backslash handling + Reported-by: kirbyn17 on hackerone - Closes #11911 + Closes #12240 -Jakub Jelen (21 Sep 2023) +- dynbuf: assert for NULL pointer inputs -- libssh: cap SFTP packet size sent + Help us catch more mistakes. - Due to libssh limitations + Closes #12238 - Signed-off-by: Jakub Jelen +- HTTP3: ngtcp2 builds are no longer experimental - Closes #11804 + The other HTTP/3 backends are still experimental. -Daniel Stenberg (21 Sep 2023) + Closes #12235 -- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 +Stefan Eissing (31 Oct 2023) - Closes #11905 +- vtls: cleanup SSL config management -- mailmap: unify Michael Osipov under a single email + - remove `Curl_ssl_get_config()`, no longer needed -Ted Lyngmo (21 Sep 2023) + Closes #12204 -- docs: use CURLSSLBACKEND_NONE +Daniel Stenberg (31 Oct 2023) - [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in - documentation and examples. +- libcurl-thread.3: simplify the TLS section - Signed-off-by: Ted Lyngmo + All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August + 2016. - Closes #11909 + Closes #12233 -Dan Fandrich (21 Sep 2023) +- configure: better --disable-http -- github/labeler: give the sync-labels config item a default value + - disable HTTPS-proxy as well, since it can't work without HTTP - This shouldn't be necessary and is likely a bug with this beta version - of the labeller. + - curl_setup: when HTTP is disabled, also disable all features that are + HTTP-only - Also, fix the negative matches for the documentation label. + - version: HTTPS-proxy only exists if HTTP support exists - Follow-up to dd12b452a - Closes #11907 + Closes #12223 -- github/labeler: fix up more the labeler config format +- http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine - The new version didn't like the workaround we had for a bug in the - previous labeler version, and it should no longer be needed. + Finding a 'Content-Range:' in the response changed the handling. - Follow-up to dd12b452a - Closes #11906 + Add test case 1475 to verify -C - with 416 and Content-Range: header, + which is almost exactly like test 194 which instead uses a fixed -C + offset. Adjusted test 194 to also be considered fine. -- github/labeler: fix indenting to try to appease labeller + Fixes #10521 + Reported-by: Smackd0wn + Fixes #12174 + Reported-by: Anubhav Rai + Closes #12176 - Follow-up to dd12b452a +Stefan Eissing (30 Oct 2023) -Jay Satiro (21 Sep 2023) +- GHA: fix checkout of quictls repository to use correct branch name -- libssh2: fix error message on failed pubkey-from-file + Follow-up to c868b0e30f10cd0ac7 - - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error - message "SSH public key authentication failed: Reason unknown (-1)". + Closes #12232 - When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a - generic error and therefore doesn't set an error message. AFAICT that is - not documented behavior. +Daniel Stenberg (30 Oct 2023) - Prior to this change libcurl retrieved the last set error message which - would be from a previous function failing. That resulted in misleading - auth failed error messages in verbose mode. +- docs/example/localport.c: show off CURLOPT_LOCALPORT - Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 - Reported-by: consulion@users.noreply.github.com + Closes #12230 - Closes https://github.com/curl/curl/pull/11881 +- docs/examples/interface.c: show CURLOPT_INTERFACE use -Stefan Eissing (21 Sep 2023) + Although super simple. -- pytest: exclude test_03_goaway in CI runs due to timing dependency + Closes #12229 - Closes #11860 +Viktor Szakats (30 Oct 2023) -- lib: disambiguate Curl_client_write flag semantics +- build: fix compiler warning with auths disabled - - use CLIENTWRITE_BODY *only* when data is actually body data - - add CLIENTWRITE_INFO for meta data that is *not* a HEADER - - debug assertions that BODY/INFO/HEADER is not used mixed - - move `data->set.include_header` check into Curl_client_write - so protocol handlers no longer have to care - - add special in FTP for `data->set.include_header` for historic, - backward compatible reasons - - move unpausing of client writes from easy.c to sendf.c, so that - code is in one place and can forward flags correctly + ``` + ./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' + [-Wunused-function] + static int is_valid_auth_separator(char ch) + ^ + 5 warnings generated. + ``` - Closes #11885 + Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 -Patrick Monnerat (21 Sep 2023) + Closes #12227 -- tftpd: always use curl's own tftp.h +- build: require Windows XP or newer - Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects - and reports a stringop-overread warning: + After this patch we assume availability of `getaddrinfo` and + `freeaddrinfo`, first introduced in Windows XP. Meaning curl + now requires building for Windows XP as a minimum. - tftpd.c: In function ‘write_behind.isra’: - tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f - rom a region of size 0 [-Wstringop-overread] - 485 | return write(test->ofile, writebuf, count); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - In file included from tftpd.c:71: - /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 - 58 | char tu_data[0]; /* data or error stri - ng */ - | ^~~~~~~ + TODO: assume these also in autotools. - This occurs because writebuf points to this field and the latter - cannot be considered as being of dynamic length because it is not - the last field in the structure. Thus it is bound to its declared - size. + Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 + Closes #12225 - This commit always uses curl's own version of tftp.h where the - target field is last in its structure, effectively avoiding the - warning. +- appveyor: bump one job to OpenSSL 3.1 (was 1.1.1) - As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for - arpa/tftp.h are removed. + Use 3.1 with the modern runner image. - Closes #11897 + We still use 1.1.1 in 8 jobs. -Dan Fandrich (20 Sep 2023) + 1.1.1 is EOL since 2023-09-11: + https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/ -- test1474: make precheck more robust on non-Solaris systems + Also: + - add missing SSL-backend to job descriptions. + - tidy up CPU in job descriptions. - If uname -r returns something odd, perl could return an error code and - the test would be erroneously skipped. The qx// syntax avoid this. + Closes #12226 - Followup to 08f9b2148 +Daniel Stenberg (30 Oct 2023) -- github/labeler: switch to the 5 beta version +- RELEASE-NOTES: synced - This version adds an important feature that will allow more PRs to be - labelled. Rather than being limited to labeling PRs with files that - match a single glob, it can now label them if multiple changed files - match any one of a number of globs. +- GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions -Daniel Stenberg (20 Sep 2023) + ngtcp2 1.0.1 + nghttp3 1.0.0 + nghttp2 1.58.0 + quictls 3.1.4+quic -- lib: enable hmac for digest as well + also sync HTTP3.md with these changes - Previously a build that disabled NTLM and aws-sigv4 would fail to build - since the hmac was disabled, but it is also needed for digest auth. + Closes #12132 - Follow-up to e92edfbef64448ef +Kareem (29 Oct 2023) - Fixes #11890 - Reported-by: Aleksander Mazur - Closes #11896 +- wolfssl: add default case for wolfssl_connect_step1 switch -- idn: if idn2_check_version returns NULL, return error + Closes #12218 - ... this avoids a NULL dereference for this unusual case. +Jay Satiro (29 Oct 2023) - Reported-by: s0urc3_ on hackerone - Closes #11898 +- curl_setup: disallow Windows IPv6 builds missing getaddrinfo -- http: fix CURL_DISABLE_BEARER_AUTH breakage + - On Windows if IPv6 is enabled but getaddrinfo is missing then #error + the build. - When bearer auth was disabled, the if/else logic got wrong and caused - problems. + curl can be built with IPv6 support (ENABLE_IPV6) but without the + ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On + Windows this is highly unlikely and should be considered a bad build + configuration. - Follow-up to e92edfbef64448ef461 - Fixes #11892 - Reported-by: Aleksander Mazur - Closes #11895 + Such a bad configuration has already given us a bug that was hard to + diagnose. See #12134 and #12136 for discussion. -Michael Osipov (20 Sep 2023) + Ref: https://github.com/curl/curl/issues/12134 + Ref: https://github.com/curl/curl/pull/12136 -- wolfssl: allow capath with CURLOPT_CAINFO_BLOB + Closes https://github.com/curl/curl/pull/12221 - Remain consistent with OpenSSL. While CAfile is nulled as documented - with CURLOPT_CAINFO_BLOB, CApath remains intact. +Nico Rieck (29 Oct 2023) - Closes #11886 +- openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs -- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c + - If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA + "CA" store after importing from root CA "ROOT" store. - Closes #11886 + This change allows curl to work in situations where a server does not + send all intermediate certs and they are present in the "CA" store (the + store with intermediate CAs). This is already allowed by the Schannel + backend. -Dan Fandrich (19 Sep 2023) + Also this change makes partial chain verification possible for those + certs since we allow partial chain verification by default for OpenSSL + (unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel + backend. -- test1474: disable test on NetBSD, OpenBSD and Solaris 10 + Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs. - These kernels only send a fraction of the requested amount of the first - large block, invalidating the assumptions of the test and causing it to - fail. + Fixes https://github.com/curl/curl/issues/12155 + Closes https://github.com/curl/curl/pull/12185 - Assisted-by: Christian Weisgerber - Ref: https://curl.se/mail/lib-2023-09/0021.html - Closes #11888 +Viktor Szakats (28 Oct 2023) -Ryan Schmidt (20 Sep 2023) +- Makefile.mk: fix `-rtmp` option for non-Windows [ci skip] -- cmake, configure: also link with CoreServices +Daniel Stenberg (28 Oct 2023) - When linking with CoreFoundation, also link with CoreServices which is - apparently required to avoid an NSInvalidArgumentException in software - linking with libcurl on macOS Sonoma 14 and later. +- asyn-ares: handle no connection in the addrinfo callback - Fixes #11893 - Closes #11894 + To avoid crashing. -Marc Hoersken (19 Sep 2023) + Follow-up from 56a4db2 + Closes #12219 -- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket +Jay Satiro (28 Oct 2023) - These dependencies are now already included in the Docker image. +- hostip6: fix DEBUG_ADDRINFO builds - Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 - b41d15606e97f38cf312c1ce56 + - Removed unused and incorrect parameter from dump_addrinfo(). - Closes #11889 + Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442 + Reported-by: Gisle Vanem -Daniel Stenberg (19 Sep 2023) + Closes https://github.com/curl/curl/pull/12212 -- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files +Viktor Szakats (28 Oct 2023) - Ref: #11883 - Reported-by: Michael Osipov - Closes #11884 +- Makefile.mk: restore `_mingw.h` for default `_WIN32_WINNT` -- RELEASE-NOTES: synced + In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support. + Turns out `_mingw.h` had the side-effect of setting a default + `_WIN32_WINNT` value expected by `lib/config-win32.h` to enable + `getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused + disabling support for this unless specifying the value manually. -- test3103: CURLOPT_COOKIELIST test + Restore this header and update its comment to tell why we continue + to need it. -- cookie: set ->running in cookie_init even if data is NULL + This triggered a regression in official Windows curl builds starting + with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.) - This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) + Regression from 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 - Test 3103 verifies. + Reported-by: zhengqwe on github + Helped-by: Nico Rieck + Fixes #12134 + Fixes #12136 + Closes #12217 - Fixes #11875 - Reported-by: wangp on github - Closes #11876 +- hostip: silence compiler warning `-Wparentheses-equality` -- test498: total header size for all redirects is larger than accepted + Seen with LLVM 17. -- http: use per-request counter to check too large headers + ``` + hostip.c:1336:22: warning: equality comparison with extraneous parentheses [- + Wparentheses-equality] + 1336 | (a->ai_family == PF_INET)) { + | ~~~~~~~~~~~~~^~~~~~~~~~ + hostip.c:1336:22: note: remove extraneous parentheses around the comparison t + o silence this warning + 1336 | (a->ai_family == PF_INET)) { + | ~ ^ ~ + hostip.c:1336:22: note: use '=' to turn this equality comparison into an assi + gnment + 1336 | (a->ai_family == PF_INET)) { + | ^~ + | = + 1 warning generated. + ``` - Not the counter that accumulates all headers over all redirects. + Follow-up to b651aba0962bb31353f55de4dc35f745952a1b10 #12145 - Follow-up to 3ee79c1674fd6 + Reviewed-by: Daniel Stenberg + Closes #12215 - Do a second check for 20 times the limit for the accumulated size for - all headers. +Stefan Eissing (27 Oct 2023) - Fixes #11871 - Reported-by: Joshix-1 on github - Closes #11872 +- doh: use PIPEWAIT when HTTP/2 is attempted -Jay Satiro (18 Sep 2023) + Closes #12214 -- THANKS: add Eric Murphy +Daniel Stenberg (27 Oct 2023) - He reported #11850 (quiche build error) but I forgot to add a - 'reported-by' entry in the fix 267e14f1. +- setopt: remove outdated cookie comment -Daniel Stenberg (18 Sep 2023) + Closes #12206 -- h2-proxy: remove left-over mistake in drain_tunnel() +Stefan Eissing (27 Oct 2023) - Left-over from 331b89a319 +- cfilter: provide call to tell connection to forget a socket - Reported-by: 南宫雪珊 + - fixed libssh.c workaround for a socket being closed by + the library + - eliminate the terrible hack in cf-socket.c to guess when + this happened and try not closing the socket again. + - fixes race in eyeballing when socket could have failed to + be closed for a discarded connect attempt - Closes https://github.com/curl/curl/pull/11877 + Closes #12207 -vvb2060 (18 Sep 2023) +- url: protocol handler lookup tidy-up -- lib: failf/infof compiler warnings + - rename lookup to what it does + - use ARRAYSIZE instead of NULL check for end + - offer alternate lookup for 0-terminated strings - Closes #11874 + Closes #12216 -Daniel Stenberg (17 Sep 2023) +Viktor Szakats (27 Oct 2023) -- rand: fix 'alnum': array is too small to include a terminating null character +- build: variadic macro tidy-ups - It was that small on purpose, but this change now adds the null byte to - avoid the error. + - delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks. + (both autotools and CMake.) + - delete duplicate `NULL` check in `Curl_trc_cf_infof()`. + - fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds. + ``` + ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parame + ter] + static void nosigpipe(struct Curl_easy *data, + ^ + ``` + - fix `#ifdef` comments in `lib/curl_trc.{c,h}`. + - fix indentation in some `infof()` calls. - Follow-up to 3aa3cc9b052353b1 + Follow-up to dac293cfb7026b1ca4175d88b80f1432d3d3c684 #12167 - Reported-by: Dan Fandrich - Ref: #11838 - Closes #11870 + Cherry-picked from #12105 + Closes #12210 -Mathias Fuchs (16 Sep 2023) +- cmake: speed up threads setup for Windows -- cmake: fix the help text to the static build option in CMakeLists.txt + Win32 threads are always available. We enabled them unconditionally + (with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection + logic has this condition hard-coded for Windows as well (since at least + 2007). - Closes #11843 + Instead of doing all the work of detecting pthread combinations on + Windows, then discarding those results, skip these efforts and assume + built-in thread support when building for Windows. -John Haugabook (16 Sep 2023) + This saves 1-3 slow CMake configuration steps. -- MANUAL.md: change domain to example.com + Reviewed-by: Daniel Stenberg + Closes #12202 - Closes #11866 +- cmake: speed up zstd detection -Daniel Stenberg (16 Sep 2023) + Before this patch we detected the presence of a specific zstd API to + see if we can use the library. zstd published that API in its first + stable release: v1.0.0 (2016-08-31). -- doh: inherit DEBUGFUNCTION/DATA + Replace that method by detecting the zstd library version instead and + accepting if it's v1.0.0 or newer. Also display this detected version + and display a warning if the zstd found is unfit for curl. - When creating new transfers for doing DoH, they now inherit the debug - settings from the initiating transfer, so that the application can - redirect and handle the verbose output correctly even for the DoH - transfers. + We use the same version detection method as zstd itself, via its public + C header. - Reported-by: calvin2021y on github - Fixes #11864 - Closes #11869 + This deviates from autotools which keeps using the slow method of + looking for the API by building a test program. The outcome is the same + as long as zstd keeps offering this API. -Dan Fandrich (16 Sep 2023) + Ref: https://github.com/facebook/zstd/commit/5a0c8e24395079f8e8cdc90aa1659cd5 + ab1b7427 (2016-08-12, committed) + Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first + released) + Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0 -- http_aws_sigv4: fix sorting with empty parts + Reviewed-by: Daniel Stenberg + Closes #12200 - When comparing with an empty part, the non-empty one is always - considered greater-than. Previously, the two would be considered equal - which would randomly place empty parts amongst non-empty ones. This - showed as a test 439 failure on Solaris as it uses a different - implementation of qsort() that compares parts differently. +Daniel Stenberg (26 Oct 2023) - Fixes #11855 - Closes #11868 +- openssl: fix infof() to avoid compiler warning for %s with null -- CI: ignore the "flaky" and "timing-dependent" test results + vtls/openssl.c: In function ‘ossl_connect_step2’: + ../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror + =format-overflow=] + 120 | Curl_infof(data, __VA_ARGS__); } while(0) + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’ + 4008 | infof(data, "SSL connection using %s / %s / %s / %s", + | ^~~~~ + vtls/openssl.c:4008:49: note: format string is defined here + 4008 | infof(data, "SSL connection using %s / %s / %s / %s", + | ^~ - CI builds will now run these tests, but will ignore the results if they - fail. The relevant tests are ones that are sensitive to timing or - have edge conditions that make them more likely to fail on CI servers, - which are often heavily overloaded and slow. + Follow-up to b6e6d4ff8f253c8b8055bab + Closes #12196 - This change only adds two additional tests to be ignored, since the - others already had the flaky keyword. +Stefan Eissing (26 Oct 2023) - Closes #11865 +- lib: apache style infof and trace macros/functions -- runtests: eliminate a warning on old perl versions + - test for a simplified C99 variadic check + - args to infof() in --disable-verbose are no longer disregarded but + must compile. - The warning "Use of implicit split to @_ is deprecated" showed between - perl versions about 5.8 through 5.11. + Closes #12167 + Fixes #12083 + Fixes #11880 + Fixes #11891 -- tests: log the test result code after each libtest +Daniel Stenberg (26 Oct 2023) - This makes it easier to determine the test status. Also, capitalize - FAILURE and ABORT messages in log lines to make them easier to spot. +- RELEASE-NOTES: synced -Harry Sintonen (16 Sep 2023) +Stefan Eissing (26 Oct 2023) -- misc: better random strings +- urldata: move async resolver state from easy handle to connectdata - Generate alphanumerical random strings. + - resolving is done for a connection, not for every transfer + - save create/dup/free of a cares channel for each transfer + - check values of setopt calls against a local channel if no + connection has been attached yet, when needed. - Prior this change curl used to create random hex strings. This was - mostly okay, but having alphanumerical random strings is better: The - strings have more entropy in the same space. + Closes #12198 - The MIME multipart boundary used to be mere 64-bits of randomness due - to being 16 hex chars. With these changes the boundary is 22 - alphanumerical chars, or little over 130 bits of randomness. +Daniel Stenberg (26 Oct 2023) - Closes #11838 +- CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERRO + R -Daniel Stenberg (15 Sep 2023) + It returns CURLE_WRITE_ERROR. It was not previously stated clearly. -- cookie: reduce variable scope, add const + Reported-by: enWILLYado on github + Fixes #12201 + Closes #12203 -- cookie: do not store the expire or max-age strings +Viktor Szakats (25 Oct 2023) - Convert it to an expire time at once and save memory. +- autotools: update references to deleted `crypt-auth` option - Closes #11862 + Delete leftovers of the `crypt-auth` `./configure` option and + add the new ones that replaced them. -- cookie: remove unnecessary struct fields + Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 - Plus: reduce the hash table size from 256 to 63. It seems unlikely to - make much of a speed difference for most use cases but saves 1.5KB of - data per instance. + Reviewed-by: Daniel Stenberg + Closes #12194 - Closes #11862 +Stefan Eissing (25 Oct 2023) -- RELEASE-NOTES: synced +- lib: introduce struct easy_poll_set for poll information - Bumped to 8.4.0, the next presumed version + Connection filter had a `get_select_socks()` method, inspired by the + various `getsocks` functions involved during the lifetime of a + transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), + return sockets to monitor and flag if this shall be done for POLLIN + and/or POLLOUT. -Dan Fandrich (14 Sep 2023) + Due to this design, sockets and flags could only be added, not + removed. This led to problems in filters like HTTP/2 where flow control + prohibits the sending of data until the peer increases the flow + window. The general transfer loop wants to write, adds POLLOUT, the + socket is writeable but no data can be written. -- test2600: remove special case handling for USE_ALARM_TIMEOUT + This leads to cpu busy loops. To prevent that, HTTP/2 did set the + `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes + further attempts. This works if only one such filter is involved. If a + HTTP/2 transfer goes through a HTTP/2 proxy, two filters are + setting/clearing this flag and may step on each other's toes. - This was originally added to handle platforms that supported only 1 - second granularity in connect timeouts, but after some recent changes - the test currently permafails on several Windows platforms. + Connection filters `get_select_socks()` is replaced by + `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps + up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` + flags. This struct is initialized in `multi_getsock()` by calling the + various `getsocks()` implementations based on transfer state, as before. - The need for this special-case was removed in commit 8627416, which - increased the connect timeout in all cases to well above 1 second. + After protocol handlers/transfer loop have set the sockets and flags + they want, the `easy_pollset` is *always* passed to the filters. Filters + "higher" in the chain are called first, starting at the first + not-yet-connection one. Each filter may add sockets and/or change + flags. When all flags are removed, the socket itself is removed from the + pollset. - Fixes #11767 - Closes #11849 + Example: -Daniel Stenberg (14 Sep 2023) + * transfer wants to send, adds POLLOUT + * http/2 filter has a flow control block, removes POLLOUT and adds + POLLIN (it is waiting on a WINDOW_UPDATE from the server) + * TLS filter is connected and changes nothing + * h2-proxy filter also has a flow control block on its tunnel stream, + removes POLLOUT and adds POLLIN also. + * socket filter is connected and changes nothing + * The resulting pollset is then mixed together with all other transfers + and their pollsets, just as before. -- SECURITY-PROCESS.md. call it vulnerability disclosure policy + Use of `SEND_HOLD` is no longer necessary in the filters. - SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md + All filters are adapted for the changed method. The handling in + `multi.c` has been adjusted, but its state handling the the protocol + handlers' `getsocks` method are untouched. - This a name commonly used for a document like this. This name helps - users find it. + The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS + filters needed to be adjusted for the connecting handshake read/write + handling. - Closes #11852 + No noticeable difference in performance was detected in local scorecard + runs. -Junho Choi (14 Sep 2023) + Closes #11833 -- quiche: fix build error with --with-ca-fallback +Daniel Stenberg (25 Oct 2023) - - Fix build error when curl is built with --with-quiche - and --with-ca-fallback. +- tests/README: SOCKS tests are not using OpenSSH, it has its own server - - Add --with-ca-fallback to the quiche CI job. + Follow-up to 04fd67555cc - Fixes https://github.com/curl/curl/issues/11850 - Closes https://github.com/curl/curl/pull/11847 + Closes #12195 -Jay Satiro (14 Sep 2023) +Jacob Hoffman-Andrews (25 Oct 2023) -- escape: replace Curl_isunreserved with ISUNRESERVED +- tets: make test documentation more user-friendly - - Use the ALLCAPS version of the macro so that it is clear a macro is - being called that evaluates the variable multiple times. + Put the instructions to run tests right at the top of tests/README.md. - - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates - a variable multiple times. + Give instructions to read the runtests.1 man page for information + about flags. Delete redundant copy of the flags documentation in the + README. - This is a follow-up to 291d225a which changed Curl_isunreserved into an - alias macro for ISUNRESERVED. The problem is the former is not easily - identified as a macro by the caller, which could lead to a bug. + Add a mention in README.md of the important parallelism flag, to make + test runs go much faster. - For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but - Curl_isunreserved(*foo++) is not even though they both are the same. + Move documentation of output line format into the runtests.1 man page, + and update it with missing flags. - Closes https://github.com/curl/curl/pull/11846 + Fix the order of two flags in the man page. -Dan Fandrich (13 Sep 2023) + Closes #12193 -- tests: increase the default server logs lock timeout +Viktor Szakats (24 Oct 2023) - This timeout is used to wait for the server to finish writing its logs - before checking them against the expected values. An overloaded machine - could take more than the two seconds previously allocated, so increase - the timeout to 5 seconds. +- cmake: pre-fill rest of detection values for Windows - Ref: #11328 - Closes #11834 + The goal of this patch is to avoid unnecessary feature detection work + when doing Windows builds with CMake. Do this by pre-filling well-known + detection results for Windows and specifically for mingw-w64 and MSVC + compilers. Also limit feature checks to platforms where the results are + actually used. Drop a few redundant ones. And some tidying up. -- tests: increase TEST_HANG_TIMEOUT in two tests + - pre-fill remaining detection values in Windows CMake builds. - These tests had a 5 second timeout compared to 60 seconds for all other - tests. Make these consistent with the others for more reliability on - heavily-loaded machines. + Based on actual detection results observed in CI runs, preceding + similar work over libssh2 and matching up values with + `lib/config-win32.h`. - Ref: #11328 + This brings down CMake configuration time from 58 to 14 seconds on the + same local machine. -- test1056: disable on Windows + On AppVeyor CI this translates to: + - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job): + https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecr + jpy7necb#L296 + https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2 + fe249uo8#L186 + - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job): + https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5iv + lcs7ub29?fullLog=true#L290 + https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjs + yc9kl13a?fullLog=true#L194 - This test relies on the IPv6 scope field being ignored when connecting to - ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit - dodgy, but it works on all our test platforms except Windows. This - test was disabled manually on all Windows CI builds already, so instead - add an incompatible feature and precheck so it's skipped on Windows - everywhere automatically. + The formula is about 1-3 seconds delay for each detection. Almost all + of these trigger a full compile-link cycle behind the scenes, slow + even today, both cross and native, mingw-w64 and apparently MSVC too. + Enabling .map files or other custom build features slows it down + further. (Similar is expected for autotools configure.) -- test587: add a slight delay after test + - stop detecting `idn2.h` if idn2 was deselected. + autotools does this. - This test is designed to connect to the server, then immediately send a - few bytes and disconnect. In some situations, such as on a loaded - server, this doesn't give the server enough time to write its lock file - before its existence is checked. The test harness then fails to find the - server's input log file (because it hasn't been written yet) and fails - the test. By adding a short delay after the test, the HTTP server has - enough time to write its lock file which gives itself more time to write - its remaining files. + - stop detecting `idn2.h` if idn2 was not found. + This deviates from autotools. Source code requires both header and + lib, so this is still correct, but faster. - Ref: #11328 + - limit `ADDRESS_FAMILY` detection to Windows. -- tests: stop overriding the lock timeout + - normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format. - These tests reduce the server lock wait timeout which can increase - flakiness on loaded machines. Since this is merely an optimization, - eliminate them in favour of reliability. + - pre-fill `HAVE_WIN32_WINNT`-dependent detection results. + Saving 4 (slow) feature-detections in most builds: `getaddrinfo`, + `freeaddrinfo`, `inet_ntop`, `inet_pton` - Ref: #11328 + - fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`, + `HAVE_GETTIMEOFDAY` for mingw-w64. + Luckily this do not change build results, as `WIN32` took + priority over `HAVE_GETTIMEOFDAY` with the current source + code. -- tests: add some --expect100-timeout to reduce timing dependencies + - limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and + `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows. + We're not using these in the source code for Windows. - These tests can fail when the test machine is so slow that the test HTTP - server didn't get a chance to complete before the client's one second - 100-continue timeout triggered. Increase that 1 second to 999 seconds so - this situation doesn't happen. + - reduce compiler warning noise in CMake internal logs: + - fix to include `winsock2.h` before `windows.h`. + Apply it to autotools test snippets too. + - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above. + - cleanup `CMake/CurlTests.c` to emit less warnings. - Ref: #11328 + - delete redundant `HAVE_MACRO_SIGSETJMP` feature check. + It was the same check as `HAVE_SIGSETJMP`. -- test661: return from test early in case of curl error + - delete 'experimental' marking from `CURL_USE_OPENSSL`. -- tests: add the timing-dependent keyword on several tests + - show CMake version via `CMakeLists.txt`. + Credit to the `zlib-ng` project for the idea: + https://github.com/zlib-ng/zlib-ng/blob/61e181c8ae93dbf56040336179c9954078b + d1399/CMakeLists.txt#L7 - These are ones likely to fail on heavily-loaded machines that alter the - normal test timing. Most of these tests already had the flaky keyword - since this condition makes them more likely to fail on CI. + - make `CMake/CurlTests.c` pass `checksrc`. -- test1592: greatly increase the maximum test timeout + - `CMake/WindowsCache.cmake` tidy-ups. - It was too short to be reliable on heavily loaded CI machines, and - as a fail-safe only, it didn't need to be short. + - replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`. - Ref: #11328 + Closes #12044 -- test: minor test cleanups +Jay Satiro (24 Oct 2023) - Remove an obsolete block of code in tests 2032 & 576. - Add a comment in test 1474. +- page-footer: clarify exit code 25 -- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts + - Clarify that curl tool exit code 25 means an upload failed to start. - This gives more of a margin for error when running on overloaded CI - servers. + Exit code 25 is equivalent to CURLE_UPLOAD_FAILED (25). Prior to this + change the documentation only mentioned the case of FTP STOR failing. - Ref: #11328 + Reported-by: Emanuele Torre -- tests: improve SLOWDOWN test reliability by reducing sent data + Ref: https://github.com/curl/curl/blob/curl-8_4_0/docs/libcurl/libcurl-errors + .3#L113-L115 - These tests are run in SLOWDOWN mode which adds a 10 msec delay after - each character output, which means it takes at least 1.6 seconds (and - 320 kernel calls) just to get through the long welcome banner. On an - overloaded system, this can end up taking much more than 1.6 seconds, - and even more than the 7 or 16 second curl timeout that the tests rely - on, causing them to fail. Reducing the size of the welcome banner drops - the total number of characters sent before the transfer starts by more - than half, which reduces the opportunity for test-breaking slowdowns by - the same amount. + Fixes https://github.com/curl/curl/issues/12189 + Closes https://github.com/curl/curl/pull/12190 - Ref: #11328 +Daniel Stenberg (24 Oct 2023) -- test650: fix an end tag typo +- scripts/cijobs.pl: adjust for appveyor -Jay Satiro (13 Sep 2023) + Follow-up to a1d73a6bb -- tool_cb_wrt: fix debug assertion +Alex Bozarth (24 Oct 2023) - - Fix off-by-one out-of-bounds array index in Windows debug assertion. +- OpenSSL: Include SIG and KEM algorithms in verbose - Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 - Reported-by: Gisle Vanem + Currently the verbose output does not include which algorithms are used + for the signature and key exchange when using OpenSSL. Including the + algorithms used will enable better debugging when working on using new + algorithm implementations. Know what algorithms are used has become more + important with the fast growing research into new quantum-safe + algorithms. -Daniel Stenberg (13 Sep 2023) + This implementation includes a build time check for the OpenSSL version + to use a new function that will be included in OpenSSL 3.2 that was + introduced in openssl/openssl@6866824 -- ctype: add ISUNRESERVED() + Based-on-patch-by: Martin Schmatz + Closes #12030 - ... and make Curl_isunreserved() use that macro instead of providing a - separate funtion for the purpose. +Daniel Stenberg (23 Oct 2023) - Closes #11840 +- http2: provide an error callback and failf the message -Version 8.3.0 (13 Sep 2023) + Getting nghttp2's error message helps users understand what's going + on. For example when the connection is brought down due a forbidden + header is used - as that header is then not displayed by curl itself. -Daniel Stenberg (13 Sep 2023) + Example: -- RELEASE-NOTES: syn ced + curl: (92) Invalid HTTP header field was received: frame type: 1, + stream: 1, name: [upgrade], value: [h2,h2c] - curl 8.3.0 release + Ref: #12172 + Closes #12179 -- THANKS: contributors from 8.3.0 +Turiiya (23 Oct 2023) -Thorsten Klein (12 Sep 2023) +- BINDINGS: add V binding -- cmake: set SIZEOF_LONG_LONG in curl_config.h + Closes #12182 - in order to support 32bit builds regarding wolfssl CTC_SETTINGS +Daniel Stenberg (22 Oct 2023) - Closes #11839 +- configure: check for the fseeko declaration too -Jay Satiro (12 Sep 2023) + ... and make the code require both symbol and declaration. -- curl_ngtcp2: fix error message + This is because for Android, the symbol is always present in the lib at + build-time even when not actually available in run-time. -- http_aws_sigv4: handle no-value user header entries + Assisted-by: Viktor Szakats + Reported-by: 12932 on github + Fixes #12086 + Closes #12158 - - Handle user headers in format 'name:' and 'name;' with no value. +Viktor Szakats (22 Oct 2023) - The former is used when the user wants to remove an internal libcurl - header and the latter is used when the user actually wants to send a - no-value header in the format 'name:' (note the semi-colon is converted - by libcurl to a colon). +- cmake: fix OpenSSL quic detection in quiche builds - Prior to this change the AWS header import code did not special case - either of those and the generated AWS SignedHeaders would be incorrect. + An orphan call to `CheckQuicSupportInOpenSSL()` remained after a recent + update when checking QUIC for quiche. Move back QUIC detection to + a function and fixup callers to use that. Also make sure that quiche + gets QUIC from BoringSSL, because it doesn't support other forks at this + time. - Reported-by: apparentorder@users.noreply.github.com + Regression from dee310d54261f9a8416e87d50bccfe2cbe404949 #11555 - Ref: https://curl.se/docs/manpage.html#-H + Reported-by: Casey Bodley + Fixes #12160 + Closes #12162 - Fixes https://github.com/curl/curl/issues/11664 - Closes https://github.com/curl/curl/pull/11668 +Daniel Stenberg (22 Oct 2023) -Dan Fandrich (11 Sep 2023) +- RELEASE-NOTES: synced -- CI: run pytest with the -v option + bump to 8.5.0 for pending release - This lists of the test cases being run so it can be tracked over time. +Dan Fandrich (21 Oct 2023) - Closes #11824 +- test3103: add missing quotes around a test tag attribute -Daniel Stenberg (11 Sep 2023) +Loïc Yhuel (21 Oct 2023) -- HTTP3: the msquic backend is not functional +- tool: fix --capath when proxy support is disabled - I ask that we do not submit bugs for this backend just yet as we know it - does not fully work. + After 95e8515ca0, --capath always sets CURLOPT_PROXY_CAPATH, which fails + with CURLE_UNKNOWN_OPTION when proxy support is disabled. - Closes #11831 - Closes #11819 + Closes #12089 -- aws_sigv4: the query canon code miscounted URL encoded input +Daniel Stenberg (21 Oct 2023) - Added some extra ampersands to test 439 to verify "blank" query parts +- openldap: move the alloc of ldapconninfo to *connect() - Follow-up to fc76a24c53b08cdf + Fixes a minor memory leak on LDAP connection reuse. - Closes #11829 + Doing the allocation already in *setup_connection() is wrong since that + connect struct might get discarded early when an existing connection is + reused instead. -vvb2060 (11 Sep 2023) + Closes #12166 -- quic: don't set SNI if hostname is an IP address +- openldap: set the callback argument in oldap_do - We already do this for TLS connections. + ... to make sure it has the current 'data' pointer and not a stale old + one. - RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in - "HostName". + Reported-by: Dan Fandrich + Closes #12166 - Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 +- gnutls: support CURLSSLOPT_NATIVE_CA - Fixes https://github.com/curl/curl/issues/11827 - Closes https://github.com/curl/curl/pull/11828 + Remove the CURL_CA_FALLBACK logic. That build option was added to allow + primarily OpenSSL to use the default paths for loading the CA certs. For + GnuTLS it was instead made to load the "system certs", which is + different and not desirable. -Daniel Stenberg (10 Sep 2023) + The native CA store loading is now asked for with this option. -- RELEASE-NOTES: synced + Follow-up to 7b55279d1d856 -Benoit Pierre (10 Sep 2023) + Co-authored-by: Jay Satiro -- configure: fix `HAVE_TIME_T_UNSIGNED` check + Closes #12137 - The syntax was incorrect (need a proper main body), and the test - condition was wrong (resulting in a signed `time_t` detected as - unsigned). +Stefan Eissing (21 Oct 2023) - Closes #11825 +- RTSP: improved RTP parser -Daniel Stenberg (9 Sep 2023) + - fix HTTP header parsing to report incomplete + lines it buffers as consumed! + - re-implement the RTP parser for interleave RTP + messages for robustness. It is now keeping its + state at the connection + - RTSP protocol handler "readwrite" implementation + now tracks if the response is before/in/after + header parsing or "in" a bod by calling + "Curl_http_readwrite_headers()" itself. This + allows it to know when non-RTP bytes are "junk" + or HEADER or BODY. + - tested with #12035 and various small receive + sizes where current master fails -- THANKS-filter: pszlazak on github + Closes #12052 -pszlazak (9 Sep 2023) +- http2: header conversion tightening -- include.d: explain headers not printed with --fail before 7.75.0 + - fold the code to convert dynhds to the nghttp2 structs + into a dynhds internal method + - saves code duplication + - pacifies compiler analyzers - Prior to 7.75.0 response headers were not printed if -f/--fail was used - and an error was reported by server. This was fixed in ab525c0 - (precedes 7.75.0). + Closes #12097 - Closes #11822 +Daniel Stenberg (21 Oct 2023) -Daniel Stenberg (8 Sep 2023) +- curl_ntlm_wb: fix elif typo -- http_aws_sigv4: skip the op if the query pair is zero bytes + Reported-by: Manfred Schwarb + Follow-up to d4314cdf65ae + Bug: https://github.com/curl/curl/commit/d4314cdf65aee295db627016934bd9eb621a + b077#r130551295 - Follow-up to fc76a24c53b08cdf +Dan Fandrich (20 Oct 2023) - Spotted by OSS-Fuzz +- test1683: remove commented-out check alternatives - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 - Closes #11823 + Python precheck/postcheck alternatives were included but commented out. + Since these are not used and perl is guaranteed to be available to run + the perl versions anyway, the Python ones are removed. -- cmdline-docs: use present tense, not future +Daniel Stenberg (20 Oct 2023) - + some smaller cleanups +- hostip: show the list of IPs when resolving is done - Closes #11821 + Getting 'curl.se' today then gets this verbose output which might help + debugging connectivity related matters. -- cmdline-docs: make sure to phrase it as "added in ...." + * Host curl.se:80 was resolved. + * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, + 2a04:4e42:600::347, 2a04:4e42:800::347, 2a04:4e42:a00::347, + 2a04:4e42:c00::347, 2a04:4e42:e00::347 + * IPv4: 151.101.193.91, 151.101.1.91, 151.101.65.91, 151.101.129.91 - References to things that were added or changed in a specific version - should be specified as "(added in [version]) for two reasons: + Co-authored-by: Jay Satiro + Closes #12145 - 1 - consistency +rilysh (20 Oct 2023) - 2 - to allow gen.pl to strip them out if deemed referring to too old - versions +- docs: fix function typo in curl_easy_option_next.3 - Closes #11821 + Closes #12170 -Jay Satiro (8 Sep 2023) +Daniel Stenberg (20 Oct 2023) -- docs: mark --ssl-revoke-best-effort as Schannel specific +- vssh: remove the #ifdef for Curl_ssh_init, use empty macro - Closes https://github.com/curl/curl/pull/11760 + In the same style as other init calls -Nathan Moinvaziri (8 Sep 2023) +- easy: remove duplicate wolfSSH init call -- schannel: fix ordering of cert chain info + It is already done in Curl_ssh_init() where it belongs. - - Use CERT_CONTEXT's pbCertEncoded to determine chain order. + Closes #12168 - CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains - end-entity/server certificate in pbCertEncoded. We can use this pointer - to determine the order of certificates when enumerating hCertStore using - CertEnumCertificatesInStore. +- socks: make SOCKS5 use the CURLOPT_IPRESOLVE choice - This change is to help ensure that the ordering of the certificate chain - requested by the user via CURLINFO_CERTINFO has the same ordering on all - versions of Windows. + Fixes #11949 + Reported-by: Ammar Faizi + Closes #12163 - Prior to this change Schannel certificate order was reversed in 8986df80 - but that was later reverted in f540a39b when it was discovered that - Windows 11 22H2 does the reversal on its own. +- urldata: move the 'internal' boolean to the state struct - Ref: https://github.com/curl/curl/issues/9706 + ... where all the other state bits for the easy handles live. - Closes https://github.com/curl/curl/pull/11632 + Closes #12165 -Chris Talbot (8 Sep 2023) +- url: don't touch the multi handle when closing internal handles -- digest: Use hostname to generate spn instead of realm + Reported-by: Maksymilian Arciemowicz + Closes #12165 - In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 +Faraz Fallahi (19 Oct 2023) - digest-uri-value should be serv-type "/" host , where host is: +- getenv: PlayStation doesn't have getenv() - The DNS host name or IP address for the service requested. The - DNS host name must be the fully-qualified canonical name of the - host. The DNS host name is the preferred form; see notes on server - processing of the digest-uri. + Closes #12140 - Realm may not be the host, so we must specify the host explicitly. +Daniel Stenberg (19 Oct 2023) - Note this change only affects the non-SSPI digest code. The digest code - used by SSPI builds already uses the hostname to generate the spn. +- transfer: only reset the FTP wildcard engine in CLEAR state - Ref: https://github.com/curl/curl/issues/11369 + To avoid the state machine to start over and redownload all the files + *again*. - Closes https://github.com/curl/curl/pull/11395 + Reported-by: lkordos on github + Regression from 843b3baa3e3cb228 (shipped in 8.1.0) + Bisect-by: Dan Fandrich + Fixes #11775 + Closes #12156 -Daniel Stenberg (7 Sep 2023) +Stefan Eissing (19 Oct 2023) -- docs: remove use of the word 'very' +- GHA: move mod_h2 version in CI to v2.0.25 - It is mostly superfluous. proselint would complain. + Closes #12157 - Closes #11818 +Daniel Stenberg (19 Oct 2023) -- curl_multi_remove_handle.3: clarify what happens with connection +- ntlm_wb: use pipe instead of socketpair when possible - Closes #11817 + Closes #12149 - RELEASE-NOTES: synced -- test439: verify query canonization for aws-sigv4 +- asyn-thread: use pipe instead of socketpair for IPC when available -- tool_operate: make aws-sigv4 not require TLS to be used + If pipe() is present. Less overhead. - Maybe not used too often, but we want it for testing and it should work. + Helped-by: Viktor Szakats + Closes #12146 -- http_aws_sigv4: canonicalize the query +Dan Fandrich (17 Oct 2023) - Percent encoding needs to be done using uppercase, and most - non-alphanumerical must be percent-encoded. +- tests: Fix Windows test helper tool search & use it for handle64 - Fixes #11794 - Reported-by: John Walker - Closes #11806 + The checkcmd() and checktestcmd() functions would not have worked on + Windows due to hard-coding the UNIX PATH separator character and not + adding .exe file extension. This meant that tools like stunnel, valgrind + and nghttpx would not have been found and used on Windows, and + inspection of previous test runs show none of those being found in pure + Windows CI builds. -Wyatt O'Day (7 Sep 2023) + With this fixed, they can be used to detect the handle64.exe program + before attempting to use it. When handle64.exe was called + unconditionally without it existing, it caused perl to abort the test + run with the error -- lib: add ability to disable auths individually + The running command stopped because the preference variable + "ErrorActionPreference" or common parameter is set to Stop: + sh: handle64.exe: command not found - Both with configure and cmake + Closes #12115 - Closes #11490 +Daniel Stenberg (17 Oct 2023) -Stefan Eissing (7 Sep 2023) +- multi: use pipe instead of socketpair to *wakeup() -- ngtcp2: fix handling of large requests + If pipe() is present. Less overhead. - - requests >64K are send in parts to the filter - - fix parsing of the request to assemble it correctly - from several sends - - open a QUIC stream only when the complete request has - been collected + Closes #12142 - Closes #11815 +Jay Satiro (17 Oct 2023) -- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before +- build: fix 'threadsafe' feature detection for older gcc - - we delay loading the x509 store to shorten the handshake time. - However an application callback installed via CURLOPT_SSL_CTX_FUNCTION - may need to have the store loaded and try to manipulate it. - - load the x509 store before invoking the app callback + - Add 'threadsafe' to the feature list shown during build if POSIX + threads are being used. - Fixes #11800 - Reported-by: guoxinvmware on github - Cloes #11805 + This is a follow-up to 5adb6000 which added support for building a + thread-safe libcurl with older versions of gcc where atomic is not + available but pthread is. -Daniel Stenberg (7 Sep 2023) + Reported-by: Dan Fandrich + Co-authored-by: Dan Fandrich -- krb5: fix "implicit conversion loses integer precision" warnings + Fixes https://github.com/curl/curl/issues/12125 + Closes https://github.com/curl/curl/pull/12127 - conversions to/from enum and unsigned chars +Daniel Stenberg (16 Oct 2023) - Closes #11814 +- test729: verify socks4a with excessive proxy user name length -Stefan Eissing (7 Sep 2023) +- socks: better buffer size checks for socks4a user and hostname -- pytest: improvements + Also limit the proxy user name to 255 bytes, which is the same limit as + in SOCKS5. - - set CURL_CI for pytest runs in CI environments - - exclude timing sensitive tests from CI runs - - for failed results, list only the log and stat of - the failed transfer + Reported-by: sd0 on hackerone + Closes #12139 - - fix type in http.c comment +- curl.h: on FreeBSD include sys/param.h instead of osreldate.h - Closes #11812 + Should things build on Playstation as well -- CI: move on to ngtcp2 v0.19.1 + Fixes #12107 + Reported-by: Faraz Fallahi + Closes #12123 - Closes #11809 +Marcin Rataj (16 Oct 2023) -Dan Fandrich (5 Sep 2023) +- tool_operate: fix links in ipfs errors -- CI: run Circle macOS builds on x86 for now + URL fragment links generated from headers in + https://curl.se/docs/ipfs.html are lowercase. - The ARM machines aren't ready for us and requesting them now causes - warnings e-mails to be sent to some PR pushers. + Closes #12133 - Ref: #11771 +Viktor Szakats (15 Oct 2023) -Viktor Szakats (5 Sep 2023) +- cmake: replace `check_library_exists_concat()` -- http3: adjust cast for ngtcp2 v0.19.0 + The idea of `check_library_exists_concat()` is that it detects an + optional component and adds it to the list of libs that we also use in + subsequent component checks. This caused problems when detecting + components with unnecessary dependencies that were not yet built. - ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` - an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. + CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used + for component checks, which we already use in most cases. That left 4 + uses of `check_library_exists_concat()`. Only one of these actually + needed the 'concat' feature (ldap/lber). - Fixes: - ``` - ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in - teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned - char') [-Wimplicit-int-conversion] - pi.ecn = (uint32_t)ecn; - ~ ^~~~~~~~~~~~~ - ``` + Delete this function and replace it with standard + `check_library_exists()` and manual management of our `CURL_LIBS` + list we use when linking build targets. And special logic to handle the + ldap/lber case. - Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our - docs and CI. + (We have a similar function for headers: `check_include_file_concat()`. + It works, but problematic for performance reasons and because it hides + the actual headers required in `check_symbol_exists()` calls.) - Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 - 782894a3 - Ref: https://github.com/ngtcp2/ngtcp2/pull/877 - Closes #11798 + Ref: #11537 #11558 + Fixes #11285 + Fixes #11648 + Closes #12070 -Stefan Eissing (5 Sep 2023) +LoRd_MuldeR (15 Oct 2023) -- http: fix sending of large requests +- tool_cb_wrt: fix write output for very old Windows versions - - refs #11342 where errors with git https interactions - were observed - - problem was caused by 1st sends of size larger than 64KB - which resulted in later retries of 64KB only - - limit sending of 1st block to 64KB - - adjust h2/h3 filters to cope with parsing the HTTP/1.1 - formatted request in chunks + - Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW() + function. - - introducing Curl_nwrite() as companion to Curl_write() - for the many cases where the sockindex is already known + Apparently this parameter was *not* optional on older Windows versions. - Fixes #11342 (again) - Closes #11803 + Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1. + So at some point between those two Microsoft changed the behavior. -- pytest: fix check for slow_network skips to only apply when intended + Prior to this change, on those versions if parameter is NULL then the + function call fails with error ERROR_INVALID_ACCESS. - Closes #11801 + Regression since af3f4e41. -Daniel Stenberg (5 Sep 2023) + Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299 -- curl_url_get/set.3: add missing semicolon in SYNOPSIS + Fixes https://github.com/curl/curl/issues/12131 + Closes https://github.com/curl/curl/pull/12130 -- CURLOPT_URL.3: explain curl_url_set() uses the same parser +Jay Satiro (15 Oct 2023) -- CURLOPT_URL.3: add two URL API calls in the see-also section +- tool_urlglob: fix build for old gcc versions -Dan Fandrich (4 Sep 2023) + - Don't use __builtin_mul_overflow for GCC 4 and earlier. -- CI: add a 32-bit i686 Linux build + The function was added in GCC 5. - This is done by cross-compiling under regular x86_64 Linux. Since the - kernel offers backwards compatibility, the binaries can be tested as - normal. + Ref: https://gcc.gnu.org/gcc-5/changes.html - Closes #11799 + Reported-by: Dan Fandrich -- tests: fix a type warning on 32-bit x86 + Fixes https://github.com/curl/curl/issues/12124 + Closes https://github.com/curl/curl/pull/12128 -Viktor Szakats (4 Sep 2023) +Carlos Henrique Lima Melara (14 Oct 2023) -- tests: delete stray `.orig` file +- docs/libcurl: fix three minor man page format mistakes - Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f - Closes #11797 + Reported-by: Samuel Henrique -Daniel Stenberg (4 Sep 2023) + Closes https://github.com/curl/curl/pull/12126 -- RELEASE-NOTES: synced +Jay Satiro (14 Oct 2023) -Viktor Szakats (4 Sep 2023) +- tests/server: add more SOCKS5 handshake error checking -- lib: silence compiler warning in inet_ntop6 + - Add additional checking for missing and too-short SOCKS5 handshake + messages. - ``` - ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her - e [-Wcomma] - cur.base = i, cur.len = 1; - ^ - ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin - g - cur.base = i, cur.len = 1; - ^~~~~~~~~~~~ - (void)( ) - ``` + Prior to this change the SOCKS5 test server did not check that all parts + of the handshake were received successfully. If those parts were missing + or too short then the server would access uninitialized memory. - Closes #11790 + This issue was discovered in CI job 'memory-sanitizer' test results. + Test 2055 was failing due to the SOCKS5 test server not running. It was + not running because either it crashed or memory sanitizer aborted it + during Test 728. Test 728 connects to the SOCKS5 test server on a + redirect but does not send any data on purpose. The test server was not + prepared for that. -Daniel Stenberg (4 Sep 2023) + Reported-by: Dan Fandrich -- transfer: also stop the sending on closed connection + Fixes https://github.com/curl/curl/issues/12117 + Closes https://github.com/curl/curl/pull/12118 - Previously this cleared the receiving bit only but in some cases it is - also still sending (like a request-body) when disconnected and neither - direction can continue then. +Daniel Stenberg (14 Oct 2023) - Fixes #11769 - Reported-by: Oleg Jukovec - Closes #11795 +- RELEASE-NOTES: synced -John Bampton (4 Sep 2023) +Sohom Datta (14 Oct 2023) -- docs: change `sub-domain` to `subdomain` +- tool_getparam: limit --rate to be smaller than number of ms - https://en.wikipedia.org/wiki/Subdomain + Currently, curl allows users to specify absurd request rates that might + be higher than the number of milliseconds in the unit (ex: curl --rate + 3600050/h http://localhost:8080 does not error out despite there being + only 3600000ms in a hour). - Closes #11793 + This change adds a conditional check before the millisecond calculation + making sure that the number is not higher than the numerator (the unit) + If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE -Stefan Eissing (4 Sep 2023) + Closes #12116 -- multi: more efficient pollfd count for poll +Daniel Stenberg (14 Oct 2023) - - do not use separate pollfds for sockets that have POLLIN+POLLOUT +- opts: fix two minor man page format mistakes - Closes #11792 +Jay Satiro (14 Oct 2023) -- http2: polish things around POST +- curl_trc: remove a bad assertion - - added test cases for various code paths - - fixed handling of blocked write when stream had - been closed inbetween attempts - - re-enabled DEBUGASSERT on send with smaller data size + - Remove DEBUGASSERT that an internal handle must not have user + private_data set before calling the user's debug callback. - - in debug builds, environment variables can be set to simulate a slow - network when sending data. cf-socket.c and vquic.c support - * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be - answered with a EAGAIN. TCP/UNIX sockets. - This is chosen randomly. - * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written - to the network. TCP/UNIX sockets. - Example: 80 means a send with 1000 bytes would only send 800 - This is applied to every send. - * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be - answered with EAGAIN. QUIC only. - This is chosen randomly. + This is a follow-up to 0dc40b2a. The user can distinguish their easy + handle from an internal easy handle by setting CURLOPT_PRIVATE on their + easy handle. I had wrongly assumed that meant the user couldn't then + set CURLOPT_PRIVATE on an internal handle as well. - Closes #11756 + Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 + Reported-by: Daniel Stenberg -Daniel Stenberg (4 Sep 2023) + Closes https://github.com/curl/curl/pull/12104 -- docs: add curl_global_trace to some SEE ALSO sections +Dan Fandrich (13 Oct 2023) - Closes #11791 +- test613: stop showing an error on missing output file -- os400: fix checksrc nits + This test would show an error message if the output was missing during + the log post-processing step, but the message was not captured by the + test harness and wasn't useful since the normal golden log file + comparison would the problem more clearly. - Closes #11789 +Stefan Eissing (13 Oct 2023) -Nicholas Nethercote (3 Sep 2023) +- quic: manage connection idle timeouts -- hyper: remove `hyptransfer->endtask` + - configure a 120s idle timeout on our side of the connection + - track the timestamp when actual socket IO happens + - check IO timestamp to our *and* the peer's idle timeouts + in "is this connection alive" checks - `Curl_hyper_stream` needs to distinguish between two kinds of - `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and - (b) background tasks that hyper produces. It does this by recording the - address of any `foreach` task in `hyptransfer->endtask` before pushing - it into the executor, and then comparing that against the address of - tasks later polled out of the executor. + Reported-by: calvin2021y on github + Fixes #12064 + Closes #12077 - This works right now, but there is no guarantee from hyper that the - addresses are stable. `hyper_executor_push` says "The executor takes - ownership of the task, which should not be accessed again unless - returned back to the user with `hyper_executor_poll`". That wording is a - bit ambiguous but with my Rust programmer's hat on I read it as meaning - the task returned with `hyper_executor_poll` may be conceptually the - same as a task that was pushed, but that there are no other guarantees - and comparing addresses is a bad idea. +Dan Fandrich (13 Oct 2023) - This commit instead uses `hyper_task_set_userdata` to mark the `foreach` - task with a `USERDATA_RESP_BODY` value which can then be checked for, - removing the need for `hyptransfer->endtask`. This makes the code look - more like that hyper C API examples, which use userdata for every task - and never look at task addresses. +- CI: ignore test 286 on Appveyor gcc 9 build - Closes #11779 + This test fails sometimes with a super fast retry loop due to what may + just be a compiler bug. The test results are ignored on the one CI job + where it occurs because there seems to be nothing we can do to fix it. -Dave Cottlehuber (3 Sep 2023) + Fixes #12040 + Closes #12106 -- ws: fix spelling mistakes in examples and tests +Viktor Szakats (13 Oct 2023) - Closes #11784 +- lib: fix gcc warning in printf call -Daniel Stenberg (3 Sep 2023) + Do not pass NULL to printf %s. -- tool_filetime: make -z work with file dates before 1970 + Seen with gcc 13.2.0 on Debian: + ``` + .../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wfo + rmat-overflow=] + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/1758442 + 6483#step:3:11104 - Fixes #11785 - Reported-by: Harry Sintonen - Closes #11786 + Ref: #10284 + Co-authored-by: Jay Satiro + Closes #12082 -Dan Fandrich (1 Sep 2023) +Alex Klyubin (13 Oct 2023) -- build: fix portability of mancheck and checksrc targets +- http2: safer invocation of populate_binsettings - At least FreeBSD preserves cwd across makefile lines, so rules - consisting of more than one "cd X; do_something" must be explicitly run - in a subshell to avoid this. This problem caused the Cirrus FreeBSD - build to fail when parallel make jobs were enabled. + populate_binsettings now returns a negative value on error, instead of a + huge positive value. Both places which call this function have been + updated to handle this change in its contract. -- CI: adjust labeler match patterns for new & obsolete files + The way populate_binsettings had been used prior to this change the huge + positive values -- due to signed->unsigned conversion of the potentially + negative result of nghttp2_pack_settings_payload which returns negative + values on error -- are not possible. But only because http2.c currently + always provides a large enough output buffer and provides H2 SETTINGS + IVs which pass the verification logic inside nghttp2. If the + verification logic were to change or if http2.c started passing in more + IVs without increasing the output buffer size, the overflow could become + reachable, and libcurl/curl might start leaking memory contents to + servers/proxies... -- configure: trust pkg-config when it's used for zlib + Closes #12101 - The library flags retrieved from pkg-config were later thrown out and - harded-coded, which negates the whole reason to use pkg-config. - Also, previously, the assumption was made that --libs-only-l and - --libs-only-L are the full decomposition of --libs, which is untrue and - would not allow linking against a static zlib. The new approach is - better in that it uses --libs, although only if --libs-only-l returns - nothing. +Daniel Stenberg (13 Oct 2023) - Bug: https://curl.se/mail/lib-2023-08/0081.html - Reported-by: Randall - Closes #11778 +- openssl: avoid BN_num_bits() NULL pointer derefs -Stefan Eissing (1 Sep 2023) + Reported-by: icy17 on github + Fixes #12099 + Closes #12100 -- CI/ngtcp2: clear wolfssl for when cache is ignored +- wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA - Closes #11783 + This define is set in wolfssl's options.h file when this function and + feature is present. Handles both builds with the feature explicitly + disabled and wolfSSL versions before 5.5.2 - which introduced this API + call. -Daniel Stenberg (1 Sep 2023) + Closes #12108 -- RELEASE-NOTES: synced +- tool_urlglob: make multiply() bail out on negative values -Nicholas Nethercote (1 Sep 2023) + - Does not work correctly with negative values + - use __builtin_mul_overflow() on gcc -- hyper: fix a progress upload counter bug + Reported-by: Torben Dury + Closes #12102 - `Curl_pgrsSetUploadCounter` should be a passed a total count, not an - increment. +Loïc Yhuel (13 Oct 2023) - This changes the failing diff for test 579 with hyper from this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 29 out of 0[LF] - ``` - to this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 40 out of 0[LF] - ``` - Presumably a step in the right direction. +- cmake: fix CURL_DISABLE_GETOPTIONS - Closes #11780 + - Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake. -Daniel Stenberg (1 Sep 2023) + Prior to this change the option had no effect because it was missing + from that file. -- awssiv4: avoid freeing the date pointer on error + Closes https://github.com/curl/curl/pull/12091 - Since it was not allocated, don't free it even if it was wrong syntax +- easy_lock: add a pthread_mutex_t fallback - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 + This allows to keep the init threadsafe with gcc < 4.9.0 (no C11 + atomics). - Follow-up to b137634ba3adb + Closes https://github.com/curl/curl/pull/12090 - Closes #11782 +Viktor Szakats (12 Oct 2023) -Stefan Eissing (1 Sep 2023) +- CI: add autotools, out-of-tree, debug build to distro check job -- CI: ngtcp2-linux: use separate caches for tls libraries + Add a job that builds curl from a generated source tarball sample, with + autotools, out-of-tree, in debug mode. - allow ever changing master for wolfssl + Ref: #12085 + Closes #12088 - Closes #11766 +Daniel Stenberg (12 Oct 2023) -- replace `master` as wolfssl-version with recent commit +- http: avoid Expect: 100-continue if Upgrade: is used -- wolfssl, use master again in CI + Reported-by: Daniel Jelinski + Fixes #12022 + Closes #12062 - - with the shared session update fix landed in master, it - is time to use that in our CI again +Jan Alexander Steffens (heftig) (12 Oct 2023) -Nicholas Nethercote (31 Aug 2023) +- docs: use SOURCE_DATE_EPOCH for generated manpages -- tests: fix formatting errors in `FILEFORMAT.md`. + This should make builds from Git reproducible. - Without the surrounding backticks, these tags get swallowed when the - markdown is rendered. + Closes #12092 - Closes #11777 +Daniel Stenberg (12 Oct 2023) -Viktor Szakats (31 Aug 2023) +- RELEASE-NOTES: synced -- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` + Bumped to 8.4.1 - Allow overriding the default TLS backend via a CMake setting. +Viktor Szakats (12 Oct 2023) - E.g.: - `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` +- cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection - Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, - schannel, secure-transport, wolfssl + Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet, + aligning this with autotools. This fixes an error when doing + cross-builds and also actually detects this feature. It affected systems + not allowlisted into this, e.g. SerenityOS. - The passed string is baked into the curl/libcurl binaries. - The value is case-insensitive. + We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`. - We added a similar option to autotools in 2017 via - c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. + Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979 + Ref: #12095 (closed in favour of this patch) + Ref: #11964 (effort to sync cmake detections with autotools) - TODO: Convert to lowercase to improve reproducibility. + Reported-by: Kartatz on Github + Assisted-by: Kartatz on Github + Fixes #12093 + Closes #12094 - Closes #11774 +- build: add `src/.checksrc` to source tarball -- sectransp: fix compiler warnings + Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958 - https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# - step:3:11046 - ``` - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 - 4: warning: unused variable 'success' [-Wunused-variable] - OSStatus success; - ^ - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 - 4: warning: unused parameter 'sha256len' [-Wunused-parameter] - size_t sha256len) - ^ - ``` + Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071 + Reported-by: Romain Geissler + Fixes #12084 + Closes #12085 - Closes #11773 +Version 8.4.0 (11 Oct 2023) -- tidy-up: mostly whitespace nits +Daniel Stenberg (11 Oct 2023) - - delete completed TODO from `./CMakeLists.txt`. - - convert a C++ comment to C89 in `./CMake/CurlTests.c`. - - delete duplicate EOLs from EOF. - - add missing EOL at EOF. - - delete whitespace at EOL (except from expected test results). - - convert tabs to spaces. - - convert CRLF EOLs to LF in GHA yaml. - - text casing fixes in `./CMakeLists.txt`. - - fix a codespell typo in `packages/OS400/initscript.sh`. +- RELEASE-NOTES: synced - Closes #11772 +- THANKS: add contributors from 8.4.0 -Dan Fandrich (31 Aug 2023) +Jay Satiro (11 Oct 2023) -- CI: remove Windows builds from Cirrus, without replacement +- socks: return error if hostname too long for remote resolve - If we don't do this, all coverage on Cirrus will cease in a few days. By - removing the Windows builds, the FreeBSD one should still continue - as before. The Windows builds will need be moved to another service to - maintain test coverage. + Prior to this change the state machine attempted to change the remote + resolve to a local resolve if the hostname was longer than 255 + characters. Unfortunately that did not work as intended and caused a + security issue. - Closes #11771 + Bug: https://curl.se/docs/CVE-2023-38545.html -- CI: switch macOS ARM build from Cirrus to Circle CI +Stefan Eissing (10 Oct 2023) - Cirrus is drastically reducing their free tier on Sept. 1, so they will - no longer perform all these builds for us. All but one build has been - moved, with the LibreSSL one being dropped because of linking problems - on Circle. +- CI: remove slowed-network tests - One important note about this change is that Circle CI is currently - directing all these builds to x86_64 hardware, despite them requesting - ARM. This is because ARM nodes are scheduled to be available on the - free tier only in December. This reduces our architectural diversity - until then but it should automatically come back once those machines are - enabled. + - remove these tests as they are currently not reliable in our CI + setups. -- CI: use the right variable for BSD make + curl handles the test cases, but CI sometimes fails on these due to + additional conditions. Rather than mix them in, an additional CI job + will be added in the future that is specific to them. - BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel - builds before. + Closes https://github.com/curl/curl/pull/12075 -- CI: drop the FreeBSD 12.X build +Jay Satiro (10 Oct 2023) - Cirrus' new free tier won't let us have many builds, so drop the - nonessential ones. The FreeBSD 13.X build will still give us the most - relevant FreeBSD coverage. +- libcurl-env-dbg.3: move debug variables from libcurl-env.3 -- CI: move the Alpine build from Cirrus to GHA + - Move documentation of libcurl environment variables used only in debug + builds from libcurl-env into a separate document libcurl-env-dbg. - Cirrus is reducing their free tier to next to nothing, so we must move - builds elsewhere. + - Document more debug environment variables. -Stefan Eissing (30 Aug 2023) + Previously undocumented or missing a description: -- test_07_upload.py: fix test_07_34 curl args + CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, + CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, + CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, + CURL_TIME. - - Pass correct filename to --data-binary. + Closes https://github.com/curl/curl/pull/11811 - Prior to this change --data-binary was passed an incorrect filename due - to a missing separator in the arguments list. Since aacbeae7 curl will - error on incorrect filenames for POST. +Dan Fandrich (9 Oct 2023) - Fixes https://github.com/curl/curl/issues/11761 - Closes https://github.com/curl/curl/pull/11763 +- test670: increase the test timeout -Nicholas Nethercote (30 Aug 2023) + This should make it more immune to loaded servers. -- tests: document which tests fail due to hyper's lack of trailer support. + Ref: #11328 - Closes #11762 +Stefan Eissing (9 Oct 2023) -- docs: removing "pausing transfers" from HYPER.md. +- MQTT: improve receive of ACKs - It's a reference to #8600, which was fixed by #9070. + - add `mq->recvbuf` to provide buffering of incomplete + ACK responses + - continue ACK reading until sufficient bytes available + - fixes test failures on low network receives - Closes #11764 + Closes #12071 -Patrick Monnerat (30 Aug 2023) +Viktor Szakats (9 Oct 2023) -- os400: handle CURL_TEMP_PRINTF() while building bind source +- quic: fix BoringSSL build - Closes #11547 + Add guard around `SSL_CTX_set_ciphersuites()` use. -- os400: build test servers + Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 - Also fix a non-compliant main prototype in disabled.c. + Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 - Closes #11547 + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes #12067 -- tests: fix compilation error for os400 +Stefan Eissing (9 Oct 2023) - OS400 uses BSD 4.3 setsockopt() prototype by default: this does not - define parameter as const, resulting in an error if actual parameter is - const. Remove the const keyword from the actual parameter cast: this - works in all conditions, even if the formal parameter uses it. +- test1540: improve reliability - Closes #11547 + - print that bytes have been received on pausing, but not how many -- os400: make programs and command name configurable + Closes #12069 - Closes #11547 +- test2302: improve reliability -- os400: move build configuration parameters to a separate script + - make result print collected write data, unless + change in meta flags is detected + - will show same result even when data arrives via + several writecb invocations - They can then easily be overriden in a script named "config400.override" - that is not part of the distribution. + Closes #12068 - Closes #11547 +Daniel Stenberg (9 Oct 2023) -- os400: implement CLI tool +- curl_easy_pause: set "in callback" true on exit if true - This is provided as a QADRT (ascii) program, a link to it in the IFS and - a minimal CL command. + Because it might have called another callback in the mean time that then + set the bit FALSE on exit. - Closes #11547 + Reported-by: Jay Satiro + Fixes #12059 + Closes #12061 -Matthias Gatto (30 Aug 2023) +Viktor Szakats (8 Oct 2023) -- lib: fix aws-sigv4 having date header twice in some cases +- h3: add support for ngtcp2 with AWS-LC builds - When the user was providing the header X-XXX-Date, the header was - re-added during signature computation, and we had it twice in the - request. + ``` + curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- + LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps + mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss + Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi + SSL NTLM SSL threadsafe UnixSockets + ``` - Reported-by: apparentorder@users.noreply.github.com + Also delete an obsolete GnuTLS TODO and update the header comment in + `FindNGTCP2.cmake`. - Signed-off-by: Matthias Gatto + Reviewed-by: Daniel Stenberg + Closes #12066 - Fixes: https://github.com/curl/curl/issues/11738 - Closes: https://github.com/curl/curl/pull/11754 +- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros -Jay Satiro (30 Aug 2023) + Syncing this up with CMake. -- multi: remove 'processing: ' debug message + Source code uses the built-in `OPENSSL_IS_AWSLC` and + `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is + necessary from the build tools. - - Remove debug message added by e024d566. + The one use of `HAVE_BORINGSSL` in the source turned out to be no longer + necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 - Closes https://github.com/curl/curl/pull/11759 + autotools detects this anyway for display purposes. + CMake detects this to decide whether to use the BoringSSL-specific + crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection + result just yet (planned in #12066). -- ftp: fix temp write of ipv6 address + Ref: #11964 - - During the check to differentiate between a port and IPv6 address - without brackets, write the binary IPv6 address to an in6_addr. + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12065 - Prior to this change the binary IPv6 address was erroneously written to - a sockaddr_in6 'sa6' when it should have been written to its in6_addr - member 'sin6_addr'. There's no fallout because no members of 'sa6' are - accessed before it is later overwritten. +Marc Hoersken (8 Oct 2023) - Closes https://github.com/curl/curl/pull/11747 +- CI: move distcheck job from Azure Pipelines to GitHub Actions -- tool: change some fopen failures from warnings to errors + This will allow for more trigger excludes within Azure Pipelines. - - Error on missing input file for --data, --data-binary, - --data-urlencode, --header, --variable, --write-out. + Also fixes seemingly broken check with scripts/installcheck.sh. + Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 - Prior to this change if a user of the curl tool specified an input file - for one of the above options and that file could not be opened then it - would be treated as zero length data instead of an error. For example, a - POST using `--data @filenametypo` would cause a zero length POST which - is probably not what the user intended. + Assisted-by: Philip Heiduck + Closes #9532 - Closes https://github.com/curl/curl/pull/11677 +Daniel Stenberg (8 Oct 2023) -- hostip: fix typo +- url: fall back to http/https proxy env-variable if ws/wss not set -Davide Masserut (29 Aug 2023) + Reported-by: Craig Andrews + Fixes #12031 + Closes #12058 -- tool: avoid including leading spaces in the Location hyperlink +Stefan Eissing (8 Oct 2023) - Co-authored-by: Dan Fandrich +- cf-socket: simulate slow/blocked receives in debug - Closes #11735 + add 2 env variables for non-UDP sockets: + 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly + should return EAGAIN + 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket -Daniel Stenberg (29 Aug 2023) + Closes #12035 -- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline +- http2: refused stream handling for retry - Closes #11757 + - answer HTTP/2 streams refused via a GOAWAY from the server to + respond with CURLE_RECV_ERROR in order to trigger a retry + on another connection -- connect: stop halving the remaining timeout when less than 600 ms left + Reported-by: black-desk on github + Ref #11859 + Closes #12054 - When curl wants to connect to a host, it always has a TIMEOUT. The - maximum time it is allowed to spend until a connect is confirmed. +Jay Satiro (8 Oct 2023) - curl will try to connect to each of the IP adresses returned for the - host. Two loops, one for each IP family. +- CURLOPT_DEBUGFUNCTION.3: warn about internal handles - During the connect loop, while curl has more than one IP address left to - try within a single address family, curl has traditionally allowed (time - left/2) for *this* connect attempt. This, to not get stuck on the - initial addresses in case the timeout but still allow later addresses to - get attempted. - - This has the downside that when users set a very short timeout and the - host has a large number of IP addresses, the effective result might be - that every attempt gets a little too short time. + - Warn that the user's debug callback may be called with the handle + parameter set to an internal handle. - This change stop doing the divided-by-two if the total time left is - below a threshold. This threshold is 600 milliseconds. + Without this warning the user may assume that the only handles their + debug callback receives are the easy handles on which they set + CURLOPT_DEBUGFUNCTION. - Closes #11693 + This is a follow-up to f8cee8cc which changed DoH handles to inherit + the debug callback function set in the user's easy handle. As a result + those handles are now passed to the user's debug callback function. -- asyn-ares: reduce timeout to 2000ms + Closes https://github.com/curl/curl/pull/12034 - When UDP packets get lost this makes for slightly faster retries. This - lower timeout is used by @c-ares itself by default starting next - release. +- url: fix typo - Closes #11753 +Daniel Stenberg (8 Oct 2023) -John Bampton (29 Aug 2023) +- test458: verify --expand-output, expanding a file name accepting option -- misc: remove duplicate words + Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) - Closes #11740 +- tool_getparam: accept variable expansion on file names too -Daniel Stenberg (29 Aug 2023) + Reported-by: PBudmark on github + Fixes #12048 + Closes #12055 - RELEASE-NOTES: synced -- wolfSSL: avoid the OpenSSL compat API when not needed +- multi: do CURLM_CALL_MULTI_PERFORM at two more places - ... and instead call wolfSSL functions directly. + ... when it does a state transition but there is no particular socket or + timer activity. This was made apparent when commit b5bb84c removed a + superfluous timer expiry. - Closes #11752 + Reported-by: Dan Fandrich. + Fixes #12033 + Closes #12056 -Viktor Szakats (28 Aug 2023) +Viktor Szakats (7 Oct 2023) -- lib: fix null ptr derefs and uninitialized vars (h2/h3) +- GHA/linux: mbedtls 3.5.0 + minor dep bumps - Fixing compiler warnings with gcc 13.2.0 in unity builds. + Closes #12057 - Assisted-by: Jay Satiro - Assisted-by: Stefan Eissing - Closes #11739 +Dan Fandrich (7 Oct 2023) -Jay Satiro (28 Aug 2023) +- CI: bump OpenLDAP package version on FreeBSD -- secureserver.pl: fix stunnel version parsing + The old one is no longer available. - - Allow the stunnel minor-version version part to be zero. +Marc Hoersken (7 Oct 2023) - Prior to this change with the stunnel version scheme of . - if either part was 0 then version parsing would fail, causing - secureserver.pl to fail with error "No stunnel", causing tests that use - the SSL protocol to be skipped. As a practical matter this bug can only - be caused by a minor-version part of 0, since the major-version part is - always greater than 0. +- docs/libcurl/opts/Makefile.inc: add missing manpage files - Closes https://github.com/curl/curl/pull/11722 + Detected with #9532 -- secureserver.pl: fix stunnel path quoting +Dan Fandrich (7 Oct 2023) - - Store the stunnel path in the private variable $stunnel unquoted and - instead quote it in the command strings. +- tests: fix a race condition in ftp server disconnect - Prior to this change the quoted stunnel path was passed to perl's file - operators which cannot handle quoted paths. For example: + If a client disconnected and reconnected quickly, before the ftp server + had a chance to respond, the protocol message/ack (ping/pong) sequence + got out of sync, causing messages sent to the old client to be delivered + to the new. A disconnect must now be acknowledged and intermediate + requests thrown out until it is, which ensures that such synchronization + problems can't occur. This problem could affect ftp, pop3, imap and smtp + tests. - $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; - if(-x $stunnel or -x "$stunnel") - # false even if path exists and is executable + Fixes #12002 + Closes #12049 - Our other test scripts written in perl, unlike this one, use servers.pm - which has a global $stunnel variable with the path stored unquoted and - therefore those scripts don't have this problem. +Viktor Szakats (7 Oct 2023) - Closes https://github.com/curl/curl/pull/11721 +- appveyor: bump mingw-w64 job to gcc 13 (was: 8) -Daniel Stenberg (28 Aug 2023) + This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). + Adding a modern gcc version to the tests. -- altsvc: accept and parse IPv6 addresses in response headers + (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 + finished in 32, 35, 34 minutes in the 3 test runs so far.) - Store numerical IPv6 addresses in the alt-svc file with the brackets - present. + It also adds a modern CMake version and OS env to our mingw-w64 builds. - Verify with test 437 and 438 + Closes #12051 - Fixes #11737 - Reported-by: oliverpool on github - Closes #11743 +David Benjamin (6 Oct 2023) -- libtest: use curl_free() to free libcurl allocated data +- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR - In several test programs. These mistakes are not detected or a problem - as long as memdebug.h is included, as that provides the debug wrappers - for all memory functions in the same style libcurl internals do it, - which makes curl_free and free effectively the same call. + While the struct is still public in OpenSSL, there is a (somewhat + inconvenient) accessor. Use it to remain compatible if it becomes opaque + in the future. - Reported-by: Nicholas Nethercote - Closes #11746 + Closes #12038 -Jay Satiro (28 Aug 2023) +Daniel Stenberg (6 Oct 2023) -- disable.d: explain --disable not implemented prior to 7.50.0 +- curl_easy_pause.3: mention it works within callbacks - Option -q/--disable was added in 5.0 but only -q was actually - implemented. Later --disable was implemented in e200034 (precedes - 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). + Reported-by: Maxim Dzhura + Bug: https://curl.se/mail/lib-2023-10/0010.html + Closes #12046 - Reported-by: pszlazak@users.noreply.github.com +- curl_easy_pause.3: mention h2/h3 buffering - Fixes https://github.com/curl/curl/issues/11710 - Closes #11712 + Asked-by: Maxim Dzhura + Ref: https://curl.se/mail/lib-2023-10/0011.html -Nicholas Nethercote (28 Aug 2023) + Closes #12045 -- hyper: fix ownership problems +Viktor Szakats (6 Oct 2023) - Some of these changes come from comparing `Curl_http` and - `start_CONNECT`, which are similar, and adding things to them that are - present in one and missing in another. +- cmake: re-add missed C89 headers for specific detections - The most important changes: - - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the - happy path. - - In `start_CONNECT`, add a missing `hyper_request_free` on the error - path. - - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. - - In `bodysend`, remove an unnecessary `hyper_body_free` on a different - error path that would cause a double-free. - https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html - says of `hyper_request_set_body`: "This takes ownership of the - hyper_body *, you must not use it or free it after setting it on the - request." This is true even if `hyper_request_set_body` returns an - error; I confirmed this by looking at the hyper source code. + We removed C89 `setjmp.h` and `signal.h` detections and excluded them + from the global header list we use when detecting functions [1]. Then + missed to re-add these headers to the specific functions which need + them to be detected [2]. Fix this omission in this patch. - Other changes are minor but make things slightly nicer. + [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 + [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Closes #11745 + Closes #12043 -Daniel Stenberg (28 Aug 2023) +Daniel Stenberg (6 Oct 2023) -- multi.h: the 'revents' field of curl_waitfd is supported +- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE - Since 6d30f8ebed34e7276 + Since there is nothing to wait for there. Avoids the test 1233 hang + reported in #12033. - Reported-by: Nicolás Ojeda Bär - Ref: #11748 - Closes #11749 + Reported-by: Dan Fandrich + Closes #12042 -Gerome Fournier (27 Aug 2023) +Dan Fandrich (5 Oct 2023) -- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() +- test1903: actually verify the cookies after the test - Closes #11742 + The test otherwise could do just about anything (except leak memory in + debug mode) and its bad behaviour wouldn't be detected. Now, check the + resulting cookie file to ensure the cookies are still there. -Daniel Stenberg (27 Aug 2023) + Closes #12041 -- docs: mention critical files in same directories as curl saves +- test: add missing s - ... cannot be fully protected. Don't do it. + The tests will otherwise fail if curl has them disabled. - Co-authored-by: Jay Satiro - Reported-by: Harry Sintonen - Fixes #11530 - Closes #11701 +- test1906: set a lower timeout since it's hit on Windows -John Hawthorn (26 Aug 2023) + msys2 builds actually hit the connect timeout in normal operation, so + lower the timeout from 5 minutes to 5 seconds to reduce test time. -- OpenSSL: clear error queue after SSL_shutdown + Ref: #11328 + Closes #12036 - We've seen errors left in the OpenSSL error queue (specifically, - "shutdown while in init") by adding some logging it revealed that the - source was this file. +Daniel Stenberg (5 Oct 2023) - Since we call SSL_read and SSL_shutdown here, but don't check the return - code for an error, we should clear the OpenSSL error queue in case one - was raised. +- RELEASE-NOTES: synced - This didn't affect curl because we call ERR_clear_error before every - write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when - libcurl is used in a process with other OpenSSL users, they may detect - an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their - own. +Jay Satiro (5 Oct 2023) - Co-authored-by: Satana de Sant'Ana +- idn: fix WinIDN null ptr deref on bad host - Closes #11736 + - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from + UTF-8 to UTF-16. -Alexander Kanavin (25 Aug 2023) + Prior to this change a failed conversion erroneously returned CURLE_OK + which meant 'decoded' pointer (what would normally point to the + punycode) would not be written to, remain NULL and be dereferenced + causing an access violation. -- tests: update cookie expiry dates to far in the future + Closes https://github.com/curl/curl/pull/11983 - This allows testing Y2038 with system time set to after that, so that - actual Y2038 issues can be exposed, and not masked by expiry errors. +Dan Fandrich (4 Oct 2023) - Fixes #11576 - Closes #11610 +- tests: close the shell used to start sshd -John Bampton (25 Aug 2023) + This shell isn't needed once sshd starts, so use "exec" so it doesn't + stick around. -- misc: fix spelling + Closes #12032 - Closes #11733 +Daniel Stenberg (4 Oct 2023) -Daniel Stenberg (25 Aug 2023) +- base64: also build for curl -- cmdline-opts/page-header: clarify stronger that !opt == URL + Since the tool itself now uses the base64 code using the curlx way, it + needs to build also when the tool needs it. Starting now, the tool build + defines BULDING_CURL to allow lib-side code to use it. - Everything provided on the command line that is not an option (or an - argument to an option) is treated as a URL. + Follow-up to 2e160c9c6525 - Closes #11734 + Closes #12010 -- tests/runner: fix %else handling +Eduard Strehlau (4 Oct 2023) - Getting the show state proper for %else and %endif did not properly work - in nested cases. +- tests: Fix zombie processes left behind by FTP tests. - Follow-up to 3d089c41ea9 + ftpserver.pl correctly cleans up spawned server processes, + but forgets to wait for the shell used to spawn them. + This is barely noticeable during a normal testrun, + but causes process exhaustion and test failure + during a complete torture run of the FTP tests. - Closes #11731 + Fixes #12018 + Closes #12020 -Nicholas Nethercote (25 Aug 2023) +Dan Fandrich (4 Oct 2023) -- docs: Remove mention of #10803 from `KNOWN_BUGS`. +- github/labeler: improve labeler matches - Because the leaks have been fixed. +- test574: add a timeout to the test -- c-hyper: fix another memory leak in `Curl_http`. + This one hangs occasionally, so this will speed up a test run and allow + logs to be seen when it does. - There is a `hyper_clientconn_free` call on the happy path, but not one - on the error path. This commit adds one. + Closes #12025 - Fixes the second memory leak reported by Valgrind in #10803. +- tests: propagate errors in libtests - Fixes #10803 - Closes #11729 + Use the test macros to automatically propagate some errors, and check + and log others while running the tests. This can help in debugging + exactly why a test has failed. -- c-hyper: fix a memory leak in `Curl_http`. +- tests: set --expect100-timeout to improve test reliability - A request created with `hyper_request_new` must be consumed by either - `hyper_clientconn_send` or `hyper_request_free`. + On an overloaded server, the default 1 second timeout can go by without + the test server having a chance to respond with the expected headers, + causing tests to fail. Increase the 1 second timeout to 99 seconds so + this failure mode is no longer a problem on test 1129. Some other tests + already set a high value, but make them consistently 99 seconds so if + something goes wrong the test is stalled for less time. - This is not terrifically clear from the hyper docs -- - `hyper_request_free` is documented only with "Free an HTTP request if - not going to send it on a client" -- but a perusal of the hyper code - confirms it. + Ref: #11328 - This commit adds a `hyper_request_free` to the `error:` path in - `Curl_http` so that the request is consumed when an error occurs after - the request is created but before it is sent. +- CI: ignore the "flaky" and "timing-dependent" test results in CMake - Fixes the first memory leak reported by Valgrind in #10803. + This was already done for automake builds but CMake builds were missed. + Test 1086 actually causes the test harness to crash with: - Closes #11729 + Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec + ts/curl/tests/ftpserver.pl line 527 -Daniel Stenberg (25 Aug 2023) + Rather than fix it now, this change leaves test 1086 entirely skipped on + those builds that show this problem. -- RELEASE-NOTES: synced + Follow-up to 589dca761 -John Bampton (25 Aug 2023) + Ref: #11865 -- misc: spellfixes +Viktor Szakats (4 Oct 2023) - Closes #11730 +- cmake: improve OpenLDAP builds -Daniel Stenberg (25 Aug 2023) + - cmake: detect OpenLDAP based on function `ldap_init_fd`. + autotools does this. autotools also publishes this detection result + in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source + doesn't use this value. (it might need to be remove-listed in + `scripts/cmp-config.pl` for future OpenLDAP test builds.) + This also deletes existing self-declaration method via the + CMake-specific `CURL_USE_OPENLDAP` configuration. -- tests: add support for nested %if conditions + - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. + Like autotools does. This fixes a long list of these warnings: + ``` + /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i + s not defined, evaluates to 0 [-Wundef] + ``` - Provides more flexiblity to test cases. + - cmake: delete LDAP TODO comment no longer relevant. - Also warn and bail out if there is an '%else' or %endif' without a - preceeding '%if'. + Also: - Ref: #11610 - Closes #11728 + - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature + detection functions. -- time-cond.d: mention what happens on a missing file + Ref: #11964 (effort to sync cmake detections with autotools) - Closes #11727 + Closes #12024 -Christian Hesse (24 Aug 2023) +- cmake: fix unity builds for more build combinations -- docs/cmdline-opts: match the current output + By using unique static function/variable names in source files + implementing these interfaces. - The release date has been added in output, reflect that in documentation. + - OpenLDAP combined with any SSH backend. - Closes #11723 + - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. -Daniel Stenberg (24 Aug 2023) + Closes #12027 -- lib: minor comment corrections +Daniel Stenberg (4 Oct 2023) -- docs: rewrite to present tense +- tests: remove leading spaces from some tags - ... instead of using future tense. + The threee tags ``, `` and `` were frequently used + with a leading space that this removes. The reason this habbit is so + widespread in testcases is probably that they have been copy and pasted. - + numerous cleanups and improvements - + stick to "reuse" not "re-use" - + fewer contractions + Hence, fixing them all now might curb this practice from now on. - Closes #11713 + Closes #12028 -- urlapi: setting a blank URL ("") is not an ok URL +Viktor Szakats (4 Oct 2023) - Test it in 1560 - Fixes #11714 - Reported-by: ad0p on github - Closes #11715 +- GHA: bump actions/checkout -- spelling: use 'reuse' not 're-use' in code and elsewhere + Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 + Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 - Unify the spelling as both versions were previously used intermittently + Closes #12023 - Closes #11717 +- spelling: fix codespell 2.2.6 typos -Michael Osipov (23 Aug 2023) + Closes #12019 -- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC +Daniel Stenberg (3 Oct 2023) - HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the - default one. Use "long long" in 32 bit mode and just "long" in 64 bit - mode. +- GHA: add workflow to compare configure vs cmake outputs - Closes #11718 + Uses scripts/cmp-config.pl two compare two curl_config.h files, + presumbly generated with configure and cmake. It displays the + differences and filters out a lot of known lines we ignore. -Dan Fandrich (22 Aug 2023) + The script also shows the matches that were *not* used. Possibly + subjects for removal. -- tests: don't call HTTP errors OK in test cases + Closes #11964 - Some HTTP errors codes were accompanied by the text OK, which causes - some cognitive dissonance when reading them. +- appveyor: enable test 571 -- http: close the connection after a late 417 is received + Follow-up from 8a940fd55c175f7 / #12013 - In this situation, only part of the data has been sent before aborting - so the connection is no longer usable. + Closes #12017 - Assisted-by: Jay Satiro - Fixes #11678 - Closes #11679 +Viktor Szakats (3 Oct 2023) -- runtests: slightly increase the longest log file displayed +- build: alpha-sort source files for lib and src - The new limit provides enough space for a 64 KiB data block to be logged - in a trace file, plus a few lines at the start and end for context. This - happens to be the amount of data sent at a time in a PUT request. + Closes #12014 -- tests: add delay command to the HTTP server +- cmake: delete old `HAVE_LDAP_URL_PARSE` logic - This adds a delay after client connect. + Left there by accident after adding proper detection for this. -Daniel Stenberg (22 Aug 2023) + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 -- cirrus: install everthing with pkg, avoid pip + Ref: #11964 (effort to sync cmake detections with autotools) - Assisted-by: Sevan Janiyan + Closes #12015 - Closes #11711 +Stefan Eissing (3 Oct 2023) -- curl_url*.3: update function descriptions +- tests: increase lib571 timeout from 3s to 30s - - expand and clarify several descriptions - - avoid using future tense all over + - 3s is too short for our CI, making this test fail occasionally + - test usually experiences no delay run locally, so 30s wont hurt - Closes #11708 + Closes #12013 -- RELEASE-NOTES: synced +Viktor Szakats (3 Oct 2023) -Stefan Eissing (21 Aug 2023) +- cmake: fix unity with Windows Unicode + TrackMemory -- CI/cirrus: disable python install on FreeBSD + Found the root cause of the startup crash in unity builds with Unicode + and TrackMemory enabled at the same time. - - python cryptography package does not build build FreeBSD - - install just mentions "error" - - this gets the build and the main test suite going again + We must make sure that the `memdebug.h` header doesn't apply to + `lib/curl_multibyte.c` (as even noted in a comment there.) In unity + builds all headers apply to all sources, including `curl_multibyte.c`. + This probably resulted in an infinite loop on startup. - Closes #11705 + Exclude this source from unity compilation with TrackMemory enabled, + in both libcurl and curl tool. Enable unity mode for a debug Unicode + CI job to keep it tested. Also delete the earlier workaround that + fully disabled unity for affected builds. -- test2600: fix flakiness on low cpu + Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - - refs #11355 where failures to to low cpu resources in CI - are reported - - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations - to test cases - - trigger Curl_expire() in test filter to allow re-checks before - the usual 1second interval + Closes #11928 - Closes #11690 +- cmake: disable unity mode with Windows Unicode + TrackMemory -Maksim Sciepanienka (20 Aug 2023) + "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, + aka `-DCURLDEBUG`). -- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf + There is an issue with memory tracking and Unicode when built in "unity" + mode, which results in the curl tool crashing right on startup, even + without any command-line option. Interestingly this doesn't happen under + WINE (at least on the system I tested this on), but consistenly happens + on real Windows machines. Crash is 0xC0000374 heap corruption. Both + shared and static curl executables are affected. - Closes #11698 + This limitation probably won't hit too many people, but it remains + a TODO to find and fix the root cause and drop this workaround. -Daniel Stenberg (20 Aug 2023) + Example builds and runs: + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd + 7iwj#L313 (static) + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby + qu9c#L317 (shared) -- test687/688: two more basic --xattr tests + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Closes #11697 + Ref: #11928 + Closes #12005 -- cmdline-opts/docs: mentioned the negative option part +- cmake: tidy-up `NOT_NEED_LBER_H` detection - ... for --no-alpn and --no-buffer in the same style done for other --no- - options: + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - "Note that this is the negated option name documented." +- appveyor: rewrite batch in PowerShell + CI improvements - Closes #11695 + 1. Rewrite in PowerShell: -Emanuele Torre (19 Aug 2023) + - rewrite MS-DOS batch build script in PowerShell. + - move some bash operations into native PowerShell. + - fixups for PowerShell insisting on failure when a command outputs + something to stderr. + - fix to actually run `curl -V` after every build. + (and exclude ARM64 builds.) + - also say why we skipped `curl -V` if we had to skip. + - fix CMake warnings about unused configuration variables, by adapting + these dynamically for build cases. + - dedupe OpenSSL path into a variable. + - disable `test1451` failing with a warning anyway due to missing python + impacket. (after trying and failing to install impacket) + PowerShell promotes these warnings to errors by PowerShell. We can also + suppress they wholesale if they start causing issues in the future, + like we already to with `autoreconf` and `./configure`. -- tool/var: also error when expansion result starts with NUL + PowerShell is better than MS-DOS batches, so the hope is this makes it + easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't + supported inline by AppVeyor on Windows build machines, but we are okay + to keep it in an external script, so it's also an option. - Expansions whose output starts with NUL were being expanded to the empty - string, and not being recognised as values that contain a NUL byte, and - should error. + 2. CI improvements: - Closes #11694 + - enable tests for a "unity" build job. + - speed-up CI initialization by using shallow clones of the curl repo. + - speed-up CMake MSVC jobs with `TrackFileAccess=false`. + - enable parallelism in `VisualStudioSolution` builds. + - display CMake version before builds. + - always show the CPU in job names. + - tell which jobs are build-only in job names. + - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. + - add `config.log` (autotools) to dumped logs (need to enable manually). -Daniel Stenberg (19 Aug 2023) + 3. Style: -- tests: add 'large-time' as a testable feature + - use single-quotes in YAML like we do in other CI YAML files. + It also allows to drop quoting characters and lighter to write/read. + (keep double quotes for PowerShell strings needing expansion.) - This allows test cases to require this feature to run and to be used in - %if conditions. + Closes #11999 - Large here means larger than 32 bits. Ie does not suffer from y2038. +- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows - Closes #11696 + - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. + Before this patch we set it based it on the presence of `stricmp`, + which correctly enabled it on e.g. Windows, but was inaccurate for + other platforms. -- tests/Makefile: add check-translatable-options.pl to tarball + - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and + LDAPS is not explicitly disabled. This mimics autotools behaviour. + Previously we set it only for Windows LDAP. After this fix, LDAPS is + correctly enabled in default macOS builds. - Used in test 1544 + - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) + to avoid deprecation warnings for LDAP API. - Follow-up to ae806395abc8c + - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. + This doesn't make much sense, but let's do it to sync behaviour with + autotools. -- gen.pl: fix a long version generation mistake + - fix benign typo in variable name. - Too excessive escaping made the parsing not find the correct long names - later and instead add "wrong" links. + Ref: #11964 (effort to sync cmake detections with autotools) - Follow-up to 439ff2052e219 + Closes #12006 - Reported-by: Lukas Tribus - Fixes #11688 - Closes #11689 +- autotools: restore `HAVE_IOCTL_*` detections -- lib: move mimepost data from ->req.p.http to ->state + This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in + 4d73854462f30948acab12984b611e9e33ee41e6 and + c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the + `HAVE_IOCTL` result it generated was unused in the source. But, + I did miss the fact that this had two dependent checks: + `CURL_CHECK_FUNC_IOCTL_FIONBIO`, + `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: + `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. - When the legacy CURLOPT_HTTPPOST option is used, it gets converted into - the modem mimpost struct at first use. This data is (now) kept for the - entire transfer and not only per single HTTP request. This re-enables - rewind in the beginning of the second request instead of in end of the - first, as brought by 1b39731. + Regression from 4d73854462f30948acab12984b611e9e33ee41e6 - The request struct is per-request data only. + Ref: #11964 (effort to sync cmake detections with autotools) - Extend test 650 to verify. + Closes #12008 - Fixes #11680 - Reported-by: yushicheng7788 on github - Closes #11682 +Daniel Stenberg (2 Oct 2023) -Patrick Monnerat (17 Aug 2023) +- RELEASE-PROCEDURE.md: updated coming release dates -- os400: do not check translatable options at build time +- RELEASE-NOTES: synced - Now that there is a test for this, the build time check is not needed - anymore. +Viktor Szakats (1 Oct 2023) - Closes #11650 +- cmake: pre-cache `HAVE_POLL_FINE` on Windows -- test1554: check translatable string options in OS400 wrapper + Windows doesn't support `poll()`, so we can safely skip checking for + fine poll. - This test runs a perl script that checks all string options are properly - translated by the OS400 character code conversion wrapper. It also - verifies these options are listed in alphanumeric order in the wrapper - switch statement. + Closes #12003 - Closes #11650 +- gha: bump actions to latest versions -Daniel Stenberg (17 Aug 2023) + - actions@checkout@v4 (from v3 and v2) -- unit3200: skip testing if function is not present + - fsfe/reuse-action@v2 (from v1) - Fake a successful run since we have no easy mechanism to skip this test - for this advanced condition. + Closes #12000 -- unit2600: fix build warning if built without verbose messages +Stefan Eissing (30 Sep 2023) -- test1608: make it build and get skipped without shuffle DNS support +- h2: testcase and fix for pausing h2 streams -- lib: --disable-bindlocal builds curl without local binding support + - refs #11982 where it was noted that paused transfers may + close successfully without delivering the complete data + - made sample poc into tests/http/client/h2-pausing.c and + added test_02_27 to reproduce -- test1304: build and skip without netrc support + Closes #11989 + Fixes #11982 + Reported-by: Harry Sintonen -- lib: build fixups when built with most things disabled +Viktor Szakats (30 Sep 2023) - Closes #11687 +- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value -- workflows/macos.yml: disable zstd and alt-svc in the http-only build + Before this patch CMake builds accepted any value and it was used at + runtime as-is. This patch make sure that the selected default backend + is also enabled in the build. It also enforces a full lowercase value. - Closes #11683 + This improves reproducibility and brings CMake in sync with autotools + which already worked like described above. -Stefan Eissing (17 Aug 2023) + Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 -- bearssl: handshake fix, provide proper get_select_socks() implementation + Closes #11998 - - bring bearssl handshake times down from +200ms down to other TLS backends - - vtls: improve generic get_select_socks() implementation - - tests: provide Apache with a suitable ssl session cache +- autotools: adjust `CURL_CA_PATH` value to CMake - Closes #11675 + autotools was using the same value as CMake, but with an ending + slash. Delete the ending slash to match configurations. -- tests: TLS session sharing test + Ref: #11964 (effort to sync cmake detections with autotools) - - test TLS session sharing with special test client - - expect failure with wolfSSL - - disable flaky wolfSSL test_02_07b + Closes #11997 - Closes #11675 +- cmake: detect `sys/wait.h` and `netinet/udp.h` -Daniel Stenberg (17 Aug 2023) + Ref: #11964 (effort to sync cmake detections with autotools) -- CURLOPT_*TIMEOUT*: extend and clarify + Closes #11996 - Closes #11686 +Daniel Stenberg (30 Sep 2023) -- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails +- lib: provide and use Curl_hexencode - And document it. Only return out of memory when it actually is a memory - problem. + Generates a lower case ASCII hex output from a binary input. - Pointed-out-by: Jacob Mealey - Closes #11674 + Closes #11990 -Mathew Benson (17 Aug 2023) +- configure: check for the capath by default -- cmake: add GnuTLS option + ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS + L - - Option to use GNUTLS was missing. Hence was not able to use GNUTLS - with ngtcp2 for http3. + cmake: synced - Closes #11685 + Assisted-by: Viktor Szakats + Closes #11987 -Daniel Stenberg (16 Aug 2023) +- wolfssl: ignore errors in CA path -- RELEASE-NOTES: synced + The default wolfSSL_CTX_load_verify_locations() function is quite picky + with the certificates it loads and will for example return error if just + one of the certs has expired. -- http: remove the p_pragma struct field + With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it + behaves more similar to what OpenSSL does by default. - unused since 40e8b4e52 (2008) + Even the set of default certs on my Debian unstable has several expired + ones. - Closes #11681 + Assisted-by: Juliusz Sosinowicz + Assisted-by: Michael Osipov -Jay Satiro (16 Aug 2023) + Closes #11987 -- CURLINFO_CERTINFO.3: better explain curl_certinfo struct +- create-dirs.d: clarify it also uses --output-dirs - Closes https://github.com/curl/curl/pull/11666 + Reported-by: Robert Simpson + Fixes #11991 + Closes #11995 -- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation +Viktor Szakats (30 Sep 2023) - - Remove the out-of-date SSL backend list supported by - CURLOPT_SSL_CTX_FUNCTION. +- appveyor: fix yamlint issues, indent - It makes more sense to just refer to that document instead of having - a separate list that has to be kept in sync. + Also: + - use double quotes in all batch if statements. - Closes https://github.com/curl/curl/pull/11665 + Closes #11994 -- write-out.d: clarify %{time_starttransfer} +- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` - sync it up with CURLINFO_STARTTRANSFER_TIME_T + Based on existing autotools logic. -Daniel Stenberg (15 Aug 2023) + Ref: #11964 (effort to sync cmake detections with autotools) -- transfer: don't set TIMER_STARTTRANSFER on first send + Closes #11981 - The time stamp is for measuring the first *received* byte +- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` - Fixes #11669 - Reported-by: JazJas on github - Closes #11670 + Based on existing autotools logic. -trrui-huawei (15 Aug 2023) + autotools checks for old versions of the allowlisted target OSes and + disables this feature when seeing them. In CMake we assume we're running + on newer systems and enable regardless of OS version. -- quiche: enable quiche to handle timeout events + autotools always runs all 3 probes for non-fast-tracked systems and + enables this feature if any one of them was successful. To save + configuration time, CMake stops at the first successful check. - In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` - interface for the application to invoke upon timer - expiration. Therefore, invoking the `on_timeout` function of the - Connection is crucial to ensure seamless functionality of quiche with - timeout events. + OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD + system. I haven't double-checked if this is correct, but looks odd. - Closes #11654 + Ref: #11964 (effort to sync cmake detections with autotools) -- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s + Closes #11979 - Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. +- cmake: fix `HAVE_WRITABLE_ARGV` detection -Daniel Stenberg (15 Aug 2023) + Move detection before the creation of detection results in + `curl_config.h`. -- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang + Ref: #11964 (effort to sync cmake detections with autotools) - Closes #9580 + Closes #11978 -- imap: add a check for failing strdup() +- appveyor: minor improvements -- imap: remove the only sscanf() call in the IMAP code + - run `curl -V` after builds to see if they run and with what features. + Except for one job where a CRT DLL is missing. And ARM64 which should + fail, but is silently not launched instead. - Avoids the use of a stack buffer. + - copy libcurl DLL next to curl tool and tests binaries in shared mode. + This makes it possible to run the tests. (We don't run tests after + these builds yet.) - Closes #11673 + - list the DLLs and EXEs present after the builds. -- imap: use a dynbuf in imap_atom + - add `DEBUG` variable for CMake builds to allow disabling it, for + testing non-debug builds. (currently enabled for all) - Avoid a calculation + malloc. Build the output in a dynbuf. + - add commented lines that dump CMake configuration logs for debugging + build/auto-detection issues. - Closes #11672 + - add gcc version to jobs where missing. -Marin Hannache (14 Aug 2023) + - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 + to the build mix. -- http: do not require a user name when using CURLAUTH_NEGOTIATE + - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. - In order to get Negotiate (SPNEGO) authentication to work in HTTP you - used to be required to provide a (fake) user name (this concerned both - curl and the lib) because the code wrongly only considered - authentication if there was a user name provided, as in: + - delete a duplicate backslash. - curl -u : --negotiate https://example.com/ + Closes #11976 - This commit leverages the `struct auth` want member to figure out if the - user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of - setting a user name both in curl and the lib. +- configure: replace adhoc domain with `localhost` in tests - Signed-off-by: Marin Hannache - Reported-by: Enrico Scholz - Fixes https://sourceforge.net/p/curl/bugs/440/ - Fixes #1161 - Closes #9047 + Reviewed-by: Daniel Stenberg + Closes #11988 -Viktor Szakats (13 Aug 2023) +- tidy-up: use more example domains -- build: streamline non-UWP wincrypt detections + Also make use of the example TLD: + https://en.wikipedia.org/wiki/.example - - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build - and disable our non-UWP-compatible use the Windows crypto API. This - allows to drop two dynamic feature checks. + Reviewed-by: Daniel Stenberg + Closes #11992 - `WINDOWS_STORE` is true when invoking CMake with - `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. +Dan Fandrich (29 Sep 2023) - Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html +- runtests: display the test status if tests appear hung - - with autotools, drop the separate feature check for `wincrypt.h`. On - one hand this header has been present for long (even Borland C 5.5 had - it from year 2000), on the other we used the check result solely to - enable another check for certain crypto functions. This fails anyway - with the header not present. We save one dynamic feature check at the - configure stage. + It sometimes happens that a test hangs during a test run and never + returns. The test harness will wait indefinitely for the results and on + CI servers the CI job will eventually be killed after an hour or two. + At the end of a test run, if results haven't come in within a couple of + minutes, display the status of all test runners and what tests they're + running to help in debugging the problem. - Reviewed-by: Marcel Raad - Closes #11657 + This feature is really only kick in with parallel testing enabled, which + is fine because without parallel testing it's usually easy to tell what + test has hung. -Nicholas Nethercote (13 Aug 2023) + Closes #11980 -- docs/HYPER.md: update hyper build instructions +- github/labeler: remove workaround for labeler - Nightly Rust and `-Z unstable-options` are not needed. + This was added due to what seemed to be a bug regarding the sync-labels: + config option, but it looks like it wasn't necessary. - The instructions here now match the hyper docs exactly: - https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 - 6f88 + Follow-up to b2b0534e7 - Closes #11662 +Viktor Szakats (29 Sep 2023) -Daniel Stenberg (13 Aug 2023) +- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] -- RELEASE-NOTES: synced +Daniel Stenberg (29 Sep 2023) -- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name +- docs: replace made up domains with example.com - Asssisted-by: Jay Satiro - Closes #11655 + in FAQ and MANUAL.md -- spellcheck: adapt to backslashed minuses + - example.com was made for this purpose. - As the curl.1 has more backslashed minus, the cleanup sed lines xneed to - adapt. + - reduces the risk that one of those domains suddenly start hosting + something nasty and we provide links to them - Adjusted some docs slighly. + Closes #11986 - Follow-up to 439ff2052e +Michael Osipov (29 Sep 2023) - Closes #11663 +- acinclude.m4: Document proper system truststore on FreeBSD -- gen: escape more minus + The default system truststore on FreeBSD has been /etc/ssl/certs for many + years now. It is managed canonically through certctl(8) and contains hashed + symlinks for OpenSSL and other TLS providers. + The previous ones require security/ca_root_nss which might not be installed o + r + will not contain any custom CA certificates. - Detected since it was still hard to search for option names using dashes - in the middle in the man page. + Closes #11985 - Closes #11660 +Daniel Stenberg (29 Sep 2023) -- cookie-jar.d: enphasize that this option is ONLY writing cookies +- FAQ: How do I upgrade curl.exe in Windows? - Reported-by: Dan Jacobson - Tweaked-by: Jay Satiro - Ref: #11642 - Closes #11661 + This is a growing question, better answer it here to get somewhere to + point users to. -Nicholas Nethercote (11 Aug 2023) + Closes #11984 -- docs/HYPER.md: document a workaround for a link error +Viktor Szakats (28 Sep 2023) - Closes #11653 +- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC -Jay Satiro (11 Aug 2023) + `basename` is present in mingw-w64, missing from MSVC. Pre-cache + accordingly to make configure faster. -- schannel: verify hostname independent of verify cert + Notice that `basename` has a bug so we later disable it even with + mingw-w64: + https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li + b/curl_setup.h#L820-L825 - Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off - and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the - hostname in schannel code. + Closes #11974 - This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and - verify hostname". We discussed a fix several years ago in #3285 but it - went stale. +Daniel Stenberg (28 Sep 2023) - Assisted-by: Daniel Stenberg +- cmake: add missing checks - Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html - Reported-by: Martin Galvan + - check for arc4random. To make rand.c use it accordingly. + - check for fcntl + - fix fseek detection + - add SIZEOF_CURL_SOCKET_T + - fix USE_UNIX_SOCKETS + - define HAVE_SNPRINTF to 1 + - check for fnmatch + - check for sched_yield + - remove HAVE_GETPPID duplicate from curl_config.h + - add HAVE_SENDMSG - Ref: https://github.com/curl/curl/pull/3285 + Ref: #11964 - Fixes https://github.com/curl/curl/issues/3284 - Closes https://github.com/curl/curl/pull/10056 + Co-authored-by: Viktor Szakats + Closes #11973 -Daniel Stenberg (11 Aug 2023) +- configure: remove unused checks -- curl_quiche: remove superfluous NULL check + - for sys/uio.h + - for fork + - for connect - 'stream' is always non-NULL at this point + Ref: #11964 - Pointed out by Coverity + Closes #11973 - Closes #11656 +- lib: remove TIME_WITH_SYS_TIME -- curl/urlapi.h: tiny typo + It is not used in any code anywhere. -- github/labeler: make HYPER.md set Hyper and not TLS + Ref: #11964 + Closes #11975 -- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 +- docs: update curl man page references - 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need - to specify version changes for earlier releases in the generated output. + Detected by the manpage-syntax update - This ups the limit from the previous 7.30.0 (Apr 12 2013) + Closes #11963 - This hides roughly 35 "added in" mentions. +- manpage-syntax: verify curl man page references - Closes #11651 + 1. References to curl symbols are now checked that they indeed exist as + man pages. This for \f references as well as the names referenced in the + SEE ALSO section. -Jay Satiro (10 Aug 2023) + Allowlist curl.1 since it is not always built in builds -- bug_report: require reporters to specify curl and os versions + 2. References to curl symbols that lack section now causes warning, since tha + t + will prevent them from getting linked properly - - Change curl version and os sections from single-line input to - multi-line textarea. + 3. Check for "bare" references to curl functions and warn, they should be + references - - Require curl version and os sections to be filled out before report - can be submitted. + Closes #11963 - Closes https://github.com/curl/curl/pull/11636 +- cmake: add check for suseconds_t -Daniel Stenberg (9 Aug 2023) + And fix the HAVE_LONGLONG define -- gen.pl: replace all single quotes with aq + Ref: #11964 + Closes #11977 - - this prevents man from using a unicode sequence for them - - which then allows search to work properly +Viktor Szakats (28 Sep 2023) - Closes #11645 +- tidy-up: whitespace fixes -Viktor Szakats (9 Aug 2023) + Closes #11972 -- cmake: fix to use variable for the curl namespace +- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS - Replace (wrong) literal with a variable to specify the curl - namespace. + With new option `CURL_DISABLE_SRP=ON` to force-disable it. + To match existing option and detection logic in autotools. - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 + Also: + - fix detecting GnuTLS. + We assume `nettle` as a GnuTLS dependency. + - add CMake GnuTLS CI job. + - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) + TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. + - fix compiler warning when building with GnuTLS and disabled TLS-SRP. + - fix comment typos, whitespace. - Reported-by: balikalina on Github - Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 - 20d30#r123923098 - Closes #11629 + Ref: #11964 -- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms + Closes #11967 - 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing - libcurl objects for shared and static targets. +- tool: use our own stderr variable - The above automatically enabled for Windows builds, with an option to - disable with `SHARE_LIB_OBJECT=OFF`. + Earlier this year we changed our own stderr variable to use the standard + name `stderr` (to avoid bugs where someone is using `stderr` instead of + the curl-tool specific variable). This solution needed to override the + standard `stderr` symbol via the preprocessor. This in turn didn't play + well with unity builds and caused curl tool to crash or stay silent due + to an uninitialized stderr. This was a hard to find issue, fixed by + manually breaking out one file from the unity sources. - This patch extend this feature to all platforms as a manual option. - You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects - are built in PIC mode, meaning the static lib will also have PIC code. + To avoid two these two tricks, this patch implements a different + solution: Restore using our own local variable for our stderr output and + leave `stderr` as-is. To avoid using `stderr` by mistake, add a + `checksrc` rule (based on logic we already used in lib for `strerror`) + that detects any `stderr` use in `src` and points to using our own + variable instead: `tool_stderr`. - [EXPERIMENTAL] + Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e + Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db - Closes #11627 + Closes #11958 -- cmake: assume `wldap32` availability on Windows +Loïc Yhuel (28 Sep 2023) - This system library first shipped with Windows ME, available as an extra - install for some older releases (according to [1]). The import library - was present already in old MinGW 3.4.2 (year 2007). +- connect: only start the happy eyeballs timer when needed - Drop the feature check and its associated `HAVE_WLDAP32` variable. + The timeout is only used when there is a second address family, for the + delayed eyeballer. - To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` - CMake option, like before. + Closes #11939 - [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html +Daniel Stenberg (28 Sep 2023) - Reviewed-by: Jay Satiro - Closes #11624 +- tool_operate: free 'gateway' correctly -Daniel Stenberg (9 Aug 2023) + Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. -- page-header: move up a URL paragraph from GLOBBING to URL + Also removed repeated wording in IPFS related error messages. -- variable.d: output the function names table style + Closes #11969 - Also correct the url function name in the header +Stefan Eissing (28 Sep 2023) - Closes #11641 +- lib: move handling of `data->req.writer_stack` into Curl_client_write() -- haproxy-clientip.d: remove backticks + - move definitions from content_encoding.h to sendf.h + - move create/cleanup/add code into sendf.c + - installed content_encoding writers will always be called + on Curl_client_write(CLIENTWRITE_BODY) + - Curl_client_cleanup() frees writers and tempbuffers from + paused transfers, irregardless of protocol - This is not markdown + Closes #11908 - Follow-up to 0a75964d0d94a4 +Loïc Yhuel (28 Sep 2023) - Closes #11639 +- multi: round the timeout up to prevent early wakeups -- RELEASE-NOTES: synced + Curl_timediff rounds down to the millisecond, so curl_multi_perform can + be called too early, then we get a timeout of 0 and call it again. -- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens + The code already handled the case of timeouts which expired less than + 1ms in the future. By rounding up, we make sure we will never ask the + platform to wake up too early. - Reported-by: FC Stegerman - Fixes #11635 - Closes #11637 + Closes #11938 -- cmdline-opts/page-header: reorder, clean up +Daniel Stenberg (28 Sep 2023) - - removed some unnecessary blurb to focus - - moved up the more important URL details - - put "globbing" into its own subtitle and moved down a little - - mention the online man page in the version section +- RELEASE-NOTES: spell out that IPFS is via gateway - Closes #11638 +- RELEASE-NOTES: synced -- c-hyper: adjust the hyper to curlcode conversion +- tool_operate: avoid strlen() -1 on zero length content from file - Closes #11621 + Follow-up to 65b563a96a226649ba12cb1e -- test2306: make it use a persistent connection + Closes #11959 - + enable verbose already from the start +- tool_operate: fix memory mixups - Closes #11621 + Switch to plain getenv() from curl_getenv() to avoid the allocation and + having to keep track of which free() or curl_free() that need to be + used. -eppesuig (8 Aug 2023) + Coverity found issues and a memory leak. -- list-only.d: mention SFTP as supported protocol + Follow-up to 65b563a96a226649ba12cb1e - Closes #11628 + Closes #11959 -Daniel Stenberg (8 Aug 2023) +Viktor Szakats (27 Sep 2023) -- request.d: use .TP for protocol "labels" +- curl-functions.m4: fixup recent bad edits - To render the section nicer in man page. + Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Closes #11630 + Closes #11966 -- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP +Daniel Stenberg (27 Sep 2023) - ... as documented. +- curl-functions.m4: fix include line - Update test 3201 and 3202 accordingly. + This made the getaddrinfo detection fail, but we did not spot it in the + CI because it graciously falled back to using legacy functions instead! - Reported-by: Markus Sommer - Fixes #11619 - Closes #11626 + Follow-up to 96c29900bcec (#11940) -- page-footer: QLOGDIR works with ngtcp2 and quiche + Closes #11965 - It previously said "both" backends which is confusing as we currently - have three... +- inet_ntop: add typecast to silence Coverity - Closes #11631 + CID 1024653: Integer handling issues (SIGN_EXTENSION) -Stefan Eissing (8 Aug 2023) + Suspicious implicit sign extension: "src[i]" with type "unsigned char + const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to + type "int" (32 bits, signed), then sign-extended to type "unsigned long" + (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than + 0x7FFFFFFF, the upper bits of the result will all be 1. -- http3: quiche, handshake optimization, trace cleanup + 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); - - load x509 store after clienthello - - cleanup of tracing + The value will not be greater than 0x7FFFFFFF so this still cannot + happen. - Closes #11618 + Also, switch to ints here instead of longs. The values stored are 16 bit + so at least no need to use 64 bit variables. Also, longs are 32 bit on + some platforms so this logic still needs to work with 32 bits. -Daniel Stenberg (8 Aug 2023) + Closes #11960 -- ngtcp2: remove dead code +- docs: adapt SEE ALSO sections to new requirements - 'result' is always zero (CURLE_OK) at this point + To please manpage-syntax.pl used by test 1173 - Detected by Coverity + Closes #11957 - Closes #11622 +- manpage-syntax.pl: verify SEE ALSO syntax -Viktor Szakats (8 Aug 2023) + - Enforce a single reference per .BR line + - Skip the quotes around the section number for example (3) + - Insist on trailing commas on all lines except the last + - Error on comma on the last SEE ALSO entry -- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` + - List the entries alpha-sorted, not enforced just recommended - OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the - popular forks (yet). Use the macro itself to detect its presence, - replacing the hard-wired fork-specific conditions. + Closes #11957 - This way the feature will enable automatically when forks implement it, - while also shorter and possibly requiring less future maintenance. +- connect: expire the timeout when trying next - Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 + ... so that it gets called again immediately and can continue trying + addresses to connect to. Otherwise it might unnecessarily wait for a + while there. - Reviewed-by: Jay Satiro - Closes #11617 + Fixes #11920 + Reported-by: Loïc Yhuel + Closes #11935 -- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 +- http: remove wrong comment for http_should_fail - LibreSSL 3.4.1 (2021-10-14) added support for - `SSL_CTX_set_ciphersuites`. + Reported-by: Christian Schmitz + Ref: #11936 + Closes #11941 - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt +Dan Fandrich (26 Sep 2023) - Reviewed-by: Jay Satiro - Closes #11616 +- tool_setopt: remove unused function tool_setopt_flags -- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 + This function is identical to tool_setopt_bitmask except that it treats + the argument as unsigned. - LibreSSL 3.5.0 (2022-02-24) added support for - `SSL_CTX_set_keylog_callback`. + Closes #11943 - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt +Viktor Szakats (26 Sep 2023) - Reviewed-by: Jay Satiro - Closes #11615 +- cmake: add feature checks for `memrchr` and `getifaddrs` -- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks + - `HAVE_MEMRCHR` for `memrchr`. + - `HAVE_GETIFADDRS` for `getifaddrs`. + This was present in `lib/curl_config.h.cmake` but missed the detection + logic. - - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is - also not used by curl, but it is by its optional dependency `librtmp`. - Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This - library has been available since the early days of Windows. + To match existing autotools feature checks. - - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present - since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and - other existing logic already assumed this lib being present, so delete - the check and replace the detection variable with `WIN32` and always - add `ws2_32` on Windows. + Closes #11954 - Closes #11612 +- cmake: move global headers to specific checks -Daniel Gustafsson (8 Aug 2023) + Before this patch we added standard headers unconditionally to the + global list of headers used for feature checks. This is unnecessary + and also doesn't help CMake 'Generate' performance. This patch moves + these headers to each feature check where they are actually needed. + Stop using `stddef.h`, as it seems unnecessary. -- crypto: ensure crypto initialization works + I've used autotools' `m4/curl-functions.m4` to figure out these + dependencies. - Make sure that context initialization during hash setup works to avoid - going forward with the risk of a null pointer dereference. + Also delete checking for the C89 standard header `time.h`, that I + missed in the earlier commit. - Reported-by: Philippe Antoine on HackerOne - Assisted-by: Jay Satiro - Assisted-by: Daniel Stenberg + Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Closes #11614 + Closes #11951 -Viktor Szakats (7 Aug 2023) +- src/mkhelp: make generated code pass `checksrc` -- openssl: switch to modern init for LibreSSL 2.7.0+ + Closes #11955 - LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, - `OPENSSL_init_ssl()` function and deprecated the old, manual init - method, as seen in OpenSSL 1.1.0. Switch to the modern method when - available. +- tests: show which curl tool `runtests.pl` is using - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt + To help debugging when there is issue finding or running it. - Reviewed-by: Daniel Stenberg - Closes #11611 - -Daniel Stenberg (7 Aug 2023) - -- gskit: remove - - We remove support for building curl with gskit. - - - This is a niche TLS library, only running on some IBM systems - - no regular curl contributors use this backend - - no CI builds use or verify this backend - - gskit, or the curl adaption for it, lacks many modern TLS features - making it an inferior solution - - build breakages in this code take weeks or more to get detected - - fixing gskit code is mostly done "flying blind" - - This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has - been mentioned on the curl-library mailing list. + Closes #11953 - It could be brought back, this is not a ban. Given proper effort and - will, gskit support is welcome back into the curl TLS backend family. +- CI/azure: make `MAKEFLAGS` global to parallelize all jobs - Closes #11460 + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi + th -j3) -- RELEASE-NOTES: synced + Closes #11952 -Dan Fandrich (7 Aug 2023) +- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 -- THANKS-filter: add a name typo + Also delete an accidental variable reference. -Stefan Eissing (7 Aug 2023) + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 -- http3/ngtcp2: shorten handshake, trace cleanup + Closes #11945 - - shorten handshake timing by delayed x509 store load (OpenSSL) - as we do for HTTP/2 - - cleanup of trace output, align with HTTP/2 output +Daniel Stenberg (26 Sep 2023) - Closes #11609 +- docs: add see also curl_multi_get_handles to some man pages -Daniel Stenberg (7 Aug 2023) + Assisted-by: Jay Satiro -- headers: accept leading whitespaces on first response header + Closes #11942 - This is a bad header fold but since the popular browsers accept this - violation, so does curl now. Unless built with hyper. +Viktor Szakats (26 Sep 2023) - Add test 1473 to verify and adjust test 2306. +- cmake: assume `_fseeki64` and no `fseeko` on Windows - Reported-by: junsik on github - Fixes #11605 - Closes #11607 + `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and + at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. -- include/curl/mprintf.h: add __attribute__ for the prototypes + `fseeko` is not present in any of these. - - if gcc or clang is used - - if __STDC_VERSION__ >= 199901L, which means greater than C90 - - if not using mingw - - if CURL_NO_FMT_CHECKS is not defined + (mingw-w64 1.0 also offers `fseeko64`.) - Closes #11589 + [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 -- tests: fix bad printf format flags in test code + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 -- tests: fix header scan tools for attribute edits in mprintf.h + Closes #11950 -- cf-socket: log successful interface bind +- build: delete checks for C89 standard headers - When the setsockopt SO_BINDTODEVICE operation succeeds, output that in - the verbose output. + Delete checks and guards for standard C89 headers and assume these are + available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, + `stddef.h`, `signal.h`. - Ref: #11599 - Closes #11608 + Some of these we already used unconditionally, some others we only used + for feature checks. -- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i + n CMake) - Ref: #11457 - Closes #11606 + Closes #11940 -- CURLOPT_SSL_VERIFYPEER.3: add two more see also options +Stefan Eissing (26 Sep 2023) - CURLINFO_CAINFO and CURLINFO_CAPATH +- multiif.h: remove Curl_multi_dump declaration - Closes #11603 + Follow-up to d850eea2 which removed the Curl_multi_dump definition. -- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice + Closes https://github.com/curl/curl/pull/11946 - Closes #11007 +Jay Satiro (26 Sep 2023) -Graham Campbell (6 Aug 2023) +- config-win32: define HAVE__FSEEKI64 -- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 + Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that + calls _fseeki64 if it is available. - Closes #11585 + Closes https://github.com/curl/curl/pull/11944 -Daniel Stenberg (6 Aug 2023) +- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER -- TODO: add *5* entries for aws-sigv4 + - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place + even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. - Closes #7559 - Closes #8107 - Closes #8810 - Closes #9717 - Closes #10129 + The behavior is verified by test2048. -- TODO: LDAP Certificate-Based Authentication + Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 + Reported-by: claudiusaiz@users.noreply.github.com - Closes #9641 + Bug: https://github.com/curl/curl/discussions/11910 + Reported-by: Hakan Sunay Halil -Stefan Eissing (6 Aug 2023) + Closes https://github.com/curl/curl/pull/11930 -- http2: cleanup trace messages +Stefan Eissing (26 Sep 2023) - - more compact format with bracketed stream id - - all frames traced in and out +- openssl: improve ssl shutdown handling - Closes #11592 + - If SSL shutdown is not finished then make an additional call to + SSL_read to gather additional tracing. -Daniel Stenberg (6 Aug 2023) + - Fix http2 and h2-proxy filters to forward do_close() calls to the next + filter. -- tests/tftpd+mqttd: make variables static to silence picky warnings + For example h2 and SSL shutdown before and after this change: - Closes #11594 + Before: -- docs/cmdline: remove repeated working for negotiate + ntlm + Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> + ssl_cf_destroy - The extra wording is added automatically by the gen.pl tool + After: - Closes #11597 + Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> + ssl_cf_close -- docs/cmdline: add small "warning" to verbose options + Note that currently the tracing does not show output on the connection + closure handle. Refer to discussion in #11878. - "Note that verbose output of curl activities and network traffic might - contain sensitive data, including user names, credentials or secret data - content. Be aware and be careful when sharing trace logs with others." + Ref: https://github.com/curl/curl/discussions/11878 - Closes #11596 + Closes https://github.com/curl/curl/pull/11858 -- RELEASE-NOTES: synced +Loïc Yhuel (26 Sep 2023) -- pingpong: don't use *bump_headersize +- multi: fix small timeouts - We use that for HTTP(S) only. + Since Curl_timediff rounds down to the millisecond, timeouts which + expire in less than 1ms are considered as outdated and removed from the + list. We can use Curl_timediff_us instead, big timeouts could saturate + but this is not an issue. - Follow-up to 3ee79c1674fd6 + Closes #11937 - Closes #11590 +Viktor Szakats (25 Sep 2023) -- urldata: remove spurious parenthesis to unbreak no-proxy build +- cmake: fix stderr initialization in unity builds - Follow-up to e12b39e13382 + Before this patch, in certain build configurations the curl tool may + not have displayed anything (debug, macOS), or crashed at startup + (debug, Windows). - Closes #11591 + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 + Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db -- easy: don't call Curl_trc_opt() in disabled-verbose builds + Closes #11929 - Follow-up to e12b39e133822c6a0 +- cmake: fix missing `zlib.h` when compiling `libcurltool` - Closes #11588 + Came up while testing debug/testing build for Windows. I'm not sure why + it didn't come up in earlier tests with similar config. + `tool_hugehelp.c` might indeed require `zlib.h` and without linking + `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected + dependency headers to the compiler command. -- http: use %u for printfing int + ``` + [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang + --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 + 6_64 + -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO + DE + -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib + -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li + ne-argument + -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static + -libgcc + -lucrt [...] -O3 -DNDEBUG -municode -MD + -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d + -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x + 64/src/tool_hugehelp.c + .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f + ile not found + 6 | #include + | ^~~~~~~~ + ``` - Follow-up to 3ee79c1674fd6f99e8efca5 + Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 - Closes #11587 + Closes #11927 -Goro FUJI (3 Aug 2023) +- cmake: fix duplicate symbols when linking tests -- vquic: show stringified messages for errno + The linker resolves this automatically in non-unity builds. In unity + builds the linker cannot drop a single object with the duplicates, + resulting in these errors. The root issue is that we started including + certain objects both via both libcurlu and libcurltool libs. - Closes #11584 + Regression from 39e7c22bb459c2e818f079984989a26a09741860 -Stefan Eissing (3 Aug 2023) + Windows errors: + ``` + [ 3%] Linking C executable unit1303.exe + [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul + tibyte.c.obj + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch + ar': + C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve + rt_UTF8_to_wchar' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:44: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT + F8': + C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve + rt_wchar_to_UTF8' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:66: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': + C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 + _open' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:92: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': + C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 + 2_fopen' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:120: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': + [...] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 + aa4ehny5q#L247 -- trace: make tracing available in non-debug builds + macOS errors: + ``` + [ 56%] Linking C executable unit1302 + duplicate symbol '_curlx_sotouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_sitouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_uztosz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + [...] + ``` + with config: + ``` + -DCMAKE_UNITY_BUILD=ON \ + -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ + -DBUILD_SHARED_LIBS=ON \ + -DBUILD_STATIC_LIBS=OFF + ``` - Add --trace-config to curl + Closes #11926 - Add curl_global_trace() to libcurl +- cmake: lib `CURL_STATICLIB` fixes (Windows) - Closes #11421 + - always define `CURL_STATICLIB` when building libcurl for Windows. -Daniel Stenberg (3 Aug 2023) + This disables `__declspec(dllexport)` for exported libcurl symbols. + In normal mode (hide symbols) these exported symbols are specified + via `libcurl.def`. When not hiding symbols, all symbols are exported + by default. -- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" + Regression from 1199308dbc902c52be67fc805c72dd2582520d30 - See also https://github.com/curl/curl/pull/7507 + Fixes #11844 -- TODO: add "WebSocket read callback" + - fix to omit `libcurl.def` when not hiding private symbols. - remove "Upgrade to websockets" as we already have this + Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 - Closes #11402 + - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also + omitting `libcurl.def` in this case, and exporting all symbols + instead. This ensures that a shared curl tool can access all debug + functions which are not normally exported from libcurl DLL. -- test497: verify rejecting too large incoming headers + - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" + target. -- http: return error when receiving too large header set + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 - To avoid abuse. The limit is set to 300 KB for the accumulated size of - all received HTTP headers for a single response. Incomplete research - suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to - 1MB. + - delete duplicate `BUILDING_LIBCURL` definitions. - Closes #11582 + - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. -Stefan Eissing (3 Aug 2023) + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 -- http2: upgrade tests and add fix for non-existing stream + Closes #11914 - - check in h2 filter recv that stream actually exists - and return error if not - - add test for parallel, extreme h2 upgrades that fail if - connections get reused before fully switched - - add h2 upgrade upload test just for completeness +Daniel Stenberg (25 Sep 2023) - Closes #11563 +- RELEASE-NOTES: synced -Viktor Szakats (3 Aug 2023) +Dan Fandrich (25 Sep 2023) -- tests: ensure `libcurl.def` contains all exports +- tests: fix log directory path in IPFS tests - Add `test1279` to verify that `libcurl.def` lists all exported API - functions found in libcurl headers. + Hard-coding the log directory name fails with parallel tests. - Also: + Follow-up to 65b563a96 - - extend test suite XML `stdout` tag with the `loadfile` attribute. + Ref: #8805 - - fix `tests/extern-scan.pl` and `test1135` to include websocket API. +Daniel Stenberg (25 Sep 2023) - - use all headers (sorted) in `test1135` instead of a manual list. +- curl_multi_get_handles: get easy handles from a multi handle - - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. + Closes #11750 - - add `libcurl.def` to the auto-labeler GHA task. +Stefan Eissing (25 Sep 2023) - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 +- http: h1/h2 proxy unification - Closes #11570 + - use shared code for setting up the CONNECT request + when tunneling, used in HTTP/1.x and HTTP/2 proxying + - eliminate use of Curl_buffer_send() and other manipulations + of `data->req` or `data->state.ulbuf` -Daniel Stenberg (2 Aug 2023) + Closes #11808 -- url: change default value for CURLOPT_MAXREDIRS to 30 +Natanael Copa (25 Sep 2023) - It was previously unlimited by default, but that's not a sensible - default. While changing this has a remote risk of breaking an existing - use case, I figure it is more likely to actually save users from loops. +- lib: use wrapper for curl_mime_data fseek callback - Closes #11581 + fseek uses long offset which does not match with curl_off_t. This leads + to undefined behavior when calling the callback and caused failure on + arm 32 bit. -- lib: fix a few *printf() flag mistakes + Use a wrapper to solve this and use fseeko which uses off_t instead of + long. - Reported-by: Gisle Vanem - Ref: #11574 - Closes #11579 + Thanks to the nice people at Libera IRC #musl for helping finding this + out. -Samuel Chiang (2 Aug 2023) + Fixes #11882 + Fixes #11900 + Closes #11918 -- openssl: make aws-lc version support OCSP +- configure: sort AC_CHECK_FUNCS - And bump version in CI + No functional changes. - Closes #11568 +Daniel Stenberg (25 Sep 2023) -Daniel Stenberg (2 Aug 2023) +- warnless: remove unused functions -- tool: make the length argument an int for printf()-.* flags + Previously put there for use with the intel compiler - Closes #11578 + Closes #11932 -- tool_operate: fix memory leak when SSL_CERT_DIR is used +- GHA/linux: run singleuse to detect single-use global functions - Detected by Coverity + Use --unit for configure --enable-debug builds - Follow-up to 29bce9857a12b6cfa726a5 + Closes #11932 - Closes #11577 +- singleuse: add scan for use in other source codes -- tool/var: free memory on OOM + This should reduce false-positive to almost zero. Checks for presence in + unit tests if --unit is specified, which is intended for debug builds + where unit testing is enabled. - Coverity detected this memory leak in OOM situation + Closes #11932 - Follow-up to 2e160c9c652504e +- multi: remove Curl_multi_dump - Closes #11575 + A debug-only function that is basically never used. Removed to ease the + use of the singleuse script to detect non-static functions not used + outside the file where it is defined. -Viktor Szakats (2 Aug 2023) + Closes #11931 -- gha: bump libressl and mbedtls versions +Viktor Szakats (24 Sep 2023) - Closes #11573 +- tests: fix compiler warnings -Jay Satiro (2 Aug 2023) + Seen with llvm 17 on Windows x64. -- schannel: fix user-set legacy algorithms in Windows 10 & 11 + ``` + .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration + for non-static variable 'logdir' [-Wmissing-variable-declarations] + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration + for non-static variable 'loglockfile' [-Wmissing-variable-declarations] + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat + ion for non-static variable 'logdir' [-Wmissing-variable-declarations] + 43 | const char *logdir = "log"; + | ^ + .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl + e is not intended to be used outside of this translation unit + 43 | const char *logdir = "log"; + | ^ + .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator + here [-Wcomma] + 350 | ++d, ++s; + | ^ + .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa + rning + 350 | ++d, ++s; + | ^~~ + | (void)( ) + ``` - - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then - use the SCHANNEL_CRED legacy structure to pass the list to Schannel. + ``` + .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; + | ~ ^ ~~~~~~~~~~~~~ + 1 warning generated. - - If the user set both a legacy algorithm list and a TLS 1.3 cipher list - then abort. + .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison + 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] + 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. - Although MS doesn't document it, Schannel will not negotiate TLS 1.3 - when SCHANNEL_CRED is used. That means setting a legacy algorithm list - limits the user to earlier versions of TLS. + .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > + 2147483647 is always false [-Wtautological-type-limit-compare] + 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + ``` - Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would - ignore legacy algorithms in Windows 10 1809 and later. + Seen on macOS Intel: + ``` + .../curl/tests/server/sws.c:440:64: warning: field precision should have type + 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] + msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" + , + ~~^~ + 1 warning generated. + ``` - Reported-by: zhihaoy@users.noreply.github.com + Closes #11925 - Fixes https://github.com/curl/curl/pull/10741 - Closes https://github.com/curl/curl/pull/10746 +Jay Satiro (24 Sep 2023) -Daniel Stenberg (2 Aug 2023) +- url: fix netrc info message -- variable.d: setting a variable again overwrites it + - Fix netrc info message to use the generic ".netrc" filename if the + user did not specify a netrc location. - Reported-by: Niall McGee - Bug: https://twitter.com/niallmcgee/status/1686523075423322113 - Closes #11571 + - Update --netrc doc to add that recent versions of curl on Windows + prefer .netrc over _netrc. -Jay Satiro (2 Aug 2023) + Before: + * Couldn't find host google.com in the (nil) file; using defaults -- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting - - - Re-wrap CURLSSLOPT_ALLOW_BEAST description. + After: + * Couldn't find host google.com in the .netrc file; using defaults -Daniel Stenberg (2 Aug 2023) + Closes https://github.com/curl/curl/pull/11904 -- RELEASE-NOTES: synced +Dan Fandrich (23 Sep 2023) -- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set +- wolfssh: do cleanup in Curl_ssh_cleanup - Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not - used, thus unnecessarily asking for addresses that will not be used. + Closes: #11921 - Reported-by: Joseph Tharayil - Fixes #11564 - Closes #11565 +Daniel Stenberg (24 Sep 2023) -- docs: link to the website versions instead of markdowns +- tool_listhelp: regenerated - ... to make the links work when the markdown is converted to webpages on - https://curl.se + Polished the --ipfs-gateway description - Reported-by: Maurício Meneghini Fauth - Fixes https://github.com/curl/curl-www/issues/272 - Closes #11569 + Fixed the --trace-config description -Viktor Szakats (1 Aug 2023) + The script also fixed some other small mistakes -- cmake: cache more config and delete unused ones + Closes #11923 - - cache more Windows config results for faster initialization. +Viktor Szakats (23 Sep 2023) - - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. +- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) - - delete dead references to `sys/utsname.h`. + Also fix to export all symbols in Windows debug builds, making + `-debug-dyn` builds work with `-DCURL_STATICLIB` set. - Closes #11551 + Ref: https://github.com/curl/curl/pull/11914 (same for CMake) -- egd: delete feature detection and related source code + Closes #11924 - EGD is Entropy Gathering Daemon, a socket-based entropy source supported - by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it - a while ago. +Daniel Stenberg (23 Sep 2023) - Its detection in CMake was broken all along because OpenSSL libs were - not linked at the point of feature check. +- quic: set ciphers/curves the same way regular TLS does - Delete detection from both cmake and autotools, along with the related - source snippet, and the `--with-egd-socket=` `./configure` option. + for OpenSSL/BoringSSL - Closes #11556 + Fixes #11796 + Reported-by: Karthikdasari0423 on github + Assisted-by: Jay Satiro + Closes #11836 -Stefan Eissing (1 Aug 2023) +- test457: verify --max-filesize with chunked encoding -- tests: fix h3 server check and parallel instances +- lib: let the max filesize option stop too big transfers too - - fix check for availability of nghttpx server - - add `tcp` frontend config for same port as quic, as - without this, port 3000 is bound which clashes for parallel - testing + Previously it would only stop them from getting started if the size is + known to be too big then. - Closes #11553 + Update the libcurl and curl docs accordingly. -Daniel Stenberg (1 Aug 2023) + Fixes #11810 + Reported-by: Elliot Killick + Assisted-by: Jay Satiro + Closes #11820 -- docs/cmdline-opts: spellfixes, typos and polish +Viktor Szakats (23 Sep 2023) - To make them accepted by the spell checker +- mingw: delete support for legacy mingw.org toolchain - Closes #11562 + Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: + https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ + Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. + It supported the x86 CPU only and used a old Windows API header and + implib set, often causing issues. It also misses most modern Windows + features, offering old versions of both binutils and gcc (no llvm/clang + support). It was last updated 2 years ago. -- CI/spellcheck: build curl.1 and spellcheck it + curl now relies on toolchains based on the mingw-w64 project: + https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ + https://www.msys2.org/ https://github.com/msys2/msys2 + https://github.com/mstorsjo/llvm-mingw + (Also available via Linux and macOS package managers.) - Added acceptable words + Closes #11625 - Closes #11562 +Mark Gaiser (23 Sep 2023) -Alexander Jaeger (1 Aug 2023) +- curl: add support for the IPFS protocols: -- misc: fix various typos + - ipfs:// + - ipns:// - Closes #11561 + This allows you tu use ipfs in curl like: + curl ipfs:// + and + curl ipns:// -Daniel Stenberg (1 Aug 2023) + For more information consult the readme at: + https://curl.se/docs/ipfs.html -- http2: avoid too early connection re-use/multiplexing + Closes #8805 - HTTP/1 connections that are upgraded to HTTP/2 should not be picked up - for reuse and multiplexing by other handles until the 101 switching - process is completed. +Daniel Stenberg (23 Sep 2023) - Lots-of-debgging-by: Stefan Eissing - Reported-by: Richard W.M. Jones - Bug: https://curl.se/mail/lib-2023-07/0045.html - Closes #11557 +- bufq: remove Curl_bufq_skip_and_shift (unused) -- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" + Closes #11915 - This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. +- scripts/singleuse.pl: add curl_global_trace - It's a user error for supplying incomplete information to the build system. +Viktor Szakats (22 Sep 2023) - Reported-by: Ryan Schmidt - Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 +- cmake: fix unity symbol collisions in h2 builds -Viktor Szakats (1 Aug 2023) + Regression from 331b89a319d0067fa1e6441719307cfef9c7960f -- cmake: add support for single libcurl compilation pass + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #11912 - Before this patch CMake builds used two separate compilation passes to - build the shared and static libcurl respectively. This patch allows to - reduce that to a single pass if the target platform and build settings - allow it. +Daniel Stenberg (22 Sep 2023) - This reduces CMake build times when building both static and shared - libcurl at the same time, making these dual builds an almost zero-cost - option. +- RELEASE-NOTES: synced - Enable this feature for Windows builds, where the difference between the - two passes was the use of `__declspec(dllexport)` attribute for exported - API functions for the shared builds. This patch replaces this method - with the use of `libcurl.def` at DLL link time. +Dan Fandrich (21 Sep 2023) - Also update `Makefile.mk` to use `libcurl.def` to export libcurl API - symbols on Windows. This simplifies (or fixes) this build method (e.g. - in curl-for-win, which generated a `libcurl.def` from `.h` files using - an elaborate set of transformations). +- github/labeler: improve the match patterns - `libcurl.def` has the maintenance cost of keeping the list of public - libcurl API symbols up-to-date. This list seldom changes, so the cost - is low. + This includes new rules for setting the appleOS and logging labels and + matches on some example files. Also, enable dot mode for wildcard + matches in the .github directory. - Closes #11546 +Daniel Stenberg (21 Sep 2023) -- cmake: detect `SSL_set0_wbio` in OpenSSL +- upload-file.d: describe the file name slash/backslash handling - Present in OpenSSL 1.1.0 and BoringSSL. - Missing from LibreSSL 3.8.0. + Closes #11911 - Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 +Jakub Jelen (21 Sep 2023) - While here, also fix `RAND_egd()` detection which was broken, likely all - along. This feature is probably broken with CMake builds and also - requires a sufficiently obsolete OpenSSL version, so this part of the - update was not tested. +- libssh: cap SFTP packet size sent - Closes #11555 + Due to libssh limitations -- cmake: fixup H2 duplicate symbols for unity builds + Signed-off-by: Jakub Jelen - Closes #11550 + Closes #11804 -Pablo Busse (1 Aug 2023) +Daniel Stenberg (21 Sep 2023) -- openssl: Support async cert verify callback +- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 - - Update the OpenSSL connect state machine to handle - SSL_ERROR_WANT_RETRY_VERIFY. + Closes #11905 - This allows libcurl users that are using custom certificate validation - to suspend processing while waiting for external I/O during certificate - validation. +- mailmap: unify Michael Osipov under a single email - Closes https://github.com/curl/curl/pull/11499 +Ted Lyngmo (21 Sep 2023) -Jay Satiro (1 Aug 2023) +- docs: use CURLSSLBACKEND_NONE -- tool_cb_wrt: fix invalid unicode for windows console + [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in + documentation and examples. - - Suppress an incomplete UTF-8 sequence at the end of the buffer. + Signed-off-by: Ted Lyngmo - - Attempt to reconstruct incomplete UTF-8 sequence from prior call(s) - in current call. + Closes #11909 - Prior to this change, in Windows console UTF-8 sequences split between - two or more calls to the write callback would cause invalid "replacement - characters" U+FFFD to be printed instead of the actual Unicode - character. This is because in Windows only UTF-16 encoded characters are - printed to the console, therefore we convert the UTF-8 contents to - UTF-16, which cannot be done with partial UTF-8 sequences. +Dan Fandrich (21 Sep 2023) - Reported-by: Maksim Arhipov +- github/labeler: give the sync-labels config item a default value - Fixes https://github.com/curl/curl/issues/9841 - Closes https://github.com/curl/curl/pull/10890 + This shouldn't be necessary and is likely a bug with this beta version + of the labeller. -Daniel Stenberg (1 Aug 2023) + Also, fix the negative matches for the documentation label. -- sectransp: prevent CFRelease() of NULL + Follow-up to dd12b452a + Closes #11907 - When SecCertificateCopyCommonName() returns NULL, the common_name - pointer remains set to NULL which apparently when calling CFRelease() on - (sometimes?) crashes. +- github/labeler: fix up more the labeler config format - Reported-by: Guillaume Algis - Fixes #9194 - Closes #11554 + The new version didn't like the workaround we had for a bug in the + previous labeler version, and it should no longer be needed. -Jay Satiro (1 Aug 2023) + Follow-up to dd12b452a + Closes #11906 -- vtls: clarify "ALPN: offers" message +- github/labeler: fix indenting to try to appease labeller - Before: - * ALPN: offers h2,http/1.1 + Follow-up to dd12b452a - After: - * ALPN: curl offers h2,http/1.1 +Jay Satiro (21 Sep 2023) - Bug: https://curl.se/mail/lib-2023-07/0041.html - Reported-by: Richard W.M. Jones - Closes #11544 +- libssh2: fix error message on failed pubkey-from-file -Daniel Stenberg (1 Aug 2023) + - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error + message "SSH public key authentication failed: Reason unknown (-1)". -- urlapi: make sure zoneid is also duplicated in curl_url_dup + When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a + generic error and therefore doesn't set an error message. AFAICT that is + not documented behavior. - Add several curl_url_dup() tests to the general lib1560 test. + Prior to this change libcurl retrieved the last set error message which + would be from a previous function failing. That resulted in misleading + auth failed error messages in verbose mode. - Reported-by: Rutger Broekhoff - Bug: https://curl.se/mail/lib-2023-07/0047.html - Closes #11549 + Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 + Reported-by: consulion@users.noreply.github.com -Sergey (1 Aug 2023) + Closes https://github.com/curl/curl/pull/11881 -- urlapi: fix heap buffer overflow +Stefan Eissing (21 Sep 2023) - `u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-ter - minator. +- pytest: exclude test_03_goaway in CI runs due to timing dependency - ``` - ==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 a - t pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4 - READ of size 78 at 0x04d48c75 thread T0 - #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\c - ompiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840 - #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_p - arty\curl\lib\strdup.c:97 - #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_part - y\curl\lib\urlapi.c:1297 - #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\clien - t\third_party\curl\lib\urlapi.c:1342 - #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_ - party\curl\lib\urlapi.c:1790 - #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client - \third_party\curl\lib\url.c:1768 - #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_p - arty\curl\lib\url.c:3403 - #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_ - party\curl\lib\url.c:3888 - #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\thi - rd_party\curl\lib\multi.c:1982 - #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\ - third_party\curl\lib\multi.c:2756 - ``` + Closes #11860 - Closes #11560 +- lib: disambiguate Curl_client_write flag semantics -Daniel Stenberg (31 Jul 2023) + - use CLIENTWRITE_BODY *only* when data is actually body data + - add CLIENTWRITE_INFO for meta data that is *not* a HEADER + - debug assertions that BODY/INFO/HEADER is not used mixed + - move `data->set.include_header` check into Curl_client_write + so protocol handlers no longer have to care + - add special in FTP for `data->set.include_header` for historic, + backward compatible reasons + - move unpausing of client writes from easy.c to sendf.c, so that + code is in one place and can forward flags correctly -- curl: make %output{} in -w specify a file to write to + Closes #11885 - It can be used multiple times. Use %output{>>name} to append. +Patrick Monnerat (21 Sep 2023) - Add docs. Test 990 and 991 verify. +- tftpd: always use curl's own tftp.h - Idea: #11400 - Suggested-by: ed0d2b2ce19451f2 - Closes #11416 + Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects + and reports a stringop-overread warning: + + tftpd.c: In function ‘write_behind.isra’: + tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f + rom a region of size 0 [-Wstringop-overread] + 485 | return write(test->ofile, writebuf, count); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + In file included from tftpd.c:71: + /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 + 58 | char tu_data[0]; /* data or error stri + ng */ + | ^~~~~~~ + + This occurs because writebuf points to this field and the latter + cannot be considered as being of dynamic length because it is not + the last field in the structure. Thus it is bound to its declared + size. + + This commit always uses curl's own version of tftp.h where the + target field is last in its structure, effectively avoiding the + warning. + + As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for + arpa/tftp.h are removed. + + Closes #11897 + +Dan Fandrich (20 Sep 2023) + +- test1474: make precheck more robust on non-Solaris systems + + If uname -r returns something odd, perl could return an error code and + the test would be erroneously skipped. The qx// syntax avoid this. + + Followup to 08f9b2148 + +- github/labeler: switch to the 5 beta version + + This version adds an important feature that will allow more PRs to be + labelled. Rather than being limited to labeling PRs with files that + match a single glob, it can now label them if multiple changed files + match any one of a number of globs. + +Daniel Stenberg (20 Sep 2023) + +- lib: enable hmac for digest as well + + Previously a build that disabled NTLM and aws-sigv4 would fail to build + since the hmac was disabled, but it is also needed for digest auth. + + Follow-up to e92edfbef64448ef + + Fixes #11890 + Reported-by: Aleksander Mazur + Closes #11896 + +- idn: if idn2_check_version returns NULL, return error + + ... this avoids a NULL dereference for this unusual case. + + Reported-by: s0urc3_ on hackerone + Closes #11898 + +- http: fix CURL_DISABLE_BEARER_AUTH breakage + + When bearer auth was disabled, the if/else logic got wrong and caused + problems. + + Follow-up to e92edfbef64448ef461 + Fixes #11892 + Reported-by: Aleksander Mazur + Closes #11895 + +Michael Osipov (20 Sep 2023) + +- wolfssl: allow capath with CURLOPT_CAINFO_BLOB + + Remain consistent with OpenSSL. While CAfile is nulled as documented + with CURLOPT_CAINFO_BLOB, CApath remains intact. + + Closes #11886 + +- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c + + Closes #11886 + +Dan Fandrich (19 Sep 2023) + +- test1474: disable test on NetBSD, OpenBSD and Solaris 10 + + These kernels only send a fraction of the requested amount of the first + large block, invalidating the assumptions of the test and causing it to + fail. + + Assisted-by: Christian Weisgerber + Ref: https://curl.se/mail/lib-2023-09/0021.html + Closes #11888 + +Ryan Schmidt (20 Sep 2023) + +- cmake, configure: also link with CoreServices + + When linking with CoreFoundation, also link with CoreServices which is + apparently required to avoid an NSInvalidArgumentException in software + linking with libcurl on macOS Sonoma 14 and later. + + Fixes #11893 + Closes #11894 + +Marc Hoersken (19 Sep 2023) + +- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket + + These dependencies are now already included in the Docker image. + + Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 + b41d15606e97f38cf312c1ce56 + + Closes #11889 + +Daniel Stenberg (19 Sep 2023) + +- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files + + Ref: #11883 + Reported-by: Michael Osipov + Closes #11884 - RELEASE-NOTES: synced -- tool: add "variable" support +- test3103: CURLOPT_COOKIELIST test - Add support for command line variables. Set variables with --variable - name=content or --variable name@file (where "file" can be stdin if set - to a single dash (-)). +- cookie: set ->running in cookie_init even if data is NULL - Variable content is expanded in option parameters using "{{name}}" - (without the quotes) if the option name is prefixed with - "--expand-". This gets the contents of the variable "name" inserted, or - a blank if the name does not exist as a variable. Insert "{{" verbatim - in the string by prefixing it with a backslash, like "\\{{". + This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) - Import an environment variable with --variable %name. It makes curl exit - with an error if the environment variable is not set. It can also rather - get a default value if the variable does not exist, using =content or - @file like shown above. + Test 3103 verifies. - Example: get the USER environment variable into the URL: + Fixes #11875 + Reported-by: wangp on github + Closes #11876 - --variable %USER - --expand-url = "https://example.com/api/{{USER}}/method" +- test498: total header size for all redirects is larger than accepted - When expanding variables, curl supports a set of functions that can make - the variable contents more convenient to use. It can trim leading and - trailing white space with "trim", output the contents as a JSON quoted - string with "json", URL encode it with "url" and base 64 encode it with - "b64". To apply functions to a variable expansion, add them colon - separated to the right side of the variable. They are then performed in - a left to right order. +- http: use per-request counter to check too large headers - Example: get the contents of a file called $HOME/.secret into a variable - called "fix". Make sure that the content is trimmed and percent-encoded - sent as POST data: + Not the counter that accumulates all headers over all redirects. - --variable %HOME=/home/default - --expand-variable fix@{{HOME}}/.secret - --expand-data "{{fix:trim:url}}" - https://example.com/ + Follow-up to 3ee79c1674fd6 - Documented. Many new test cases. + Do a second check for 20 times the limit for the accumulated size for + all headers. + + Fixes #11871 + Reported-by: Joshix-1 on github + Closes #11872 + +Jay Satiro (18 Sep 2023) + +- THANKS: add Eric Murphy + + He reported #11850 (quiche build error) but I forgot to add a + 'reported-by' entry in the fix 267e14f1. + +Daniel Stenberg (18 Sep 2023) + +- h2-proxy: remove left-over mistake in drain_tunnel() + + Left-over from 331b89a319 + + Reported-by: 南宫雪珊 + + Closes https://github.com/curl/curl/pull/11877 + +vvb2060 (18 Sep 2023) + +- lib: failf/infof compiler warnings + + Closes #11874 + +Daniel Stenberg (17 Sep 2023) + +- rand: fix 'alnum': array is too small to include a terminating null character + + It was that small on purpose, but this change now adds the null byte to + avoid the error. + + Follow-up to 3aa3cc9b052353b1 + + Reported-by: Dan Fandrich + Ref: #11838 + Closes #11870 + +Mathias Fuchs (16 Sep 2023) + +- cmake: fix the help text to the static build option in CMakeLists.txt + + Closes #11843 + +John Haugabook (16 Sep 2023) + +- MANUAL.md: change domain to example.com + + Closes #11866 + +Daniel Stenberg (16 Sep 2023) + +- doh: inherit DEBUGFUNCTION/DATA + + When creating new transfers for doing DoH, they now inherit the debug + settings from the initiating transfer, so that the application can + redirect and handle the verbose output correctly even for the DoH + transfers. + + Reported-by: calvin2021y on github + Fixes #11864 + Closes #11869 + +Dan Fandrich (16 Sep 2023) + +- http_aws_sigv4: fix sorting with empty parts + + When comparing with an empty part, the non-empty one is always + considered greater-than. Previously, the two would be considered equal + which would randomly place empty parts amongst non-empty ones. This + showed as a test 439 failure on Solaris as it uses a different + implementation of qsort() that compares parts differently. + + Fixes #11855 + Closes #11868 + +- CI: ignore the "flaky" and "timing-dependent" test results + + CI builds will now run these tests, but will ignore the results if they + fail. The relevant tests are ones that are sensitive to timing or + have edge conditions that make them more likely to fail on CI servers, + which are often heavily overloaded and slow. + + This change only adds two additional tests to be ignored, since the + others already had the flaky keyword. + + Closes #11865 + +- runtests: eliminate a warning on old perl versions + + The warning "Use of implicit split to @_ is deprecated" showed between + perl versions about 5.8 through 5.11. + +- tests: log the test result code after each libtest + + This makes it easier to determine the test status. Also, capitalize + FAILURE and ABORT messages in log lines to make them easier to spot. + +Harry Sintonen (16 Sep 2023) + +- misc: better random strings + + Generate alphanumerical random strings. + + Prior this change curl used to create random hex strings. This was + mostly okay, but having alphanumerical random strings is better: The + strings have more entropy in the same space. + + The MIME multipart boundary used to be mere 64-bits of randomness due + to being 16 hex chars. With these changes the boundary is 22 + alphanumerical chars, or little over 130 bits of randomness. + + Closes #11838 + +Daniel Stenberg (15 Sep 2023) + +- cookie: reduce variable scope, add const + +- cookie: do not store the expire or max-age strings + + Convert it to an expire time at once and save memory. + + Closes #11862 + +- cookie: remove unnecessary struct fields + + Plus: reduce the hash table size from 256 to 63. It seems unlikely to + make much of a speed difference for most use cases but saves 1.5KB of + data per instance. + + Closes #11862 + +- RELEASE-NOTES: synced + + Bumped to 8.4.0, the next presumed version + +Dan Fandrich (14 Sep 2023) + +- test2600: remove special case handling for USE_ALARM_TIMEOUT + + This was originally added to handle platforms that supported only 1 + second granularity in connect timeouts, but after some recent changes + the test currently permafails on several Windows platforms. + + The need for this special-case was removed in commit 8627416, which + increased the connect timeout in all cases to well above 1 second. + + Fixes #11767 + Closes #11849 + +Daniel Stenberg (14 Sep 2023) + +- SECURITY-PROCESS.md. call it vulnerability disclosure policy + + SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md + + This a name commonly used for a document like this. This name helps + users find it. + + Closes #11852 + +Junho Choi (14 Sep 2023) + +- quiche: fix build error with --with-ca-fallback + + - Fix build error when curl is built with --with-quiche + and --with-ca-fallback. + + - Add --with-ca-fallback to the quiche CI job. + + Fixes https://github.com/curl/curl/issues/11850 + Closes https://github.com/curl/curl/pull/11847 + +Jay Satiro (14 Sep 2023) + +- escape: replace Curl_isunreserved with ISUNRESERVED + + - Use the ALLCAPS version of the macro so that it is clear a macro is + being called that evaluates the variable multiple times. + + - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates + a variable multiple times. + + This is a follow-up to 291d225a which changed Curl_isunreserved into an + alias macro for ISUNRESERVED. The problem is the former is not easily + identified as a macro by the caller, which could lead to a bug. + + For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but + Curl_isunreserved(*foo++) is not even though they both are the same. + + Closes https://github.com/curl/curl/pull/11846 + +Dan Fandrich (13 Sep 2023) + +- tests: increase the default server logs lock timeout + + This timeout is used to wait for the server to finish writing its logs + before checking them against the expected values. An overloaded machine + could take more than the two seconds previously allocated, so increase + the timeout to 5 seconds. + + Ref: #11328 + Closes #11834 + +- tests: increase TEST_HANG_TIMEOUT in two tests + + These tests had a 5 second timeout compared to 60 seconds for all other + tests. Make these consistent with the others for more reliability on + heavily-loaded machines. + + Ref: #11328 + +- test1056: disable on Windows + + This test relies on the IPv6 scope field being ignored when connecting to + ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit + dodgy, but it works on all our test platforms except Windows. This + test was disabled manually on all Windows CI builds already, so instead + add an incompatible feature and precheck so it's skipped on Windows + everywhere automatically. + +- test587: add a slight delay after test + + This test is designed to connect to the server, then immediately send a + few bytes and disconnect. In some situations, such as on a loaded + server, this doesn't give the server enough time to write its lock file + before its existence is checked. The test harness then fails to find the + server's input log file (because it hasn't been written yet) and fails + the test. By adding a short delay after the test, the HTTP server has + enough time to write its lock file which gives itself more time to write + its remaining files. + + Ref: #11328 + +- tests: stop overriding the lock timeout + + These tests reduce the server lock wait timeout which can increase + flakiness on loaded machines. Since this is merely an optimization, + eliminate them in favour of reliability. + + Ref: #11328 + +- tests: add some --expect100-timeout to reduce timing dependencies + + These tests can fail when the test machine is so slow that the test HTTP + server didn't get a chance to complete before the client's one second + 100-continue timeout triggered. Increase that 1 second to 999 seconds so + this situation doesn't happen. + + Ref: #11328 + +- test661: return from test early in case of curl error + +- tests: add the timing-dependent keyword on several tests + + These are ones likely to fail on heavily-loaded machines that alter the + normal test timing. Most of these tests already had the flaky keyword + since this condition makes them more likely to fail on CI. + +- test1592: greatly increase the maximum test timeout + + It was too short to be reliable on heavily loaded CI machines, and + as a fail-safe only, it didn't need to be short. + + Ref: #11328 + +- test: minor test cleanups + + Remove an obsolete block of code in tests 2032 & 576. + Add a comment in test 1474. + +- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts + + This gives more of a margin for error when running on overloaded CI + servers. + + Ref: #11328 + +- tests: improve SLOWDOWN test reliability by reducing sent data + + These tests are run in SLOWDOWN mode which adds a 10 msec delay after + each character output, which means it takes at least 1.6 seconds (and + 320 kernel calls) just to get through the long welcome banner. On an + overloaded system, this can end up taking much more than 1.6 seconds, + and even more than the 7 or 16 second curl timeout that the tests rely + on, causing them to fail. Reducing the size of the welcome banner drops + the total number of characters sent before the transfer starts by more + than half, which reduces the opportunity for test-breaking slowdowns by + the same amount. + + Ref: #11328 + +- test650: fix an end tag typo + +Jay Satiro (13 Sep 2023) + +- tool_cb_wrt: fix debug assertion + + - Fix off-by-one out-of-bounds array index in Windows debug assertion. + + Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 + Reported-by: Gisle Vanem + +Daniel Stenberg (13 Sep 2023) + +- ctype: add ISUNRESERVED() + + ... and make Curl_isunreserved() use that macro instead of providing a + separate funtion for the purpose. + + Closes #11840 + +Version 8.3.0 (13 Sep 2023) + +Daniel Stenberg (13 Sep 2023) + +- RELEASE-NOTES: syn ced + + curl 8.3.0 release - Co-brainstormed-by: Emanuele Torre - Assisted-by: Jat Satiro - Closes #11346 +- THANKS: contributors from 8.3.0 -- KNOWN_BUGS: cygwin: make install installs curl-config.1 twice +Thorsten Klein (12 Sep 2023) - Closes #8839 +- cmake: set SIZEOF_LONG_LONG in curl_config.h -- KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14 + in order to support 32bit builds regarding wolfssl CTC_SETTINGS - Closes #11215 + Closes #11839 -- KNOWN_BUGS: cmake outputs: no version information available +Jay Satiro (12 Sep 2023) - Closes #11158 +- curl_ngtcp2: fix error message -- KNOWN_BUGS: APOP authentication fails on POP3 +- http_aws_sigv4: handle no-value user header entries - Closes #10073 + - Handle user headers in format 'name:' and 'name;' with no value. -- KNOWN_BUGS: hyper is slow + The former is used when the user wants to remove an internal libcurl + header and the latter is used when the user actually wants to send a + no-value header in the format 'name:' (note the semi-colon is converted + by libcurl to a colon). - Closes #11203 + Prior to this change the AWS header import code did not special case + either of those and the generated AWS SignedHeaders would be incorrect. -Patrick Monnerat (31 Jul 2023) + Reported-by: apparentorder@users.noreply.github.com -- configure, cmake, lib: more form api deprecation + Ref: https://curl.se/docs/manpage.html#-H - Introduce a --enable-form-api configure option to control its inclusion - in builds. The condition name defined for it is CURL_DISABLE_FORM_API. + Fixes https://github.com/curl/curl/issues/11664 + Closes https://github.com/curl/curl/pull/11668 - Form api code is dependent of MIME: configure and CMake handle this - dependency automatically: CMake by making it a dependent option - explicitly, configure by inheriting the MIME value by default and - rejecting explicit incompatible values. +Dan Fandrich (11 Sep 2023) - "form-api" is now a new hidden test feature. +- CI: run pytest with the -v option - Update libcurl modules to respect this option and adjust tests - accordingly. + This lists of the test cases being run so it can be tracked over time. - Closes #9621 + Closes #11824 -Daniel Stenberg (31 Jul 2023) +Daniel Stenberg (11 Sep 2023) -- mailmap: add Derzsi Dániel +- HTTP3: the msquic backend is not functional -Derzsi Dániel (31 Jul 2023) + I ask that we do not submit bugs for this backend just yet as we know it + does not fully work. -- wolfssl: support loading system CA certificates + Closes #11831 + Closes #11819 - Closes #11452 +- aws_sigv4: the query canon code miscounted URL encoded input -Viktor Szakats (30 Jul 2023) + Added some extra ampersands to test 439 to verify "blank" query parts -- nss: delete more NSS references + Follow-up to fc76a24c53b08cdf - Fix the distcheck CI failure and delete more NSS references. + Closes #11829 - Follow-up to 7c8bae0d9c9b2dfeeb008b9a316117d7b9675175 +vvb2060 (11 Sep 2023) - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Closes #11548 +- quic: don't set SNI if hostname is an IP address -Daniel Stenberg (29 Jul 2023) + We already do this for TLS connections. -- nss: remove support for this TLS library + RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in + "HostName". - Closes #11459 + Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 -Ryan Schmidt (29 Jul 2023) + Fixes https://github.com/curl/curl/issues/11827 + Closes https://github.com/curl/curl/pull/11828 -- macOS: fix target detection more +Daniel Stenberg (10 Sep 2023) - Now SCDynamicStoreCopyProxies is called (and the required frameworks are - linked in) on all versions of macOS and only on macOS. Fixes crash due - to undefined symbol when built with the macOS 10.11 SDK or earlier. +- RELEASE-NOTES: synced - CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and - is now only defined when SCDynamicStoreCopyProxies will actually be - called. Previously, it was defined when ENABLE_IPV6 was not defined but - SCDynamicStoreCopyProxies is not called in that case. +Benoit Pierre (10 Sep 2023) - TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only - when dynamic targets are enabled. TARGET_OS_MAC is always defined but - means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS. - TARGET_OS_IPHONE means any Darwin OS other than macOS. +- configure: fix `HAVE_TIME_T_UNSIGNED` check - Follow-up to c73b2f82 + The syntax was incorrect (need a proper main body), and the test + condition was wrong (resulting in a signed `time_t` detected as + unsigned). - Fixes #11502 - Closes #11516 + Closes #11825 -Daniel Stenberg (29 Jul 2023) +Daniel Stenberg (9 Sep 2023) -- tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR +- THANKS-filter: pszlazak on github - ... used at once. +pszlazak (9 Sep 2023) - Reported-by: Gabriel Corona - Fixes #11325 - Closes #11531 +- include.d: explain headers not printed with --fail before 7.75.0 -Thomas M. DuBuisson (29 Jul 2023) + Prior to 7.75.0 response headers were not printed if -f/--fail was used + and an error was reported by server. This was fixed in ab525c0 + (precedes 7.75.0). -- CI: remove Lift's configuration + Closes #11822 - The Lift tool is being retired. Their site reads: +Daniel Stenberg (8 Sep 2023) - "Sonatype Lift will be retiring on Sep 12, 2023, with its analysis - stopping on Aug 12, 2023." +- http_aws_sigv4: skip the op if the query pair is zero bytes - Closes #11541 + Follow-up to fc76a24c53b08cdf -Nathan Moinvaziri (29 Jul 2023) + Spotted by OSS-Fuzz -- Revert "schannel: reverse the order of certinfo insertions" + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 + Closes #11823 - This reverts commit 8986df802db9b5338d9d50a54232ebae4dbcf6dd. +- cmdline-docs: use present tense, not future - Windows does not guarantee a particular certificate ordering, even - though TLS may have its own ordering/relationship guarantees. Recent - versions of Windows 11 reversed the ordering of ceritifcates returned by - CertEnumCertificatesInStore, therefore this commit no longer works as - initially intended. libcurl makes no guarantees about certificate - ordering if the operating system can't. + + some smaller cleanups - Ref: https://github.com/curl/curl/issues/9706 + Closes #11821 - Closes https://github.com/curl/curl/pull/11536 +- cmdline-docs: make sure to phrase it as "added in ...." -wangzhikun (29 Jul 2023) + References to things that were added or changed in a specific version + should be specified as "(added in [version]) for two reasons: -- winbuild: improve check for static zlib + 1 - consistency - - Check for zlib static library name zlibstatic.lib. + 2 - to allow gen.pl to strip them out if deemed referring to too old + versions - zlib's static library has a different name depending on how it was - built. zlibstatic.lib is output by cmake. zlibstat.lib is output by - their pre-generated Visual Studio project files (in the contrib - directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall - instead of cdecl if you end up exporting the zlib functions). + Closes #11821 - Prior to this change the makefile only checked for the latter. +Jay Satiro (8 Sep 2023) - Closes https://github.com/curl/curl/pull/11521 +- docs: mark --ssl-revoke-best-effort as Schannel specific -Daniel Stenberg (29 Jul 2023) + Closes https://github.com/curl/curl/pull/11760 -- configure: use the pkg-config --libs-only-l flag for libssh2 +Nathan Moinvaziri (8 Sep 2023) - ... instead of --libs, as that one also returns -L flags. +- schannel: fix ordering of cert chain info - Reported-by: Wilhelm von Thiele - Fixes #11538 - Closes #11539 + - Use CERT_CONTEXT's pbCertEncoded to determine chain order. -Viktor Szakats (29 Jul 2023) + CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains + end-entity/server certificate in pbCertEncoded. We can use this pointer + to determine the order of certificates when enumerating hCertStore using + CertEnumCertificatesInStore. -- cmake: support building static and shared libcurl in one go + This change is to help ensure that the ordering of the certificate chain + requested by the user via CURLINFO_CERTINFO has the same ordering on all + versions of Windows. - This patch adds the ability to build a static and shared libcurl library - in a single build session. It also adds an option to select which one to - use when building the curl executable. + Prior to this change Schannel certificate order was reversed in 8986df80 + but that was later reverted in f540a39b when it was discovered that + Windows 11 22H2 does the reversal on its own. - New build options: - - `BUILD_STATIC_LIBS`. Default: `OFF`. - Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`. - - `BUILD_STATIC_CURL`. Default: `OFF`. - Requires `BUILD_STATIC_LIBS` enabled. - Enabled automatically if building static libcurl only. - - `STATIC_LIB_SUFFIX`. Default: empty. - - `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide - with static lib name (typically with MSVC) in Windows builds. - Otherwise empty. + Ref: https://github.com/curl/curl/issues/9706 - Also: + Closes https://github.com/curl/curl/pull/11632 - - Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass - it directly to the compiler. This also allows to delete a condition - from `tests/server/CMakeLists.txt`. +Chris Talbot (8 Sep 2023) - - Complete a TODO by following the logic used in autotools (also for - `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of - `libcurl.pc` for _static-only_ curl builds. +- digest: Use hostname to generate spn instead of realm - - Convert an existing CI test to build both shared and static libcurl. + In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 - Closes #11505 + digest-uri-value should be serv-type "/" host , where host is: -Stefan Eissing (28 Jul 2023) + The DNS host name or IP address for the service requested. The + DNS host name must be the fully-qualified canonical name of the + host. The DNS host name is the preferred form; see notes on server + processing of the digest-uri. -- CI/awslc: add cache for build awslc library + Realm may not be the host, so we must specify the host explicitly. - Closes #11535 + Note this change only affects the non-SSPI digest code. The digest code + used by SSPI builds already uses the hostname to generate the spn. -- GHA/linux.yml: add caching + Ref: https://github.com/curl/curl/issues/11369 - Closes #11532 + Closes https://github.com/curl/curl/pull/11395 -Daniel Stenberg (27 Jul 2023) +Daniel Stenberg (7 Sep 2023) -- RELEASE-NOTES: synced +- docs: remove use of the word 'very' - Bump working version to 8.3.0 + It is mostly superfluous. proselint would complain. -- url: remove infof() output for "still name resolving" + Closes #11818 - The message does not help and might get spewed a lot during times. +- curl_multi_remove_handle.3: clarify what happens with connection - Reported-by: yushicheng7788 on github - Fixes #11394 - Closes #11529 + Closes #11817 -- KNOWN_BUGS: cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!" +- RELEASE-NOTES: synced - Closes #11244 +- test439: verify query canonization for aws-sigv4 -Stefan Eissing (27 Jul 2023) +- tool_operate: make aws-sigv4 not require TLS to be used -- CI: quiche updates + Maybe not used too often, but we want it for testing and it should work. - - remove quiche from standard `linux` workflow - - add mod_h2 caching to quiche workflow - - rename quiche to quiche-linux - - move version definitions into env section +- http_aws_sigv4: canonicalize the query - Closes #11528 + Percent encoding needs to be done using uppercase, and most + non-alphanumerical must be percent-encoded. -- http2: disable asssertion blocking OSSFuzz testing + Fixes #11794 + Reported-by: John Walker + Closes #11806 - - not clear how this triggers and it blocks OSSFuzz testing other - things. Since we handle the case with an error return, disabling the - assertion for now seems the best way forward. +Wyatt O'Day (7 Sep 2023) - Fixes #11500 - Closes #11519 +- lib: add ability to disable auths individually -- http2: fix in h2 proxy tunnel: progress in ingress on sending + Both with configure and cmake - - depending on what is tunneled, the proxy may never get invoked for - receiving data explicitly. Not progressing ingress may lead to stalls - due to missed WINDOW_UPDATEs. + Closes #11490 - CI: - - add a chache for building mod_h2 +Stefan Eissing (7 Sep 2023) - Closes #11527 +- ngtcp2: fix handling of large requests -- CI ngtcp2+quictls: use nghttpx cache as in quiche build + - requests >64K are send in parts to the filter + - fix parsing of the request to assemble it correctly + from several sends + - open a QUIC stream only when the complete request has + been collected -Jay Satiro (27 Jul 2023) + Closes #11815 -- bearssl: don't load CA certs when peer verification is disabled +- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before - We already do this for other SSL backends. + - we delay loading the x509 store to shorten the handshake time. + However an application callback installed via CURLOPT_SSL_CTX_FUNCTION + may need to have the store loaded and try to manipulate it. + - load the x509 store before invoking the app callback - Bug: https://github.com/curl/curl/pull/11457#issuecomment-1644587473 - Reported-by: kyled-dell@users.noreply.github.com + Fixes #11800 + Reported-by: guoxinvmware on github + Cloes #11805 - Closes https://github.com/curl/curl/pull/11497 +Daniel Stenberg (7 Sep 2023) -Daniel Stenberg (26 Jul 2023) +- krb5: fix "implicit conversion loses integer precision" warnings -- easy: remove #ifdefs to make code easier on the eye + conversions to/from enum and unsigned chars - Closes #11525 + Closes #11814 -Stefan Eissing (26 Jul 2023) +Stefan Eissing (7 Sep 2023) -- GHA: adding quiche workflow +- pytest: improvements - - adding separate quiche workflow to also build nghttpx server for testing + - set CURL_CI for pytest runs in CI environments + - exclude timing sensitive tests from CI runs + - for failed results, list only the log and stat of + the failed transfer - Closes #11517 + - fix type in http.c comment -Version 8.2.1 (26 Jul 2023) + Closes #11812 -Daniel Stenberg (26 Jul 2023) +- CI: move on to ngtcp2 v0.19.1 -- RELEASE-NOTES: synced + Closes #11809 - curl 8.2.1 release +Dan Fandrich (5 Sep 2023) -- THANKS: add contributors from 8.2.1 +- CI: run Circle macOS builds on x86 for now -- docs: provide more see also for cipher options + The ARM machines aren't ready for us and requesting them now causes + warnings e-mails to be sent to some PR pushers. - More cross references. Hide nroff errors. + Ref: #11771 - Closes #11513 +Viktor Szakats (5 Sep 2023) -- docs: mark two TLS options for TLS, not SSL +- http3: adjust cast for ngtcp2 v0.19.0 - Closes #11514 + ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` + an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. -Brad Harder (25 Jul 2023) + Fixes: + ``` + ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in + teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned + char') [-Wimplicit-int-conversion] + pi.ecn = (uint32_t)ecn; + ~ ^~~~~~~~~~~~~ + ``` -- curl_multi_wait.3: fix arg quoting to doc macro .BR + Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our + docs and CI. - Closes #11511 + Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 + 782894a3 + Ref: https://github.com/ngtcp2/ngtcp2/pull/877 + Closes #11798 -Daniel Stenberg (24 Jul 2023) +Stefan Eissing (5 Sep 2023) -- RELEASE-NOTES: synced +- http: fix sending of large requests -Viktor Szakats (24 Jul 2023) + - refs #11342 where errors with git https interactions + were observed + - problem was caused by 1st sends of size larger than 64KB + which resulted in later retries of 64KB only + - limit sending of 1st block to 64KB + - adjust h2/h3 filters to cope with parsing the HTTP/1.1 + formatted request in chunks -- cmake: update ngtcp2 detection + - introducing Curl_nwrite() as companion to Curl_write() + for the many cases where the sockindex is already known - Replace `OpenSSL` with `quictls` to follow the same change - in the v0.17.0 ngtcp2 release. + Fixes #11342 (again) + Closes #11803 - Follow-up to e0093b4b732f6495b0fb1cd6747cbfedcdcf63ed +- pytest: fix check for slow_network skips to only apply when intended - Closes #11508 + Closes #11801 -Stefan Eissing (24 Jul 2023) +Daniel Stenberg (5 Sep 2023) -- http: VLH, very large header test and fixes +- curl_url_get/set.3: add missing semicolon in SYNOPSIS - - adding tests using very large passwords in auth - - fixes general http sending to treat h3 like h2, and - not like http1.1 - - eliminate H2_HEADER max definitions and use the commmon - DYN_HTTP_REQUEST everywhere, different limits do not help - - fix http2 handling of requests denied by nghttp2 on send - to immediately report the refused stream +- CURLOPT_URL.3: explain curl_url_set() uses the same parser - Closes #11509 +- CURLOPT_URL.3: add two URL API calls in the see-also section -Andrei Rybak (23 Jul 2023) +Dan Fandrich (4 Sep 2023) -- CONTRIBUTE: drop mention of copyright year ranges +- CI: add a 32-bit i686 Linux build - Year ranges in copyrights were dropped in commits [1] and [2]. - Verification of year ranges in copyrights was dropped from script - 'scripts/copyright.pl' in commit [3]. However, the corresponding - passages in file 'docs/CONTRIBUTE.md' weren't updated. + This is done by cross-compiling under regular x86_64 Linux. Since the + kernel offers backwards compatibility, the binaries can be tested as + normal. - Drop mentions of copyright year ranges from 'docs/CONTRIBUTE.md'. + Closes #11799 - [1] 2bc1d775f (copyright: update all copyright lines and remove year - ranges, 2023-01-02) - [2] c46761bd8 (tests/http: remove year ranges from copyrights, - 2023-03-14) - [3] 0e293bacb (copyright.pl: cease doing year verifications, 2023-01-28) +- tests: fix a type warning on 32-bit x86 - Closes #11504 +Viktor Szakats (4 Sep 2023) -- CONTRIBUTE: fix syntax in commit message description +- tests: delete stray `.orig` file - File 'docs/CONTRIBUTE.md' includes a description of how one should write - commit messages in the curl project. Different possible parts of the - message are enclosed in square brackets. One exception is the section - describing how the curl project doesn't use "Signed-off-by" commit - trailers [1], which is enclosed in an opening curly brace paired with a - closing square bracket. + Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f + Closes #11797 - Fix the enclosing square brackets in description of "Signed-off-by" - trailers in commit messages in file 'docs/CONTRIBUTE.md'. +Daniel Stenberg (4 Sep 2023) - [1] See description of option '--signoff' in Git documentation: - https://git-scm.com/docs/git-commit +- RELEASE-NOTES: synced - Closes #11504 +Viktor Szakats (4 Sep 2023) -Daniel Stenberg (23 Jul 2023) +- lib: silence compiler warning in inet_ntop6 -- src/mkhelp: strip off escape sequences + ``` + ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her + e [-Wcomma] + cur.base = i, cur.len = 1; + ^ + ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin + g + cur.base = i, cur.len = 1; + ^~~~~~~~~~~~ + (void)( ) + ``` - At some point the nroff command stopped stripping off escape sequences, - so then this script needs to do the job instead. + Closes #11790 - Reported-by: VictorVG on github - Fixes #11501 - Closes #11503 +Daniel Stenberg (4 Sep 2023) -- KNOWN_BUGS: building for old macOS fails with gcc +- transfer: also stop the sending on closed connection - Closes #11441 + Previously this cleared the receiving bit only but in some cases it is + also still sending (like a request-body) when disconnected and neither + direction can continue then. -Jacob Hoffman-Andrews (22 Jul 2023) + Fixes #11769 + Reported-by: Oleg Jukovec + Closes #11795 -- rustls: update rustls-ffi 0.10.0 +John Bampton (4 Sep 2023) - This brings in version 0.21.0 of the upstream rustls implementation, - which notable includes support for IP address certificates. +- docs: change `sub-domain` to `subdomain` - Closes #10865 + https://en.wikipedia.org/wiki/Subdomain -Brad Harder (22 Jul 2023) + Closes #11793 -- websocket: rename arguments/variables to match docs +Stefan Eissing (4 Sep 2023) - Pedantry/semantic-alignment between functions, docs, comments with - respect to websocket protocol code; No functional change intended. +- multi: more efficient pollfd count for poll - * "totalsize", "framesize" becomes "fragsize" (we deal in frame fragments). + - do not use separate pollfds for sockets that have POLLIN+POLLOUT - * "sendflags" becomes "flags" + Closes #11792 - * use canonical CURL *handle +- http2: polish things around POST - Closes #11493 + - added test cases for various code paths + - fixed handling of blocked write when stream had + been closed inbetween attempts + - re-enabled DEBUGASSERT on send with smaller data size -Jan Macku (21 Jul 2023) + - in debug builds, environment variables can be set to simulate a slow + network when sending data. cf-socket.c and vquic.c support + * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be + answered with a EAGAIN. TCP/UNIX sockets. + This is chosen randomly. + * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written + to the network. TCP/UNIX sockets. + Example: 80 means a send with 1000 bytes would only send 800 + This is applied to every send. + * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be + answered with EAGAIN. QUIC only. + This is chosen randomly. -- bug_report: use issue forms instead of markdown template + Closes #11756 - Issue forms allow you to define web-like input forms using YAML - syntax. It allows you to guide the reporter to get the required - information. +Daniel Stenberg (4 Sep 2023) - Signed-off-by: Jan Macku - Closes #11474 +- docs: add curl_global_trace to some SEE ALSO sections -Daniel Stenberg (21 Jul 2023) + Closes #11791 -- TODO: Obey Retry-After in redirects +- os400: fix checksrc nits - (remove "Set custom client ip when using haproxy protocol" which was - shipped in 8.2.0) + Closes #11789 - Mentioned-by: Yair Lenga - Closes #11447 +Nicholas Nethercote (3 Sep 2023) -- RELEASE-NOTES: synced +- hyper: remove `hyptransfer->endtask` -Oliver Roberts (21 Jul 2023) + `Curl_hyper_stream` needs to distinguish between two kinds of + `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and + (b) background tasks that hyper produces. It does this by recording the + address of any `foreach` task in `hyptransfer->endtask` before pushing + it into the executor, and then comparing that against the address of + tasks later polled out of the executor. -- amissl: fix AmiSSL v5 detection + This works right now, but there is no guarantee from hyper that the + addresses are stable. `hyper_executor_push` says "The executor takes + ownership of the task, which should not be accessed again unless + returned back to the user with `hyper_executor_poll`". That wording is a + bit ambiguous but with my Rust programmer's hat on I read it as meaning + the task returned with `hyper_executor_poll` may be conceptually the + same as a task that was pushed, but that there are no other guarantees + and comparing addresses is a bad idea. - Due to changes in the AmiSSL SDK, the detection needed adjusting. + This commit instead uses `hyper_task_set_userdata` to mark the `foreach` + task with a `USERDATA_RESP_BODY` value which can then be checked for, + removing the need for `hyptransfer->endtask`. This makes the code look + more like that hyper C API examples, which use userdata for every task + and never look at task addresses. - Closes #11477 + Closes #11779 -Alois Klink (21 Jul 2023) +Dave Cottlehuber (3 Sep 2023) -- unittest/makefile: remove unneeded unit1621_LDADD +- ws: fix spelling mistakes in examples and tests - The `unit1621_LDADD` variable has the exact same value as the `LDADD` - flag in `Makefile.am`, except without `@LDFLAGS@ @LIBCURL_LIBS@`. + Closes #11784 - This was originally added by [98e6629][], but I can't see any reason - why it exists, so we should remove it to clean things up. +Daniel Stenberg (3 Sep 2023) - [98e6629]: https://github.com/curl/curl/commit/98e6629154044e4ab1ee7cff8351c7 - ebcb131e88 +- tool_filetime: make -z work with file dates before 1970 - Closes #11494 + Fixes #11785 + Reported-by: Harry Sintonen + Closes #11786 -- unittest/makefile: remove unneeded unit1394_LDADD +Dan Fandrich (1 Sep 2023) - These custom `unit1394_LDADD` and similar automake overrides are no - longer neded. They were originally added by added by [8dac7be][] for - metalink support, but are no longer after [265b14d][] removed metalink. +- build: fix portability of mancheck and checksrc targets - [8dac7be]: https://github.com/curl/curl/commit/8dac7be438512a8725d3c71e9139bd - fdcac1ed8c - [265b14d]: https://github.com/curl/curl/commit/265b14d6b37c4298bd5556fabcbc37 - d36f911693 + At least FreeBSD preserves cwd across makefile lines, so rules + consisting of more than one "cd X; do_something" must be explicitly run + in a subshell to avoid this. This problem caused the Cirrus FreeBSD + build to fail when parallel make jobs were enabled. - Closes #11494 +- CI: adjust labeler match patterns for new & obsolete files -- cmake: add `libcurlu`/`libcurltool` for unit tests +- configure: trust pkg-config when it's used for zlib - Add a `libcurlu`/`libcurltool` static library that is compiled only for - unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not - built by default, they're only built if unit tests are built. + The library flags retrieved from pkg-config were later thrown out and + harded-coded, which negates the whole reason to use pkg-config. + Also, previously, the assumption was made that --libs-only-l and + --libs-only-L are the full decomposition of --libs, which is untrue and + would not allow linking against a static zlib. The new approach is + better in that it uses --libs, although only if --libs-only-l returns + nothing. - These libraries allow us to compile every unit test with CMake. + Bug: https://curl.se/mail/lib-2023-08/0081.html + Reported-by: Randall + Closes #11778 - Closes #11446 +Stefan Eissing (1 Sep 2023) -Daniel Stenberg (21 Jul 2023) +- CI/ngtcp2: clear wolfssl for when cache is ignored -- test979: test -u with redirect to (the same) absolute host + Closes #11783 - Verifies #11492 +Daniel Stenberg (1 Sep 2023) -- transfer: do not clear the credentials on redirect to absolute URL +- RELEASE-NOTES: synced - Makes test 979 work. Regression shipped in 8.2.0 from commit - dd4d1a26959f63a2c +Nicholas Nethercote (1 Sep 2023) - Fixes #11486 - Reported-by: Cloudogu Siebels - Closes #11492 +- hyper: fix a progress upload counter bug -Jon Rumsey (20 Jul 2023) + `Curl_pgrsSetUploadCounter` should be a passed a total count, not an + increment. -- os400: correct EXPECTED_STRING_LASTZEROTERMINATED + This changes the failing diff for test 579 with hyper from this: + ``` + Progress callback called with UL 0 out of 0[LF] + -Progress callback called with UL 8 out of 0[LF] + -Progress callback called with UL 16 out of 0[LF] + -Progress callback called with UL 26 out of 0[LF] + -Progress callback called with UL 61 out of 0[LF] + -Progress callback called with UL 66 out of 0[LF] + +Progress callback called with UL 29 out of 0[LF] + ``` + to this: + ``` + Progress callback called with UL 0 out of 0[LF] + -Progress callback called with UL 8 out of 0[LF] + -Progress callback called with UL 16 out of 0[LF] + -Progress callback called with UL 26 out of 0[LF] + -Progress callback called with UL 61 out of 0[LF] + -Progress callback called with UL 66 out of 0[LF] + +Progress callback called with UL 40 out of 0[LF] + ``` + Presumably a step in the right direction. - Correct EXPECTED_STRING_LASTZEROTERMINATED to account for - CURLOPT_HAPROXY_CLIENT_IP which requires EBCDIC to ASCII conversion when - passed into curl_easy_setopt(). + Closes #11780 - Closes #11476 +Daniel Stenberg (1 Sep 2023) -Oliver Roberts (20 Jul 2023) +- awssiv4: avoid freeing the date pointer on error -- amissl: add missing signal.h include + Since it was not allocated, don't free it even if it was wrong syntax - In some environments, signal.h is already included, but not in others - which cause compilation to fail, so explictly include it. + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 - Closes #11478 + Follow-up to b137634ba3adb -- amigaos: fix sys/mbuf.h m_len macro clash + Closes #11782 - The updated Curl_http_req_make and Curl_http_req_make2 functions spawned - a parameter called m_len. The AmigaOS networking headers, derived from - NetBSD, contain "#define m_len m_hdr.mh_len" which clashes with - this. Since we do not actually use mbuf, force the include file to be - ignored, removing the clash. +Stefan Eissing (1 Sep 2023) - Closes #11479 +- CI: ngtcp2-linux: use separate caches for tls libraries -Daniel Stenberg (20 Jul 2023) + allow ever changing master for wolfssl -- socks: print ipv6 address within brackets + Closes #11766 - Fixes #11483 - Closes #11484 +- replace `master` as wolfssl-version with recent commit -Christian Schmitz (20 Jul 2023) +- wolfssl, use master again in CI -- libcurl-errors.3: add CURLUE_OK + - with the shared session update fix landed in master, it + is time to use that in our CI again - Closes #11488 +Nicholas Nethercote (31 Aug 2023) -Oliver Roberts (20 Jul 2023) +- tests: fix formatting errors in `FILEFORMAT.md`. -- cfilters: rename close/connect functions to avoid clashes + Without the surrounding backticks, these tags get swallowed when the + markdown is rendered. - Rename `close` and `connect` in `struct Curl_cftype` for - consistency and to avoid clashes with macros of the same name - (the standard AmigaOS networking connect() function is implemented - via a macro). + Closes #11777 - Closes #11491 +Viktor Szakats (31 Aug 2023) -Stefan Eissing (20 Jul 2023) +- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` -- http2: fix regression on upload EOF handling + Allow overriding the default TLS backend via a CMake setting. - - a regression introduced by c9ec85121110d7cbbbed2990024222c8f5b8afe5 - where optimization of small POST bodies leads to a new code path - for such uploads that did not trigger the "done sending" event - - add triggering this event for early "upload_done" situations + E.g.: + `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` - Fixes #11485 - Closes #11487 - Reported-by: Aleksander Mazur + Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, + schannel, secure-transport, wolfssl -Daniel Stenberg (19 Jul 2023) + The passed string is baked into the curl/libcurl binaries. + The value is case-insensitive. -- configure: check for nghttp2_session_get_stream_local_window_size + We added a similar option to autotools in 2017 via + c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. - The http2 code uses it now. Introduced in nghttp2 1.15.0 (Sep 2016) + TODO: Convert to lowercase to improve reproducibility. - Fixes #11470 - Reported-by: Paul Howarth - Closes #11473 + Closes #11774 -Stefan Eissing (19 Jul 2023) +- sectransp: fix compiler warnings -- quiche: fix segfault and other things + https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# + step:3:11046 + ``` + /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 + 4: warning: unused variable 'success' [-Wunused-variable] + OSStatus success; + ^ + /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 + 4: warning: unused parameter 'sha256len' [-Wunused-parameter] + size_t sha256len) + ^ + ``` - - refs #11449 where a segfault is reported when IP Eyeballing did - not immediately connect but made several attempts - - The transfer initiating the eyeballing was initialized too early, - leadding to references to the filter instance that was then - replaced in the subsequent eyeball attempts. That led to a use - after free in the buffer handling for the transfer - - transfers are initiated now more lazy (like in the ngtcp2 filter), - when the stream is actually opened - - suppress reporting on quiche event errors for "other" transfers - than the current one to not fail a transfer due to faults in - another one. - - revert recent return value handling for quiche_h3_recv_body() - to not indicate an error but an EAGAIN situation. We wish quiche - would document what functions return. + Closes #11773 - Fixes #11449 - Closes #11469 - Reported-by: ウさん +- tidy-up: mostly whitespace nits -Daniel Stenberg (19 Jul 2023) + - delete completed TODO from `./CMakeLists.txt`. + - convert a C++ comment to C89 in `./CMake/CurlTests.c`. + - delete duplicate EOLs from EOF. + - add missing EOL at EOF. + - delete whitespace at EOL (except from expected test results). + - convert tabs to spaces. + - convert CRLF EOLs to LF in GHA yaml. + - text casing fixes in `./CMakeLists.txt`. + - fix a codespell typo in `packages/OS400/initscript.sh`. -- hostip: return IPv6 first for localhost resolves + Closes #11772 - Fixes #11465 - Reported-by: Chilledheart on github - Closes #11466 +Dan Fandrich (31 Aug 2023) + +- CI: remove Windows builds from Cirrus, without replacement -Harry Sintonen (19 Jul 2023) + If we don't do this, all coverage on Cirrus will cease in a few days. By + removing the Windows builds, the FreeBSD one should still continue + as before. The Windows builds will need be moved to another service to + maintain test coverage. -- tool: fix tool_seek_cb build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T + Closes #11771 - - a variable was renamed, and some use of it wasn't. this fixes the - build. +- CI: switch macOS ARM build from Cirrus to Circle CI - Closes #11468 + Cirrus is drastically reducing their free tier on Sept. 1, so they will + no longer perform all these builds for us. All but one build has been + moved, with the LibreSSL one being dropped because of linking problems + on Circle. -Stefan Eissing (19 Jul 2023) + One important note about this change is that Circle CI is currently + directing all these builds to x86_64 hardware, despite them requesting + ARM. This is because ARM nodes are scheduled to be available on the + free tier only in December. This reduces our architectural diversity + until then but it should automatically come back once those machines are + enabled. -- quiche: fix lookup of transfer at multi +- CI: use the right variable for BSD make - - refs #11449 where weirdness in quiche multi connection tranfers was - observed - - fixes lookup of transfer for a quiche event to take the connection - into account - - formerly, a transfer with the same stream_id, but on another connection - could be found + BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel + builds before. - Closes #11462 +- CI: drop the FreeBSD 12.X build -Daniel Stenberg (19 Jul 2023) + Cirrus' new free tier won't let us have many builds, so drop the + nonessential ones. The FreeBSD 13.X build will still give us the most + relevant FreeBSD coverage. -- RELEASE-NOTES: synced +- CI: move the Alpine build from Cirrus to GHA - bump to 8.2.1 + Cirrus is reducing their free tier to next to nothing, so we must move + builds elsewhere. -John Haugabook (19 Jul 2023) +Stefan Eissing (30 Aug 2023) -- ciphers.d: put URL in first column +- test_07_upload.py: fix test_07_34 curl args - This makes the URL turn into a link properly when "webified". + - Pass correct filename to --data-binary. - Fixes https://github.com/curl/curl-www/issues/270 - Closes #11464 + Prior to this change --data-binary was passed an incorrect filename due + to a missing separator in the arguments list. Since aacbeae7 curl will + error on incorrect filenames for POST. -Version 8.2.0 (19 Jul 2023) + Fixes https://github.com/curl/curl/issues/11761 + Closes https://github.com/curl/curl/pull/11763 -Daniel Stenberg (19 Jul 2023) +Nicholas Nethercote (30 Aug 2023) -- RELEASE-NOTES: synced +- tests: document which tests fail due to hyper's lack of trailer support. - 8.2.0 release + Closes #11762 -- THANKS-filter: strip out "GitHub" +- docs: removing "pausing transfers" from HYPER.md. -- THANKS: add contributors from 8.2.0 + It's a reference to #8600, which was fixed by #9070. -- RELEASE-PROCEDURE.md: adjust the release dates + Closes #11764 -Stefan Eissing (17 Jul 2023) +Patrick Monnerat (30 Aug 2023) -- quiche: fix defects found in latest coverity report +- os400: handle CURL_TEMP_PRINTF() while building bind source - Closes #11455 + Closes #11547 -Daniel Stenberg (17 Jul 2023) +- os400: build test servers -- quiche: avoid NULL deref in debug logging + Also fix a non-compliant main prototype in disabled.c. - Coverity reported "Dereference after null check" + Closes #11547 - If stream is NULL and the function exits, the logging must not deref it. +- tests: fix compilation error for os400 - Closes #11454 + OS400 uses BSD 4.3 setsockopt() prototype by default: this does not + define parameter as const, resulting in an error if actual parameter is + const. Remove the const keyword from the actual parameter cast: this + works in all conditions, even if the formal parameter uses it. -Stefan Eissing (17 Jul 2023) + Closes #11547 -- http2: treat initial SETTINGS as a WINDOW_UPDATE +- os400: make programs and command name configurable - - refs #11426 where spurious stalls on large POST requests - are reported - - the issue seems to involve the following - * first stream on connection adds up to 64KB of POST - data, which is the max default HTTP/2 stream window size - transfer is set to HOLD - * initial SETTINGS from server arrive, enlarging the stream - window. But no WINDOW_UPDATE is received. - * curl stalls - - the fix un-HOLDs a stream on receiving SETTINGS, not - relying on a WINDOW_UPDATE from lazy servers + Closes #11547 - Closes #11450 +- os400: move build configuration parameters to a separate script -Daniel Stenberg (17 Jul 2023) + They can then easily be overriden in a script named "config400.override" + that is not part of the distribution. -- ngtcp2: assigning timeout, but value is overwritten before used + Closes #11547 - Reported by Coverity +- os400: implement CLI tool - Closes #11453 + This is provided as a QADRT (ascii) program, a link to it in the IFS and + a minimal CL command. -- krb5: add typecast to please Coverity + Closes #11547 -Derzsi Dániel (16 Jul 2023) +Matthias Gatto (30 Aug 2023) -- wolfssl: support setting CA certificates as blob +- lib: fix aws-sigv4 having date header twice in some cases - Closes #11445 + When the user was providing the header X-XXX-Date, the header was + re-added during signature computation, and we had it twice in the + request. -- wolfssl: detect when TLS 1.2 support is not built into wolfssl + Reported-by: apparentorder@users.noreply.github.com - Closes #11444 + Signed-off-by: Matthias Gatto -Graham Campbell (15 Jul 2023) + Fixes: https://github.com/curl/curl/issues/11738 + Closes: https://github.com/curl/curl/pull/11754 -- CI: bump nghttp2 from 1.55.0 to 1.55.1 +Jay Satiro (30 Aug 2023) - Closes #11442 +- multi: remove 'processing: ' debug message -Daniel Stenberg (15 Jul 2023) + - Remove debug message added by e024d566. -- curl: return error when asked to use an unsupported HTTP version + Closes https://github.com/curl/curl/pull/11759 - When one of the following options are used but the libcurl in use does - not support it: +- ftp: fix temp write of ipv6 address - --http2 - --http2-prior-knowledge - --proxy-http2 + - During the check to differentiate between a port and IPv6 address + without brackets, write the binary IPv6 address to an in6_addr. - Closes #11440 + Prior to this change the binary IPv6 address was erroneously written to + a sockaddr_in6 'sa6' when it should have been written to its in6_addr + member 'sin6_addr'. There's no fallout because no members of 'sa6' are + accessed before it is later overwritten. -Chris Paulson-Ellis (14 Jul 2023) + Closes https://github.com/curl/curl/pull/11747 -- cf-socket: don't bypass fclosesocket callback if cancelled before connect +- tool: change some fopen failures from warnings to errors - After upgrading to 8.1.2 from 7.84.0, I found that sockets were being - closed without calling the fclosesocket callback if a request was - cancelled after the associated socket was created, but before the socket - was connected. This lead to an imbalance of fopensocket & fclosesocket - callbacks, causing problems with a custom event loop integration using - the multi-API. + - Error on missing input file for --data, --data-binary, + --data-urlencode, --header, --variable, --write-out. - This was caused by cf_socket_close() calling sclose() directly instead - of calling socket_close() if the socket was not active. For regular TCP - client connections, the socket is activated by cf_socket_active(), which - is only called when the socket completes the connect. + Prior to this change if a user of the curl tool specified an input file + for one of the above options and that file could not be opened then it + would be treated as zero length data instead of an error. For example, a + POST using `--data @filenametypo` would cause a zero length POST which + is probably not what the user intended. - As far as I can tell, this issue has existed since 7.88.0. That is, - since the code in question was introduced by: - commit 71b7e0161032927cdfb4e75ea40f65b8898b3956 - Author: Stefan Eissing - Date: Fri Dec 30 09:14:55 2022 +0100 + Closes https://github.com/curl/curl/pull/11677 - lib: connect/h2/h3 refactor +- hostip: fix typo - Closes #11439 +Davide Masserut (29 Aug 2023) -Daniel Stenberg (13 Jul 2023) +- tool: avoid including leading spaces in the Location hyperlink -- tool_parsecfg: accept line lengths up to 10M + Co-authored-by: Dan Fandrich - Bumped from 100K set in 47dd957daff9 + Closes #11735 - Reported-by: Antoine du Hamel - Fixes #11431 - Closes #11435 +Daniel Stenberg (29 Aug 2023) -Stefan Eissing (13 Jul 2023) +- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline -- CI: brew fix for openssl in default path + Closes #11757 - If brew install/update links openssl into /usr/local, it will be found - before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of - that by unlinking the keg. +- connect: stop halving the remaining timeout when less than 600 ms left - Fixes #11413 - Closes #11436 + When curl wants to connect to a host, it always has a TIMEOUT. The + maximum time it is allowed to spend until a connect is confirmed. -Daniel Stenberg (13 Jul 2023) + curl will try to connect to each of the IP adresses returned for the + host. Two loops, one for each IP family. -- RELEASE-NOTES: synced + During the connect loop, while curl has more than one IP address left to + try within a single address family, curl has traditionally allowed (time + left/2) for *this* connect attempt. This, to not get stuck on the + initial addresses in case the timeout but still allow later addresses to + get attempted. -Ondřej Koláček (13 Jul 2023) + This has the downside that when users set a very short timeout and the + host has a large number of IP addresses, the effective result might be + that every attempt gets a little too short time. -- sectransp: fix EOF handling + This change stop doing the divided-by-two if the total time left is + below a threshold. This threshold is 600 milliseconds. - Regression since the large refactor from 2022 + Closes #11693 - Closes #11427 +- asyn-ares: reduce timeout to 2000ms -Daniel Stenberg (13 Jul 2023) + When UDP packets get lost this makes for slightly faster retries. This + lower timeout is used by @c-ares itself by default starting next + release. -- checksrc: quote the file name to work with "funny" letters + Closes #11753 - Closes #11437 +John Bampton (29 Aug 2023) -Karthikdasari0423 (13 Jul 2023) +- misc: remove duplicate words -- HTTP3.md: ngtcp2 updated to v0.17.0 and nghttp3 to v0.13.0 + Closes #11740 - Follow-up to e0093b4b732f6 +Daniel Stenberg (29 Aug 2023) - Closes #11433 +- RELEASE-NOTES: synced -Daniel Stenberg (13 Jul 2023) +- wolfSSL: avoid the OpenSSL compat API when not needed -- CURLOPT_MIMEPOST.3: clarify what setting to NULL means + ... and instead call wolfSSL functions directly. - Follow-up to e08382a208d4e480 + Closes #11752 - Closes #11430 +Viktor Szakats (28 Aug 2023) -Tatsuhiro Tsujikawa (12 Jul 2023) +- lib: fix null ptr derefs and uninitialized vars (h2/h3) -- ngtcp2: build with 0.17.0 and nghttp3 0.13.0 + Fixing compiler warnings with gcc 13.2.0 in unity builds. - - ngtcp2_crypto_openssl was renamed to ngtcp2_crypto_quictls. + Assisted-by: Jay Satiro + Assisted-by: Stefan Eissing + Closes #11739 - Closes #11428 +Jay Satiro (28 Aug 2023) -- CI: Bump ngtcp2, nghttp3, and nghttp2 +- secureserver.pl: fix stunnel version parsing - Closes #11428 + - Allow the stunnel minor-version version part to be zero. -James Fuller (11 Jul 2023) + Prior to this change with the stunnel version scheme of . + if either part was 0 then version parsing would fail, causing + secureserver.pl to fail with error "No stunnel", causing tests that use + the SSL protocol to be skipped. As a practical matter this bug can only + be caused by a minor-version part of 0, since the major-version part is + always greater than 0. -- example/maxconnects: set maxconnect example + Closes https://github.com/curl/curl/pull/11722 - Closes #11343 +- secureserver.pl: fix stunnel path quoting -Pontakorn Prasertsuk (11 Jul 2023) + - Store the stunnel path in the private variable $stunnel unquoted and + instead quote it in the command strings. -- http2: send HEADER & DATA together if possible + Prior to this change the quoted stunnel path was passed to perl's file + operators which cannot handle quoted paths. For example: - Closes #11420 + $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; + if(-x $stunnel or -x "$stunnel") + # false even if path exists and is executable -Daniel Stenberg (11 Jul 2023) + Our other test scripts written in perl, unlike this one, use servers.pm + which has a global $stunnel variable with the path stored unquoted and + therefore those scripts don't have this problem. -- CI: use wolfSSL 5.6.3 in builds + Closes https://github.com/curl/curl/pull/11721 - No using master anymore +Daniel Stenberg (28 Aug 2023) - Closes #11424 +- altsvc: accept and parse IPv6 addresses in response headers -SaltyMilk (11 Jul 2023) + Store numerical IPv6 addresses in the alt-svc file with the brackets + present. -- fopen: optimize + Verify with test 437 and 438 - Closes #11419 + Fixes #11737 + Reported-by: oliverpool on github + Closes #11743 -Daniel Stenberg (11 Jul 2023) +- libtest: use curl_free() to free libcurl allocated data -- cmake: make use of snprintf + In several test programs. These mistakes are not detected or a problem + as long as memdebug.h is included, as that provides the debug wrappers + for all memory functions in the same style libcurl internals do it, + which makes curl_free and free effectively the same call. - Follow-up to 935b1bd4544a23a91d68 + Reported-by: Nicholas Nethercote + Closes #11746 - Closes #11423 +Jay Satiro (28 Aug 2023) -Stefan Eissing (11 Jul 2023) +- disable.d: explain --disable not implemented prior to 7.50.0 -- macOS: fix taget detection + Option -q/--disable was added in 5.0 but only -q was actually + implemented. Later --disable was implemented in e200034 (precedes + 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). - - TARGET_OS_OSX is not always defined on macOS - - this leads to missing symbol Curl_macos_init() - - TargetConditionals.h seems to define these only when - dynamic targets are enabled (somewhere?) - - this PR fixes that on my macOS 13.4.1 - - I have no clue why CI builds worked without it + Reported-by: pszlazak@users.noreply.github.com - Follow-up to c7308592fb8ba213fc2c1 - Closes #11417 + Fixes https://github.com/curl/curl/issues/11710 + Closes #11712 -Stan Hu (9 Jul 2023) +Nicholas Nethercote (28 Aug 2023) -- hostip.c: Move macOS-specific calls into global init call +- hyper: fix ownership problems - https://github.com/curl/curl/pull/7121 introduced a macOS system call - to `SCDynamicStoreCopyProxies`, which is invoked every time an IP - address needs to be resolved. + Some of these changes come from comparing `Curl_http` and + `start_CONNECT`, which are similar, and adding things to them that are + present in one and missing in another. - However, this system call is not thread-safe, and macOS will kill the - process if the system call is run first in a fork. To make it possible - for the parent process to call this once and prevent the crash, only - invoke this system call in the global initialization routine. + The most important changes: + - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the + happy path. + - In `start_CONNECT`, add a missing `hyper_request_free` on the error + path. + - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. + - In `bodysend`, remove an unnecessary `hyper_body_free` on a different + error path that would cause a double-free. + https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html + says of `hyper_request_set_body`: "This takes ownership of the + hyper_body *, you must not use it or free it after setting it on the + request." This is true even if `hyper_request_set_body` returns an + error; I confirmed this by looking at the hyper source code. - In addition, this change is beneficial because it: + Other changes are minor but make things slightly nicer. - 1. Avoids extra macOS system calls for every IP lookup. - 2. Consolidates macOS-specific initialization in a separate file. + Closes #11745 - Fixes #11252 - Closes #11254 +Daniel Stenberg (28 Aug 2023) -Daniel Stenberg (9 Jul 2023) +- multi.h: the 'revents' field of curl_waitfd is supported -- docs: use a space after RFC when spelling out RFC numbers + Since 6d30f8ebed34e7276 - Closes #11382 + Reported-by: Nicolás Ojeda Bär + Ref: #11748 + Closes #11749 -Margu (9 Jul 2023) +Gerome Fournier (27 Aug 2023) -- imap-append.c: update to make it more likely to work +- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() - Fixes #10300 - Closes #11397 + Closes #11742 -Emanuele Torre (9 Jul 2023) +Daniel Stenberg (27 Aug 2023) -- tool_writeout_json: fix encoding of control characters +- docs: mention critical files in same directories as curl saves - Control characters without a special escape sequence e.g. %00 or %06 - were being encoded as "u0006" instead of "\u0006". + ... cannot be fully protected. Don't do it. - Ref: https://github.com/curl/trurl/pull/214#discussion_r1257487858 - Closes #11414 + Co-authored-by: Jay Satiro + Reported-by: Harry Sintonen + Fixes #11530 + Closes #11701 -Stefan Eissing (9 Jul 2023) +John Hawthorn (26 Aug 2023) -- http3/ngtcp2: upload EAGAIN handling +- OpenSSL: clear error queue after SSL_shutdown - - refs #11389 where IDLE timeouts on upload are reported - - reword ngtcp2 expiry handling to apply to both send+recv - calls into the filter - - EAGAIN uploads similar to the recent changes in HTTP/2, e.g. - report success only when send data was ACKed. - - HOLD sending of EAGAINed uploads to avoid cpu busy loops - - rename internal function for consistency with HTTP/2 - implementation + We've seen errors left in the OpenSSL error queue (specifically, + "shutdown while in init") by adding some logging it revealed that the + source was this file. - Fixes #11389 - Closes #11390 + Since we call SSL_read and SSL_shutdown here, but don't check the return + code for an error, we should clear the OpenSSL error queue in case one + was raised. -Brian Nixon (9 Jul 2023) + This didn't affect curl because we call ERR_clear_error before every + write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when + libcurl is used in a process with other OpenSSL users, they may detect + an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their + own. -- tool_easysrc.h: correct `easysrc_perform` for `CURL_DISABLE_LIBCURL_OPTION` + Co-authored-by: Satana de Sant'Ana - Closes #11398 + Closes #11736 -Daniel Stenberg (9 Jul 2023) +Alexander Kanavin (25 Aug 2023) -- RELEASE-NOTES: synced +- tests: update cookie expiry dates to far in the future -- transfer: clear credentials when redirecting to absolute URL + This allows testing Y2038 with system time set to after that, so that + actual Y2038 issues can be exposed, and not masked by expiry errors. - Make sure the user and password for the second request is taken from the - redirected-to URL. + Fixes #11576 + Closes #11610 - Add test case 899 to verify. +John Bampton (25 Aug 2023) - Reported-by: James Lucas - Fixes #11410 - Closes #11412 +- misc: fix spelling -Stefan Eissing (8 Jul 2023) + Closes #11733 -- hyper: fix EOF handling on input +Daniel Stenberg (25 Aug 2023) - We ran out of disc space due to an infinite loop with debug logging +- cmdline-opts/page-header: clarify stronger that !opt == URL - Fixes #11377 - Closes #11385 - Reported-by: Dan Fandrich + Everything provided on the command line that is not an option (or an + argument to an option) is treated as a URL. -- http2: raise header limitations above and beyond + Closes #11734 - - not quite to infinity - - rewrote the implementation of our internal HTTP/1.x request - parsing to work with very large lines using dynbufs. - - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which - is also the limit of curl's general HTTP request processing. +- tests/runner: fix %else handling - Fixes #11405 - Closes #11407 + Getting the show state proper for %else and %endif did not properly work + in nested cases. -Juan Cruz Viotti (8 Jul 2023) + Follow-up to 3d089c41ea9 -- curl_easy_nextheader.3: add missing open parenthesis examples + Closes #11731 - Closes #11409 - Signed-off-by: Juan Cruz Viotti +Nicholas Nethercote (25 Aug 2023) -Dan Fandrich (7 Jul 2023) +- docs: Remove mention of #10803 from `KNOWN_BUGS`. -- CI: enable verbose test output on pytest + Because the leaks have been fixed. - This shows individual pass/fail status on tests and makes this output - consistent with other jobs' pytest invocations. +- c-hyper: fix another memory leak in `Curl_http`. -Stefan Eissing (28 Jun 2023) + There is a `hyper_clientconn_free` call on the happy path, but not one + on the error path. This commit adds one. -- http2: fix crash in handling stream weights + Fixes the second memory leak reported by Valgrind in #10803. - - Delay the priority handling until the stream has been opened. + Fixes #10803 + Closes #11729 - - Add test2404 to reproduce and verify. +- c-hyper: fix a memory leak in `Curl_http`. - Weights may change "on the run", which is why there are checks in - general egress handling. These must not trigger when the stream has not - been opened yet. + A request created with `hyper_request_new` must be consumed by either + `hyper_clientconn_send` or `hyper_request_free`. - Reported-by: jbgoog@users.noreply.github.com + This is not terrifically clear from the hyper docs -- + `hyper_request_free` is documented only with "Free an HTTP request if + not going to send it on a client" -- but a perusal of the hyper code + confirms it. - Fixes https://github.com/curl/curl/issues/11379 - Closes https://github.com/curl/curl/pull/11384 + This commit adds a `hyper_request_free` to the `error:` path in + `Curl_http` so that the request is consumed when an error occurs after + the request is created but before it is sent. -- tests/http: Add mod_h2 directive `H2ProxyRequests` + Fixes the first memory leak reported by Valgrind in #10803. - master of mod_h2 now requires H2ProxyRequests directives for forward - proxying with HTTP/2 to work. + Closes #11729 - Ref: https://github.com/icing/mod_h2/commit/3897a7086 +Daniel Stenberg (25 Aug 2023) - Closes https://github.com/curl/curl/pull/11392 +- RELEASE-NOTES: synced -Dan Fandrich (28 Jun 2023) +John Bampton (25 Aug 2023) -- CI: make Appveyor job names unique +- misc: spellfixes - Two otherwise identical mingw-w64 jobs now have their differing compiler - versions mentioned in their names. + Closes #11730 -Sheshadri.V (25 Jun 2023) +Daniel Stenberg (25 Aug 2023) -- curl.h: include for vxworks +- tests: add support for nested %if conditions - Closes #11356 + Provides more flexiblity to test cases. -Dan Fandrich (24 Jun 2023) + Also warn and bail out if there is an '%else' or %endif' without a + preceeding '%if'. -- CI: enable parallel make in more builds + Ref: #11610 + Closes #11728 - Most CI services provide at least two cores, so enable parallel make - jobs to take advantage of that for builds. Some dependencies aren't safe - to build in parallel so leave those as-is. Also, rename a few - workflows to eliminate duplicate names and provide a better idea what - they're about. +- time-cond.d: mention what happens on a missing file -- CI: don't install impacket if tests are not run + Closes #11727 - It just wastes time and bandwidth and isn't even used. +Christian Hesse (24 Aug 2023) -divinity76 (24 Jun 2023) +- docs/cmdline-opts: match the current output -- configure: the --without forms of the options are also gone + The release date has been added in output, reflect that in documentation. - --without-darwin-ssl and --without-metalink + Closes #11723 - Closes #11378 +Daniel Stenberg (24 Aug 2023) -Daniel Stenberg (23 Jun 2023) +- lib: minor comment corrections -- configure: add check for ldap_init_fd +- docs: rewrite to present tense - ... as otherwise the configure script will say it is OpenLDAP in the - summary, but not set the USE_OPENLDAP define, therefor not using the - intended OpenLDAP code paths. + ... instead of using future tense. - Regression since 4d7385446 (7.85.0) - Fixes #11372 - Closes #11374 - Reported-by: vlkl-sap on github + + numerous cleanups and improvements + + stick to "reuse" not "re-use" + + fewer contractions -Michał Petryka (23 Jun 2023) + Closes #11713 -- cmake: stop CMake from quietly ignoring missing Brotli +- urlapi: setting a blank URL ("") is not an ok URL - The CMake project was set to `QUIET` for Brotli instead of - `REQUIRED`. This makes builds unexpectedly ignore missing Brotli even - when `CURL_BROTLI` is enabled. + Test it in 1560 + Fixes #11714 + Reported-by: ad0p on github + Closes #11715 - Closes #11376 +- spelling: use 'reuse' not 're-use' in code and elsewhere -Emanuele Torre (22 Jun 2023) + Unify the spelling as both versions were previously used intermittently -- docs: add more .IP after .RE to fix indentation of generate paragraphs + Closes #11717 - follow-up from 099f41e097c030077b8ec078f2c2d4038d31353b +Michael Osipov (23 Aug 2023) - I just thought of checking all the other files with .RE, and I found 6 - other files that were missing .IP at the end. +- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC - Closes #11375 + HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the + default one. Use "long long" in 32 bit mode and just "long" in 64 bit + mode. -Stefan Eissing (22 Jun 2023) + Closes #11718 -- http2: h2 and h2-PROXY connection alive check fixes +Dan Fandrich (22 Aug 2023) - - fix HTTP/2 check to not declare a connection dead when - the read attempt results in EAGAIN - - add H2-PROXY alive check as for HTTP/2 that was missing - and is needed - - add attach/detach around Curl_conn_is_alive() and remove - these in filter methods - - add checks for number of connections used in some test_10 - proxy tunneling tests +- tests: don't call HTTP errors OK in test cases - Closes #11368 + Some HTTP errors codes were accompanied by the text OK, which causes + some cognitive dissonance when reading them. -- http2: error stream resets with code CURLE_HTTP2_STREAM +- http: close the connection after a late 417 is received - - refs #11357, where it was reported that HTTP/1.1 downgrades - no longer works - - fixed with suggested change - - added test_05_03 and a new handler in the curltest module - to reproduce that downgrades work + In this situation, only part of the data has been sent before aborting + so the connection is no longer usable. - Fixes #11357 - Closes #11362 - Reported-by: Jay Satiro + Assisted-by: Jay Satiro + Fixes #11678 + Closes #11679 -Daniel Stenberg (22 Jun 2023) +- runtests: slightly increase the longest log file displayed -- connect-timeout.d: mention that the DNS lookup is included + The new limit provides enough space for a 64 KiB data block to be logged + in a trace file, plus a few lines at the start and end for context. This + happens to be the amount of data sent at a time in a PUT request. - Closes #11370 +- tests: add delay command to the HTTP server -Emanuele Torre (22 Jun 2023) + This adds a delay after client connect. -- quote.d: fix indentation of generated paragraphs +Daniel Stenberg (22 Aug 2023) - quote.d was missing a .IP at the end which caused the paragraphs - generated for See-also, Multi, and Example to not be indented correctly. +- cirrus: install everthing with pkg, avoid pip - I also remove a redundant "This option can be used multiple times.", and - replaced .IP "item" with .TP .B "item" to make more clear which lines - are part of the list of commands and which aren't. + Assisted-by: Sevan Janiyan - Closes #11371 + Closes #11711 -Paul Wise (22 Jun 2023) +- curl_url*.3: update function descriptions -- checksrc: modernise perl file open + - expand and clarify several descriptions + - avoid using future tense all over - Use regular variables and separate file open modes from filenames. + Closes #11708 - Suggested by perlcritic +- RELEASE-NOTES: synced - Copied from https://github.com/curl/trurl/commit/f2784a9240f47ee28a845 +Stefan Eissing (21 Aug 2023) - Closes #11358 +- CI/cirrus: disable python install on FreeBSD -Dan Fandrich (21 Jun 2023) + - python cryptography package does not build build FreeBSD + - install just mentions "error" + - this gets the build and the main test suite going again -- runtests: work around a perl without SIGUSR1 + Closes #11705 - At least msys2 perl v5.32.1 doesn't seem to define this signal. Since - this signal is only used for debugging, just ignore if setting it fails. +- test2600: fix flakiness on low cpu - Reported-by: Marcel Raad - Fixes #11350 - Closes #11366 + - refs #11355 where failures to to low cpu resources in CI + are reported + - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations + to test cases + - trigger Curl_expire() in test filter to allow re-checks before + the usual 1second interval -- runtests: include missing valgrind package + Closes #11690 - use valgrind was missing which caused torture tests with valgrind - enabled to fail. +Maksim Sciepanienka (20 Aug 2023) - Reported-by: Daniel Stenberg - Fixes #11364 - Closes #11365 +- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf -- runtests: use more consistent failure lines + Closes #11698 - After a test failure log a consistent log message to make it easier to - parse the log file. Also, log a consistent message with "ignored" for - failures that cause the test to be not considered at all. These should - perhaps be counted in the skipped category, but this commit does not - change that behaviour. +Daniel Stenberg (20 Aug 2023) -- runtests: consistently write the test check summary block +- test687/688: two more basic --xattr tests - The memory check character was erroneously omitted if the memory - checking file was not available for some reason, making the block of - characters an inconsistent length. + Closes #11697 -- test2600: fix the description +- cmdline-opts/docs: mentioned the negative option part - It looks like it was cut-and-pasted. + ... for --no-alpn and --no-buffer in the same style done for other --no- + options: - Closes #11354 + "Note that this is the negated option name documented." -Daniel Stenberg (21 Jun 2023) + Closes #11695 -- TODO: "Support HTTP/2 for HTTP(S) proxies" *done* +Emanuele Torre (19 Aug 2023) -humbleacolyte (21 Jun 2023) +- tool/var: also error when expansion result starts with NUL -- cf-socket: move ctx declaration under HAVE_GETPEERNAME + Expansions whose output starts with NUL were being expanded to the empty + string, and not being recognised as values that contain a NUL byte, and + should error. - Closes #11352 + Closes #11694 -Daniel Stenberg (20 Jun 2023) +Daniel Stenberg (19 Aug 2023) -- RELEASE-NOTES: synced +- tests: add 'large-time' as a testable feature -- example/connect-to: show CURLOPT_CONNECT_TO + This allows test cases to require this feature to run and to be used in + %if conditions. - Closes #11340 + Large here means larger than 32 bits. Ie does not suffer from y2038. -Stefan Eissing (20 Jun 2023) + Closes #11696 -- hyper: unslow +- tests/Makefile: add check-translatable-options.pl to tarball - - refs #11203 where hyper was reported as being slow - - fixes hyper_executor_poll to loop until it is out of - tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issue - s/3237 - - added a fix in hyper io handling for detecting EAGAIN - - added some debug logs to see IO results - - pytest http/1.1 test cases pass - - pytest h2 test cases fail on connection reuse. HTTP/2 - connection reuse does not seem to work. Hyper submits - a request on a reused connection, curl's IO works and - thereafter hyper declares `Hyper: [1] operation was canceled: connection cl - osed` - on stderr without any error being logged before. + Used in test 1544 - Fixes #11203 - Reported-by: Gisle Vanem - Advised-by: Sean McArthur - Closes #11344 + Follow-up to ae806395abc8c -- HTTP/2: upload handling fixes +- gen.pl: fix a long version generation mistake - - fixes #11242 where 100% CPU on uploads was reported - - fixes possible stalls on last part of a request body when - that information could not be fully send on the connection - due to an EAGAIN - - applies the same EGAIN handling to HTTP/2 proxying + Too excessive escaping made the parsing not find the correct long names + later and instead add "wrong" links. - Reported-by: Sergey Alirzaev - Fixed #11242 - Closes #11342 + Follow-up to 439ff2052e219 -Daniel Stenberg (20 Jun 2023) + Reported-by: Lukas Tribus + Fixes #11688 + Closes #11689 -- example/opensslthreadlock: remove +- lib: move mimepost data from ->req.p.http to ->state - This shows how to setup OpenSSL mutex callbacks, but this is not - necessary since OpenSSL 1.1.0 - meaning that no currently supported - OpenSSL version requires this anymore + When the legacy CURLOPT_HTTPPOST option is used, it gets converted into + the modem mimpost struct at first use. This data is (now) kept for the + entire transfer and not only per single HTTP request. This re-enables + rewind in the beginning of the second request instead of in end of the + first, as brought by 1b39731. - Closes #11341 + The request struct is per-request data only. -Dan Fandrich (19 Jun 2023) + Extend test 650 to verify. -- libtest: display the times after a test timeout error + Fixes #11680 + Reported-by: yushicheng7788 on github + Closes #11682 - This is to help with test failure debugging. +Patrick Monnerat (17 Aug 2023) - Ref: #11328 - Closes #11329 +- os400: do not check translatable options at build time -- test2600: bump a test timeout + Now that there is a test for this, the build time check is not needed + anymore. - Case 1 failed at least once on GHA by going 30 msec too long. + Closes #11650 - Ref: #11328 +- test1554: check translatable string options in OS400 wrapper -- runtests: better detect and handle pipe errors in the controller + This test runs a perl script that checks all string options are properly + translated by the OS400 character code conversion wrapper. It also + verifies these options are listed in alphanumeric order in the wrapper + switch statement. - Errors reading and writing to the pipes are now better detected and - propagated up to the main test loop so it can be cleanly shut down. Such - errors are usually due to a runner dying so it doesn't make much sense - to try to continue the test run. + Closes #11650 -- runtests: cleanly abort the runner if the controller dies +Daniel Stenberg (17 Aug 2023) - If the controller dies unexpectedly, have the runner stop its servers - and exit cleanly. Otherwise, the orphaned servers will stay running in - the background. +- unit3200: skip testing if function is not present -- runtests: improve error logging + Fake a successful run since we have no easy mechanism to skip this test + for this advanced condition. - Give more information about test harness error conditions to help figure - out what might be wrong. Print some internal test state when SIGUSR1 is - sent to runtests.pl. +- unit2600: fix build warning if built without verbose messages - Ref: #11328 +- test1608: make it build and get skipped without shuffle DNS support -- runtests: better handle ^C during slow tests +- lib: --disable-bindlocal builds curl without local binding support - Since the SIGINT handler now just sets a flag that must be checked in the - main controller loop, make sure that runs periodically. Rather than - blocking on a response from a test runner near the end of the test run, - add a short timeout to allow it. +- test1304: build and skip without netrc support -- runtests: rename server command file +- lib: build fixups when built with most things disabled - The name ftpserver.cmd was historical and has been used for more than - ftp for many years now. Rename it to plain server.cmd to reduce - confusion. + Closes #11687 -- tests: improve reliability of TFTP tests +- workflows/macos.yml: disable zstd and alt-svc in the http-only build - Stop checking the timeout used by the client under test (for most - tests). The timeout will change if the TFTP test server is slow (such as - happens on an overprovisioned CI server) because the client will retry - and reduce its timeout, and the actual value is not important for most - tests. + Closes #11683 - test285 is changed a different way, by increasing the connect timeout. - This improves test coverage by allowing the changed timeout value to be - checked, but improves reliability with a carefully-chosen timeout that - not only allows twice the time to respond as before, but also allows - several retries before the client will change its timeout value. +Stefan Eissing (17 Aug 2023) - Ref: #11328 +- bearssl: handshake fix, provide proper get_select_socks() implementation -Daniel Stenberg (19 Jun 2023) + - bring bearssl handshake times down from +200ms down to other TLS backends + - vtls: improve generic get_select_socks() implementation + - tests: provide Apache with a suitable ssl session cache -- cf-socket: skip getpeername()/getsockname for TFTP + Closes #11675 - Since the socket is not connected then the call fails. When the call - fails, failf() is called to write an error message that is then - surviving and is returned when the *real* error occurs later. The - earlier, incorrect, error therefore hides the actual error message. +- tests: TLS session sharing test - This could be seen in stderr for test 1007 + - test TLS session sharing with special test client + - expect failure with wolfSSL + - disable flaky wolfSSL test_02_07b - Test 1007 has now been extended to verify the stderr message. + Closes #11675 - Closes #11332 +Daniel Stenberg (17 Aug 2023) -- example/crawler: make it use a few more options +- CURLOPT_*TIMEOUT*: extend and clarify - For show, but reasonable + Closes #11686 -- libcurl-ws.3: mention raw mode +- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails - Closes #11339 + And document it. Only return out of memory when it actually is a memory + problem. -- example/default-scheme: set the default scheme for schemeless URLs + Pointed-out-by: Jacob Mealey + Closes #11674 - Closes #11338 +Mathew Benson (17 Aug 2023) -- example/hsts-preload: show one way to HSTS preload +- cmake: add GnuTLS option - Closes #11337 + - Option to use GNUTLS was missing. Hence was not able to use GNUTLS + with ngtcp2 for http3. -- examples/http-options: show how to send "OPTIONS *" + Closes #11685 - With CURLOPT_REQUEST_TARGET. +Daniel Stenberg (16 Aug 2023) - Also add use of CURLOPT_QUICK_EXIT to show. +- RELEASE-NOTES: synced - Closes #11333 +- http: remove the p_pragma struct field -- examples: make use of CURLOPT_(REDIR_|)PROTOCOLS_STR + unused since 40e8b4e52 (2008) - To show how to use them + Closes #11681 - Closes #11334 +Jay Satiro (16 Aug 2023) -- examples/smtp-mime: use CURLOPT_MAIL_RCPT_ALLOWFAILS +- CURLINFO_CERTINFO.3: better explain curl_certinfo struct - For show + Closes https://github.com/curl/curl/pull/11666 - Closes #11335 +- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation -- http: rectify the outgoing Cookie: header field size check + - Remove the out-of-date SSL backend list supported by + CURLOPT_SSL_CTX_FUNCTION. - Previously it would count the size of the entire outgoing request and - not just the size of only the Cookie: header field - which was the - intention. + It makes more sense to just refer to that document instead of having + a separate list that has to be kept in sync. - This could make the check be off by several hundred bytes in some cases. + Closes https://github.com/curl/curl/pull/11665 - Closes #11331 +- write-out.d: clarify %{time_starttransfer} -Jay Satiro (17 Jun 2023) + sync it up with CURLINFO_STARTTRANSFER_TIME_T -- lib: fix some format specifiers +Daniel Stenberg (15 Aug 2023) - - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some - curl_off_t variables. +- transfer: don't set TIMER_STARTTRANSFER on first send - - Use %zu where %zd was erroneously used for some size_t variables. + The time stamp is for measuring the first *received* byte - Prior to this change some of the Windows CI tests were failing because - in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t. - When %zd was used for some curl_off_t variables then only the lower - 32-bits was read and the upper 32-bits would be read for part or all of - the next specifier. + Fixes #11669 + Reported-by: JazJas on github + Closes #11670 - Fixes https://github.com/curl/curl/issues/11327 - Closes https://github.com/curl/curl/pull/11321 +trrui-huawei (15 Aug 2023) -Marcel Raad (16 Jun 2023) +- quiche: enable quiche to handle timeout events -- test427: add `cookies` feature and keyword + In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` + interface for the application to invoke upon timer + expiration. Therefore, invoking the `on_timeout` function of the + Connection is crucial to ensure seamless functionality of quiche with + timeout events. - This test doesn't work with `--disable-cookies`. + Closes #11654 - Closes https://github.com/curl/curl/pull/11320 +- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s -Chris Talbot (15 Jun 2023) + Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. -- imap: Provide method to disable SASL if it is advertised +Daniel Stenberg (15 Aug 2023) - - Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext - LOGIN over SASL auth. +- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang - Prior to this change there was no method to be able to fall back to - LOGIN if an IMAP server advertises SASL capabilities. However, this may - be desirable for e.g. a misconfigured server. + Closes #9580 - Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2 +- imap: add a check for failing strdup() - ";AUTH=" looks to be the correct way to specify what - authenication method to use, regardless of SASL or not. +- imap: remove the only sscanf() call in the IMAP code - Closes https://github.com/curl/curl/pull/10041 + Avoids the use of a stack buffer. -Daniel Stenberg (15 Jun 2023) + Closes #11673 -- RELEASE-NOTES: synced +- imap: use a dynbuf in imap_atom -- examples/multi-debugcallback.c: avoid the bool typedef + Avoid a calculation + malloc. Build the output in a dynbuf. - Apparently this cannot be done in c23 + Closes #11672 - Reported-by: Cristian Rodríguez - Fixes #11299 - Closes #11319 +Marin Hannache (14 Aug 2023) -- docs/libcurl/libcurl.3: cleanups and improvements +- http: do not require a user name when using CURLAUTH_NEGOTIATE - Closes #11317 + In order to get Negotiate (SPNEGO) authentication to work in HTTP you + used to be required to provide a (fake) user name (this concerned both + curl and the lib) because the code wrongly only considered + authentication if there was a user name provided, as in: -- libcurl-ws.3: fix typo + curl -u : --negotiate https://example.com/ -- curl_ws_*.3: enhance + This commit leverages the `struct auth` want member to figure out if the + user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of + setting a user name both in curl and the lib. - - all: SEE ALSO the libcurl-ws man page - - send: add example and return value information - - meta: mention that the returned data is read-only + Signed-off-by: Marin Hannache + Reported-by: Enrico Scholz + Fixes https://sourceforge.net/p/curl/bugs/440/ + Fixes #1161 + Closes #9047 - Closes #11318 +Viktor Szakats (13 Aug 2023) -- docs/libcurl/libcurl-ws.3: see also CURLOPT_WS_OPTIONS +- build: streamline non-UWP wincrypt detections -- docs/libcurl/libcurl-ws.3: minor polish + - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build + and disable our non-UWP-compatible use the Windows crypto API. This + allows to drop two dynamic feature checks. -- libcurl-ws.3. WebSocket API overview + `WINDOWS_STORE` is true when invoking CMake with + `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. - Closes #11314 + Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html -- libcurl-url.3: also mention CURLUPART_ZONEID + - with autotools, drop the separate feature check for `wincrypt.h`. On + one hand this header has been present for long (even Borland C 5.5 had + it from year 2000), on the other we used the check result solely to + enable another check for certain crypto functions. This fails anyway + with the header not present. We save one dynamic feature check at the + configure stage. - ... and sort the two part-using lists alphabetically + Reviewed-by: Marcel Raad + Closes #11657 -Marcel Raad (14 Jun 2023) +Nicholas Nethercote (13 Aug 2023) -- fopen: fix conversion warning on 32-bit Android +- docs/HYPER.md: update hyper build instructions - When building for 32-bit ARM or x86 Android, `st_mode` is defined as - `unsigned int` instead of `mode_t`, resulting in a - -Wimplicit-int-conversion clang warning because `mode_t` is - `unsigned short`. Add a cast to silence the warning. + Nightly Rust and `-Z unstable-options` are not needed. - Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li - bc/include/sys/stat.h#86 - Closes https://github.com/curl/curl/pull/11313 + The instructions here now match the hyper docs exactly: + https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 + 6f88 -- http2: fix variable type + Closes #11662 - `max_recv_speed` is `curl_off_t`, so using `size_t` might result in - -Wconversion GCC warnings for 32-bit `size_t`. Visible in the NetBSD - ARM autobuilds. +Daniel Stenberg (13 Aug 2023) - Closes https://github.com/curl/curl/pull/11312 +- RELEASE-NOTES: synced -Daniel Stenberg (13 Jun 2023) +- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name -- vtls: fix potentially uninitialized local variable warnings + Asssisted-by: Jay Satiro + Closes #11655 - Follow-up from a4a5e438ae533c +- spellcheck: adapt to backslashed minuses - Closes #11310 + As the curl.1 has more backslashed minus, the cleanup sed lines xneed to + adapt. -- timeval: use CLOCK_MONOTONIC_RAW if available + Adjusted some docs slighly. - Reported-by: Harry Sintonen - Ref: #11288 - Closes #11291 + Follow-up to 439ff2052e -Stefan Eissing (12 Jun 2023) + Closes #11663 -- tool: add curl command line option `--trace-ids` +- gen: escape more minus - - added and documented --trace-ids to prepend (after the timestamp) - the transfer and connection identifiers to each verbose log line - - format is [n-m] with `n` being the transfer id and `m` being the - connection id. In case there is not valid connection id, print 'x'. - - Log calls with a handle that has no transfer id yet, are written - without any ids. + Detected since it was still hard to search for option names using dashes + in the middle in the man page. - Closes #11185 + Closes #11660 -- lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID +- cookie-jar.d: enphasize that this option is ONLY writing cookies - - add an `id` long to Curl_easy, -1 on init - - once added to a multi (or its own multi), it gets - a non-negative number assigned by the connection cache - - `id` is unique among all transfers using the same - cache until reaching LONG_MAX where it will wrap - around. So, not unique eternally. - - CURLINFO_CONN_ID returns the connection id attached to - data or, if none present, data->state.lastconnect_id - - variables and type declared in tool for write out + Reported-by: Dan Jacobson + Tweaked-by: Jay Satiro + Ref: #11642 + Closes #11661 - Closes #11185 +Nicholas Nethercote (11 Aug 2023) -Daniel Stenberg (12 Jun 2023) +- docs/HYPER.md: document a workaround for a link error -- CURLOPT_INFILESIZE.3: mention -1 triggers chunked + Closes #11653 - Ref: #11300 - Closes #11304 +Jay Satiro (11 Aug 2023) -Philip Heiduck (12 Jun 2023) +- schannel: verify hostname independent of verify cert -- CI: openssl-3.0.9+quic + Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off + and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the + hostname in schannel code. - Closes #11296 + This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and + verify hostname". We discussed a fix several years ago in #3285 but it + went stale. -Karthikdasari0423 (12 Jun 2023) + Assisted-by: Daniel Stenberg -- HTTP3.md: update openssl version + Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html + Reported-by: Martin Galvan - Closes #11297 + Ref: https://github.com/curl/curl/pull/3285 -Daniel Stenberg (12 Jun 2023) + Fixes https://github.com/curl/curl/issues/3284 + Closes https://github.com/curl/curl/pull/10056 -- vtls: avoid memory leak if sha256 call fails +Daniel Stenberg (11 Aug 2023) - ... in the pinned public key handling function. +- curl_quiche: remove superfluous NULL check - Reported-by: lizhuang0630 on github - Fixes #11306 - Closes #11307 + 'stream' is always non-NULL at this point -- examples/ipv6: disable on win32 + Pointed out by Coverity - I can't make if_nametoindex() work there + Closes #11656 - Follow-up to c23dc42f3997acf23 +- curl/urlapi.h: tiny typo - Closes #11305 +- github/labeler: make HYPER.md set Hyper and not TLS -- tool_operate: allow cookie lines up to 8200 bytes +- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 - Since this option might set multiple cookies in the same line, it does - not make total sense to cap this at 4096 bytes, which is the limit for a - single cookie name or value. + 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need + to specify version changes for earlier releases in the generated output. - Closes #11303 + This ups the limit from the previous 7.30.0 (Apr 12 2013) -- test427: verify sending more cookies than fit in a 8190 bytes line + This hides roughly 35 "added in" mentions. - curl will then only populate the header with cookies that fit, dropping - ones that otherwise would have been sent + Closes #11651 - Ref: https://curl.se/mail/lib-2023-06/0020.html +Jay Satiro (10 Aug 2023) - Closes #11303 +- bug_report: require reporters to specify curl and os versions -- testutil: allow multiple %-operators on the same line + - Change curl version and os sections from single-line input to + multi-line textarea. - Closes #11303 + - Require curl version and os sections to be filled out before report + can be submitted. -Oleg Jukovec (12 Jun 2023) + Closes https://github.com/curl/curl/pull/11636 -- docs: update CURLOPT_UPLOAD.3 +Daniel Stenberg (9 Aug 2023) - The behavior of CURLOPT_UPLOAD differs from what is described in the - documentation. The option automatically adds the 'Transfer-Encoding: - chunked' header if the upload size is unknown. +- gen.pl: replace all single quotes with aq - Closes #11300 + - this prevents man from using a unicode sequence for them + - which then allows search to work properly -Daniel Stenberg (12 Jun 2023) + Closes #11645 -- RELEASE-NOTES: synced +Viktor Szakats (9 Aug 2023) -- CURLOPT_AWS_SIGV4.3: remove unused variable from example +- cmake: fix to use variable for the curl namespace - Closes #11302 + Replace (wrong) literal with a variable to specify the curl + namespace. -- examples/https.c: use CURLOPT_CA_CACHE_TIMEOUT + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 - for demonstration purposes + Reported-by: balikalina on Github + Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 + 20d30#r123923098 + Closes #11629 - Closes #11290 +- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms -- example/ipv6: feature CURLOPT_ADDRESS_SCOPE in use + 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing + libcurl objects for shared and static targets. - Closes #11282 + The above automatically enabled for Windows builds, with an option to + disable with `SHARE_LIB_OBJECT=OFF`. -Karthikdasari0423 (10 Jun 2023) + This patch extend this feature to all platforms as a manual option. + You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects + are built in PIC mode, meaning the static lib will also have PIC code. -- docs: Update HTTP3.md for newer ngtcp2 and nghttp3 + [EXPERIMENTAL] - Follow-up to fb9b9b58 + Closes #11627 - Ref: #11184 - Closes #11295 +- cmake: assume `wldap32` availability on Windows -Dan Fandrich (10 Jun 2023) + This system library first shipped with Windows ME, available as an extra + install for some older releases (according to [1]). The import library + was present already in old MinGW 3.4.2 (year 2007). -- docs: update the supported ngtcp2 and nghttp3 versions + Drop the feature check and its associated `HAVE_WLDAP32` variable. - Follow-up to cae9d10b + To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` + CMake option, like before. - Ref: #11184 - Closes #11294 + [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html -- tests: fix error messages & handling around sockets + Reviewed-by: Jay Satiro + Closes #11624 - The wrong error code was checked on Windows on UNIX socket failures, - which could have caused all UNIX sockets to be reported as having - errored and the tests therefore skipped. Also, a useless error message - was displayed on socket errors in many test servers on Windows because - strerror() doesn't work on WinSock error codes; perror() is overridden - there to work on all errors and is used instead. +Daniel Stenberg (9 Aug 2023) - Ref #11258 - Closes #11265 +- page-header: move up a URL paragraph from GLOBBING to URL -Daniel Stenberg (9 Jun 2023) +- variable.d: output the function names table style -- CURLOPT_SSH_PRIVATE_KEYFILE.3: expand on the file search + Also correct the url function name in the header - Reported-by: atjg on github - Ref: #11287 - Closes #11289 + Closes #11641 -Stefan Eissing (9 Jun 2023) +- haproxy-clientip.d: remove backticks -- ngtcp2: use ever increasing timestamp in io + This is not markdown - - ngtcp2 v0.16.0 asserts that timestamps passed to its function - will only ever increase. - - Use a context shared between ingress/egress operations that - uses a shared timestamp, regularly updated during calls. + Follow-up to 0a75964d0d94a4 - Closes #11288 + Closes #11639 -Daniel Stenberg (9 Jun 2023) +- RELEASE-NOTES: synced -- GHA: use nghttp2 1.54.0 for the ngtcp2 jobs +- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens -Philip Heiduck (9 Jun 2023) + Reported-by: FC Stegerman + Fixes #11635 + Closes #11637 -- GHA: ngtcp2: use 0.16.0 and nghttp3 0.12.0 +- cmdline-opts/page-header: reorder, clean up -Daniel Stenberg (9 Jun 2023) + - removed some unnecessary blurb to focus + - moved up the more important URL details + - put "globbing" into its own subtitle and moved down a little + - mention the online man page in the version section -- ngtcp2: build with 0.16.0 and nghttp3 0.12.0 + Closes #11638 - - moved to qlog_write - - crypto => encryption - - CRYPTO => ENCRYPTION - - removed "_is_" - - ngtcp2_conn_shutdown_stream_read and - ngtcp2_conn_shutdown_stream_write got flag arguments - - the nghttp3_callbacks struct got a recv_settings callback +- c-hyper: adjust the hyper to curlcode conversion - Closes #11184 + Closes #11621 -- example/http2-download: set CURLOPT_BUFFERSIZE +- test2306: make it use a persistent connection - Primarily because no other example sets it, and remove the disabling of - the certificate check because we should not recommend that. + + enable verbose already from the start - Closes #11284 + Closes #11621 -- example/crawler: also set CURLOPT_AUTOREFERER +eppesuig (8 Aug 2023) - Could make sense, and it was not used in any example before. +- list-only.d: mention SFTP as supported protocol - Closes #11283 + Closes #11628 -Wyatt OʼDay (9 Jun 2023) +Daniel Stenberg (8 Aug 2023) -- tls13-ciphers.d: include Schannel +- request.d: use .TP for protocol "labels" - Closes #11271 + To render the section nicer in man page. -Daniel Stenberg (9 Jun 2023) + Closes #11630 -- curl_pushheader_byname/bynum.3: document in their own man pages +- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP - These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was - introduced but always lived a life in the shadows, embedded in the - CURLMOPT_PUSHFUNCTION man page. Until now. + ... as documented. - It makes better sense and gives more visibility to document them in - their own stand-alone man pages. + Update test 3201 and 3202 accordingly. - Closes #11286 + Reported-by: Markus Sommer + Fixes #11619 + Closes #11626 -- curl_mprintf.3: minor fix of the example +- page-footer: QLOGDIR works with ngtcp2 and quiche -- curl_url_set: enforce the max string length check for all parts + It previously said "both" backends which is confusing as we currently + have three... - Update the docs and test 1559 accordingly + Closes #11631 - Closes #11273 +Stefan Eissing (8 Aug 2023) -- examples/ftpuploadresume.c: add use of CURLOPT_ACCEPTTIMEOUT_MS +- http3: quiche, handshake optimization, trace cleanup - For show + - load x509 store after clienthello + - cleanup of tracing - Closes #11277 + Closes #11618 -- examples/unixsocket.c: example using CURLOPT_UNIX_SOCKET_PATH +Daniel Stenberg (8 Aug 2023) - and alternatively CURLOPT_ABSTRACT_UNIX_SOCKET +- ngtcp2: remove dead code - Closes #11276 + 'result' is always zero (CURLE_OK) at this point -Anssi Kolehmainen (8 Jun 2023) + Detected by Coverity -- docs: fix missing parameter names in examples + Closes #11622 - Closes #11278 +Viktor Szakats (8 Aug 2023) -Daniel Stenberg (8 Jun 2023) +- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` -- urlapi: have *set(PATH) prepend a slash if one is missing + OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the + popular forks (yet). Use the macro itself to detect its presence, + replacing the hard-wired fork-specific conditions. - Previously the code would just do that for the path when extracting the - full URL, which made a subsequent curl_url_get() of the path to - (unexpectedly) still return it without the leading path. + This way the feature will enable automatically when forks implement it, + while also shorter and possibly requiring less future maintenance. - Amend lib1560 to verify this. Clarify the curl_url_set() docs about it. + Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 - Bug: https://curl.se/mail/lib-2023-06/0015.html - Closes #11272 - Reported-by: Pedro Henrique + Reviewed-by: Jay Satiro + Closes #11617 -Dan Fandrich (7 Jun 2023) +- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 -- runtests; give each server a unique log lock file + LibreSSL 3.4.1 (2021-10-14) added support for + `SSL_CTX_set_ciphersuites`. - Logs are written by several servers and all of them must be finished - writing before the test results can be determined. This means each - server must have its own lock file rather than sharing a single one, - which is how it was done up to now. Previously, the first server to - complete a test would clear the lock before the other server was done, - which caused flaky tests. + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt - Lock files are now all found in their own directory, so counting locks - equals counting the files in that directory. The result is that the - proxy logs are now reliably written which actually changes the expected - output for two tests. + Reviewed-by: Jay Satiro + Closes #11616 - Fixes #11231 - Closes #11259 +- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 -- runtests: make test file directories in log/N + LibreSSL 3.5.0 (2022-02-24) added support for + `SSL_CTX_set_keylog_callback`. - Test files in subdirectories were not created after parallel test log - directories were moved down a level due to a now-bad comparison. + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt - Follow-up to 92d7dd39 + Reviewed-by: Jay Satiro + Closes #11615 - Ref #11264 - Closes #11267 +- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks -Daniel Stenberg (7 Jun 2023) + - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is + also not used by curl, but it is by its optional dependency `librtmp`. + Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This + library has been available since the early days of Windows. -- ws: make the curl_ws_meta() return pointer a const + - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present + since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and + other existing logic already assumed this lib being present, so delete + the check and replace the detection variable with `WIN32` and always + add `ws2_32` on Windows. - The returned info is read-only for the user. + Closes #11612 - Closes #11261 +Daniel Gustafsson (8 Aug 2023) -- RELEASE-NOTES: synced +- crypto: ensure crypto initialization works -- runtests: move parallel log dirs from logN to log/N + Make sure that context initialization during hash setup works to avoid + going forward with the risk of a null pointer dereference. - Having several hundreds of them in there gets annoying. + Reported-by: Philippe Antoine on HackerOne + Assisted-by: Jay Satiro + Assisted-by: Daniel Stenberg - Closes #11264 + Closes #11614 -Dan Fandrich (7 Jun 2023) +Viktor Szakats (7 Aug 2023) -- test447: move the test file into %LOGDIR +- openssl: switch to modern init for LibreSSL 2.7.0+ -Viktor Szakats (7 Jun 2023) + LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, + `OPENSSL_init_ssl()` function and deprecated the old, manual init + method, as seen in OpenSSL 1.1.0. Switch to the modern method when + available. -- cmake: add support for "unity" builds + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt - Aka "jumbo" or "amalgamation" builds. It means to compile all sources - per target as a single C source. This is experimental. + Reviewed-by: Daniel Stenberg + Closes #11611 - You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. - It requires CMake 3.16 or newer. +Daniel Stenberg (7 Aug 2023) - It makes builds (much) faster, allows for better optimizations and tends - to promote less ambiguous code. +- gskit: remove - Also add a new AppVeyor CI job and convert an existing one to use - "unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job. + We remove support for building curl with gskit. - Fix related issues: - - add missing include guard to `easy_lock.h`. - - rename static variables and functions (and a macro) with names reused - across sources, or shadowed by local variables. - - add an `#undef` after use. - - add a missing `#undef` before use. - - move internal definitions from `ftp.h` to `ftp.c`. - - `curl_memory.h` fixes to make it work when included repeatedly. - - stop building/linking curlx bits twice for a static-mode curl tool. - These caused doubly defined symbols in unity builds. - - silence missing extern declarations compiler warning for ` _CRT_glob`. - - fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`. - - fix colliding static symbols in debug mode: `debugtime()` and - `statename`. - - rename `ssl_backend_data` structure to unique names for each - TLS-backend, along with the `ssl_connect_data` struct member - referencing them. This required adding casts for each access. - - add workaround for missing `[P]UNICODE_STRING` types in certain Windows - builds when compiling `lib/ldap.c`. To support "unity" builds, we had - to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows - `schannel.h` option) _globally_. This caused an indirect inclusion of - Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled - as well. This requires `[P]UNICODE_STRING` types, which is apperantly - not defined automatically (as seen with both MSVS and mingw-w64). - This patch includes `` to fix it. - Ref: https://github.com/curl/curl/runs/13987772013 - Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&vie - w=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f - 38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c - - tweak unity builds to compile `lib/memdebug.c` separately in memory - trace builds to avoid PP confusion. - - force-disable unity for test programs. - - do not compile and link libcurl sources to libtests _twice_ when libcurl - is built in static mode. + - This is a niche TLS library, only running on some IBM systems + - no regular curl contributors use this backend + - no CI builds use or verify this backend + - gskit, or the curl adaption for it, lacks many modern TLS features + making it an inferior solution + - build breakages in this code take weeks or more to get detected + - fixing gskit code is mostly done "flying blind" - KNOWN ISSUES: - - running tests with unity builds may fail in cases. - - some build configurations/env may not compile in unity mode. E.g.: - https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfu - auwl8q#L250 + This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has + been mentioned on the curl-library mailing list. - Ref: https://github.com/libssh2/libssh2/issues/1034 - Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html - Ref: https://en.wikipedia.org/wiki/Unity_build + It could be brought back, this is not a ban. Given proper effort and + will, gskit support is welcome back into the curl TLS backend family. - Closes #11095 + Closes #11460 -Daniel Stenberg (7 Jun 2023) +- RELEASE-NOTES: synced -- examples/websocket.c: websocket example using CONNECT_ONLY +Dan Fandrich (7 Aug 2023) - Closes #11262 +- THANKS-filter: add a name typo -- websocket-cb: example doing WebSocket download using callback +Stefan Eissing (7 Aug 2023) - Very basic +- http3/ngtcp2: shorten handshake, trace cleanup - Closes #11260 + - shorten handshake timing by delayed x509 store load (OpenSSL) + as we do for HTTP/2 + - cleanup of trace output, align with HTTP/2 output -- test/.gitignore: ignore log* + Closes #11609 -Dan Fandrich (5 Jun 2023) +Daniel Stenberg (7 Aug 2023) -- runtests: document the -j parallel testing option +- headers: accept leading whitespaces on first response header - Reported-by: Daniel Stenberg - Ref: #10818 - Closes #11255 + This is a bad header fold but since the popular browsers accept this + violation, so does curl now. Unless built with hyper. -- runtests: create multiple test runners when requested + Add test 1473 to verify and adjust test 2306. - Parallel testing is enabled by using a nonzero value for the -j option - to runtests.pl. Performant values seem to be about 7*num CPU cores, or - 1.3*num CPU cores if Valgrind is in use. + Reported-by: junsik on github + Fixes #11605 + Closes #11607 - Flaky tests due to improper log locking (bug #11231) are exacerbated - while parallel testing, so it is not enabled by default yet. +- include/curl/mprintf.h: add __attribute__ for the prototypes - Fixes #10818 - Closes #11246 + - if gcc or clang is used + - if __STDC_VERSION__ >= 199901L, which means greater than C90 + - if not using mingw + - if CURL_NO_FMT_CHECKS is not defined -- runtests: handle repeating tests in multiprocess mode + Closes #11589 - Such as what happens with the --repeat option. Some functions are - changed to pass the runner ID instead of relying on the non-unique test - number. +- tests: fix bad printf format flags in test code - Ref: #10818 +- tests: fix header scan tools for attribute edits in mprintf.h -- runtests: buffer logmsg while running singletest() +- cf-socket: log successful interface bind - This allows all messages relating to a single test case to be displayed - together at the end of the test. + When the setsockopt SO_BINDTODEVICE operation succeeds, output that in + the verbose output. - Ref: #10818 + Ref: #11599 + Closes #11608 -- runtests: call initserverconfig() in the runner +- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled - This must be done so variables pick up the runner's unique $LOGDIR. + Ref: #11457 + Closes #11606 - Ref: #10818 +- CURLOPT_SSL_VERIFYPEER.3: add two more see also options -- runtests: use a per-runner random seed + CURLINFO_CAINFO and CURLINFO_CAPATH - Each runner needs a unique random seed to reduce the chance of port - number collisions. The new scheme uses a consistent per-runner source of - randomness which results in deterministic behaviour, as it did before. + Closes #11603 - Ref: #10818 +- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice -- runtests: complete main test loop refactor for multiple runners + Closes #11007 - The main test loop is now able to handle multiple runners, or no - additional runner processes at all. At most one process is still - created, however. +Graham Campbell (6 Aug 2023) - Ref: #10818 +- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 -- runtests: prepare main test loop for multiple runners + Closes #11585 - Some variables are expanded to arrays and hashes so that multiple - runners can be used for running tests. +Daniel Stenberg (6 Aug 2023) - Ref: #10818 +- TODO: add *5* entries for aws-sigv4 -Stefan Eissing (5 Jun 2023) + Closes #7559 + Closes #8107 + Closes #8810 + Closes #9717 + Closes #10129 -- bufq: make write/pass methods more robust +- TODO: LDAP Certificate-Based Authentication - - related to #11242 where curl enters busy loop when - sending http2 data to the server + Closes #9641 - Closes #11247 +Stefan Eissing (6 Aug 2023) -Boris Verkhovskiy (5 Jun 2023) +- http2: cleanup trace messages -- tool_getparam: fix comment + - more compact format with bracketed stream id + - all frames traced in and out - Closes #11253 + Closes #11592 -Raito Bezarius (5 Jun 2023) +Daniel Stenberg (6 Aug 2023) -- haproxy: add --haproxy-clientip flag to spoof client IPs +- tests/tftpd+mqttd: make variables static to silence picky warnings - CURLOPT_HAPROXY_CLIENT_IP in the library + Closes #11594 - Closes #10779 +- docs/cmdline: remove repeated working for negotiate + ntlm -Daniel Stenberg (5 Jun 2023) + The extra wording is added automatically by the gen.pl tool -- curl: add --ca-native and --proxy-ca-native + Closes #11597 - These are two boolean options to ask curl to use the native OS's CA - store when verifying TLS servers. For peers and for proxies - respectively. +- docs/cmdline: add small "warning" to verbose options - They currently only have an effect for curl on Windows when built to use - OpenSSL for TLS. + "Note that verbose output of curl activities and network traffic might + contain sensitive data, including user names, credentials or secret data + content. Be aware and be careful when sharing trace logs with others." - Closes #11049 + Closes #11596 -Viktor Szakats (5 Jun 2023) +- RELEASE-NOTES: synced -- build: drop unused/redundant `HAVE_WINLDAP_H` +- pingpong: don't use *bump_headersize - Sources did not use it. Autotools used it when checking for the - `winldap` library, which is redundant. + We use that for HTTP(S) only. - With CMake, detection was broken: - ``` - Run Build Command(s):/usr/local/Cellar/cmake/3.26.3/bin/cmake -E env VERBOSE= - 1 /usr/bin/make -f Makefile cmTC_2d8fe/fast && /Library/Developer/CommandLine - Tools/usr/bin/make -f CMakeFiles/cmTC_2d8fe.dir/build.make CMakeFiles/cmTC_2 - d8fe.dir/build - Building C object CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj - /usr/local/opt/llvm/bin/clang --target=x86_64-w64-mingw32 --sysroot=/usr/loca - l/opt/mingw-w64/toolchain-x86_64 -D_WINSOCKAPI_="" -I/my/quictls/x64-ucrt/usr - /include -I/my/zlib/x64-ucrt/usr/include -I/my/brotli/x64-ucrt/usr/include -W - no-unused-command-line-argument -D_UCRT -DCURL_HIDDEN_SYMBOLS -DHAVE_SSL_SE - T0_WBIO -DHAS_ALPN -DNGHTTP2_STATICLIB -DNGHTTP3_STATICLIB -DNGTCP2_STATICLIB - -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static-libgcc -lucrt -Wextra -Wall -p - edantic -Wbad-function-cast -Wconversion -Winline -Wmissing-declarations -Wmi - ssing-prototypes -Wnested-externs -Wno-long-long -Wno-multichar -Wpointer-ari - th -Wshadow -Wsign-compare -Wundef -Wunused -Wwrite-strings -Wcast-align -Wde - claration-after-statement -Wempty-body -Wendif-labels -Wfloat-equal -Wignored - -qualifiers -Wno-format-nonliteral -Wno-sign-conversion -Wno-system-headers - - Wstrict-prototypes -Wtype-limits -Wvla -Wshift-sign-overflow -Wshorten-64-to- - 32 -Wdouble-promotion -Wenum-conversion -Wunused-const-variable -Wcomma -Wmis - sing-variable-declarations -Wassign-enum -Wextra-semi-stmt -MD -MT CMakeFile - s/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -MF CMakeFiles/cmTC_2d8fe.dir/HAVE_WINL - DAP_H.c.obj.d -o CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -c /my/curl/b - ld-cmake-llvm-x64-shared/CMakeFiles/CMakeScratch/TryCompile-3JP6dR/HAVE_WINLD - AP_H.c - In file included from /my/curl/bld-cmake-llvm-x64-shared/CMakeFiles/CMakeScra - tch/TryCompile-3JP6dR/HAVE_WINLDAP_H.c:2: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/winldap.h:17: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/schnlsp.h:9: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/schannel.h:10: - /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt - .h:5041:254: error: unknown type name 'PSYSTEMTIME' - WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ - NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, - DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER - pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT - ENSIONS pExtensions); - - - - ^ - /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt - .h:5041:278: error: unknown type name 'PSYSTEMTIME' - WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ - NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, - DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER - pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT - ENSIONS pExtensions); - - - - ^ - 2 errors generated. - make[1]: *** [CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj] Error 1 - make: *** [cmTC_2d8fe/fast] Error 2 - exitCode: 2 - ``` + Follow-up to 3ee79c1674fd6 - Cherry-picked from #11095 88e4a21ff70ccef391cf99c8165281ff81374503 - Reviewed-by: Daniel Stenberg - Closes #11245 + Closes #11590 -Daniel Stenberg (5 Jun 2023) +- urldata: remove spurious parenthesis to unbreak no-proxy build -- urlapi: scheme starts with alpha + Follow-up to e12b39e13382 - Add multiple tests to lib1560 to verify + Closes #11591 - Fixes #11249 - Reported-by: ad0p on github - Closes #11250 +- easy: don't call Curl_trc_opt() in disabled-verbose builds -- RELEASE-NOTES: synced + Follow-up to e12b39e133822c6a0 -- CURLOPT_MAIL_RCPT_ALLOWFAILS: replace CURLOPT_MAIL_RCPT_ALLLOWFAILS + Closes #11588 - Deprecate the name using three Ls and prefer the name with two. +- http: use %u for printfing int - Replaces #10047 - Closes #11218 + Follow-up to 3ee79c1674fd6f99e8efca5 -- tests/servers: generate temp names in /tmp for unix domain sockets + Closes #11587 - ... instead of putting them in the regular pid directories because - systems generally have strict length requirements for the path name to - be shorter than 107 bytes and we easily hit that boundary otherwise. +Goro FUJI (3 Aug 2023) - The new concept generates two random names: one for the socks daemon and - one for http. +- vquic: show stringified messages for errno - Reported-by: Andy Fiddaman - Fixes #11152 - Closes #11166 + Closes #11584 -Stefan Eissing (2 Jun 2023) +Stefan Eissing (3 Aug 2023) -- http2: better support for --limit-rate +- trace: make tracing available in non-debug builds - - leave transfer loop when --limit-rate is in effect and has - been received - - adjust stream window size to --limit-rate plus some slack - to make the server observe the pacing we want - - add test case to confirm behaviour + Add --trace-config to curl - Closes #11115 + Add curl_global_trace() to libcurl -- curl_log: evaluate log statement only when transfer is verbose + Closes #11421 - Closes #11238 +Daniel Stenberg (3 Aug 2023) -Daniel Stenberg (2 Jun 2023) +- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" -- libssh2: provide error message when setting host key type fails + See also https://github.com/curl/curl/pull/7507 - Ref: https://curl.se/mail/archive-2023-06/0001.html +- TODO: add "WebSocket read callback" - Closes #11240 + remove "Upgrade to websockets" as we already have this -Igor Todorovski (2 Jun 2023) + Closes #11402 -- system.h: remove __IBMC__/__IBMCPP__ guards and apply to all z/OS compiles +- test497: verify rejecting too large incoming headers - Closes #11241 +- http: return error when receiving too large header set -Daniel Stenberg (2 Jun 2023) + To avoid abuse. The limit is set to 300 KB for the accumulated size of + all received HTTP headers for a single response. Incomplete research + suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to + 1MB. -- docs/SECURITY-PROCESS.md: link to example of previous critical flaw + Closes #11582 -Mark Seuffert (2 Jun 2023) +Stefan Eissing (3 Aug 2023) -- README.md: updated link to opencollective +- http2: upgrade tests and add fix for non-existing stream - Closes #11232 + - check in h2 filter recv that stream actually exists + and return error if not + - add test for parallel, extreme h2 upgrades that fail if + connections get reused before fully switched + - add h2 upgrade upload test just for completeness -Daniel Stenberg (1 Jun 2023) + Closes #11563 -- libssh2: use custom memory functions +Viktor Szakats (3 Aug 2023) - Because of how libssh2_userauth_keyboard_interactive_ex() works: the - libcurl callback allocates memory that is later free()d by libssh2, we - must set the custom memory functions. +- tests: ensure `libcurl.def` contains all exports + + Add `test1279` to verify that `libcurl.def` lists all exported API + functions found in libcurl headers. - Reverts 8b5f100db388ee60118c08aa28 + Also: - Ref: https://github.com/libssh2/libssh2/issues/1078 - Closes #11235 + - extend test suite XML `stdout` tag with the `loadfile` attribute. -- test447: test PUTting a file that grows + - fix `tests/extern-scan.pl` and `test1135` to include websocket API. - ... and have curl trim the end when it reaches the expected total amount - of bytes instead of over-sending. + - use all headers (sorted) in `test1135` instead of a manual list. - Reported-by: JustAnotherArchivist on github - Closes #11223 + - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. -- curl: count uploaded data to stop at the originally given size + - add `libcurl.def` to the auto-labeler GHA task. - Closes #11223 - Fixes #11222 - Reported-by: JustAnotherArchivist on github + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 -- tool: remove exclamation marks from error/warning messages + Closes #11570 -- tool: use errorf() for error output +Daniel Stenberg (2 Aug 2023) - Convert a number of fprintf() calls. +- url: change default value for CURLOPT_MAXREDIRS to 30 -- tool: remove newlines from all helpf/notef/warnf/errorf calls + It was previously unlimited by default, but that's not a sensible + default. While changing this has a remote risk of breaking an existing + use case, I figure it is more likely to actually save users from loops. - Make voutf() always add one. + Closes #11581 - Closes #11226 +- lib: fix a few *printf() flag mistakes -- tests/servers.pm: pick unused port number with a server socket + Reported-by: Gisle Vanem + Ref: #11574 + Closes #11579 - This change replaces the previous method of picking a port number at - random to try to start servers on, then retrying up to ten times with - new random numbers each time, with a function that creates a server - socket on port zero, thereby getting a suitable random port set by the - kernel. That server socket is then closed and that port number is used - to setup the actual test server on. +Samuel Chiang (2 Aug 2023) - There is a risk that *another* server can be started on the machine in - the time gap, but the server verification feature will detect that. +- openssl: make aws-lc version support OCSP - Closes #11220 + And bump version in CI -- RELEASE-NOTES: synced + Closes #11568 - bump to 8.2.0 +Daniel Stenberg (2 Aug 2023) -Alejandro R. Sedeño (31 May 2023) +- tool: make the length argument an int for printf()-.* flags -- configure: fix run-compiler for old /bin/sh + Closes #11578 - If you try to assign and export on the same line on some older /bin/sh - implementations, it complains: +- tool_operate: fix memory leak when SSL_CERT_DIR is used - ``` - $ export "NAME=value" - NAME=value: is not an identifier - ``` + Detected by Coverity - This commit rewrites run-compiler's assignments and exports to work with - old /bin/sh, splitting assignment and export into two separate - statements, and only quote the value. So now we have: + Follow-up to 29bce9857a12b6cfa726a5 - ``` - NAME="value" - export NAME - ``` + Closes #11577 - While we're here, make the same change to the two supporting - assign+export lines preceeding the script to be consistent with how - exports work throughout the rest of configure.ac. +- tool/var: free memory on OOM - Closes #11228 + Coverity detected this memory leak in OOM situation -Philip Heiduck (31 May 2023) + Follow-up to 2e160c9c652504e -- circleci: install impacket & wolfssl 5.6.0 + Closes #11575 - Closes #11221 +Viktor Szakats (2 Aug 2023) -Daniel Stenberg (31 May 2023) +- gha: bump libressl and mbedtls versions -- tool_urlglob: use curl_off_t instead of longs + Closes #11573 - To handle more globs better (especially on Windows) +Jay Satiro (2 Aug 2023) - Closes #11224 +- schannel: fix user-set legacy algorithms in Windows 10 & 11 -Dan Fandrich (30 May 2023) + - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then + use the SCHANNEL_CRED legacy structure to pass the list to Schannel. -- scripts: Fix GHA matrix job detection in cijobs.pl + - If the user set both a legacy algorithm list and a TLS 1.3 cipher list + then abort. - The parsing is pretty brittle and it broke detecting some jobs at some - point. Also, detect if Windows is used in GHA. + Although MS doesn't document it, Schannel will not negotiate TLS 1.3 + when SCHANNEL_CRED is used. That means setting a legacy algorithm list + limits the user to earlier versions of TLS. -- runtests: abort test run after failure without -a + Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would + ignore legacy algorithms in Windows 10 1809 and later. - This was broken in a recent refactor and test runs would not stop. + Reported-by: zhihaoy@users.noreply.github.com - Follow-up to d4a1b5b6 + Fixes https://github.com/curl/curl/pull/10741 + Closes https://github.com/curl/curl/pull/10746 - Reported-by: Daniel Stenberg - Fixes #11225 - Closes #11227 +Daniel Stenberg (2 Aug 2023) -Version 8.1.2 (30 May 2023) +- variable.d: setting a variable again overwrites it -Daniel Stenberg (30 May 2023) + Reported-by: Niall McGee + Bug: https://twitter.com/niallmcgee/status/1686523075423322113 + Closes #11571 -- RELEASE-NOTES: synced +Jay Satiro (2 Aug 2023) - 8.1.2 release +- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting -- THANKS: contributors from 8.1.2 + - Re-wrap CURLSSLOPT_ALLOW_BEAST description. -- lib1560: verify more scheme guessing +Daniel Stenberg (2 Aug 2023) - - on 2nd level domains - - on names without dots +- RELEASE-NOTES: synced - As mentioned in #11161, "imap.com" will be guessed IMAP +- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set - Closes #11219 + Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not + used, thus unnecessarily asking for addresses that will not be used. -- page-header: minor wording polish in the URL segment + Reported-by: Joseph Tharayil + Fixes #11564 + Closes #11565 - Closes #11217 +- docs: link to the website versions instead of markdowns -- page-header: mention curl version and how to figure out current release + ... to make the links work when the markdown is converted to webpages on + https://curl.se - Closes #11216 + Reported-by: Maurício Meneghini Fauth + Fixes https://github.com/curl/curl-www/issues/272 + Closes #11569 -- RELEASE-NOTES: synced +Viktor Szakats (1 Aug 2023) -- configure: without pkg-config and no custom path, use -lnghttp2 +- cmake: cache more config and delete unused ones - Reported-by: correctmost on github - Fixes #11186 - Closes #11210 + - cache more Windows config results for faster initialization. -Stefan Eissing (28 May 2023) + - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. -- curl: cache the --trace-time value for a second + - delete dead references to `sys/utsname.h`. - - caches HH:MM:SS computed and reuses it for logging during - the same second. - - common function for plain log line start formatting + Closes #11551 - Closes #11211 +- egd: delete feature detection and related source code -Kev Jackson (28 May 2023) + EGD is Entropy Gathering Daemon, a socket-based entropy source supported + by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it + a while ago. -- libcurl.m4: remove trailing 'dnl' that causes this to break autoconf + Its detection in CMake was broken all along because OpenSSL libs were + not linked at the point of feature check. - Closes #11212 + Delete detection from both cmake and autotools, along with the related + source snippet, and the `--with-egd-socket=` `./configure` option. -Stefan Eissing (26 May 2023) + Closes #11556 -- http3: send EOF indicator early as possible +Stefan Eissing (1 Aug 2023) - - ngtcp2 and quiche implementations relied on the DONE_SEND event - to forward the EOF for uploads to the libraries. This often - result in a last 0 length EOF data. Tracking the amount of - data left to upload allows EOF indication earlier. - - refs #11205 where CloudFlare DoH servers did not like to - receive the initial upload DATA without EOF and returned - a 400 Bad Request +- tests: fix h3 server check and parallel instances - Reported-by: Sergey Fionov - Fixes #11205 - Closes #11207 + - fix check for availability of nghttpx server + - add `tcp` frontend config for same port as quic, as + without this, port 3000 is bound which clashes for parallel + testing -Daniel Stenberg (26 May 2023) + Closes #11553 -- scripts/contri*sh: no longer grep -v ' ' +Daniel Stenberg (1 Aug 2023) - Originally these scripts filtered out names that have no space so that - they better avoid nick names not intended for credits. Such names are - not too commonly used, plus we now give credit even to those. +- docs/cmdline-opts: spellfixes, typos and polish - Additionally: non-latin names, like Asian, don't have spaces at all so - they were also filtered out and had to be manually added which made it - an error-prone operation where Asian names eventually easily fell off by - mistake. + To make them accepted by the spell checker - Closes #11206 + Closes #11562 -- cf-socket: restore Curl_sock_assign_addr() +- CI/spellcheck: build curl.1 and spellcheck it - Regression since it was not private. Also used by msh3.c + Added acceptable words - Follow-up to 8e85764b7bd7f05f5 - Reported-by: Gisle Vanem - Fixes #11202 - Closes #11204 + Closes #11562 -- RELEASE-NOTES: synced +Alexander Jaeger (1 Aug 2023) - Taken down to 8.1.2 now for pending patch release +- misc: fix various typos -- libssh: when keyboard-interactive auth fails, try password + Closes #11561 - The state machine had a mistake in that it would not carry on to that - next step. +Daniel Stenberg (1 Aug 2023) - This also adds a verbose output what methods that are available from the - server and renames the macros that change to the next auth methods to - try. +- http2: avoid too early connection re-use/multiplexing - Reported-by: 左潇峰 - Fixes #11196 - Closes #11197 + HTTP/1 connections that are upgraded to HTTP/2 should not be picked up + for reuse and multiplexing by other handles until the 101 switching + process is completed. -Emanuele Torre (25 May 2023) + Lots-of-debgging-by: Stefan Eissing + Reported-by: Richard W.M. Jones + Bug: https://curl.se/mail/lib-2023-07/0045.html + Closes #11557 -- configure: fix build with arbitrary CC and LD_LIBRARY_PATH +- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" - Since ./configure and processes that inherit its environment variables - are the only callers of the run-compiler script, we can just save the - current value of the LD_LIBRARY_PATH and CC variables to another pair of - environment variables, and make run-compiler a static script that - simply restores CC and LD_LIBRARY_PATH to the saved value, and before - running the compiler. + This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. - This avoids having to inject the values of the variables in the script, - possibly causing problems if they contains spaces, quotes, and other - special characters. + It's a user error for supplying incomplete information to the build system. - Also add exports in the script just in case LD_LIBRARY_PATH and CC are - not already in the environment. + Reported-by: Ryan Schmidt + Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 - follow-up from 471dab2 +Viktor Szakats (1 Aug 2023) - Closes #11182 +- cmake: add support for single libcurl compilation pass -Daniel Stenberg (25 May 2023) + Before this patch CMake builds used two separate compilation passes to + build the shared and static libcurl respectively. This patch allows to + reduce that to a single pass if the target platform and build settings + allow it. -- urlapi: remove superfluous host name check + This reduces CMake build times when building both static and shared + libcurl at the same time, making these dual builds an almost zero-cost + option. - ... as it is checked later more proper. + Enable this feature for Windows builds, where the difference between the + two passes was the use of `__declspec(dllexport)` attribute for exported + API functions for the shared builds. This patch replaces this method + with the use of `libcurl.def` at DLL link time. - Closes #11195 + Also update `Makefile.mk` to use `libcurl.def` to export libcurl API + symbols on Windows. This simplifies (or fixes) this build method (e.g. + in curl-for-win, which generated a `libcurl.def` from `.h` files using + an elaborate set of transformations). -Stefan Eissing (25 May 2023) + `libcurl.def` has the maintenance cost of keeping the list of public + libcurl API symbols up-to-date. This list seldom changes, so the cost + is low. -- http2: fix EOF handling on uploads with auth negotiation + Closes #11546 - - doing a POST with `--digest` does an override on the initial request - with `Content-Length: 0`, but the http2 filter was unaware of that - and expected the originally request body. It did therefore not - send a final DATA frame with EOF flag to the server. - - The fix overrides any initial notion of post size when the `done_send` - event is triggered by the transfer loop, leading to the EOF that - is necessary. - - refs #11194. The fault did not happen in testing, as Apache httpd - never tries to read the request body of the initial request, - sends the 401 reply and closes the stream. The server used in the - reported issue however tried to read the EOF and timed out on the - request. +- cmake: detect `SSL_set0_wbio` in OpenSSL - Reported-by: Aleksander Mazur - Fixes #11194 - Cloes #11200 + Present in OpenSSL 1.1.0 and BoringSSL. + Missing from LibreSSL 3.8.0. -Daniel Stenberg (23 May 2023) + Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 -- RELEASE-NOTES: synced + While here, also fix `RAND_egd()` detection which was broken, likely all + along. This feature is probably broken with CMake builds and also + requires a sufficiently obsolete OpenSSL version, so this part of the + update was not tested. - bump to 8.2.0 + Closes #11555 -- lib: remove unused functions, make single-use static +- cmake: fixup H2 duplicate symbols for unity builds - Closes #11174 + Closes #11550 -- scripts/singleuse.pl: add more API calls +Pablo Busse (1 Aug 2023) -Christian Hesse (23 May 2023) +- openssl: Support async cert verify callback -- configure: quote the assignments for run-compiler + - Update the OpenSSL connect state machine to handle + SSL_ERROR_WANT_RETRY_VERIFY. - Building for multilib failed, as the compiler command contains an - extra argument. That needs quoting. + This allows libcurl users that are using custom certificate validation + to suspend processing while waiting for external I/O during certificate + validation. - Regression from b78ca50cb3dda361f9c1 + Closes https://github.com/curl/curl/pull/11499 - Fixes #11179 - Closes #11180 +Jay Satiro (1 Aug 2023) -Daniel Stenberg (23 May 2023) +- tool_cb_wrt: fix invalid unicode for windows console -- misc: fix spelling mistakes + - Suppress an incomplete UTF-8 sequence at the end of the buffer. - Reported-by: musvaage on github - Fixes #11171 - Closes #11172 + - Attempt to reconstruct incomplete UTF-8 sequence from prior call(s) + in current call. -Version 8.1.1 (23 May 2023) + Prior to this change, in Windows console UTF-8 sequences split between + two or more calls to the write callback would cause invalid "replacement + characters" U+FFFD to be printed instead of the actual Unicode + character. This is because in Windows only UTF-16 encoded characters are + printed to the console, therefore we convert the UTF-8 contents to + UTF-16, which cannot be done with partial UTF-8 sequences. -Daniel Stenberg (23 May 2023) + Reported-by: Maksim Arhipov -- RELEASE-NOTES: synced + Fixes https://github.com/curl/curl/issues/9841 + Closes https://github.com/curl/curl/pull/10890 - curl 8.1.1 +Daniel Stenberg (1 Aug 2023) -- THANKS: contributors from the 8.1.1 release +- sectransp: prevent CFRelease() of NULL -Dan Fandrich (22 May 2023) + When SecCertificateCopyCommonName() returns NULL, the common_name + pointer remains set to NULL which apparently when calling CFRelease() on + (sometimes?) crashes. -- docs: fix fuzzing documentation link + Reported-by: Guillaume Algis + Fixes #9194 + Closes #11554 - Follow-up to 4c712a1b +Jay Satiro (1 Aug 2023) -- CI: add an Alpine build with MUSL +- vtls: clarify "ALPN: offers" message - MUSL is another libc implementation which has its own unique issues - worth testing. + Before: + * ALPN: offers h2,http/1.1 - Ref: #11140 - Closes #11178 + After: + * ALPN: curl offers h2,http/1.1 -- runtests: add a missing \n at the end of a log message + Bug: https://curl.se/mail/lib-2023-07/0041.html + Reported-by: Richard W.M. Jones + Closes #11544 -correctmost on github (22 May 2023) +Daniel Stenberg (1 Aug 2023) -- SECURITY-PROCESS.md: link security advisory doc and fix typo +- urlapi: make sure zoneid is also duplicated in curl_url_dup - Closes #11177 + Add several curl_url_dup() tests to the general lib1560 test. -Daniel Stenberg (22 May 2023) + Reported-by: Rutger Broekhoff + Bug: https://curl.se/mail/lib-2023-07/0047.html + Closes #11549 -- TODO: build curl with Windows Unicode support +Sergey (1 Aug 2023) - Closes #7229 +- urlapi: fix heap buffer overflow -- KNOWN_BUGS: hyper memory-leaks + `u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-ter + minator. - Closes #10803 + ``` + ==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 a + t pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4 + READ of size 78 at 0x04d48c75 thread T0 + #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\c + ompiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840 + #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_p + arty\curl\lib\strdup.c:97 + #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_part + y\curl\lib\urlapi.c:1297 + #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\clien + t\third_party\curl\lib\urlapi.c:1342 + #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_ + party\curl\lib\urlapi.c:1790 + #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client + \third_party\curl\lib\url.c:1768 + #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_p + arty\curl\lib\url.c:3403 + #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_ + party\curl\lib\url.c:3888 + #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\thi + rd_party\curl\lib\multi.c:1982 + #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\ + third_party\curl\lib\multi.c:2756 + ``` -Stefan Eissing (22 May 2023) + Closes #11560 -- http/2: unstick uploads +Daniel Stenberg (31 Jul 2023) - - refs #11157 and #11175 where uploads get stuck or lead to RST streams - - fixes our h2 send behaviour to continue sending in the nghttp2 session - as long as it wants to. This will empty our send buffer as long as - the remote stream/connection window allows. - - in case the window is exhausted, the data remaining in the send buffer - will wait for a WINDOW_UPDATE from the server. Which is a socket event - that engages our transfer loop again - - the problem in the issue was that we did not exhaust the window, but - left data in the sendbuffer and no further socket events did happen. - The server was just waiting for us to send more. - - relatedly, there was an issue fixed that closing a stream with KEEP_HOLD - set kept the transfer from shutting down - as it should have - leading - to a timeout. +- curl: make %output{} in -w specify a file to write to - Closes #11176 + It can be used multiple times. Use %output{>>name} to append. -Daniel Stenberg (21 May 2023) + Add docs. Test 990 and 991 verify. -- workflows/macos: add a job using gcc + debug + secure transport + Idea: #11400 + Suggested-by: ed0d2b2ce19451f2 + Closes #11416 -Jay Satiro (21 May 2023) +- RELEASE-NOTES: synced -- lib: fix conversion warnings with gcc on macOS +- tool: add "variable" support -Daniel Stenberg (21 May 2023) + Add support for command line variables. Set variables with --variable + name=content or --variable name@file (where "file" can be stdin if set + to a single dash (-)). -- sectransp.c: make the code c89 compatible + Variable content is expanded in option parameters using "{{name}}" + (without the quotes) if the option name is prefixed with + "--expand-". This gets the contents of the variable "name" inserted, or + a blank if the name does not exist as a variable. Insert "{{" verbatim + in the string by prefixing it with a backslash, like "\\{{". - Follow-up to dd2bb485521c2ec713001b3a + Import an environment variable with --variable %name. It makes curl exit + with an error if the environment variable is not set. It can also rather + get a default value if the variable does not exist, using =content or + @file like shown above. - Reported-by: FeignClaims on github - Fixes #11155 - Closes #11159 + Example: get the USER environment variable into the URL: -Emanuele Torre (21 May 2023) + --variable %USER + --expand-url = "https://example.com/api/{{USER}}/method" -- Revert "urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirect - s" + When expanding variables, curl supports a set of functions that can make + the variable contents more convenient to use. It can trim leading and + trailing white space with "trim", output the contents as a JSON quoted + string with "json", URL encode it with "url" and base 64 encode it with + "b64". To apply functions to a variable expansion, add them colon + separated to the right side of the variable. They are then performed in + a left to right order. - This reverts commit df6c2f7b544f1f35f2a3e0be11f345affeb6fe9c. - (It only keep the test case that checks redirection to an absolute URL - without hostname and CURLU_NO_AUTHORITY). + Example: get the contents of a file called $HOME/.secret into a variable + called "fix". Make sure that the content is trimmed and percent-encoded + sent as POST data: - I originally wanted to make CURLU_ALLOW_SPACE accept spaces in the - hostname only because I thought - curl_url_set(CURLUPART_URL, CURLU_ALLOW_SPACE) was already accepting - them, and they were only not being accepted in the hostname when - curl_url_set(CURLUPART_URL) was used for a redirection. + --variable %HOME=/home/default + --expand-variable fix@{{HOME}}/.secret + --expand-data "{{fix:trim:url}}" + https://example.com/ - That is not actually the case, urlapi never accepted hostnames with - spaces, and a hostname with a space in it never makes sense. - I probably misread the output of my original test when I they were - normally accepted when using CURLU_ALLOW_SPACE, and not redirecting. + Documented. Many new test cases. - Some other URL parsers seems to allow space in the host part of the URL, - e.g. both python3's urllib.parse module, and Chromium's javascript URL - object allow spaces (chromium percent escapes the spaces with %20), - (they also both ignore TABs, and other whitespace characters), but those - URLs with spaces in the hostname are useless, neither python3's requests - module nor Chromium's window.location can actually use them. + Co-brainstormed-by: Emanuele Torre + Assisted-by: Jat Satiro + Closes #11346 - There is no reason to add support for URLs with spaces in the host, - since it was not a inconsistency bug; let's revert that patch before it - makes it into release. Sorry about that. +- KNOWN_BUGS: cygwin: make install installs curl-config.1 twice - I also reverted the extra check for CURLU_NO_AUTHORITY since that does - not seem to be necessary, CURLU_NO_AUTHORITY already worked for - redirects. + Closes #8839 - Closes #11169 +- KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14 -Dan Fandrich (20 May 2023) + Closes #11215 -- runtests: use the correct fd after select +- KNOWN_BUGS: cmake outputs: no version information available - The code was using the wrong fd when determining which runner was ready - with a response. + Closes #11158 - Ref: #10818 - Closes #11160 +- KNOWN_BUGS: APOP authentication fails on POP3 -- test425: fix the log directory for the upload + Closes #10073 - This must be %LOGDIR to let it work with parallel tests. +- KNOWN_BUGS: hyper is slow - Ref: #10969 + Closes #11203 -- runtests: handle interrupted reads from IPC pipes +Patrick Monnerat (31 Jul 2023) - These can be interrupted by signals, especially SIGINT to shut down, and - must be restarted so the IPC call arrives correctly. If the read just - returns an error instead, the IPC calling state will go out of sync and - a proper shutdown won't happen. +- configure, cmake, lib: more form api deprecation - Ref: #10818 + Introduce a --enable-form-api configure option to control its inclusion + in builds. The condition name defined for it is CURL_DISABLE_FORM_API. -Stefan Eissing (20 May 2023) + Form api code is dependent of MIME: configure and CMake handle this + dependency automatically: CMake by making it a dependent option + explicitly, configure by inheriting the MIME value by default and + rejecting explicit incompatible values. -- http2: upload improvements + "form-api" is now a new hidden test feature. - Make send buffer smaller to have progress and "upload done" reporting - closer to reality. Fix handling of send "drain" condition to no longer - trigger once the transfer loop reports it is done sending. Also do not - trigger the send "drain" on RST streams. + Update libcurl modules to respect this option and adjust tests + accordingly. - Background: - - a upload stall was reported in #11157 that timed out - - test_07_33a reproduces a problem with such a stall if the - server 404s the request and RSTs the stream. - - test_07_33b verifies a successful PUT, using the parameters - from #11157 and checks success + Closes #9621 - Ref: #11157 - Closes #11165 +Daniel Stenberg (31 Jul 2023) -- http2: increase stream window size to 10 MB +- mailmap: add Derzsi Dániel - Reported-by: pandada8 on github +Derzsi Dániel (31 Jul 2023) - Fixes #11162 - Closes #11167 +- wolfssl: support loading system CA certificates -Daniel Stenberg (20 May 2023) + Closes #11452 -- lib: rename struct 'http_req' to 'httpreq' +Viktor Szakats (30 Jul 2023) - Because FreeBSD 14 kidnapped the name. - Ref: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271526 +- nss: delete more NSS references - Fixes #11163 - Closes #11164 + Fix the distcheck CI failure and delete more NSS references. -Viktor Szakats (20 May 2023) + Follow-up to 7c8bae0d9c9b2dfeeb008b9a316117d7b9675175 -- cmake: avoid `list(PREPEND)` for compatibility + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Closes #11548 - `list(PREPEND)` requires CMake v3.15, our minimum is v3.7. +Daniel Stenberg (29 Jul 2023) - Ref: https://cmake.org/cmake/help/latest/command/list.html#prepend +- nss: remove support for this TLS library - Regression from 1e3319a167d2f32d295603167486e9e88af9bb4e + Closes #11459 - Reported-by: Keitagit-kun on Github - Fixes #11141 - Closes #11144 +Ryan Schmidt (29 Jul 2023) -Daniel Stenberg (19 May 2023) +- macOS: fix target detection more -- RELEASE-NOTES: synced + Now SCDynamicStoreCopyProxies is called (and the required frameworks are + linked in) on all versions of macOS and only on macOS. Fixes crash due + to undefined symbol when built with the macOS 10.11 SDK or earlier. -Stefan Eissing (19 May 2023) + CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and + is now only defined when SCDynamicStoreCopyProxies will actually be + called. Previously, it was defined when ENABLE_IPV6 was not defined but + SCDynamicStoreCopyProxies is not called in that case. -- ngtcp2: proper handling of uint64_t when adjusting send buffer + TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only + when dynamic targets are enabled. TARGET_OS_MAC is always defined but + means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS. + TARGET_OS_IPHONE means any Darwin OS other than macOS. - Fixes #11149 - Closes #11153 + Follow-up to c73b2f82 -- ngtcp2: fix compiler warning about possible null-deref + Fixes #11502 + Closes #11516 - - compiler analyzer did not include the call context for this - static function where the condition had already been checked. - - eleminating the problem by making stream a call parameter +Daniel Stenberg (29 Jul 2023) - Fixes #11147 - Closes #11151 +- tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR -Emanuele Torre (19 May 2023) + ... used at once. -- docs: document that curl_url_cleanup(NULL) is a safe no-op + Reported-by: Gabriel Corona + Fixes #11325 + Closes #11531 - This has always been the case, but it was not documented. +Thomas M. DuBuisson (29 Jul 2023) - The paragraph was copied verbatim from curl_easy_cleanup.3 +- CI: remove Lift's configuration - Closes #11150 + The Lift tool is being retired. Their site reads: -Antoine Pitrou (19 May 2023) + "Sonatype Lift will be retiring on Sep 12, 2023, with its analysis + stopping on Aug 12, 2023." -- select: avoid returning an error on EINTR from select() or poll() + Closes #11541 - This was already done for the poll() and select() calls - made directly from Curl_poll(), but was missed in - Curl_wait_ms(), which is called when there are no fds - to wait on. +Nathan Moinvaziri (29 Jul 2023) - Fixes #11135 - Closes #11143 +- Revert "schannel: reverse the order of certinfo insertions" -Daniel Stenberg (19 May 2023) + This reverts commit 8986df802db9b5338d9d50a54232ebae4dbcf6dd. -- vquic.c: make recvfrom_packets static, avoid compiler warning + Windows does not guarantee a particular certificate ordering, even + though TLS may have its own ordering/relationship guarantees. Recent + versions of Windows 11 reversed the ordering of ceritifcates returned by + CertEnumCertificatesInStore, therefore this commit no longer works as + initially intended. libcurl makes no guarantees about certificate + ordering if the operating system can't. - warning: no previous prototype for 'recvfrom_packets' + Ref: https://github.com/curl/curl/issues/9706 - Reported-by: Keitagit-kun on github - Fixes #11146 - Closes #11148 + Closes https://github.com/curl/curl/pull/11536 -- urlapi: allow numerical parts in the host name +wangzhikun (29 Jul 2023) - It can only be an IPv4 address if all parts are all digits and no more than - four parts, otherwise it is a host name. Even slightly wrong IPv4 will now be - passed through as a host name. +- winbuild: improve check for static zlib - Regression from 17a15d88467 shipped in 8.1.0 + - Check for zlib static library name zlibstatic.lib. - Extended test 1560 accordingly. + zlib's static library has a different name depending on how it was + built. zlibstatic.lib is output by cmake. zlibstat.lib is output by + their pre-generated Visual Studio project files (in the contrib + directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall + instead of cdecl if you end up exporting the zlib functions). - Reported-by: Pavel Kalyugin - Fixes #11129 - Closes #11131 + Prior to this change the makefile only checked for the latter. -Emilio Cobos Álvarez (19 May 2023) + Closes https://github.com/curl/curl/pull/11521 -- http2: double http request parser max line length +Daniel Stenberg (29 Jul 2023) - This works around #11138, by doubling the limit, and should be a - relatively safe fix. +- configure: use the pkg-config --libs-only-l flag for libssh2 - Ideally the buffer would grow as needed and there would be no need for a - limit? But that might be follow-up material. + ... instead of --libs, as that one also returns -L flags. - Fixes #11138 - Closes #11139 + Reported-by: Wilhelm von Thiele + Fixes #11538 + Closes #11539 -Emanuele Torre (18 May 2023) +Viktor Szakats (29 Jul 2023) -- configure: fix --help alignment +- cmake: support building static and shared libcurl in one go - AC_ARG_ENABLE seems to only trim off whitespace from the start and end - of its help-string argument, while prepending two spaces of indentation - to all lines. + This patch adds the ability to build a static and shared libcurl library + in a single build session. It also adds an option to select which one to + use when building the curl executable. - This means that the two spaces of indentation between the --enable-rtsp - and the --disable-rtsp line were not removed causing ./configure --help - to print: + New build options: + - `BUILD_STATIC_LIBS`. Default: `OFF`. + Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`. + - `BUILD_STATIC_CURL`. Default: `OFF`. + Requires `BUILD_STATIC_LIBS` enabled. + Enabled automatically if building static libcurl only. + - `STATIC_LIB_SUFFIX`. Default: empty. + - `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide + with static lib name (typically with MSVC) in Windows builds. + Otherwise empty. - Optional Features: - [...] - --enable-rtsp Enable RTSP support - --disable-rtsp Disable RTSP support + Also: - I removed the indentation to fix the issue, now it prints: + - Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass + it directly to the compiler. This also allows to delete a condition + from `tests/server/CMakeLists.txt`. - Optional Features: - [...] - --enable-rtsp Enable RTSP support - --disable-rtsp Disable RTSP support + - Complete a TODO by following the logic used in autotools (also for + `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of + `libcurl.pc` for _static-only_ curl builds. - The --enable-hsts and --disable-hsts lines had the same problems, and - have been fixed too. + - Convert an existing CI test to build both shared and static libcurl. - Closes #11142 + Closes #11505 -Deal(一线灵) (18 May 2023) +Stefan Eissing (28 Jul 2023) -- cmake: repair cross compiling +- CI/awslc: add cache for build awslc library - It cannot *run* code for testing purposes when cross-compiling. + Closes #11535 - Closes #11130 +- GHA/linux.yml: add caching -Daniel Stenberg (18 May 2023) + Closes #11532 -- configure: generate a script to run the compiler +Daniel Stenberg (27 Jul 2023) - in the CURL_RUN_IFELSE macro, with LD_LIBRARY_PATH set to the value of - the configure invoke, and not the value that might be used later, - intended for the execution of the output the compiler ouputs. +- RELEASE-NOTES: synced - For example when the compiler uses the same library (like libz) that - configure checks for. + Bump working version to 8.3.0 - Reported-by: Jonas Bülow - Fixes #11114 - Closes #11120 +- url: remove infof() output for "still name resolving" -Stefan Eissing (18 May 2023) + The message does not help and might get spewed a lot during times. -- cf-socket: completely remove the disabled USE_RECV_BEFORE_SEND_WORKAROUND + Reported-by: yushicheng7788 on github + Fixes #11394 + Closes #11529 - Closes #11118 +- KNOWN_BUGS: cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!" -Emanuele Torre (18 May 2023) + Closes #11244 -- urlapi: respect CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY for redirects +Stefan Eissing (27 Jul 2023) - curl_url_set(uh, CURLUPART_URL, redirurl, flags) was not respecing - CURLU_ALLOW_SPACE and CURLU_NO_AUTHORITY in the host part of redirurl - when redirecting to an absolute URL. +- CI: quiche updates - Closes #11136 + - remove quiche from standard `linux` workflow + - add mod_h2 caching to quiche workflow + - rename quiche to quiche-linux + - move version definitions into env section -Colin Cross (18 May 2023) + Closes #11528 -- hostip: move easy_lock.h include above curl_memory.h +- http2: disable asssertion blocking OSSFuzz testing - Similar to #9561, move easy_lock.h above curl_memory.h to fix building - against musl libc. + - not clear how this triggers and it blocks OSSFuzz testing other + things. Since we handle the case with an error return, disabling the + assertion for now seems the best way forward. - Closes #11140 + Fixes #11500 + Closes #11519 -Hind Montassif (18 May 2023) +- http2: fix in h2 proxy tunnel: progress in ingress on sending -- curl_easy_getinfo: clarify on return data types + - depending on what is tunneled, the proxy may never get invoked for + receiving data explicitly. Not progressing ingress may lead to stalls + due to missed WINDOW_UPDATEs. - Closes #11126 + CI: + - add a chache for building mod_h2 -Emanuele Torre (18 May 2023) + Closes #11527 -- checksrc: disallow spaces before labels +- CI ngtcp2+quictls: use nghttpx cache as in quiche build - Out of 415 labels throughout the code base, 86 of those labels were - not at the start of the line. Which means labels always at the start of - the line is the favoured style overall with 329 instances. +Jay Satiro (27 Jul 2023) - Out of the 86 labels not at the start of the line: - * 75 were indented with the same indentation level of the following line - * 8 were indented with exactly one space - * 2 were indented with one fewer indentation level then the following - line - * 1 was indented with the indentation level of the following line minus - three space (probably unintentional) +- bearssl: don't load CA certs when peer verification is disabled - Co-Authored-By: Viktor Szakats + We already do this for other SSL backends. - Closes #11134 + Bug: https://github.com/curl/curl/pull/11457#issuecomment-1644587473 + Reported-by: kyled-dell@users.noreply.github.com -Daniel Stenberg (18 May 2023) + Closes https://github.com/curl/curl/pull/11497 -- cookie: update the comment on cookie length and size limits +Daniel Stenberg (26 Jul 2023) - To refer to the proper cookie RFC and the upcoming RFC refresh. +- easy: remove #ifdefs to make code easier on the eye - Closes #11127 + Closes #11525 -- url: provide better error message when URLs fail to parse +Stefan Eissing (26 Jul 2023) - By providing the URL API error message into the error message. +- GHA: adding quiche workflow - Ref: #11129 - Closes #11137 + - adding separate quiche workflow to also build nghttpx server for testing -- RELEASE-NOTES: synced + Closes #11517 - bumped to 8.1.1 +Version 8.2.1 (26 Jul 2023) -Jon Rumsey (18 May 2023) +Daniel Stenberg (26 Jul 2023) -- os400: update chkstrings.c +- RELEASE-NOTES: synced - Compensate changes for recent changes to urldata.h to reclassify - STRING_AWS_SIGV4. + curl 8.2.1 release - Fixes #11132 - Closes #11133 +- THANKS: add contributors from 8.2.1 -Version 8.1.0 (17 May 2023) +- docs: provide more see also for cipher options -Daniel Stenberg (17 May 2023) + More cross references. Hide nroff errors. -- RELEASE-NOTES: synced + Closes #11513 -- THANKS: contributors from the 8.1.0 release +- docs: mark two TLS options for TLS, not SSL -- hostip: include easy_lock.h before using GLOBAL_INIT_IS_THREADSAFE + Closes #11514 - Since that header file is the only place that define can be defined. +Brad Harder (25 Jul 2023) - Reported-by: Marc Deslauriers +- curl_multi_wait.3: fix arg quoting to doc macro .BR - Follow-up to 13718030ad4b3209 + Closes #11511 - Closes #11121 +Daniel Stenberg (24 Jul 2023) -Thomas Taylor (16 May 2023) +- RELEASE-NOTES: synced -- aws-sigv4.d: fix region identifier in example +Viktor Szakats (24 Jul 2023) - Closes #11117 +- cmake: update ngtcp2 detection -Philip Heiduck (15 May 2023) + Replace `OpenSSL` with `quictls` to follow the same change + in the v0.17.0 ngtcp2 release. -- mlc_config.json: remove this linkcheck CI job config file + Follow-up to e0093b4b732f6495b0fb1cd6747cbfedcdcf63ed - Closes #11113 + Closes #11508 -Daniel Silverstone (15 May 2023) +Stefan Eissing (24 Jul 2023) -- ssh: Add support for libssh2 read timeout +- http: VLH, very large header test and fixes - Hook the new (1.11.0 or newer) libssh2 support for setting a read timeout - into the SERVER_RESPONSE_TIMEOUT option. With this done, clients can use - the standard curl response timeout setting to also control the time that - libssh2 will wait for packets from a slow server. This is necessary to - enable use of very slow SFTP servers. + - adding tests using very large passwords in auth + - fixes general http sending to treat h3 like h2, and + not like http1.1 + - eliminate H2_HEADER max definitions and use the commmon + DYN_HTTP_REQUEST everywhere, different limits do not help + - fix http2 handling of requests denied by nghttp2 on send + to immediately report the refused stream - Signed-off-by: Daniel Silverstone + Closes #11509 - Closes #10965 +Andrei Rybak (23 Jul 2023) -Osama Albahrani (14 May 2023) +- CONTRIBUTE: drop mention of copyright year ranges -- GIT-INFO: add --with-openssl + Year ranges in copyrights were dropped in commits [1] and [2]. + Verification of year ranges in copyrights was dropped from script + 'scripts/copyright.pl' in commit [3]. However, the corresponding + passages in file 'docs/CONTRIBUTE.md' weren't updated. - Closes #11110 + Drop mentions of copyright year ranges from 'docs/CONTRIBUTE.md'. -Daniel Stenberg (13 May 2023) + [1] 2bc1d775f (copyright: update all copyright lines and remove year + ranges, 2023-01-02) + [2] c46761bd8 (tests/http: remove year ranges from copyrights, + 2023-03-14) + [3] 0e293bacb (copyright.pl: cease doing year verifications, 2023-01-28) -- RELEASE-NOTES: synced + Closes #11504 -Marcel Raad (13 May 2023) +- CONTRIBUTE: fix syntax in commit message description -- md(4|5): don't use deprecated iOS functions + File 'docs/CONTRIBUTE.md' includes a description of how one should write + commit messages in the curl project. Different possible parts of the + message are enclosed in square brackets. One exception is the section + describing how the curl project doesn't use "Signed-off-by" commit + trailers [1], which is enclosed in an opening curly brace paired with a + closing square bracket. - They are marked as deprecated in iOS 13.0, which might result in - warnings-as-errors. + Fix the enclosing square brackets in description of "Signed-off-by" + trailers in commit messages in file 'docs/CONTRIBUTE.md'. - Also, use `*_MIN_REQUIRED` instead of `*_MIN_ALLOWED`, which seems to - be what's currently used. + [1] See description of option '--signoff' in Git documentation: + https://git-scm.com/docs/git-commit - Bug: https://github.com/curl/curl/issues/11098 - Closes https://github.com/curl/curl/pull/11102 + Closes #11504 -- md4: only build when used +Daniel Stenberg (23 Jul 2023) - Its only usage in curl_ntlm_core.c is guarded by `USE_CURL_NTLM_CORE`, - so let's use this here too. +- src/mkhelp: strip off escape sequences - Ref: https://github.com/curl/curl/issues/11098 - Closes https://github.com/curl/curl/pull/11102 + At some point the nroff command stopped stripping off escape sequences, + so then this script needs to do the job instead. -Vítor Galvão (12 May 2023) + Reported-by: VictorVG on github + Fixes #11501 + Closes #11503 -- write-out.d: Use response_code in example +- KNOWN_BUGS: building for old macOS fails with gcc - Closes #11107 + Closes #11441 -Shohei Maeda (12 May 2023) +Jacob Hoffman-Andrews (22 Jul 2023) -- url: fix null dispname for --connect-to option +- rustls: update rustls-ffi 0.10.0 - Closes #11106 + This brings in version 0.21.0 of the upstream rustls implementation, + which notable includes support for IP address certificates. -Daniel Stenberg (12 May 2023) + Closes #10865 -- test2306: verify getting a second response with folded headers +Brad Harder (22 Jul 2023) - Reproduces the isue #11101 and verifies the fix. +- websocket: rename arguments/variables to match docs - Verifies a17b2a503f + Pedantry/semantic-alignment between functions, docs, comments with + respect to websocket protocol code; No functional change intended. -- headers: clear (possibly) lingering pointer in init + * "totalsize", "framesize" becomes "fragsize" (we deal in frame fragments). - The "prevhead" pointer is used for the headers storage but was not - cleared correctly in init, which made it possible to act up when a - handle is reused. + * "sendflags" becomes "flags" - Reported-by: Steve Herrell - Fixes #11101 - Closes #11103 + * use canonical CURL *handle -- RELEASE-NOTES: synced + Closes #11493 -- ngtcp2: use 0.15.0 +Jan Macku (21 Jul 2023) - - nghttp3 0.11.0 - - nghttp2 1.53.0 +- bug_report: use issue forms instead of markdown template - Adapt to new API calls + Issue forms allow you to define web-like input forms using YAML + syntax. It allows you to guide the reporter to get the required + information. - Closes #11031 + Signed-off-by: Jan Macku + Closes #11474 -Jay Satiro (10 May 2023) +Daniel Stenberg (21 Jul 2023) -- openssl: fix indent +- TODO: Obey Retry-After in redirects -Daniel Stenberg (10 May 2023) + (remove "Set custom client ip when using haproxy protocol" which was + shipped in 8.2.0) -- CURLOPT_DNS_CACHE_TIMEOUT.3: fix spelling + Mentioned-by: Yair Lenga + Closes #11447 - Follow-up to 9ed7d56e044f5aa1b29 +- RELEASE-NOTES: synced - Closes #11096 +Oliver Roberts (21 Jul 2023) -- hostip: use time_t for storing oldest DNS entry +- amissl: fix AmiSSL v5 detection - Theoretically, the oldest time could overflow an int. In practice that - won't happen, but let's do this to please analyzers. + Due to changes in the AmiSSL SDK, the detection needed adjusting. - Follow-up to 9ed7d56e044f5aa1b2928ccde6245d0 + Closes #11477 - Pointed out by Coverity. - Closes #11094 +Alois Klink (21 Jul 2023) -- http: free the url before storing a new copy +- unittest/makefile: remove unneeded unit1621_LDADD - To avoid a memory-leak. + The `unit1621_LDADD` variable has the exact same value as the `LDADD` + flag in `Makefile.am`, except without `@LDFLAGS@ @LIBCURL_LIBS@`. - Reported-by: Hiroki Kurosawa + This was originally added by [98e6629][], but I can't see any reason + why it exists, so we should remove it to clean things up. - Closes #11093 + [98e6629]: https://github.com/curl/curl/commit/98e6629154044e4ab1ee7cff8351c7 + ebcb131e88 -- compressed.d: clarify the words on "not notifying headers" + Closes #11494 - Reported-by: Dylan Anthony - Fixes #11091 - Closes #11092 +- unittest/makefile: remove unneeded unit1394_LDADD -- libssh2: free fingerprint better + These custom `unit1394_LDADD` and similar automake overrides are no + longer neded. They were originally added by added by [8dac7be][] for + metalink support, but are no longer after [265b14d][] removed metalink. - Reported-by: Wei Chong Tan - Closes #11088 + [8dac7be]: https://github.com/curl/curl/commit/8dac7be438512a8725d3c71e9139bd + fdcac1ed8c + [265b14d]: https://github.com/curl/curl/commit/265b14d6b37c4298bd5556fabcbc37 + d36f911693 -- CURLOPT_IPRESOLVE.3: clarify that this for host names, not IP addresses + Closes #11494 - Reported-by: Harry Sintonen - Closes #11087 +- cmake: add `libcurlu`/`libcurltool` for unit tests -- hostip: enforce a maximum DNS cache size independent of timeout value + Add a `libcurlu`/`libcurltool` static library that is compiled only for + unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not + built by default, they're only built if unit tests are built. - To reduce the damage an application can cause if using -1 or other - ridiculous timeout values and letting the cache live long times. + These libraries allow us to compile every unit test with CMake. - The maximum number of entries in the DNS cache is now totally - arbitrarily and hard-coded set to 29999. + Closes #11446 - Closes #11084 +Daniel Stenberg (21 Jul 2023) -- hostip: store dns timeout as 'int' +- test979: test -u with redirect to (the same) absolute host - ... because it set and held as an 'int' elsewhere and can never be - larger. + Verifies #11492 -- RELEASE-NOTES: synced +- transfer: do not clear the credentials on redirect to absolute URL -- tool_operate: refuse (--data or --form) and --continue-at combo + Makes test 979 work. Regression shipped in 8.2.0 from commit + dd4d1a26959f63a2c - libcurl assumes that a --continue-at resumption is done to continue an - upload using the read callback and neither --data nor --form use - that and thus won't do what the user wants. Whatever the user wants - with this strange combination. + Fixes #11486 + Reported-by: Cloudogu Siebels + Closes #11492 - Add test 426 to verify. +Jon Rumsey (20 Jul 2023) - Reported-by: Smackd0wn on github - Fixes #11081 - Closes #11083 +- os400: correct EXPECTED_STRING_LASTZEROTERMINATED -- transfer: refuse POSTFIELDS + RESUME_FROM combo + Correct EXPECTED_STRING_LASTZEROTERMINATED to account for + CURLOPT_HAPROXY_CLIENT_IP which requires EBCDIC to ASCII conversion when + passed into curl_easy_setopt(). - The code assumes that such a resume is wanting to continue an upload - using the read callback, and since POSTFIELDS is done without callback - libcurl will just misbehave. + Closes #11476 - This combo will make the transfer fail with CURLE_BAD_FUNCTION_ARGUMENT - with an explanation in the error message. +Oliver Roberts (20 Jul 2023) - Reported-by: Smackd0wn on github - Fixes #11081 - Closes #11083 +- amissl: add missing signal.h include -- ipv4.d/ipv6.d: they are "mutex", not "boolean" + In some environments, signal.h is already included, but not in others + which cause compilation to fail, so explictly include it. - ... which for example means they do not have --no-* versions. + Closes #11478 - Reported-by: Harry Sintonen - Fixes #11085 - Closes #11086 +- amigaos: fix sys/mbuf.h m_len macro clash -- docs/SECURITY-ADVISORY.md: how to write a curl security advisory + The updated Curl_http_req_make and Curl_http_req_make2 functions spawned + a parameter called m_len. The AmigaOS networking headers, derived from + NetBSD, contain "#define m_len m_hdr.mh_len" which clashes with + this. Since we do not actually use mbuf, force the include file to be + ignored, removing the clash. - Closes #11080 + Closes #11479 -nobedee on github (5 May 2023) +Daniel Stenberg (20 Jul 2023) -- MANUAL.md: add dict example for looking up a single definition +- socks: print ipv6 address within brackets - Closes #11077 + Fixes #11483 + Closes #11484 -Dan Fandrich (5 May 2023) +Christian Schmitz (20 Jul 2023) -- runtests: fix -c option when run with valgrind +- libcurl-errors.3: add CURLUE_OK - The curl binary argument wasn't being quoted properly. This seems to - have broken at some point after quoting was added in commit 606b29fe. + Closes #11488 - Reported-by: Daniel Stenberg - Ref: #11073 - Fixes #11074 - Closes #11076 +Oliver Roberts (20 Jul 2023) -- runtests: support creating more than one runner process +- cfilters: rename close/connect functions to avoid clashes - The controller currently only creates and uses one, but more are now - possible. + Rename `close` and `connect` in `struct Curl_cftype` for + consistency and to avoid clashes with macros of the same name + (the standard AmigaOS networking connect() function is implemented + via a macro). - Ref: #10818 + Closes #11491 -- runtests: spawn a new process for the test runner +Stefan Eissing (20 Jul 2023) - When the -j option is given, a new process is spawned in which the test - programs are run and from which test servers are started. Only one - process can be started at once, but this is sufficient to test that the - infrastructure can isolate those functions in a new task. There should - be no visible difference between the two modes at the moment. +- http2: fix regression on upload EOF handling - Ref: #10818 - Closes #11064 + - a regression introduced by c9ec85121110d7cbbbed2990024222c8f5b8afe5 + where optimization of small POST bodies leads to a new code path + for such uploads that did not trigger the "done sending" event + - add triggering this event for early "upload_done" situations -- runtests: turn singletest() into a state machine + Fixes #11485 + Closes #11487 + Reported-by: Aleksander Mazur - This allows it to run in a non-blocking manner. +Daniel Stenberg (19 Jul 2023) - Ref: #10818 +- configure: check for nghttp2_session_get_stream_local_window_size -- runtests: change runner interface to be asynchronous + The http2 code uses it now. Introduced in nghttp2 1.15.0 (Sep 2016) - Program arguments are marshalled and then written to the end of a pipe - which is later read from and the arguments unmarshalled before the - desired function is called normally. The function return values are - then marshalled and written into another pipe when is later read from - and unmarshalled before being returned to the caller. + Fixes #11470 + Reported-by: Paul Howarth + Closes #11473 - The implementation is currently blocking but can be made non-blocking - without any changes to the API. This allows calling multiple runners - without blocking in the future. +Stefan Eissing (19 Jul 2023) - Ref: #10818 +- quiche: fix segfault and other things -- runtests: call citest_finishtest in singletest + - refs #11449 where a segfault is reported when IP Eyeballing did + not immediately connect but made several attempts + - The transfer initiating the eyeballing was initialized too early, + leadding to references to the filter instance that was then + replaced in the subsequent eyeball attempts. That led to a use + after free in the buffer handling for the transfer + - transfers are initiated now more lazy (like in the ngtcp2 filter), + when the stream is actually opened + - suppress reporting on quiche event errors for "other" transfers + than the current one to not fail a transfer due to faults in + another one. + - revert recent return value handling for quiche_h3_recv_body() + to not indicate an error but an EAGAIN situation. We wish quiche + would document what functions return. - This is where citest_starttest is called. + Fixes #11449 + Closes #11469 + Reported-by: ウさん - Ref: #10818 +Daniel Stenberg (19 Jul 2023) -- runtests: add a runner initialization function +- hostip: return IPv6 first for localhost resolves - This sets up the runner environment to start running tests. + Fixes #11465 + Reported-by: Chilledheart on github + Closes #11466 - Ref: #10818 +Harry Sintonen (19 Jul 2023) -- runtests: remove directory from server filename variables +- tool: fix tool_seek_cb build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T - There will soon be multiple log directories so the paths will no longer - be static in runtests.pl. Also, get rid of $SERVER2IN which was not - used. + - a variable was renamed, and some use of it wasn't. this fixes the + build. - Ref: #10818 + Closes #11468 -- runtests: reduce package exports after refactoring +Stefan Eissing (19 Jul 2023) - Some recent refactoring made these export no longer necessary. Also, - stop displaying the Unix socket paths at startup since there will soon - be many of them and they're not that interesting. +- quiche: fix lookup of transfer at multi - Ref: #10818 + - refs #11449 where weirdness in quiche multi connection tranfers was + observed + - fixes lookup of transfer for a quiche event to take the connection + into account + - formerly, a transfer with the same stream_id, but on another connection + could be found -- runtests: use a function to obtain $LOGDIR for a test + Closes #11462 - This will no longer be static soon. +Daniel Stenberg (19 Jul 2023) - Ref: #10818 +- RELEASE-NOTES: synced -Jay Satiro (5 May 2023) + bump to 8.2.1 -- tool_cb_hdr: Fix 'Location:' formatting for early VTE terminals +John Haugabook (19 Jul 2023) - - Disable hyperlink formatting for the 'Location:' header value in VTE - 0.48.1 and earlier, since it is buggy in some of those versions. +- ciphers.d: put URL in first column - Prior to this change those terminals may show the location header value - as gibberish or show it twice. + This makes the URL turn into a link properly when "webified". - Ref: https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda#backw - ard-compatibility + Fixes https://github.com/curl/curl-www/issues/270 + Closes #11464 - Fixes https://github.com/curl/curl/issues/10428 - Closes https://github.com/curl/curl/pull/11071 +Version 8.2.0 (19 Jul 2023) -François Michel (3 May 2023) +Daniel Stenberg (19 Jul 2023) -- quiche: disable pacing while pacing is not actually performed +- RELEASE-NOTES: synced - Closes #11068 + 8.2.0 release -Daniel Stenberg (2 May 2023) +- THANKS-filter: strip out "GitHub" -- easy_cleanup: require a "good" handle to act +- THANKS: add contributors from 8.2.0 - By insisting that the passed in handle is "good" (the magic number is - intact), this can limit the potential damage if a bad pointer is passed - in. Like when this function is called twice on the same handle pointer. +- RELEASE-PROCEDURE.md: adjust the release dates - Ref: #10964 - Closes #11061 +Stefan Eissing (17 Jul 2023) -Andreas Falkenhahn (1 May 2023) +- quiche: fix defects found in latest coverity report -- amiga: Fix CA certificate paths for AmiSSL and MorphOS + Closes #11455 - AmiSSL stores certificates in `AmiSSL:Certs` and MorphOS stores them in - `MOSSYS:Data/SSL/curl-ca-bundle.crt`. +Daniel Stenberg (17 Jul 2023) - Closes https://github.com/curl/curl/pull/11059 +- quiche: avoid NULL deref in debug logging -Daniel Stenberg (30 Apr 2023) + Coverity reported "Dereference after null check" -- http2: (void)-mark when we explicitly ignore the return code + If stream is NULL and the function exits, the logging must not deref it. - When h2_progress_egress() is called. Pointed out by Coverity. + Closes #11454 - Closes #11057 +Stefan Eissing (17 Jul 2023) -- checksrc: find bad indentation in conditions without open brace +- http2: treat initial SETTINGS as a WINDOW_UPDATE - If the previous line starts with if/while/for AND ends with a closed - parenthesis and there's an equal number of open and closed parentheses - on that line, verify that this line is indented $indent more steps, if - not a cpp line. + - refs #11426 where spurious stalls on large POST requests + are reported + - the issue seems to involve the following + * first stream on connection adds up to 64KB of POST + data, which is the max default HTTP/2 stream window size + transfer is set to HOLD + * initial SETTINGS from server arrive, enlarging the stream + window. But no WINDOW_UPDATE is received. + * curl stalls + - the fix un-HOLDs a stream on receiving SETTINGS, not + relying on a WINDOW_UPDATE from lazy servers - Also adjust the fall-out from this fix. + Closes #11450 - Closes #11054 +Daniel Stenberg (17 Jul 2023) -Diogo Teles Sant'Anna (28 Apr 2023) +- ngtcp2: assigning timeout, but value is overwritten before used -- CI: Set minimal permissions on workflow ngtcp2-quictls.yml + Reported by Coverity - Signed-off-by: Diogo Teles Sant'Anna + Closes #11453 - Closes #11055 +- krb5: add typecast to please Coverity -Dan Fandrich (28 Apr 2023) +Derzsi Dániel (16 Jul 2023) -- CI: use another glob syntax for matching files on Appveyor +- wolfssl: support setting CA certificates as blob - The previous globbing syntax was not matching files recursively in - directories, so try appending a /* to more closely match the examples at - https://www.appveyor.com/docs/how-to/filtering-commits/ + Closes #11445 -Daniel Stenberg (28 Apr 2023) +- wolfssl: detect when TLS 1.2 support is not built into wolfssl -- multi: add multi-ignore logic to multi_socket_action + Closes #11444 - The multi-ignore logic that was previously applied to - curl_multi_perform() (#10750) is here applied to the loop within - curl_multi_socket_action() to make it use the same optimization: most - handles have the same signal-ignore option state so this drastically - reduces the number of ignore/unignore calls per libcurl function invoke. +Graham Campbell (15 Jul 2023) - Follow-up to bc90308328afb8 +- CI: bump nghttp2 from 1.55.0 to 1.55.1 - Closes #11045 + Closes #11442 -Stefan Eissing (28 Apr 2023) +Daniel Stenberg (15 Jul 2023) -- http2: do flow window accounting for cancelled streams +- curl: return error when asked to use an unsupported HTTP version - - nghttp2 does not free connection level window flow for - aborted streams - - when closing transfers, make sure that any buffered - response data is "given back" to the flow control window - - add tests test_02_22 and test_02_23 to reproduce + When one of the following options are used but the libcurl in use does + not support it: - Closes #11052 + --http2 + --http2-prior-knowledge + --proxy-http2 -- pingpong: fix compiler warning "assigning an enum to unsigned char" + Closes #11440 - Closes #11050 +Chris Paulson-Ellis (14 Jul 2023) -Daniel Stenberg (28 Apr 2023) +- cf-socket: don't bypass fclosesocket callback if cancelled before connect -- configure: fix detection of apxs (for httpd) + After upgrading to 8.1.2 from 7.84.0, I found that sockets were being + closed without calling the fclosesocket callback if a request was + cancelled after the associated socket was created, but before the socket + was connected. This lead to an imbalance of fopensocket & fclosesocket + callbacks, causing problems with a custom event loop integration using + the multi-API. - The condition check was turned the wrong way around! + This was caused by cf_socket_close() calling sclose() directly instead + of calling socket_close() if the socket was not active. For regular TCP + client connections, the socket is activated by cf_socket_active(), which + is only called when the socket completes the connect. - Closes #11051 + As far as I can tell, this issue has existed since 7.88.0. That is, + since the code in question was introduced by: + commit 71b7e0161032927cdfb4e75ea40f65b8898b3956 + Author: Stefan Eissing + Date: Fri Dec 30 09:14:55 2022 +0100 -Viktor Szakats (28 Apr 2023) + lib: connect/h2/h3 refactor -- ci: `-Wno-vla` no longer necessary + Closes #11439 - We handle this issue in the source now. +Daniel Stenberg (13 Jul 2023) - Follow-up to b725fe1944b45406676ea3aff333ae3085a848d9 +- tool_parsecfg: accept line lengths up to 10M - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Closes #11048 + Bumped from 100K set in 47dd957daff9 -Marcel Raad (28 Apr 2023) + Reported-by: Antoine du Hamel + Fixes #11431 + Closes #11435 -- tests/http: make curl_setup.h the first include +Stefan Eissing (13 Jul 2023) - This is required for the macros there to take effect for system - libraries. Specifically, including the system libraries first led to - warnings about `_FILE_OFFSET_BITS` being redefined in curl_config.h on - the Solaris autobuilds for ws-data.c and ws-pingpong.c. - Also make the curl includes come first for the other source files here - for consistency. +- CI: brew fix for openssl in default path - Closes https://github.com/curl/curl/pull/11046 + If brew install/update links openssl into /usr/local, it will be found + before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of + that by unlinking the keg. -Emanuele Torre (27 Apr 2023) + Fixes #11413 + Closes #11436 -- checksrc: check for spaces before the colon of switch labels +Daniel Stenberg (13 Jul 2023) - Closes #11047 +- RELEASE-NOTES: synced -Daniel Stenberg (27 Apr 2023) +Ondřej Koláček (13 Jul 2023) -- RELEASE-NOTES: synced +- sectransp: fix EOF handling -- libssh: tell it to use SFTP non-blocking + Regression since the large refactor from 2022 - Reported-by: Andreas Huebner - Fixes #11020 - Closes #11039 + Closes #11427 -Stefan Eissing (27 Apr 2023) +Daniel Stenberg (13 Jul 2023) -- http2: enlarge the connection window +- checksrc: quote the file name to work with "funny" letters - - fixes stalled connections + Closes #11437 - - Make the connection window large enough, so that there is - some room left should 99/100 streams be PAUSED by the application +Karthikdasari0423 (13 Jul 2023) - Reported-by: Paweł Wegner - Fixes #10988 - Closes #11043 +- HTTP3.md: ngtcp2 updated to v0.17.0 and nghttp3 to v0.13.0 -Daniel Stenberg (27 Apr 2023) + Follow-up to e0093b4b732f6 -- checksrc: fix SPACEBEFOREPAREN for conditions starting with "*" + Closes #11433 - The open paren check wants to warn for spaces before open parenthesis - for if/while/for but also for any function call. In order to avoid - catching function pointer declarations, the logic allows a space if the - first character after the open parenthesis is an asterisk. +Daniel Stenberg (13 Jul 2023) - I also spotted what we did not include "switch" in the check but we should. +- CURLOPT_MIMEPOST.3: clarify what setting to NULL means - This check is a little lame, but we reduce this problem by not allowing - that space for if/while/for/switch. + Follow-up to e08382a208d4e480 - Reported-by: Emanuele Torre - Closes #11044 + Closes #11430 -- docs: minor polish +Tatsuhiro Tsujikawa (12 Jul 2023) - - "an HTTP*" (not "a") - - remove a few contractions - - remove a spurious "a" - - reduce use of "I" in texts +- ngtcp2: build with 0.17.0 and nghttp3 0.13.0 - Closes #11040 + - ngtcp2_crypto_openssl was renamed to ngtcp2_crypto_quictls. -- ws: fix CONT opcode check + Closes #11428 - Detected by Coverity. Follow-up to 930c00c259 +- CI: Bump ngtcp2, nghttp3, and nghttp2 - Closes #11037 + Closes #11428 -Dan Fandrich (27 Apr 2023) +James Fuller (11 Jul 2023) -- CI: switch the awslc builds to build out-of-tree +- example/maxconnects: set maxconnect example - This is a common configuration that should be tested to avoid - regressions. The awsls cmake build was already out-of-tree so the - automake build now joins it. + Closes #11343 - Ref: #11006 +Pontakorn Prasertsuk (11 Jul 2023) -- tests/http: fix out-of-tree builds +- http2: send HEADER & DATA together if possible - Add both lib/ directories (src & build) to the search path so - curl_setup.h and its dependencies can be found. + Closes #11420 - Followup-to acd82c8b +Daniel Stenberg (11 Jul 2023) - Ref: #11006 - Closes #11036 +- CI: use wolfSSL 5.6.3 in builds -Daniel Stenberg (27 Apr 2023) + No using master anymore -- urlapi: make internal function start with Curl_ + Closes #11424 - Curl_url_set_authority() it is. +SaltyMilk (11 Jul 2023) - Follow-up to acd82c8bfd +- fopen: optimize - Closes #11035 + Closes #11419 -YX Hao (26 Apr 2023) +Daniel Stenberg (11 Jul 2023) -- cf-socket: turn off IPV6_V6ONLY on Windows if it is supported +- cmake: make use of snprintf - IPV6_V6ONLY refs: - https://en.wikipedia.org/wiki/IPv6#IPv4-mapped_IPv6_addresses - https://github.com/golang/go/blob/master/src/net/ipsock_posix.go - https://en.wikipedia.org/wiki/Unix-like - https://learn.microsoft.com/en-us/windows/win32/winsock/ipproto-ipv6-socket-o - ptions + Follow-up to 935b1bd4544a23a91d68 - default value refs: - https://datatracker.ietf.org/doc/html/rfc3493#section-5.3 - https://www.kernel.org/doc/html/latest/networking/ip-sysctl.html#proc-sys-net - -ipv6-variables + Closes #11423 - Closes #10975 +Stefan Eissing (11 Jul 2023) -Daniel Stenberg (26 Apr 2023) +- macOS: fix taget detection -- urldata: shrink *select_bits int => unsigned char + - TARGET_OS_OSX is not always defined on macOS + - this leads to missing symbol Curl_macos_init() + - TargetConditionals.h seems to define these only when + dynamic targets are enabled (somewhere?) + - this PR fixes that on my macOS 13.4.1 + - I have no clue why CI builds worked without it - - dselect_bits - - cselect_bits + Follow-up to c7308592fb8ba213fc2c1 + Closes #11417 - ... are using less than 8 bits. Changed types and moved them towards - the end of the structs to fit better. +Stan Hu (9 Jul 2023) - Closes #11025 +- hostip.c: Move macOS-specific calls into global init call -Stefan Eissing (26 Apr 2023) + https://github.com/curl/curl/pull/7121 introduced a macOS system call + to `SCDynamicStoreCopyProxies`, which is invoked every time an IP + address needs to be resolved. -- tests/http: more tests with specific clients + However, this system call is not thread-safe, and macOS will kill the + process if the system call is run first in a fork. To make it possible + for the parent process to call this once and prevent the crash, only + invoke this system call in the global initialization routine. - - Makefile support for building test specific clients in tests/http/clients - - auto-make of clients when invoking pytest - - added test_09_02 for server PUSH_PROMISEs using clients/h2-serverpush - - added test_02_21 for lib based downloads and pausing/unpausing transfers + In addition, this change is beneficial because it: - curl url parser: - - added internal method `curl_url_set_authority()` for setting the - authority part of a url (used for PUSH_PROMISE) + 1. Avoids extra macOS system calls for every IP lookup. + 2. Consolidates macOS-specific initialization in a separate file. - http2: - - made logging of PUSH_PROMISE handling nicer + Fixes #11252 + Closes #11254 - Placing python test requirements in requirements.txt files - - separate files to base test suite and http tests since use - and module lists differ - - using the files in the gh workflows +Daniel Stenberg (9 Jul 2023) - websocket test cases, fixes for we and bufq - - bufq: account for spare chunks in space calculation - - bufq: reset chunks that are skipped empty - - ws: correctly encode frames with 126 bytes payload - - ws: update frame meta information on first call of collect - callback that fills user buffer - - test client ws-data: some test/reporting improvements +- docs: use a space after RFC when spelling out RFC numbers - Closes #11006 + Closes #11382 -Jay Satiro (26 Apr 2023) +Margu (9 Jul 2023) -- libssh2: fix crash in keyboard callback +- imap-append.c: update to make it more likely to work - - Always set the libssh2 'abstract' user-pointer to the libcurl easy - handle associated with the ssh session, so it is always passed to the - ssh keyboard callback. + Fixes #10300 + Closes #11397 - Prior to this change and since 8b5f100 (precedes curl 8.0.0), if libcurl - was built without CURL_DEBUG then it could crash during the ssh auth - phase due to a null dereference in the ssh keyboard callback. +Emanuele Torre (9 Jul 2023) - Reported-by: Andreas Falkenhahn +- tool_writeout_json: fix encoding of control characters - Fixes https://github.com/curl/curl/pull/11024 - Closes https://github.com/curl/curl/pull/11026 + Control characters without a special escape sequence e.g. %00 or %06 + were being encoded as "u0006" instead of "\u0006". -Daniel Stenberg (26 Apr 2023) + Ref: https://github.com/curl/trurl/pull/214#discussion_r1257487858 + Closes #11414 -- docs: clarify that more backends have HTTPS proxy support +Stefan Eissing (9 Jul 2023) - Closes #11033 +- http3/ngtcp2: upload EAGAIN handling -- KNOWN_BUGS: remove two not-bugs + - refs #11389 where IDLE timeouts on upload are reported + - reword ngtcp2 expiry handling to apply to both send+recv + calls into the filter + - EAGAIN uploads similar to the recent changes in HTTP/2, e.g. + report success only when send data was ACKed. + - HOLD sending of EAGAINed uploads to avoid cpu busy loops + - rename internal function for consistency with HTTP/2 + implementation - - 11.7 signal-based resolver timeouts + Fixes #11389 + Closes #11390 - Not considered a bug anymore but just implementation details. People - should avoid using timeouts with the synchronous name resolver. +Brian Nixon (9 Jul 2023) - - 11.16 libcurl uses renames instead of locking for atomic operations +- tool_easysrc.h: correct `easysrc_perform` for `CURL_DISABLE_LIBCURL_OPTION` - Not a bug, just a description of how it works + Closes #11398 - Closes #11032 +Daniel Stenberg (9 Jul 2023) -Harry Sintonen (26 Apr 2023) +- RELEASE-NOTES: synced -- hostip: add locks around use of global buffer for alarm() +- transfer: clear credentials when redirecting to absolute URL - When building with the sync name resolver and timeout ability we now - require thread-safety to be present to enable it. + Make sure the user and password for the second request is taken from the + redirected-to URL. - Closes #11030 + Add test case 899 to verify. -Daniel Stenberg (26 Apr 2023) + Reported-by: James Lucas + Fixes #11410 + Closes #11412 -- curl_path: bring back support for SFTP path ending in /~ +Stefan Eissing (8 Jul 2023) - libcurl used to do a directory listing for this case (even though the - documentation says a URL needs to end in a slash for this), but - 4e2b52b5f7a3 modified the behavior. +- hyper: fix EOF handling on input - This change brings back a directory listing for SFTP paths that are - specified exactly as /~ in the URL. + We ran out of disc space due to an infinite loop with debug logging - Reported-by: Pavel Mayorov - Fixes #11001 - Closes #11023 + Fixes #11377 + Closes #11385 + Reported-by: Dan Fandrich -Emanuele Torre (26 Apr 2023) +- http2: raise header limitations above and beyond -- docs/libcurl/curl_*escape.3: rename "url" argument to "input"/"string" + - not quite to infinity + - rewrote the implementation of our internal HTTP/1.x request + parsing to work with very large lines using dynbufs. + - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which + is also the limit of curl's general HTTP request processing. - Also reword the DESCRIPTION section to mention "input"/"string" argument - in bold. + Fixes #11405 + Closes #11407 - Closes #11027 +Juan Cruz Viotti (8 Jul 2023) -- docs/libcurl: minor cleanups +- curl_easy_nextheader.3: add missing open parenthesis examples - I was reading curl_unescape(3) and I noticed that there was an extra - space after the open parenthesis in the SYNOPSIS; I removed the extra - space. + Closes #11409 + Signed-off-by: Juan Cruz Viotti - I also ran a few grep -r commands to find and remove extra spaces - after '(' in other files, and to find and replace uses of `T*' instead - of `T *'. Some of the instances of `T*` where unnecessary casts that I - removed. +Dan Fandrich (7 Jul 2023) - I also fixed a comment that was misaligned in CURLMOPT_SOCKETFUNCTION.3. +- CI: enable verbose test output on pytest - And I fixed some formatting inconsistencies: in curl_unescape(3), all - function parameter were mentioned with bold text except length, that was - mentioned as 'length'; and, in curl_easy_unescape(3), all parameters - were mentioned in bold text except url that was italicised. Now they are - all mentioned in bold. - Documentation is not very consistent in how function parameter are - formatted: many pages italicise them, and others display them in bold - text; but I think it makes sense to at least be consistent with - formatting within the same page. + This shows individual pass/fail status on tests and makes this output + consistent with other jobs' pytest invocations. - Closes #11027 +Stefan Eissing (28 Jun 2023) -Daniel Stenberg (26 Apr 2023) +- http2: fix crash in handling stream weights -- man pages: simplify the .TH sections + - Delay the priority handling until the stream has been opened. - - remove the version numbers - - simplify the texts + - Add test2404 to reproduce and verify. - The date and version number will be put there for releases when maketgz - runs the updatemanpages.pl script. + Weights may change "on the run", which is why there are checks in + general egress handling. These must not trigger when the stream has not + been opened yet. - Closes #11029 + Reported-by: jbgoog@users.noreply.github.com -- hostcheck: fix host name wildcard checking + Fixes https://github.com/curl/curl/issues/11379 + Closes https://github.com/curl/curl/pull/11384 - The leftmost "label" of the host name can now only match against single - '*'. Like the browsers have worked for a long time. +- tests/http: Add mod_h2 directive `H2ProxyRequests` - - extended unit test 1397 for this - - move some SOURCE variables from unit/Makefile.am to unit/Makefile.inc + master of mod_h2 now requires H2ProxyRequests directives for forward + proxying with HTTP/2 to work. - Reported-by: Hiroki Kurosawa - Closes #11018 + Ref: https://github.com/icing/mod_h2/commit/3897a7086 -Dan Fandrich (25 Apr 2023) + Closes https://github.com/curl/curl/pull/11392 -- smbserver: remove temporary files before exit +Dan Fandrich (28 Jun 2023) - Each execution of test 1451 would leave a file in /tmp before. Since - Windows can't delete a file while it's open, all the temporary file - names are stored and deleted on exit. +- CI: make Appveyor job names unique - Closes #10990 + Two otherwise identical mingw-w64 jobs now have their differing compiler + versions mentioned in their names. -Stefan Eissing (25 Apr 2023) +Sheshadri.V (25 Jun 2023) -- Websocket en-/decoding +- curl.h: include for vxworks - - state is fully kept at connection, since curl_ws_send() and - curl_ws_rec() have lifetime beyond usual transfers - - no more limit on frame sizes + Closes #11356 - Reported-by: simplerobot on github - Fixes #10962 - Closes #10999 +Dan Fandrich (24 Jun 2023) -Patrick Monnerat (25 Apr 2023) +- CI: enable parallel make in more builds -- urldata: copy CURLOPT_AWS_SIGV4 value on handle duplication + Most CI services provide at least two cores, so enable parallel make + jobs to take advantage of that for builds. Some dependencies aren't safe + to build in parallel so leave those as-is. Also, rename a few + workflows to eliminate duplicate names and provide a better idea what + they're about. - Prior to this change STRING_AWS_SIGV4 (CURLOPT_AWS_SIGV4) was wrongly - marked as binary data that could not be duplicated. +- CI: don't install impacket if tests are not run - Without this fix, this option's value is not copied upon calling - curl_easy_duphandle(). + It just wastes time and bandwidth and isn't even used. - Closes https://github.com/curl/curl/pull/11021 +divinity76 (24 Jun 2023) -Stefan Eissing (25 Apr 2023) +- configure: the --without forms of the options are also gone -- http3: expire unpaused transfers in all HTTP/3 backends + --without-darwin-ssl and --without-metalink - Closes #11005 + Closes #11378 -- http2: always EXPIRE_RUN_NOW unpaused http/2 transfers +Daniel Stenberg (23 Jun 2023) - - just increasing the http/2 flow window does not necessarily - make a server send new data. It may already have exhausted - the window before +- configure: add check for ldap_init_fd - Closes #11005 + ... as otherwise the configure script will say it is OpenLDAP in the + summary, but not set the USE_OPENLDAP define, therefor not using the + intended OpenLDAP code paths. -- http2: pass `stream` to http2_handle_stream_close to avoid NULL checks + Regression since 4d7385446 (7.85.0) + Fixes #11372 + Closes #11374 + Reported-by: vlkl-sap on github - Closes #11005 +Michał Petryka (23 Jun 2023) -- h2/h3: replace `state.drain` counter with `state.dselect_bits` +- cmake: stop CMake from quietly ignoring missing Brotli - - `drain` was used by http/2 and http/3 implementations to indicate - that the transfer requires send/recv independant from its socket - poll state. Intended as a counter, it was used as bool flag only. - - a similar mechanism exists on `connectdata->cselect_bits` where - specific protocols can indicate something similar, only for the - whole connection. - - `cselect_bits` are cleard in transfer.c on use and, importantly, - also set when the transfer loop expended its `maxloops` tries. - `drain` was not cleared by transfer and the http2/3 implementations - had to take care of that. - - `dselect_bits` is cleared *and* set by the transfer loop. http2/3 - does no longer clear it, only set when new events happen. + The CMake project was set to `QUIET` for Brotli instead of + `REQUIRED`. This makes builds unexpectedly ignore missing Brotli even + when `CURL_BROTLI` is enabled. - This change unifies the handling of socket poll overrides, extending - `cselect_bits` by a easy handle specific value and a common treatment in - transfers. + Closes #11376 - Closes #11005 +Emanuele Torre (22 Jun 2023) -Daniel Stenberg (25 Apr 2023) +- docs: add more .IP after .RE to fix indentation of generate paragraphs -- socketpair: verify with a random value + follow-up from 099f41e097c030077b8ec078f2c2d4038d31353b - ... instead of using the curl time struct, since it would use a few - uninitialized bytes and the sanitizers would complain. This is a neater - approach I think. + I just thought of checking all the other files with .RE, and I found 6 + other files that were missing .IP at the end. - Reported-by: Boris Kuschel - Fixes #10993 - Closes #11015 + Closes #11375 -Stefan Eissing (25 Apr 2023) +Stefan Eissing (22 Jun 2023) -- HTTP3: document the ngtcp2/nghttp3 versions to use for building curl +- http2: h2 and h2-PROXY connection alive check fixes - - refs #11011 to clarify this for people building curl themselves + - fix HTTP/2 check to not declare a connection dead when + the read attempt results in EAGAIN + - add H2-PROXY alive check as for HTTP/2 that was missing + and is needed + - add attach/detach around Curl_conn_is_alive() and remove + these in filter methods + - add checks for number of connections used in some test_10 + proxy tunneling tests - Closes #11019 + Closes #11368 -Daniel Stenberg (25 Apr 2023) +- http2: error stream resets with code CURLE_HTTP2_STREAM -- lib: unify the upload/method handling + - refs #11357, where it was reported that HTTP/1.1 downgrades + no longer works + - fixed with suggested change + - added test_05_03 and a new handler in the curltest module + to reproduce that downgrades work - By making sure we set state.upload based on the set.method value and not - independently as set.upload, we reduce confusion and mixup risks, both - internally and externally. + Fixes #11357 + Closes #11362 + Reported-by: Jay Satiro - Closes #11017 +Daniel Stenberg (22 Jun 2023) -- RELEASE-NOTES: synced +- connect-timeout.d: mention that the DNS lookup is included -Dan Fandrich (24 Apr 2023) + Closes #11370 -- CI: don't run CI jobs if only another CI was changed +Emanuele Torre (22 Jun 2023) - A few paths were missed in the last commit, as well as a job added since - then. +- quote.d: fix indentation of generated paragraphs - Followup-to 395b9175 + quote.d was missing a .IP at the end which caused the paragraphs + generated for See-also, Multi, and Example to not be indented correctly. -- CI: adjust labeler match patterns + I also remove a redundant "This option can be used multiple times.", and + replaced .IP "item" with .TP .B "item" to make more clear which lines + are part of the list of commands and which aren't. -- runtests: support buffering log messages in runner & servers + Closes #11371 - Log messages generated with logmsg can now be buffered and returned from - the runner as a return value. This will be needed with parallel testing - to allow all messages for one test to be displayed together instead of - interspersed with messages of multiple tests. Buffering can be disabled - by setting a logging callback function with setlogfunc, which is - currently being done to preserve existing logging behaviour for now. +Paul Wise (22 Jun 2023) - Some additional output is generated in verbose and debugprotocol modes, - which don't always use logmsg. These modes also impact some servers - which generate extra messages. No attempt is made to buffer everything - if these modes are enabled. +- checksrc: modernise perl file open - Ref: #10818 - Closes #11016 + Use regular variables and separate file open modes from filenames. -- runtests: more consistently use logmsg in server control code + Suggested by perlcritic - Also, display an error when sshversioninfo returns one. + Copied from https://github.com/curl/trurl/commit/f2784a9240f47ee28a845 - Ref: #10818 + Closes #11358 -- runtests: create runner functions for clearlocks and stopservers +Dan Fandrich (21 Jun 2023) - runtests.pl now uses runner for all server actions beyond the initial - variable configuration. +- runtests: work around a perl without SIGUSR1 - Ref: #10818 + At least msys2 perl v5.32.1 doesn't seem to define this signal. Since + this signal is only used for debugging, just ignore if setting it fails. -- runtests: tightened servers package exports + Reported-by: Marcel Raad + Fixes #11350 + Closes #11366 - The defaults are intended for runtests.pl, whereas runner.pm needs to - explicitly specify them. +- runtests: include missing valgrind package -- runtests: display logs on server failure in singletest() + use valgrind was missing which caused torture tests with valgrind + enabled to fail. - This is closer to the place where logs are displayed on test failure. - Also, only display these logs if -p is given, which is the same flag - that controls display of test failure logs. Some server log files - need to be deleted later so that they stay around long enough to be - displayed on failure. + Reported-by: Daniel Stenberg + Fixes #11364 + Closes #11365 - Ref: #10818 +- runtests: use more consistent failure lines -- runtests: turn a print into a logmsg + After a test failure log a consistent log message to make it easier to + parse the log file. Also, log a consistent message with "ignored" for + failures that cause the test to be not considered at all. These should + perhaps be counted in the skipped category, but this commit does not + change that behaviour. - Also enable another couple of useful messages in verbose mode. +- runtests: consistently write the test check summary block - Ref: #10818 + The memory check character was erroneously omitted if the memory + checking file was not available for some reason, making the block of + characters an inconsistent length. -Daniel Stenberg (24 Apr 2023) +- test2600: fix the description -- http: store the password in the correct variable + It looks like it was cut-and-pasted. - Typo from fc2f1e547a4a, detected by Coverity (because there's dead code - due to this). + Closes #11354 - Closes #11002 +Daniel Stenberg (21 Jun 2023) -Stefan Eissing (24 Apr 2023) +- TODO: "Support HTTP/2 for HTTP(S) proxies" *done* -- HTTP3/quiche: terminate h1 response header when no body is sent +humbleacolyte (21 Jun 2023) - - fixes a failure in test2501 where a response without body was missing - the final empty line +- cf-socket: move ctx declaration under HAVE_GETPEERNAME - Closes #11003 + Closes #11352 -Dan Fandrich (22 Apr 2023) +Daniel Stenberg (20 Jun 2023) -- runtests: move showdiff into runtests.pl +- RELEASE-NOTES: synced - It's not used anywhere else. +- example/connect-to: show CURLOPT_CONNECT_TO -- devtest: add a new script for testing the test harness + Closes #11340 - This is currently useful for starting a test server on its own without - an associated test, which can be used for interactive curl testing or - for validating parts of the test harness itself. More commands can be - added to perform additional functions in the future. +Stefan Eissing (20 Jun 2023) - Ref: #10818 - Closes #11008 +- hyper: unslow -- runtests: refactor the main test loop into two + - refs #11203 where hyper was reported as being slow + - fixes hyper_executor_poll to loop until it is out of + tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issue + s/3237 + - added a fix in hyper io handling for detecting EAGAIN + - added some debug logs to see IO results + - pytest http/1.1 test cases pass + - pytest h2 test cases fail on connection reuse. HTTP/2 + connection reuse does not seem to work. Hyper submits + a request on a reused connection, curl's IO works and + thereafter hyper declares `Hyper: [1] operation was canceled: connection cl + osed` + on stderr without any error being logged before. - The test loop now has an initial loop that first runs through all - possible tests to build a set of those to attempt on this run based on - features and keywords and only then goes through that new list to run - them. This actually makes it three loops through all tests cases, as - there is an existing loop that gathers possible test numbers from the - test files on disk. + Fixes #11203 + Reported-by: Gisle Vanem + Advised-by: Sean McArthur + Closes #11344 - This has two minor effects on the output: all the tests that will be - skipped are displayed at the start (instead of being interspersed with - other tests) and the -l option no longer shows a count of tests at the - end or a (misleading) statement that tests have run successfully. The - skipped tests are also omitted from the test results sent to AppVeyor - and Azure in CI builds. +- HTTP/2: upload handling fixes - Another effect is a reduction in the amount of work considered part of - the "Test definition reading and preparation time" reported with -r - making those figures slightly lower than before. + - fixes #11242 where 100% CPU on uploads was reported + - fixes possible stalls on last part of a request body when + that information could not be fully send on the connection + due to an EAGAIN + - applies the same EGAIN handling to HTTP/2 proxying - Ref: #10818 + Reported-by: Sergey Alirzaev + Fixed #11242 + Closes #11342 -- runtests: track only the current test timings in runner.pm +Daniel Stenberg (20 Jun 2023) - This avoids passing these data through through global variables, which - soon won't be possible. +- example/opensslthreadlock: remove - Ref: #10818 + This shows how to setup OpenSSL mutex callbacks, but this is not + necessary since OpenSSL 1.1.0 - meaning that no currently supported + OpenSSL version requires this anymore -- runtests: skip test preprocessing when doing -l + Closes #11341 - This speeds up the output tremendously by avoiding unnecessary work. +Dan Fandrich (19 Jun 2023) -- runtests: simplify value returned regarding use of valgrind +- libtest: display the times after a test timeout error - As a side effect this will now also show in verbose mode that valgrind - is being skipped on tests that explicitly disable it, such as 600. + This is to help with test failure debugging. - Ref: #10818 + Ref: #11328 + Closes #11329 -- runtests: fix quoting in Appveyor and Azure test integration +- test2600: bump a test timeout - Test 1442's name was not quoted correctly so wasn't registered in - Appveyor and it had the wrong name in Azure. The JSON string quotes were - also invalid, even though both servers happened to accept it regardless. + Case 1 failed at least once on GHA by going 30 msec too long. - Closes #11010 + Ref: #11328 -Daniel Stenberg (19 Apr 2023) +- runtests: better detect and handle pipe errors in the controller -- RELEASE-NOTES: synced + Errors reading and writing to the pipes are now better detected and + propagated up to the main test loop so it can be cleanly shut down. Such + errors are usually due to a runner dying so it doesn't make much sense + to try to continue the test run. -Dan Fandrich (18 Apr 2023) +- runtests: cleanly abort the runner if the controller dies -- runtests: spread out the port numbers used by servers + If the controller dies unexpectedly, have the runner stop its servers + and exit cleanly. Otherwise, the orphaned servers will stay running in + the background. - The server ports are chosen randomly for each server, but the random - ranges chosen were inconsistently-sized and overlapping. Now, they are - spread out more so at least the first random port chosen for each server - is guaranteed to not also be chosen by another server. The starting port - numbers are also raised to put them in the Ephemeral Port range—not the - range defined by RFC 6335 but the one used by Linux, which starts lower - and gives us more room to work with. +- runtests: improve error logging - Reported-by: Daniel Stenberg + Give more information about test harness error conditions to help figure + out what might be wrong. Print some internal test state when SIGUSR1 is + sent to runtests.pl. -- runtests: fix problems on failure + Ref: #11328 - The verify time must be set in this case, like all cases. An error - message needs to be displayed as well. +- runtests: better handle ^C during slow tests -- runtests: fix perl warning when is wrong + Since the SIGINT handler now just sets a flag that must be checked in the + main controller loop, make sure that runs periodically. Rather than + blocking on a response from a test runner near the end of the test run, + add a short timeout to allow it. -- runtests: don't try to stop stunnel before trying again +- runtests: rename server command file - Calling stopserver() before retrying stunnel due to an error would stop - the dependent server (such as HTTP) meaning stunnel would have nothing - to talk to when it came up. Don't try to force a stop when it didn't - actually start. Also, don't mark the server as bad for future use when - it starts up on a retry. + The name ftpserver.cmd was historical and has been used for more than + ftp for many years now. Rename it to plain server.cmd to reduce + confusion. - Reported-by: eaglegai at github - Tested-by: eaglegai at github - Fixes #10976 +- tests: improve reliability of TFTP tests -- runtests: don't accidentally randomly choose the same port + Stop checking the timeout used by the client under test (for most + tests). The timeout will change if the TFTP test server is slow (such as + happens on an overprovisioned CI server) because the client will retry + and reduce its timeout, and the actual value is not important for most + tests. - If a server couldn't be started on a port, a new one is randomly chosen - and the server is tried again. Avoid accidentally using a - randomly-chosen 0 port offset by adding 1 to the random number. + test285 is changed a different way, by increasing the connect timeout. + This improves test coverage by allowing the changed timeout value to be + checked, but improves reliability with a carefully-chosen timeout that + not only allows twice the time to respond as before, but also allows + several retries before the client will change its timeout value. - Found-by: Daniel Stenberg + Ref: #11328 -- runtests: don't attempt to use a port we know is in use +Daniel Stenberg (19 Jun 2023) - This reduces the startup time when there is a known conflict on the - random port chosen for a server. This was already done for stunnel, but - now it's done for all servers. +- cf-socket: skip getpeername()/getsockname for TFTP -- http-server: fix server name in a log message + Since the socket is not connected then the call fails. When the call + fails, failf() is called to write an error message that is then + surviving and is returned when the *real* error occurs later. The + earlier, incorrect, error therefore hides the actual error message. - This changed when the file was renamed in commit cbf57176 + This could be seen in stderr for test 1007 -- runtests: refactor into more packages + Test 1007 has now been extended to verify the stderr message. - testutil.pm now contains a few miscellaneous functions that are used in - several places but have no better place to live. subvariables moves to - servers.pm since most variables that it substitutes relate to servers, - so this is the most appropriate place. Rename a few functions for better - naming consistency. + Closes #11332 - Ref: #10818 - Closes #10995 +- example/crawler: make it use a few more options -- runtests: call timestampskippedevents() in singletest + For show, but reasonable - ..rather than by the runner +- libcurl-ws.3: mention raw mode -- runtests: assume a newer Valgrind by default + Closes #11339 - The tests for an older Valgrind version should probably just be deleted, - given that they're testing for an 18-year-old version. +- example/default-scheme: set the default scheme for schemeless URLs -- runtests: refactor test runner code into runner.pm + Closes #11338 - This is code that is directly responsible for running a single test. - This will eventually run in a separate process as part of the parallel - testing project. +- example/hsts-preload: show one way to HSTS preload - Ref: #10818 + Closes #11337 -- runtests: skip unneeded work if test won't be running +- examples/http-options: show how to send "OPTIONS *" - This speeds up tests by avoiding unnecessary processing. + With CURLOPT_REQUEST_TARGET. - Ref: #10818 + Also add use of CURLOPT_QUICK_EXIT to show. -- runtests: factor out singletest_postcheck + Closes #11333 - This will eventually need to be part of the test runner. +- examples: make use of CURLOPT_(REDIR_|)PROTOCOLS_STR - Ref: #10818 + To show how to use them -- test303: kill server after test + Closes #11334 - Otherwise, an HTTP test closely following this one with a tight time - constraint (e.g. 672) could fail because the test server stays sitting - with the wait command for a while. +- examples/smtp-mime: use CURLOPT_MAIL_RCPT_ALLOWFAILS -Patrick Monnerat (18 Apr 2023) + For show -- OS400: provide ILE/RPG usage examples + Closes #11335 - Closes https://github.com/curl/curl/pull/10994 +- http: rectify the outgoing Cookie: header field size check -- OS400: improve vararg emulation + Previously it would count the size of the entire outgoing request and + not just the size of only the Cookie: header field - which was the + intention. - - Use V7R4 RPG procedure overloading to improve vararg emulation. + This could make the check be off by several hundred bytes in some cases. - From OS400 V7R4 and above, ILE/RPG implements a limited procedure - overloading feature that can be used to improve curl's typed - implementation of varargs procedures. This commit applies it to - curl_easy_setopt(), curl_multi_setopt(), curl_share_setopt() and - curl_easy_getinfo(). + Closes #11331 - Closes https://github.com/curl/curl/pull/10994 +Jay Satiro (17 Jun 2023) -- OS400: fix and complete ILE/RPG binding +- lib: fix some format specifiers - - Fix wrong definitions of CURL_ZERO_TERNINATED, curl_mime_data() and - curl_mime_data_ccsid(). + - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some + curl_off_t variables. - - Add recent definitions, in particular blob, header API and WebSockets - API. + - Use %zu where %zd was erroneously used for some size_t variables. - - Support for CURLVERSION_ELEVENTH. + Prior to this change some of the Windows CI tests were failing because + in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t. + When %zd was used for some curl_off_t variables then only the lower + 32-bits was read and the upper 32-bits would be read for part or all of + the next specifier. - - New functions for EBCDIC support. + Fixes https://github.com/curl/curl/issues/11327 + Closes https://github.com/curl/curl/pull/11321 - Reflect these changes in README.OS400. +Marcel Raad (16 Jun 2023) - Closes https://github.com/curl/curl/pull/10994 +- test427: add `cookies` feature and keyword -- OS400: implement EBCDIC support for recent features + This test doesn't work with `--disable-cookies`. - - Support CURLVERSION_ELEVENTH. + Closes https://github.com/curl/curl/pull/11320 - - New function curl_url_strerror_ccsid(). +Chris Talbot (15 Jun 2023) - - curl_easy_setopt_ccsid() supports blobs and 3 recent string options. +- imap: Provide method to disable SASL if it is advertised - - New function curl_easy_header_ccsid(). + - Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext + LOGIN over SASL auth. - - New generic latin1<-->ccsid conversion functions curl_from_ccsid() and - curl_to_ccsid() for user convenience. + Prior to this change there was no method to be able to fall back to + LOGIN if an IMAP server advertises SASL capabilities. However, this may + be desirable for e.g. a misconfigured server. - - README.OS400 updated accordingly. + Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2 - - Removed a leftover QsoSSL support identifier. + ";AUTH=" looks to be the correct way to specify what + authenication method to use, regardless of SASL or not. - Closes https://github.com/curl/curl/pull/10994 + Closes https://github.com/curl/curl/pull/10041 -- OS400: rework build scripts +Daniel Stenberg (15 Jun 2023) - - Rename shell function "system" to "CLcommand" to avoid confusion with - built-in command. +- RELEASE-NOTES: synced - - Reformat scripts. Fix some indentations. Avoid lines > 80 characters - where possible. +- examples/multi-debugcallback.c: avoid the bool typedef - - Support ASCII runtime development files in a user-defined directory - path. + Apparently this cannot be done in c23 - - FIX SONAME detection. + Reported-by: Cristian Rodríguez + Fixes #11299 + Closes #11319 - - Drop form API test program compilation (does not exist anymore). +- docs/libcurl/libcurl.3: cleanups and improvements - Closes https://github.com/curl/curl/pull/10994 + Closes #11317 -Sevan Janiyan (18 Apr 2023) +- libcurl-ws.3: fix typo -- tests/sshserver.pl: Define AddressFamily earlier +- curl_ws_*.3: enhance - As the comment states "Address family must be specified before ListenAddress" - , otherwise the tests fail to run - `"failed starting SSH server" 52 times (582, 583, 600, 601, 602, 603, 604, 60 - 5, 606 and 43 more)` + - all: SEE ALSO the libcurl-ws man page + - send: add example and return value information + - meta: mention that the returned data is read-only - Closes #10983 + Closes #11318 -Stefan Eissing (18 Apr 2023) +- docs/libcurl/libcurl-ws.3: see also CURLOPT_WS_OPTIONS -- quiche: Enable IDLE egress handling +- docs/libcurl/libcurl-ws.3: minor polish - Follow-up to 544abeea which added the handling but wrongly left it - commented out. +- libcurl-ws.3. WebSocket API overview - Closes https://github.com/curl/curl/pull/11000 + Closes #11314 -Daniel Stenberg (18 Apr 2023) +- libcurl-url.3: also mention CURLUPART_ZONEID -- docs/examples/protofeats.c: Outputs all protocols and features + ... and sort the two part-using lists alphabetically - Showing off one way to get to char pointer arrays of info returned by - curl_version_info() +Marcel Raad (14 Jun 2023) - Closes #10991 +- fopen: fix conversion warning on 32-bit Android -- tests/keywords.pl: remove + When building for 32-bit ARM or x86 Android, `st_mode` is defined as + `unsigned int` instead of `mode_t`, resulting in a + -Wimplicit-int-conversion clang warning because `mode_t` is + `unsigned short`. Add a cast to silence the warning. - This script does not work since the introduction of the test - preprocessing. If we need this functionality, it probably needs to be - moved into the runtests tool or similar. + Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li + bc/include/sys/stat.h#86 + Closes https://github.com/curl/curl/pull/11313 - Reported-by: Dan Fandrich - Fixes #10895 - Closes #10987 +- http2: fix variable type -Stefan Eissing (17 Apr 2023) + `max_recv_speed` is `curl_off_t`, so using `size_t` might result in + -Wconversion GCC warnings for 32-bit `size_t`. Visible in the NetBSD + ARM autobuilds. -- http2: support HTTP/2 to forward proxies, non-tunneling + Closes https://github.com/curl/curl/pull/11312 - - with `--proxy-http2` allow h2 ALPN negotiation to - forward proxies - - applies to http: requests against a https: proxy only, - as https: requests will auto-tunnel - - adding a HTTP/1 request parser in http1.c - - removed h2h3.c - - using new request parser in nghttp2 and all h3 backends - - adding test 2603 for request parser - - adding h2 proxy test cases to test_10_* +Daniel Stenberg (13 Jun 2023) - scorecard.py: request scoring accidentally always run curl - with '-v'. Removed that, expect double numbers. +- vtls: fix potentially uninitialized local variable warnings - labeller: added http1.* and h2-proxy sources to detection + Follow-up from a4a5e438ae533c - Closes #10967 + Closes #11310 -Daniel Stenberg (17 Apr 2023) +- timeval: use CLOCK_MONOTONIC_RAW if available -- curl_easy_unescape.3: rename the argument + Reported-by: Harry Sintonen + Ref: #11288 + Closes #11291 - and highlight it appropriately in the text. +Stefan Eissing (12 Jun 2023) - Closes #10979 +- tool: add curl command line option `--trace-ids` -Viktor Szakats (17 Apr 2023) + - added and documented --trace-ids to prepend (after the timestamp) + the transfer and connection identifiers to each verbose log line + - format is [n-m] with `n` being the transfer id and `m` being the + connection id. In case there is not valid connection id, print 'x'. + - Log calls with a handle that has no transfer id yet, are written + without any ids. -- autotools: sync up clang picky warnings with cmake + Closes #11185 - Bringing missing options over from CMake. +- lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID - Move around existing `-Wno-pointer-bool-conversion` option to come - _after_ `-Wconversion`. + - add an `id` long to Curl_easy, -1 on init + - once added to a multi (or its own multi), it gets + a non-negative number assigned by the connection cache + - `id` is unique among all transfers using the same + cache until reaching LONG_MAX where it will wrap + around. So, not unique eternally. + - CURLINFO_CONN_ID returns the connection id attached to + data or, if none present, data->state.lastconnect_id + - variables and type declared in tool for write out - Reviewed-by: Marcel Raad - Closes #10974 + Closes #11185 -Daniel Stenberg (17 Apr 2023) +Daniel Stenberg (12 Jun 2023) -- tests/libtest/lib1900.c: remove +- CURLOPT_INFILESIZE.3: mention -1 triggers chunked - This file was left behind when the rest of the test was previously removed. + Ref: #11300 + Closes #11304 - Follow-up to e50a877df74f +Philip Heiduck (12 Jun 2023) -- src/tool_operhlp.c: fix value stored to 'uerr' is never read +- CI: openssl-3.0.9+quic - Ref: https://github.com/curl/curl/pull/10974#issuecomment-1510461343 - Reported-by: Viktor Szakats - Closes #10982 + Closes #11296 -Viktor Szakats (16 Apr 2023) +Karthikdasari0423 (12 Jun 2023) -- cmake: speed up and extend picky clang/gcc options +- HTTP3.md: update openssl version - Extend existing picky compiler options with ones missing compared to - autotools builds. Also sync options between clang and gcc. + Closes #11297 - Redesign the way we enable these options to avoid the slow option - detection almost completely. +Daniel Stenberg (12 Jun 2023) - This reduces the number of detections from 35 to zero for clang and - 3 for gcc, even after adding a bunch of new options. +- vtls: avoid memory leak if sha256 call fails - clang 3.0 (2011-11-29) and gcc 2.95 (1999-07-31) now required. + ... in the pinned public key handling function. - Also show enabled picky options. + Reported-by: lizhuang0630 on github + Fixes #11306 + Closes #11307 - Ref: https://github.com/libssh2/libssh2/pull/952 +- examples/ipv6: disable on win32 - Reviewed-by: Daniel Stenberg - Closes #10973 + I can't make if_nametoindex() work there -Andreas Falkenhahn (16 Apr 2023) + Follow-up to c23dc42f3997acf23 -- nbtlm: use semicolons instead of commas for (void) args + Closes #11305 - Closes #10978 +- tool_operate: allow cookie lines up to 8200 bytes -Daniel Stenberg (15 Apr 2023) + Since this option might set multiple cookies in the same line, it does + not make total sense to cap this at 4096 bytes, which is the limit for a + single cookie name or value. -- multi: free up more data earleier in DONE + Closes #11303 - Before checking for more users of the connection and possibly bailing - out. +- test427: verify sending more cookies than fit in a 8190 bytes line - Fixes #10971 - Reported-by: Paweł Wegner - Closes #10972 + curl will then only populate the header with cookies that fit, dropping + ones that otherwise would have been sent -- RELEASE-NOTES: synced + Ref: https://curl.se/mail/lib-2023-06/0020.html -- curl: do NOT append file name to path for upload when there's a query + Closes #11303 - Added test 425 to verify. +- testutil: allow multiple %-operators on the same line - Reported-by: Dirk Rosenkranz - Bug: https://curl.se/mail/archive-2023-04/0008.html - Closes #10969 + Closes #11303 -- libcurl-thread.3: improved name resolver wording +Oleg Jukovec (12 Jun 2023) - And make better .SH sections +- docs: update CURLOPT_UPLOAD.3 - Closes #10966 + The behavior of CURLOPT_UPLOAD differs from what is described in the + documentation. The option automatically adds the 'Transfer-Encoding: + chunked' header if the upload size is unknown. -Colman Mbuya (14 Apr 2023) + Closes #11300 -- CURLOPT_PROXY_SSL_VERIFYPEER.3: fix minor grammar mistake +Daniel Stenberg (12 Jun 2023) - Closes #10968 +- RELEASE-NOTES: synced -Daniel Stenberg (14 Apr 2023) +- CURLOPT_AWS_SIGV4.3: remove unused variable from example -- curl: add --proxy-http2 + Closes #11302 - For trying HTTP/2 with an HTTPS proxy. +- examples/https.c: use CURLOPT_CA_CACHE_TIMEOUT - Closes #10926 + for demonstration purposes -- KNOWN_BUGS: remove fixed or outdated issues, move non-bugs + Closes #11290 - - remove h3 issues believed to be fixed +- example/ipv6: feature CURLOPT_ADDRESS_SCOPE in use - - make the flaky CI issue be generic and not Windows specific + Closes #11282 - - "TLS session cache does not work with TFO" now documented +Karthikdasari0423 (10 Jun 2023) - This is now a documented restriction and not a bug. TFO in general is - rarely used and has other problems, making it a low-priotity thing to - work on. +- docs: Update HTTP3.md for newer ngtcp2 and nghttp3 - - remove "Renegotiate from server may cause hang for OpenSSL backend" + Follow-up to fb9b9b58 - This is an OpenSSL issue, not a curl one. Even if it taints curl. + Ref: #11184 + Closes #11295 - - rm "make distclean loops forever" +Dan Fandrich (10 Jun 2023) - - rm "configure finding libs in wrong directory" +- docs: update the supported ngtcp2 and nghttp3 versions - Added a section to docs/INSTALL.md about it. + Follow-up to cae9d10b - - "A shared connection cache is not thread-safe" + Ref: #11184 + Closes #11294 - Moved over to TODO and expanded for other sharing improvements we - could do +- tests: fix error messages & handling around sockets - - rm "CURLOPT_OPENSOCKETPAIRFUNCTION is missing" + The wrong error code was checked on Windows on UNIX socket failures, + which could have caused all UNIX sockets to be reported as having + errored and the tests therefore skipped. Also, a useless error message + was displayed on socket errors in many test servers on Windows because + strerror() doesn't work on WinSock error codes; perror() is overridden + there to work on all errors and is used instead. - - rm "Blocking socket operations in non-blocking API" + Ref #11258 + Closes #11265 - Already listed as a TODO +Daniel Stenberg (9 Jun 2023) - - rm "curl compiled on OSX 10.13 failed to run on OSX 10.10" +- CURLOPT_SSH_PRIVATE_KEYFILE.3: expand on the file search - Water under the bridge. No one cares about this anymore. + Reported-by: atjg on github + Ref: #11287 + Closes #11289 - - rm "build on Linux links libcurl to libdl" +Stefan Eissing (9 Jun 2023) - Verified to not be true (anymore). +- ngtcp2: use ever increasing timestamp in io - - rm "libpsl is not supported" + - ngtcp2 v0.16.0 asserts that timestamps passed to its function + will only ever increase. + - Use a context shared between ingress/egress operations that + uses a shared timestamp, regularly updated during calls. - The cmake build supports it since cafb356e19cda22 + Closes #11288 - Closes #10963 +Daniel Stenberg (9 Jun 2023) -- url: fix PVS nits +- GHA: use nghttp2 1.54.0 for the ngtcp2 jobs - - expression 'hostptr' is always true - - a part of conditional expression is always true: proxypasswd - - expression 'proxyuser' is always true - - avoid multiple Curl_now() calls in allocate_conn +Philip Heiduck (9 Jun 2023) - Ref: #10929 - Closes #10959 +- GHA: ngtcp2: use 0.16.0 and nghttp3 0.12.0 -- bufq: simplify since expression is always true +Daniel Stenberg (9 Jun 2023) - The check for 'len' is already done so it will remain true until - updated. Pointed out by PVS. +- ngtcp2: build with 0.16.0 and nghttp3 0.12.0 - Ref: #10929 - Closes #10958 + - moved to qlog_write + - crypto => encryption + - CRYPTO => ENCRYPTION + - removed "_is_" + - ngtcp2_conn_shutdown_stream_read and + ngtcp2_conn_shutdown_stream_write got flag arguments + - the nghttp3_callbacks struct got a recv_settings callback -- hash: fix assigning same value + Closes #11184 - Pointed out by PVS +- example/http2-download: set CURLOPT_BUFFERSIZE - Ref: #10929 - Closes #10956 + Primarily because no other example sets it, and remove the disabling of + the certificate check because we should not recommend that. -- cookie: address PVS nits + Closes #11284 - - avoid assigning the same value again - - remove superfluous check of co->domain - - reduce variable scope for namep/valuep +- example/crawler: also set CURLOPT_AUTOREFERER - Ref: #10929 - Closes #10954 + Could make sense, and it was not used in any example before. -Stefan Eissing (14 Apr 2023) + Closes #11283 -- cf-socket: Disable socket receive buffer by default +Wyatt OʼDay (9 Jun 2023) - - Disable socket receive buffer unless USE_RECV_BEFORE_SEND_WORKAROUND - is in place. +- tls13-ciphers.d: include Schannel - While we would like to use the receive buffer, we have stalls in - parallel transfers where not all buffered data is consumed and no socket - events happen. + Closes #11271 - Note USE_RECV_BEFORE_SEND_WORKAROUND is a Windows sockets workaround - that has been disabled by default since b4b6e4f1, due to other bugs. +Daniel Stenberg (9 Jun 2023) - Closes https://github.com/curl/curl/pull/10961 +- curl_pushheader_byname/bynum.3: document in their own man pages -- cf-h2-proxy: fix processing ingress to stop too early + These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was + introduced but always lived a life in the shadows, embedded in the + CURLMOPT_PUSHFUNCTION man page. Until now. - - progress ingress stopped too early, causing data - from the underlying filters to not be processed and - report that no tunnel data was available - - this lead to "hangers" where no socket activity was - seen but data rested in buffers + It makes better sense and gives more visibility to document them in + their own stand-alone man pages. - Closes #10952 + Closes #11286 -- http3: check stream_ctx more thoroughly in all backends +- curl_mprintf.3: minor fix of the example - - callbacks and filter methods might be invoked at unexpected - times, e.g. when the transfer's stream_ctx has not been initialized - yet or, more likely, has already been taken down. - - check for existance of stream_ctx in such places and return - an error or silently succeed the call. +- curl_url_set: enforce the max string length check for all parts - Closes #10951 + Update the docs and test 1559 accordingly -Daniel Stenberg (13 Apr 2023) + Closes #11273 -- ftp: fix 'portsock' variable was assigned the same value +- examples/ftpuploadresume.c: add use of CURLOPT_ACCEPTTIMEOUT_MS - Pointed out by PVS + For show - Ref: #10929 - Closes #10955 + Closes #11277 -- ftp: remove dead code +- examples/unixsocket.c: example using CURLOPT_UNIX_SOCKET_PATH - This condition can never be true here since it is handled already 28 - lines above. + and alternatively CURLOPT_ABSTRACT_UNIX_SOCKET - Pointed out by PVS. + Closes #11276 - Ref: #10929 - Closes #10957 +Anssi Kolehmainen (8 Jun 2023) -- cf-h1-proxy: skip an extra NULL assign +- docs: fix missing parameter names in examples - and use Curl_safefree() once to save another NULL assign. Found by PVS. + Closes #11278 - Ref. #10929 - Closes #10953 +Daniel Stenberg (8 Jun 2023) -Philip Heiduck (13 Apr 2023) +- urlapi: have *set(PATH) prepend a slash if one is missing -- GHA: suppress git clone output + Previously the code would just do that for the path when extracting the + full URL, which made a subsequent curl_url_get() of the path to + (unexpectedly) still return it without the leading path. - Follow-up: https://github.com/curl/curl/commit/8203aa6ed405ec832d2c62f18dfda2 - 93f89a23f9 + Amend lib1560 to verify this. Clarify the curl_url_set() docs about it. - Closes #10949 + Bug: https://curl.se/mail/lib-2023-06/0015.html + Closes #11272 + Reported-by: Pedro Henrique -Stefan Eissing (13 Apr 2023) +Dan Fandrich (7 Jun 2023) -- cf-socket: remove dead code discovered by PVS +- runtests; give each server a unique log lock file - Closes #10960 + Logs are written by several servers and all of them must be finished + writing before the test results can be determined. This means each + server must have its own lock file rather than sharing a single one, + which is how it was done up to now. Previously, the first server to + complete a test would clear the lock before the other server was done, + which caused flaky tests. -Daniel Stenberg (13 Apr 2023) + Lock files are now all found in their own directory, so counting locks + equals counting the files in that directory. The result is that the + proxy logs are now reliably written which actually changes the expected + output for two tests. -- http: skip a double NULL assign + Fixes #11231 + Closes #11259 - and also use a local variable to shorten the long names and increase - readability in the function. Pointed out by PVS. +- runtests: make test file directories in log/N - Ref: #10929 - Closes #10950 + Test files in subdirectories were not created after parallel test log + directories were moved down a level due to a now-bad comparison. -- mime: skip NULL assigns after Curl_safefree() + Follow-up to 92d7dd39 - Pointed out by PVS. + Ref #11264 + Closes #11267 - Ref: #10929 - Closes #10947 +Daniel Stenberg (7 Jun 2023) -- rtsp: skip NULL assigns after Curl_safefree() +- ws: make the curl_ws_meta() return pointer a const - ... since this is a macro that assigns NULL itself. Pointed out by PVS. + The returned info is read-only for the user. - Ref: #10929 - Closes #10946 + Closes #11261 -- smb: remove double assign +- RELEASE-NOTES: synced - The same value is assigned the same value already a few lines above. - Pointed out by PVS. +- runtests: move parallel log dirs from logN to log/N - Ref: #10929 - Closes #10945 + Having several hundreds of them in there gets annoying. -- transfer: skip extra assign + Closes #11264 - The 'result' variable already contains CURLE_OK at this point, no use in - setting it again. Pointed out by PVS. +Dan Fandrich (7 Jun 2023) - Ref: #10929 - Closes #10944 +- test447: move the test file into %LOGDIR -- urlapi: skip a pointless assign +Viktor Szakats (7 Jun 2023) - It stores a null byte after already having confirmed there is a null - byte there. Detected by PVS. +- cmake: add support for "unity" builds - Ref: #10929 - Closes #10943 + Aka "jumbo" or "amalgamation" builds. It means to compile all sources + per target as a single C source. This is experimental. -Philip Heiduck (13 Apr 2023) + You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. + It requires CMake 3.16 or newer. -- GHA: suppress git clone output + It makes builds (much) faster, allows for better optimizations and tends + to promote less ambiguous code. - Closes #10939 + Also add a new AppVeyor CI job and convert an existing one to use + "unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job. -Stefan Eissing (13 Apr 2023) + Fix related issues: + - add missing include guard to `easy_lock.h`. + - rename static variables and functions (and a macro) with names reused + across sources, or shadowed by local variables. + - add an `#undef` after use. + - add a missing `#undef` before use. + - move internal definitions from `ftp.h` to `ftp.c`. + - `curl_memory.h` fixes to make it work when included repeatedly. + - stop building/linking curlx bits twice for a static-mode curl tool. + These caused doubly defined symbols in unity builds. + - silence missing extern declarations compiler warning for ` _CRT_glob`. + - fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`. + - fix colliding static symbols in debug mode: `debugtime()` and + `statename`. + - rename `ssl_backend_data` structure to unique names for each + TLS-backend, along with the `ssl_connect_data` struct member + referencing them. This required adding casts for each access. + - add workaround for missing `[P]UNICODE_STRING` types in certain Windows + builds when compiling `lib/ldap.c`. To support "unity" builds, we had + to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows + `schannel.h` option) _globally_. This caused an indirect inclusion of + Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled + as well. This requires `[P]UNICODE_STRING` types, which is apperantly + not defined automatically (as seen with both MSVS and mingw-w64). + This patch includes `` to fix it. + Ref: https://github.com/curl/curl/runs/13987772013 + Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&vie + w=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f + 38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c + - tweak unity builds to compile `lib/memdebug.c` separately in memory + trace builds to avoid PP confusion. + - force-disable unity for test programs. + - do not compile and link libcurl sources to libtests _twice_ when libcurl + is built in static mode. -- tests: make test_12_01 a bit more forgiving on connection counts + KNOWN ISSUES: + - running tests with unity builds may fail in cases. + - some build configurations/env may not compile in unity mode. E.g.: + https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfu + auwl8q#L250 -- cf-socket: add socket recv buffering for most tcp cases + Ref: https://github.com/libssh2/libssh2/issues/1034 + Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html + Ref: https://en.wikipedia.org/wiki/Unity_build - - use bufq as recv buffer, also for Windows pre-receive handling - - catch small reads followed by larger ones in a single socket - call. A common pattern on TLS connections. + Closes #11095 - Closes #10787 +Daniel Stenberg (7 Jun 2023) -Daniel Stenberg (13 Apr 2023) +- examples/websocket.c: websocket example using CONNECT_ONLY -- urlapi: cleanups + Closes #11262 - - move host checks together - - simplify the scheme parser loop and the end of host name parser - - avoid itermediate buffer storing in multiple places - - reduce scope for several variables - - skip the Curl_dyn_tail() call for speed - - detect IPv6 earlier and skip extra checks for such hosts - - normalize directly in dynbuf instead of itermediate buffer - - split out the IPv6 parser into its own funciton - - call the IPv6 parser directly for ipv6 addresses - - remove (unused) special treatment of % in host names - - junkscan() once in the beginning instead of scattered - - make junkscan return error code - - remove unused query management from dedotdotify() - - make Curl_parse_login_details use memchr - - more use of memchr() instead of strchr() and less strlen() calls - - make junkscan check and return the URL length +- websocket-cb: example doing WebSocket download using callback - An optimized build runs one of my benchmark URL parsing programs ~41% - faster using this branch. (compared against the shipped 7.88.1 library - in Debian) + Very basic - Closes #10935 + Closes #11260 -Josh McCullough (13 Apr 2023) +- test/.gitignore: ignore log* -- http2: fix typo in infof() call +Dan Fandrich (5 Jun 2023) - Closes #10940 +- runtests: document the -j parallel testing option -Daniel Stenberg (12 Apr 2023) + Reported-by: Daniel Stenberg + Ref: #10818 + Closes #11255 -- noproxy: pointer to local array 'hostip' is stored outside scope +- runtests: create multiple test runners when requested - Ref: #10929 - Closes #10933 + Parallel testing is enabled by using a nonzero value for the -j option + to runtests.pl. Performant values seem to be about 7*num CPU cores, or + 1.3*num CPU cores if Valgrind is in use. -Stefan Eissing (12 Apr 2023) + Flaky tests due to improper log locking (bug #11231) are exacerbated + while parallel testing, so it is not enabled by default yet. -- connect: fix https connection setup to treat ssl_mode correctly + Fixes #10818 + Closes #11246 - - for HTTPS protocol, a disabled ssl should never be acceptables. +- runtests: handle repeating tests in multiprocess mode - Closes #10934 + Such as what happens with the --repeat option. Some functions are + changed to pass the runner ID instead of relying on the non-unique test + number. -Douglas R. Reno (12 Apr 2023) + Ref: #10818 -- CMakeLists.txt: fix typo for Haiku detection +- runtests: buffer logmsg while running singletest() - Closes #10937 + This allows all messages relating to a single test case to be displayed + together at the end of the test. -Dan Fandrich (11 Apr 2023) + Ref: #10818 -- pathhelp: use the cached $use_cygpath when available +- runtests: call initserverconfig() in the runner -- runtests: eliminate unneeded variable + This must be done so variables pick up the runner's unique $LOGDIR. -- runtests: make the # of server start attempts a constant + Ref: #10818 -- runtests: on startup failure call displaylogs only in serverfortest +- runtests: use a per-runner random seed - This reduces the number of calls spread throughout the code. + Each runner needs a unique random seed to reduce the chance of port + number collisions. The new scheme uses a consistent per-runner source of + randomness which results in deterministic behaviour, as it did before. Ref: #10818 - Closes #10919 -- runtests: return an error code with startservers() +- runtests: complete main test loop refactor for multiple runners - The code indicates the kind of failure encountered in starting a server, - which can be used by the caller to tailor the user experience. + The main test loop is now able to handle multiple runners, or no + additional runner processes at all. At most one process is still + created, however. Ref: #10818 -- runtests: abort early if runpingpongserver is given a bad server type +- runtests: prepare main test loop for multiple runners -- runtests: don't use the SMB server verification time as reference + Some variables are expanded to arrays and hashes so that multiple + runners can be used for running tests. - %FTPTIME2 and %FTPTIME3 should be set by the FTP server only, for - consistency. + Ref: #10818 -- tests: factor out the test server management code +Stefan Eissing (5 Jun 2023) - This now lives in servers.pm with some configuration variables moved to - globalconfig.pm +- bufq: make write/pass methods more robust - Ref: #10818 + - related to #11242 where curl enters busy loop when + sending http2 data to the server -- runtests: remove an inappropriate use of runclientoutput + Closes #11247 - This function is intended for running client code, not servers. +Boris Verkhovskiy (5 Jun 2023) -- runtests: only add $LIBDIR to the path for checktestcmd +- tool_getparam: fix comment - Since checkcmd is for finding servers, there will never be anything in - this directory of interest to them. + Closes #11253 - Ref: #10818 +Raito Bezarius (5 Jun 2023) -- tests: log sshserver.pl messages to a file +- haproxy: add --haproxy-clientip flag to spoof client IPs - The logmsg messages were thrown away before, so they are now available - for debugging. + CURLOPT_HAPROXY_CLIENT_IP in the library -- runtests: also show DISABLED tests with -l + Closes #10779 - Other reasons for skipping tests are ignored for -l, so being explicitly - disabled should be too. +Daniel Stenberg (5 Jun 2023) -- runtests: move the UNIX sockets into $PIDDIR +- curl: add --ca-native and --proxy-ca-native - These were missed when the other server files were moved there. + These are two boolean options to ask curl to use the native OS's CA + store when verifying TLS servers. For peers and for proxies + respectively. - Follow-up to 70d2fca2 + They currently only have an effect for curl on Windows when built to use + OpenSSL for TLS. - Ref: #10818 + Closes #11049 -- tests: tighten up perl exports +Viktor Szakats (5 Jun 2023) - This reduces namespace pollution a little. +- build: drop unused/redundant `HAVE_WINLDAP_H` - Ref: #10818 + Sources did not use it. Autotools used it when checking for the + `winldap` library, which is redundant. -- tests: turn perl modules into full packages + With CMake, detection was broken: + ``` + Run Build Command(s):/usr/local/Cellar/cmake/3.26.3/bin/cmake -E env VERBOSE= + 1 /usr/bin/make -f Makefile cmTC_2d8fe/fast && /Library/Developer/CommandLine + Tools/usr/bin/make -f CMakeFiles/cmTC_2d8fe.dir/build.make CMakeFiles/cmTC_2 + d8fe.dir/build + Building C object CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj + /usr/local/opt/llvm/bin/clang --target=x86_64-w64-mingw32 --sysroot=/usr/loca + l/opt/mingw-w64/toolchain-x86_64 -D_WINSOCKAPI_="" -I/my/quictls/x64-ucrt/usr + /include -I/my/zlib/x64-ucrt/usr/include -I/my/brotli/x64-ucrt/usr/include -W + no-unused-command-line-argument -D_UCRT -DCURL_HIDDEN_SYMBOLS -DHAVE_SSL_SE + T0_WBIO -DHAS_ALPN -DNGHTTP2_STATICLIB -DNGHTTP3_STATICLIB -DNGTCP2_STATICLIB + -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static-libgcc -lucrt -Wextra -Wall -p + edantic -Wbad-function-cast -Wconversion -Winline -Wmissing-declarations -Wmi + ssing-prototypes -Wnested-externs -Wno-long-long -Wno-multichar -Wpointer-ari + th -Wshadow -Wsign-compare -Wundef -Wunused -Wwrite-strings -Wcast-align -Wde + claration-after-statement -Wempty-body -Wendif-labels -Wfloat-equal -Wignored + -qualifiers -Wno-format-nonliteral -Wno-sign-conversion -Wno-system-headers - + Wstrict-prototypes -Wtype-limits -Wvla -Wshift-sign-overflow -Wshorten-64-to- + 32 -Wdouble-promotion -Wenum-conversion -Wunused-const-variable -Wcomma -Wmis + sing-variable-declarations -Wassign-enum -Wextra-semi-stmt -MD -MT CMakeFile + s/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -MF CMakeFiles/cmTC_2d8fe.dir/HAVE_WINL + DAP_H.c.obj.d -o CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -c /my/curl/b + ld-cmake-llvm-x64-shared/CMakeFiles/CMakeScratch/TryCompile-3JP6dR/HAVE_WINLD + AP_H.c + In file included from /my/curl/bld-cmake-llvm-x64-shared/CMakeFiles/CMakeScra + tch/TryCompile-3JP6dR/HAVE_WINLDAP_H.c:2: + In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi + ngw32/include/winldap.h:17: + In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi + ngw32/include/schnlsp.h:9: + In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi + ngw32/include/schannel.h:10: + /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt + .h:5041:254: error: unknown type name 'PSYSTEMTIME' + WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ + NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, + DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER + pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT + ENSIONS pExtensions); + + + + ^ + /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt + .h:5041:278: error: unknown type name 'PSYSTEMTIME' + WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ + NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, + DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER + pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT + ENSIONS pExtensions); + + + + ^ + 2 errors generated. + make[1]: *** [CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj] Error 1 + make: *** [cmTC_2d8fe/fast] Error 2 + exitCode: 2 + ``` - This helps enforce more modularization and encapsulation. Enable and fix - warnings on a few packages. Also, rename ftp.pm to processhelp.pm since - there's really nothing ftp-specific in it. + Cherry-picked from #11095 88e4a21ff70ccef391cf99c8165281ff81374503 + Reviewed-by: Daniel Stenberg + Closes #11245 - Ref: #10818 +Daniel Stenberg (5 Jun 2023) -Daniel Stenberg (11 Apr 2023) +- urlapi: scheme starts with alpha -- multi: remove a few superfluous assigns + Add multiple tests to lib1560 to verify - PVS found these "The 'rc' variable was assigned the same value." cases. + Fixes #11249 + Reported-by: ad0p on github + Closes #11250 - Ref: #10929 - Closes #10932 +- RELEASE-NOTES: synced -- schannel: add clarifying comment +- CURLOPT_MAIL_RCPT_ALLOWFAILS: replace CURLOPT_MAIL_RCPT_ALLLOWFAILS - Explaining how the PVS warning in #10929 is wrong: Dereferencing of the - null pointer 'backend->cred' might take place. + Deprecate the name using three Ls and prefer the name with two. - Closes #10931 + Replaces #10047 + Closes #11218 -- cookie: clarify that init with data set to NULL reads no file +- tests/servers: generate temp names in /tmp for unix domain sockets - ... and make Curl_cookie_add() require 'data' being set proper with an - assert. + ... instead of putting them in the regular pid directories because + systems generally have strict length requirements for the path name to + be shorter than 107 bytes and we easily hit that boundary otherwise. - The function has not worked with a NULL data for quite some time so this - just corrects the code and comment. + The new concept generates two random names: one for the socks daemon and + one for http. - This is a different take than the proposed fixed in #10927 + Reported-by: Andy Fiddaman + Fixes #11152 + Closes #11166 - Reported-by: Kvarec Lezki - Ref: #10929 - Closes #10930 +Stefan Eissing (2 Jun 2023) -Kvarec Lezki (11 Apr 2023) +- http2: better support for --limit-rate -- vtls: remove int typecast for sizeof() + - leave transfer loop when --limit-rate is in effect and has + been received + - adjust stream window size to --limit-rate plus some slack + to make the server observe the pacing we want + - add test case to confirm behaviour - V220 Suspicious sequence of types castings: memsize -> 32-bit integer -> - memsize. The value being cast: 'sizeof - (buf->data)'. curl\lib\vtls\vtls.c 2025 + Closes #11115 - https://pvs-studio.com/en/docs/warnings/v220/ +- curl_log: evaluate log statement only when transfer is verbose - Closes #10928 + Closes #11238 -Stefan Eissing (11 Apr 2023) +Daniel Stenberg (2 Jun 2023) -- http2: fix copynpaste error reported by coverity +- libssh2: provide error message when setting host key type fails - - move all code handling HTTP/2 frames for a particular - stream into a separate function to keep from confusing - the call `data` with the stream `data`. + Ref: https://curl.se/mail/archive-2023-06/0001.html - Closes #10924 + Closes #11240 -Dan Fandrich (11 Apr 2023) +Igor Todorovski (2 Jun 2023) -- tests: log a too-long Unix socket path in sws and socksd +- system.h: remove __IBMC__/__IBMCPP__ guards and apply to all z/OS compiles - Ref: #10919 + Closes #11241 -Daniel Stenberg (11 Apr 2023) +Daniel Stenberg (2 Jun 2023) -- gen.pl: error on duplicated See-Also fields +- docs/SECURITY-PROCESS.md: link to example of previous critical flaw - Updated http2.d accordingly. +Mark Seuffert (2 Jun 2023) - Closes #10925 +- README.md: updated link to opencollective -- http2: avoid possible null pointer dereference + Closes #11232 - Reported-by: Dan Fandrich - Fixes #10920 - Closes #10923 +Daniel Stenberg (1 Jun 2023) -- lib1560: verify that more bad host names are rejected +- libssh2: use custom memory functions - when setting the hostname component of a URL + Because of how libssh2_userauth_keyboard_interactive_ex() works: the + libcurl callback allocates memory that is later free()d by libssh2, we + must set the custom memory functions. - Closes #10922 + Reverts 8b5f100db388ee60118c08aa28 -- curl_url_set.3: mention that users can set content rather freely + Ref: https://github.com/libssh2/libssh2/issues/1078 + Closes #11235 - ... which then might render bad URLs if you extract a URL later. +- test447: test PUTting a file that grows - Closes #10921 + ... and have curl trim the end when it reaches the expected total amount + of bytes instead of over-sending. -Dan Fandrich (10 Apr 2023) + Reported-by: JustAnotherArchivist on github + Closes #11223 -- CI: retry failed downloads of aws-lc +- curl: count uploaded data to stop at the originally given size - Don't fail the build in case of a temporary server problem. + Closes #11223 + Fixes #11222 + Reported-by: JustAnotherArchivist on github -- test1169: fix so it works properly everywhere +- tool: remove exclamation marks from error/warning messages - - Use an absolute path for the -L option since the module isn't in the - perl path - - Create the needed test file in a section; isn't - intended for this - - Fix the test number in the file name, which was wrong +- tool: use errorf() for error output - Follow-up to f754990a + Convert a number of fprintf() calls. - Ref: #10818 - Fixes #10889 - Closes #10917 +- tool: remove newlines from all helpf/notef/warnf/errorf calls -- tests: stop using strndup(), which isn't portable + Make voutf() always add one. - It's not available on Solaris 10, for example. Since this is just test - code that doesn't need to use an optimized system version, replace it - with the implementation copied from tool_cb_hdr.c. + Closes #11226 -- runtests: fix an incorrect comment about the ld_preload feature +- tests/servers.pm: pick unused port number with a server socket - Follow-up to 1f631864 + This change replaces the previous method of picking a port number at + random to try to start servers on, then retrying up to ten times with + new random numbers each time, with a function that creates a server + socket on port zero, thereby getting a suitable random port set by the + kernel. That server socket is then closed and that port number is used + to setup the actual test server on. - Ref: #10818 + There is a risk that *another* server can be started on the machine in + the time gap, but the server verification feature will detect that. -Daniel Stenberg (9 Apr 2023) + Closes #11220 -- urlapi: prevent setting invalid schemes with *url_set() +- RELEASE-NOTES: synced - A typical mistake would be to try to set "https://" - including the - separator - this is now rejected as that would then lead to - url_get(... URL...) would get an invalid URL extracted. + bump to 8.2.0 - Extended test 1560 to verify. +Alejandro R. Sedeño (31 May 2023) - Closes #10911 +- configure: fix run-compiler for old /bin/sh -Biswapriyo Nath (9 Apr 2023) + If you try to assign and export on the same line on some older /bin/sh + implementations, it complains: -- http2: remove unused Curl_http2_strerror function declaration + ``` + $ export "NAME=value" + NAME=value: is not an identifier + ``` - Curl_http2_strerror was renamed to http2_strerror in - 05b100aee247bb9bec8e9a1b0 and then http2_strerror was removed in - 5808a0d0f5ea0399d4a2a2 + This commit rewrites run-compiler's assignments and exports to work with + old /bin/sh, splitting assignment and export into two separate + statements, and only quote the value. So now we have: - This also fixes the following compiler error + ``` + NAME="value" + export NAME + ``` - lib/http2.h:41:33: error: unknown type name 'uint32_t' - lib/http2.h:1:1: note: 'uint32_t' is defined in header '' + While we're here, make the same change to the two supporting + assign+export lines preceeding the script to be consistent with how + exports work throughout the rest of configure.ac. - Closes #10912 + Closes #11228 -Daniel Stenberg (8 Apr 2023) +Philip Heiduck (31 May 2023) -- RELEASE-NOTES: synced +- circleci: install impacket & wolfssl 5.6.0 -SuperIlu on github (8 Apr 2023) + Closes #11221 -- config-dos.h: fix SIZEOF_CURL_OFF_T for MS-DOS/DJGPP +Daniel Stenberg (31 May 2023) - Fixes #10905 - Closes #10910 +- tool_urlglob: use curl_off_t instead of longs -Daniel Stenberg (8 Apr 2023) + To handle more globs better (especially on Windows) -- lib: remove CURLX_NO_MEMORY_CALLBACKS + Closes #11224 - The only user of this define was 'chkdecimalpoint' - a special purpose - test tool that was built but not used anymore (since 17c18fbc3 - Apr - 2020). +Dan Fandrich (30 May 2023) - Closes #10908 +- scripts: Fix GHA matrix job detection in cijobs.pl -- CURLPROXY_HTTPS2: for HTTPS proxy that may speak HTTP/2 + The parsing is pretty brittle and it broke detecting some jobs at some + point. Also, detect if Windows is used in GHA. - Setting this proxy type allows curl to negotiate and use HTTP/2 with - HTTPS proxies. +- runtests: abort test run after failure without -a - Closes #10900 + This was broken in a recent refactor and test runs would not stop. -Ali Khodkar (8 Apr 2023) + Follow-up to d4a1b5b6 -- write-out.d: add missing periods + Reported-by: Daniel Stenberg + Fixes #11225 + Closes #11227 - Closes #10897 +Version 8.1.2 (30 May 2023) -Daniel Stenberg (7 Apr 2023) +Daniel Stenberg (30 May 2023) -- http2: remove check for !data after it was already dereferenced +- RELEASE-NOTES: synced - Pointed out by Coverity + 8.1.2 release - Closes #10906 +- THANKS: contributors from 8.1.2 diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index b8e754e4ac..d7e05f07ef 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -71,6 +71,7 @@ Alessandro Vesely Alex aka WindEagle Alex Baines Alex Bligh +Alex Bozarth Alex Chan Alex Crichton Alex Fishman @@ -78,6 +79,7 @@ Alex Gaynor Alex Grebenschikov Alex Gruz Alex Kiernan +Alex Klyubin Alex Konev Alex Malinovich Alex Mayorga @@ -111,6 +113,7 @@ Alexandre Ferrieux Alexandre Pion Alexey Borzov Alexey Eremikhin +Alexey Larikov Alexey Melnichuk Alexey Pesternikov Alexey Savchuk @@ -130,6 +133,7 @@ Alona Rossen Amaury Denoyelle amishmm on github Amit Katyal +Ammar Faizi Amol Pattekar Amr Shahin Anatol Belski @@ -227,6 +231,7 @@ Antoni Villalonga Antonio Larrosa Antony74 on github Antti Hätälä +Anubhav Rai apparentorder on github April King arainchik on github @@ -349,6 +354,7 @@ Bob Schader bobmitchell1956 on github Bodo Bergmann Bogdan Nicula +boilingoden Boris Kuschel Boris Okunskiy Boris Rasin @@ -422,6 +428,7 @@ Carlo Cannas Carlo Marcelo Arenas Belón Carlo Teubner Carlo Wood +Carlos Henrique Lima Melara Carlos ORyan Carsten Lange Casey Bodley @@ -437,6 +444,7 @@ Chandrakant Bagul Charles Cazabon Charles Kerr Charles Romestant +Charlie C Chen Prog Cherish98 on github Chester Liu @@ -649,6 +657,7 @@ David Sanderson David Schweikert David Shaw David Strauss +David Suter David Tarendash David Thiel David Walser @@ -780,6 +789,7 @@ Edward Sheldrake Edward Thomson Eelco Dolstra Eetu Ojanen +eeverettrbx on github Egon Eckert Egor Pugin Ehren Bendler @@ -806,8 +816,10 @@ Emiliano Ida Emilio Cobos Álvarez Emilio López Emmanuel Tychon +Enno Boland Enrico Scholz Enrik Berkhan +enWILLYado on github eppesuig Eramoto Masaya Eric Cooper @@ -868,6 +880,7 @@ Fabrice Fontaine Fabrizio Ammollo Fahim Chandurwala Faizur Rahman +Faraz Fallahi Farzin on github Fata Nugraha Fawad Mirza @@ -1048,6 +1061,7 @@ Hao Wu Hardeep Singh Haris Okanovic Harold Stuart +Harry Mallon Harry Sarson Harry Sintonen Harshal Pradhan @@ -1103,6 +1117,8 @@ Ian Lynagh Ian Spence Ian Turner Ian Wilkes +iconoclasthero +icy17 on github Ignacio Vazquez-Abrams Igor Franchuk Igor Khristophorov @@ -1279,6 +1295,7 @@ Jesse Tan jethrogb on github jhoyla on github Jie He +Jiehong on github Jilayne Lovejoy Jim Beveridge Jim Drash @@ -1467,10 +1484,12 @@ Kane York Kang Lin Kang-Jin Lee Kantanat Wannapaka +Kareem Kari Pahula Karl Chen Karl Moerder Karol Pietrzak +Kartatz on Github Karthikdasari0423 Karthikdasari0423 on github Kartik Mahajan @@ -1512,6 +1531,7 @@ Kim Minjoong Kim Rinnewitz Kim Vandry Kimmo Kinnunen +kirbyn17 on hackerone Kirill Efimov Kirill Marchuk Kjell Ericson @@ -1570,6 +1590,7 @@ Lars J. Aas Lars Johannesen Lars Nilsson Lars Torben Wilson +Lau Laurent Bonnans Laurent Dufresne Laurent Rabret @@ -1615,15 +1636,18 @@ Litter White Liviu Chircu Liza Alenchery lizhuang0630 on github +lkordos on github lllaffer on github Lloyd Fournier Lluís Batlle i Rossell locpyl-tidnyd on github Loganaden Velvindron Loic Dachary +LoRd_MuldeR Loren Kirkby Lorenzo Miniero Loïc Yhuel +lRoccoon on github Luan Cestari Luca Altea Luca Boccassi @@ -1667,6 +1691,7 @@ Maksim Arhipov Maksim Kuzevanov Maksim Sciepanienka Maksim Stsepanenka +Maksymilian Arciemowicz Malik Idrees Hasan Khan Mamoru Tasaka Mamta Upadhyay @@ -1693,6 +1718,7 @@ Marcelo Juchem Marcin Adamski Marcin Gryszkalis Marcin Konicki +Marcin Rataj Marco Deckel Marco G. Salvagno Marco Kamner @@ -1756,6 +1782,7 @@ Martin Jansen Martin Kammerhofer Martin Kepplinger Martin Lemke +Martin Schmatz Martin Skinner Martin Staael Martin Storsjö @@ -1995,6 +2022,7 @@ Nick Zitzmann nick-telia on github Nicklas Avén Nico Baggus +Nico Rieck nico-abram on github Nicolas Berloquin Nicolas Croiset @@ -2018,6 +2046,7 @@ nimaje on github niner on github Ning Dong Nir Soffer +Niracler Li Niranjan Hasabnis Nis Jorgensen nk @@ -2037,6 +2066,7 @@ Nuru on github Octavio Schroeder odek86 on github Ofer +ohyeaah on github Okhin Vasilij Ola Mork Olaf Flebbe @@ -2063,6 +2093,7 @@ omau on github Ondřej Koláček opensignature on github opensslonzos-github on github +Ophir Lojkine Orange Tsai Oren Souroujon Oren Tirosh @@ -2346,6 +2377,7 @@ Ricky-Tigg on github Rider Linden RiderALT on github Rikard Falkeborn +rilysh rl1987 on github Rob Boeckermann Rob Cotrone @@ -2373,6 +2405,7 @@ Robert Prag Robert Ronto Robert Schumann Robert Simpson +Robert Southee Robert Weaver Robert Wruck Robin A. Meade @@ -2448,6 +2481,7 @@ Salvador Dávila Salvatore Sorrentino Sam Deane Sam Hurst +Sam James Sam Roth Sam Schanken Samanta Navarro @@ -2482,6 +2516,7 @@ Scott Barrett Scott Cantor Scott Davis Scott McCreary +sd0 on hackerone Sean Boudreau Sean Burford Sean MacLennan @@ -2559,10 +2594,12 @@ Simon Warta simplerobot on github Siva Sivaraman SLDiggie on github +Smackd0wn Smackd0wn on github smuellerDD on github sn on hackerone sofaboss on github +Sohom Datta Somnath Kundu Song Ma Sonia Subramanian @@ -2629,6 +2666,7 @@ Steve Marx Steve Oliphant Steve Roskowski Steve Walch +Steven Allen Steven Bazyl Steven G. Johnson Steven Gu @@ -2714,6 +2752,7 @@ Tim Chen Tim Costello Tim Harder Tim Heckman +Tim Hill Tim Mcdonough Tim Newsome Tim Rühsen @@ -2793,6 +2832,7 @@ tonystz on Github Toon Verwaest Tor Arntsen Torben Dannhauer +Torben Dury Torsten Foertsch Toshio Kuratomi Toshiyuki Maezawa @@ -2809,6 +2849,7 @@ Tseng Jun Tuomas Siipola Tuomo Rinne Tupone Alfredo +Turiiya Tyler Hall Török Edwin u20221022 on github @@ -2976,6 +3017,7 @@ Zhang Xiuhua zhanghu on xiaomi Zhao Yisha Zhaoyang Wu +zhengqwe on github Zhibiao Wu zhihaoy on github Zhouyihai Ding -- cgit v1.2.3