From 1e6eb7b2eec5520b510b4437a6f13578f0acddc6 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Wed, 31 Jan 2024 18:58:27 +0300 Subject: libcurl: update to 8.6.0 --- libs/libcurl/docs/CHANGES | 13478 +++++++++++++++++++++++--------------------- libs/libcurl/docs/COPYING | 2 +- libs/libcurl/docs/THANKS | 39 + 3 files changed, 6940 insertions(+), 6579 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 85fa4522fb..8e70215477 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,10369 +6,10691 @@ Changelog -Version 8.5.0 (6 Dec 2023) +Version 8.6.0 (31 Jan 2024) -Daniel Stenberg (6 Dec 2023) +Daniel Stenberg (31 Jan 2024) - RELEASE-NOTES: synced - The curl 8.5.0 release. + curl 8.6.0 -Dan Fandrich (5 Dec 2023) +- THANKS: new contributors from 8.5.0 -- github/labeler: switch from the beta to labeler v5 +Jay Satiro (31 Jan 2024) - Some keys were renamed and the dot option was made default. +- cd2nroff: use perl 'strict' and 'warnings' - Closes #12458 + - Use strict and warnings pragmas. -Daniel Stenberg (5 Dec 2023) + - If open() fails then show the reason. -- DEPRECATE: remove NTLM_WB in June 2024 + - Set STDIN io layer :crlf so that input is properly read on Windows. - Ref: https://curl.se/mail/lib-2023-12/0010.html + - When STDIN is used as input, the filename $f is now set to "STDIN". - Closes #12451 + Various error messages in single() use $f for the filename and this way + it is not undefined when STDIN. -Jacob Hoffman-Andrews (4 Dec 2023) + Closes https://github.com/curl/curl/pull/12819 -- rustls: implement connect_blocking +Daniel Stenberg (30 Jan 2024) - Closes #11647 +- cd2nroff: fix duplicate output issue -Daniel Stenberg (4 Dec 2023) + Assisted-by: Jay Satiro + Fixes https://github.com/curl/curl-www/issues/321 + Closes #12818 -- examples/rtsp-options.c: add +- lib: error out on multissl + http3 - Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and - CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. + Since the QUIC/h3 code has no knowledge or handling of multissl it might + bring unintended consequences if we allow it. - Closes #12452 + configure, cmake and curl_setup.h all now reject this combination. -Stefan Eissing (4 Dec 2023) + Assisted-by: Viktor Szakats + Assisted-by: Gisle Vanem + Ref: #12806 + Closes #12807 -- ngtcp2: ignore errors on unknown streams +Patrick Monnerat (29 Jan 2024) - - expecially in is_alive checks on connections, we might - see incoming packets on streams already forgotten and closed, - leading to errors reported by nghttp3. Ignore those. +- OS400: sync ILE/RPG binding - Closes #12449 + Also do not force git CRLF line endings on *.cmd files for OS400. -Daniel Stenberg (4 Dec 2023) + Closes #12815 -- docs: make all examples in all libcurl man pages compile +Viktor Szakats (28 Jan 2024) - Closes #12448 +- build: delete/replace 3 more clang warning pragmas -- checksrc.pl: support #line instructions + - tool_msgs: delete redundant `-Wformat-nonliteral` suppression pragma. - makes it identify the correct source file and line + - whitespace formatting in `mprintf.h`, lib518, lib537. -- GHA/man-examples: verify libcurl man page examples + - lib518: fix wrong variable in `sizeof()`. -- verify-examples.pl: verify that all man page examples compile clean + - lib518: bump variables to `rlim_t`. + Follow-up to e2b394106d543c4615a60795b7fdce04bd4e5090 #1469 -- RELEASE-NOTES: synced + - lib518: sync error message with lib537 + Follow-up to 365322b8bcf9efb6a361473d227b70f2032212ce -Graham Campbell (2 Dec 2023) + - lib518, lib537: replace `-Wformat-nonliteral` suppression pragmas + by reworking test code. -- http3: bump ngtcp2 and nghttp3 versions + Follow-up to 5b286c250829e06a135a6ba998e80beb7f43a734 #12812 + Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 + Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - nghttp3 v1.1.0 - ngtcp2 v1.1.0 + Reviewed-by: Daniel Stenberg + Closes #12814 - In docs and CI +Richard Levitte (27 Jan 2024) - Closes #12446 +- cmake: freshen up docs/INSTALL.cmake -- CI/quiche: use `3.1.4+quic` consistently in CI workflows + - Turn docs/INSTALL.cmake into a proper markdown file, + docs/INSTALL-CMAKE.md + - Move things around to divide the description into configuration, + building and installing sections + - Mention the more modern cmake options to configure, build and install, + but also retain the older variants as fallbacks - Closes #12447 + Closes #12772 -Viktor Szakats (2 Dec 2023) +Viktor Szakats (27 Jan 2024) -- test1545: disable deprecation warnings +- build: delete/replace clang warning pragmas - Fixes: - https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr - p6pk#L1205 + - delete redundant warning suppressions for `-Wformat-nonliteral`. + This now relies on `CURL_PRINTF()` and it's theoratically possible + that this macro isn't active but the warning is. We're ignoring this + as a corner-case here. - Same with details: - https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil - b6wt#L1263 - ``` - tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 - .0. Use curl_mime_init() [-Werror=deprecated-declarations] - 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", - | ^~~~~~~~~~~~ - [...] - ``` + - replace two pragmas with code changes to avoid the warnings. - Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 + Follow-up to aee4ebe59161d0a5281743f96e7738ad97fe1cd4 #12803 + Follow-up to 09230127589eccc7e01c1a7217787ef8e64f3328 #12540 + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - Fixes #12445 - Closes #12444 + Reviewed-by: Daniel Stenberg + Closes #12812 -Daniel Stenberg (2 Dec 2023) +Daniel Stenberg (27 Jan 2024) -- INSTALL: update list of ports and CPU archs +- RELEASE-NOTES: synced -- symbols-in-versions: the CLOSEPOLICY options are deprecated +- http: only act on 101 responses when they are HTTP/1.1 - The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. + For 101 responses claiming to be any other protocol, bail out. This + would previously trigger an assert. -z2_ (1 Dec 2023) + Add test 1704 to verify. -- build: fix builds that disable protocols but not digest auth + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=66184 + Closes #12811 - - Build base64 functions if digest auth is not disabled. +Scarlett McAllister (27 Jan 2024) - Prior to this change if some protocols were disabled but not digest auth - then a build error would occur due to missing base64 functions. +- _VARIABLES.md: add missing 'be' into the sentence - Fixes https://github.com/curl/curl/issues/12440 - Closes https://github.com/curl/curl/pull/12442 + Closes #12809 -Michał Antoniak (1 Dec 2023) +Stefan Eissing (27 Jan 2024) -- connect: reduce number of transportation providers +- mqtt, remove remaining use of data->state.buffer - Use only the ones necessary - the ones that are built-in. Saves a few - bytes in the resulting code. + Closes #12799 - Closes #12438 +Daniel Stenberg (27 Jan 2024) -David Benjamin (1 Dec 2023) +- x509asn1: switch from malloc to dynbuf -- vtls: consistently use typedef names for OpenSSL structs + Closes #12808 - The foo_st names don't appear in OpenSSL public API documentation. The - FOO typedefs are more common. This header was already referencing - SSL_CTX via . There is a comment about avoiding - , but OpenSSL actually declares all the typedefs in - , which is already included by (and - every other OpenSSL header), so just use that. Though I've included it - just to be explicit. +- x509asn1: make utf8asn1str() use dynbuf instead of malloc + memcpy - (I'm also fairly sure including already triggers the - Schannel conflicts anyway. The comment was probably just out of date.) + Closes #12808 - Closes #12439 +- x509asn1: reduce malloc in Curl_extract_certinfo -Lau (1 Dec 2023) + Using dynbuf -- libcurl-security.3: fix typo + Closes #12808 - Fixed minimal typo. +Jay Satiro (27 Jan 2024) - Closes #12437 +- THANKS: add Alexander Bartel and Brennan Kinney -Stefan Eissing (1 Dec 2023) + They reported and investigated #10259 which was fixed by 7b2d98df. -- ngtcp2: fix races in stream handling + Ref: https://github.com/curl/curl/issues/10259 - - fix cases where ngtcp2 invokes callbacks on streams that - nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND - in these cases as it is normal behaviour. +Daniel Stenberg (26 Jan 2024) - Closes #12435 +- krb5: add prototype to silence clang warnings on mvsnprintf() -Emanuele Torre (1 Dec 2023) + "error: format string is not a string literal" -- tool_writeout_json: fix JSON encoding of non-ascii bytes + Follow-up to 09230127589eccc7 which made the warning appear - char variables if unspecified can be either signed or unsigned depending - on the platform according to the C standard; in most platforms, they are - signed. + Assisted-by: Viktor Szakats + Closes #12803 - This meant that the *i<32 waas always true for bytes with the top bit - set. So they were always getting encoded as \uXXXX, and then since they - were also signed negative, they were getting extended with 1s causing - '\xe2' to be expanded to \uffffffe2, for example: +- x509asn1: remove code for WANT_VERIFYHOST - $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul - l - \uffffffe2\uffffff80\uffffff9c + No code ever sets this anymore since we dropped gskit - I fixed this bug by making the code use explicitly unsigned char* - variables instead of char* variables. + Follow-up to 78d6232f1f326b9ab4d - Test 268 verifies + Closes #12804 - Reported-by: iconoclasthero - Closes #12434 +- socks: reduce the buffer size to 600 (from 8K) -Stefan Eissing (1 Dec 2023) + This is malloc'ed memory and it does not more. Test 742 helps us verify + this. -- cf-socket: TCP trace output local address used in connect + Closes #12789 - Closes #12427 +Stefan Eissing (26 Jan 2024) -Jay Satiro (1 Dec 2023) +- file+ftp: use stack buffers instead of data->state.buffer -- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation + Closes #12789 - - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it - includes protocol-specific instructions that trigger a transfer. +- vtls: receive max buffer - Prior to this change it explicitly said that it did not include those - instructions in the time, but that is incorrect. + - do not only receive one TLS record, but try to fill + the passed buffer + - consider <4K remaning space is "filled". - The change is a copy of the fixed explanation already in - CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. + Closes #12801 - Reported-by: eeverettrbx@users.noreply.github.com +Daniel Stenberg (26 Jan 2024) - Fixes https://github.com/curl/curl/issues/12431 - Closes https://github.com/curl/curl/pull/12432 +- docs: do not start lines/sentences with So, But nor And -Daniel Stenberg (30 Nov 2023) + Closes #12802 -- multi: during ratelimit multi_getsock should return no sockets +- docs: remove spurious ampersands from markdown - ... as there is nothing to wait for then, it just waits. Otherwise, this - causes much more CPU work and updates than necessary during ratelimit - periods. + They were leftovers from the nroff conversion. - Ref: https://curl.se/mail/lib-2023-11/0056.html - Closes #12430 + Follow-up to eefcc1bda4bccd800f5a5 -Dmitry Karpov (30 Nov 2023) + Closes #12800 -- transfer: abort pause send when connection is marked for closing +Patrick Monnerat (26 Jan 2024) - This handles cases of some bi-directional "upgrade" scenarios - (i.e. WebSockets) where sending is paused until some "upgrade" handshake - is completed, but server rejects the handshake and closes the - connection. +- sasl: make login option string override http auth - Closes #12428 + - Use http authentication mechanisms as a default, not a preset. -Daniel Stenberg (28 Nov 2023) + Consider http authentication options which are mapped to SASL options as + a default (overriding the hardcoded default mask for the protocol) that + is ignored if a login option string is given. -- RELEASE-NOTES: synced + Prior to this change, if some HTTP auth options were given, sasl mapped + http authentication options to sasl ones but merged them with the login + options. -- openssl: when a session-ID is reused, skip OCSP stapling + That caused problems with the cli tool that sets the http login option + CURLAUTH_BEARER as a side-effect of --oauth2-bearer, because this flag + maps to more than one sasl mechanisms and the latter cannot be cleared + individually by the login options string. - Fixes #12399 - Reported-by: Alexey Larikov - Closes #12418 + New test 992 checks this. -- test1545: test doing curl_formadd twice with missing file + Fixes https://github.com/curl/curl/issues/10259 + Closes https://github.com/curl/curl/pull/12790 - Reproduces #12410 - Verifies the fix - Closes #12421 +Stefan Eissing (26 Jan 2024) -- Curl_http_body: cleanup properly when Curl_getformdata errors +- socks: use own buffer instead of data->state.buffer - Reported-by: yushicheng7788 on github - Based-on-work-by: yushicheng7788 on github - Fixes #12410 - Closes #12421 + Closes #12788 -- test1477: verify that libcurl-errors.3 and public headers are synced +Daniel Stenberg (26 Jan 2024) - The script errorcodes.pl extracts all error codes from all headers and - checks that they are all documented, then checks that all documented - error codes are also specified in a header file. +- socks: fix generic output string to say SOCKS instead of SOCKS4 - Closes #12424 + ... since it was also logged for SOCKS5. -- libcurl-errors.3: sync with current public headers + Closes #12797 - Closes #12424 +- test742: test SOCKS5 with max length user, password and hostname -Stefan Eissing (28 Nov 2023) + Adjusted the socksd server accordingly to allow for configuring that + long user name and password. -- test459: fix for parallel runs + Closes #12797 - - change warniing message to work better with varying filename - length. - - adapt test output check to new formatting +Stefan Eissing (25 Jan 2024) - Follow-up to 97ccc4479f77ba3191c6 - Closes #12423 +- ssh: use stack scratch buffer for seeks -Daniel Stenberg (27 Nov 2023) + - instead of data->state.buffer -- tool_cb_prg: make the carriage return fit for wide progress bars + Closes #12794 - When the progress bar was made max width (256 columns), the fly() - function attempted to generate its output buffer too long so that the - trailing carriage return would not fit and then the output would show - wrongly. The fly function is called when the expected total transfer is - unknown, which could be one or more progress calls before the actual - progress meter get shown when the expected transfer size is provided. +Daniel Stenberg (25 Jan 2024) - This new take also replaces the msnprintf() call with a much simpler - memset() for speed. +- krb5: access the response buffer correctly - Reported-by: Tim Hill - Fixes #12407 - Closes #12415 + As the pingpong code no longer uses the download buffer. -- tool_parsecfg: make warning output propose double-quoting + Folllow-up to c2d973627bab12ab + Pointed-out-by: Stefan Eissing + Closes #12796 - When the config file parser detects a word that *probably* should be - quoted, mention double-quotes as a possible remedy. +Stefan Eissing (25 Jan 2024) - Test 459 verifies. +- mqtt: use stack scratch buffer for recv+publish - Proposed-by: Jiehong on github - Fixes #12409 - Closes #12412 + - instead of data->state.buffer -Jay Satiro (26 Nov 2023) + Closes #12792 -- curl.rc: switch out the copyright symbol for plain ASCII +- telnet, use stack scratch buffer for do - .. like we already do for libcurl.rc. + - instead of data->state.buffer - libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" - warning so it was switched to ascii. + Closes #12793 - Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 +- http, use stack scratch buffer - Suggested-by: Robert Southee + - instead of data->state.buffer - Closes https://github.com/curl/curl/pull/12403 + Closes #12791 -Daniel Stenberg (26 Nov 2023) +- ntlm_wb: do not use data->state.buf any longer -- conncache: use the closure handle when disconnecting surplus connections + Closes #12787 - Use the closure handle for disconnecting connection cache entries so - that anything that happens during the disconnect is not stored and - associated with the 'data' handle which already just finished a transfer - and it is important that details from the unrelated disconnect does not - taint meta-data in the data handle. +- gitignore: the generated `libcurl-symbols.md` - Like storing the response code. + Closes #12795 - This also adjust test 1506. Unfortunately it also removes a key part of - the test that verifies that a connection is closed since when this - output vanishes (because the closure handle is used), we don't know - exactly that the connection actually gets closed in this test... +Daniel Stenberg (25 Jan 2024) - Reported-by: ohyeaah on github - Fixes #12367 - Closes #12405 +- tool: fix the listhelp generation command -- RELEASE-NOTES: synced + The previous command line to generate the tool_listhelp.c source file + broke with 2494b8dd5175cee7. -Stefan Eissing (24 Nov 2023) + Make 'make listhelp' invoked in src/ generate it. Also update the + comment in the file to mention the right procedure. -- quic: make eyeballers connect retries stop at weird replies + Closes #12786 - - when a connect immediately goes into DRAINING state, do - not attempt retries in the QUIC connection filter. Instead, - return CURLE_WEIRD_SERVER_REPLY - - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an - inconclusive answer. When all addresses have been attempted, - rewind the address list once on an inconclusive answer. - - refs #11832 where connects were retried indefinitely until - the overall timeout fired +- http: check for "Host:" case insensitively - Closes #12400 + When checking if the user wants to replace the header, the check should + be case insensitive. -Daniel Stenberg (24 Nov 2023) + Adding test 461 to verify -- CI: verify libcurl function SYNPOSIS sections + Found-by: Dan Fandrich + Ref: #12782 + Closes #12784 - With the .github/scripits/verify-synopsis.pl script +Tatsuhiro Tsujikawa (25 Jan 2024) - Closes #12402 +- configure: add libngtcp2_crypto_boringssl detection -- docs/libcurl: SYNSOPSIS cleanup + If OpenSSL is found to be BoringSSL or AWS-LC, and ngtcp2 is requested, + try to detect libngtcp2_crypto_boringssl. - - use the correct include file - - make sure they are declared as in the header file - - fix minor nroff syntax mistakes (missing .fi) + Reported-by: ウさん + Fixes #12724 + Closes #12769 - These are verified by verify-synopsis.pl, which extracts the SYNPOSIS - code and runs it through gcc. +Daniel Stenberg (25 Jan 2024) - Closes #12402 +- http: remove comment reference to a removed solution -- sendf: fix comment typo + Follow-up to 58974d25d -- fopen: allocate the dir after fopen + Closes #12785 - Move the allocation of the directory name down to after the fopen() call - to allow that shortcut code path to avoid a superfluous malloc+free - cycle. +Stefan Eissing (25 Jan 2024) - Follow-up to 73b65e94f35311 +- pytest: Scorecard tracking CPU and RSS - Closes #12398 + Closes #12765 -Stefan Eissing (24 Nov 2023) +Graham Campbell (25 Jan 2024) -- transfer: cleanup done+excess handling +- GHA: bump ngtcp2, gnutls, mod_h2, quiche - - add `SingleRequest->download_done` as indicator that - all download bytes have been received - - remove `stop_reading` bool from readwrite functions - - move excess body handling into client download writer + - ngtcp2 to v1.2.0 + - gnutls to 3.8.3 + - mod_h2 to 2.0.26 + - quiche to 0.20.0 - Closes #12371 + Closes #12778 + Closes #12779 + Closes #12780 + Closes #12781 -Daniel Stenberg (23 Nov 2023) +Daniel Stenberg (25 Jan 2024) -- fopen: create new file using old file's mode +- ftpserver.pl: send 213 SIZE response without spurious newline - Because the function renames the temp file to the target name as a last - step, if the file was previously owned by a different user, not ORing - the old mode could otherwise end up creating a file that was no longer - readable by the original owner after save. +- pingpong: stop using the download buffer - Reported-by: Loïc Yhuel - Fixes #12299 - Closes #12395 + The pingpong logic now uses its own dynbuf for receiving command + response data. -- test1476: require proxy + When the "final" response header for a commanad has been received, that + final line is left first in the recvbuf for the protocols to parse at + will. If there is additional data behind the final response line, the + 'overflow' counter is indicate how many bytes. - Follow-up from 323df4261c3542 + Closes #12757 - Closes #12394 +- gen.pl: remove bold from .IP used for ## -- fopen: create short(er) temporary file name + Reported-by: Viktor Szakats + Fixes #12776 + Closes #12777 - Only using random letters in the name plus a ".tmp" extension. Not by - appending characters to the final file name. +Viktor Szakats (24 Jan 2024) - Reported-by: Maksymilian Arciemowicz +- cmake: rework options to enable curl and libcurl docs - Closes #12388 + Rework CMake options for building/using curl tool and libcurl manuals. -Stefan Eissing (23 Nov 2023) + - rename `ENABLE_MANUAL` to `ENABLE_CURL_MANUAL`, meaning: + to build man page and built-in manual for curl tool. -- tests: git ignore generated second-hsts.txt file + - rename `BUILD_DOCS` to `BUILD_LIBCURL_DOCS`, meaning: + to build man pages for libcurl. - File is generated in test lib1900 + - `BUILD_LIBCURL_DOCS` now works without having to enable + `ENABLE_CURL_MANUAL` too. - Follow-up to 7cb03229d9e9c5 + - drop support for existing CMake-level `USE_MANUAL` option to avoid + confusion. (It used to work with the effect of current + `ENABLE_CURL_MANUAL`, but only by accident.) - Closes #12393 + Assisted-by: Richard Levitte + Ref: #12771 + Closes #12773 -Viktor Szakats (23 Nov 2023) +Daniel Stenberg (24 Jan 2024) -- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 +- urlapi: remove assert - Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to - OpenSSL 1.1.x, and LibreSSL 3.6 or upper. + This assert triggers wrongly when CURLU_GUESS_SCHEME and + CURLU_NO_AUTHORITY are both set and the URL is a single path. - With the caveat, that "group name" and "type name" are missing from - the log output with these TLS backends. + I think this assert has played out its role. It was introduced in a + rather big refactor. - Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + Follow-up to 4cfa5bcc9a - Reviewed-by: Daniel Stenberg - Closes #12385 + Reported-by: promptfuzz_ on hackerone + Closes #12775 -Daniel Stenberg (23 Nov 2023) +Patrick Monnerat (24 Jan 2024) -- urldata: fix typo in comment +- tests: avoid int/size_t conversion size/sign warnings -- CI: codespell + Closes #12768 - The list of words to ignore is in the file - .github/scripts/codespell-ignore.txt +Daniel Stenberg (24 Jan 2024) - Closes #12390 +- GHA: add a job scanning for "bad words" in markdown -- lib: fix comment typos + This means words, phrases or things we have decided not to use - words that + are spelled right according to the dictionary but we want to avoid. In the + name of consistency and better documentation. - Five separate ones, found by codespell + Closes #12764 - Closes #12390 +Viktor Szakats (23 Jan 2024) -- test1476: verify cookie PSL mixed case +- cmake: speed up curldown processing, enable by default -- cookie: lowercase the domain names before PSL checks + - cmake: enable `BUILD_DOCS` by default (this controls converting and + installing `.3` files from `.md` sources) - Reported-by: Harry Sintonen + - cmake: speed up generating `.3` files by using a single command per + directory, instead of a single command per file. This reduces external + commands by about a thousand. (There remains some CMake logic kicking + in resulting in 500 -one per file- external `-E touch_nocreate` calls.) - Closes #12387 + - cd2nroff: add ability to process multiple input files. -Viktor Szakats (23 Nov 2023) + - cd2nroff: add `-k` option to use the source filename to form the + output filename. (instead of the default in-file `Title:` line.) -- openssl: fix building with v3 `no-deprecated` + add CI test + Follow-up to 3f08d80b2244524646ce86915c585509ac54fb4c + Follow-up to ea0b575dab86a3c44dd1d547dc500276266aa382 #12753 + Follow-up to eefcc1bda4bccd800f5a56a0fe17a2f44a96e88b #12730 - - build quictls with `no-deprecated` in CI to have test coverage for - this OpenSSL 3 configuration. + Closes #12762 - - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. - The caller code is meant for OpenSSL 3, while these two functions were - only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 - if built with option `no-deprecated`, causing build errors: - ``` - vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ - algorithms'; ISO C99 and later do not support implicit function declaration - s [-Wimplicit-function-declaration] - vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ - digests'; ISO C99 and later do not support implicit function declarations [ - -Wimplicit-function-declaration] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f - ullLog=true#L7667 +Richard Levitte (23 Jan 2024) - Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 - Reviewed-by: Alex Bozarth +- docs: install curl.1 with cmake as well - - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with - `no-deprecated` quictls 3 builds. - Do it by moving an existing solution for this from `vtls/openssl.c` - to `vtls/openssl.h` and adjusting caller code. - ``` - vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g - et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit - -function-declaration] - ``` - Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s - tep:24:1178 + Closes #12759 - - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and - `-Wunused-function` when trying to build curl with NTLM enabled but - without the necessary TLS backend (with DES) support. +Daniel Stenberg (23 Jan 2024) - Closes #12384 +- osslq: remove the TLS library from the version output -- curl.h: delete Symbian OS references + Since we only support using a single TLS library at any one time, we + know that the TLS library for QUIC is the same that is also shown for + regular TLS. - curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 - via #5989. Delete references to it from public headers, because there - is no fresh release to use those headers with. + Fixes #12763 + Reported-by: Viktor Szakats + Closes #12767 - Reviewed-by: Dan Fandrich - Reviewed-by: Jay Satiro - Closes #12378 +Stefan Eissing (23 Jan 2024) -- windows: use built-in `_WIN32` macro to detect Windows +- CI: remove unnecessary OpenSSL 3 option `enable-tls1_3` - Windows compilers define `_WIN32` automatically. Windows SDK headers - or build env defines `WIN32`, or we have to take care of it. The - agreement seems to be that `_WIN32` is the preferred practice here. - Make the source code rely on that to detect we're building for Windows. + .. and switch OpenSSL 3 libdir from lib64 to lib for consistency. - Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for - Windows detection, next to the official `_WIN32`. After this patch it - only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. + Closes https://github.com/curl/curl/pull/12758 - There is a slight chance these break compatibility with Windows - compilers that fail to define `_WIN32`. I'm not aware of any obsolete - or modern compiler affected, but in case there is one, one possible - solution is to define this macro manually. +- GHA: bump nghttp2 version to v1.59.0 - grepping for `WIN32` remains useful to discover Windows-specific code. + - Switch to v1.59.0 for GHA CI jobs that use a specific nghttp2-version. - Also: + Closes https://github.com/curl/curl/pull/12766 - - extend `checksrc` to ensure we're not using `WIN32` anymore. +Daniel Stenberg (23 Jan 2024) - - apply minor formatting here and there. +- RELEASE-NOTES: synced - - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. +- docs/cmdline: change to .md for cmdline docs - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg + - switch all invidual files documenting command line options into .md, + as the documentation is now markdown-looking. - Closes #12376 + - made the parser treat 4-space indents as quotes -Stefan Eissing (22 Nov 2023) + - switch to building the curl.1 manpage using the "mainpage.idx" file, + which lists the files to include to generate it, instead of using the + previous page-footer/headers. Also, those files are now also .md + ones, using the same format. I gave them underscore prefixes to make + them sort separately: + _NAME.md, _SYNOPSIS.md, _DESCRIPTION.md, _URL.md, _GLOBBING.md, + _VARIABLES.md, _OUTPUT.md, _PROTOCOLS.md, _PROGRESS.md, _VERSION.md, + _OPTIONS.md, _FILES.md, _ENVIRONMENT.md, _PROXYPREFIX.md, + _EXITCODES.md, _BUGS.md, _AUTHORS.md, _WWW.md, _SEEALSO.md -- url: ConnectionExists revisited + - updated test cases accordingly - - have common pattern of `if not match, continue` - - revert pages long if()s to return early - - move dead connection check to later since it may - be relatively expensive - - check multiuse also when NOT building with NGHTTP2 - - for MULTIUSE bundles, verify that the inspected - connection indeed supports multiplexing when in use - (bundles may contain a mix of connection, afaict) + Closes #12751 - Closes #12373 +dependabot[bot] (23 Jan 2024) -Daniel Stenberg (22 Nov 2023) +- CI: bump actions/cache from 3 to 4 -- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range + Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4. + - [Release notes](https://github.com/actions/cache/releases) + - [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md) + - [Commits](https://github.com/actions/cache/compare/v3...v4) - ... or use the default value. + --- + updated-dependencies: + - dependency-name: actions/cache + dependency-type: direct:production + update-type: version-update:semver-major + ... - Also clarify the documentation language somewhat. + Signed-off-by: dependabot[bot] + Closes #12756 - Closes #12382 +Daniel Stenberg (23 Jan 2024) -- urldata: make maxconnects a 32 bit value +- openssl: when verifystatus fails, remove session id from cache - "2^32 idle connections ought to be enough for anybody" + To prevent that it gets used in a subsequent transfer that skips the + verifystatus check since that check can't be done when the session id is + reused. - Closes #12375 + Reported-by: Hiroki Kurosawa + Closes #12760 -- FEATURES: update the URL phrasing +Viktor Szakats (23 Jan 2024) - The URL is length limited since a while back so "no limit" simply is not - true anymore. Mention the URL RFC standard used instead. +- cmake: add option to disable building docs - Closes #12383 +Richard Levitte (23 Jan 2024) -- wolfssh: remove redundant static prototypes +- cmake: use curldown to build man pages - vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We - rror=redundant-decls] + This throws away the previous HTML and PDF producers, to mimic what + Makefile.am does as faithfully as possible. - Closes #12381 + Closes #12753 -- setopt: remove superfluous use of ternary expressions +Daniel Stenberg (23 Jan 2024) - Closes #12374 +- mksymbolsmanpage.pl: provide references to where the symbol is used -- mime: store "form escape" as a single bit +- docs: introduce "curldown" for libcurl man page format - Closes #12374 + curldown is this new file format for libcurl man pages. It is markdown + inspired with differences: -- setopt: check CURLOPT_TFTP_BLKSIZE range on set + - Each file has a set of leading headers with meta-data + - Supports a small subset of markdown + - Uses .md file extensions for editors/IDE/GitHub to treat them nicely + - Generates man pages very similar to the previous ones + - Generates man pages that still convert nicely to HTML on the website + - Detects and highlights mentions of curl symbols automatically (when + their man page section is specified) - ... instead of later when the transfer is about to happen. + tools: - Closes #12374 + - cd2nroff: converts from curldown to nroff man page + - nroff2cd: convert an (old) nroff man page to curldown + - cdall: convert many nroff pages to curldown versions + - cd2cd: verifies and updates a curldown to latest curldown -Viktor Szakats (21 Nov 2023) + This setup generates .3 versions of all the curldown versions at build time. -- build: add more picky warnings and fix them + CI: - Enable more picky compiler warnings. I've found these options in the - nghttp3 project when implementing the CMake quick picky warning - functionality for it [1]. + Since the documentation is now technically markdown in the eyes of many + things, the CI runs many more tests and checks on this documentation, + including proselint, link checkers and tests that make sure we capitalize the + first letter after a period... - `-Wunused-macros` was too noisy to keep around, but fixed a few issues - it revealed while testing. + Closes #12730 - - autotools: reflect the more precisely-versioned clang warnings. - Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 - - autotools: sync between clang and gcc the way we set `no-multichar`. - - autotools: avoid setting `-Wstrict-aliasing=3` twice. - - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. - It triggers in libtool-generated stub code. +Viktor Szakats (22 Jan 2024) - - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. +- libssh2: use `libssh2_session_callback_set2()` with v1.11.1 - - lib/curl_setup.h: delete duplicate declaration for `fileno`. - Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d - (1999-12-29). This suggests this may not be needed anymore, but if - it does, we may restore this for those specific (non-Windows) systems. - - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since - c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. - - lib: delete unused macro `isxdigit_ascii` since - f65f750742068f579f4ee6d8539ed9d5f0afcb85. - - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. - - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. - - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` - macro. - - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. - - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. - - lib/setopt: fix `-Wunreachable-code-break`. - - lib/system_win32 and lib/timeval: fix double declarations for - `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. - - lib/warnless: fix double declarations in CMake UNITY mode [5]. - This was due to force-disabling the header guard of `warnless.h` to - to reapply it to source code coming after `warnless.c` in UNITY - builds. This reapplied declarations too, causing the warnings. - Solved by adding a header guard for the lines that actually need - to be reapplied. - - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. - - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant - block. - - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. - - lib/vtls/sectransp: suppress `-Wunreachable-code`. - Detected in `else` branches of dynamic feature checks, with results - known at compile-time, e.g. - ```c - if(SecCertificateCopySubjectSummary) /* -> true */ - ``` - Likely fixable as a separate micro-project, but given SecureTransport - is deprecated anyway, let's just silence these locally. - - src/tool_help: delete duplicate declaration for `helptext`. - - src/tool_xattr: fix `-Wunreachable-code`. - - tests: delete duplicate declaration for `unitfail` [8]. - - tests: delete duplicate declaration for `strncasecompare`. - - tests/libtest: delete duplicate declaration for `gethostname`. - Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 - (2010-08-02). - Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc - If there are still systems around with warnings, we may restore the - prototype, but limited for those systems. - - tests/lib2305: delete duplicate declaration for - `libtest_debug_config`. - - tests/h2-download: fix `-Wunreachable-code-break`. + To avoid a local hack to pass function pointers and to avoid + deprecation warnings when building with libssh2 v1.11.1 or newer: + ``` + lib/vssh/libssh2.c:3324:5: warning: 'libssh2_session_callback_set' is depreca + ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- + declarations] + lib/vssh/libssh2.c:3326:5: warning: 'libssh2_session_callback_set' is depreca + ted: since libssh2 1.11.1. Use libssh2_session_callback_set2() [-Wdeprecated- + declarations] + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/7609484879/job/2072082 + 1100#step:3:4982 - [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 - f8bf9f/cmake/PickyWarningsC.cmake - [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui - qla5fj45?fullLog=true#L1675 - [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 - 2331#step:7:72 - https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 - 2331#step:7:109 - [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L204 - [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr - iklpf1ut#L218 - [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 - 2331#step:7:290 - [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 - 2331#step:9:1193 - [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 - 2331#step:33:1870 + Ref: https://github.com/libssh2/libssh2/pull/1285 + Ref: https://github.com/libssh2/libssh2/commit/c0f69548be902147ce014ffa40b8db + 3cf1d4b0b4 + Reviewed-by: Daniel Stenberg + Closes #12754 - Closes #12331 +Daniel Stenberg (22 Jan 2024) -Daniel Stenberg (21 Nov 2023) +- transfer: make the select_bits_paused condition check both directions -- transfer: avoid unreachable expression + If there is activity in a direction that is not paused, return false. - If curl_off_t and size_t have the same size (which is common on modern - 64 bit systems), a condition cannot occur which Coverity pointed - out. Avoid the warning by having the code conditionally only used if - curl_off_t actually is larger. + Reported-by: Sergey Bronnikov + Bug: https://curl.se/mail/lib-2024-01/0049.html + Closes #12740 - Follow-up to 1cd2f0072fa482e25baa2 +Stefan Eissing (22 Jan 2024) - Closes #12370 +- http3: initial support for OpenSSL 3.2 QUIC stack -Stefan Eissing (21 Nov 2023) + - HTTP/3 for curl using OpenSSL's own QUIC stack together + with nghttp3 + - configure with `--with-openssl-quic` to enable curl to + build this. This requires the nghttp3 library + - implementation with the following restrictions: + * macOS has to use an unconnected UDP socket due to an + issue in OpenSSL's datagram implementation + See https://github.com/openssl/openssl/issues/23251 + This makes connections to non-reponsive servers hang. + * GET requests will send the indicator that they have + no body in a separate QUIC packet. This may result + in processing delays or Transfer-Encodings on proxied + requests + * uploads that encounter blocks will use 100% cpu as + detection of these flow control issue is not working + (we have not figured out to pry that from OpenSSL). -- transfer: readwrite improvements + Closes #12734 - - changed header/chunk/handler->readwrite prototypes to accept `buf`, - `blen` and a `pconsumed` pointer. They now get the buffer to work on - and report back how many bytes they consumed - - eliminated `k->str` in SingleRequest - - improved excess data handling to properly calculate with any body data - left in the headerb buffer - - eliminated `k->badheader` enum to only be a bool +Viktor Szakats (22 Jan 2024) - Closes #12283 +- cmake: fix `ENABLE_MANUAL` option -Daniel Stenberg (21 Nov 2023) + Fix the `ENABLE_MANUAL` option. Set it to default to `OFF`. -- RELEASE-NOTES: synced + Before this patch `ENABLE_MANUAL=ON` was a no-op, even though it was the + option designed to enable building and using the built-in curl manual. + (`USE_MANUAL=ON` option worked for this instead, by accident). -Jiří Hruška (21 Nov 2023) + Ref: https://github.com/curl/curl/pull/12730#issuecomment-1902572409 + Closes #12749 -- transfer: avoid calling the read callback again after EOF +Mohammadreza Hendiani (19 Jan 2024) - Regression since 7f43f3dc5994d01b12 (7.84.0) +- TODO: update broken link to ratelimit-headers draft - Bug: https://curl.se/mail/lib-2023-11/0017.html + Closes #12741 - Closes #12363 +Daniel Stenberg (19 Jan 2024) -Daniel Stenberg (21 Nov 2023) +- cmake: when USE_MANUAL=YES, build the curl.1 man page -- doh: provide better return code for responses w/o addresses + Fixes KNOWN_BUG 15.4 - Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the - response did not contain any addresses. Now it more accurately returns - CURLE_COULDNT_RESOLVE_HOST. + Closes #12742 - Reported-by: lRoccoon on github +- cmdline-opts/write-out.d: remove spurious double quotes - Fixes #12365 - Closes #12366 +Stefan Eissing (19 Jan 2024) -Stefan Eissing (21 Nov 2023) +- rtsp: Convert assertion into debug log -- HTTP/2, HTTP/3: handle detach of onoing transfers + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65934 - - refs #12356 where a UAF is reported when closing a connection - with a stream whose easy handle was cleaned up already - - handle DETACH events same as DONE events in h2/h3 filters + - write excess bytes to the client where the standard excess bytes + checks will report any wrongness and fail the transfer - Fixes #12356 - Reported-by: Paweł Wegner - Closes #12364 + Fixes #12738 + Closes #12739 -Viktor Szakats (20 Nov 2023) +Daniel Stenberg (19 Jan 2024) -- autotools: stop setting `-std=gnu89` with `--enable-warnings` +- headers: remove assert from Curl_headers_push - Do not alter the C standard when building with `--enable-warnings` when - building with gcc. + The fuzzer managed to reach the function without a terminating CR or LF + so let's handle it normally. While there, remove the goto. - On one hand this alters warning results compared to a default build. - On the other, it may produce different binaries, which is unexpected. + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65839 - Also fix new warnings that appeared after removing `-std=gnu89`: + Closes #12721 - - include: fix public curl headers to use the correct printf mask for - `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 - and Visual Studio 2013 and newer. This fixes the printf mask warnings - in examples and tests. E.g. [1] +- curl_easy_getinfo.3: remove the wrong time value count - - conncache: fix printf format string [2]. + It said "six" time values but they are eight by now. Remove the mention + of the amount. - - http2: fix potential null pointer dereference [3]. - (seen on Slackware with gcc 11.) + Closes #12727 - - libssh: fix printf format string in SFTP code [4]. - Also make MSVC builds compatible with old CRT versions. +Viktor Szakats (18 Jan 2024) - - libssh2: fix printf format string in SFTP code for MSVC. - Applying the same fix as for libssh above. +- mbedtls: fix `-Wnull-dereference` and `-Wredundant-decls` - - unit1395: fix `argument is null` and related issues [5]: - - stop calling `strcmp()` with NULL to avoid undefined behaviour. - - fix checking results if some of them were NULL. - - do not pass NULL to printf `%s`. + - Silence warning in mbedTLS v3.5.1 public headers: + ``` + ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_extra.h:489:14: warning: r + edundant redeclaration of 'psa_set_key_domain_parameters' [-Wredundant-decls] + ./mbedtls/_x64-linux-musl/usr/include/psa/crypto_struct.h:354:14: note: pre + vious declaration of 'psa_set_key_domain_parameters' was here + ``` + Ref: https://github.com/libssh2/libssh2/commit/ecec68a2c13a9c63fe8c2dc457ae + 785a513e157c + Ref: https://github.com/libssh2/libssh2/pull/1226 - - ci: keep a build job with `-std=gnu89` to continue testing for - C89-compliance. We can apply this to other gcc jobs as needed. - Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 + - Fix compiler warnings seen with gcc 9.2.0 + cmake unity: + ``` + ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_read': + ./curl/lib/vtls/mbedtls.c:189:11: warning: null pointer dereference [-Wnull + -dereference] + 189 | nread = Curl_conn_cf_recv(cf->next, data, (char *)buf, blen, &res + ult); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~ + ./curl/lib/vtls/mbedtls.c: In function 'mbedtls_bio_cf_write': + ./curl/lib/vtls/mbedtls.c:168:14: warning: null pointer dereference [-Wnull + -dereference] + 168 | nwritten = Curl_conn_cf_send(cf->next, data, (char *)buf, blen, & + result); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~ + ``` - [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l - ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b - [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 - 2346#step:6:67 - [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 - 2346#step:30:214 - [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 - 2346#step:29:895 - [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 - 2346#step:33:1689 + - delete stray `#else`. - Closes #12346 + Closes #12720 -- autotools: fix/improve gcc and Apple clang version detection +Daniel Stenberg (17 Jan 2024) - - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu - may return `n-win32` (also with `-dumpfullversion`). Causing these - errors and failing to enable picky warnings: - ``` - ../configure: line 23845: test: : integer expression expected - ``` - Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 - 3718#step:5:143 +- docs: cleanup nroff format use - Fix that by stripping any dash-suffix and handling a dotless (major-only) - version number by assuming `.0` in that case. + - remove use of .BI for code snippet + - stop using .br, just do a blank line + - remove use of .PP + - remove use for .sp + - remove backslash in .IP + - use .IP instead of .TP - `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` - Ref: https://github.com/mamedev/mame/pull/9767 + Closes #12731 - - fix Apple clang version detection for releases between - 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the - version was under-detected as 3.7 llvm/clang equivalent. +Stefan Eissing (17 Jan 2024) - - fix Apple clang version detection for 'Apple clang version 11.0.0' - and newer where the Apple clang version was detected, instead of its - llvm/clang equivalent. +- test2307: fix expected failure code after ws refactoring - - display detected clang/gcc/icc compiler version. + Fixes #12722 + Closes #12728 - Via libssh2: - - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d - 89875ad - https://github.com/libssh2/libssh2/pull/1187 - - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 - 42b592e - https://github.com/libssh2/libssh2/pull/1232 +Jay Satiro (17 Jan 2024) - Closes #12362 +- cf-socket: show errno in tcpkeepalive error messages -- autotools: delete LCC compiler support bits + - If the socket keepalive options (TCP_KEEPIDLE, etc) cannot be set + then show the errno in the verbose error messages. - Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 + Ref: https://github.com/curl/curl/discussions/12715#discussioncomment-8151652 - Closes #12357 + Closes https://github.com/curl/curl/pull/12726 -- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` +- tool_getparam: stop supporting `@filename` style for --cookie - - tests: verify CMake `DISABLE` options. + The `@filename` style was never documented for --cookie + but prior to this change curl would accept it anyway and always treat a + @ prefixed string as a filename. - Make an exception for 2 CMake-only ones, and one more that's - using a different naming scheme, also in autotools and source. + That's a problem if the string also contains a = sign because then it is + documented to be interpreted as a cookie string and not a filename. - - cmake: add support for `CURL_DISABLE_HEADERS_API`. + Example: - Suggested-by: Daniel Stenberg - Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 + `--cookie @foo=bar` - Closes #12353 + Before: Interpreted as load cookies from filename foo=bar. -Jacob Hoffman-Andrews (20 Nov 2023) + After: Interpreted as cookie `@foo=bar` (name `@foo` and value `bar`). -- hyper: temporarily remove HTTP/2 support + Other curl options with a data/filename option-value use the `@filename` + to distinguish filenames which is probably how this happened. The + --cookie option has never been documented that way. - The current design of the Hyper integration requires rebuilding the - Hyper clientconn for each request. However, building the clientconn - requires resending the HTTP/2 connection preface, which is incorrect - from a protocol perspective. That in turn causes servers to send GOAWAY - frames, effectively degrading performance to "no connection reuse" in - the best case. It may also be triggering some bugs where requests get - dropped entirely and reconnects take too long. + Ref: https://curl.se/docs/manpage.html#-b - This doesn't rule out HTTP/2 support with Hyper, but it may take a - redesign of the Hyper integration in order to make things work. + Closes https://github.com/curl/curl/pull/12645 - Closes #12191 +Stefan Eissing (16 Jan 2024) -Jay Satiro (20 Nov 2023) +- websockets: refactor decode chain -- schannel: fix unused variable warning + - use client writer stack for decoding frames + - move websocket protocol handler to ws.c - Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 - Reported-by: Viktor Szakats + Closes #12713 - Closes https://github.com/curl/curl/pull/12361 +- websockets: check for negative payload lengths -Daniel Stenberg (19 Nov 2023) + - in en- and decoding, check the websocket frame payload lengths for + negative values (from curl_off_t) and error the operation in that case + - add test 2307 to verify -- url: find scheme with a "perfect hash" + Closes #12707 - Instead of a loop to scan over the potentially 30+ scheme names, this - uses a "perfect hash" table. This works fine because the set of schemes - is known and cannot change in a build. The hash algorithm and table size - is made to only make a single scheme index per table entry. +Daniel Stenberg (16 Jan 2024) - The perfect hash is generated by a separate tool (scripts/schemetable.c) +- docs: mention env vars not used by schannel - Closes #12347 + Ref: #12704 -- scripts: add schemetable.c + Co-authored-by: Jay Satiro - This tool generates a scheme-matching table. + Closes #12711 - It iterates over a number of different initial and shift values in order - to find the hash algorithm that needs the smallest possible table. +- tool_operate: make --remove-on-error only remove "real" files - The generated hash function, table and table size then needs to be used - by the url.c:Curl_getn_scheme_handler() function. + Reported-by: Harry Sintonen + Assisted-by: Dan Fandrich -Stefan Eissing (19 Nov 2023) + Closes #12710 -- vtls/vquic, keep peer name information together +Jay Wu (16 Jan 2024) - - add `struct ssl_peer` to keep hostname, dispname and sni - for a filter - - allocate `sni` for use in VTLS backend - - eliminate `Curl_ssl_snihost()` and its use of the download buffer - - use ssl_peer in SSL and QUIC filters +- url: don't set default CA paths for Secure Transport backend - Closes #12349 + As the default for this backend is the native CA store. -Viktor Szakats (18 Nov 2023) + Closes #12704 -- build: always revert `#pragma GCC diagnostic` after use +Lin Sun (16 Jan 2024) - Before this patch some source files were overriding gcc warning options, - but without restoring them at the end of the file. In CMake UNITY builds - these options spilled over to the remainder of the source code, - effecitvely disabling them for a larger portion of the codebase than - intended. +- asyn-ares: with modern c-ares, use its default timeout - `#pragma clang diagnostic` didn't have such issue in the codebase. + Closes #12703 - Reviewed-by: Marcel Raad - Closes #12352 +Daniel Stenberg (15 Jan 2024) -- tidy-up: casing typos, delete unused Windows version aliases +- tool_operate: stop setting the file comment on Amiga - - cmake: fix casing of `UnixSockets` to match the rest of the codebase. + - the URL is capped at 80 cols, which ruins it if longer + - it does not strip off URL credentials + - it is done unconditonally, not on --xattr + - we don't have Amiga in the CI which makes fixing it blindly fragile - - curl-compilers.m4: fix casing in a comment. + Someone who builds and tests on Amiga can add it back correctly in a + future if there is a desire. - - setup-win32: delete unused Windows version constant aliases. + Reported-by: Harry Sintonen + Closes #12709 - Reviewed-by: Marcel Raad - Closes #12351 +Stefan Eissing (15 Jan 2024) -- keylog: disable if unused +- rtsp: deal with borked server responses - Fully disable keylog code if there is no TLS or QUIC subsystem using it. + - enforce a response body length of 0, if the + response has no Content-lenght. This is according + to the RTSP spec. + - excess bytes in a response body are forwarded to + the client writers which will report and fail the + transfer - Closes #12350 + Follow-up to d7b6ce6 + Fixes #12701 + Closes #12706 -- cmake: add `CURL_DISABLE_BINDLOCAL` option +Daniel Stenberg (14 Jan 2024) - To match similar autotools option. +- version: show only the libpsl version, not its dependencies - Default is `ON`. + The libpsl version output otherwise also includes version number for its + dependencies, like IDN lib, but since libcurl does not use libpsl's IDN + functionality those components are not important. - Reviewed-by: Daniel Stenberg - Closes #12345 + Ref: https://github.com/curl/curl-for-win/issues/63 + Closes #12700 -- url: fix `-Wzero-length-array` with no protocols +Brad Harder (14 Jan 2024) - Fixes: - ``` - ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ - -Wc2x-extensions] - 178 | static const struct Curl_handler * const protocols[] = { - | ^ - ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length - -array] - ``` +- curl.h: CURLOPT_DNS_SERVERS is only available with c-ares - Closes #12344 + Closes #12695 -- url: fix builds with `CURL_DISABLE_HTTP` +Daniel Stenberg (14 Jan 2024) - Fixes: - ``` - ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' - 456 | Curl_mime_cleanpart(data->state.formp); - | ~~~~~~~~~~~ ^ - ``` +- cmdline-opts/gen.pl: error on initital blank line - Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 + After the "---" separator, there should be no blank line and this script + now errors out if one is detected. - Closes #12343 + Ref: #12696 + Closes #12698 -- http: fix `-Wunused-parameter` with no auth and no proxy +- cf-h1-proxy: no CURLOPT_USERAGENT in CONNECT with hyper - ``` - lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] - bool proxy) - ^ - ``` + Follow-up to 693cd1679361828a which was incomplete - Reviewed-by: Marcel Raad - Closes #12338 + Ref #12680 + Closes #12697 -Daniel Stenberg (16 Nov 2023) +- curl_multi_fdset.3: remove mention of null pointer support -- TODO: Some TLS options are not offered for HTTPS proxies + ... since this funtion has not supported null pointer fd_set arguments since + at least 2006. (That's when I stopped my git blame journey) - Closes #12286 - Closes #12342 + Fixes #12691 + Reported-by: sfan5 on github + Closes #12692 -- RELEASE-NOTES: synced +Mark Huang (14 Jan 2024) -- duphandle: make dupset() not return with pointers to old alloced data +- docs/cmdline: remove unnecessary line breaks - As the blob pointers are to be duplicated, the function must not return - mid-function with lingering pointers to the old handle's allocated data, - as that would lead to double-free in OOM situations. + Closes #12696 - Make sure to clear all destination pointers first to avoid this risk. +Daniel Stenberg (14 Jan 2024) - Closes #12337 +- transfer: remove warning: Value stored to 'blen' is never read -Viktor Szakats (16 Nov 2023) + Detected by scan-build -- http: fix `-Wunused-variable` compiler warning + Follow-up from 1cd2f0072f - Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. + Closes #12693 - E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + - `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + - `CURL_DISABLE_NTLM` on non-Windows. +Stefan Eissing (13 Jan 2024) + +- lib: replace readwrite with write_resp + + This clarifies the handling of server responses by folding the code for + the complicated protocols into their protocol handlers. This concerns + mainly HTTP and its bastard sibling RTSP. + The terms "read" and "write" are often used without clear context if + they refer to the connect or the client/application side of a + transfer. This PR uses "read/write" for operations on the client side + and "send/receive" for the connection, e.g. server side. If this is + considered useful, we can revisit renaming of further methods in another + PR. + + Curl's protocol handler `readwrite()` method been changed: + + ```diff + - CURLcode (*readwrite)(struct Curl_easy *data, struct connectdata *conn, + - const char *buf, size_t blen, + - size_t *pconsumed, bool *readmore); + + CURLcode (*write_resp)(struct Curl_easy *data, const char *buf, size_t ble + n, + + bool is_eos, bool *done); ``` - ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl - e] - CURLcode result = CURLE_OK; - ^ - ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus - ed-but-set-variable] - unsigned long *availp; - ^ - ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse - d-but-set-variable] - struct auth *authp; - ^ + + The name was changed to clarify that this writes reponse data to the + client side. The parameter changes are: + + * `conn` removed as it always operates on `data->conn` + * `pconsumed` removed as the method needs to handle all data on success + * `readmore` removed as no longer necessary + * `is_eos` as indicator that this is the last call for the transfer + response (end-of-stream). + * `done` TRUE on return iff the transfer response is to be treated as + finished + + This change affects many files only because of updated comments in + handlers that provide no implementation. The real change is that the + HTTP protocol handlers now provide an implementation. + + The HTTP protocol handlers `write_resp()` implementation will get passed + **all** raw data of a server response for the transfer. The HTTP/1.x + formatted status and headers, as well as the undecoded response + body. `Curl_http_write_resp_hds()` is used internally to parse the + response headers and pass them on. This method is public as the RTSP + protocol handler also uses it. + + HTTP/1.1 "chunked" transport encoding is now part of the general + *content encoding* writer stack, just like other encodings. A new flag + `CLIENTWRITE_EOS` was added for the last client write. This allows + writers to verify that they are in a valid end state. The chunked + decoder will check if it indeed has seen the last chunk. + + The general response handling in `transfer.c:466` happens in function + `readwrite_data()`. This mainly operates now like: + + ``` + static CURLcode readwrite_data(data, ...) + { + do { + Curl_xfer_recv_resp(data, buf) + ... + Curl_xfer_write_resp(data, buf) + ... + } while(interested); + ... + } ``` - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 + All the response data handling is implemented in + `Curl_xfer_write_resp()`. It calls the protocol handler's `write_resp()` + implementation if available, or does the default behaviour. - Fixes #12228 - Closes #12335 + All raw response data needs to pass through this function. Which also + means that anyone in possession of such data may call + `Curl_xfer_write_resp()`. -Jay Satiro (16 Nov 2023) + Closes #12480 -- tool: support bold headers in Windows +Daniel Stenberg (13 Jan 2024) - - If virtual terminal processing is enabled in Windows then use ANSI - escape codes Esc[1m and Esc[22m to turn bold on and off. +- RELEASE-NOTES: synced - Suggested-by: Gisle Vanem +- TODO: TFTP doesn't convert LF to CRLF for mode=netascii - Ref: https://github.com/curl/curl/discussions/11770 + Closes #12655 + Closes #12690 - Closes https://github.com/curl/curl/pull/12321 +- gen: do italics/bold for a range of letters, not just single word -Viktor Szakats (15 Nov 2023) + Previously it would match only on a sequence of non-space, which made it + miss to highlight for example "public suffix list". -- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` + Updated the recent cookie.d edit from 5da57193b732 to use bold instead + of italics. - Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + - `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, - or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while - compiling due to a missing HMAC declaration. + Closes #12689 - The reason is that HMAC is required by `lib/sha256.c` which publishes - `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when - building for libssh2 v1.8.2 (2019-05-25) or older. +- docs: describe and highlight super cookies - Make sure to compile the HMAC bits for a successful build. + Reported-by: Yadhu Krishna M - Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting - them into separate sources isn't practical. + Closes #12687 - Fixes: - ``` - [...] - In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc - url_object.dir/Unity/unity_0_c.c:310: - ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s - truct HMAC_params' - 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { - | ^ - ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par - ams' - [...] - ``` +- configure: when enabling QUIC, check that TLS supports QUIC - Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 + Most importantly perhaps is when using OpenSSL that the used + build/flavor has the QUIC API: the vanilla OpenSSL does not, only + BoringSSL, libressl, AWS-LC and quictls do. - Fixes #12273 - Closes #12332 + Ref: https://github.com/curl/curl/commit/5d044ad9480a9f556f4b6a252d7533b1ba7f + e57e#r136780413 -Daniel Stenberg (15 Nov 2023) + Closes #12683 -- duphandle: also free 'outcurl->cookies' in error path +Stefan Eissing (11 Jan 2024) - Fixes memory-leak when OOM mid-function +- vquic: extract TLS setup into own source - Use plain free instead of safefree, since the entire struct is - freed below. + - separate ngtcp2 specific parts out + - provide callback during init to allow ngtcp2 to apply its defaults - Remove some free calls that is already freed in Curl_freeset() + Closes #12678 - Closes #12329 +Sergey Markelov (11 Jan 2024) -Viktor Szakats (15 Nov 2023) +- multi: remove total timer reset in file_do() while fetching file:// -- config-win32: set `HAVE_SNPRINTF` for mingw-w64 + The total timer is properly reset in MSTATE_INIT. MSTATE_CONNECT starts + with resetting the timer that is a start point for further multi states. + If file://, MSTATE_DO calls file_do() that should not reset the total + timer. Otherwise, the total time is always less than the pre-transfer + and the start transfer times. - It's available in all mingw-w64 releases. We already pre-fill this - detection in CMake. + Closes #12682 - Closes #12325 +Daniel Stenberg (11 Jan 2024) -- sasl: fix `-Wunused-function` compiler warning +- http_proxy: a blank CURLOPT_USERAGENT should not be used in CONNECT - In builds with disabled auths. + Extended test 80 to verify this. - ``` - lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus - ed-function] - static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, - ^ - 1 warning generated. - ``` - Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st - ep:3:3822 + Reported-by: Stefan Eissing + Fixes #12680 + Closes #12681 - Reviewed-by: Daniel Stenberg - Closes #12326 +- sectransp: do verify_cert without memdup for blobs -- build: picky warning updates + Since the information is then already stored in memory, this can avoid + an extra set of malloc + free calls. - - cmake: sync some picky gcc warnings with autotools. - - cmake, autotools: add `-Wold-style-definition` for clang too. - - cmake: more precise version info for old clang options. - - cmake: use `IN LISTS` syntax in `foreach()`. + Closes #12679 - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #12324 +- hsts: remove assert for zero length domain -Daniel Stenberg (15 Nov 2023) + A zero length domain can happen if the HSTS parser is given invalid + input data which is not unheard of and is done by the fuzzer. -- urldata: move cookielist from UserDefined to UrlState + Follow-up from cfe7902111ae547873 - 1. Because the value is not strictly set with a setopt option. + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65661 - 2. Because otherwise when duping a handle when all the set.* fields are - first copied and an error happens (think out of memory mid-function), - the function would easily free the list *before* it was deep-copied, - which could lead to a double-free. + Closes #12676 - Closes #12323 +- headers: make sure the trailing newline is not stored -Viktor Szakats (14 Nov 2023) + extended test1940 to verify blank header fields too -- autotools: avoid passing `LDFLAGS` twice to libcurl + Bug: https://curl.se/mail/lib-2024-01/0019.html + Reported-by: Dmitry Karpov + Closes #12675 - autotools passes `LDFLAGS` automatically linker commands. curl's - `lib/Makefile.am` customizes libcurl linker flags. In that - customization, it added `LDFLAGS` to the custom flags. This resulted in - passing `LDFLAGS` _twice_ to the `libtool` command. +- curl_easy_header.3: tiny language fix - Most of the time this is benign, but some `LDFLAGS` options can break - the build when passed twice. One such example is passing `.o` files, - e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for - MUSL builds. + Closes #12672 - Passing them twice resulted in duplicate symbol errors: - ``` - libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a - arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] - ld.lld-15: error: duplicate symbol: _start - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - >>> defined at crt1.c - >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) - [...] - clang: error: linker command failed with exit code 1 (use -v to see invocatio - n) - ``` +- examples/range.c: add - This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. - The patch was a works-for-me hack that ended up merged in curl: - https://sourceforge.net/p/curl/bugs/1217/#06ef - With the root cause remaining unclear. + Closes #12671 - Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring - `-L` first? This would be unusual and suggests a bug in either the - linker or in `libtool`. +- examples/netrc.c: add - The curl build does pass the list of detected libs via its own - `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` - command line, but it's the job of `libtool` to ensure that even - a peculiar linker gets the options in the expected order. Also because - autotools passes `LDFLAGS` last, making it hardly possible to pass - anything after it. + Closes #12671 - Perhaps in the 10 years since this issue, this already got a fix - upstream. +- examples/ipv6.c: new example showing IPv6-only internet transfer - This patch deletes `LDFLAGS` from our customized libcurl options, - leaving a single copy of them as passed by autotools automatically. + Closes #12671 - Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 - Closes #12310 +- examples/address-scope.c: renamed from ipv6.c -- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` + It shows address scope use really - To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and - curl tool (`CURL_LDFLAGS_BIN`). + Closes #12671 - This makes it possible to build libcurl and curl with a single - invocation with lib- and tool-specific custom linker flags. +Stefan Eissing (9 Jan 2024) - Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, - controlling static/shared, incl. requesting a static curl tool (with - `-static-libtool-libs`) while building both shared and static libcurl. +- multi: pollset adjust, init with FIRSTSOCKET during connect - curl-for-win uses the above and some more. + - `conn->sockfd` is set by `Curl_setup_transfer()`, but that + is called *after* the connection has been established + - use `conn->sock[FIRSTSOCKET]` instead - These options are already supported in `Makefile.mk`. CMake has built-in - variables for this. + Follow-up to a0f94800d507de + Closes #12664 - Closes #12312 +Daniel Stenberg (9 Jan 2024) -Jay Satiro (14 Nov 2023) +- WEBSOCKET.md: remove dead link -- tool_cb_hdr: add an additional parsing check +- CI: spellcheck/appveyor: invoke configure --without-libpsl - - Don't dereference the past-the-end element when parsing the server's - Content-disposition header. + Follow-up to 2998874bb61ac6 - As 'p' is advanced it can point to the past-the-end element and prior - to this change 'p' could be dereferenced in that case. +- cmdline/docs/*.d: switch to using ## instead of .IP - Technically the past-the-end element is not out of bounds because dynbuf - (which manages the header line) automatically adds a null terminator to - every buffer and that is not included in the buffer length passed to - the header callback. + To make the editing easier. To write and to read. - Closes https://github.com/curl/curl/pull/12320 + Closes #12667 -Philip Heiduck (14 Nov 2023) +- gen.pl: support ## for doing .IP in table-like lists -- .cirrus.yml: freebsd 14 + Warn on use of .RS/.IP/.RE - ensure curl works on latest freebsd version + Closes #12667 - Closes #12053 +Jay Satiro (9 Jan 2024) -Daniel Stenberg (13 Nov 2023) +- cookie.d: Document use of empty string to enable cookie engine -- easy: in duphandle, init the cookies for the new handle + - Explain that --cookie "" can be used to enable the cookie engine + without reading any initial cookies. - ... not the source handle. + As is documented in CURLOPT_COOKIEFILE. - Closes #12318 + Ref: https://curl.se/libcurl/c/CURLOPT_COOKIEFILE.html -- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set + Bug: https://github.com/curl/curl/issues/12643#issuecomment-1879844420 + Reported-by: janko-js@users.noreply.github.com - Previously it would unconditionally use the size, which is set to -1 - when strlen is requested. + Closes https://github.com/curl/curl/pull/12646 - Updated test 544 to verify. +Daniel Stenberg (9 Jan 2024) - Closes #12317 +- setopt: use memdup0 when cloning COPYPOSTFIELDS -- RELEASE-NOTES: synced + Closes #12651 -- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped +- telnet: use dynbuf instad of malloc for escape buffer - Closes #12315 + Previously, send_telnet_data() would malloc + free a buffer every time + for escaping IAC codes. Now, it reuses a dynbuf for this purpose. -- urldata: move hstslist from 'set' to 'state' + Closes #12652 - To make it work properly with curl_easy_duphandle(). This, because - duphandle duplicates the entire 'UserDefined' struct by plain copy while - 'hstslist' is a linked curl_list of file names. This would lead to a - double-free when the second of the two involved easy handles were - closed. +- CI: install libpsl or configure --without-libpsl in builds - Closes #12315 + As a follow-up to the stricted libpsl check in configure -- test1900: verify duphandle with HSTS using multiple files +- configure: make libpsl detection failure cause error - Closes #12315 + To force users to explictily disable it if they really don't want it + used and make it harder to accidentally miss it. -Goro FUJI (13 Nov 2023) + --without-libpsl is the option to use if PSL is not wanted. -- http: allow longer HTTP/2 request method names + Closes #12661 - - Increase the maximum request method name length from 11 to 23. +- RELEASE-NOTES: synced - For HTTP/1.1 and earlier there's not a specific limit in libcurl for - method length except that it is limited by the initial HTTP request - limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same - and there was no specific limit. +- pop3: replace calloc + memcpy with memdup0 - According to Internet Assigned Numbers Authority (IANA) the longest - registered method is UPDATEREDIRECTREF which is 17 characters. + ... and make sure to return error on out of memory. - Also there are unregistered methods used by some companies that are - longer than 11 characters. + Closes #12650 - The limit was originally added by 61f52a97 but not used until fc2f1e54. +- lib: add debug log outputs for CURLE_BAD_FUNCTION_ARGUMENT - Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml + Closes #12658 - Closes https://github.com/curl/curl/pull/12311 +- mime: use memdup0 instead of malloc + memcpy -Jay Satiro (12 Nov 2023) + Closes #12649 -- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does +- tool_getparam: move the --rate logic into set_rate() - - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB - and CURLOPT_PROXY_CAINFO_BLOB docs. +- tool_getparam: switch to an enum for every option - All the other _BLOB option docs already have the same explanation. + To make the big switch much easier to read/understand and to make it + easier to add new options. - Closes https://github.com/curl/curl/pull/12277 +- tool_getparam: build post data using dynbuf (more) -Viktor Szakats (11 Nov 2023) +- tool_getparam: replace malloc + copy by dynbuf for --data -- tidy-up: dedupe Windows system libs in cmake +- tool_getparam: make data_urlencode avoid direct malloc - Reviewed-by: Daniel Stenberg - Closes #12307 + use aprintf() instead -Junho Choi (11 Nov 2023) +- tool_getparam: move the --url-query logic into url_query() -- ci: test with latest quiche release (0.19.0) + This function is not doing post at all so it was always weirdly placed. - Closes #12180 +- tool_getparam: move the --data logic into set_data() -- quiche: use quiche_conn_peer_transport_params() +- tool_getparam: unify the cmdline switch() into a single one - In recent quiche, transport parameter API is separated - with quiche_conn_peer_transport_params(). - (https://github.com/cloudflare/quiche/pull/1575) - It breaks with bulding with latest(post 0.18.0) quiche. + - easier to follow, easier to modify, easier to extend, possibly slightly + faster - Closes #12180 + - each case now has the long option as a comment -Daniel Stenberg (11 Nov 2023) +- tool_getparam: bsearch cmdline options -- Makefile: generate the VC 14.20 project files at dist-time + - the option names are now alpha sorted and lookup is a lot faster - Follow-up to 28287092cc5a6d6ef8 (#12282) + - use case sensitive matching. It was previously case insensitive, but that + was not documented nor tested. - Closes #12290 + - remove "partial match" feature. It was not documented, not tested and + was always fragile as existing use could break when we add a new + option -Sam James (11 Nov 2023) + - lookup short options via a table -- misc: fix -Walloc-size warnings + Closes #12631 - GCC 14 introduces a new -Walloc-size included in -Wextra which gives: +Gabe (8 Jan 2024) - ``` - src/tool_operate.c: In function ‘add_per_transfer’: - src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo - r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] - 213 | p = calloc(sizeof(struct per_transfer), 1); - | ^ - src/var.c: In function ‘addvariable’: - src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type - struct var’ with size ‘32’ [-Walloc-size] - 361 | p = calloc(sizeof(struct var), 1); - | ^ - ``` +- COPYING: update copyright year - The calloc prototype is: - ``` - void *calloc(size_t nmemb, size_t size); - ``` + Closes #12654 - So, just swap the number of members and size arguments to match the - prototype, as we're initialising 1 struct of size `sizeof(struct - ...)`. GCC then sees we're not doing anything wrong. +Stefan Eissing (8 Jan 2024) - Closes #12292 +- url: init conn->sockfd and writesockfd to CURL_SOCKET_BAD -Mark Gaiser (11 Nov 2023) + Also add more tracing to test 19 -- IPFS: bugfixes + Follow-up to a0f9480 - - Fixed endianness bug in gateway file parsing - - Use IPFS_PATH in tests where IPFS_DATA was used - - Fixed typos from traling -> trailing - - Fixed broken link in IPFS.md + Fixes #12657 + Closes #12659 - Follow-up to 859e88f6533f9e +Daniel Stenberg (8 Jan 2024) - Reported-by: Michael Kaufmann - Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 - Closes #12305 +- connect: remove margin from eyeballer alloc -Daniel Stenberg (11 Nov 2023) + Presumably leftovers from debugging -- VULN-DISCLOSURE-POLIC: remove broken link to hackerone + Closes #12647 - It should ideally soon not be done from hackerone anyway +- ftp: only consider entry path if it has a length - Closes #12308 + Follow-up from 8edcfedc1a144f438bd1cdf814a0016cb -Andrew Kurushin (11 Nov 2023) + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65631 -- schannel: add CA cache support for files and memory blobs + Avoids a NULL pointer deref. - - Support CA bundle and blob caching. + Closes #12648 - Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. +Stefan Eissing (7 Jan 2024) - Closes https://github.com/curl/curl/pull/12261 +- transfer: adjust_pollset improvements -Daniel Stenberg (10 Nov 2023) + - let `multi_getsock()` initialize the pollset in what the + transfer state requires in regards to SEND/RECV + - change connection filters `adjust_pollset()` implementation + to react on the presence of POLLIN/-OUT in the pollset and + no longer check CURL_WANT_SEND/CURL_WANT_RECV + - cf-socket will no longer add POLLIN on its own + - http2 and http/3 filters will only do adjustments if the + passed pollset wants to POLLIN/OUT for the transfer on + the socket. This is similar to the HTTP/2 proxy filter + and works in stacked filters. -- RELEASE-NOTES: synced + Closes #12640 -Charlie C (10 Nov 2023) +Daniel Stenberg (6 Jan 2024) -- cmake: option to disable install & drop `curlu` target when unused +- ftp: use memdup0 to store the OS from a SYST 215 response - This patch makes the following changes: - - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. - - Removes the target `curlu` when the option `BUILD_TESTING` is set to - `OFF` - to prevent it from being loaded in Visual Studio. + avoid malloc + direct buffer fiddle - Closes #12287 + Closes #12639 -Kai Pastor (10 Nov 2023) +- ftp: use dynbuf to store entrypath -- cmake: fix multiple include of CURL package + avoid direct malloc - Fixes errors on second `find_package(CURL)`. This is a frequent case - with transitive dependencies: - ``` - CMake Error at ...: - add_library cannot create ALIAS target "CURL::libcurl" because another - target with the same name already exists. - ``` + Closes #12638 - Test to reproduce: - ```cmake - cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher +Lealem Amedie (6 Jan 2024) - project(curl) +- wolfssl: load certificate *chain* for PEM client certs - set(CURL_DIR "example/lib/cmake/CURL/") - find_package(CURL CONFIG REQUIRED) - find_package(CURL CONFIG REQUIRED) # fails + Closes #12634 - add_executable(main main.c) - target_link_libraries(main CURL::libcurl) - ``` +Stefan Eissing (4 Jan 2024) - Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes - Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html - Ref: #12300 - Assisted-by: Harry Mallon - Closes #11913 +- http: adjust_pollset fix -Viktor Szakats (8 Nov 2023) + do not add a socket for POLLIN when the transfer does not want to send + (for example is paused). -- tidy-up: use `OPENSSL_VERSION_NUMBER` + Follow-up to 47f5b1a - Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. - Before this patch some places used `OPENSSL_VERSION_MAJOR`. - - Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't - define any version number in these implementations: BoringSSL, AWS-LC, - LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to - `opensslv.h`. This wasn't causing a deeper problem because the code is - looking for v3, which is only provided by OpenSSL/quictls as of now. - - According to https://github.com/openssl/openssl/issues/17517, the macro - `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. + Reported-by: bubbleguuum on github + Fixes #12632 + Closes #12633 - Reviewed-by: Marcel Raad - Closes #12298 +Daniel Stenberg (3 Jan 2024) -Daniel Stenberg (8 Nov 2023) +- tool: make parser reject blank arguments if not supported -- resolve.d: drop a multi use-sentence + Already in the getstr() function that clones the input argument. - Since the `multi:` keyword adds that message. + Closes #12620 - Reported-by: 積丹尼 Dan Jacobson - Fixes https://github.com/curl/curl/discussions/12294 - Closes #12295 +dependabot[bot] (3 Jan 2024) -- content_encoding: make Curl_all_content_encodings allocless +- build(deps): bump github/codeql-action from 2 to 3 - - Fixes a memory leak pointed out by Coverity - - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? - id=63947 - - Avoids unncessary allocations + Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 + to 3. + - [Release notes](https://github.com/github/codeql-action/releases) + - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) + - [Commits](https://github.com/github/codeql-action/compare/v2...v3) - Follow-up ad051e1cbec68b2456a22661b + --- + updated-dependencies: + - dependency-name: github/codeql-action + dependency-type: direct:production + update-type: version-update:semver-major + ... - Closes #12289 + Signed-off-by: dependabot[bot] -Michael Kaufmann (7 Nov 2023) + Closes #12625 -- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 +- build(deps): bump actions/checkout from 3 to 4 - Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), - avoid it and use "http/1.1" instead. + Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4. + - [Release notes](https://github.com/actions/checkout/releases) + - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) + - [Commits](https://github.com/actions/checkout/compare/v3...v4) - This reverts commit df856cb5c9 (#10183). + --- + updated-dependencies: + - dependency-name: actions/checkout + dependency-type: direct:production + update-type: version-update:semver-major + ... - Fixes #12259 - Closes #12285 + Signed-off-by: dependabot[bot] -Daniel Stenberg (7 Nov 2023) + Closes #12624 -- Makefile.am: drop vc10, vc11 and vc12 projects from dist +- build(deps): bump actions/upload-artifact from 3 to 4 - They are end of life products. Support for generating them remain in the - repo for a while but this change drops them from distribution. + Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) f + rom 3 to 4. + - [Release notes](https://github.com/actions/upload-artifact/releases) + - [Commits](https://github.com/actions/upload-artifact/compare/v3...v4) - Closes #12288 + --- + updated-dependencies: + - dependency-name: actions/upload-artifact + dependency-type: direct:production + update-type: version-update:semver-major + ... -David Suter (7 Nov 2023) + Signed-off-by: dependabot[bot] -- projects: add VC14.20 project files + Closes #12627 - Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. - OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why - this is missing. Updated the templates to produce a VC14.20 project. - Project opens in Visual Studio 2019 as expected. +- build(deps): bump actions/download-artifact from 3 to 4 - Closes #12282 + Bumps [actions/download-artifact](https://github.com/actions/download-artifac + t) from 3 to 4. + - [Release notes](https://github.com/actions/download-artifact/releases) + - [Commits](https://github.com/actions/download-artifact/compare/v3...v4) -Daniel Stenberg (7 Nov 2023) + --- + updated-dependencies: + - dependency-name: actions/download-artifact + dependency-type: direct:production + update-type: version-update:semver-major + ... -- curl: move IPFS code into src/tool_ipfs.[ch] + Signed-off-by: dependabot[bot] - - convert ensure_trailing into ensure_trailing_slash - - strdup the URL string to own it proper - - use shorter variable names - - combine some expressions - - simplify error handling in ipfs_gateway() - - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached - - ipfs-gateway.d polish and simplification - - shorten ipfs error message + make them "synthetic" + Closes #12626 - Closes #12281 +Stefan Eissing (3 Jan 2024) -Viktor Szakats (6 Nov 2023) +- http3/quiche: fix result code on a stream reset -- build: delete support bits for obsolete Windows compilers + - fixes pytest failures in test 07_22 + - aligns CURLcode values on stream reset with ngtcp2 - - Pelles C: Unclear status, failed to obtain a fresh copy a few months - ago. Possible website is HTTP-only. ~10 years ago I left this compiler - dealing with crashes and other issues with no response on the forum - for years. It has seen some activity in curl back in 2021. - - LCC: Last stable release in September 2002. - - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned - in 2006. - - Borland C++: We dropped Borland C++ support in 2018. - - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 - (or possibly 2008) as a minimum. + Closes #12629 - Closes #12222 +Daniel Stenberg (2 Jan 2024) -- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` +- setopt: clear mimepost when formp is freed - We use `stdint.h` unconditionally in all places except one. These uses - are imposed by external dependencies / features. nghttp2, quic, wolfSSL - and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that - any of these features make curl require a C99 compiler. (In case of - MSVC, this means Visual Studio 2010 or newer.) + A precaution to avoid a possibly dangling pointer left behind. - This patch changes the single use of `stdint.h` guarded by - `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using - `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used - anywhere else, allowing to delete this feature check as well. + Reported-by: Thomas Ferguson + Fixes #12608 + Closes #12621 - Closes #12275 +Andy Alt (2 Jan 2024) -Daniel Stenberg (6 Nov 2023) +- CI: Add dependabot.yml -- tool_operate: do not mix memory models + This will cause dependabot to open a PR when various actions are + updated, provided that the action maintainer has issued a release. - Make sure 'inputpath' only points to memory allocated by libcurl so that - curl_free works correctly. + Closes #12623 - Pointed out by Coverity +Gisle Vanem (2 Jan 2024) - Follow-up to 859e88f6533f9e1f890 +- content_encoding: change return code to typedef'ed enum - Closes #12280 + ... to work around a clang ubsan warning. -Stefan Eissing (6 Nov 2023) + Fixes #12618 + Closes #12622 -- lib: client writer, part 2, accounting + logging +Daniel Stenberg (2 Jan 2024) - This PR has these changes: +- tool: prepend output_dir in header callback - Renaming of unencode_* to cwriter, e.g. client writers - - documentation of sendf.h functions - - move max decode stack checks back to content_encoding.c - - define writer phase which was used as order before - - introduce phases for monitoring inbetween decode phases - - offering default implementations for init/write/close + When Content-Disposition parsing is used and an output dir is prepended, + make sure to store that new file name correctly so that it can be used + for setting the file timestamp when --remote-time is used. - Add type paramter to client writer's do_write() - - always pass all writes through the writer stack - - writers who only care about BODY data will pass other writes unchanged + Extended test 3012 to verify. - add RAW and PROTOCOL client writers - - RAW used for Curl_debug() logging of CURLINFO_DATA_IN - - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and - Curl_pgrsSetDownloadCounter() - - remove all updates of data->req.bytecount and calls to - Curl_pgrsSetDownloadCounter() and Curl_debug() from other code - - adjust test457 expected output to no longer see the excess write + Co-Authored-by: Jay Satiro + Reported-by: hgdagon on github + Fixes #12614 + Closes #12617 - Closes #12184 +- test1254: fix typo in name plus shorten it -Daniel Stenberg (6 Nov 2023) +- RELEASE-NOTES: synced -- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw +Viktor Szakats (2 Jan 2024) - Closes #12278 +- schannel: fix `-Warith-conversion` gcc 13 warning -Viktor Szakats (6 Nov 2023) + ``` + lib/vtls/schannel.c:1201:22: warning: conversion to 'unsigned int' from 'int' + may change the sign of the result [-Warith-conversion] + 1201 | *extension_len = *list_len + + | ^ + ``` -- rand: fix build error with autotools + LibreSSL + Closes #12616 - autotools unexpectedly detects `arc4random` because it is also looking - into dependency libs. One dependency, LibreSSL, happens to publish an - `arc4random` function (via its shared lib before v3.7, also via static - lib as of v3.8.2). When trying to use this function in `lib/rand.c`, - its protoype is missing. To fix that, curl included a prototype, but - that used a C99 type without including `stdint.h`, causing: +- asyn-thread: silence `-Wcast-align` warning for Windows + Seen with llvm/clang 17: ``` - ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' - 37 | uint32_t arc4random(void); - | ^ - 1 error generated. + lib/asyn-thread.c:310:5: warning: cast from 'PCHAR' (aka 'char *') to 'struct + thread_sync_data *' increases required alignment from 1 to 8 [-Wcast-align] + 310 | CONTAINING_RECORD(overlapped, struct thread_sync_data, w8.overlap + ped); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ~~~~ + .../llvm-mingw/aarch64-w64-mingw32/include/winnt.h:717:48: note: expanded fro + m macro 'CONTAINING_RECORD' + 717 | #define CONTAINING_RECORD(address,type,field) ((type *)((PCHAR)(addre + ss) - (ULONG_PTR)(&((type *)0)->field))) + | ^~~~~~~~~~~~~~~~~~~~~~ + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``` - This patch improves this by dropping the local prototype and instead - limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide - their own random source anyway. + Follow-up to a6bbc87f9e9ffb46a1801dfb983e7534825ed56b #12482 - The better fix would be to teach autotools to not link dependency libs - while detecting `arc4random`. + Ref: https://github.com/curl/curl/pull/12482#issuecomment-1873017261 + Closes #12615 - LibreSSL publishing a non-namespaced `arc4random` tracked here: - https://github.com/libressl/portable/issues/928 +Daniel Stenberg (2 Jan 2024) - Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 +- tool_listhelp: regenerate after recent .d updates - Reviewed-by: Daniel Stenberg - Fixes #12257 - Closes #12274 + Makes it survive test 1478 -Daniel Stenberg (5 Nov 2023) + Closes #12612 -- RELEASE-NOTES: synced +- test1478: verify src/tool_listhelp.c -- strdup: do Curl_strndup without strncpy + Verify that the source file on disk is identical to the output of gen.pl + listhelp, as otherwise they are out of sync and need attention. - To avoid (false positive) gcc-13 compiler warnings. + Closes #12612 - Follow-up to 4855debd8a2c1cb +- testutil: make runtests support %include - Assisted-by: Jay Satiro - Reported-by: Viktor Szakats - Fixes #12258 + Using this instruction, a test case can include the contents of a file + into the test during the preprocessing. -Enno Boland (5 Nov 2023) + Closes #12612 -- HTTP: fix empty-body warning +- runtests: for mode="text" on , fix newlines on both parts - This change fixes a compiler warning with gcc-12.2.0 when - `-DCURL_DISABLE_BEARER_AUTH=ON` is used. + Closes #12612 - /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': - /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp - ty body in an 'else' statement [-Wempty-body] - 1147 | ; - | ^ +Jay Satiro (2 Jan 2024) - Closes #12262 +- quiche: return CURLE_HTTP3 on send to invalid stream -Daniel Stenberg (5 Nov 2023) + Prior to this change if a send failed on a stream in an invalid state + (according to quiche) and not marked as closed (according to libcurl) + then the send function would return CURLE_SEND_ERROR. -- openssl: identify the "quictls" backend correctly + We already have similar code for ngtcp2 to return CURLE_HTTP3 in this + case. - Since vanilla OpenSSL does not support the QUIC API I think it helps - users to identify the correct OpenSSL fork in version output. The best - (crude) way to do that right now seems to be to check if ngtcp2 support - is enabled. + Caught by test test_07_upload.py: test_07_22_upload_parallel_fail. - Closes #12270 + Fixes https://github.com/curl/curl/issues/12590 + Closes https://github.com/curl/curl/pull/12597 -Mark Gaiser (5 Nov 2023) +Daniel Stenberg (1 Jan 2024) -- curl: improved IPFS and IPNS URL support +- cmdline-opts: update availability for the *-ca-native options - Previously just ipfs:// and ipns:// was supported, which is - too strict for some usecases. + Closes #12613 - This patch allows paths and query arguments to be used too. - Making this work according to normal http semantics: +Patrick Monnerat (31 Dec 2023) - ipfs:///foo/bar?key=val - ipns:///foo/bar?key=val +- openldap: fix STARTTLS - The gateway url support is changed. - It now only supports gateways in the form of: + It was not working anymore since introduction of connection filters. - http:///foo/bar - http:// + Also do not attempt to recover from a failing TLS negotiation with + CURLUSESSL_TRY. - Query arguments here are explicitly not allowed and trigger an intended - malformed url error. + Closes #12610 - There also was a crash when IPFS_PATH was set with a non trailing - forward slash. This has been fixed. +Daniel Stenberg (31 Dec 2023) - Lastly, a load of test cases have been added to verify the above. +- haproxy-clientip.d: document the arg - Reported-by: Steven Allen - Fixes #12148 - Closes #12152 + The arg keyword was missing and therefore not present in the man page. -Harry Mallon (5 Nov 2023) + Closes #12611 -- docs: KNOWN_BUGS cleanup +annalee (29 Dec 2023) - * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. - Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 +- configure: fix no default int compile error in ipv6 detection - * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. - Fixed in #11806 + Closes #12607 - * Remove mention of aws-sigv4 query empty value problems +Dan Fandrich (28 Dec 2023) - * Remove mention of aws-sigv4 missing amz-content-sha256 - Fixed in #9995 +- CI: Fix use of any-glob-to-all-files in the labeler -- http_aws_sigv4: canonicalise valueless query params + Despite its name, this atom acts like one-glob-to-all-files and a + different syntax with braces must be used to get + any-glob-to-all-files semantics. Unfortunately, this makes the file + completely unreadable. - Fixes #8107 - Closes #12244 + Ref: https://github.com/actions/labeler/issues/731 -Michael Kaufmann (4 Nov 2023) +Daniel Stenberg (29 Dec 2023) -- docs: preserve the modification date when copying the prebuilt man page +- CURLOPT_AUTOREFERER.3: mention CURLINFO_REFERER - The previously built man page "curl.1" must be copied with the original - modification date, otherwise the man page is never updated. +- CURLINFO_REFERER.3: clarify that it is the *request* header - This fixes a bug that has been introduced with commit 2568441cab. + That libcurl itself sent in the most recent request - Reviewed-by: Dan Fandrich - Reviewed-by: Daniel Stenberg + Closes #12605 - Closes #12199 +Jay Satiro (28 Dec 2023) -Daniel Stenberg (4 Nov 2023) +- system_win32: fix a function pointer assignment warning -- docs: remove bold from some man page SYNOPSIS sections + - Use CURLX_FUNCTION_CAST to suppress a function pointer assignment + warning. - In the name of consistency + a6bbc87f added lookups of some Windows API functions and then cast them + like `*(FARPROC*)&Curl_funcname = address`. Some versions of gcc warn + about that as breaking strict-aliasing rules so this PR changes those + assignments to use CURLX_FUNCTION_CAST. - Closes #12267 + Bug: https://github.com/curl/curl/pull/12581#issuecomment-1869804317 + Reported-by: Marcel Raad -- openssl: two multi pointer checks should probably rather be asserts + Closes https://github.com/curl/curl/pull/12602 - ... so add the asserts now and consider removing the dynamic checks in a - future. +- verify-examples.pl: fail verification on unescaped backslash - Ref: #12261 - Closes #12264 + - Check that all backslashes in EXAMPLE are properly escaped. -boilingoden (4 Nov 2023) + eg manpage must always use `\\n` never `\n`. -- docs: add supported version for the json write-out + This is because the manpage requires we always double blackslash to show + a single backslash. Prior to this change an erroneous single backslash + would pass through and compile even though it would not show correctly + in the manpage. - xref: https://curl.se/changes.html#7_70_0 + Co-authored-by: Daniel Stenberg - Closes #12266 + Ref: https://github.com/curl/curl/pull/12588 -Viktor Szakats (3 Nov 2023) + Closes https://github.com/curl/curl/pull/12589 -- appveyor: make VS2008-built curl tool runnable +- vtls: fix missing multissl version info - By linking the CRT statically. This avoids the error about missing - runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`. + - Fix erroneous buffer copy logic from ff74cef5. - Closes #12263 + Prior to this change the MultiSSL version info returned to the user + was empty. -Stefan Eissing (3 Nov 2023) + Closes https://github.com/curl/curl/pull/12599 -- url: proxy ssl connection reuse fix +Daniel Stenberg (27 Dec 2023) - - tunnel https proxy used for http: transfers does - no check if proxy-ssl configuration matches - - test cases added, test_10_12 fails on 8.4.0 +- KNOWN_BUGS: [RTSP] Some methods do not support response bodies - Closes #12255 + Closes #12414 -Jay Satiro (3 Nov 2023) +Patrick Monnerat (27 Dec 2023) -- curl_sspi: support more revocation error names in error messages +- openldap: fix an LDAP crash - - Add these revocation errors to sspi error list: - CRYPT_E_NO_REVOCATION_DLL, CRYPT_E_NO_REVOCATION_CHECK, - CRYPT_E_REVOCATION_OFFLINE and CRYPT_E_NOT_IN_REVOCATION_DATABASE. + Reported-by: Ozan Cansel + Fixes #12593 + Closes #12600 - Prior to this change those error codes were not matched to their macro - name and instead shown as "unknown error". +Daniel Stenberg (27 Dec 2023) - Before: +- getinfo: CURLINFO_QUEUE_TIME_T - schannel: next InitializeSecurityContext failed: - Unknown error (0x80092013) - The revocation function was - unable to check revocation because the revocation server was offline. + Returns the time, in microseconds, during which this transfer was held + in a waiting queue before it started "for real". A transfer might be put + in a queue if after getting started, it cannot create a new connection + etc due to set conditions and limits imposed by the application. - After: + Ref: #12293 + Closes #12368 - schannel: next InitializeSecurityContext failed: - CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was - unable to check revocation because the revocation server was offline. +- RELEASE-NOTES: synced - Bug: https://github.com/curl/curl/issues/12239 - Reported-by: Niracler Li +Jay Satiro (26 Dec 2023) - Closes https://github.com/curl/curl/pull/12241 +- examples/sendrecv: fix comment line length -- strdup: don't allow Curl_strndup to read past a null terminator + Caught by checksrc. - - Use malloc + strncpy instead of Curl_memdup to dupe the string before - null terminating it. +Haydar Alaidrus (23 Dec 2023) - Prior to this change if Curl_strndup was passed a length longer than - the allocated string then it could copy out of bounds. +- CURLOPT_POSTFIELDS.3: fix incorrect C string escape in example - This change is for posterity. Curl_strndup was added in the parent - commit and currently none of the calls to it pass a length that would - cause it to read past the allocated length of the input. + - Escape inner quotes with two backslashes. - Follow-up to d3b3ba35. + Two backslashes escapes the backslash for the man page and will show as + a single backslash. - Closes https://github.com/curl/curl/pull/12254 + eg: "{\\"name\\": \\"daniel\\"}" shows as "{\"name\": \"daniel\"}". -Daniel Stenberg (2 Nov 2023) + Closes https://github.com/curl/curl/pull/12588 -- lib: add and use Curl_strndup() +Viktor Szakats (23 Dec 2023) - The Curl_strndup() function is similar to memdup(), but copies 'n' bytes - then adds a terminating null byte ('\0'). +- appveyor: tidy-ups - Closes #12251 + - replace two remaining backslashes with forward slashes. + - tidy up the way we form and pass `TFLAGS`. -- CURPOST_POSTFIELDS.3: add CURLOPT_COPYPOSTFIELDS in SEE ALSO + Follow-up to 2d4d0c1fd32f5cc3f946c407c8eccd5477b287df #12572 -Stefan Eissing (2 Nov 2023) + Closes #12582 -- pytest: use lower count in repeat tests +Stefan Eissing (22 Dec 2023) - - lower large iteration counts in some tests somewhat for - the same coverage with less duration +- transfer: fix upload rate limiting, add test cases - Closes #12248 + - add test cases for rate limiting uploads for all + http versions + - fix transfer loop handling of limits. Signal a re-receive + attempt only on exhausting maxloops without an EAGAIN + - fix `data->state.selectbits` forcing re-receive to also + set re-sending when transfer is doing this. -Daniel Stenberg (2 Nov 2023) + Reported-by: Karthikdasari0423 on github + Fixes #12559 + Closes #12586 -- RELEASE-NOTES: synced +Daniel Stenberg (22 Dec 2023) -- docs: clarify that curl passes on input unfiltered +- mbedtls: free the entropy when threaded - ... for several options. + The entropy_free was never done for threaded builds, causing a small + (fixed) memory leak. - Reported-by: Ophir Lojkine + Reported-by: RevaliQaQ on github + Fixes #12584 + Closes #12585 - Closes #12249 +Stefan Eissing (22 Dec 2023) -- urlapi: when URL encoding the fragment, pass in the right length +- http2: improved on_stream_close/data_done handling - A benign bug because it would only add an extra null terminator. + - there seems to be a code path that cleans up easy handles without + triggering DONE or DETACH events to the connection filters. This + would explain wh nghttp2 still holds stream user data + - add GOOD check to easy handle used in on_close_callback to + prevent crashes, ASSERTs in debug builds. + - NULL the stream user data early before submitting RST + - add checks in on_stream_close() to identify UNGOOD easy handles - Made lib1560 get a test that runs this code. + Reported-by: Hans-Christian Egtvedt + Fixes #10936 + Closes #12562 - Closes #12250 +Daniel Stenberg (22 Dec 2023) -Stefan Eissing (2 Nov 2023) +- mprintf: overhaul and bugfixes -- vtls: late clone of connection ssl config + In a test case using lots of snprintf() calls using many commonly used + %-codes per call, this version is around 30% faster than previous + version. - - perform connection cache matching against `data->set.ssl.primary` - and proxy counterpart - - fully clone connection ssl config only when connection is used + It also fixes the #12561 bug which made it not behave correctly when + given unknown %-sequences. Fixing that flaw required a different take on + the problem, which resulted in the new two-arrays model. - Closes #12237 + lib557: extended - Verify the #12561 fix and test more printf features -- msh3: error when built with CURL_DISABLE_SOCKETPAIR set + unit1398: fix test: It used a $ only for one argument, which is not + supported. - Reported-by: Gisle Vanem - Closes #12252 - Fixes #12213 + Fixes #12561 + Closes #12563 -Daniel Stenberg (2 Nov 2023) +Viktor Szakats (21 Dec 2023) -- hsts: skip single-dot hostname +- appveyor: replace PowerShell with bash + parallel autotools - Reported-by: Maksymilian Arciemowicz + PowerShell works (after a steep development curve), but one property of + it stuck and kept causing unresolvable usability issues: With + `$ErrorActionPreference=Stop`, it does abort on failures, but shows only + the first line of the error message. In `Continue` mode, it shows the + full error message, but doesn't stop on all errors. Another issue is + PowerShell considering any stderr output as if the command failed (this + has been improved in 7.2 (2021-Nov), but fixed versions aren't running + in CI and will not be for a long time in all test images.) - Closes #12247 + Thus, we're going with bash. -- vtls: fix build without proxy + Also: + - use `-j2` with autotools tests, making them finish 5-15 minutes per + job faster. + - omit `POSIX_PATH_PREFIX`. + - use `WINDIR`. + - prefer forward slashes. - Follow-up to bf0e278a3c54bc7fee7360da17c + Follow-up to: 75078a415d9c769419aed4153d3d525a8eba95af #11999 + Ref: #12444 - closes #12243 + Fixes #12560 + Closes #12572 -- docs/example/keepalive.c: show TCP keep-alive options +Pavel Pavlov (21 Dec 2023) - Closes #12242 +- asyn-thread: use GetAddrInfoExW on >= Windows 8 -- lib1560: verify appending blank URL encoded query string + For doing async DNS resolution instead of starting a thread for each + request. -- urlapi: skip appending NULL pointer query + Fixes #12481 + Closes #12482 - Reported-by: kirbyn17 on hackerone +Daniel Stenberg (21 Dec 2023) - Closes #12240 +- strerror: repair get_winsock_error() -- lib1560: verify setting host to "" with and without URL encode + It would try to read longer than the provided string and crash. -- urlapi: avoid null deref if setting blank host to url encode + Follow-up to ff74cef5d4a0cf60106517a1c7384 + Reported-by: calvin2021y on github + Fixes #12578 + Closes #12579 - Reported-by: kirbyn17 on hackerone +- CURLOPT_SSH_*_KEYFILE: clarify - Closes #12240 + Closes #12554 -- dynbuf: assert for NULL pointer inputs +ivanfywang (21 Dec 2023) - Help us catch more mistakes. +- ngtcp2: put h3 at the front of alpn - Closes #12238 + Closes #12576 -- HTTP3: ngtcp2 builds are no longer experimental +Daniel Stenberg (21 Dec 2023) - The other HTTP/3 backends are still experimental. +- test460: verify a command line using --expand with no argument - Closes #12235 + This verifies the fix for #12565 -Stefan Eissing (31 Oct 2023) +- tool_getparam: do not try to expand without an argument -- vtls: cleanup SSL config management + This would lead to a segfault. - - remove `Curl_ssl_get_config()`, no longer needed + Fixes #12565 + Reported-by: Geeknik Labs + Closes #12575 - Closes #12204 +- RELEASE-NOTES: synced -Daniel Stenberg (31 Oct 2023) + Bumped version to 8.6.0 because of changes -- libcurl-thread.3: simplify the TLS section +- Makefile.am: fix the MSVC project generation - All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August - 2016. + It made the vcxproj files not get included in dist tarballs. - Closes #12233 + Regression since 74423b5df4c8117891eb89 (8.5.0) -- configure: better --disable-http + Reported-by: iAroc on github + Fixes #12564 + Closes #12567 - - disable HTTPS-proxy as well, since it can't work without HTTP +zengwei2000 (21 Dec 2023) - - curl_setup: when HTTP is disabled, also disable all features that are - HTTP-only +- altsvc: free 'as' when returning error - - version: HTTPS-proxy only exists if HTTP support exists + Closes #12570 - Closes #12223 + Signed-off-by: zengwei -- http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine +Viktor Szakats (20 Dec 2023) - Finding a 'Content-Range:' in the response changed the handling. +- build: fix `-Wconversion`/`-Wsign-conversion` warnings - Add test case 1475 to verify -C - with 416 and Content-Range: header, - which is almost exactly like test 194 which instead uses a fixed -C - offset. Adjusted test 194 to also be considered fine. + Fix remaining warnings in examples and tests which are not suppressed + by the pragma in `lib/curl_setup.h`. - Fixes #10521 - Reported-by: Smackd0wn - Fixes #12174 - Reported-by: Anubhav Rai - Closes #12176 + Silence a toolchain issue causing warnings in `FD_SET()` calls with + older Cygwin/MSYS2 builds. Likely fixed on 2020-08-03 by: + https://cygwin.com/git/?p=newlib-cygwin.git;a=commitdiff;h=5717262b8ecfed0f7f + ab63e2c09c78991e36f9dd -Stefan Eissing (30 Oct 2023) + Follow-up to 2dbe75bd7f3c36837aa06fd87a442bdf3fb7faef #12492 -- GHA: fix checkout of quictls repository to use correct branch name + Closes #12557 - Follow-up to c868b0e30f10cd0ac7 +- build: fix some `-Wsign-conversion`/`-Warith-conversion` warnings - Closes #12232 + - enable `-Wsign-conversion` warnings, but also setting them to not + raise errors. + - fix `-Warith-conversion` warnings seen in CI. + These are triggered by `-Wsign-converion` and causing errors unless + explicitly silenced. It makes more sense to fix them, there just a few + of them. + - fix some `-Wsign-conversion` warnings. + - hide `-Wsign-conversion` warnings with a `#pragma`. + - add macro `CURL_WARN_SIGN_CONVERSION` to unhide them on a per-build + basis. + - update a CI job to unhide them with the above macro: + https://github.com/curl/curl/actions/workflows/linux.yml -> OpenSSL -O3 -Daniel Stenberg (30 Oct 2023) + Closes #12492 -- docs/example/localport.c: show off CURLOPT_LOCALPORT +- cmake: tidy-up `OtherTests.cmake` - Closes #12230 + - make more obvious which detection uses which prep steps. + - merge and streamline conditions. + - these should not alter detection results. -- docs/examples/interface.c: show CURLOPT_INTERFACE use + Also align log output messages from + `Macros.cmake` / `curl_internal_test` with rest of the build. - Although super simple. + Closes #12551 - Closes #12229 +- appveyor: switch to out-of-tree builds -Viktor Szakats (30 Oct 2023) + With cmake and autotools. -- build: fix compiler warning with auths disabled + Closes #12550 - ``` - ./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' - [-Wunused-function] - static int is_valid_auth_separator(char ch) - ^ - 5 warnings generated. - ``` +Daniel Stenberg (19 Dec 2023) - Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 +- DEPRECATE.md: mention that NTLM_WB no longer works - Closes #12227 + Ref: #12479 + Closes #12553 -- build: require Windows XP or newer +- CURLOPT_SERVER_RESPONSE_TIMEOUT_MS: add - After this patch we assume availability of `getaddrinfo` and - `freeaddrinfo`, first introduced in Windows XP. Meaning curl - now requires building for Windows XP as a minimum. + Proposed-by: Yifei Kong + Ref: https://curl.se/mail/lib-2023-11/0023.html + Closes #12369 - TODO: assume these also in autotools. +Viktor Szakats (18 Dec 2023) - Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 - Closes #12225 +- build: more `-Wformat` fixes -- appveyor: bump one job to OpenSSL 3.1 (was 1.1.1) + - memdebug: update to not trigger `-Wformat-nonliteral` warnings. + - imap: mark `imap_sendf()` with `CURL_PRINTF()`. + - tool_msgs: mark static function with `CURL_PRINTF()`. - Use 3.1 with the modern runner image. + Follow-up to 3829759bd042c03225ae862062560f568ba1a231 #12489 - We still use 1.1.1 in 8 jobs. + Closes #12540 - 1.1.1 is EOL since 2023-09-11: - https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/ +- windows: delete redundant headers - Also: - - add missing SSL-backend to job descriptions. - - tidy up CPU in job descriptions. + `winsock2.h` pulls in `windows.h`. `ws2tcpip.h` pulls in `winsock2.h`. + `winsock2.h` and `ws2tcpip.h` are also pulled by `curl/curl.h`. - Closes #12226 + Keep only those headers that are not already included, or the code under + it uses something from that specific header. -Daniel Stenberg (30 Oct 2023) + Closes #12539 -- RELEASE-NOTES: synced +- cmake: prefill/cache `HAVE_STRUCT_SOCKADDR_STORAGE` -- GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions + Also add missing include to `OtherTests.cmake`. It didn't cause an issue + because the parent already included this earlier by chance. - ngtcp2 1.0.1 - nghttp3 1.0.0 - nghttp2 1.58.0 - quictls 3.1.4+quic + Closes #12537 - also sync HTTP3.md with these changes +Daniel Stenberg (18 Dec 2023) - Closes #12132 +- runner.pm: fix perl warning when running tests -Kareem (29 Oct 2023) + Use of uninitialized value $runner::gdbthis in numeric eq (==) at runner. + pm -- wolfssl: add default case for wolfssl_connect_step1 switch + Follow-up from 3dcf301752a09d9 - Closes #12218 + Closes #12549 -Jay Satiro (29 Oct 2023) +- runtests: support -gl. Like -g but for lldb. -- curl_setup: disallow Windows IPv6 builds missing getaddrinfo + Follow-up to 63b5748 - - On Windows if IPv6 is enabled but getaddrinfo is missing then #error - the build. + Invokes the test case via lldb instead of gdb. Since using gdb is such a + pain on mac, using lldb is sometimes less quirky. - curl can be built with IPv6 support (ENABLE_IPV6) but without the - ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On - Windows this is highly unlikely and should be considered a bad build - configuration. + Closes #12547 - Such a bad configuration has already given us a bug that was hard to - diagnose. See #12134 and #12136 for discussion. +- curl.h: add CURLE_TOO_LARGE - Ref: https://github.com/curl/curl/issues/12134 - Ref: https://github.com/curl/curl/pull/12136 + A new error code to be used when an internal field grows too large, like + when a dynbuf reaches its maximum. Previously it would return + CURLE_OUT_OF_MEMORY for this, which is highly misleading. - Closes https://github.com/curl/curl/pull/12221 + Ref: #12268 + Closes #12269 -Nico Rieck (29 Oct 2023) +- CI/circleci: disable MQTT in the HTTP-only build -- openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs + And remove the use of configure options that don't actually exist - - If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA - "CA" store after importing from root CA "ROOT" store. + Closes #12546 - This change allows curl to work in situations where a server does not - send all intermediate certs and they are present in the "CA" store (the - store with intermediate CAs). This is already allowed by the Schannel - backend. +Yedaya Katsman (18 Dec 2023) - Also this change makes partial chain verification possible for those - certs since we allow partial chain verification by default for OpenSSL - (unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel - backend. +- tests: respect $TMPDIR when creating unix domain sockets - Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs. + When running on termux, where $TMPDIR isn't /tmp, running the tests + failed, since the server config tried creating sockets in /tmp, without + checking the temp dir config. Use the TMPDIR variable that makes it find + the correct directory everywhere [0] - Fixes https://github.com/curl/curl/issues/12155 - Closes https://github.com/curl/curl/pull/12185 + [0] https://perldoc.perl.org/File::Temp#tempfile -Viktor Szakats (28 Oct 2023) + Closes #12545 -- Makefile.mk: fix `-rtmp` option for non-Windows [ci skip] +Viktor Szakats (17 Dec 2023) -Daniel Stenberg (28 Oct 2023) +- ssh: fix namespace of two local macros -- asyn-ares: handle no connection in the addrinfo callback + Avoid using the libssh and libssh2 macro namespaces by prefixing + these local macro names with `CURL_`. - To avoid crashing. + Follow-up to 413a0fedd02c8c6df1d294534b8c6e306fcca7a2 #12346 - Follow-up from 56a4db2 - Closes #12219 + Reviewed-by: Daniel Stenberg + Closes #12544 -Jay Satiro (28 Oct 2023) +- cmake: whitespace tidy-up in `OtherTests.cmake` -- hostip6: fix DEBUG_ADDRINFO builds + Closes #12538 - - Removed unused and incorrect parameter from dump_addrinfo(). +Mark Sinkovics (16 Dec 2023) - Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442 - Reported-by: Gisle Vanem +- cmake: fix generation for system name iOS - Closes https://github.com/curl/curl/pull/12212 + This PR fixes a problem that happens during CMake configuration when + the `CMAKE_SYSTEM_NAME` set to `iOS` and not `Darwin`. This value is + available (as far as I remember) version 3.14. The final solution + (thanks to @vszakats) is to use `APPLE` which contains all the Apple + platforms https://cmake.org/cmake/help/latest/variable/APPLE.html. -Viktor Szakats (28 Oct 2023) + This issue was found when during vcpkg installation. Running command + `vcpkg install curl:arm64-ios` and `vcpkg install curl:x64-ios` failed + with message: + ``` + CMake Error: try_run() invoked in cross-compiling mode, please set the follow + ing cache variables appropriately: + HAVE_H_ERRNO_ASSIGNABLE_EXITCODE (advanced) + ``` + After this fix, I was able to compile the compile the binary without + any issue. -- Makefile.mk: restore `_mingw.h` for default `_WIN32_WINNT` + In addition to that fix, this PR also contains an simplification to + check if the platform is not APPLE. - In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support. - Turns out `_mingw.h` had the side-effect of setting a default - `_WIN32_WINNT` value expected by `lib/config-win32.h` to enable - `getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused - disabling support for this unless specifying the value manually. + Co-authored-by: Viktor Szakats + Closes #12515 - Restore this header and update its comment to tell why we continue - to need it. +Daniel Stenberg (16 Dec 2023) - This triggered a regression in official Windows curl builds starting - with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.) +- RELEASE-NOTES: synced - Regression from 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 +Baruch Siach (16 Dec 2023) - Reported-by: zhengqwe on github - Helped-by: Nico Rieck - Fixes #12134 - Fixes #12136 - Closes #12217 +- gnutls: fix build with --disable-verbose -- hostip: silence compiler warning `-Wparentheses-equality` + infof() parameters must be defined event with --disable-verbose since + commit dac293cfb702 ("lib: apache style infof and trace + macros/functions"). - Seen with LLVM 17. + Move also 'ptr' definition under !CURL_DISABLE_VERBOSE_STRINGS. - ``` - hostip.c:1336:22: warning: equality comparison with extraneous parentheses [- - Wparentheses-equality] - 1336 | (a->ai_family == PF_INET)) { - | ~~~~~~~~~~~~~^~~~~~~~~~ - hostip.c:1336:22: note: remove extraneous parentheses around the comparison t - o silence this warning - 1336 | (a->ai_family == PF_INET)) { - | ~ ^ ~ - hostip.c:1336:22: note: use '=' to turn this equality comparison into an assi - gnment - 1336 | (a->ai_family == PF_INET)) { - | ^~ - | = - 1 warning generated. - ``` + Fixes the following build failure: - Follow-up to b651aba0962bb31353f55de4dc35f745952a1b10 #12145 + In file included from ../lib/sendf.h:29, + from vtls/gtls.c:44: + vtls/gtls.c: In function 'Curl_gtls_verifyserver': + vtls/gtls.c:841:34: error: 'version' undeclared (first use in this function); + did you mean 'session'? + 841 | gnutls_protocol_get_name(version), ptr); + | ^~~~~~~ - Reviewed-by: Daniel Stenberg - Closes #12215 + Closes #12505 -Stefan Eissing (27 Oct 2023) +Viktor Szakats (16 Dec 2023) -- doh: use PIPEWAIT when HTTP/2 is attempted +- build: delete unused `HAVE_{GSSHEIMDAL,GSSMIT,HEIMDAL}` - Closes #12214 + Stop setting `HAVE_GSSHEIMDAL`, `HAVE_GSSMIT` and `HAVE_HEIMDAL`. + There was no place in the build system or source code that used them. -Daniel Stenberg (27 Oct 2023) + Reviewed-by: Daniel Stenberg + Closes #12506 -- setopt: remove outdated cookie comment +- build: remove redundant `CURL_PULL_*` settings - Closes #12206 + These macros were not propagated to the source code from CMake. -Stefan Eissing (27 Oct 2023) + autotools set only one of them (`CURL_PULL_SYS_POLL_H`), initially to + address an AIX issue [1]. This later broke when introducing `system.h` + [2] without the logic it enabled. A subsequent fix [3] re-added the + logic, and also enabled it for AIX before its use, directly in + `system.h`. -- cfilter: provide call to tell connection to forget a socket + [1] 2012-11-23: 665adcd4b7bcdb7deb638cdc499fbe71f8d777f2 + [2] 2017-03-29: 9506d01ee50d5908138ebad0fd9fbd39b66bd64d #1373 + [3] 2017-08-25: 8a84fcc4b59e8b78d2acc6febf44a43d6bc81b59 #1828 #1833 - - fixed libssh.c workaround for a socket being closed by - the library - - eliminate the terrible hack in cf-socket.c to guess when - this happened and try not closing the socket again. - - fixes race in eyeballing when socket could have failed to - be closed for a discarded connect attempt + Reviewed-by: Daniel Stenberg + Closes #12502 - Closes #12207 +- system.h: sync mingw `CURL_TYPEOF_CURL_SOCKLEN_T` with other compilers -- url: protocol handler lookup tidy-up + Align mingw with the other Windows compilers and use the `int` type for + `CURL_TYPEOF_CURL_SOCKLEN_T` (and thus for `curl_socklent_t`). This + makes it unnecessary to make a mingw-specific trick and pull all Windows + headers early just for this type definition. This type is specific to + Windows, not to the compiler. mingw-w64's Windows header maps it to + `int` too. - - rename lookup to what it does - - use ARRAYSIZE instead of NULL check for end - - offer alternate lookup for 0-terminated strings + With this we also delete all remaining uses of `CURL_PULL_WS2TCPIP_H`. - Closes #12216 + [ The official solution is to use `socklen_t` for all Windows compilers. + In this case we may want to update `curl/curl.h` to pull in Windows + headers before `system.h`. ] -Viktor Szakats (27 Oct 2023) + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12501 + +- windows: simplify detecting and using system headers + + - autotools, cmake: assume that if we detect Windows, `windows.h`, + `winsock2.h` and `ws2tcpip.h` do exist. + - lib: fix 3 outlier `#if` conditions to use `USE_WINSOCK` instead of + looking for `winsock2.h`. + - autotools: merge 3 Windows check methods into one. + - move Watt-32 and lwIP socket support to `setup-win32.h` from + `config-win32.h`. It opens up using these with all build tools. Also + merge logic with Windows Sockets. + - fix to assume Windows sockets with the mingw32ce toolchain. + Follow-up to: 2748c64d605b19fb419ae56810ad8da36487a2d4 + - cmake: delete unused variable `signature_call_conv` since + eb33ccd5332435fa50f1758e5debb869c6942b7f. + - autotools: simplify `CURL_CHECK_WIN32_LARGEFILE` detection. + - examples/externalsocket: fix header order. + - cmake/OtherTests.cmake: delete Windows-specific `_source_epilogue` + that wasn't used anymore. + - cmake/OtherTests.cmake: set `WIN32_LEAN_AND_MEAN` for test + `SIZEOF_STRUCT_SOCKADDR_STORAGE`. + + After this patch curl universally uses `_WIN32` to guard + Windows-specific logic. It guards Windows Sockets-specific logic with + `USE_WINSOCK` (this might need further work). -- build: variadic macro tidy-ups + Reviewed-by: Jay Satiro + Closes #12495 + +- build: enable missing OpenSSF-recommended warnings, with fixes + + https://best.openssf.org/Compiler-Hardening-Guides/Compiler-Options-Hardening + -Guide-for-C-and-C++.html + as of 2023-11-29 [1]. + + Enable new recommended warnings (except `-Wsign-conversion`): + + - enable `-Wformat=2` for clang (in both cmake and autotools). + - add `CURL_PRINTF()` internal attribute and mark functions accepting + printf arguments with it. This is a copy of existing + `CURL_TEMP_PRINTF()` but using `__printf__` to make it compatible + with redefinting the `printf` symbol: + https://gcc.gnu.org/onlinedocs/gcc-3.0.4/gcc_5.html#SEC94 + - fix `CURL_PRINTF()` and existing `CURL_TEMP_PRINTF()` for + mingw-w64 and enable it on this platform. + - enable `-Wimplicit-fallthrough`. + - enable `-Wtrampolines`. + - add `-Wsign-conversion` commented with a FIXME. + - cmake: enable `-pedantic-errors` the way we do it with autotools. + Follow-up to d5c0351055d5709da8f3e16c91348092fdb481aa #2747 + - lib/curl_trc.h: use `CURL_FORMAT()`, this also fixes it to enable format + checks. Previously it was always disabled due to the internal `printf` + macro. - - delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks. - (both autotools and CMake.) - - delete duplicate `NULL` check in `Curl_trc_cf_infof()`. - - fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds. - ``` - ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parame - ter] - static void nosigpipe(struct Curl_easy *data, - ^ - ``` - - fix `#ifdef` comments in `lib/curl_trc.{c,h}`. - - fix indentation in some `infof()` calls. + Fix them: - Follow-up to dac293cfb7026b1ca4175d88b80f1432d3d3c684 #12167 + - fix bug where an `set_ipv6_v6only()` call was missed in builds with + `--disable-verbose` / `CURL_DISABLE_VERBOSE_STRINGS=ON`. + - add internal `FALLTHROUGH()` macro. + - replace obsolete fall-through comments with `FALLTHROUGH()`. + - fix fallthrough markups: Delete redundant ones (showing up as + warnings in most cases). Add missing ones. Fix indentation. + - silence `-Wformat-nonliteral` warnings with llvm/clang. + - fix one `-Wformat-nonliteral` warning. + - fix new `-Wformat` and `-Wformat-security` warnings. + - fix `CURL_FORMAT_SOCKET_T` value for mingw-w64. Also move its + definition to `lib/curl_setup.h` allowing use in `tests/server`. + - lib: fix two wrongly passed string arguments in log outputs. + Co-authored-by: Jay Satiro + - fix new `-Wformat` warnings on mingw-w64. - Cherry-picked from #12105 - Closes #12210 + [1] https://github.com/ossf/wg-best-practices-os-developers/blob/56c0fde3895b + fc55c8a973ef49a2572c507b2ae1/docs/Compiler-Hardening-Guides/Compiler-Options- + Hardening-Guide-for-C-and-C%2B%2B.md -- cmake: speed up threads setup for Windows + Closes #12489 - Win32 threads are always available. We enabled them unconditionally - (with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection - logic has this condition hard-coded for Windows as well (since at least - 2007). +- Makefile.mk: drop Windows support - Instead of doing all the work of detecting pthread combinations on - Windows, then discarding those results, skip these efforts and assume - built-in thread support when building for Windows. + And DLL-support with it. This leaves `Makefile.mk` for MS-DOS and Amiga. - This saves 1-3 slow CMake configuration steps. + We recommend CMake instead. With unity mode it's much faster, and about + the same without. + Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 Reviewed-by: Daniel Stenberg - Closes #12202 + Closes #12224 -- cmake: speed up zstd detection +Daniel Stenberg (16 Dec 2023) - Before this patch we detected the presence of a specific zstd API to - see if we can use the library. zstd published that API in its first - stable release: v1.0.0 (2016-08-31). +- cmdline-docs: use .IP consistently - Replace that method by detecting the zstd library version instead and - accepting if it's v1.0.0 or newer. Also display this detected version - and display a warning if the zstd found is unfit for curl. + Remove use of .TP and some .B. The idea is to reduce nroff syntax as + much as possible and to use it consistently. Ultimately, we should be + able to introduce our own easier-to-use-and-read syntax/formatting and + convert on generation time. - We use the same version detection method as zstd itself, via its public - C header. + Closes #12535 - This deviates from autotools which keeps using the slow method of - looking for the API by building a test program. The outcome is the same - as long as zstd keeps offering this API. +Tatsuhiko Miyagawa (16 Dec 2023) - Ref: https://github.com/facebook/zstd/commit/5a0c8e24395079f8e8cdc90aa1659cd5 - ab1b7427 (2016-08-12, committed) - Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first - released) - Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0 +- http: fix off-by-one error in request method length check - Reviewed-by: Daniel Stenberg - Closes #12200 + It should allow one more byte. -Daniel Stenberg (26 Oct 2023) + Closes #12534 -- openssl: fix infof() to avoid compiler warning for %s with null +Daniel Stenberg (15 Dec 2023) - vtls/openssl.c: In function ‘ossl_connect_step2’: - ../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror - =format-overflow=] - 120 | Curl_infof(data, __VA_ARGS__); } while(0) - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’ - 4008 | infof(data, "SSL connection using %s / %s / %s / %s", - | ^~~~~ - vtls/openssl.c:4008:49: note: format string is defined here - 4008 | infof(data, "SSL connection using %s / %s / %s / %s", - | ^~ +- curl: show ipfs and ipns as supported "protocols" - Follow-up to b6e6d4ff8f253c8b8055bab - Closes #12196 + They are accepted schemes in URLs passed to curl (the tool, not the + library). -Stefan Eissing (26 Oct 2023) + Also makes curl-config show the same list. -- lib: apache style infof and trace macros/functions + Co-Authored-by: Jay Satiro + Reported-by: Chara White + Bug: https://curl.se/mail/archive-2023-12/0026.html + Closes #12508 - - test for a simplified C99 variadic check - - args to infof() in --disable-verbose are no longer disregarded but - must compile. +- Revert "urldata: move async resolver state from easy handle to connectdata" - Closes #12167 - Fixes #12083 - Fixes #11880 - Fixes #11891 + This reverts commit 56a4db2e4e2bcb9a0dcb75b83560a78ef231fcc8 (#12198) -Daniel Stenberg (26 Oct 2023) + We want the c-ares channel to be held in the easy handle, not per + connection - for performance. -- RELEASE-NOTES: synced + Closes #12524 -Stefan Eissing (26 Oct 2023) +Viktor Szakats (15 Dec 2023) -- urldata: move async resolver state from easy handle to connectdata +- openssl: re-match LibreSSL deinit with init - - resolving is done for a connection, not for every transfer - - save create/dup/free of a cares channel for each transfer - - check values of setopt calls against a local channel if no - connection has been attached yet, when needed. + Earlier we switched to use modern initialization with LibreSSL v2.7.0 + and up, but did not touch deinitialization [1]. Fix it in this patch. - Closes #12198 + Regression from bec0c5bbf34369920598678161d2df8bea0e243b #11611 -Daniel Stenberg (26 Oct 2023) + [1] https://github.com/curl/curl/pull/11611#issuecomment-1668654014 -- CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERRO - R + Reported-by: Mike Hommey + Reviewed-by: Daniel Stenberg + Fixes #12525 + Closes #12526 - It returns CURLE_WRITE_ERROR. It was not previously stated clearly. +Daniel Stenberg (14 Dec 2023) - Reported-by: enWILLYado on github - Fixes #12201 - Closes #12203 +- libssh: supress warnings without version check -Viktor Szakats (25 Oct 2023) + Define unconditionally. -- autotools: update references to deleted `crypt-auth` option + Follow-up from d21bd2190c46ad7fa - Delete leftovers of the `crypt-auth` `./configure` option and - add the new ones that replaced them. + Closes #12523 - Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 +- hostip: return error immediately when Curl_ip2addr() fails - Reviewed-by: Daniel Stenberg - Closes #12194 + Closes #12522 -Stefan Eissing (25 Oct 2023) +Theo (14 Dec 2023) -- lib: introduce struct easy_poll_set for poll information +- libssh: improve the deprecation warning dismissal - Connection filter had a `get_select_socks()` method, inspired by the - various `getsocks` functions involved during the lifetime of a - transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), - return sockets to monitor and flag if this shall be done for POLLIN - and/or POLLOUT. + Previous code was compiler dependant, and dismissed all deprecation warnings + indiscriminately. - Due to this design, sockets and flags could only be added, not - removed. This led to problems in filters like HTTP/2 where flow control - prohibits the sending of data until the peer increases the flow - window. The general transfer loop wants to write, adds POLLOUT, the - socket is writeable but no data can be written. + libssh provides a way to disable the deprecation warnings for libssh only, an + d + naturally this is the preferred way. - This leads to cpu busy loops. To prevent that, HTTP/2 did set the - `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes - further attempts. This works if only one such filter is involved. If a - HTTP/2 transfer goes through a HTTP/2 proxy, two filters are - setting/clearing this flag and may step on each other's toes. + This commit uses that, to prevent the erroneous hiding of potential, unrelate + d + deprecation warnings. - Connection filters `get_select_socks()` is replaced by - `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps - up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` - flags. This struct is initialized in `multi_getsock()` by calling the - various `getsocks()` implementations based on transfer state, as before. + Fixes #12519 + Closes #12520 - After protocol handlers/transfer loop have set the sockets and flags - they want, the `easy_pollset` is *always* passed to the filters. Filters - "higher" in the chain are called first, starting at the first - not-yet-connection one. Each filter may add sockets and/or change - flags. When all flags are removed, the socket itself is removed from the - pollset. +Daniel Stenberg (14 Dec 2023) - Example: +- test1474: removed - * transfer wants to send, adds POLLOUT - * http/2 filter has a flow control block, removes POLLOUT and adds - POLLIN (it is waiting on a WINDOW_UPDATE from the server) - * TLS filter is connected and changes nothing - * h2-proxy filter also has a flow control block on its tunnel stream, - removes POLLOUT and adds POLLIN also. - * socket filter is connected and changes nothing - * The resulting pollset is then mixed together with all other transfers - and their pollsets, just as before. + The test was already somewhat flaky and disabled on several platforms, + and after 1da640abb688 even more unstable. - Use of `SEND_HOLD` is no longer necessary in the filters. +- readwrite_data: loop less - All filters are adapted for the changed method. The handling in - `multi.c` has been adjusted, but its state handling the the protocol - handlers' `getsocks` method are untouched. + This function is made to loop in order to drain incoming data + faster. Completely removing the loop has a measerably negative impact on + transfer speeds. - The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS - filters needed to be adjusted for the connecting handshake read/write - handling. + Downsides with the looping include - No noticeable difference in performance was detected in local scorecard - runs. + - it might call the progress callback much more seldom. Especially if + the write callback is slow. - Closes #11833 + - rate limiting becomes less exact -Daniel Stenberg (25 Oct 2023) + - a single transfer might "starve out" other parallel transfers -- tests/README: SOCKS tests are not using OpenSSH, it has its own server + - QUIC timers for other connections can't be maintained correctly - Follow-up to 04fd67555cc + The long term fix should be to remove the loop and optimize coming back + to avoid the transfer speed penalty. - Closes #12195 + This fix lower the max loop count to reduce the starvation problem, and + avoids the loop completely for when rate-limiting is in progress. -Jacob Hoffman-Andrews (25 Oct 2023) + Ref: #12488 + Ref: https://curl.se/mail/lib-2023-12/0012.html + Closes #12504 -- tets: make test documentation more user-friendly +Stefan Eissing (14 Dec 2023) - Put the instructions to run tests right at the top of tests/README.md. +- lib: eliminate `conn->cselect_bits` - Give instructions to read the runtests.1 man page for information - about flags. Delete redundant copy of the flags documentation in the - README. + - use `data->state.dselect_bits` everywhere instead + - remove `bool *comeback` parameter as non-zero + `data->state.dselect_bits` will indicate that IO is + incomplete. - Add a mention in README.md of the important parallelism flag, to make - test runs go much faster. + Closes #12512 - Move documentation of output line format into the runtests.1 man page, - and update it with missing flags. +- connect: refactor `Curl_timeleft()` - Fix the order of two flags in the man page. + - less local vars, "better" readability + - added documentation - Closes #12193 + Closes #12518 -Viktor Szakats (24 Oct 2023) +Dmitry Karpov (14 Dec 2023) -- cmake: pre-fill rest of detection values for Windows +- cookie: avoid fopen with empty file name - The goal of this patch is to avoid unnecessary feature detection work - when doing Windows builds with CMake. Do this by pre-filling well-known - detection results for Windows and specifically for mingw-w64 and MSVC - compilers. Also limit feature checks to platforms where the results are - actually used. Drop a few redundant ones. And some tidying up. + Closes #12514 - - pre-fill remaining detection values in Windows CMake builds. +Viktor Szakats (13 Dec 2023) - Based on actual detection results observed in CI runs, preceding - similar work over libssh2 and matching up values with - `lib/config-win32.h`. +- tests/server: delete workaround for old-mingw - This brings down CMake configuration time from 58 to 14 seconds on the - same local machine. + mingw-w64 1.0 comes with w32api v3.12, thus doesn't need this. - On AppVeyor CI this translates to: - - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job): - https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecr - jpy7necb#L296 - https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2 - fe249uo8#L186 - - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job): - https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5iv - lcs7ub29?fullLog=true#L290 - https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjs - yc9kl13a?fullLog=true#L194 + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 - The formula is about 1-3 seconds delay for each detection. Almost all - of these trigger a full compile-link cycle behind the scenes, slow - even today, both cross and native, mingw-w64 and apparently MSVC too. - Enabling .map files or other custom build features slows it down - further. (Similar is expected for autotools configure.) + Reviewed-by: Jay Satiro + Closes #12510 - - stop detecting `idn2.h` if idn2 was deselected. - autotools does this. +- cmake: delete obsolete TODOs more [ci skip] - - stop detecting `idn2.h` if idn2 was not found. - This deviates from autotools. Source code requires both header and - lib, so this is still correct, but faster. + - manual completed: 898b012a9bf388590c4be7f526815b5ab74feca1 #1288 + - soname completed: 5de6848f104d7cb0017080e31216265ac19d0dde #10023 + - bunch of others that are completed + - `NTLM_WB_ENABLED` is implemented in a basic form, and now also + scheduled for removal, so a TODO at this point isn't useful. - - limit `ADDRESS_FAMILY` detection to Windows. + And this 'to-check' item: - - normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format. + Q: "The cmake build selected to run gcc with -fPIC on my box while the + plain configure script did not." - - pre-fill `HAVE_WIN32_WINNT`-dependent detection results. - Saving 4 (slow) feature-detections in most builds: `getaddrinfo`, - `freeaddrinfo`, `inet_ntop`, `inet_pton` + A: With CMake, since 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 + and fc9bfb14520712672b4784e8b48256fb29204011 #11627, we explicitly + enable PIC for libcurl shared lib. Or when building libcurl for + shared and static lib in a single pass. We do this by default for + Windows or when enabled by the user via `SHARE_LIB_OBJECT`. + Otherwise we don't touch this setting. Meaning the default set by + CMake (if any) or the toolchain is used. On Debian Bookworm, this + means that PIC is disabled for static libs by default. Some platforms + (like macOS), has PIC enabled by default. + autotools supports the double-pass mode only, and in that case + CMake seems to match PIC behaviour now (as tested on Linux with gcc.) - - fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`, - `HAVE_GETTIMEOFDAY` for mingw-w64. - Luckily this do not change build results, as `WIN32` took - priority over `HAVE_GETTIMEOFDAY` with the current source - code. + Follow-up to 5d5dfdbd1a6c40bd75e982b66f49e1fa3a7eeae7 #12500 - - limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and - `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows. - We're not using these in the source code for Windows. + Reviewed-by: Jay Satiro + Closes #12509 - - reduce compiler warning noise in CMake internal logs: - - fix to include `winsock2.h` before `windows.h`. - Apply it to autotools test snippets too. - - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above. - - cleanup `CMake/CurlTests.c` to emit less warnings. +Stefan Eissing (12 Dec 2023) - - delete redundant `HAVE_MACRO_SIGSETJMP` feature check. - It was the same check as `HAVE_SIGSETJMP`. +- CLIENT-WRITERS: design and use documentation - - delete 'experimental' marking from `CURL_USE_OPENSSL`. + Closes #12507 - - show CMake version via `CMakeLists.txt`. - Credit to the `zlib-ng` project for the idea: - https://github.com/zlib-ng/zlib-ng/blob/61e181c8ae93dbf56040336179c9954078b - d1399/CMakeLists.txt#L7 +Viktor Szakats (12 Dec 2023) - - make `CMake/CurlTests.c` pass `checksrc`. +- cmake: delete obsolete TODO items [ci skip] - - `CMake/WindowsCache.cmake` tidy-ups. + There is always room for improvement, but CMake is up to par now with + autotools, so there is no longer a good reason to keep around these + inline TODO items. - - replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`. + Answering one of questions: - Closes #12044 + Q: "The gcc command line use neither -g nor any -O options. As a + developer, I also treasure our configure scripts's --enable-debug + option that sets a long range of "picky" compiler options." -Jay Satiro (24 Oct 2023) + A: CMake offers the `CMAKE_BUILD_TYPE` variable to control debug info + and optimization level. E.g.: + - `Release` = `-O3` + no debug info + - `MinSizeRel` = `-Os` + no debug info + - `Debug` = `-O0` + debug info -- page-footer: clarify exit code 25 + https://stackoverflow.com/questions/48754619/what-are-cmake-build-type-deb + ug-release-relwithdebinfo-and-minsizerel/59314670#59314670 + https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#defaul + t-and-custom-configurations - - Clarify that curl tool exit code 25 means an upload failed to start. + For picky warnings we have the `PICKY_COMPILER` options, enabled by + default. - Exit code 25 is equivalent to CURLE_UPLOAD_FAILED (25). Prior to this - change the documentation only mentioned the case of FTP STOR failing. + Closes #12500 - Reported-by: Emanuele Torre +Stefan Eissing (11 Dec 2023) - Ref: https://github.com/curl/curl/blob/curl-8_4_0/docs/libcurl/libcurl-errors - .3#L113-L115 +- CONNECTION-FILTERS: update documentation - Fixes https://github.com/curl/curl/issues/12189 - Closes https://github.com/curl/curl/pull/12190 + Closes #12497 -Daniel Stenberg (24 Oct 2023) +Daniel Stenberg (11 Dec 2023) -- scripts/cijobs.pl: adjust for appveyor +- lib: reduce use of strncpy - Follow-up to a1d73a6bb + - bearssl: select cipher without buffer copies + - http_aws_sigv4: avoid strncpy, require exact timestamp length + - http_aws_sigv4: use memcpy isntead of strncpy + - openssl: avoid strncpy calls + - schannel: check for 1.3 algos without buffer copies + - strerror: avoid strncpy calls + - telnet: avoid strncpy, return error on too long inputs + - vtls: avoid strncpy in multissl_version() -Alex Bozarth (24 Oct 2023) + Closes #12499 -- OpenSSL: Include SIG and KEM algorithms in verbose +- CI/distcheck: run full tests - Currently the verbose output does not include which algorithms are used - for the signature and key exchange when using OpenSSL. Including the - algorithms used will enable better debugging when working on using new - algorithm implementations. Know what algorithms are used has become more - important with the fast growing research into new quantum-safe - algorithms. + To be able to detect missing files better, this now runs the full CI + test suite. If done before, it would have detected #12462 before + release. - This implementation includes a build time check for the OpenSSL version - to use a new function that will be included in OpenSSL 3.2 that was - introduced in openssl/openssl@6866824 + Closes #12503 - Based-on-patch-by: Martin Schmatz - Closes #12030 +- docs: clean up Protocols: for cmdline options -Daniel Stenberg (23 Oct 2023) + ... and some other minor polish. -- http2: provide an error callback and failf the message + Closes #12496 - Getting nghttp2's error message helps users understand what's going - on. For example when the connection is brought down due a forbidden - header is used - as that header is then not displayed by curl itself. +- cmdline/gen: fix the sorting of the man page options - Example: + They were previously sorted based on the file names, which use a .d + extension, making "data" get placed after "data-binary" etc. Making the + sort ignore the extention fixes the ordering. - curl: (92) Invalid HTTP header field was received: frame type: 1, - stream: 1, name: [upgrade], value: [h2,h2c] + Reported-by: Boris Verkhovskiy + Bug: https://curl.se/mail/archive-2023-12/0014.html + Closes #12494 - Ref: #12172 - Closes #12179 +Daniel Gustafsson (9 Dec 2023) -Turiiya (23 Oct 2023) +- doh: remove unused local variable -- BINDINGS: add V binding + The nurl variable is no longer used during probing following + a refactoring, so remove. - Closes #12182 + Closes #12491 -Daniel Stenberg (22 Oct 2023) +Jay Satiro (8 Dec 2023) -- configure: check for the fseeko declaration too +- build: fix Windows ADDRESS_FAMILY detection - ... and make the code require both symbol and declaration. + - Include winsock2.h for Windows ADDRESS_FAMILY detection. - This is because for Android, the symbol is always present in the lib at - build-time even when not actually available in run-time. + Prior to this change cmake detection didn't work because it included + ws2def.h by itself, which is missing needed types from winsock2.h. - Assisted-by: Viktor Szakats - Reported-by: 12932 on github - Fixes #12086 - Closes #12158 - -Viktor Szakats (22 Oct 2023) + Prior to this change autotools detection didn't work because it did not + include any Windows header. -- cmake: fix OpenSSL quic detection in quiche builds - - An orphan call to `CheckQuicSupportInOpenSSL()` remained after a recent - update when checking QUIC for quiche. Move back QUIC detection to - a function and fixup callers to use that. Also make sure that quiche - gets QUIC from BoringSSL, because it doesn't support other forks at this - time. + In both cases libcurl would fall back on unsigned short as the address + family type, which is the same as ADDRESS_FAMILY. - Regression from dee310d54261f9a8416e87d50bccfe2cbe404949 #11555 + Co-authored-by: Viktor Szakats - Reported-by: Casey Bodley - Fixes #12160 - Closes #12162 + Closes https://github.com/curl/curl/pull/12441 -Daniel Stenberg (22 Oct 2023) +Daniel Stenberg (8 Dec 2023) -- RELEASE-NOTES: synced +- lib: rename Curl_strndup to Curl_memdup0 to avoid misunderstanding - bump to 8.5.0 for pending release + Since the copy does not stop at a null byte, let's not call it anything + that makes you think it works like the common strndup() function. -Dan Fandrich (21 Oct 2023) + Based on feedback from Jay Satiro, Stefan Eissing and Patrick Monnerat -- test3103: add missing quotes around a test tag attribute + Closes #12490 -Loïc Yhuel (21 Oct 2023) +- convsrctest.pl: removed: not used, not shipped in tarballs -- tool: fix --capath when proxy support is disabled +- tests: rename tests scripts to the test number - After 95e8515ca0, --capath always sets CURLOPT_PROXY_CAPATH, which fails - with CURLE_UNKNOWN_OPTION when proxy support is disabled. + It is hard to name the scripts sensibly. Lots of them are similarly + named and the name did not tell which test that used them. - Closes #12089 + The new approach is rather to name them based on the test number that + runs them. Also helps us see which scripts are for individual tests + rather than for general test infra. -Daniel Stenberg (21 Oct 2023) + - badsymbols.pl -> test1167.pl + - check-deprecated.pl -> test1222.pl + - check-translatable-options.pl -> test1544.pl + - disable-scan.pl -> test1165.pl + - error-codes.pl -> test1175.pl + - errorcodes.pl -> test1477.pl + - extern-scan.pl -> test1135.pl + - manpage-scan.pl -> test1139.pl + - manpage-syntax.pl -> test1173.pl + - markdown-uppercase.pl -> test1275.pl + - mem-include-scan.pl -> test1132.pl + - nroff-scan.pl -> test1140.pl + - option-check.pl -> test1276.pl + - options-scan.pl -> test971.pl + - symbol-scan.pl -> test1119.pl + - version-scan.pl -> test1177.pl -- openldap: move the alloc of ldapconninfo to *connect() + Closes #12487 - Fixes a minor memory leak on LDAP connection reuse. +Michał Antoniak (8 Dec 2023) - Doing the allocation already in *setup_connection() is wrong since that - connect struct might get discarded early when an existing connection is - reused instead. +- sendf: fix compiler warning with CURL_DISABLE_HEADERS_API - Closes #12166 + fix MSVC warning C4189: 'htype': local variable is initialized but not + referenced - when CURL_DISABLE_HEADERS_API is defined. -- openldap: set the callback argument in oldap_do + Closes #12485 - ... to make sure it has the current 'data' pointer and not a stale old - one. +Viktor Szakats (8 Dec 2023) - Reported-by: Dan Fandrich - Closes #12166 +- tidy-up: whitespace -- gnutls: support CURLSSLOPT_NATIVE_CA + Closes #12484 - Remove the CURL_CA_FALLBACK logic. That build option was added to allow - primarily OpenSSL to use the default paths for loading the CA certs. For - GnuTLS it was instead made to load the "system certs", which is - different and not desirable. +Stefan Eissing (7 Dec 2023) - The native CA store loading is now asked for with this option. +- test_02_download: fix paramters to test_02_27 - Follow-up to 7b55279d1d856 + - it is a special client that only ever uses http/2 - Co-authored-by: Jay Satiro + Closes #12467 - Closes #12137 +Michał Antoniak (7 Dec 2023) -Stefan Eissing (21 Oct 2023) +- vtls: remove the Curl_cft_ssl_proxy object if CURL_DISABLE_PROXY -- RTSP: improved RTP parser + Closes #12459 - - fix HTTP header parsing to report incomplete - lines it buffers as consumed! - - re-implement the RTP parser for interleave RTP - messages for robustness. It is now keeping its - state at the connection - - RTSP protocol handler "readwrite" implementation - now tracks if the response is before/in/after - header parsing or "in" a bod by calling - "Curl_http_readwrite_headers()" itself. This - allows it to know when non-RTP bytes are "junk" - or HEADER or BODY. - - tested with #12035 and various small receive - sizes where current master fails +Daniel Stenberg (7 Dec 2023) - Closes #12052 +- lib: strndup/memdup instead of malloc, memcpy and null-terminate -- http2: header conversion tightening + - bufref: use strndup + - cookie: use strndup + - formdata: use strndup + - ftp: use strndup + - gtls: use aprintf instead of malloc + strcpy * 2 + - http: use strndup + - mbedtls: use strndup + - md4: use memdup + - ntlm: use memdup + - ntlm_sspi: use strndup + - pingpong: use memdup + - rtsp: use strndup instead of malloc, memcpy and null-terminate + - sectransp: use strndup + - socks_gssapi.c: use memdup + - vtls: use dynbuf instead of malloc, snprintf and memcpy + - vtls: use strdup instead of malloc + memcpy + - wolfssh: use strndup - - fold the code to convert dynhds to the nghttp2 structs - into a dynhds internal method - - saves code duplication - - pacifies compiler analyzers + Closes #12453 - Closes #12097 +- strdup: remove the memchr check from Curl_strndup -Daniel Stenberg (21 Oct 2023) + It makes it possible to clone a binary chunk of data. -- curl_ntlm_wb: fix elif typo + Closes #12453 - Reported-by: Manfred Schwarb - Follow-up to d4314cdf65ae - Bug: https://github.com/curl/curl/commit/d4314cdf65aee295db627016934bd9eb621a - b077#r130551295 +- ftp: handle the PORT parsing without allocation -Dan Fandrich (20 Oct 2023) + Also reduces amount of *cpy() calls. -- test1683: remove commented-out check alternatives + Closes #12456 - Python precheck/postcheck alternatives were included but commented out. - Since these are not used and perl is guaranteed to be available to run - the perl versions anyway, the Python ones are removed. +- RELEASE-NOTES: synced -Daniel Stenberg (20 Oct 2023) + Bumped to 8.5.1 -- hostip: show the list of IPs when resolving is done +- url: for disabled protocols, mention if found in redirect - Getting 'curl.se' today then gets this verbose output which might help - debugging connectivity related matters. + To help users better understand where the URL (and denied scheme) comes + from. Also removed "in libcurl" from the message, since the disabling + can be done by the application. - * Host curl.se:80 was resolved. - * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, - 2a04:4e42:600::347, 2a04:4e42:800::347, 2a04:4e42:a00::347, - 2a04:4e42:c00::347, 2a04:4e42:e00::347 - * IPv4: 151.101.193.91, 151.101.1.91, 151.101.65.91, 151.101.129.91 + The error message now says "not supported" or "disabled" depending on + why it was denied: - Co-authored-by: Jay Satiro - Closes #12145 + Protocol "hej" not supported + Protocol "http" disabled -rilysh (20 Oct 2023) + And in redirects: -- docs: fix function typo in curl_easy_option_next.3 + Protocol "hej" not supported (in redirect) + Protocol "http" disabled (in redirect) - Closes #12170 + Reported-by: Mauricio Scheffer + Fixes #12465 + Closes #12469 -Daniel Stenberg (20 Oct 2023) +Stefan Eissing (6 Dec 2023) -- vssh: remove the #ifdef for Curl_ssh_init, use empty macro +- sectransp_ make TLSCipherNameForNumber() available in non-verbose config - In the same style as other init calls + Reported-by: Cajus Pollmeier + Closes #12476 + Fixes #12474 -- easy: remove duplicate wolfSSH init call +YX Hao (6 Dec 2023) - It is already done in Curl_ssh_init() where it belongs. +- lib: fix variable undeclared error caused by `infof` changes - Closes #12168 + `--disable-verbose` yields `CURL_DISABLE_VERBOSE_STRINGS` defined. + `infof` isn't `Curl_nop_stmt` anymore: dac293c. -- socks: make SOCKS5 use the CURLOPT_IPRESOLVE choice + Follow-up to dac293c - Fixes #11949 - Reported-by: Ammar Faizi - Closes #12163 + Closes #12470 -- urldata: move the 'internal' boolean to the state struct +Viktor Szakats (6 Dec 2023) - ... where all the other state bits for the easy handles live. +- tidy-up: fix yamllint whitespace issues in labeler.yml - Closes #12165 + Follow-up to bda212911457c6fadfbba50be61afc4ca513fa56 #12466 -- url: don't touch the multi handle when closing internal handles + Reviewed-by: Dan Fandrich + Closes #12475 - Reported-by: Maksymilian Arciemowicz - Closes #12165 +- tidy-up: fix yamllint whitespace issues -Faraz Fallahi (19 Oct 2023) + Closes #12466 -- getenv: PlayStation doesn't have getenv() +Chris Sauer (6 Dec 2023) - Closes #12140 +- cmake: fix typo -Daniel Stenberg (19 Oct 2023) + Follow-up to aace27b + Closes #12464 -- transfer: only reset the FTP wildcard engine in CLEAR state +Daniel Stenberg (6 Dec 2023) - To avoid the state machine to start over and redownload all the files - *again*. +- dist: add tests/errorcodes.pl to the tarball - Reported-by: lkordos on github - Regression from 843b3baa3e3cb228 (shipped in 8.1.0) - Bisect-by: Dan Fandrich - Fixes #11775 - Closes #12156 + Used by test 1477 -Stefan Eissing (19 Oct 2023) + Reported-by: Xi Ruoyao + Follow-up to 0ca3a4ec9a7 + Fixes #12462 + Closes #12463 -- GHA: move mod_h2 version in CI to v2.0.25 +Dan Fandrich (6 Dec 2023) - Closes #12157 +- github/labeler: update a missed key in the v5 upgrade -Daniel Stenberg (19 Oct 2023) + Follow-up to ce03fe3ba -- ntlm_wb: use pipe instead of socketpair when possible +Version 8.5.0 (6 Dec 2023) - Closes #12149 +Daniel Stenberg (6 Dec 2023) - RELEASE-NOTES: synced -- asyn-thread: use pipe instead of socketpair for IPC when available - - If pipe() is present. Less overhead. - - Helped-by: Viktor Szakats - Closes #12146 + The curl 8.5.0 release. -Dan Fandrich (17 Oct 2023) +Dan Fandrich (5 Dec 2023) -- tests: Fix Windows test helper tool search & use it for handle64 +- github/labeler: switch from the beta to labeler v5 - The checkcmd() and checktestcmd() functions would not have worked on - Windows due to hard-coding the UNIX PATH separator character and not - adding .exe file extension. This meant that tools like stunnel, valgrind - and nghttpx would not have been found and used on Windows, and - inspection of previous test runs show none of those being found in pure - Windows CI builds. + Some keys were renamed and the dot option was made default. - With this fixed, they can be used to detect the handle64.exe program - before attempting to use it. When handle64.exe was called - unconditionally without it existing, it caused perl to abort the test - run with the error + Closes #12458 - The running command stopped because the preference variable - "ErrorActionPreference" or common parameter is set to Stop: - sh: handle64.exe: command not found +Daniel Stenberg (5 Dec 2023) - Closes #12115 +- DEPRECATE: remove NTLM_WB in June 2024 -Daniel Stenberg (17 Oct 2023) + Ref: https://curl.se/mail/lib-2023-12/0010.html -- multi: use pipe instead of socketpair to *wakeup() + Closes #12451 - If pipe() is present. Less overhead. +Jacob Hoffman-Andrews (4 Dec 2023) - Closes #12142 +- rustls: implement connect_blocking -Jay Satiro (17 Oct 2023) + Closes #11647 -- build: fix 'threadsafe' feature detection for older gcc +Daniel Stenberg (4 Dec 2023) - - Add 'threadsafe' to the feature list shown during build if POSIX - threads are being used. +- examples/rtsp-options.c: add - This is a follow-up to 5adb6000 which added support for building a - thread-safe libcurl with older versions of gcc where atomic is not - available but pthread is. + Just a bare bones RTSP example using CURLOPT_RTSP_SESSION_ID and + CURLOPT_RTSP_REQUEST set to CURL_RTSPREQ_OPTIONS. - Reported-by: Dan Fandrich - Co-authored-by: Dan Fandrich + Closes #12452 - Fixes https://github.com/curl/curl/issues/12125 - Closes https://github.com/curl/curl/pull/12127 +Stefan Eissing (4 Dec 2023) -Daniel Stenberg (16 Oct 2023) +- ngtcp2: ignore errors on unknown streams -- test729: verify socks4a with excessive proxy user name length + - expecially in is_alive checks on connections, we might + see incoming packets on streams already forgotten and closed, + leading to errors reported by nghttp3. Ignore those. -- socks: better buffer size checks for socks4a user and hostname + Closes #12449 - Also limit the proxy user name to 255 bytes, which is the same limit as - in SOCKS5. +Daniel Stenberg (4 Dec 2023) - Reported-by: sd0 on hackerone - Closes #12139 +- docs: make all examples in all libcurl man pages compile -- curl.h: on FreeBSD include sys/param.h instead of osreldate.h + Closes #12448 - Should things build on Playstation as well +- checksrc.pl: support #line instructions - Fixes #12107 - Reported-by: Faraz Fallahi - Closes #12123 + makes it identify the correct source file and line -Marcin Rataj (16 Oct 2023) +- GHA/man-examples: verify libcurl man page examples -- tool_operate: fix links in ipfs errors +- verify-examples.pl: verify that all man page examples compile clean - URL fragment links generated from headers in - https://curl.se/docs/ipfs.html are lowercase. +- RELEASE-NOTES: synced - Closes #12133 +Graham Campbell (2 Dec 2023) -Viktor Szakats (15 Oct 2023) +- http3: bump ngtcp2 and nghttp3 versions -- cmake: replace `check_library_exists_concat()` + nghttp3 v1.1.0 + ngtcp2 v1.1.0 - The idea of `check_library_exists_concat()` is that it detects an - optional component and adds it to the list of libs that we also use in - subsequent component checks. This caused problems when detecting - components with unnecessary dependencies that were not yet built. + In docs and CI - CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used - for component checks, which we already use in most cases. That left 4 - uses of `check_library_exists_concat()`. Only one of these actually - needed the 'concat' feature (ldap/lber). + Closes #12446 - Delete this function and replace it with standard - `check_library_exists()` and manual management of our `CURL_LIBS` - list we use when linking build targets. And special logic to handle the - ldap/lber case. +- CI/quiche: use `3.1.4+quic` consistently in CI workflows - (We have a similar function for headers: `check_include_file_concat()`. - It works, but problematic for performance reasons and because it hides - the actual headers required in `check_symbol_exists()` calls.) + Closes #12447 - Ref: #11537 #11558 - Fixes #11285 - Fixes #11648 - Closes #12070 +Viktor Szakats (2 Dec 2023) -LoRd_MuldeR (15 Oct 2023) +- test1545: disable deprecation warnings -- tool_cb_wrt: fix write output for very old Windows versions + Fixes: + https://ci.appveyor.com/project/curlorg/curl/builds/48631551/job/bhx74e0i66yr + p6pk#L1205 - - Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW() - function. + Same with details: + https://ci.appveyor.com/project/curlorg/curl/builds/48662893/job/ol8a78q9gmil + b6wt#L1263 + ``` + tests/libtest/lib1545.c:38:3: error: 'curl_formadd' is deprecated: since 7.56 + .0. Use curl_mime_init() [-Werror=deprecated-declarations] + 38 | curl_formadd(&m_formpost, &lastptr, CURLFORM_COPYNAME, "file", + | ^~~~~~~~~~~~ + [...] + ``` - Apparently this parameter was *not* optional on older Windows versions. + Follow-up to 07a3cd83e0456ca17dfd8c3104af7cf45b7a1ff5 #12421 - Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1. - So at some point between those two Microsoft changed the behavior. + Fixes #12445 + Closes #12444 - Prior to this change, on those versions if parameter is NULL then the - function call fails with error ERROR_INVALID_ACCESS. +Daniel Stenberg (2 Dec 2023) - Regression since af3f4e41. +- INSTALL: update list of ports and CPU archs - Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299 +- symbols-in-versions: the CLOSEPOLICY options are deprecated - Fixes https://github.com/curl/curl/issues/12131 - Closes https://github.com/curl/curl/pull/12130 + The were used with the CURLOPT_CLOSEPOLICY option, which *never* worked. -Jay Satiro (15 Oct 2023) +z2_ (1 Dec 2023) -- tool_urlglob: fix build for old gcc versions +- build: fix builds that disable protocols but not digest auth - - Don't use __builtin_mul_overflow for GCC 4 and earlier. + - Build base64 functions if digest auth is not disabled. - The function was added in GCC 5. + Prior to this change if some protocols were disabled but not digest auth + then a build error would occur due to missing base64 functions. - Ref: https://gcc.gnu.org/gcc-5/changes.html + Fixes https://github.com/curl/curl/issues/12440 + Closes https://github.com/curl/curl/pull/12442 - Reported-by: Dan Fandrich +Michał Antoniak (1 Dec 2023) - Fixes https://github.com/curl/curl/issues/12124 - Closes https://github.com/curl/curl/pull/12128 +- connect: reduce number of transportation providers -Carlos Henrique Lima Melara (14 Oct 2023) + Use only the ones necessary - the ones that are built-in. Saves a few + bytes in the resulting code. -- docs/libcurl: fix three minor man page format mistakes + Closes #12438 - Reported-by: Samuel Henrique +David Benjamin (1 Dec 2023) - Closes https://github.com/curl/curl/pull/12126 +- vtls: consistently use typedef names for OpenSSL structs -Jay Satiro (14 Oct 2023) + The foo_st names don't appear in OpenSSL public API documentation. The + FOO typedefs are more common. This header was already referencing + SSL_CTX via . There is a comment about avoiding + , but OpenSSL actually declares all the typedefs in + , which is already included by (and + every other OpenSSL header), so just use that. Though I've included it + just to be explicit. -- tests/server: add more SOCKS5 handshake error checking + (I'm also fairly sure including already triggers the + Schannel conflicts anyway. The comment was probably just out of date.) - - Add additional checking for missing and too-short SOCKS5 handshake - messages. + Closes #12439 - Prior to this change the SOCKS5 test server did not check that all parts - of the handshake were received successfully. If those parts were missing - or too short then the server would access uninitialized memory. +Lau (1 Dec 2023) - This issue was discovered in CI job 'memory-sanitizer' test results. - Test 2055 was failing due to the SOCKS5 test server not running. It was - not running because either it crashed or memory sanitizer aborted it - during Test 728. Test 728 connects to the SOCKS5 test server on a - redirect but does not send any data on purpose. The test server was not - prepared for that. +- libcurl-security.3: fix typo - Reported-by: Dan Fandrich + Fixed minimal typo. - Fixes https://github.com/curl/curl/issues/12117 - Closes https://github.com/curl/curl/pull/12118 + Closes #12437 -Daniel Stenberg (14 Oct 2023) +Stefan Eissing (1 Dec 2023) -- RELEASE-NOTES: synced +- ngtcp2: fix races in stream handling -Sohom Datta (14 Oct 2023) + - fix cases where ngtcp2 invokes callbacks on streams that + nghttp3 has already forgotten. Ignore the NGHTTP3_ERR_STREAM_NOT_FOUND + in these cases as it is normal behaviour. -- tool_getparam: limit --rate to be smaller than number of ms + Closes #12435 - Currently, curl allows users to specify absurd request rates that might - be higher than the number of milliseconds in the unit (ex: curl --rate - 3600050/h http://localhost:8080 does not error out despite there being - only 3600000ms in a hour). +Emanuele Torre (1 Dec 2023) - This change adds a conditional check before the millisecond calculation - making sure that the number is not higher than the numerator (the unit) - If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE +- tool_writeout_json: fix JSON encoding of non-ascii bytes - Closes #12116 + char variables if unspecified can be either signed or unsigned depending + on the platform according to the C standard; in most platforms, they are + signed. -Daniel Stenberg (14 Oct 2023) + This meant that the *i<32 waas always true for bytes with the top bit + set. So they were always getting encoded as \uXXXX, and then since they + were also signed negative, they were getting extended with 1s causing + '\xe2' to be expanded to \uffffffe2, for example: -- opts: fix two minor man page format mistakes + $ curl --variable 'v=“' --expand-write-out '{{v:json}}\n' file:///dev/nul + l + \uffffffe2\uffffff80\uffffff9c -Jay Satiro (14 Oct 2023) + I fixed this bug by making the code use explicitly unsigned char* + variables instead of char* variables. -- curl_trc: remove a bad assertion + Test 268 verifies - - Remove DEBUGASSERT that an internal handle must not have user - private_data set before calling the user's debug callback. + Reported-by: iconoclasthero + Closes #12434 - This is a follow-up to 0dc40b2a. The user can distinguish their easy - handle from an internal easy handle by setting CURLOPT_PRIVATE on their - easy handle. I had wrongly assumed that meant the user couldn't then - set CURLOPT_PRIVATE on an internal handle as well. +Stefan Eissing (1 Dec 2023) - Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 - Reported-by: Daniel Stenberg +- cf-socket: TCP trace output local address used in connect - Closes https://github.com/curl/curl/pull/12104 + Closes #12427 -Dan Fandrich (13 Oct 2023) +Jay Satiro (1 Dec 2023) -- test613: stop showing an error on missing output file +- CURLINFO_PRETRANSFER_TIME_T.3: fix time explanation - This test would show an error message if the output was missing during - the log post-processing step, but the message was not captured by the - test harness and wasn't useful since the normal golden log file - comparison would the problem more clearly. + - Change CURLINFO_PRETRANSFER_TIME_T explanation to say that it + includes protocol-specific instructions that trigger a transfer. -Stefan Eissing (13 Oct 2023) + Prior to this change it explicitly said that it did not include those + instructions in the time, but that is incorrect. -- quic: manage connection idle timeouts + The change is a copy of the fixed explanation already in + CURLINFO_PRETRANSFER_TIME, fixed by ec8dcd7b. - - configure a 120s idle timeout on our side of the connection - - track the timestamp when actual socket IO happens - - check IO timestamp to our *and* the peer's idle timeouts - in "is this connection alive" checks + Reported-by: eeverettrbx@users.noreply.github.com - Reported-by: calvin2021y on github - Fixes #12064 - Closes #12077 + Fixes https://github.com/curl/curl/issues/12431 + Closes https://github.com/curl/curl/pull/12432 -Dan Fandrich (13 Oct 2023) +Daniel Stenberg (30 Nov 2023) -- CI: ignore test 286 on Appveyor gcc 9 build +- multi: during ratelimit multi_getsock should return no sockets - This test fails sometimes with a super fast retry loop due to what may - just be a compiler bug. The test results are ignored on the one CI job - where it occurs because there seems to be nothing we can do to fix it. + ... as there is nothing to wait for then, it just waits. Otherwise, this + causes much more CPU work and updates than necessary during ratelimit + periods. - Fixes #12040 - Closes #12106 + Ref: https://curl.se/mail/lib-2023-11/0056.html + Closes #12430 -Viktor Szakats (13 Oct 2023) +Dmitry Karpov (30 Nov 2023) -- lib: fix gcc warning in printf call +- transfer: abort pause send when connection is marked for closing - Do not pass NULL to printf %s. + This handles cases of some bi-directional "upgrade" scenarios + (i.e. WebSockets) where sending is paused until some "upgrade" handshake + is completed, but server rejects the handshake and closes the + connection. - Seen with gcc 13.2.0 on Debian: - ``` - .../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wfo - rmat-overflow=] - ``` - Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/1758442 - 6483#step:3:11104 + Closes #12428 - Ref: #10284 - Co-authored-by: Jay Satiro - Closes #12082 +Daniel Stenberg (28 Nov 2023) -Alex Klyubin (13 Oct 2023) +- RELEASE-NOTES: synced -- http2: safer invocation of populate_binsettings +- openssl: when a session-ID is reused, skip OCSP stapling - populate_binsettings now returns a negative value on error, instead of a - huge positive value. Both places which call this function have been - updated to handle this change in its contract. + Fixes #12399 + Reported-by: Alexey Larikov + Closes #12418 - The way populate_binsettings had been used prior to this change the huge - positive values -- due to signed->unsigned conversion of the potentially - negative result of nghttp2_pack_settings_payload which returns negative - values on error -- are not possible. But only because http2.c currently - always provides a large enough output buffer and provides H2 SETTINGS - IVs which pass the verification logic inside nghttp2. If the - verification logic were to change or if http2.c started passing in more - IVs without increasing the output buffer size, the overflow could become - reachable, and libcurl/curl might start leaking memory contents to - servers/proxies... +- test1545: test doing curl_formadd twice with missing file - Closes #12101 + Reproduces #12410 + Verifies the fix + Closes #12421 -Daniel Stenberg (13 Oct 2023) +- Curl_http_body: cleanup properly when Curl_getformdata errors -- openssl: avoid BN_num_bits() NULL pointer derefs + Reported-by: yushicheng7788 on github + Based-on-work-by: yushicheng7788 on github + Fixes #12410 + Closes #12421 - Reported-by: icy17 on github - Fixes #12099 - Closes #12100 +- test1477: verify that libcurl-errors.3 and public headers are synced -- wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA + The script errorcodes.pl extracts all error codes from all headers and + checks that they are all documented, then checks that all documented + error codes are also specified in a header file. - This define is set in wolfssl's options.h file when this function and - feature is present. Handles both builds with the feature explicitly - disabled and wolfSSL versions before 5.5.2 - which introduced this API - call. + Closes #12424 - Closes #12108 +- libcurl-errors.3: sync with current public headers -- tool_urlglob: make multiply() bail out on negative values + Closes #12424 - - Does not work correctly with negative values - - use __builtin_mul_overflow() on gcc +Stefan Eissing (28 Nov 2023) - Reported-by: Torben Dury - Closes #12102 +- test459: fix for parallel runs -Loïc Yhuel (13 Oct 2023) + - change warniing message to work better with varying filename + length. + - adapt test output check to new formatting -- cmake: fix CURL_DISABLE_GETOPTIONS + Follow-up to 97ccc4479f77ba3191c6 + Closes #12423 - - Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake. - - Prior to this change the option had no effect because it was missing - from that file. +Daniel Stenberg (27 Nov 2023) - Closes https://github.com/curl/curl/pull/12091 +- tool_cb_prg: make the carriage return fit for wide progress bars -- easy_lock: add a pthread_mutex_t fallback + When the progress bar was made max width (256 columns), the fly() + function attempted to generate its output buffer too long so that the + trailing carriage return would not fit and then the output would show + wrongly. The fly function is called when the expected total transfer is + unknown, which could be one or more progress calls before the actual + progress meter get shown when the expected transfer size is provided. - This allows to keep the init threadsafe with gcc < 4.9.0 (no C11 - atomics). + This new take also replaces the msnprintf() call with a much simpler + memset() for speed. - Closes https://github.com/curl/curl/pull/12090 + Reported-by: Tim Hill + Fixes #12407 + Closes #12415 -Viktor Szakats (12 Oct 2023) +- tool_parsecfg: make warning output propose double-quoting -- CI: add autotools, out-of-tree, debug build to distro check job + When the config file parser detects a word that *probably* should be + quoted, mention double-quotes as a possible remedy. - Add a job that builds curl from a generated source tarball sample, with - autotools, out-of-tree, in debug mode. + Test 459 verifies. - Ref: #12085 - Closes #12088 + Proposed-by: Jiehong on github + Fixes #12409 + Closes #12412 -Daniel Stenberg (12 Oct 2023) +Jay Satiro (26 Nov 2023) -- http: avoid Expect: 100-continue if Upgrade: is used +- curl.rc: switch out the copyright symbol for plain ASCII - Reported-by: Daniel Jelinski - Fixes #12022 - Closes #12062 + .. like we already do for libcurl.rc. -Jan Alexander Steffens (heftig) (12 Oct 2023) + libcurl.rc copyright symbol used to cause a "non-ascii 8-bit codepoint" + warning so it was switched to ascii. -- docs: use SOURCE_DATE_EPOCH for generated manpages + Ref: https://github.com/curl/curl/commit/1ca62bb5#commitcomment-133474972 - This should make builds from Git reproducible. + Suggested-by: Robert Southee - Closes #12092 + Closes https://github.com/curl/curl/pull/12403 -Daniel Stenberg (12 Oct 2023) +Daniel Stenberg (26 Nov 2023) -- RELEASE-NOTES: synced +- conncache: use the closure handle when disconnecting surplus connections - Bumped to 8.4.1 + Use the closure handle for disconnecting connection cache entries so + that anything that happens during the disconnect is not stored and + associated with the 'data' handle which already just finished a transfer + and it is important that details from the unrelated disconnect does not + taint meta-data in the data handle. -Viktor Szakats (12 Oct 2023) + Like storing the response code. -- cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection + This also adjust test 1506. Unfortunately it also removes a key part of + the test that verifies that a connection is closed since when this + output vanishes (because the closure handle is used), we don't know + exactly that the connection actually gets closed in this test... - Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet, - aligning this with autotools. This fixes an error when doing - cross-builds and also actually detects this feature. It affected systems - not allowlisted into this, e.g. SerenityOS. + Reported-by: ohyeaah on github + Fixes #12367 + Closes #12405 - We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`. +- RELEASE-NOTES: synced - Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979 - Ref: #12095 (closed in favour of this patch) - Ref: #11964 (effort to sync cmake detections with autotools) +Stefan Eissing (24 Nov 2023) - Reported-by: Kartatz on Github - Assisted-by: Kartatz on Github - Fixes #12093 - Closes #12094 +- quic: make eyeballers connect retries stop at weird replies -- build: add `src/.checksrc` to source tarball + - when a connect immediately goes into DRAINING state, do + not attempt retries in the QUIC connection filter. Instead, + return CURLE_WEIRD_SERVER_REPLY + - When eyeballing, interpret CURLE_WEIRD_SERVER_REPLY as an + inconclusive answer. When all addresses have been attempted, + rewind the address list once on an inconclusive answer. + - refs #11832 where connects were retried indefinitely until + the overall timeout fired - Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958 + Closes #12400 - Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071 - Reported-by: Romain Geissler - Fixes #12084 - Closes #12085 +Daniel Stenberg (24 Nov 2023) -Version 8.4.0 (11 Oct 2023) +- CI: verify libcurl function SYNPOSIS sections -Daniel Stenberg (11 Oct 2023) + With the .github/scripits/verify-synopsis.pl script -- RELEASE-NOTES: synced + Closes #12402 -- THANKS: add contributors from 8.4.0 +- docs/libcurl: SYNSOPSIS cleanup -Jay Satiro (11 Oct 2023) + - use the correct include file + - make sure they are declared as in the header file + - fix minor nroff syntax mistakes (missing .fi) -- socks: return error if hostname too long for remote resolve + These are verified by verify-synopsis.pl, which extracts the SYNPOSIS + code and runs it through gcc. - Prior to this change the state machine attempted to change the remote - resolve to a local resolve if the hostname was longer than 255 - characters. Unfortunately that did not work as intended and caused a - security issue. + Closes #12402 - Bug: https://curl.se/docs/CVE-2023-38545.html +- sendf: fix comment typo -Stefan Eissing (10 Oct 2023) +- fopen: allocate the dir after fopen -- CI: remove slowed-network tests + Move the allocation of the directory name down to after the fopen() call + to allow that shortcut code path to avoid a superfluous malloc+free + cycle. - - remove these tests as they are currently not reliable in our CI - setups. + Follow-up to 73b65e94f35311 - curl handles the test cases, but CI sometimes fails on these due to - additional conditions. Rather than mix them in, an additional CI job - will be added in the future that is specific to them. + Closes #12398 - Closes https://github.com/curl/curl/pull/12075 +Stefan Eissing (24 Nov 2023) -Jay Satiro (10 Oct 2023) +- transfer: cleanup done+excess handling -- libcurl-env-dbg.3: move debug variables from libcurl-env.3 + - add `SingleRequest->download_done` as indicator that + all download bytes have been received + - remove `stop_reading` bool from readwrite functions + - move excess body handling into client download writer - - Move documentation of libcurl environment variables used only in debug - builds from libcurl-env into a separate document libcurl-env-dbg. + Closes #12371 - - Document more debug environment variables. +Daniel Stenberg (23 Nov 2023) - Previously undocumented or missing a description: +- fopen: create new file using old file's mode - CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, - CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, - CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, - CURL_TIME. + Because the function renames the temp file to the target name as a last + step, if the file was previously owned by a different user, not ORing + the old mode could otherwise end up creating a file that was no longer + readable by the original owner after save. - Closes https://github.com/curl/curl/pull/11811 + Reported-by: Loïc Yhuel + Fixes #12299 + Closes #12395 -Dan Fandrich (9 Oct 2023) +- test1476: require proxy -- test670: increase the test timeout + Follow-up from 323df4261c3542 - This should make it more immune to loaded servers. + Closes #12394 - Ref: #11328 +- fopen: create short(er) temporary file name -Stefan Eissing (9 Oct 2023) + Only using random letters in the name plus a ".tmp" extension. Not by + appending characters to the final file name. -- MQTT: improve receive of ACKs + Reported-by: Maksymilian Arciemowicz - - add `mq->recvbuf` to provide buffering of incomplete - ACK responses - - continue ACK reading until sufficient bytes available - - fixes test failures on low network receives + Closes #12388 - Closes #12071 +Stefan Eissing (23 Nov 2023) -Viktor Szakats (9 Oct 2023) +- tests: git ignore generated second-hsts.txt file -- quic: fix BoringSSL build + File is generated in test lib1900 - Add guard around `SSL_CTX_set_ciphersuites()` use. + Follow-up to 7cb03229d9e9c5 - Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 + Closes #12393 - Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 +Viktor Szakats (23 Nov 2023) - Co-authored-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes #12067 +- openssl: enable `infof_certstack` for 1.1 and LibreSSL 3.6 -Stefan Eissing (9 Oct 2023) + Lower the barrier to enable `infof_certstack()` from OpenSSL 3 to + OpenSSL 1.1.x, and LibreSSL 3.6 or upper. -- test1540: improve reliability + With the caveat, that "group name" and "type name" are missing from + the log output with these TLS backends. - - print that bytes have been received on pausing, but not how many + Follow-up to b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 - Closes #12069 + Reviewed-by: Daniel Stenberg + Closes #12385 -- test2302: improve reliability +Daniel Stenberg (23 Nov 2023) - - make result print collected write data, unless - change in meta flags is detected - - will show same result even when data arrives via - several writecb invocations +- urldata: fix typo in comment - Closes #12068 +- CI: codespell -Daniel Stenberg (9 Oct 2023) + The list of words to ignore is in the file + .github/scripts/codespell-ignore.txt -- curl_easy_pause: set "in callback" true on exit if true + Closes #12390 - Because it might have called another callback in the mean time that then - set the bit FALSE on exit. +- lib: fix comment typos - Reported-by: Jay Satiro - Fixes #12059 - Closes #12061 + Five separate ones, found by codespell -Viktor Szakats (8 Oct 2023) + Closes #12390 -- h3: add support for ngtcp2 with AWS-LC builds +- test1476: verify cookie PSL mixed case - ``` - curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- - LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 - Release-Date: [unreleased] - Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps - mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss - Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi - SSL NTLM SSL threadsafe UnixSockets - ``` +- cookie: lowercase the domain names before PSL checks - Also delete an obsolete GnuTLS TODO and update the header comment in - `FindNGTCP2.cmake`. + Reported-by: Harry Sintonen - Reviewed-by: Daniel Stenberg - Closes #12066 + Closes #12387 -- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros +Viktor Szakats (23 Nov 2023) - Syncing this up with CMake. +- openssl: fix building with v3 `no-deprecated` + add CI test - Source code uses the built-in `OPENSSL_IS_AWSLC` and - `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is - necessary from the build tools. - - The one use of `HAVE_BORINGSSL` in the source turned out to be no longer - necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 - - autotools detects this anyway for display purposes. - CMake detects this to decide whether to use the BoringSSL-specific - crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection - result just yet (planned in #12066). - - Ref: #11964 - - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #12065 + - build quictls with `no-deprecated` in CI to have test coverage for + this OpenSSL 3 configuration. -Marc Hoersken (8 Oct 2023) + - don't call `OpenSSL_add_all_algorithms()`, `OpenSSL_add_all_digests()`. + The caller code is meant for OpenSSL 3, while these two functions were + only necessary before OpenSSL 1.1.0. They are missing from OpenSSL 3 + if built with option `no-deprecated`, causing build errors: + ``` + vtls/openssl.c:4097:3: error: call to undeclared function 'OpenSSL_add_all_ + algorithms'; ISO C99 and later do not support implicit function declaration + s [-Wimplicit-function-declaration] + vtls/openssl.c:4098:3: error: call to undeclared function 'OpenSSL_add_all_ + digests'; ISO C99 and later do not support implicit function declarations [ + -Wimplicit-function-declaration] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl-for-win/builds/48587418?f + ullLog=true#L7667 -- CI: move distcheck job from Azure Pipelines to GitHub Actions + Regression from b6e6d4ff8f253c8b8055bab9d4d6a10f9be109f3 #12030 + Bug: https://github.com/curl/curl/issues/12380#issuecomment-1822944669 + Reviewed-by: Alex Bozarth - This will allow for more trigger excludes within Azure Pipelines. + - vquic/curl_ngtcp2: fix using `SSL_get_peer_certificate` with + `no-deprecated` quictls 3 builds. + Do it by moving an existing solution for this from `vtls/openssl.c` + to `vtls/openssl.h` and adjusting caller code. + ``` + vquic/curl_ngtcp2.c:1950:19: error: implicit declaration of function 'SSL_g + et_peer_certificate'; did you mean 'SSL_get1_peer_certificate'? [-Wimplicit + -function-declaration] + ``` + Ref: https://github.com/curl/curl/actions/runs/6960723097/job/18940818625#s + tep:24:1178 - Also fixes seemingly broken check with scripts/installcheck.sh. - Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 + - curl_ntlm_core: fix `-Wunused-parameter`, `-Wunused-variable` and + `-Wunused-function` when trying to build curl with NTLM enabled but + without the necessary TLS backend (with DES) support. - Assisted-by: Philip Heiduck - Closes #9532 + Closes #12384 -Daniel Stenberg (8 Oct 2023) +- curl.h: delete Symbian OS references -- url: fall back to http/https proxy env-variable if ws/wss not set + curl deprecated Symbian OS in 3d64031fa7a80ac4ae3fd09a5939196268b92f81 + via #5989. Delete references to it from public headers, because there + is no fresh release to use those headers with. - Reported-by: Craig Andrews - Fixes #12031 - Closes #12058 + Reviewed-by: Dan Fandrich + Reviewed-by: Jay Satiro + Closes #12378 -Stefan Eissing (8 Oct 2023) +- windows: use built-in `_WIN32` macro to detect Windows -- cf-socket: simulate slow/blocked receives in debug + Windows compilers define `_WIN32` automatically. Windows SDK headers + or build env defines `WIN32`, or we have to take care of it. The + agreement seems to be that `_WIN32` is the preferred practice here. + Make the source code rely on that to detect we're building for Windows. - add 2 env variables for non-UDP sockets: - 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly - should return EAGAIN - 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket + Public `curl.h` was using `WIN32`, `__WIN32__` and `CURL_WIN32` for + Windows detection, next to the official `_WIN32`. After this patch it + only uses `_WIN32` for this. Also, make it stop defining `CURL_WIN32`. - Closes #12035 + There is a slight chance these break compatibility with Windows + compilers that fail to define `_WIN32`. I'm not aware of any obsolete + or modern compiler affected, but in case there is one, one possible + solution is to define this macro manually. -- http2: refused stream handling for retry + grepping for `WIN32` remains useful to discover Windows-specific code. - - answer HTTP/2 streams refused via a GOAWAY from the server to - respond with CURLE_RECV_ERROR in order to trigger a retry - on another connection + Also: - Reported-by: black-desk on github - Ref #11859 - Closes #12054 + - extend `checksrc` to ensure we're not using `WIN32` anymore. -Jay Satiro (8 Oct 2023) + - apply minor formatting here and there. -- CURLOPT_DEBUGFUNCTION.3: warn about internal handles + - delete unnecessary checks for `!MSDOS` when `_WIN32` is present. - - Warn that the user's debug callback may be called with the handle - parameter set to an internal handle. + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Without this warning the user may assume that the only handles their - debug callback receives are the easy handles on which they set - CURLOPT_DEBUGFUNCTION. + Closes #12376 - This is a follow-up to f8cee8cc which changed DoH handles to inherit - the debug callback function set in the user's easy handle. As a result - those handles are now passed to the user's debug callback function. +Stefan Eissing (22 Nov 2023) - Closes https://github.com/curl/curl/pull/12034 +- url: ConnectionExists revisited -- url: fix typo + - have common pattern of `if not match, continue` + - revert pages long if()s to return early + - move dead connection check to later since it may + be relatively expensive + - check multiuse also when NOT building with NGHTTP2 + - for MULTIUSE bundles, verify that the inspected + connection indeed supports multiplexing when in use + (bundles may contain a mix of connection, afaict) -Daniel Stenberg (8 Oct 2023) + Closes #12373 -- test458: verify --expand-output, expanding a file name accepting option +Daniel Stenberg (22 Nov 2023) - Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) +- CURLMOPT_MAX_CONCURRENT_STREAMS: make sure the set value is within range -- tool_getparam: accept variable expansion on file names too + ... or use the default value. - Reported-by: PBudmark on github - Fixes #12048 - Closes #12055 + Also clarify the documentation language somewhat. -- RELEASE-NOTES: synced + Closes #12382 -- multi: do CURLM_CALL_MULTI_PERFORM at two more places +- urldata: make maxconnects a 32 bit value - ... when it does a state transition but there is no particular socket or - timer activity. This was made apparent when commit b5bb84c removed a - superfluous timer expiry. + "2^32 idle connections ought to be enough for anybody" - Reported-by: Dan Fandrich. - Fixes #12033 - Closes #12056 + Closes #12375 -Viktor Szakats (7 Oct 2023) +- FEATURES: update the URL phrasing -- GHA/linux: mbedtls 3.5.0 + minor dep bumps + The URL is length limited since a while back so "no limit" simply is not + true anymore. Mention the URL RFC standard used instead. - Closes #12057 + Closes #12383 -Dan Fandrich (7 Oct 2023) +- wolfssh: remove redundant static prototypes -- CI: bump OpenLDAP package version on FreeBSD + vssh/wolfssh.c:346:18: error: redundant redeclaration of ‘wscp_recv’ [-We + rror=redundant-decls] - The old one is no longer available. + Closes #12381 -Marc Hoersken (7 Oct 2023) +- setopt: remove superfluous use of ternary expressions -- docs/libcurl/opts/Makefile.inc: add missing manpage files + Closes #12374 - Detected with #9532 +- mime: store "form escape" as a single bit -Dan Fandrich (7 Oct 2023) + Closes #12374 -- tests: fix a race condition in ftp server disconnect +- setopt: check CURLOPT_TFTP_BLKSIZE range on set - If a client disconnected and reconnected quickly, before the ftp server - had a chance to respond, the protocol message/ack (ping/pong) sequence - got out of sync, causing messages sent to the old client to be delivered - to the new. A disconnect must now be acknowledged and intermediate - requests thrown out until it is, which ensures that such synchronization - problems can't occur. This problem could affect ftp, pop3, imap and smtp - tests. + ... instead of later when the transfer is about to happen. - Fixes #12002 - Closes #12049 + Closes #12374 -Viktor Szakats (7 Oct 2023) +Viktor Szakats (21 Nov 2023) -- appveyor: bump mingw-w64 job to gcc 13 (was: 8) +- build: add more picky warnings and fix them - This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). - Adding a modern gcc version to the tests. + Enable more picky compiler warnings. I've found these options in the + nghttp3 project when implementing the CMake quick picky warning + functionality for it [1]. - (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 - finished in 32, 35, 34 minutes in the 3 test runs so far.) + `-Wunused-macros` was too noisy to keep around, but fixed a few issues + it revealed while testing. - It also adds a modern CMake version and OS env to our mingw-w64 builds. + - autotools: reflect the more precisely-versioned clang warnings. + Follow-up to 033f8e2a08eb1d3102f08c4d8c8e85470f8b460e #12324 + - autotools: sync between clang and gcc the way we set `no-multichar`. + - autotools: avoid setting `-Wstrict-aliasing=3` twice. + - autotools: disable `-Wmissing-noreturn` for MSYS gcc targets [2]. + It triggers in libtool-generated stub code. - Closes #12051 + - lib/timeval: delete a redundant `!MSDOS` guard from a `WIN32` branch. -David Benjamin (6 Oct 2023) + - lib/curl_setup.h: delete duplicate declaration for `fileno`. + Added in initial commit ae1912cb0d494b48d514d937826c9fe83ec96c4d + (1999-12-29). This suggests this may not be needed anymore, but if + it does, we may restore this for those specific (non-Windows) systems. + - lib: delete unused macro `FTP_BUFFER_ALLOCSIZE` since + c1d6fe2aaa5a26e49a69a4f2495b3cc7a24d9394. + - lib: delete unused macro `isxdigit_ascii` since + f65f750742068f579f4ee6d8539ed9d5f0afcb85. + - lib/mqtt: delete unused macro `MQTT_HEADER_LEN`. + - lib/multi: delete unused macro `SH_READ`/`SH_WRITE`. + - lib/hostip: add `noreturn` function attribute via new `CURL_NORETURN` + macro. + - lib/mprintf: delete duplicate declaration for `Curl_dyn_vprintf`. + - lib/rand: fix `-Wunreachable-code` and related fallouts [3]. + - lib/setopt: fix `-Wunreachable-code-break`. + - lib/system_win32 and lib/timeval: fix double declarations for + `Curl_freq` and `Curl_isVistaOrGreater` in CMake UNITY mode [4]. + - lib/warnless: fix double declarations in CMake UNITY mode [5]. + This was due to force-disabling the header guard of `warnless.h` to + to reapply it to source code coming after `warnless.c` in UNITY + builds. This reapplied declarations too, causing the warnings. + Solved by adding a header guard for the lines that actually need + to be reapplied. + - lib/vauth/digest: fix `-Wunreachable-code-break` [6]. + - lib/vssh/libssh2: fix `-Wunreachable-code-break` and delete redundant + block. + - lib/vtls/sectransp: fix `-Wunreachable-code-break` [7]. + - lib/vtls/sectransp: suppress `-Wunreachable-code`. + Detected in `else` branches of dynamic feature checks, with results + known at compile-time, e.g. + ```c + if(SecCertificateCopySubjectSummary) /* -> true */ + ``` + Likely fixable as a separate micro-project, but given SecureTransport + is deprecated anyway, let's just silence these locally. + - src/tool_help: delete duplicate declaration for `helptext`. + - src/tool_xattr: fix `-Wunreachable-code`. + - tests: delete duplicate declaration for `unitfail` [8]. + - tests: delete duplicate declaration for `strncasecompare`. + - tests/libtest: delete duplicate declaration for `gethostname`. + Originally added in 687df5c8c39c370a59999b9afc0917d808d978b7 + (2010-08-02). + Got complicated later: c49e9683b85ba9d12cbb6eebc4ab2c8dba68fbdc + If there are still systems around with warnings, we may restore the + prototype, but limited for those systems. + - tests/lib2305: delete duplicate declaration for + `libtest_debug_config`. + - tests/h2-download: fix `-Wunreachable-code-break`. -- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR + [1] https://github.com/ngtcp2/nghttp3/blob/a70edb08e954d690e8fb2c1df999b5a056 + f8bf9f/cmake/PickyWarningsC.cmake + [2] https://ci.appveyor.com/project/curlorg/curl/builds/48553586/job/3qkgjaui + qla5fj45?fullLog=true#L1675 + [3] https://github.com/curl/curl/actions/runs/6880886309/job/18716044703?pr=1 + 2331#step:7:72 + https://github.com/curl/curl/actions/runs/6883016087/job/18722707368?pr=1 + 2331#step:7:109 + [4] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L204 + [5] https://ci.appveyor.com/project/curlorg/curl/builds/48555101/job/9g15qkrr + iklpf1ut#L218 + [6] https://github.com/curl/curl/actions/runs/6880886309/job/18716042927?pr=1 + 2331#step:7:290 + [7] https://github.com/curl/curl/actions/runs/6891484996/job/18746659406?pr=1 + 2331#step:9:1193 + [8] https://github.com/curl/curl/actions/runs/6882803986/job/18722082562?pr=1 + 2331#step:33:1870 - While the struct is still public in OpenSSL, there is a (somewhat - inconvenient) accessor. Use it to remain compatible if it becomes opaque - in the future. + Closes #12331 - Closes #12038 +Daniel Stenberg (21 Nov 2023) -Daniel Stenberg (6 Oct 2023) +- transfer: avoid unreachable expression -- curl_easy_pause.3: mention it works within callbacks + If curl_off_t and size_t have the same size (which is common on modern + 64 bit systems), a condition cannot occur which Coverity pointed + out. Avoid the warning by having the code conditionally only used if + curl_off_t actually is larger. - Reported-by: Maxim Dzhura - Bug: https://curl.se/mail/lib-2023-10/0010.html - Closes #12046 + Follow-up to 1cd2f0072fa482e25baa2 -- curl_easy_pause.3: mention h2/h3 buffering + Closes #12370 - Asked-by: Maxim Dzhura - Ref: https://curl.se/mail/lib-2023-10/0011.html +Stefan Eissing (21 Nov 2023) - Closes #12045 +- transfer: readwrite improvements -Viktor Szakats (6 Oct 2023) + - changed header/chunk/handler->readwrite prototypes to accept `buf`, + `blen` and a `pconsumed` pointer. They now get the buffer to work on + and report back how many bytes they consumed + - eliminated `k->str` in SingleRequest + - improved excess data handling to properly calculate with any body data + left in the headerb buffer + - eliminated `k->badheader` enum to only be a bool -- cmake: re-add missed C89 headers for specific detections + Closes #12283 - We removed C89 `setjmp.h` and `signal.h` detections and excluded them - from the global header list we use when detecting functions [1]. Then - missed to re-add these headers to the specific functions which need - them to be detected [2]. Fix this omission in this patch. +Daniel Stenberg (21 Nov 2023) - [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 - [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 +- RELEASE-NOTES: synced - Closes #12043 +Jiří Hruška (21 Nov 2023) -Daniel Stenberg (6 Oct 2023) +- transfer: avoid calling the read callback again after EOF -- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE + Regression since 7f43f3dc5994d01b12 (7.84.0) - Since there is nothing to wait for there. Avoids the test 1233 hang - reported in #12033. + Bug: https://curl.se/mail/lib-2023-11/0017.html - Reported-by: Dan Fandrich - Closes #12042 + Closes #12363 -Dan Fandrich (5 Oct 2023) +Daniel Stenberg (21 Nov 2023) -- test1903: actually verify the cookies after the test +- doh: provide better return code for responses w/o addresses - The test otherwise could do just about anything (except leak memory in - debug mode) and its bad behaviour wouldn't be detected. Now, check the - resulting cookie file to ensure the cookies are still there. + Previously it was wrongly returning CURLE_OUT_OF_MEMORY when the + response did not contain any addresses. Now it more accurately returns + CURLE_COULDNT_RESOLVE_HOST. - Closes #12041 + Reported-by: lRoccoon on github -- test: add missing s + Fixes #12365 + Closes #12366 - The tests will otherwise fail if curl has them disabled. +Stefan Eissing (21 Nov 2023) -- test1906: set a lower timeout since it's hit on Windows +- HTTP/2, HTTP/3: handle detach of onoing transfers - msys2 builds actually hit the connect timeout in normal operation, so - lower the timeout from 5 minutes to 5 seconds to reduce test time. + - refs #12356 where a UAF is reported when closing a connection + with a stream whose easy handle was cleaned up already + - handle DETACH events same as DONE events in h2/h3 filters - Ref: #11328 - Closes #12036 + Fixes #12356 + Reported-by: Paweł Wegner + Closes #12364 -Daniel Stenberg (5 Oct 2023) +Viktor Szakats (20 Nov 2023) -- RELEASE-NOTES: synced +- autotools: stop setting `-std=gnu89` with `--enable-warnings` -Jay Satiro (5 Oct 2023) + Do not alter the C standard when building with `--enable-warnings` when + building with gcc. -- idn: fix WinIDN null ptr deref on bad host + On one hand this alters warning results compared to a default build. + On the other, it may produce different binaries, which is unexpected. - - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from - UTF-8 to UTF-16. + Also fix new warnings that appeared after removing `-std=gnu89`: - Prior to this change a failed conversion erroneously returned CURLE_OK - which meant 'decoded' pointer (what would normally point to the - punycode) would not be written to, remain NULL and be dereferenced - causing an access violation. + - include: fix public curl headers to use the correct printf mask for + `CURL_FORMAT_CURL_OFF_T` and `CURL_FORMAT_CURL_OFF_TU` with mingw-w64 + and Visual Studio 2013 and newer. This fixes the printf mask warnings + in examples and tests. E.g. [1] - Closes https://github.com/curl/curl/pull/11983 + - conncache: fix printf format string [2]. -Dan Fandrich (4 Oct 2023) + - http2: fix potential null pointer dereference [3]. + (seen on Slackware with gcc 11.) -- tests: close the shell used to start sshd + - libssh: fix printf format string in SFTP code [4]. + Also make MSVC builds compatible with old CRT versions. - This shell isn't needed once sshd starts, so use "exec" so it doesn't - stick around. + - libssh2: fix printf format string in SFTP code for MSVC. + Applying the same fix as for libssh above. - Closes #12032 + - unit1395: fix `argument is null` and related issues [5]: + - stop calling `strcmp()` with NULL to avoid undefined behaviour. + - fix checking results if some of them were NULL. + - do not pass NULL to printf `%s`. -Daniel Stenberg (4 Oct 2023) + - ci: keep a build job with `-std=gnu89` to continue testing for + C89-compliance. We can apply this to other gcc jobs as needed. + Ref: b23ce2cee7329bbf425f18b49973b7a5f23dfcb4 (2022-09-23) #9542 -- base64: also build for curl + [1] https://dev.azure.com/daniel0244/curl/_build/results?buildId=18581&view=l + ogs&jobId=ccf9cc6d-2ef1-5cf2-2c09-30f0c14f923b + [2] https://github.com/curl/curl/actions/runs/6896854263/job/18763831142?pr=1 + 2346#step:6:67 + [3] https://github.com/curl/curl/actions/runs/6896854253/job/18763839238?pr=1 + 2346#step:30:214 + [4] https://github.com/curl/curl/actions/runs/6896854253/job/18763838007?pr=1 + 2346#step:29:895 + [5] https://github.com/curl/curl/actions/runs/6896854253/job/18763836775?pr=1 + 2346#step:33:1689 - Since the tool itself now uses the base64 code using the curlx way, it - needs to build also when the tool needs it. Starting now, the tool build - defines BULDING_CURL to allow lib-side code to use it. + Closes #12346 - Follow-up to 2e160c9c6525 +- autotools: fix/improve gcc and Apple clang version detection - Closes #12010 + - Before this patch we expected `n.n` `-dumpversion` output, but Ubuntu + may return `n-win32` (also with `-dumpfullversion`). Causing these + errors and failing to enable picky warnings: + ``` + ../configure: line 23845: test: : integer expression expected + ``` + Ref: https://github.com/libssh2/libssh2/actions/runs/6263453828/job/1700789 + 3718#step:5:143 -Eduard Strehlau (4 Oct 2023) + Fix that by stripping any dash-suffix and handling a dotless (major-only) + version number by assuming `.0` in that case. -- tests: Fix zombie processes left behind by FTP tests. + `9.3-posix`, `9.3-win32`, `6`, `9.3.0`, `11`, `11.2`, `11.2.0` + Ref: https://github.com/mamedev/mame/pull/9767 - ftpserver.pl correctly cleans up spawned server processes, - but forgets to wait for the shell used to spawn them. - This is barely noticeable during a normal testrun, - but causes process exhaustion and test failure - during a complete torture run of the FTP tests. + - fix Apple clang version detection for releases between + 'Apple LLVM version 7.3.0' and 'Apple LLVM version 10.0.1' where the + version was under-detected as 3.7 llvm/clang equivalent. - Fixes #12018 - Closes #12020 + - fix Apple clang version detection for 'Apple clang version 11.0.0' + and newer where the Apple clang version was detected, instead of its + llvm/clang equivalent. -Dan Fandrich (4 Oct 2023) + - display detected clang/gcc/icc compiler version. -- github/labeler: improve labeler matches + Via libssh2: + - https://github.com/libssh2/libssh2/commit/00a3b88c51cdb407fbbb347a2e38c5c7d + 89875ad + https://github.com/libssh2/libssh2/pull/1187 + - https://github.com/libssh2/libssh2/commit/89ccc83c7da73e7ca3a112e3500081319 + 42b592e + https://github.com/libssh2/libssh2/pull/1232 -- test574: add a timeout to the test + Closes #12362 - This one hangs occasionally, so this will speed up a test run and allow - logs to be seen when it does. +- autotools: delete LCC compiler support bits - Closes #12025 + Follow-up to fd7ef00f4305a2919e6950def1cf83d0110a4acd #12222 -- tests: propagate errors in libtests + Closes #12357 - Use the test macros to automatically propagate some errors, and check - and log others while running the tests. This can help in debugging - exactly why a test has failed. +- cmake: add test for `DISABLE` options, add `CURL_DISABLE_HEADERS_API` -- tests: set --expect100-timeout to improve test reliability + - tests: verify CMake `DISABLE` options. - On an overloaded server, the default 1 second timeout can go by without - the test server having a chance to respond with the expected headers, - causing tests to fail. Increase the 1 second timeout to 99 seconds so - this failure mode is no longer a problem on test 1129. Some other tests - already set a high value, but make them consistently 99 seconds so if - something goes wrong the test is stalled for less time. + Make an exception for 2 CMake-only ones, and one more that's + using a different naming scheme, also in autotools and source. - Ref: #11328 + - cmake: add support for `CURL_DISABLE_HEADERS_API`. -- CI: ignore the "flaky" and "timing-dependent" test results in CMake + Suggested-by: Daniel Stenberg + Ref: https://github.com/curl/curl/pull/12345#pullrequestreview-1736238641 - This was already done for automake builds but CMake builds were missed. - Test 1086 actually causes the test harness to crash with: + Closes #12353 - Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec - ts/curl/tests/ftpserver.pl line 527 +Jacob Hoffman-Andrews (20 Nov 2023) - Rather than fix it now, this change leaves test 1086 entirely skipped on - those builds that show this problem. +- hyper: temporarily remove HTTP/2 support - Follow-up to 589dca761 + The current design of the Hyper integration requires rebuilding the + Hyper clientconn for each request. However, building the clientconn + requires resending the HTTP/2 connection preface, which is incorrect + from a protocol perspective. That in turn causes servers to send GOAWAY + frames, effectively degrading performance to "no connection reuse" in + the best case. It may also be triggering some bugs where requests get + dropped entirely and reconnects take too long. - Ref: #11865 + This doesn't rule out HTTP/2 support with Hyper, but it may take a + redesign of the Hyper integration in order to make things work. -Viktor Szakats (4 Oct 2023) + Closes #12191 -- cmake: improve OpenLDAP builds +Jay Satiro (20 Nov 2023) - - cmake: detect OpenLDAP based on function `ldap_init_fd`. - autotools does this. autotools also publishes this detection result - in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source - doesn't use this value. (it might need to be remove-listed in - `scripts/cmp-config.pl` for future OpenLDAP test builds.) - This also deletes existing self-declaration method via the - CMake-specific `CURL_USE_OPENLDAP` configuration. - - - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. - Like autotools does. This fixes a long list of these warnings: - ``` - /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i - s not defined, evaluates to 0 [-Wundef] - ``` +- schannel: fix unused variable warning - - cmake: delete LDAP TODO comment no longer relevant. + Bug: https://github.com/curl/curl/pull/12349#issuecomment-1818000846 + Reported-by: Viktor Szakats - Also: + Closes https://github.com/curl/curl/pull/12361 - - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature - detection functions. +Daniel Stenberg (19 Nov 2023) - Ref: #11964 (effort to sync cmake detections with autotools) +- url: find scheme with a "perfect hash" - Closes #12024 + Instead of a loop to scan over the potentially 30+ scheme names, this + uses a "perfect hash" table. This works fine because the set of schemes + is known and cannot change in a build. The hash algorithm and table size + is made to only make a single scheme index per table entry. -- cmake: fix unity builds for more build combinations + The perfect hash is generated by a separate tool (scripts/schemetable.c) - By using unique static function/variable names in source files - implementing these interfaces. + Closes #12347 - - OpenLDAP combined with any SSH backend. +- scripts: add schemetable.c - - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. + This tool generates a scheme-matching table. - Closes #12027 + It iterates over a number of different initial and shift values in order + to find the hash algorithm that needs the smallest possible table. -Daniel Stenberg (4 Oct 2023) + The generated hash function, table and table size then needs to be used + by the url.c:Curl_getn_scheme_handler() function. -- tests: remove leading spaces from some tags +Stefan Eissing (19 Nov 2023) - The threee tags ``, `` and `` were frequently used - with a leading space that this removes. The reason this habbit is so - widespread in testcases is probably that they have been copy and pasted. +- vtls/vquic, keep peer name information together - Hence, fixing them all now might curb this practice from now on. + - add `struct ssl_peer` to keep hostname, dispname and sni + for a filter + - allocate `sni` for use in VTLS backend + - eliminate `Curl_ssl_snihost()` and its use of the download buffer + - use ssl_peer in SSL and QUIC filters - Closes #12028 + Closes #12349 -Viktor Szakats (4 Oct 2023) +Viktor Szakats (18 Nov 2023) -- GHA: bump actions/checkout +- build: always revert `#pragma GCC diagnostic` after use - Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 - Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 + Before this patch some source files were overriding gcc warning options, + but without restoring them at the end of the file. In CMake UNITY builds + these options spilled over to the remainder of the source code, + effecitvely disabling them for a larger portion of the codebase than + intended. - Closes #12023 + `#pragma clang diagnostic` didn't have such issue in the codebase. -- spelling: fix codespell 2.2.6 typos + Reviewed-by: Marcel Raad + Closes #12352 - Closes #12019 +- tidy-up: casing typos, delete unused Windows version aliases -Daniel Stenberg (3 Oct 2023) + - cmake: fix casing of `UnixSockets` to match the rest of the codebase. -- GHA: add workflow to compare configure vs cmake outputs + - curl-compilers.m4: fix casing in a comment. - Uses scripts/cmp-config.pl two compare two curl_config.h files, - presumbly generated with configure and cmake. It displays the - differences and filters out a lot of known lines we ignore. + - setup-win32: delete unused Windows version constant aliases. - The script also shows the matches that were *not* used. Possibly - subjects for removal. + Reviewed-by: Marcel Raad + Closes #12351 - Closes #11964 +- keylog: disable if unused -- appveyor: enable test 571 + Fully disable keylog code if there is no TLS or QUIC subsystem using it. - Follow-up from 8a940fd55c175f7 / #12013 + Closes #12350 - Closes #12017 +- cmake: add `CURL_DISABLE_BINDLOCAL` option -Viktor Szakats (3 Oct 2023) + To match similar autotools option. -- build: alpha-sort source files for lib and src + Default is `ON`. - Closes #12014 + Reviewed-by: Daniel Stenberg + Closes #12345 -- cmake: delete old `HAVE_LDAP_URL_PARSE` logic +- url: fix `-Wzero-length-array` with no protocols - Left there by accident after adding proper detection for this. + Fixes: + ``` + ./lib/url.c:178:56: warning: use of an empty initializer is a C2x extension [ + -Wc2x-extensions] + 178 | static const struct Curl_handler * const protocols[] = { + | ^ + ./lib/url.c:178:56: warning: zero size arrays are an extension [-Wzero-length + -array] + ``` - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 + Closes #12344 - Ref: #11964 (effort to sync cmake detections with autotools) +- url: fix builds with `CURL_DISABLE_HTTP` - Closes #12015 + Fixes: + ``` + ./lib/url.c:456:35: error: no member named 'formp' in 'struct UrlState' + 456 | Curl_mime_cleanpart(data->state.formp); + | ~~~~~~~~~~~ ^ + ``` -Stefan Eissing (3 Oct 2023) + Regression from 74b87a8af13a155c659227f5acfa78243a8b2aa6 #11682 -- tests: increase lib571 timeout from 3s to 30s + Closes #12343 - - 3s is too short for our CI, making this test fail occasionally - - test usually experiences no delay run locally, so 30s wont hurt +- http: fix `-Wunused-parameter` with no auth and no proxy - Closes #12013 + ``` + lib/http.c:734:26: warning: unused parameter 'proxy' [-Wunused-parameter] + bool proxy) + ^ + ``` -Viktor Szakats (3 Oct 2023) + Reviewed-by: Marcel Raad + Closes #12338 -- cmake: fix unity with Windows Unicode + TrackMemory +Daniel Stenberg (16 Nov 2023) - Found the root cause of the startup crash in unity builds with Unicode - and TrackMemory enabled at the same time. +- TODO: Some TLS options are not offered for HTTPS proxies - We must make sure that the `memdebug.h` header doesn't apply to - `lib/curl_multibyte.c` (as even noted in a comment there.) In unity - builds all headers apply to all sources, including `curl_multibyte.c`. - This probably resulted in an infinite loop on startup. + Closes #12286 + Closes #12342 - Exclude this source from unity compilation with TrackMemory enabled, - in both libcurl and curl tool. Enable unity mode for a debug Unicode - CI job to keep it tested. Also delete the earlier workaround that - fully disabled unity for affected builds. +- RELEASE-NOTES: synced - Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 +- duphandle: make dupset() not return with pointers to old alloced data - Closes #11928 + As the blob pointers are to be duplicated, the function must not return + mid-function with lingering pointers to the old handle's allocated data, + as that would lead to double-free in OOM situations. -- cmake: disable unity mode with Windows Unicode + TrackMemory + Make sure to clear all destination pointers first to avoid this risk. - "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, - aka `-DCURLDEBUG`). + Closes #12337 - There is an issue with memory tracking and Unicode when built in "unity" - mode, which results in the curl tool crashing right on startup, even - without any command-line option. Interestingly this doesn't happen under - WINE (at least on the system I tested this on), but consistenly happens - on real Windows machines. Crash is 0xC0000374 heap corruption. Both - shared and static curl executables are affected. +Viktor Szakats (16 Nov 2023) - This limitation probably won't hit too many people, but it remains - a TODO to find and fix the root cause and drop this workaround. +- http: fix `-Wunused-variable` compiler warning - Example builds and runs: - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd - 7iwj#L313 (static) - https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby - qu9c#L317 (shared) + Fix compiler warnings in builds with disabled auths, NTLM and SPNEGO. - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 + E.g. with `CURL_DISABLE_BASIC_AUTH` + `CURL_DISABLE_BEARER_AUTH` + + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_NEGOTIATE_AUTH` + + `CURL_DISABLE_NTLM` on non-Windows. - Ref: #11928 - Closes #12005 + ``` + ./curl/lib/http.c:737:12: warning: unused variable 'result' [-Wunused-variabl + e] + CURLcode result = CURLE_OK; + ^ + ./curl/lib/http.c:995:18: warning: variable 'availp' set but not used [-Wunus + ed-but-set-variable] + unsigned long *availp; + ^ + ./curl/lib/http.c:996:16: warning: variable 'authp' set but not used [-Wunuse + d-but-set-variable] + struct auth *authp; + ^ + ``` -- cmake: tidy-up `NOT_NEED_LBER_H` detection + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 - Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 + Fixes #12228 + Closes #12335 -- appveyor: rewrite batch in PowerShell + CI improvements +Jay Satiro (16 Nov 2023) - 1. Rewrite in PowerShell: +- tool: support bold headers in Windows - - rewrite MS-DOS batch build script in PowerShell. - - move some bash operations into native PowerShell. - - fixups for PowerShell insisting on failure when a command outputs - something to stderr. - - fix to actually run `curl -V` after every build. - (and exclude ARM64 builds.) - - also say why we skipped `curl -V` if we had to skip. - - fix CMake warnings about unused configuration variables, by adapting - these dynamically for build cases. - - dedupe OpenSSL path into a variable. - - disable `test1451` failing with a warning anyway due to missing python - impacket. (after trying and failing to install impacket) - PowerShell promotes these warnings to errors by PowerShell. We can also - suppress they wholesale if they start causing issues in the future, - like we already to with `autoreconf` and `./configure`. + - If virtual terminal processing is enabled in Windows then use ANSI + escape codes Esc[1m and Esc[22m to turn bold on and off. - PowerShell is better than MS-DOS batches, so the hope is this makes it - easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't - supported inline by AppVeyor on Windows build machines, but we are okay - to keep it in an external script, so it's also an option. + Suggested-by: Gisle Vanem - 2. CI improvements: + Ref: https://github.com/curl/curl/discussions/11770 - - enable tests for a "unity" build job. - - speed-up CI initialization by using shallow clones of the curl repo. - - speed-up CMake MSVC jobs with `TrackFileAccess=false`. - - enable parallelism in `VisualStudioSolution` builds. - - display CMake version before builds. - - always show the CPU in job names. - - tell which jobs are build-only in job names. - - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. - - add `config.log` (autotools) to dumped logs (need to enable manually). + Closes https://github.com/curl/curl/pull/12321 - 3. Style: +Viktor Szakats (15 Nov 2023) - - use single-quotes in YAML like we do in other CI YAML files. - It also allows to drop quoting characters and lighter to write/read. - (keep double quotes for PowerShell strings needing expansion.) +- build: fix libssh2 + `CURL_DISABLE_DIGEST_AUTH` + `CURL_DISABLE_AWS` - Closes #11999 + Builds with libssh2 + `-DCURL_DISABLE_DIGEST_AUTH=ON` + + `-DCURL_DISABLE_AWS=ON` in combination with either Schannel on Windows, + or `-DCURL_DISABLE_NTLM=ON` on other operating systems failed while + compiling due to a missing HMAC declaration. -- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows - - - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. - Before this patch we set it based it on the presence of `stricmp`, - which correctly enabled it on e.g. Windows, but was inaccurate for - other platforms. + The reason is that HMAC is required by `lib/sha256.c` which publishes + `Curl_sha256it()` which is required by `lib/vssh/libssh2.c` when + building for libssh2 v1.8.2 (2019-05-25) or older. - - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and - LDAPS is not explicitly disabled. This mimics autotools behaviour. - Previously we set it only for Windows LDAP. After this fix, LDAPS is - correctly enabled in default macOS builds. + Make sure to compile the HMAC bits for a successful build. - - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) - to avoid deprecation warnings for LDAP API. + Both HMAC and `Curl_sha256it()` rely on the same internals, so splitting + them into separate sources isn't practical. - - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. - This doesn't make much sense, but let's do it to sync behaviour with - autotools. + Fixes: + ``` + [...] + In file included from ./curl/_x64-win-ucrt-cmake-llvm-bld/lib/CMakeFiles/libc + url_object.dir/Unity/unity_0_c.c:310: + ./curl/lib/sha256.c:527:42: error: array has incomplete element type 'const s + truct HMAC_params' + 527 | const struct HMAC_params Curl_HMAC_SHA256[] = { + | ^ + ./curl/lib/curl_sha256.h:34:21: note: forward declaration of 'struct HMAC_par + ams' + [...] + ``` - - fix benign typo in variable name. + Regression from e92edfbef64448ef461117769881f3ed776dec4e #11490 - Ref: #11964 (effort to sync cmake detections with autotools) + Fixes #12273 + Closes #12332 - Closes #12006 +Daniel Stenberg (15 Nov 2023) -- autotools: restore `HAVE_IOCTL_*` detections +- duphandle: also free 'outcurl->cookies' in error path - This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in - 4d73854462f30948acab12984b611e9e33ee41e6 and - c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the - `HAVE_IOCTL` result it generated was unused in the source. But, - I did miss the fact that this had two dependent checks: - `CURL_CHECK_FUNC_IOCTL_FIONBIO`, - `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: - `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. + Fixes memory-leak when OOM mid-function - Regression from 4d73854462f30948acab12984b611e9e33ee41e6 + Use plain free instead of safefree, since the entire struct is + freed below. - Ref: #11964 (effort to sync cmake detections with autotools) + Remove some free calls that is already freed in Curl_freeset() - Closes #12008 + Closes #12329 -Daniel Stenberg (2 Oct 2023) +Viktor Szakats (15 Nov 2023) -- RELEASE-PROCEDURE.md: updated coming release dates +- config-win32: set `HAVE_SNPRINTF` for mingw-w64 -- RELEASE-NOTES: synced + It's available in all mingw-w64 releases. We already pre-fill this + detection in CMake. -Viktor Szakats (1 Oct 2023) + Closes #12325 -- cmake: pre-cache `HAVE_POLL_FINE` on Windows +- sasl: fix `-Wunused-function` compiler warning - Windows doesn't support `poll()`, so we can safely skip checking for - fine poll. + In builds with disabled auths. - Closes #12003 + ``` + lib/curl_sasl.c:266:17: warning: unused function 'get_server_message' [-Wunus + ed-function] + static CURLcode get_server_message(struct SASL *sasl, struct Curl_easy *data, + ^ + 1 warning generated. + ``` + Ref: https://github.com/curl/trurl/actions/runs/6871732122/job/18689066151#st + ep:3:3822 -- gha: bump actions to latest versions + Reviewed-by: Daniel Stenberg + Closes #12326 - - actions@checkout@v4 (from v3 and v2) +- build: picky warning updates - - fsfe/reuse-action@v2 (from v1) + - cmake: sync some picky gcc warnings with autotools. + - cmake, autotools: add `-Wold-style-definition` for clang too. + - cmake: more precise version info for old clang options. + - cmake: use `IN LISTS` syntax in `foreach()`. - Closes #12000 + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Closes #12324 -Stefan Eissing (30 Sep 2023) +Daniel Stenberg (15 Nov 2023) -- h2: testcase and fix for pausing h2 streams +- urldata: move cookielist from UserDefined to UrlState - - refs #11982 where it was noted that paused transfers may - close successfully without delivering the complete data - - made sample poc into tests/http/client/h2-pausing.c and - added test_02_27 to reproduce + 1. Because the value is not strictly set with a setopt option. - Closes #11989 - Fixes #11982 - Reported-by: Harry Sintonen + 2. Because otherwise when duping a handle when all the set.* fields are + first copied and an error happens (think out of memory mid-function), + the function would easily free the list *before* it was deep-copied, + which could lead to a double-free. -Viktor Szakats (30 Sep 2023) + Closes #12323 -- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value +Viktor Szakats (14 Nov 2023) - Before this patch CMake builds accepted any value and it was used at - runtime as-is. This patch make sure that the selected default backend - is also enabled in the build. It also enforces a full lowercase value. +- autotools: avoid passing `LDFLAGS` twice to libcurl - This improves reproducibility and brings CMake in sync with autotools - which already worked like described above. + autotools passes `LDFLAGS` automatically linker commands. curl's + `lib/Makefile.am` customizes libcurl linker flags. In that + customization, it added `LDFLAGS` to the custom flags. This resulted in + passing `LDFLAGS` _twice_ to the `libtool` command. - Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 + Most of the time this is benign, but some `LDFLAGS` options can break + the build when passed twice. One such example is passing `.o` files, + e.g. `crt*.o` files necessary when customizing the C runtime, e.g. for + MUSL builds. - Closes #11998 + Passing them twice resulted in duplicate symbol errors: + ``` + libtool: link: clang-15 --target=aarch64-unknown-linux-musl [...] /usr/lib/a + arch64-linux-musl/crt1.o [...] /usr/lib/aarch64-linux-musl/crt1.o [...] + ld.lld-15: error: duplicate symbol: _start + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + >>> defined at crt1.c + >>> /usr/lib/aarch64-linux-musl/crt1.o:(.text+0x0) + [...] + clang: error: linker command failed with exit code 1 (use -v to see invocatio + n) + ``` -- autotools: adjust `CURL_CA_PATH` value to CMake + This behaviour came with commit 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + (2013-07-23) as a fix for bug https://curl.haxx.se/bug/view.cgi?id=1217. + The patch was a works-for-me hack that ended up merged in curl: + https://sourceforge.net/p/curl/bugs/1217/#06ef + With the root cause remaining unclear. - autotools was using the same value as CMake, but with an ending - slash. Delete the ending slash to match configurations. + Perhaps the SUNPro 12 linker was sensitive to `-L` `-l` order, requiring + `-L` first? This would be unusual and suggests a bug in either the + linker or in `libtool`. - Ref: #11964 (effort to sync cmake detections with autotools) + The curl build does pass the list of detected libs via its own + `LIBCURL_LIBS` variable, which ends up before `LDFLAGS` on the `libtool` + command line, but it's the job of `libtool` to ensure that even + a peculiar linker gets the options in the expected order. Also because + autotools passes `LDFLAGS` last, making it hardly possible to pass + anything after it. - Closes #11997 + Perhaps in the 10 years since this issue, this already got a fix + upstream. -- cmake: detect `sys/wait.h` and `netinet/udp.h` + This patch deletes `LDFLAGS` from our customized libcurl options, + leaving a single copy of them as passed by autotools automatically. - Ref: #11964 (effort to sync cmake detections with autotools) + Reverts 1a593191c2769a47b8c3e4d9715ec9f6dddf5e36 + Closes #12310 - Closes #11996 +- autotools: accept linker flags via `CURL_LDFLAGS_{LIB,BIN}` -Daniel Stenberg (30 Sep 2023) + To allow passing `LDFLAGS` specific to libcurl (`CURL_LDFLAGS_LIB`) and + curl tool (`CURL_LDFLAGS_BIN`). -- lib: provide and use Curl_hexencode + This makes it possible to build libcurl and curl with a single + invocation with lib- and tool-specific custom linker flags. - Generates a lower case ASCII hex output from a binary input. + Such flag can be enabling `.map` files, a `.def` file for libcurl DLL, + controlling static/shared, incl. requesting a static curl tool (with + `-static-libtool-libs`) while building both shared and static libcurl. - Closes #11990 + curl-for-win uses the above and some more. -- configure: check for the capath by default + These options are already supported in `Makefile.mk`. CMake has built-in + variables for this. - ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS - L + Closes #12312 - cmake: synced +Jay Satiro (14 Nov 2023) - Assisted-by: Viktor Szakats - Closes #11987 +- tool_cb_hdr: add an additional parsing check -- wolfssl: ignore errors in CA path + - Don't dereference the past-the-end element when parsing the server's + Content-disposition header. - The default wolfSSL_CTX_load_verify_locations() function is quite picky - with the certificates it loads and will for example return error if just - one of the certs has expired. + As 'p' is advanced it can point to the past-the-end element and prior + to this change 'p' could be dereferenced in that case. - With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it - behaves more similar to what OpenSSL does by default. + Technically the past-the-end element is not out of bounds because dynbuf + (which manages the header line) automatically adds a null terminator to + every buffer and that is not included in the buffer length passed to + the header callback. - Even the set of default certs on my Debian unstable has several expired - ones. + Closes https://github.com/curl/curl/pull/12320 - Assisted-by: Juliusz Sosinowicz - Assisted-by: Michael Osipov +Philip Heiduck (14 Nov 2023) - Closes #11987 +- .cirrus.yml: freebsd 14 -- create-dirs.d: clarify it also uses --output-dirs + ensure curl works on latest freebsd version - Reported-by: Robert Simpson - Fixes #11991 - Closes #11995 + Closes #12053 -Viktor Szakats (30 Sep 2023) +Daniel Stenberg (13 Nov 2023) -- appveyor: fix yamlint issues, indent +- easy: in duphandle, init the cookies for the new handle - Also: - - use double quotes in all batch if statements. + ... not the source handle. - Closes #11994 + Closes #12318 -- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` +- duphandle: use strdup to clone *COPYPOSTFIELDS if size is not set - Based on existing autotools logic. + Previously it would unconditionally use the size, which is set to -1 + when strlen is requested. - Ref: #11964 (effort to sync cmake detections with autotools) + Updated test 544 to verify. - Closes #11981 + Closes #12317 -- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` +- RELEASE-NOTES: synced - Based on existing autotools logic. +- curl_easy_duphandle.3: clarify how HSTS and alt-svc are duped - autotools checks for old versions of the allowlisted target OSes and - disables this feature when seeing them. In CMake we assume we're running - on newer systems and enable regardless of OS version. + Closes #12315 - autotools always runs all 3 probes for non-fast-tracked systems and - enables this feature if any one of them was successful. To save - configuration time, CMake stops at the first successful check. +- urldata: move hstslist from 'set' to 'state' - OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD - system. I haven't double-checked if this is correct, but looks odd. + To make it work properly with curl_easy_duphandle(). This, because + duphandle duplicates the entire 'UserDefined' struct by plain copy while + 'hstslist' is a linked curl_list of file names. This would lead to a + double-free when the second of the two involved easy handles were + closed. - Ref: #11964 (effort to sync cmake detections with autotools) + Closes #12315 - Closes #11979 +- test1900: verify duphandle with HSTS using multiple files -- cmake: fix `HAVE_WRITABLE_ARGV` detection + Closes #12315 - Move detection before the creation of detection results in - `curl_config.h`. +Goro FUJI (13 Nov 2023) - Ref: #11964 (effort to sync cmake detections with autotools) +- http: allow longer HTTP/2 request method names - Closes #11978 + - Increase the maximum request method name length from 11 to 23. -- appveyor: minor improvements + For HTTP/1.1 and earlier there's not a specific limit in libcurl for + method length except that it is limited by the initial HTTP request + limit (DYN_HTTP_REQUEST). Prior to fc2f1e54 HTTP/2 was treated the same + and there was no specific limit. - - run `curl -V` after builds to see if they run and with what features. - Except for one job where a CRT DLL is missing. And ARM64 which should - fail, but is silently not launched instead. + According to Internet Assigned Numbers Authority (IANA) the longest + registered method is UPDATEREDIRECTREF which is 17 characters. - - copy libcurl DLL next to curl tool and tests binaries in shared mode. - This makes it possible to run the tests. (We don't run tests after - these builds yet.) + Also there are unregistered methods used by some companies that are + longer than 11 characters. - - list the DLLs and EXEs present after the builds. + The limit was originally added by 61f52a97 but not used until fc2f1e54. - - add `DEBUG` variable for CMake builds to allow disabling it, for - testing non-debug builds. (currently enabled for all) + Ref: https://www.iana.org/assignments/http-methods/http-methods.xhtml - - add commented lines that dump CMake configuration logs for debugging - build/auto-detection issues. + Closes https://github.com/curl/curl/pull/12311 - - add gcc version to jobs where missing. +Jay Satiro (12 Nov 2023) - - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 - to the build mix. +- CURLOPT_CAINFO_BLOB.3: explain what CURL_BLOB_COPY does - - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. + - Add an explanation of the CURL_BLOB_COPY flag to CURLOPT_CAINFO_BLOB + and CURLOPT_PROXY_CAINFO_BLOB docs. - - delete a duplicate backslash. + All the other _BLOB option docs already have the same explanation. - Closes #11976 + Closes https://github.com/curl/curl/pull/12277 -- configure: replace adhoc domain with `localhost` in tests +Viktor Szakats (11 Nov 2023) + +- tidy-up: dedupe Windows system libs in cmake Reviewed-by: Daniel Stenberg - Closes #11988 + Closes #12307 -- tidy-up: use more example domains +Junho Choi (11 Nov 2023) - Also make use of the example TLD: - https://en.wikipedia.org/wiki/.example +- ci: test with latest quiche release (0.19.0) - Reviewed-by: Daniel Stenberg - Closes #11992 + Closes #12180 -Dan Fandrich (29 Sep 2023) +- quiche: use quiche_conn_peer_transport_params() -- runtests: display the test status if tests appear hung + In recent quiche, transport parameter API is separated + with quiche_conn_peer_transport_params(). + (https://github.com/cloudflare/quiche/pull/1575) + It breaks with bulding with latest(post 0.18.0) quiche. - It sometimes happens that a test hangs during a test run and never - returns. The test harness will wait indefinitely for the results and on - CI servers the CI job will eventually be killed after an hour or two. - At the end of a test run, if results haven't come in within a couple of - minutes, display the status of all test runners and what tests they're - running to help in debugging the problem. + Closes #12180 - This feature is really only kick in with parallel testing enabled, which - is fine because without parallel testing it's usually easy to tell what - test has hung. +Daniel Stenberg (11 Nov 2023) - Closes #11980 +- Makefile: generate the VC 14.20 project files at dist-time -- github/labeler: remove workaround for labeler + Follow-up to 28287092cc5a6d6ef8 (#12282) - This was added due to what seemed to be a bug regarding the sync-labels: - config option, but it looks like it wasn't necessary. + Closes #12290 - Follow-up to b2b0534e7 +Sam James (11 Nov 2023) -Viktor Szakats (29 Sep 2023) +- misc: fix -Walloc-size warnings -- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] + GCC 14 introduces a new -Walloc-size included in -Wextra which gives: -Daniel Stenberg (29 Sep 2023) + ``` + src/tool_operate.c: In function ‘add_per_transfer’: + src/tool_operate.c:213:5: warning: allocation of insufficient size ‘1’ fo + r type ‘struct per_transfer’ with size ‘480’ [-Walloc-size] + 213 | p = calloc(sizeof(struct per_transfer), 1); + | ^ + src/var.c: In function ‘addvariable’: + src/var.c:361:5: warning: allocation of insufficient size ‘1’ for type + struct var’ with size ‘32’ [-Walloc-size] + 361 | p = calloc(sizeof(struct var), 1); + | ^ + ``` -- docs: replace made up domains with example.com + The calloc prototype is: + ``` + void *calloc(size_t nmemb, size_t size); + ``` - in FAQ and MANUAL.md + So, just swap the number of members and size arguments to match the + prototype, as we're initialising 1 struct of size `sizeof(struct + ...)`. GCC then sees we're not doing anything wrong. - - example.com was made for this purpose. + Closes #12292 - - reduces the risk that one of those domains suddenly start hosting - something nasty and we provide links to them +Mark Gaiser (11 Nov 2023) - Closes #11986 +- IPFS: bugfixes -Michael Osipov (29 Sep 2023) + - Fixed endianness bug in gateway file parsing + - Use IPFS_PATH in tests where IPFS_DATA was used + - Fixed typos from traling -> trailing + - Fixed broken link in IPFS.md -- acinclude.m4: Document proper system truststore on FreeBSD + Follow-up to 859e88f6533f9e - The default system truststore on FreeBSD has been /etc/ssl/certs for many - years now. It is managed canonically through certctl(8) and contains hashed - symlinks for OpenSSL and other TLS providers. - The previous ones require security/ca_root_nss which might not be installed o - r - will not contain any custom CA certificates. + Reported-by: Michael Kaufmann + Bug: https://github.com/curl/curl/pull/12152#issuecomment-1798214137 + Closes #12305 - Closes #11985 +Daniel Stenberg (11 Nov 2023) -Daniel Stenberg (29 Sep 2023) +- VULN-DISCLOSURE-POLIC: remove broken link to hackerone -- FAQ: How do I upgrade curl.exe in Windows? + It should ideally soon not be done from hackerone anyway - This is a growing question, better answer it here to get somewhere to - point users to. + Closes #12308 - Closes #11984 +Andrew Kurushin (11 Nov 2023) -Viktor Szakats (28 Sep 2023) +- schannel: add CA cache support for files and memory blobs -- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC + - Support CA bundle and blob caching. - `basename` is present in mingw-w64, missing from MSVC. Pre-cache - accordingly to make configure faster. + Cache timeout is 24 hours or can be set via CURLOPT_CA_CACHE_TIMEOUT. - Notice that `basename` has a bug so we later disable it even with - mingw-w64: - https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li - b/curl_setup.h#L820-L825 + Closes https://github.com/curl/curl/pull/12261 - Closes #11974 +Daniel Stenberg (10 Nov 2023) -Daniel Stenberg (28 Sep 2023) +- RELEASE-NOTES: synced -- cmake: add missing checks +Charlie C (10 Nov 2023) - - check for arc4random. To make rand.c use it accordingly. - - check for fcntl - - fix fseek detection - - add SIZEOF_CURL_SOCKET_T - - fix USE_UNIX_SOCKETS - - define HAVE_SNPRINTF to 1 - - check for fnmatch - - check for sched_yield - - remove HAVE_GETPPID duplicate from curl_config.h - - add HAVE_SENDMSG +- cmake: option to disable install & drop `curlu` target when unused - Ref: #11964 + This patch makes the following changes: + - adds the option `CURL_DISABLE_INSTALL` - to disable 'install' targets. + - Removes the target `curlu` when the option `BUILD_TESTING` is set to + `OFF` - to prevent it from being loaded in Visual Studio. - Co-authored-by: Viktor Szakats - Closes #11973 + Closes #12287 -- configure: remove unused checks +Kai Pastor (10 Nov 2023) - - for sys/uio.h - - for fork - - for connect +- cmake: fix multiple include of CURL package - Ref: #11964 + Fixes errors on second `find_package(CURL)`. This is a frequent case + with transitive dependencies: + ``` + CMake Error at ...: + add_library cannot create ALIAS target "CURL::libcurl" because another + target with the same name already exists. + ``` - Closes #11973 + Test to reproduce: + ```cmake + cmake_minimum_required(VERSION 3.27) # must be 3.18 or higher -- lib: remove TIME_WITH_SYS_TIME + project(curl) - It is not used in any code anywhere. + set(CURL_DIR "example/lib/cmake/CURL/") + find_package(CURL CONFIG REQUIRED) + find_package(CURL CONFIG REQUIRED) # fails - Ref: #11964 - Closes #11975 + add_executable(main main.c) + target_link_libraries(main CURL::libcurl) + ``` -- docs: update curl man page references + Ref: https://cmake.org/cmake/help/latest/release/3.18.html#other-changes + Ref: https://cmake.org/cmake/help/v3.18/policy/CMP0107.html + Ref: #12300 + Assisted-by: Harry Mallon + Closes #11913 - Detected by the manpage-syntax update +Viktor Szakats (8 Nov 2023) - Closes #11963 +- tidy-up: use `OPENSSL_VERSION_NUMBER` -- manpage-syntax: verify curl man page references + Uniformly use `OPENSSL_VERSION_NUMBER` to check for OpenSSL version. + Before this patch some places used `OPENSSL_VERSION_MAJOR`. - 1. References to curl symbols are now checked that they indeed exist as - man pages. This for \f references as well as the names referenced in the - SEE ALSO section. + Also fix `lib/md4.c`, which included `opensslconf.h`, but that doesn't + define any version number in these implementations: BoringSSL, AWS-LC, + LibreSSL, wolfSSL. (Only in mainline OpenSSL/quictls). Switch that to + `opensslv.h`. This wasn't causing a deeper problem because the code is + looking for v3, which is only provided by OpenSSL/quictls as of now. - Allowlist curl.1 since it is not always built in builds + According to https://github.com/openssl/openssl/issues/17517, the macro + `OPENSSL_VERSION_NUMBER` is safe to use and not deprecated. - 2. References to curl symbols that lack section now causes warning, since tha - t - will prevent them from getting linked properly + Reviewed-by: Marcel Raad + Closes #12298 - 3. Check for "bare" references to curl functions and warn, they should be - references +Daniel Stenberg (8 Nov 2023) - Closes #11963 +- resolve.d: drop a multi use-sentence -- cmake: add check for suseconds_t - - And fix the HAVE_LONGLONG define + Since the `multi:` keyword adds that message. - Ref: #11964 - Closes #11977 + Reported-by: 積丹尼 Dan Jacobson + Fixes https://github.com/curl/curl/discussions/12294 + Closes #12295 -Viktor Szakats (28 Sep 2023) +- content_encoding: make Curl_all_content_encodings allocless -- tidy-up: whitespace fixes + - Fixes a memory leak pointed out by Coverity + - Also found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail? + id=63947 + - Avoids unncessary allocations - Closes #11972 + Follow-up ad051e1cbec68b2456a22661b -- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS + Closes #12289 - With new option `CURL_DISABLE_SRP=ON` to force-disable it. - To match existing option and detection logic in autotools. +Michael Kaufmann (7 Nov 2023) - Also: - - fix detecting GnuTLS. - We assume `nettle` as a GnuTLS dependency. - - add CMake GnuTLS CI job. - - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) - TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. - - fix compiler warning when building with GnuTLS and disabled TLS-SRP. - - fix comment typos, whitespace. +- vtls: use ALPN "http/1.1" for HTTP/1.x, including HTTP/1.0 - Ref: #11964 + Some servers don't support the ALPN protocol "http/1.0" (e.g. IIS 10), + avoid it and use "http/1.1" instead. - Closes #11967 + This reverts commit df856cb5c9 (#10183). -- tool: use our own stderr variable + Fixes #12259 + Closes #12285 - Earlier this year we changed our own stderr variable to use the standard - name `stderr` (to avoid bugs where someone is using `stderr` instead of - the curl-tool specific variable). This solution needed to override the - standard `stderr` symbol via the preprocessor. This in turn didn't play - well with unity builds and caused curl tool to crash or stay silent due - to an uninitialized stderr. This was a hard to find issue, fixed by - manually breaking out one file from the unity sources. +Daniel Stenberg (7 Nov 2023) - To avoid two these two tricks, this patch implements a different - solution: Restore using our own local variable for our stderr output and - leave `stderr` as-is. To avoid using `stderr` by mistake, add a - `checksrc` rule (based on logic we already used in lib for `strerror`) - that detects any `stderr` use in `src` and points to using our own - variable instead: `tool_stderr`. +- Makefile.am: drop vc10, vc11 and vc12 projects from dist - Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e - Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db + They are end of life products. Support for generating them remain in the + repo for a while but this change drops them from distribution. - Closes #11958 + Closes #12288 -Loïc Yhuel (28 Sep 2023) +David Suter (7 Nov 2023) -- connect: only start the happy eyeballs timer when needed +- projects: add VC14.20 project files - The timeout is only used when there is a second address family, for the - delayed eyeballer. + Windows projects included VC14, VC14.10, VC14.30 but not VC14.20. + OpenSSL and Wolf SSL scripts mention VC14.20 so I don't see a reason why + this is missing. Updated the templates to produce a VC14.20 project. + Project opens in Visual Studio 2019 as expected. - Closes #11939 + Closes #12282 -Daniel Stenberg (28 Sep 2023) +Daniel Stenberg (7 Nov 2023) -- tool_operate: free 'gateway' correctly +- curl: move IPFS code into src/tool_ipfs.[ch] - Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. + - convert ensure_trailing into ensure_trailing_slash + - strdup the URL string to own it proper + - use shorter variable names + - combine some expressions + - simplify error handling in ipfs_gateway() + - add MAX_GATEWAY_URL_LEN + proper bailout if maximum is reached + - ipfs-gateway.d polish and simplification + - shorten ipfs error message + make them "synthetic" - Also removed repeated wording in IPFS related error messages. + Closes #12281 - Closes #11969 +Viktor Szakats (6 Nov 2023) -Stefan Eissing (28 Sep 2023) +- build: delete support bits for obsolete Windows compilers -- lib: move handling of `data->req.writer_stack` into Curl_client_write() + - Pelles C: Unclear status, failed to obtain a fresh copy a few months + ago. Possible website is HTTP-only. ~10 years ago I left this compiler + dealing with crashes and other issues with no response on the forum + for years. It has seen some activity in curl back in 2021. + - LCC: Last stable release in September 2002. + - Salford C: Misses winsock2 support, possibly abandoned? Last mentioned + in 2006. + - Borland C++: We dropped Borland C++ support in 2018. + - MS Visual C++ 6.0: Released in 1998. curl already requires VS 2010 + (or possibly 2008) as a minimum. - - move definitions from content_encoding.h to sendf.h - - move create/cleanup/add code into sendf.c - - installed content_encoding writers will always be called - on Curl_client_write(CLIENTWRITE_BODY) - - Curl_client_cleanup() frees writers and tempbuffers from - paused transfers, irregardless of protocol + Closes #12222 - Closes #11908 +- build: delete `HAVE_STDINT_H` and `HAVE_INTTYPES_H` -Loïc Yhuel (28 Sep 2023) + We use `stdint.h` unconditionally in all places except one. These uses + are imposed by external dependencies / features. nghttp2, quic, wolfSSL + and `HAVE_MACH_ABSOLUTE_TIME` do require this C99 header. It means that + any of these features make curl require a C99 compiler. (In case of + MSVC, this means Visual Studio 2010 or newer.) -- multi: round the timeout up to prevent early wakeups + This patch changes the single use of `stdint.h` guarded by + `HAVE_STDINT_H` to use `stdint.h` unconditionally. Also stop using + `inttypes.h` as an alternative there. `HAVE_INTTYPES_H` wasn't used + anywhere else, allowing to delete this feature check as well. - Curl_timediff rounds down to the millisecond, so curl_multi_perform can - be called too early, then we get a timeout of 0 and call it again. + Closes #12275 - The code already handled the case of timeouts which expired less than - 1ms in the future. By rounding up, we make sure we will never ask the - platform to wake up too early. +Daniel Stenberg (6 Nov 2023) - Closes #11938 +- tool_operate: do not mix memory models -Daniel Stenberg (28 Sep 2023) + Make sure 'inputpath' only points to memory allocated by libcurl so that + curl_free works correctly. -- RELEASE-NOTES: spell out that IPFS is via gateway + Pointed out by Coverity -- RELEASE-NOTES: synced + Follow-up to 859e88f6533f9e1f890 -- tool_operate: avoid strlen() -1 on zero length content from file + Closes #12280 - Follow-up to 65b563a96a226649ba12cb1e +Stefan Eissing (6 Nov 2023) - Closes #11959 +- lib: client writer, part 2, accounting + logging -- tool_operate: fix memory mixups + This PR has these changes: - Switch to plain getenv() from curl_getenv() to avoid the allocation and - having to keep track of which free() or curl_free() that need to be - used. + Renaming of unencode_* to cwriter, e.g. client writers + - documentation of sendf.h functions + - move max decode stack checks back to content_encoding.c + - define writer phase which was used as order before + - introduce phases for monitoring inbetween decode phases + - offering default implementations for init/write/close - Coverity found issues and a memory leak. + Add type paramter to client writer's do_write() + - always pass all writes through the writer stack + - writers who only care about BODY data will pass other writes unchanged - Follow-up to 65b563a96a226649ba12cb1e + add RAW and PROTOCOL client writers + - RAW used for Curl_debug() logging of CURLINFO_DATA_IN + - PROTOCOL used for updates to data->req.bytecount, max_filesize checks and + Curl_pgrsSetDownloadCounter() + - remove all updates of data->req.bytecount and calls to + Curl_pgrsSetDownloadCounter() and Curl_debug() from other code + - adjust test457 expected output to no longer see the excess write - Closes #11959 + Closes #12184 -Viktor Szakats (27 Sep 2023) +Daniel Stenberg (6 Nov 2023) -- curl-functions.m4: fixup recent bad edits +- VULN-DISCLOSURE-POLICY: escape sequences are not a security flaw - Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + Closes #12278 - Closes #11966 +Viktor Szakats (6 Nov 2023) -Daniel Stenberg (27 Sep 2023) +- rand: fix build error with autotools + LibreSSL -- curl-functions.m4: fix include line + autotools unexpectedly detects `arc4random` because it is also looking + into dependency libs. One dependency, LibreSSL, happens to publish an + `arc4random` function (via its shared lib before v3.7, also via static + lib as of v3.8.2). When trying to use this function in `lib/rand.c`, + its protoype is missing. To fix that, curl included a prototype, but + that used a C99 type without including `stdint.h`, causing: - This made the getaddrinfo detection fail, but we did not spot it in the - CI because it graciously falled back to using legacy functions instead! + ``` + ../../lib/rand.c:37:1: error: unknown type name 'uint32_t' + 37 | uint32_t arc4random(void); + | ^ + 1 error generated. + ``` - Follow-up to 96c29900bcec (#11940) + This patch improves this by dropping the local prototype and instead + limiting `arc4random` use for non-OpenSSL builds. OpenSSL builds provide + their own random source anyway. - Closes #11965 + The better fix would be to teach autotools to not link dependency libs + while detecting `arc4random`. -- inet_ntop: add typecast to silence Coverity + LibreSSL publishing a non-namespaced `arc4random` tracked here: + https://github.com/libressl/portable/issues/928 - CID 1024653: Integer handling issues (SIGN_EXTENSION) + Regression from 755ddbe901cd0c921fbc3ac5b3775c0dc683bc73 #10672 - Suspicious implicit sign extension: "src[i]" with type "unsigned char - const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to - type "int" (32 bits, signed), then sign-extended to type "unsigned long" - (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than - 0x7FFFFFFF, the upper bits of the result will all be 1. + Reviewed-by: Daniel Stenberg + Fixes #12257 + Closes #12274 - 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); +Daniel Stenberg (5 Nov 2023) - The value will not be greater than 0x7FFFFFFF so this still cannot - happen. +- RELEASE-NOTES: synced - Also, switch to ints here instead of longs. The values stored are 16 bit - so at least no need to use 64 bit variables. Also, longs are 32 bit on - some platforms so this logic still needs to work with 32 bits. +- strdup: do Curl_strndup without strncpy - Closes #11960 + To avoid (false positive) gcc-13 compiler warnings. -- docs: adapt SEE ALSO sections to new requirements + Follow-up to 4855debd8a2c1cb - To please manpage-syntax.pl used by test 1173 + Assisted-by: Jay Satiro + Reported-by: Viktor Szakats + Fixes #12258 - Closes #11957 +Enno Boland (5 Nov 2023) -- manpage-syntax.pl: verify SEE ALSO syntax +- HTTP: fix empty-body warning - - Enforce a single reference per .BR line - - Skip the quotes around the section number for example (3) - - Insist on trailing commas on all lines except the last - - Error on comma on the last SEE ALSO entry + This change fixes a compiler warning with gcc-12.2.0 when + `-DCURL_DISABLE_BEARER_AUTH=ON` is used. - - List the entries alpha-sorted, not enforced just recommended + /home/tox/src/curl/lib/http.c: In function 'Curl_http_input_auth': + /home/tox/src/curl/lib/http.c:1147:12: warning: suggest braces around emp + ty body in an 'else' statement [-Wempty-body] + 1147 | ; + | ^ - Closes #11957 + Closes #12262 -- connect: expire the timeout when trying next +Daniel Stenberg (5 Nov 2023) - ... so that it gets called again immediately and can continue trying - addresses to connect to. Otherwise it might unnecessarily wait for a - while there. +- openssl: identify the "quictls" backend correctly - Fixes #11920 - Reported-by: Loïc Yhuel - Closes #11935 + Since vanilla OpenSSL does not support the QUIC API I think it helps + users to identify the correct OpenSSL fork in version output. The best + (crude) way to do that right now seems to be to check if ngtcp2 support + is enabled. -- http: remove wrong comment for http_should_fail + Closes #12270 - Reported-by: Christian Schmitz - Ref: #11936 - Closes #11941 +Mark Gaiser (5 Nov 2023) -Dan Fandrich (26 Sep 2023) +- curl: improved IPFS and IPNS URL support -- tool_setopt: remove unused function tool_setopt_flags + Previously just ipfs:// and ipns:// was supported, which is + too strict for some usecases. - This function is identical to tool_setopt_bitmask except that it treats - the argument as unsigned. + This patch allows paths and query arguments to be used too. + Making this work according to normal http semantics: - Closes #11943 + ipfs:///foo/bar?key=val + ipns:///foo/bar?key=val -Viktor Szakats (26 Sep 2023) + The gateway url support is changed. + It now only supports gateways in the form of: -- cmake: add feature checks for `memrchr` and `getifaddrs` + http:///foo/bar + http:// - - `HAVE_MEMRCHR` for `memrchr`. - - `HAVE_GETIFADDRS` for `getifaddrs`. - This was present in `lib/curl_config.h.cmake` but missed the detection - logic. + Query arguments here are explicitly not allowed and trigger an intended + malformed url error. - To match existing autotools feature checks. + There also was a crash when IPFS_PATH was set with a non trailing + forward slash. This has been fixed. - Closes #11954 + Lastly, a load of test cases have been added to verify the above. -- cmake: move global headers to specific checks + Reported-by: Steven Allen + Fixes #12148 + Closes #12152 - Before this patch we added standard headers unconditionally to the - global list of headers used for feature checks. This is unnecessary - and also doesn't help CMake 'Generate' performance. This patch moves - these headers to each feature check where they are actually needed. - Stop using `stddef.h`, as it seems unnecessary. +Harry Mallon (5 Nov 2023) - I've used autotools' `m4/curl-functions.m4` to figure out these - dependencies. +- docs: KNOWN_BUGS cleanup - Also delete checking for the C89 standard header `time.h`, that I - missed in the earlier commit. + * Remove other mention of hyper memory-leaks from `KNOWN_BUGS`. + Should have been removed in 629723ecf22a8eae78d64cceec2f3bdae703ec95 - Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 + * Remove mention of aws-sigv4 sort query string from `KNOWN_BUGS`. + Fixed in #11806 - Closes #11951 + * Remove mention of aws-sigv4 query empty value problems -- src/mkhelp: make generated code pass `checksrc` + * Remove mention of aws-sigv4 missing amz-content-sha256 + Fixed in #9995 - Closes #11955 +- http_aws_sigv4: canonicalise valueless query params -- tests: show which curl tool `runtests.pl` is using + Fixes #8107 + Closes #12244 - To help debugging when there is issue finding or running it. +Michael Kaufmann (4 Nov 2023) - Closes #11953 +- docs: preserve the modification date when copying the prebuilt man page -- CI/azure: make `MAKEFLAGS` global to parallelize all jobs + The previously built man page "curl.1" must be copied with the original + modification date, otherwise the man page is never updated. - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) - https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi - th -j3) + This fixes a bug that has been introduced with commit 2568441cab. - Closes #11952 + Reviewed-by: Dan Fandrich + Reviewed-by: Daniel Stenberg -- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 + Closes #12199 - Also delete an accidental variable reference. +Daniel Stenberg (4 Nov 2023) - Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 +- docs: remove bold from some man page SYNOPSIS sections - Closes #11945 + In the name of consistency -Daniel Stenberg (26 Sep 2023) + Closes #12267 -- docs: add see also curl_multi_get_handles to some man pages +- openssl: two multi pointer checks should probably rather be asserts - Assisted-by: Jay Satiro + ... so add the asserts now and consider removing the dynamic checks in a + future. - Closes #11942 + Ref: #12261 + Closes #12264 -Viktor Szakats (26 Sep 2023) +boilingoden (4 Nov 2023) -- cmake: assume `_fseeki64` and no `fseeko` on Windows +- docs: add supported version for the json write-out - `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and - at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. + xref: https://curl.se/changes.html#7_70_0 - `fseeko` is not present in any of these. + Closes #12266 - (mingw-w64 1.0 also offers `fseeko64`.) +Viktor Szakats (3 Nov 2023) - [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 +- appveyor: make VS2008-built curl tool runnable - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 + By linking the CRT statically. This avoids the error about missing + runtime DLL `MSVCR90.dll` when running the freshly built `curl.exe`. - Closes #11950 + Closes #12263 -- build: delete checks for C89 standard headers +Stefan Eissing (3 Nov 2023) - Delete checks and guards for standard C89 headers and assume these are - available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, - `stddef.h`, `signal.h`. +- url: proxy ssl connection reuse fix - Some of these we already used unconditionally, some others we only used - for feature checks. + - tunnel https proxy used for http: transfers does + no check if proxy-ssl configuration matches + - test cases added, test_10_12 fails on 8.4.0 - Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i - n CMake) + Closes #12255 - Closes #11940 +Jay Satiro (3 Nov 2023) -Stefan Eissing (26 Sep 2023) +- curl_sspi: support more revocation error names in error messages -- multiif.h: remove Curl_multi_dump declaration + - Add these revocation errors to sspi error list: + CRYPT_E_NO_REVOCATION_DLL, CRYPT_E_NO_REVOCATION_CHECK, + CRYPT_E_REVOCATION_OFFLINE and CRYPT_E_NOT_IN_REVOCATION_DATABASE. - Follow-up to d850eea2 which removed the Curl_multi_dump definition. + Prior to this change those error codes were not matched to their macro + name and instead shown as "unknown error". - Closes https://github.com/curl/curl/pull/11946 + Before: -Jay Satiro (26 Sep 2023) + schannel: next InitializeSecurityContext failed: + Unknown error (0x80092013) - The revocation function was + unable to check revocation because the revocation server was offline. -- config-win32: define HAVE__FSEEKI64 + After: - Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that - calls _fseeki64 if it is available. + schannel: next InitializeSecurityContext failed: + CRYPT_E_REVOCATION_OFFLINE (0x80092013) - The revocation function was + unable to check revocation because the revocation server was offline. - Closes https://github.com/curl/curl/pull/11944 + Bug: https://github.com/curl/curl/issues/12239 + Reported-by: Niracler Li -- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER + Closes https://github.com/curl/curl/pull/12241 - - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place - even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. +- strdup: don't allow Curl_strndup to read past a null terminator - The behavior is verified by test2048. + - Use malloc + strncpy instead of Curl_memdup to dupe the string before + null terminating it. - Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 - Reported-by: claudiusaiz@users.noreply.github.com + Prior to this change if Curl_strndup was passed a length longer than + the allocated string then it could copy out of bounds. - Bug: https://github.com/curl/curl/discussions/11910 - Reported-by: Hakan Sunay Halil + This change is for posterity. Curl_strndup was added in the parent + commit and currently none of the calls to it pass a length that would + cause it to read past the allocated length of the input. - Closes https://github.com/curl/curl/pull/11930 + Follow-up to d3b3ba35. -Stefan Eissing (26 Sep 2023) + Closes https://github.com/curl/curl/pull/12254 -- openssl: improve ssl shutdown handling +Daniel Stenberg (2 Nov 2023) - - If SSL shutdown is not finished then make an additional call to - SSL_read to gather additional tracing. +- lib: add and use Curl_strndup() - - Fix http2 and h2-proxy filters to forward do_close() calls to the next - filter. + The Curl_strndup() function is similar to memdup(), but copies 'n' bytes + then adds a terminating null byte ('\0'). - For example h2 and SSL shutdown before and after this change: + Closes #12251 - Before: +- CURPOST_POSTFIELDS.3: add CURLOPT_COPYPOSTFIELDS in SEE ALSO - Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> - ssl_cf_destroy +Stefan Eissing (2 Nov 2023) - After: +- pytest: use lower count in repeat tests - Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> - ssl_cf_close + - lower large iteration counts in some tests somewhat for + the same coverage with less duration - Note that currently the tracing does not show output on the connection - closure handle. Refer to discussion in #11878. + Closes #12248 - Ref: https://github.com/curl/curl/discussions/11878 +Daniel Stenberg (2 Nov 2023) - Closes https://github.com/curl/curl/pull/11858 +- RELEASE-NOTES: synced -Loïc Yhuel (26 Sep 2023) +- docs: clarify that curl passes on input unfiltered -- multi: fix small timeouts + ... for several options. - Since Curl_timediff rounds down to the millisecond, timeouts which - expire in less than 1ms are considered as outdated and removed from the - list. We can use Curl_timediff_us instead, big timeouts could saturate - but this is not an issue. + Reported-by: Ophir Lojkine - Closes #11937 + Closes #12249 -Viktor Szakats (25 Sep 2023) +- urlapi: when URL encoding the fragment, pass in the right length -- cmake: fix stderr initialization in unity builds + A benign bug because it would only add an extra null terminator. - Before this patch, in certain build configurations the curl tool may - not have displayed anything (debug, macOS), or crashed at startup - (debug, Windows). + Made lib1560 get a test that runs this code. - Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 - Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db + Closes #12250 - Closes #11929 +Stefan Eissing (2 Nov 2023) -- cmake: fix missing `zlib.h` when compiling `libcurltool` +- vtls: late clone of connection ssl config - Came up while testing debug/testing build for Windows. I'm not sure why - it didn't come up in earlier tests with similar config. - `tool_hugehelp.c` might indeed require `zlib.h` and without linking - `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected - dependency headers to the compiler command. + - perform connection cache matching against `data->set.ssl.primary` + and proxy counterpart + - fully clone connection ssl config only when connection is used - ``` - [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang - --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 - 6_64 - -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO - DE - -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib - -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li - ne-argument - -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static - -libgcc - -lucrt [...] -O3 -DNDEBUG -municode -MD - -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj - -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d - -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x - 64/src/tool_hugehelp.c - .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f - ile not found - 6 | #include - | ^~~~~~~~ - ``` + Closes #12237 - Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 +- msh3: error when built with CURL_DISABLE_SOCKETPAIR set - Closes #11927 + Reported-by: Gisle Vanem + Closes #12252 + Fixes #12213 -- cmake: fix duplicate symbols when linking tests +Daniel Stenberg (2 Nov 2023) - The linker resolves this automatically in non-unity builds. In unity - builds the linker cannot drop a single object with the duplicates, - resulting in these errors. The root issue is that we started including - certain objects both via both libcurlu and libcurltool libs. +- hsts: skip single-dot hostname - Regression from 39e7c22bb459c2e818f079984989a26a09741860 + Reported-by: Maksymilian Arciemowicz - Windows errors: - ``` - [ 3%] Linking C executable unit1303.exe - [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul - tibyte.c.obj - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch - ar': - C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve - rt_UTF8_to_wchar' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:44: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT - F8': - C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve - rt_wchar_to_UTF8' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:66: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': - C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 - _open' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:92: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': - C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 - 2_fopen' - ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. - c:120: first defined here - ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': - [...] - ``` - Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 - aa4ehny5q#L247 + Closes #12247 - macOS errors: - ``` - [ 56%] Linking C executable unit1302 - duplicate symbol '_curlx_sotouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_sitouz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - duplicate symbol '_curlx_uztosz' in: - ../../lib/libcurlu.a(unity_0_c.c.o) - ../../src/libcurltool.a(unity_0_c.c.o) - [...] - ``` - with config: - ``` - -DCMAKE_UNITY_BUILD=ON \ - -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ - -DBUILD_SHARED_LIBS=ON \ - -DBUILD_STATIC_LIBS=OFF - ``` +- vtls: fix build without proxy - Closes #11926 + Follow-up to bf0e278a3c54bc7fee7360da17c -- cmake: lib `CURL_STATICLIB` fixes (Windows) + closes #12243 - - always define `CURL_STATICLIB` when building libcurl for Windows. +- docs/example/keepalive.c: show TCP keep-alive options - This disables `__declspec(dllexport)` for exported libcurl symbols. - In normal mode (hide symbols) these exported symbols are specified - via `libcurl.def`. When not hiding symbols, all symbols are exported - by default. + Closes #12242 - Regression from 1199308dbc902c52be67fc805c72dd2582520d30 +- lib1560: verify appending blank URL encoded query string - Fixes #11844 +- urlapi: skip appending NULL pointer query - - fix to omit `libcurl.def` when not hiding private symbols. + Reported-by: kirbyn17 on hackerone - Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 + Closes #12240 - - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also - omitting `libcurl.def` in this case, and exporting all symbols - instead. This ensures that a shared curl tool can access all debug - functions which are not normally exported from libcurl DLL. +- lib1560: verify setting host to "" with and without URL encode - - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" - target. +- urlapi: avoid null deref if setting blank host to url encode - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 + Reported-by: kirbyn17 on hackerone - - delete duplicate `BUILDING_LIBCURL` definitions. + Closes #12240 - - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. +- dynbuf: assert for NULL pointer inputs - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 + Help us catch more mistakes. - Closes #11914 + Closes #12238 -Daniel Stenberg (25 Sep 2023) +- HTTP3: ngtcp2 builds are no longer experimental -- RELEASE-NOTES: synced + The other HTTP/3 backends are still experimental. -Dan Fandrich (25 Sep 2023) + Closes #12235 -- tests: fix log directory path in IPFS tests +Stefan Eissing (31 Oct 2023) - Hard-coding the log directory name fails with parallel tests. +- vtls: cleanup SSL config management - Follow-up to 65b563a96 + - remove `Curl_ssl_get_config()`, no longer needed - Ref: #8805 + Closes #12204 -Daniel Stenberg (25 Sep 2023) +Daniel Stenberg (31 Oct 2023) -- curl_multi_get_handles: get easy handles from a multi handle +- libcurl-thread.3: simplify the TLS section - Closes #11750 + All TLS libraries curl can use are threadsafe since OpenSSL 1.1.x, August + 2016. -Stefan Eissing (25 Sep 2023) + Closes #12233 -- http: h1/h2 proxy unification +- configure: better --disable-http - - use shared code for setting up the CONNECT request - when tunneling, used in HTTP/1.x and HTTP/2 proxying - - eliminate use of Curl_buffer_send() and other manipulations - of `data->req` or `data->state.ulbuf` + - disable HTTPS-proxy as well, since it can't work without HTTP - Closes #11808 + - curl_setup: when HTTP is disabled, also disable all features that are + HTTP-only -Natanael Copa (25 Sep 2023) + - version: HTTPS-proxy only exists if HTTP support exists -- lib: use wrapper for curl_mime_data fseek callback + Closes #12223 - fseek uses long offset which does not match with curl_off_t. This leads - to undefined behavior when calling the callback and caused failure on - arm 32 bit. +- http: consider resume with CURLOPT_FAILONERRROR and 416 to be fine - Use a wrapper to solve this and use fseeko which uses off_t instead of - long. + Finding a 'Content-Range:' in the response changed the handling. - Thanks to the nice people at Libera IRC #musl for helping finding this - out. + Add test case 1475 to verify -C - with 416 and Content-Range: header, + which is almost exactly like test 194 which instead uses a fixed -C + offset. Adjusted test 194 to also be considered fine. - Fixes #11882 - Fixes #11900 - Closes #11918 + Fixes #10521 + Reported-by: Smackd0wn + Fixes #12174 + Reported-by: Anubhav Rai + Closes #12176 -- configure: sort AC_CHECK_FUNCS +Stefan Eissing (30 Oct 2023) - No functional changes. +- GHA: fix checkout of quictls repository to use correct branch name -Daniel Stenberg (25 Sep 2023) + Follow-up to c868b0e30f10cd0ac7 -- warnless: remove unused functions + Closes #12232 - Previously put there for use with the intel compiler +Daniel Stenberg (30 Oct 2023) - Closes #11932 +- docs/example/localport.c: show off CURLOPT_LOCALPORT -- GHA/linux: run singleuse to detect single-use global functions + Closes #12230 - Use --unit for configure --enable-debug builds +- docs/examples/interface.c: show CURLOPT_INTERFACE use - Closes #11932 + Although super simple. -- singleuse: add scan for use in other source codes + Closes #12229 - This should reduce false-positive to almost zero. Checks for presence in - unit tests if --unit is specified, which is intended for debug builds - where unit testing is enabled. +Viktor Szakats (30 Oct 2023) - Closes #11932 +- build: fix compiler warning with auths disabled -- multi: remove Curl_multi_dump + ``` + ./curl/lib/http.c:979:12: warning: unused function 'is_valid_auth_separator' + [-Wunused-function] + static int is_valid_auth_separator(char ch) + ^ + 5 warnings generated. + ``` - A debug-only function that is basically never used. Removed to ease the - use of the singleuse script to detect non-static functions not used - outside the file where it is defined. + Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 - Closes #11931 + Closes #12227 -Viktor Szakats (24 Sep 2023) +- build: require Windows XP or newer -- tests: fix compiler warnings + After this patch we assume availability of `getaddrinfo` and + `freeaddrinfo`, first introduced in Windows XP. Meaning curl + now requires building for Windows XP as a minimum. - Seen with llvm 17 on Windows x64. + TODO: assume these also in autotools. - ``` - .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration - for non-static variable 'logdir' [-Wmissing-variable-declarations] - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 136 | const char *logdir = "log"; - | ^ - .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration - for non-static variable 'loglockfile' [-Wmissing-variable-declarations] - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i - s not intended to be used outside of this translation unit - 137 | char loglockfile[256]; - | ^ - .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat - ion for non-static variable 'logdir' [-Wmissing-variable-declarations] - 43 | const char *logdir = "log"; - | ^ - .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl - e is not intended to be used outside of this translation unit - 43 | const char *logdir = "log"; - | ^ - .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator - here [-Wcomma] - 350 | ++d, ++s; - | ^ - .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa - rning - 350 | ++d, ++s; - | ^~~ - | (void)( ) - ``` + Ref: https://github.com/curl/curl/pull/12221#issuecomment-1783761806 + Closes #12225 - ``` - .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; - | ~ ^ ~~~~~~~~~~~~~ - 1 warning generated. +- appveyor: bump one job to OpenSSL 3.1 (was 1.1.1) - .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison - 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] - 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. + Use 3.1 with the modern runner image. - .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' - > 2147483647 is always false [-Wtautological-type-limit-compare] - 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ - 1 warning generated. - .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > - 2147483647 is always false [-Wtautological-type-limit-compare] - 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo - ut; - | ~~~~~~~ ^ ~~~~~~~~~~~~~ + We still use 1.1.1 in 8 jobs. + + 1.1.1 is EOL since 2023-09-11: + https://www.openssl.org/blog/blog/2023/03/28/1.1.1-EOL/ + + Also: + - add missing SSL-backend to job descriptions. + - tidy up CPU in job descriptions. + + Closes #12226 + +Daniel Stenberg (30 Oct 2023) + +- RELEASE-NOTES: synced + +- GHA: bump ngtcp2, nghttp3, nghttp2 and quictls versions + + ngtcp2 1.0.1 + nghttp3 1.0.0 + nghttp2 1.58.0 + quictls 3.1.4+quic + + also sync HTTP3.md with these changes + + Closes #12132 + +Kareem (29 Oct 2023) + +- wolfssl: add default case for wolfssl_connect_step1 switch + + Closes #12218 + +Jay Satiro (29 Oct 2023) + +- curl_setup: disallow Windows IPv6 builds missing getaddrinfo + + - On Windows if IPv6 is enabled but getaddrinfo is missing then #error + the build. + + curl can be built with IPv6 support (ENABLE_IPV6) but without the + ability to resolve hosts to IPv6 addresses (HAVE_GETADDRINFO). On + Windows this is highly unlikely and should be considered a bad build + configuration. + + Such a bad configuration has already given us a bug that was hard to + diagnose. See #12134 and #12136 for discussion. + + Ref: https://github.com/curl/curl/issues/12134 + Ref: https://github.com/curl/curl/pull/12136 + + Closes https://github.com/curl/curl/pull/12221 + +Nico Rieck (29 Oct 2023) + +- openssl: make CURLSSLOPT_NATIVE_CA import Windows intermediate CAs + + - If CURLSSLOPT_NATIVE_CA on Windows then import from intermediate CA + "CA" store after importing from root CA "ROOT" store. + + This change allows curl to work in situations where a server does not + send all intermediate certs and they are present in the "CA" store (the + store with intermediate CAs). This is already allowed by the Schannel + backend. + + Also this change makes partial chain verification possible for those + certs since we allow partial chain verification by default for OpenSSL + (unless CURLSSLOPT_NO_PARTIALCHAIN). This is not allowed by the Schannel + backend. + + Prior to this change CURLSSLOPT_NATIVE_CA only imported "ROOT" certs. + + Fixes https://github.com/curl/curl/issues/12155 + Closes https://github.com/curl/curl/pull/12185 + +Viktor Szakats (28 Oct 2023) + +- Makefile.mk: fix `-rtmp` option for non-Windows [ci skip] + +Daniel Stenberg (28 Oct 2023) + +- asyn-ares: handle no connection in the addrinfo callback + + To avoid crashing. + + Follow-up from 56a4db2 + Closes #12219 + +Jay Satiro (28 Oct 2023) + +- hostip6: fix DEBUG_ADDRINFO builds + + - Removed unused and incorrect parameter from dump_addrinfo(). + + Bug: https://github.com/curl/curl/commit/56a4db2e#commitcomment-131050442 + Reported-by: Gisle Vanem + + Closes https://github.com/curl/curl/pull/12212 + +Viktor Szakats (28 Oct 2023) + +- Makefile.mk: restore `_mingw.h` for default `_WIN32_WINNT` + + In 8.4.0 we deleted `_mingw.h` as part of purging old-mingw support. + Turns out `_mingw.h` had the side-effect of setting a default + `_WIN32_WINNT` value expected by `lib/config-win32.h` to enable + `getaddrinfo` support in `Makefile.mk` mingw-w64 builds. This caused + disabling support for this unless specifying the value manually. + + Restore this header and update its comment to tell why we continue + to need it. + + This triggered a regression in official Windows curl builds starting + with 8.4.0_1. Fixed in 8.4.0_6. (8.5.0 will be using CMake.) + + Regression from 38029101e2d78ba125732b3bab6ec267b80a0e72 #11625 + + Reported-by: zhengqwe on github + Helped-by: Nico Rieck + Fixes #12134 + Fixes #12136 + Closes #12217 + +- hostip: silence compiler warning `-Wparentheses-equality` + + Seen with LLVM 17. + + ``` + hostip.c:1336:22: warning: equality comparison with extraneous parentheses [- + Wparentheses-equality] + 1336 | (a->ai_family == PF_INET)) { + | ~~~~~~~~~~~~~^~~~~~~~~~ + hostip.c:1336:22: note: remove extraneous parentheses around the comparison t + o silence this warning + 1336 | (a->ai_family == PF_INET)) { + | ~ ^ ~ + hostip.c:1336:22: note: use '=' to turn this equality comparison into an assi + gnment + 1336 | (a->ai_family == PF_INET)) { + | ^~ + | = 1 warning generated. ``` - Seen on macOS Intel: - ``` - .../curl/tests/server/sws.c:440:64: warning: field precision should have type - 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] - msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" - , - ~~^~ - 1 warning generated. - ``` + Follow-up to b651aba0962bb31353f55de4dc35f745952a1b10 #12145 + + Reviewed-by: Daniel Stenberg + Closes #12215 + +Stefan Eissing (27 Oct 2023) + +- doh: use PIPEWAIT when HTTP/2 is attempted + + Closes #12214 + +Daniel Stenberg (27 Oct 2023) + +- setopt: remove outdated cookie comment + + Closes #12206 + +Stefan Eissing (27 Oct 2023) + +- cfilter: provide call to tell connection to forget a socket + + - fixed libssh.c workaround for a socket being closed by + the library + - eliminate the terrible hack in cf-socket.c to guess when + this happened and try not closing the socket again. + - fixes race in eyeballing when socket could have failed to + be closed for a discarded connect attempt + + Closes #12207 + +- url: protocol handler lookup tidy-up + + - rename lookup to what it does + - use ARRAYSIZE instead of NULL check for end + - offer alternate lookup for 0-terminated strings + + Closes #12216 + +Viktor Szakats (27 Oct 2023) + +- build: variadic macro tidy-ups + + - delete unused `HAVE_VARIADIC_MACROS_C99/GCC` feature checks. + (both autotools and CMake.) + - delete duplicate `NULL` check in `Curl_trc_cf_infof()`. + - fix compiler warning in `CURL_DISABLE_VERBOSE_STRINGS` builds. + ``` + ./lib/cf-socket.c:122:41: warning: unused parameter 'data' [-Wunused-parame + ter] + static void nosigpipe(struct Curl_easy *data, + ^ + ``` + - fix `#ifdef` comments in `lib/curl_trc.{c,h}`. + - fix indentation in some `infof()` calls. + + Follow-up to dac293cfb7026b1ca4175d88b80f1432d3d3c684 #12167 + + Cherry-picked from #12105 + Closes #12210 + +- cmake: speed up threads setup for Windows + + Win32 threads are always available. We enabled them unconditionally + (with `ENABLE_THREADED_RESOLVER`). CMake built-in thread detection + logic has this condition hard-coded for Windows as well (since at least + 2007). + + Instead of doing all the work of detecting pthread combinations on + Windows, then discarding those results, skip these efforts and assume + built-in thread support when building for Windows. + + This saves 1-3 slow CMake configuration steps. + + Reviewed-by: Daniel Stenberg + Closes #12202 + +- cmake: speed up zstd detection + + Before this patch we detected the presence of a specific zstd API to + see if we can use the library. zstd published that API in its first + stable release: v1.0.0 (2016-08-31). + + Replace that method by detecting the zstd library version instead and + accepting if it's v1.0.0 or newer. Also display this detected version + and display a warning if the zstd found is unfit for curl. + + We use the same version detection method as zstd itself, via its public + C header. + + This deviates from autotools which keeps using the slow method of + looking for the API by building a test program. The outcome is the same + as long as zstd keeps offering this API. + + Ref: https://github.com/facebook/zstd/commit/5a0c8e24395079f8e8cdc90aa1659cd5 + ab1b7427 (2016-08-12, committed) + Ref: https://github.com/facebook/zstd/releases/tag/v0.8.1 (2016-08-18, first + released) + Ref: https://github.com/facebook/zstd/releases/tag/v1.0.0 + + Reviewed-by: Daniel Stenberg + Closes #12200 + +Daniel Stenberg (26 Oct 2023) + +- openssl: fix infof() to avoid compiler warning for %s with null + + vtls/openssl.c: In function ‘ossl_connect_step2’: + ../lib/curl_trc.h:120:10: error: ‘%s’ directive argument is null [-Werror + =format-overflow=] + 120 | Curl_infof(data, __VA_ARGS__); } while(0) + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + vtls/openssl.c:4008:5: note: in expansion of macro ‘infof’ + 4008 | infof(data, "SSL connection using %s / %s / %s / %s", + | ^~~~~ + vtls/openssl.c:4008:49: note: format string is defined here + 4008 | infof(data, "SSL connection using %s / %s / %s / %s", + | ^~ + + Follow-up to b6e6d4ff8f253c8b8055bab + Closes #12196 + +Stefan Eissing (26 Oct 2023) + +- lib: apache style infof and trace macros/functions + + - test for a simplified C99 variadic check + - args to infof() in --disable-verbose are no longer disregarded but + must compile. + + Closes #12167 + Fixes #12083 + Fixes #11880 + Fixes #11891 + +Daniel Stenberg (26 Oct 2023) + +- RELEASE-NOTES: synced + +Stefan Eissing (26 Oct 2023) + +- urldata: move async resolver state from easy handle to connectdata + + - resolving is done for a connection, not for every transfer + - save create/dup/free of a cares channel for each transfer + - check values of setopt calls against a local channel if no + connection has been attached yet, when needed. + + Closes #12198 + +Daniel Stenberg (26 Oct 2023) + +- CURLOPT_WRITEFUNCTION.3: clarify what libcurl returns for CURL_WRITEFUNC_ERRO + R + + It returns CURLE_WRITE_ERROR. It was not previously stated clearly. + + Reported-by: enWILLYado on github + Fixes #12201 + Closes #12203 + +Viktor Szakats (25 Oct 2023) + +- autotools: update references to deleted `crypt-auth` option + + Delete leftovers of the `crypt-auth` `./configure` option and + add the new ones that replaced them. + + Follow-up to e92edfbef64448ef461117769881f3ed776dec4e #11490 + + Reviewed-by: Daniel Stenberg + Closes #12194 + +Stefan Eissing (25 Oct 2023) + +- lib: introduce struct easy_poll_set for poll information + + Connection filter had a `get_select_socks()` method, inspired by the + various `getsocks` functions involved during the lifetime of a + transfer. These, depending on transfer state (CONNECT/DO/DONE/ etc.), + return sockets to monitor and flag if this shall be done for POLLIN + and/or POLLOUT. + + Due to this design, sockets and flags could only be added, not + removed. This led to problems in filters like HTTP/2 where flow control + prohibits the sending of data until the peer increases the flow + window. The general transfer loop wants to write, adds POLLOUT, the + socket is writeable but no data can be written. + + This leads to cpu busy loops. To prevent that, HTTP/2 did set the + `SEND_HOLD` flag of such a blocked transfer, so the transfer loop cedes + further attempts. This works if only one such filter is involved. If a + HTTP/2 transfer goes through a HTTP/2 proxy, two filters are + setting/clearing this flag and may step on each other's toes. + + Connection filters `get_select_socks()` is replaced by + `adjust_pollset()`. They get passed a `struct easy_pollset` that keeps + up to `MAX_SOCKSPEREASYHANDLE` sockets and their `POLLIN|POLLOUT` + flags. This struct is initialized in `multi_getsock()` by calling the + various `getsocks()` implementations based on transfer state, as before. + + After protocol handlers/transfer loop have set the sockets and flags + they want, the `easy_pollset` is *always* passed to the filters. Filters + "higher" in the chain are called first, starting at the first + not-yet-connection one. Each filter may add sockets and/or change + flags. When all flags are removed, the socket itself is removed from the + pollset. + + Example: - Closes #11925 + * transfer wants to send, adds POLLOUT + * http/2 filter has a flow control block, removes POLLOUT and adds + POLLIN (it is waiting on a WINDOW_UPDATE from the server) + * TLS filter is connected and changes nothing + * h2-proxy filter also has a flow control block on its tunnel stream, + removes POLLOUT and adds POLLIN also. + * socket filter is connected and changes nothing + * The resulting pollset is then mixed together with all other transfers + and their pollsets, just as before. -Jay Satiro (24 Sep 2023) + Use of `SEND_HOLD` is no longer necessary in the filters. -- url: fix netrc info message + All filters are adapted for the changed method. The handling in + `multi.c` has been adjusted, but its state handling the the protocol + handlers' `getsocks` method are untouched. - - Fix netrc info message to use the generic ".netrc" filename if the - user did not specify a netrc location. + The most affected filters are http/2, ngtcp2, quiche and h2-proxy. TLS + filters needed to be adjusted for the connecting handshake read/write + handling. - - Update --netrc doc to add that recent versions of curl on Windows - prefer .netrc over _netrc. + No noticeable difference in performance was detected in local scorecard + runs. - Before: - * Couldn't find host google.com in the (nil) file; using defaults + Closes #11833 - After: - * Couldn't find host google.com in the .netrc file; using defaults +Daniel Stenberg (25 Oct 2023) - Closes https://github.com/curl/curl/pull/11904 +- tests/README: SOCKS tests are not using OpenSSH, it has its own server -Dan Fandrich (23 Sep 2023) + Follow-up to 04fd67555cc -- wolfssh: do cleanup in Curl_ssh_cleanup + Closes #12195 - Closes: #11921 +Jacob Hoffman-Andrews (25 Oct 2023) -Daniel Stenberg (24 Sep 2023) +- tets: make test documentation more user-friendly -- tool_listhelp: regenerated + Put the instructions to run tests right at the top of tests/README.md. - Polished the --ipfs-gateway description + Give instructions to read the runtests.1 man page for information + about flags. Delete redundant copy of the flags documentation in the + README. - Fixed the --trace-config description + Add a mention in README.md of the important parallelism flag, to make + test runs go much faster. - The script also fixed some other small mistakes + Move documentation of output line format into the runtests.1 man page, + and update it with missing flags. - Closes #11923 + Fix the order of two flags in the man page. -Viktor Szakats (23 Sep 2023) + Closes #12193 -- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) +Viktor Szakats (24 Oct 2023) - Also fix to export all symbols in Windows debug builds, making - `-debug-dyn` builds work with `-DCURL_STATICLIB` set. +- cmake: pre-fill rest of detection values for Windows - Ref: https://github.com/curl/curl/pull/11914 (same for CMake) + The goal of this patch is to avoid unnecessary feature detection work + when doing Windows builds with CMake. Do this by pre-filling well-known + detection results for Windows and specifically for mingw-w64 and MSVC + compilers. Also limit feature checks to platforms where the results are + actually used. Drop a few redundant ones. And some tidying up. - Closes #11924 + - pre-fill remaining detection values in Windows CMake builds. -Daniel Stenberg (23 Sep 2023) + Based on actual detection results observed in CI runs, preceding + similar work over libssh2 and matching up values with + `lib/config-win32.h`. -- quic: set ciphers/curves the same way regular TLS does + This brings down CMake configuration time from 58 to 14 seconds on the + same local machine. - for OpenSSL/BoringSSL + On AppVeyor CI this translates to: + - 128 seconds -> 50 seconds VS2022 MSVC with OpenSSL (per CMake job): + https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/4gw66ecr + jpy7necb#L296 + https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/8m4fwrr2 + fe249uo8#L186 + - 62 seconds -> 16 seconds VS2017 MINGW (per CMake job): + https://ci.appveyor.com/project/curlorg/curl/builds/48208419/job/s1y8q5iv + lcs7ub29?fullLog=true#L290 + https://ci.appveyor.com/project/curlorg/curl/builds/48217440/job/pchpxyjs + yc9kl13a?fullLog=true#L194 - Fixes #11796 - Reported-by: Karthikdasari0423 on github - Assisted-by: Jay Satiro - Closes #11836 + The formula is about 1-3 seconds delay for each detection. Almost all + of these trigger a full compile-link cycle behind the scenes, slow + even today, both cross and native, mingw-w64 and apparently MSVC too. + Enabling .map files or other custom build features slows it down + further. (Similar is expected for autotools configure.) -- test457: verify --max-filesize with chunked encoding + - stop detecting `idn2.h` if idn2 was deselected. + autotools does this. -- lib: let the max filesize option stop too big transfers too + - stop detecting `idn2.h` if idn2 was not found. + This deviates from autotools. Source code requires both header and + lib, so this is still correct, but faster. - Previously it would only stop them from getting started if the size is - known to be too big then. + - limit `ADDRESS_FAMILY` detection to Windows. - Update the libcurl and curl docs accordingly. + - normalize `HAVE_WIN32_WINNT` value to lowercase `0x0a12` format. - Fixes #11810 - Reported-by: Elliot Killick - Assisted-by: Jay Satiro - Closes #11820 + - pre-fill `HAVE_WIN32_WINNT`-dependent detection results. + Saving 4 (slow) feature-detections in most builds: `getaddrinfo`, + `freeaddrinfo`, `inet_ntop`, `inet_pton` -Viktor Szakats (23 Sep 2023) + - fix pre-filled `HAVE_SYS_TIME_H`, `HAVE_SYS_PARAM_H`, + `HAVE_GETTIMEOFDAY` for mingw-w64. + Luckily this do not change build results, as `WIN32` took + priority over `HAVE_GETTIMEOFDAY` with the current source + code. -- mingw: delete support for legacy mingw.org toolchain + - limit `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` and + `HAVE_CLOCK_GETTIME_MONOTONIC` detections to non-Windows. + We're not using these in the source code for Windows. - Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: - https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ - Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. - It supported the x86 CPU only and used a old Windows API header and - implib set, often causing issues. It also misses most modern Windows - features, offering old versions of both binutils and gcc (no llvm/clang - support). It was last updated 2 years ago. + - reduce compiler warning noise in CMake internal logs: + - fix to include `winsock2.h` before `windows.h`. + Apply it to autotools test snippets too. + - delete previous `-D_WINSOCKAPI_=` hack that aimed to fix the above. + - cleanup `CMake/CurlTests.c` to emit less warnings. - curl now relies on toolchains based on the mingw-w64 project: - https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ - https://www.msys2.org/ https://github.com/msys2/msys2 - https://github.com/mstorsjo/llvm-mingw - (Also available via Linux and macOS package managers.) + - delete redundant `HAVE_MACRO_SIGSETJMP` feature check. + It was the same check as `HAVE_SIGSETJMP`. - Closes #11625 + - delete 'experimental' marking from `CURL_USE_OPENSSL`. -Mark Gaiser (23 Sep 2023) + - show CMake version via `CMakeLists.txt`. + Credit to the `zlib-ng` project for the idea: + https://github.com/zlib-ng/zlib-ng/blob/61e181c8ae93dbf56040336179c9954078b + d1399/CMakeLists.txt#L7 -- curl: add support for the IPFS protocols: + - make `CMake/CurlTests.c` pass `checksrc`. - - ipfs:// - - ipns:// + - `CMake/WindowsCache.cmake` tidy-ups. - This allows you tu use ipfs in curl like: - curl ipfs:// - and - curl ipns:// + - replace `WIN32` guard with `_WIN32` in `CMake/CurlTests.c`. - For more information consult the readme at: - https://curl.se/docs/ipfs.html + Closes #12044 - Closes #8805 +Jay Satiro (24 Oct 2023) -Daniel Stenberg (23 Sep 2023) +- page-footer: clarify exit code 25 -- bufq: remove Curl_bufq_skip_and_shift (unused) + - Clarify that curl tool exit code 25 means an upload failed to start. - Closes #11915 + Exit code 25 is equivalent to CURLE_UPLOAD_FAILED (25). Prior to this + change the documentation only mentioned the case of FTP STOR failing. -- scripts/singleuse.pl: add curl_global_trace + Reported-by: Emanuele Torre -Viktor Szakats (22 Sep 2023) + Ref: https://github.com/curl/curl/blob/curl-8_4_0/docs/libcurl/libcurl-errors + .3#L113-L115 -- cmake: fix unity symbol collisions in h2 builds + Fixes https://github.com/curl/curl/issues/12189 + Closes https://github.com/curl/curl/pull/12190 - Regression from 331b89a319d0067fa1e6441719307cfef9c7960f +Daniel Stenberg (24 Oct 2023) - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Closes #11912 +- scripts/cijobs.pl: adjust for appveyor -Daniel Stenberg (22 Sep 2023) + Follow-up to a1d73a6bb -- RELEASE-NOTES: synced +Alex Bozarth (24 Oct 2023) -Dan Fandrich (21 Sep 2023) +- OpenSSL: Include SIG and KEM algorithms in verbose -- github/labeler: improve the match patterns + Currently the verbose output does not include which algorithms are used + for the signature and key exchange when using OpenSSL. Including the + algorithms used will enable better debugging when working on using new + algorithm implementations. Know what algorithms are used has become more + important with the fast growing research into new quantum-safe + algorithms. - This includes new rules for setting the appleOS and logging labels and - matches on some example files. Also, enable dot mode for wildcard - matches in the .github directory. + This implementation includes a build time check for the OpenSSL version + to use a new function that will be included in OpenSSL 3.2 that was + introduced in openssl/openssl@6866824 -Daniel Stenberg (21 Sep 2023) + Based-on-patch-by: Martin Schmatz + Closes #12030 -- upload-file.d: describe the file name slash/backslash handling +Daniel Stenberg (23 Oct 2023) - Closes #11911 +- http2: provide an error callback and failf the message -Jakub Jelen (21 Sep 2023) + Getting nghttp2's error message helps users understand what's going + on. For example when the connection is brought down due a forbidden + header is used - as that header is then not displayed by curl itself. -- libssh: cap SFTP packet size sent + Example: - Due to libssh limitations + curl: (92) Invalid HTTP header field was received: frame type: 1, + stream: 1, name: [upgrade], value: [h2,h2c] - Signed-off-by: Jakub Jelen + Ref: #12172 + Closes #12179 - Closes #11804 +Turiiya (23 Oct 2023) -Daniel Stenberg (21 Sep 2023) +- BINDINGS: add V binding -- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 + Closes #12182 - Closes #11905 +Daniel Stenberg (22 Oct 2023) -- mailmap: unify Michael Osipov under a single email +- configure: check for the fseeko declaration too -Ted Lyngmo (21 Sep 2023) + ... and make the code require both symbol and declaration. -- docs: use CURLSSLBACKEND_NONE + This is because for Android, the symbol is always present in the lib at + build-time even when not actually available in run-time. - [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in - documentation and examples. + Assisted-by: Viktor Szakats + Reported-by: 12932 on github + Fixes #12086 + Closes #12158 - Signed-off-by: Ted Lyngmo +Viktor Szakats (22 Oct 2023) - Closes #11909 +- cmake: fix OpenSSL quic detection in quiche builds -Dan Fandrich (21 Sep 2023) + An orphan call to `CheckQuicSupportInOpenSSL()` remained after a recent + update when checking QUIC for quiche. Move back QUIC detection to + a function and fixup callers to use that. Also make sure that quiche + gets QUIC from BoringSSL, because it doesn't support other forks at this + time. -- github/labeler: give the sync-labels config item a default value + Regression from dee310d54261f9a8416e87d50bccfe2cbe404949 #11555 - This shouldn't be necessary and is likely a bug with this beta version - of the labeller. + Reported-by: Casey Bodley + Fixes #12160 + Closes #12162 - Also, fix the negative matches for the documentation label. +Daniel Stenberg (22 Oct 2023) - Follow-up to dd12b452a - Closes #11907 +- RELEASE-NOTES: synced -- github/labeler: fix up more the labeler config format + bump to 8.5.0 for pending release - The new version didn't like the workaround we had for a bug in the - previous labeler version, and it should no longer be needed. +Dan Fandrich (21 Oct 2023) - Follow-up to dd12b452a - Closes #11906 +- test3103: add missing quotes around a test tag attribute -- github/labeler: fix indenting to try to appease labeller +Loïc Yhuel (21 Oct 2023) - Follow-up to dd12b452a +- tool: fix --capath when proxy support is disabled -Jay Satiro (21 Sep 2023) + After 95e8515ca0, --capath always sets CURLOPT_PROXY_CAPATH, which fails + with CURLE_UNKNOWN_OPTION when proxy support is disabled. -- libssh2: fix error message on failed pubkey-from-file + Closes #12089 - - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error - message "SSH public key authentication failed: Reason unknown (-1)". +Daniel Stenberg (21 Oct 2023) - When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a - generic error and therefore doesn't set an error message. AFAICT that is - not documented behavior. +- openldap: move the alloc of ldapconninfo to *connect() - Prior to this change libcurl retrieved the last set error message which - would be from a previous function failing. That resulted in misleading - auth failed error messages in verbose mode. + Fixes a minor memory leak on LDAP connection reuse. - Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 - Reported-by: consulion@users.noreply.github.com + Doing the allocation already in *setup_connection() is wrong since that + connect struct might get discarded early when an existing connection is + reused instead. - Closes https://github.com/curl/curl/pull/11881 + Closes #12166 -Stefan Eissing (21 Sep 2023) +- openldap: set the callback argument in oldap_do -- pytest: exclude test_03_goaway in CI runs due to timing dependency + ... to make sure it has the current 'data' pointer and not a stale old + one. - Closes #11860 + Reported-by: Dan Fandrich + Closes #12166 -- lib: disambiguate Curl_client_write flag semantics +- gnutls: support CURLSSLOPT_NATIVE_CA - - use CLIENTWRITE_BODY *only* when data is actually body data - - add CLIENTWRITE_INFO for meta data that is *not* a HEADER - - debug assertions that BODY/INFO/HEADER is not used mixed - - move `data->set.include_header` check into Curl_client_write - so protocol handlers no longer have to care - - add special in FTP for `data->set.include_header` for historic, - backward compatible reasons - - move unpausing of client writes from easy.c to sendf.c, so that - code is in one place and can forward flags correctly + Remove the CURL_CA_FALLBACK logic. That build option was added to allow + primarily OpenSSL to use the default paths for loading the CA certs. For + GnuTLS it was instead made to load the "system certs", which is + different and not desirable. - Closes #11885 + The native CA store loading is now asked for with this option. -Patrick Monnerat (21 Sep 2023) + Follow-up to 7b55279d1d856 -- tftpd: always use curl's own tftp.h + Co-authored-by: Jay Satiro - Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects - and reports a stringop-overread warning: + Closes #12137 - tftpd.c: In function ‘write_behind.isra’: - tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f - rom a region of size 0 [-Wstringop-overread] - 485 | return write(test->ofile, writebuf, count); - | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - In file included from tftpd.c:71: - /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 - 58 | char tu_data[0]; /* data or error stri - ng */ - | ^~~~~~~ +Stefan Eissing (21 Oct 2023) - This occurs because writebuf points to this field and the latter - cannot be considered as being of dynamic length because it is not - the last field in the structure. Thus it is bound to its declared - size. +- RTSP: improved RTP parser - This commit always uses curl's own version of tftp.h where the - target field is last in its structure, effectively avoiding the - warning. + - fix HTTP header parsing to report incomplete + lines it buffers as consumed! + - re-implement the RTP parser for interleave RTP + messages for robustness. It is now keeping its + state at the connection + - RTSP protocol handler "readwrite" implementation + now tracks if the response is before/in/after + header parsing or "in" a bod by calling + "Curl_http_readwrite_headers()" itself. This + allows it to know when non-RTP bytes are "junk" + or HEADER or BODY. + - tested with #12035 and various small receive + sizes where current master fails - As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for - arpa/tftp.h are removed. + Closes #12052 - Closes #11897 +- http2: header conversion tightening -Dan Fandrich (20 Sep 2023) + - fold the code to convert dynhds to the nghttp2 structs + into a dynhds internal method + - saves code duplication + - pacifies compiler analyzers -- test1474: make precheck more robust on non-Solaris systems + Closes #12097 - If uname -r returns something odd, perl could return an error code and - the test would be erroneously skipped. The qx// syntax avoid this. +Daniel Stenberg (21 Oct 2023) - Followup to 08f9b2148 +- curl_ntlm_wb: fix elif typo -- github/labeler: switch to the 5 beta version + Reported-by: Manfred Schwarb + Follow-up to d4314cdf65ae + Bug: https://github.com/curl/curl/commit/d4314cdf65aee295db627016934bd9eb621a + b077#r130551295 - This version adds an important feature that will allow more PRs to be - labelled. Rather than being limited to labeling PRs with files that - match a single glob, it can now label them if multiple changed files - match any one of a number of globs. +Dan Fandrich (20 Oct 2023) -Daniel Stenberg (20 Sep 2023) +- test1683: remove commented-out check alternatives -- lib: enable hmac for digest as well + Python precheck/postcheck alternatives were included but commented out. + Since these are not used and perl is guaranteed to be available to run + the perl versions anyway, the Python ones are removed. - Previously a build that disabled NTLM and aws-sigv4 would fail to build - since the hmac was disabled, but it is also needed for digest auth. +Daniel Stenberg (20 Oct 2023) - Follow-up to e92edfbef64448ef +- hostip: show the list of IPs when resolving is done - Fixes #11890 - Reported-by: Aleksander Mazur - Closes #11896 + Getting 'curl.se' today then gets this verbose output which might help + debugging connectivity related matters. -- idn: if idn2_check_version returns NULL, return error + * Host curl.se:80 was resolved. + * IPv6: 2a04:4e42::347, 2a04:4e42:200::347, 2a04:4e42:400::347, + 2a04:4e42:600::347, 2a04:4e42:800::347, 2a04:4e42:a00::347, + 2a04:4e42:c00::347, 2a04:4e42:e00::347 + * IPv4: 151.101.193.91, 151.101.1.91, 151.101.65.91, 151.101.129.91 - ... this avoids a NULL dereference for this unusual case. + Co-authored-by: Jay Satiro + Closes #12145 - Reported-by: s0urc3_ on hackerone - Closes #11898 +rilysh (20 Oct 2023) -- http: fix CURL_DISABLE_BEARER_AUTH breakage +- docs: fix function typo in curl_easy_option_next.3 - When bearer auth was disabled, the if/else logic got wrong and caused - problems. + Closes #12170 - Follow-up to e92edfbef64448ef461 - Fixes #11892 - Reported-by: Aleksander Mazur - Closes #11895 +Daniel Stenberg (20 Oct 2023) -Michael Osipov (20 Sep 2023) +- vssh: remove the #ifdef for Curl_ssh_init, use empty macro -- wolfssl: allow capath with CURLOPT_CAINFO_BLOB + In the same style as other init calls - Remain consistent with OpenSSL. While CAfile is nulled as documented - with CURLOPT_CAINFO_BLOB, CApath remains intact. +- easy: remove duplicate wolfSSH init call - Closes #11886 + It is already done in Curl_ssh_init() where it belongs. -- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c + Closes #12168 - Closes #11886 +- socks: make SOCKS5 use the CURLOPT_IPRESOLVE choice -Dan Fandrich (19 Sep 2023) + Fixes #11949 + Reported-by: Ammar Faizi + Closes #12163 -- test1474: disable test on NetBSD, OpenBSD and Solaris 10 +- urldata: move the 'internal' boolean to the state struct - These kernels only send a fraction of the requested amount of the first - large block, invalidating the assumptions of the test and causing it to - fail. + ... where all the other state bits for the easy handles live. - Assisted-by: Christian Weisgerber - Ref: https://curl.se/mail/lib-2023-09/0021.html - Closes #11888 + Closes #12165 -Ryan Schmidt (20 Sep 2023) +- url: don't touch the multi handle when closing internal handles -- cmake, configure: also link with CoreServices + Reported-by: Maksymilian Arciemowicz + Closes #12165 - When linking with CoreFoundation, also link with CoreServices which is - apparently required to avoid an NSInvalidArgumentException in software - linking with libcurl on macOS Sonoma 14 and later. +Faraz Fallahi (19 Oct 2023) - Fixes #11893 - Closes #11894 +- getenv: PlayStation doesn't have getenv() -Marc Hoersken (19 Sep 2023) + Closes #12140 -- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket +Daniel Stenberg (19 Oct 2023) - These dependencies are now already included in the Docker image. +- transfer: only reset the FTP wildcard engine in CLEAR state - Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 - b41d15606e97f38cf312c1ce56 + To avoid the state machine to start over and redownload all the files + *again*. - Closes #11889 + Reported-by: lkordos on github + Regression from 843b3baa3e3cb228 (shipped in 8.1.0) + Bisect-by: Dan Fandrich + Fixes #11775 + Closes #12156 -Daniel Stenberg (19 Sep 2023) +Stefan Eissing (19 Oct 2023) -- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files +- GHA: move mod_h2 version in CI to v2.0.25 - Ref: #11883 - Reported-by: Michael Osipov - Closes #11884 + Closes #12157 -- RELEASE-NOTES: synced +Daniel Stenberg (19 Oct 2023) -- test3103: CURLOPT_COOKIELIST test +- ntlm_wb: use pipe instead of socketpair when possible -- cookie: set ->running in cookie_init even if data is NULL + Closes #12149 - This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) +- RELEASE-NOTES: synced - Test 3103 verifies. +- asyn-thread: use pipe instead of socketpair for IPC when available - Fixes #11875 - Reported-by: wangp on github - Closes #11876 + If pipe() is present. Less overhead. -- test498: total header size for all redirects is larger than accepted + Helped-by: Viktor Szakats + Closes #12146 -- http: use per-request counter to check too large headers +Dan Fandrich (17 Oct 2023) - Not the counter that accumulates all headers over all redirects. +- tests: Fix Windows test helper tool search & use it for handle64 - Follow-up to 3ee79c1674fd6 + The checkcmd() and checktestcmd() functions would not have worked on + Windows due to hard-coding the UNIX PATH separator character and not + adding .exe file extension. This meant that tools like stunnel, valgrind + and nghttpx would not have been found and used on Windows, and + inspection of previous test runs show none of those being found in pure + Windows CI builds. - Do a second check for 20 times the limit for the accumulated size for - all headers. + With this fixed, they can be used to detect the handle64.exe program + before attempting to use it. When handle64.exe was called + unconditionally without it existing, it caused perl to abort the test + run with the error - Fixes #11871 - Reported-by: Joshix-1 on github - Closes #11872 + The running command stopped because the preference variable + "ErrorActionPreference" or common parameter is set to Stop: + sh: handle64.exe: command not found -Jay Satiro (18 Sep 2023) + Closes #12115 -- THANKS: add Eric Murphy +Daniel Stenberg (17 Oct 2023) - He reported #11850 (quiche build error) but I forgot to add a - 'reported-by' entry in the fix 267e14f1. +- multi: use pipe instead of socketpair to *wakeup() -Daniel Stenberg (18 Sep 2023) + If pipe() is present. Less overhead. -- h2-proxy: remove left-over mistake in drain_tunnel() + Closes #12142 - Left-over from 331b89a319 +Jay Satiro (17 Oct 2023) - Reported-by: 南宫雪珊 +- build: fix 'threadsafe' feature detection for older gcc - Closes https://github.com/curl/curl/pull/11877 + - Add 'threadsafe' to the feature list shown during build if POSIX + threads are being used. -vvb2060 (18 Sep 2023) + This is a follow-up to 5adb6000 which added support for building a + thread-safe libcurl with older versions of gcc where atomic is not + available but pthread is. -- lib: failf/infof compiler warnings + Reported-by: Dan Fandrich + Co-authored-by: Dan Fandrich - Closes #11874 + Fixes https://github.com/curl/curl/issues/12125 + Closes https://github.com/curl/curl/pull/12127 -Daniel Stenberg (17 Sep 2023) +Daniel Stenberg (16 Oct 2023) -- rand: fix 'alnum': array is too small to include a terminating null character +- test729: verify socks4a with excessive proxy user name length - It was that small on purpose, but this change now adds the null byte to - avoid the error. +- socks: better buffer size checks for socks4a user and hostname - Follow-up to 3aa3cc9b052353b1 + Also limit the proxy user name to 255 bytes, which is the same limit as + in SOCKS5. - Reported-by: Dan Fandrich - Ref: #11838 - Closes #11870 + Reported-by: sd0 on hackerone + Closes #12139 -Mathias Fuchs (16 Sep 2023) +- curl.h: on FreeBSD include sys/param.h instead of osreldate.h -- cmake: fix the help text to the static build option in CMakeLists.txt + Should things build on Playstation as well - Closes #11843 + Fixes #12107 + Reported-by: Faraz Fallahi + Closes #12123 -John Haugabook (16 Sep 2023) +Marcin Rataj (16 Oct 2023) -- MANUAL.md: change domain to example.com +- tool_operate: fix links in ipfs errors - Closes #11866 + URL fragment links generated from headers in + https://curl.se/docs/ipfs.html are lowercase. -Daniel Stenberg (16 Sep 2023) + Closes #12133 -- doh: inherit DEBUGFUNCTION/DATA +Viktor Szakats (15 Oct 2023) - When creating new transfers for doing DoH, they now inherit the debug - settings from the initiating transfer, so that the application can - redirect and handle the verbose output correctly even for the DoH - transfers. +- cmake: replace `check_library_exists_concat()` - Reported-by: calvin2021y on github - Fixes #11864 - Closes #11869 + The idea of `check_library_exists_concat()` is that it detects an + optional component and adds it to the list of libs that we also use in + subsequent component checks. This caused problems when detecting + components with unnecessary dependencies that were not yet built. -Dan Fandrich (16 Sep 2023) + CMake offers the `CMAKE_REQUIRED_LIBRARIES` variable to set libs used + for component checks, which we already use in most cases. That left 4 + uses of `check_library_exists_concat()`. Only one of these actually + needed the 'concat' feature (ldap/lber). -- http_aws_sigv4: fix sorting with empty parts + Delete this function and replace it with standard + `check_library_exists()` and manual management of our `CURL_LIBS` + list we use when linking build targets. And special logic to handle the + ldap/lber case. - When comparing with an empty part, the non-empty one is always - considered greater-than. Previously, the two would be considered equal - which would randomly place empty parts amongst non-empty ones. This - showed as a test 439 failure on Solaris as it uses a different - implementation of qsort() that compares parts differently. + (We have a similar function for headers: `check_include_file_concat()`. + It works, but problematic for performance reasons and because it hides + the actual headers required in `check_symbol_exists()` calls.) - Fixes #11855 - Closes #11868 + Ref: #11537 #11558 + Fixes #11285 + Fixes #11648 + Closes #12070 -- CI: ignore the "flaky" and "timing-dependent" test results +LoRd_MuldeR (15 Oct 2023) - CI builds will now run these tests, but will ignore the results if they - fail. The relevant tests are ones that are sensitive to timing or - have edge conditions that make them more likely to fail on CI servers, - which are often heavily overloaded and slow. +- tool_cb_wrt: fix write output for very old Windows versions - This change only adds two additional tests to be ignored, since the - others already had the flaky keyword. + - Pass missing parameter for 'lpNumberOfCharsWritten' to WriteConsoleW() + function. - Closes #11865 + Apparently this parameter was *not* optional on older Windows versions. -- runtests: eliminate a warning on old perl versions + Issue observed on Windows XP SP2. Issue not observed on Windows 7 SP1. + So at some point between those two Microsoft changed the behavior. - The warning "Use of implicit split to @_ is deprecated" showed between - perl versions about 5.8 through 5.11. + Prior to this change, on those versions if parameter is NULL then the + function call fails with error ERROR_INVALID_ACCESS. -- tests: log the test result code after each libtest + Regression since af3f4e41. - This makes it easier to determine the test status. Also, capitalize - FAILURE and ABORT messages in log lines to make them easier to spot. + Ref: https://github.com/MicrosoftDocs/Console-Docs/issues/299 -Harry Sintonen (16 Sep 2023) + Fixes https://github.com/curl/curl/issues/12131 + Closes https://github.com/curl/curl/pull/12130 -- misc: better random strings +Jay Satiro (15 Oct 2023) - Generate alphanumerical random strings. +- tool_urlglob: fix build for old gcc versions - Prior this change curl used to create random hex strings. This was - mostly okay, but having alphanumerical random strings is better: The - strings have more entropy in the same space. + - Don't use __builtin_mul_overflow for GCC 4 and earlier. - The MIME multipart boundary used to be mere 64-bits of randomness due - to being 16 hex chars. With these changes the boundary is 22 - alphanumerical chars, or little over 130 bits of randomness. + The function was added in GCC 5. - Closes #11838 + Ref: https://gcc.gnu.org/gcc-5/changes.html -Daniel Stenberg (15 Sep 2023) + Reported-by: Dan Fandrich -- cookie: reduce variable scope, add const + Fixes https://github.com/curl/curl/issues/12124 + Closes https://github.com/curl/curl/pull/12128 -- cookie: do not store the expire or max-age strings +Carlos Henrique Lima Melara (14 Oct 2023) - Convert it to an expire time at once and save memory. +- docs/libcurl: fix three minor man page format mistakes - Closes #11862 + Reported-by: Samuel Henrique -- cookie: remove unnecessary struct fields + Closes https://github.com/curl/curl/pull/12126 - Plus: reduce the hash table size from 256 to 63. It seems unlikely to - make much of a speed difference for most use cases but saves 1.5KB of - data per instance. +Jay Satiro (14 Oct 2023) - Closes #11862 +- tests/server: add more SOCKS5 handshake error checking -- RELEASE-NOTES: synced + - Add additional checking for missing and too-short SOCKS5 handshake + messages. - Bumped to 8.4.0, the next presumed version + Prior to this change the SOCKS5 test server did not check that all parts + of the handshake were received successfully. If those parts were missing + or too short then the server would access uninitialized memory. -Dan Fandrich (14 Sep 2023) + This issue was discovered in CI job 'memory-sanitizer' test results. + Test 2055 was failing due to the SOCKS5 test server not running. It was + not running because either it crashed or memory sanitizer aborted it + during Test 728. Test 728 connects to the SOCKS5 test server on a + redirect but does not send any data on purpose. The test server was not + prepared for that. -- test2600: remove special case handling for USE_ALARM_TIMEOUT + Reported-by: Dan Fandrich - This was originally added to handle platforms that supported only 1 - second granularity in connect timeouts, but after some recent changes - the test currently permafails on several Windows platforms. + Fixes https://github.com/curl/curl/issues/12117 + Closes https://github.com/curl/curl/pull/12118 - The need for this special-case was removed in commit 8627416, which - increased the connect timeout in all cases to well above 1 second. +Daniel Stenberg (14 Oct 2023) - Fixes #11767 - Closes #11849 +- RELEASE-NOTES: synced -Daniel Stenberg (14 Sep 2023) +Sohom Datta (14 Oct 2023) -- SECURITY-PROCESS.md. call it vulnerability disclosure policy +- tool_getparam: limit --rate to be smaller than number of ms - SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md + Currently, curl allows users to specify absurd request rates that might + be higher than the number of milliseconds in the unit (ex: curl --rate + 3600050/h http://localhost:8080 does not error out despite there being + only 3600000ms in a hour). - This a name commonly used for a document like this. This name helps - users find it. + This change adds a conditional check before the millisecond calculation + making sure that the number is not higher than the numerator (the unit) + If the number is higher, curl errors out with PARAM_NUMBER_TOO_LARGE - Closes #11852 + Closes #12116 -Junho Choi (14 Sep 2023) +Daniel Stenberg (14 Oct 2023) -- quiche: fix build error with --with-ca-fallback +- opts: fix two minor man page format mistakes - - Fix build error when curl is built with --with-quiche - and --with-ca-fallback. +Jay Satiro (14 Oct 2023) - - Add --with-ca-fallback to the quiche CI job. +- curl_trc: remove a bad assertion - Fixes https://github.com/curl/curl/issues/11850 - Closes https://github.com/curl/curl/pull/11847 + - Remove DEBUGASSERT that an internal handle must not have user + private_data set before calling the user's debug callback. -Jay Satiro (14 Sep 2023) + This is a follow-up to 0dc40b2a. The user can distinguish their easy + handle from an internal easy handle by setting CURLOPT_PRIVATE on their + easy handle. I had wrongly assumed that meant the user couldn't then + set CURLOPT_PRIVATE on an internal handle as well. -- escape: replace Curl_isunreserved with ISUNRESERVED + Bug: https://github.com/curl/curl/pull/12060#issuecomment-1754594697 + Reported-by: Daniel Stenberg - - Use the ALLCAPS version of the macro so that it is clear a macro is - being called that evaluates the variable multiple times. + Closes https://github.com/curl/curl/pull/12104 - - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates - a variable multiple times. +Dan Fandrich (13 Oct 2023) - This is a follow-up to 291d225a which changed Curl_isunreserved into an - alias macro for ISUNRESERVED. The problem is the former is not easily - identified as a macro by the caller, which could lead to a bug. +- test613: stop showing an error on missing output file - For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but - Curl_isunreserved(*foo++) is not even though they both are the same. + This test would show an error message if the output was missing during + the log post-processing step, but the message was not captured by the + test harness and wasn't useful since the normal golden log file + comparison would the problem more clearly. - Closes https://github.com/curl/curl/pull/11846 +Stefan Eissing (13 Oct 2023) -Dan Fandrich (13 Sep 2023) +- quic: manage connection idle timeouts -- tests: increase the default server logs lock timeout + - configure a 120s idle timeout on our side of the connection + - track the timestamp when actual socket IO happens + - check IO timestamp to our *and* the peer's idle timeouts + in "is this connection alive" checks - This timeout is used to wait for the server to finish writing its logs - before checking them against the expected values. An overloaded machine - could take more than the two seconds previously allocated, so increase - the timeout to 5 seconds. + Reported-by: calvin2021y on github + Fixes #12064 + Closes #12077 - Ref: #11328 - Closes #11834 +Dan Fandrich (13 Oct 2023) -- tests: increase TEST_HANG_TIMEOUT in two tests +- CI: ignore test 286 on Appveyor gcc 9 build - These tests had a 5 second timeout compared to 60 seconds for all other - tests. Make these consistent with the others for more reliability on - heavily-loaded machines. + This test fails sometimes with a super fast retry loop due to what may + just be a compiler bug. The test results are ignored on the one CI job + where it occurs because there seems to be nothing we can do to fix it. - Ref: #11328 + Fixes #12040 + Closes #12106 -- test1056: disable on Windows +Viktor Szakats (13 Oct 2023) - This test relies on the IPv6 scope field being ignored when connecting to - ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit - dodgy, but it works on all our test platforms except Windows. This - test was disabled manually on all Windows CI builds already, so instead - add an incompatible feature and precheck so it's skipped on Windows - everywhere automatically. +- lib: fix gcc warning in printf call -- test587: add a slight delay after test + Do not pass NULL to printf %s. - This test is designed to connect to the server, then immediately send a - few bytes and disconnect. In some situations, such as on a loaded - server, this doesn't give the server enough time to write its lock file - before its existence is checked. The test harness then fails to find the - server's input log file (because it hasn't been written yet) and fails - the test. By adding a short delay after the test, the HTTP server has - enough time to write its lock file which gives itself more time to write - its remaining files. + Seen with gcc 13.2.0 on Debian: + ``` + .../curl/lib/connect.c:696:27: warning: '%s' directive argument is null [-Wfo + rmat-overflow=] + ``` + Ref: https://github.com/curl/curl-for-win/actions/runs/6476161689/job/1758442 + 6483#step:3:11104 - Ref: #11328 + Ref: #10284 + Co-authored-by: Jay Satiro + Closes #12082 -- tests: stop overriding the lock timeout +Alex Klyubin (13 Oct 2023) - These tests reduce the server lock wait timeout which can increase - flakiness on loaded machines. Since this is merely an optimization, - eliminate them in favour of reliability. +- http2: safer invocation of populate_binsettings - Ref: #11328 + populate_binsettings now returns a negative value on error, instead of a + huge positive value. Both places which call this function have been + updated to handle this change in its contract. -- tests: add some --expect100-timeout to reduce timing dependencies + The way populate_binsettings had been used prior to this change the huge + positive values -- due to signed->unsigned conversion of the potentially + negative result of nghttp2_pack_settings_payload which returns negative + values on error -- are not possible. But only because http2.c currently + always provides a large enough output buffer and provides H2 SETTINGS + IVs which pass the verification logic inside nghttp2. If the + verification logic were to change or if http2.c started passing in more + IVs without increasing the output buffer size, the overflow could become + reachable, and libcurl/curl might start leaking memory contents to + servers/proxies... - These tests can fail when the test machine is so slow that the test HTTP - server didn't get a chance to complete before the client's one second - 100-continue timeout triggered. Increase that 1 second to 999 seconds so - this situation doesn't happen. + Closes #12101 - Ref: #11328 +Daniel Stenberg (13 Oct 2023) -- test661: return from test early in case of curl error +- openssl: avoid BN_num_bits() NULL pointer derefs -- tests: add the timing-dependent keyword on several tests + Reported-by: icy17 on github + Fixes #12099 + Closes #12100 - These are ones likely to fail on heavily-loaded machines that alter the - normal test timing. Most of these tests already had the flaky keyword - since this condition makes them more likely to fail on CI. +- wolfssl: require WOLFSSL_SYS_CA_CERTS for loading system CA -- test1592: greatly increase the maximum test timeout + This define is set in wolfssl's options.h file when this function and + feature is present. Handles both builds with the feature explicitly + disabled and wolfSSL versions before 5.5.2 - which introduced this API + call. - It was too short to be reliable on heavily loaded CI machines, and - as a fail-safe only, it didn't need to be short. + Closes #12108 - Ref: #11328 +- tool_urlglob: make multiply() bail out on negative values -- test: minor test cleanups + - Does not work correctly with negative values + - use __builtin_mul_overflow() on gcc - Remove an obsolete block of code in tests 2032 & 576. - Add a comment in test 1474. + Reported-by: Torben Dury + Closes #12102 -- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts +Loïc Yhuel (13 Oct 2023) - This gives more of a margin for error when running on overloaded CI - servers. +- cmake: fix CURL_DISABLE_GETOPTIONS - Ref: #11328 + - Add CURL_DISABLE_GETOPTIONS to curl_config.h.cmake. -- tests: improve SLOWDOWN test reliability by reducing sent data + Prior to this change the option had no effect because it was missing + from that file. - These tests are run in SLOWDOWN mode which adds a 10 msec delay after - each character output, which means it takes at least 1.6 seconds (and - 320 kernel calls) just to get through the long welcome banner. On an - overloaded system, this can end up taking much more than 1.6 seconds, - and even more than the 7 or 16 second curl timeout that the tests rely - on, causing them to fail. Reducing the size of the welcome banner drops - the total number of characters sent before the transfer starts by more - than half, which reduces the opportunity for test-breaking slowdowns by - the same amount. + Closes https://github.com/curl/curl/pull/12091 - Ref: #11328 +- easy_lock: add a pthread_mutex_t fallback -- test650: fix an end tag typo + This allows to keep the init threadsafe with gcc < 4.9.0 (no C11 + atomics). -Jay Satiro (13 Sep 2023) + Closes https://github.com/curl/curl/pull/12090 -- tool_cb_wrt: fix debug assertion +Viktor Szakats (12 Oct 2023) - - Fix off-by-one out-of-bounds array index in Windows debug assertion. +- CI: add autotools, out-of-tree, debug build to distro check job - Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 - Reported-by: Gisle Vanem + Add a job that builds curl from a generated source tarball sample, with + autotools, out-of-tree, in debug mode. -Daniel Stenberg (13 Sep 2023) + Ref: #12085 + Closes #12088 -- ctype: add ISUNRESERVED() +Daniel Stenberg (12 Oct 2023) - ... and make Curl_isunreserved() use that macro instead of providing a - separate funtion for the purpose. +- http: avoid Expect: 100-continue if Upgrade: is used - Closes #11840 + Reported-by: Daniel Jelinski + Fixes #12022 + Closes #12062 -Version 8.3.0 (13 Sep 2023) +Jan Alexander Steffens (heftig) (12 Oct 2023) -Daniel Stenberg (13 Sep 2023) +- docs: use SOURCE_DATE_EPOCH for generated manpages -- RELEASE-NOTES: syn ced + This should make builds from Git reproducible. - curl 8.3.0 release + Closes #12092 -- THANKS: contributors from 8.3.0 +Daniel Stenberg (12 Oct 2023) -Thorsten Klein (12 Sep 2023) +- RELEASE-NOTES: synced -- cmake: set SIZEOF_LONG_LONG in curl_config.h + Bumped to 8.4.1 - in order to support 32bit builds regarding wolfssl CTC_SETTINGS +Viktor Szakats (12 Oct 2023) - Closes #11839 +- cmake: fix `HAVE_H_ERRNO_ASSIGNABLE` detection -Jay Satiro (12 Sep 2023) + Fix `HAVE_H_ERRNO_ASSIGNABLE` to not run, only compile its test snippet, + aligning this with autotools. This fixes an error when doing + cross-builds and also actually detects this feature. It affected systems + not allowlisted into this, e.g. SerenityOS. -- curl_ngtcp2: fix error message + We used this detection result to enable `HAVE_GETADDRINFO_THREADSAFE`. -- http_aws_sigv4: handle no-value user header entries + Follow-up to 04a3a377d83fd72c4cf7a96c9cb6d44785e33264 #11979 + Ref: #12095 (closed in favour of this patch) + Ref: #11964 (effort to sync cmake detections with autotools) - - Handle user headers in format 'name:' and 'name;' with no value. + Reported-by: Kartatz on Github + Assisted-by: Kartatz on Github + Fixes #12093 + Closes #12094 - The former is used when the user wants to remove an internal libcurl - header and the latter is used when the user actually wants to send a - no-value header in the format 'name:' (note the semi-colon is converted - by libcurl to a colon). +- build: add `src/.checksrc` to source tarball - Prior to this change the AWS header import code did not special case - either of those and the generated AWS SignedHeaders would be incorrect. + Regression from e5bb88b8f824ed87620bd923552534c83c2a516e #11958 - Reported-by: apparentorder@users.noreply.github.com + Bug: https://github.com/curl/curl/pull/11958#issuecomment-1757079071 + Reported-by: Romain Geissler + Fixes #12084 + Closes #12085 - Ref: https://curl.se/docs/manpage.html#-H +Version 8.4.0 (11 Oct 2023) - Fixes https://github.com/curl/curl/issues/11664 - Closes https://github.com/curl/curl/pull/11668 +Daniel Stenberg (11 Oct 2023) -Dan Fandrich (11 Sep 2023) +- RELEASE-NOTES: synced -- CI: run pytest with the -v option +- THANKS: add contributors from 8.4.0 - This lists of the test cases being run so it can be tracked over time. +Jay Satiro (11 Oct 2023) - Closes #11824 +- socks: return error if hostname too long for remote resolve -Daniel Stenberg (11 Sep 2023) + Prior to this change the state machine attempted to change the remote + resolve to a local resolve if the hostname was longer than 255 + characters. Unfortunately that did not work as intended and caused a + security issue. -- HTTP3: the msquic backend is not functional + Bug: https://curl.se/docs/CVE-2023-38545.html - I ask that we do not submit bugs for this backend just yet as we know it - does not fully work. +Stefan Eissing (10 Oct 2023) - Closes #11831 - Closes #11819 +- CI: remove slowed-network tests -- aws_sigv4: the query canon code miscounted URL encoded input + - remove these tests as they are currently not reliable in our CI + setups. - Added some extra ampersands to test 439 to verify "blank" query parts + curl handles the test cases, but CI sometimes fails on these due to + additional conditions. Rather than mix them in, an additional CI job + will be added in the future that is specific to them. - Follow-up to fc76a24c53b08cdf + Closes https://github.com/curl/curl/pull/12075 - Closes #11829 +Jay Satiro (10 Oct 2023) -vvb2060 (11 Sep 2023) +- libcurl-env-dbg.3: move debug variables from libcurl-env.3 -- quic: don't set SNI if hostname is an IP address + - Move documentation of libcurl environment variables used only in debug + builds from libcurl-env into a separate document libcurl-env-dbg. - We already do this for TLS connections. + - Document more debug environment variables. - RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in - "HostName". + Previously undocumented or missing a description: - Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 + CURL_ALTSVC_HTTP, CURL_DBG_SOCK_WBLOCK, CURL_DBG_SOCK_WPARTIAL, + CURL_DBG_QUIC_WBLOCK, CURL_DEBUG, CURL_DEBUG_SIZE, CURL_GETHOSTNAME, + CURL_HSTS_HTTP, CURL_FORCETIME, CURL_SMALLREQSEND, CURL_SMALLSENDS, + CURL_TIME. - Fixes https://github.com/curl/curl/issues/11827 - Closes https://github.com/curl/curl/pull/11828 + Closes https://github.com/curl/curl/pull/11811 -Daniel Stenberg (10 Sep 2023) +Dan Fandrich (9 Oct 2023) -- RELEASE-NOTES: synced +- test670: increase the test timeout -Benoit Pierre (10 Sep 2023) + This should make it more immune to loaded servers. -- configure: fix `HAVE_TIME_T_UNSIGNED` check + Ref: #11328 - The syntax was incorrect (need a proper main body), and the test - condition was wrong (resulting in a signed `time_t` detected as - unsigned). +Stefan Eissing (9 Oct 2023) - Closes #11825 +- MQTT: improve receive of ACKs -Daniel Stenberg (9 Sep 2023) + - add `mq->recvbuf` to provide buffering of incomplete + ACK responses + - continue ACK reading until sufficient bytes available + - fixes test failures on low network receives -- THANKS-filter: pszlazak on github + Closes #12071 -pszlazak (9 Sep 2023) +Viktor Szakats (9 Oct 2023) -- include.d: explain headers not printed with --fail before 7.75.0 +- quic: fix BoringSSL build - Prior to 7.75.0 response headers were not printed if -f/--fail was used - and an error was reported by server. This was fixed in ab525c0 - (precedes 7.75.0). + Add guard around `SSL_CTX_set_ciphersuites()` use. - Closes #11822 + Bug: https://github.com/curl/curl/pull/12065#issuecomment-1752171885 -Daniel Stenberg (8 Sep 2023) + Follow-up to aa9a6a177017e4b74d33cdf85a3594900f4a7f81 -- http_aws_sigv4: skip the op if the query pair is zero bytes + Co-authored-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes #12067 - Follow-up to fc76a24c53b08cdf +Stefan Eissing (9 Oct 2023) - Spotted by OSS-Fuzz +- test1540: improve reliability - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 - Closes #11823 + - print that bytes have been received on pausing, but not how many -- cmdline-docs: use present tense, not future + Closes #12069 - + some smaller cleanups +- test2302: improve reliability - Closes #11821 + - make result print collected write data, unless + change in meta flags is detected + - will show same result even when data arrives via + several writecb invocations -- cmdline-docs: make sure to phrase it as "added in ...." + Closes #12068 - References to things that were added or changed in a specific version - should be specified as "(added in [version]) for two reasons: +Daniel Stenberg (9 Oct 2023) - 1 - consistency +- curl_easy_pause: set "in callback" true on exit if true - 2 - to allow gen.pl to strip them out if deemed referring to too old - versions + Because it might have called another callback in the mean time that then + set the bit FALSE on exit. - Closes #11821 + Reported-by: Jay Satiro + Fixes #12059 + Closes #12061 -Jay Satiro (8 Sep 2023) +Viktor Szakats (8 Oct 2023) -- docs: mark --ssl-revoke-best-effort as Schannel specific +- h3: add support for ngtcp2 with AWS-LC builds - Closes https://github.com/curl/curl/pull/11760 + ``` + curl 8.4.0-DEV (x86_64-apple-darwin) libcurl/8.4.0-DEV (SecureTransport) AWS- + LC/1.15.0 nghttp2/1.56.0 ngtcp2/0.19.1 nghttp3/0.15.0 + Release-Date: [unreleased] + Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps + mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss + Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy IPv6 Largefile Multi + SSL NTLM SSL threadsafe UnixSockets + ``` -Nathan Moinvaziri (8 Sep 2023) + Also delete an obsolete GnuTLS TODO and update the header comment in + `FindNGTCP2.cmake`. -- schannel: fix ordering of cert chain info + Reviewed-by: Daniel Stenberg + Closes #12066 - - Use CERT_CONTEXT's pbCertEncoded to determine chain order. +- build: do not publish `HAVE_BORINGSSL`, `HAVE_AWSLC` macros - CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains - end-entity/server certificate in pbCertEncoded. We can use this pointer - to determine the order of certificates when enumerating hCertStore using - CertEnumCertificatesInStore. + Syncing this up with CMake. - This change is to help ensure that the ordering of the certificate chain - requested by the user via CURLINFO_CERTINFO has the same ordering on all - versions of Windows. + Source code uses the built-in `OPENSSL_IS_AWSLC` and + `OPENSSL_IS_BORINSSL` macros to detect BoringSSL and AWS-LC. No help is + necessary from the build tools. - Prior to this change Schannel certificate order was reversed in 8986df80 - but that was later reverted in f540a39b when it was discovered that - Windows 11 22H2 does the reversal on its own. + The one use of `HAVE_BORINGSSL` in the source turned out to be no longer + necessary for warning-free BoringSSL + Schannel builds. Ref: #1610 #2634 - Ref: https://github.com/curl/curl/issues/9706 + autotools detects this anyway for display purposes. + CMake detects this to decide whether to use the BoringSSL-specific + crypto lib with ngtcp2. It detects AWS-LC, but doesn't use the detection + result just yet (planned in #12066). - Closes https://github.com/curl/curl/pull/11632 + Ref: #11964 -Chris Talbot (8 Sep 2023) + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #12065 -- digest: Use hostname to generate spn instead of realm +Marc Hoersken (8 Oct 2023) - In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 +- CI: move distcheck job from Azure Pipelines to GitHub Actions - digest-uri-value should be serv-type "/" host , where host is: + This will allow for more trigger excludes within Azure Pipelines. - The DNS host name or IP address for the service requested. The - DNS host name must be the fully-qualified canonical name of the - host. The DNS host name is the preferred form; see notes on server - processing of the digest-uri. + Also fixes seemingly broken check with scripts/installcheck.sh. + Ref: 190374c74ec4e5247d9066544c86e8d095e1d7b5 - Realm may not be the host, so we must specify the host explicitly. + Assisted-by: Philip Heiduck + Closes #9532 - Note this change only affects the non-SSPI digest code. The digest code - used by SSPI builds already uses the hostname to generate the spn. +Daniel Stenberg (8 Oct 2023) - Ref: https://github.com/curl/curl/issues/11369 +- url: fall back to http/https proxy env-variable if ws/wss not set - Closes https://github.com/curl/curl/pull/11395 + Reported-by: Craig Andrews + Fixes #12031 + Closes #12058 -Daniel Stenberg (7 Sep 2023) +Stefan Eissing (8 Oct 2023) -- docs: remove use of the word 'very' +- cf-socket: simulate slow/blocked receives in debug - It is mostly superfluous. proselint would complain. + add 2 env variables for non-UDP sockets: + 1. CURL_DBG_SOCK_RBLOCK: percentage of receive calls that randomly + should return EAGAIN + 2. CURL_DBG_SOCK_RMAX: max amount of bytes read from socket - Closes #11818 + Closes #12035 -- curl_multi_remove_handle.3: clarify what happens with connection +- http2: refused stream handling for retry - Closes #11817 + - answer HTTP/2 streams refused via a GOAWAY from the server to + respond with CURLE_RECV_ERROR in order to trigger a retry + on another connection -- RELEASE-NOTES: synced + Reported-by: black-desk on github + Ref #11859 + Closes #12054 -- test439: verify query canonization for aws-sigv4 +Jay Satiro (8 Oct 2023) -- tool_operate: make aws-sigv4 not require TLS to be used +- CURLOPT_DEBUGFUNCTION.3: warn about internal handles - Maybe not used too often, but we want it for testing and it should work. + - Warn that the user's debug callback may be called with the handle + parameter set to an internal handle. -- http_aws_sigv4: canonicalize the query + Without this warning the user may assume that the only handles their + debug callback receives are the easy handles on which they set + CURLOPT_DEBUGFUNCTION. - Percent encoding needs to be done using uppercase, and most - non-alphanumerical must be percent-encoded. + This is a follow-up to f8cee8cc which changed DoH handles to inherit + the debug callback function set in the user's easy handle. As a result + those handles are now passed to the user's debug callback function. - Fixes #11794 - Reported-by: John Walker - Closes #11806 + Closes https://github.com/curl/curl/pull/12034 -Wyatt O'Day (7 Sep 2023) +- url: fix typo -- lib: add ability to disable auths individually +Daniel Stenberg (8 Oct 2023) - Both with configure and cmake +- test458: verify --expand-output, expanding a file name accepting option - Closes #11490 + Verifies the fix in #12055 (commit f2c8086ff15e6e995e1) -Stefan Eissing (7 Sep 2023) +- tool_getparam: accept variable expansion on file names too -- ngtcp2: fix handling of large requests + Reported-by: PBudmark on github + Fixes #12048 + Closes #12055 - - requests >64K are send in parts to the filter - - fix parsing of the request to assemble it correctly - from several sends - - open a QUIC stream only when the complete request has - been collected +- RELEASE-NOTES: synced - Closes #11815 +- multi: do CURLM_CALL_MULTI_PERFORM at two more places -- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before + ... when it does a state transition but there is no particular socket or + timer activity. This was made apparent when commit b5bb84c removed a + superfluous timer expiry. - - we delay loading the x509 store to shorten the handshake time. - However an application callback installed via CURLOPT_SSL_CTX_FUNCTION - may need to have the store loaded and try to manipulate it. - - load the x509 store before invoking the app callback + Reported-by: Dan Fandrich. + Fixes #12033 + Closes #12056 - Fixes #11800 - Reported-by: guoxinvmware on github - Cloes #11805 +Viktor Szakats (7 Oct 2023) -Daniel Stenberg (7 Sep 2023) +- GHA/linux: mbedtls 3.5.0 + minor dep bumps -- krb5: fix "implicit conversion loses integer precision" warnings + Closes #12057 - conversions to/from enum and unsigned chars +Dan Fandrich (7 Oct 2023) - Closes #11814 +- CI: bump OpenLDAP package version on FreeBSD -Stefan Eissing (7 Sep 2023) + The old one is no longer available. -- pytest: improvements +Marc Hoersken (7 Oct 2023) - - set CURL_CI for pytest runs in CI environments - - exclude timing sensitive tests from CI runs - - for failed results, list only the log and stat of - the failed transfer +- docs/libcurl/opts/Makefile.inc: add missing manpage files - - fix type in http.c comment + Detected with #9532 - Closes #11812 +Dan Fandrich (7 Oct 2023) -- CI: move on to ngtcp2 v0.19.1 +- tests: fix a race condition in ftp server disconnect - Closes #11809 + If a client disconnected and reconnected quickly, before the ftp server + had a chance to respond, the protocol message/ack (ping/pong) sequence + got out of sync, causing messages sent to the old client to be delivered + to the new. A disconnect must now be acknowledged and intermediate + requests thrown out until it is, which ensures that such synchronization + problems can't occur. This problem could affect ftp, pop3, imap and smtp + tests. -Dan Fandrich (5 Sep 2023) + Fixes #12002 + Closes #12049 -- CI: run Circle macOS builds on x86 for now +Viktor Szakats (7 Oct 2023) - The ARM machines aren't ready for us and requesting them now causes - warnings e-mails to be sent to some PR pushers. +- appveyor: bump mingw-w64 job to gcc 13 (was: 8) - Ref: #11771 + This sets gcc 6, 7, 9, 13 in our test mix (was: 6, 7, 8, 9). + Adding a modern gcc version to the tests. -Viktor Szakats (5 Sep 2023) + (The gcc 8 job used to take around 50 minutes. The new image with gcc 13 + finished in 32, 35, 34 minutes in the 3 test runs so far.) -- http3: adjust cast for ngtcp2 v0.19.0 + It also adds a modern CMake version and OS env to our mingw-w64 builds. - ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` - an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. + Closes #12051 - Fixes: - ``` - ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in - teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned - char') [-Wimplicit-int-conversion] - pi.ecn = (uint32_t)ecn; - ~ ^~~~~~~~~~~~~ - ``` +David Benjamin (6 Oct 2023) - Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our - docs and CI. +- openssl: use X509_ALGOR_get0 instead of reaching into X509_ALGOR - Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 - 782894a3 - Ref: https://github.com/ngtcp2/ngtcp2/pull/877 - Closes #11798 + While the struct is still public in OpenSSL, there is a (somewhat + inconvenient) accessor. Use it to remain compatible if it becomes opaque + in the future. -Stefan Eissing (5 Sep 2023) + Closes #12038 -- http: fix sending of large requests +Daniel Stenberg (6 Oct 2023) - - refs #11342 where errors with git https interactions - were observed - - problem was caused by 1st sends of size larger than 64KB - which resulted in later retries of 64KB only - - limit sending of 1st block to 64KB - - adjust h2/h3 filters to cope with parsing the HTTP/1.1 - formatted request in chunks +- curl_easy_pause.3: mention it works within callbacks - - introducing Curl_nwrite() as companion to Curl_write() - for the many cases where the sockindex is already known + Reported-by: Maxim Dzhura + Bug: https://curl.se/mail/lib-2023-10/0010.html + Closes #12046 - Fixes #11342 (again) - Closes #11803 +- curl_easy_pause.3: mention h2/h3 buffering -- pytest: fix check for slow_network skips to only apply when intended + Asked-by: Maxim Dzhura + Ref: https://curl.se/mail/lib-2023-10/0011.html - Closes #11801 + Closes #12045 -Daniel Stenberg (5 Sep 2023) +Viktor Szakats (6 Oct 2023) -- curl_url_get/set.3: add missing semicolon in SYNOPSIS +- cmake: re-add missed C89 headers for specific detections -- CURLOPT_URL.3: explain curl_url_set() uses the same parser + We removed C89 `setjmp.h` and `signal.h` detections and excluded them + from the global header list we use when detecting functions [1]. Then + missed to re-add these headers to the specific functions which need + them to be detected [2]. Fix this omission in this patch. -- CURLOPT_URL.3: add two URL API calls in the see-also section + [1] Follow-up to 3795fcde995d96db641ddbcc8a04f9f0f03bef9f #11951 + [2] Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 -Dan Fandrich (4 Sep 2023) + Closes #12043 -- CI: add a 32-bit i686 Linux build +Daniel Stenberg (6 Oct 2023) - This is done by cross-compiling under regular x86_64 Linux. Since the - kernel offers backwards compatibility, the binaries can be tested as - normal. +- multi: set CURLM_CALL_MULTI_PERFORM after switch to DOING_MORE - Closes #11799 + Since there is nothing to wait for there. Avoids the test 1233 hang + reported in #12033. -- tests: fix a type warning on 32-bit x86 + Reported-by: Dan Fandrich + Closes #12042 -Viktor Szakats (4 Sep 2023) +Dan Fandrich (5 Oct 2023) -- tests: delete stray `.orig` file +- test1903: actually verify the cookies after the test - Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f - Closes #11797 + The test otherwise could do just about anything (except leak memory in + debug mode) and its bad behaviour wouldn't be detected. Now, check the + resulting cookie file to ensure the cookies are still there. -Daniel Stenberg (4 Sep 2023) + Closes #12041 -- RELEASE-NOTES: synced +- test: add missing s -Viktor Szakats (4 Sep 2023) + The tests will otherwise fail if curl has them disabled. -- lib: silence compiler warning in inet_ntop6 +- test1906: set a lower timeout since it's hit on Windows - ``` - ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her - e [-Wcomma] - cur.base = i, cur.len = 1; - ^ - ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin - g - cur.base = i, cur.len = 1; - ^~~~~~~~~~~~ - (void)( ) - ``` + msys2 builds actually hit the connect timeout in normal operation, so + lower the timeout from 5 minutes to 5 seconds to reduce test time. - Closes #11790 + Ref: #11328 + Closes #12036 -Daniel Stenberg (4 Sep 2023) +Daniel Stenberg (5 Oct 2023) -- transfer: also stop the sending on closed connection +- RELEASE-NOTES: synced - Previously this cleared the receiving bit only but in some cases it is - also still sending (like a request-body) when disconnected and neither - direction can continue then. +Jay Satiro (5 Oct 2023) - Fixes #11769 - Reported-by: Oleg Jukovec - Closes #11795 +- idn: fix WinIDN null ptr deref on bad host -John Bampton (4 Sep 2023) + - Return CURLE_URL_MALFORMAT if IDN hostname cannot be converted from + UTF-8 to UTF-16. -- docs: change `sub-domain` to `subdomain` + Prior to this change a failed conversion erroneously returned CURLE_OK + which meant 'decoded' pointer (what would normally point to the + punycode) would not be written to, remain NULL and be dereferenced + causing an access violation. - https://en.wikipedia.org/wiki/Subdomain + Closes https://github.com/curl/curl/pull/11983 - Closes #11793 +Dan Fandrich (4 Oct 2023) -Stefan Eissing (4 Sep 2023) +- tests: close the shell used to start sshd -- multi: more efficient pollfd count for poll + This shell isn't needed once sshd starts, so use "exec" so it doesn't + stick around. - - do not use separate pollfds for sockets that have POLLIN+POLLOUT + Closes #12032 - Closes #11792 +Daniel Stenberg (4 Oct 2023) -- http2: polish things around POST +- base64: also build for curl - - added test cases for various code paths - - fixed handling of blocked write when stream had - been closed inbetween attempts - - re-enabled DEBUGASSERT on send with smaller data size + Since the tool itself now uses the base64 code using the curlx way, it + needs to build also when the tool needs it. Starting now, the tool build + defines BULDING_CURL to allow lib-side code to use it. - - in debug builds, environment variables can be set to simulate a slow - network when sending data. cf-socket.c and vquic.c support - * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be - answered with a EAGAIN. TCP/UNIX sockets. - This is chosen randomly. - * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written - to the network. TCP/UNIX sockets. - Example: 80 means a send with 1000 bytes would only send 800 - This is applied to every send. - * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be - answered with EAGAIN. QUIC only. - This is chosen randomly. + Follow-up to 2e160c9c6525 - Closes #11756 + Closes #12010 -Daniel Stenberg (4 Sep 2023) +Eduard Strehlau (4 Oct 2023) -- docs: add curl_global_trace to some SEE ALSO sections +- tests: Fix zombie processes left behind by FTP tests. - Closes #11791 + ftpserver.pl correctly cleans up spawned server processes, + but forgets to wait for the shell used to spawn them. + This is barely noticeable during a normal testrun, + but causes process exhaustion and test failure + during a complete torture run of the FTP tests. -- os400: fix checksrc nits + Fixes #12018 + Closes #12020 - Closes #11789 +Dan Fandrich (4 Oct 2023) -Nicholas Nethercote (3 Sep 2023) +- github/labeler: improve labeler matches -- hyper: remove `hyptransfer->endtask` +- test574: add a timeout to the test - `Curl_hyper_stream` needs to distinguish between two kinds of - `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and - (b) background tasks that hyper produces. It does this by recording the - address of any `foreach` task in `hyptransfer->endtask` before pushing - it into the executor, and then comparing that against the address of - tasks later polled out of the executor. + This one hangs occasionally, so this will speed up a test run and allow + logs to be seen when it does. - This works right now, but there is no guarantee from hyper that the - addresses are stable. `hyper_executor_push` says "The executor takes - ownership of the task, which should not be accessed again unless - returned back to the user with `hyper_executor_poll`". That wording is a - bit ambiguous but with my Rust programmer's hat on I read it as meaning - the task returned with `hyper_executor_poll` may be conceptually the - same as a task that was pushed, but that there are no other guarantees - and comparing addresses is a bad idea. + Closes #12025 - This commit instead uses `hyper_task_set_userdata` to mark the `foreach` - task with a `USERDATA_RESP_BODY` value which can then be checked for, - removing the need for `hyptransfer->endtask`. This makes the code look - more like that hyper C API examples, which use userdata for every task - and never look at task addresses. +- tests: propagate errors in libtests - Closes #11779 + Use the test macros to automatically propagate some errors, and check + and log others while running the tests. This can help in debugging + exactly why a test has failed. -Dave Cottlehuber (3 Sep 2023) +- tests: set --expect100-timeout to improve test reliability -- ws: fix spelling mistakes in examples and tests + On an overloaded server, the default 1 second timeout can go by without + the test server having a chance to respond with the expected headers, + causing tests to fail. Increase the 1 second timeout to 99 seconds so + this failure mode is no longer a problem on test 1129. Some other tests + already set a high value, but make them consistently 99 seconds so if + something goes wrong the test is stalled for less time. - Closes #11784 + Ref: #11328 -Daniel Stenberg (3 Sep 2023) +- CI: ignore the "flaky" and "timing-dependent" test results in CMake -- tool_filetime: make -z work with file dates before 1970 + This was already done for automake builds but CMake builds were missed. + Test 1086 actually causes the test harness to crash with: - Fixes #11785 - Reported-by: Harry Sintonen - Closes #11786 + Warning: unable to close filehandle DWRITE properly: Broken pipe at C:/projec + ts/curl/tests/ftpserver.pl line 527 -Dan Fandrich (1 Sep 2023) + Rather than fix it now, this change leaves test 1086 entirely skipped on + those builds that show this problem. -- build: fix portability of mancheck and checksrc targets + Follow-up to 589dca761 - At least FreeBSD preserves cwd across makefile lines, so rules - consisting of more than one "cd X; do_something" must be explicitly run - in a subshell to avoid this. This problem caused the Cirrus FreeBSD - build to fail when parallel make jobs were enabled. + Ref: #11865 -- CI: adjust labeler match patterns for new & obsolete files +Viktor Szakats (4 Oct 2023) -- configure: trust pkg-config when it's used for zlib +- cmake: improve OpenLDAP builds - The library flags retrieved from pkg-config were later thrown out and - harded-coded, which negates the whole reason to use pkg-config. - Also, previously, the assumption was made that --libs-only-l and - --libs-only-L are the full decomposition of --libs, which is untrue and - would not allow linking against a static zlib. The new approach is - better in that it uses --libs, although only if --libs-only-l returns - nothing. + - cmake: detect OpenLDAP based on function `ldap_init_fd`. + autotools does this. autotools also publishes this detection result + in `HAVE_LDAP_INIT_FD`. We don't mimic that with CMake as the source + doesn't use this value. (it might need to be remove-listed in + `scripts/cmp-config.pl` for future OpenLDAP test builds.) + This also deletes existing self-declaration method via the + CMake-specific `CURL_USE_OPENLDAP` configuration. - Bug: https://curl.se/mail/lib-2023-08/0081.html - Reported-by: Randall - Closes #11778 + - cmake: define `LDAP_DEPRECATED=1` for OpenLDAP. + Like autotools does. This fixes a long list of these warnings: + ``` + /usr/local/opt/openldap/include/ldap.h:1049:5: warning: 'LDAP_DEPRECATED' i + s not defined, evaluates to 0 [-Wundef] + ``` -Stefan Eissing (1 Sep 2023) + - cmake: delete LDAP TODO comment no longer relevant. -- CI/ngtcp2: clear wolfssl for when cache is ignored + Also: - Closes #11783 + - autotools: replace domain name `dummy` with `0.0.0.0` in LDAP feature + detection functions. -Daniel Stenberg (1 Sep 2023) + Ref: #11964 (effort to sync cmake detections with autotools) -- RELEASE-NOTES: synced + Closes #12024 -Nicholas Nethercote (1 Sep 2023) +- cmake: fix unity builds for more build combinations -- hyper: fix a progress upload counter bug + By using unique static function/variable names in source files + implementing these interfaces. - `Curl_pgrsSetUploadCounter` should be a passed a total count, not an - increment. + - OpenLDAP combined with any SSH backend. - This changes the failing diff for test 579 with hyper from this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 29 out of 0[LF] - ``` - to this: - ``` - Progress callback called with UL 0 out of 0[LF] - -Progress callback called with UL 8 out of 0[LF] - -Progress callback called with UL 16 out of 0[LF] - -Progress callback called with UL 26 out of 0[LF] - -Progress callback called with UL 61 out of 0[LF] - -Progress callback called with UL 66 out of 0[LF] - +Progress callback called with UL 40 out of 0[LF] - ``` - Presumably a step in the right direction. + - MultiSSL with mbedTLS, OpenSSL, wolfSSL, SecureTransport. - Closes #11780 + Closes #12027 -Daniel Stenberg (1 Sep 2023) +Daniel Stenberg (4 Oct 2023) -- awssiv4: avoid freeing the date pointer on error +- tests: remove leading spaces from some tags - Since it was not allocated, don't free it even if it was wrong syntax + The threee tags ``, `` and `` were frequently used + with a leading space that this removes. The reason this habbit is so + widespread in testcases is probably that they have been copy and pasted. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 + Hence, fixing them all now might curb this practice from now on. - Follow-up to b137634ba3adb + Closes #12028 - Closes #11782 +Viktor Szakats (4 Oct 2023) -Stefan Eissing (1 Sep 2023) +- GHA: bump actions/checkout -- CI: ngtcp2-linux: use separate caches for tls libraries + Follow-up to 2e0fa50fc16b9339f51e0a7bfff0352829323acb #11964 + Follow-up to c39585d9b7ef3cbfc1380812dec60e7b275b6af3 #12000 - allow ever changing master for wolfssl + Closes #12023 - Closes #11766 +- spelling: fix codespell 2.2.6 typos -- replace `master` as wolfssl-version with recent commit + Closes #12019 -- wolfssl, use master again in CI +Daniel Stenberg (3 Oct 2023) - - with the shared session update fix landed in master, it - is time to use that in our CI again +- GHA: add workflow to compare configure vs cmake outputs -Nicholas Nethercote (31 Aug 2023) + Uses scripts/cmp-config.pl two compare two curl_config.h files, + presumbly generated with configure and cmake. It displays the + differences and filters out a lot of known lines we ignore. -- tests: fix formatting errors in `FILEFORMAT.md`. + The script also shows the matches that were *not* used. Possibly + subjects for removal. - Without the surrounding backticks, these tags get swallowed when the - markdown is rendered. + Closes #11964 - Closes #11777 +- appveyor: enable test 571 -Viktor Szakats (31 Aug 2023) + Follow-up from 8a940fd55c175f7 / #12013 -- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` + Closes #12017 - Allow overriding the default TLS backend via a CMake setting. +Viktor Szakats (3 Oct 2023) - E.g.: - `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` +- build: alpha-sort source files for lib and src - Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, - schannel, secure-transport, wolfssl + Closes #12014 - The passed string is baked into the curl/libcurl binaries. - The value is case-insensitive. +- cmake: delete old `HAVE_LDAP_URL_PARSE` logic - We added a similar option to autotools in 2017 via - c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. + Left there by accident after adding proper detection for this. - TODO: Convert to lowercase to improve reproducibility. + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - Closes #11774 + Ref: #11964 (effort to sync cmake detections with autotools) -- sectransp: fix compiler warnings + Closes #12015 - https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# - step:3:11046 - ``` - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 - 4: warning: unused variable 'success' [-Wunused-variable] - OSStatus success; - ^ - /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 - 4: warning: unused parameter 'sha256len' [-Wunused-parameter] - size_t sha256len) - ^ - ``` +Stefan Eissing (3 Oct 2023) - Closes #11773 +- tests: increase lib571 timeout from 3s to 30s -- tidy-up: mostly whitespace nits + - 3s is too short for our CI, making this test fail occasionally + - test usually experiences no delay run locally, so 30s wont hurt - - delete completed TODO from `./CMakeLists.txt`. - - convert a C++ comment to C89 in `./CMake/CurlTests.c`. - - delete duplicate EOLs from EOF. - - add missing EOL at EOF. - - delete whitespace at EOL (except from expected test results). - - convert tabs to spaces. - - convert CRLF EOLs to LF in GHA yaml. - - text casing fixes in `./CMakeLists.txt`. - - fix a codespell typo in `packages/OS400/initscript.sh`. + Closes #12013 - Closes #11772 +Viktor Szakats (3 Oct 2023) -Dan Fandrich (31 Aug 2023) +- cmake: fix unity with Windows Unicode + TrackMemory -- CI: remove Windows builds from Cirrus, without replacement + Found the root cause of the startup crash in unity builds with Unicode + and TrackMemory enabled at the same time. - If we don't do this, all coverage on Cirrus will cease in a few days. By - removing the Windows builds, the FreeBSD one should still continue - as before. The Windows builds will need be moved to another service to - maintain test coverage. + We must make sure that the `memdebug.h` header doesn't apply to + `lib/curl_multibyte.c` (as even noted in a comment there.) In unity + builds all headers apply to all sources, including `curl_multibyte.c`. + This probably resulted in an infinite loop on startup. - Closes #11771 + Exclude this source from unity compilation with TrackMemory enabled, + in both libcurl and curl tool. Enable unity mode for a debug Unicode + CI job to keep it tested. Also delete the earlier workaround that + fully disabled unity for affected builds. -- CI: switch macOS ARM build from Cirrus to Circle CI + Follow-up to d82b080f6374433ce7c98241329189ad2d3976f8 #12005 + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Cirrus is drastically reducing their free tier on Sept. 1, so they will - no longer perform all these builds for us. All but one build has been - moved, with the LibreSSL one being dropped because of linking problems - on Circle. + Closes #11928 - One important note about this change is that Circle CI is currently - directing all these builds to x86_64 hardware, despite them requesting - ARM. This is because ARM nodes are scheduled to be available on the - free tier only in December. This reduces our architectural diversity - until then but it should automatically come back once those machines are - enabled. +- cmake: disable unity mode with Windows Unicode + TrackMemory -- CI: use the right variable for BSD make + "TrackMemory" is `ENABLE_DEBUG=ON` (aka `ENABLE_CURLDEBUG=ON`, + aka `-DCURLDEBUG`). - BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel - builds before. + There is an issue with memory tracking and Unicode when built in "unity" + mode, which results in the curl tool crashing right on startup, even + without any command-line option. Interestingly this doesn't happen under + WINE (at least on the system I tested this on), but consistenly happens + on real Windows machines. Crash is 0xC0000374 heap corruption. Both + shared and static curl executables are affected. -- CI: drop the FreeBSD 12.X build + This limitation probably won't hit too many people, but it remains + a TODO to find and fix the root cause and drop this workaround. - Cirrus' new free tier won't let us have many builds, so drop the - nonessential ones. The FreeBSD 13.X build will still give us the most - relevant FreeBSD coverage. + Example builds and runs: + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/17cptxhtpubd + 7iwj#L313 (static) + https://ci.appveyor.com/project/curlorg/curl/builds/48169111/job/76e1ge758tby + qu9c#L317 (shared) -- CI: move the Alpine build from Cirrus to GHA + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 #11095 - Cirrus is reducing their free tier to next to nothing, so we must move - builds elsewhere. + Ref: #11928 + Closes #12005 -Stefan Eissing (30 Aug 2023) +- cmake: tidy-up `NOT_NEED_LBER_H` detection -- test_07_upload.py: fix test_07_34 curl args + Follow-up to 772f0d8edf1c3c2745543f42388ccec5a16ee2c0 #12006 - - Pass correct filename to --data-binary. +- appveyor: rewrite batch in PowerShell + CI improvements - Prior to this change --data-binary was passed an incorrect filename due - to a missing separator in the arguments list. Since aacbeae7 curl will - error on incorrect filenames for POST. + 1. Rewrite in PowerShell: - Fixes https://github.com/curl/curl/issues/11761 - Closes https://github.com/curl/curl/pull/11763 + - rewrite MS-DOS batch build script in PowerShell. + - move some bash operations into native PowerShell. + - fixups for PowerShell insisting on failure when a command outputs + something to stderr. + - fix to actually run `curl -V` after every build. + (and exclude ARM64 builds.) + - also say why we skipped `curl -V` if we had to skip. + - fix CMake warnings about unused configuration variables, by adapting + these dynamically for build cases. + - dedupe OpenSSL path into a variable. + - disable `test1451` failing with a warning anyway due to missing python + impacket. (after trying and failing to install impacket) + PowerShell promotes these warnings to errors by PowerShell. We can also + suppress they wholesale if they start causing issues in the future, + like we already to with `autoreconf` and `./configure`. -Nicholas Nethercote (30 Aug 2023) + PowerShell is better than MS-DOS batches, so the hope is this makes it + easier to extend and maintain the AppVeyor build logic. POSIX/bash isn't + supported inline by AppVeyor on Windows build machines, but we are okay + to keep it in an external script, so it's also an option. -- tests: document which tests fail due to hyper's lack of trailer support. + 2. CI improvements: - Closes #11762 + - enable tests for a "unity" build job. + - speed-up CI initialization by using shallow clones of the curl repo. + - speed-up CMake MSVC jobs with `TrackFileAccess=false`. + - enable parallelism in `VisualStudioSolution` builds. + - display CMake version before builds. + - always show the CPU in job names. + - tell which jobs are build-only in job names. + - move `TESTING:` value next to `DISABLED_TESTS:` in two jobs. + - add `config.log` (autotools) to dumped logs (need to enable manually). -- docs: removing "pausing transfers" from HYPER.md. + 3. Style: - It's a reference to #8600, which was fixed by #9070. + - use single-quotes in YAML like we do in other CI YAML files. + It also allows to drop quoting characters and lighter to write/read. + (keep double quotes for PowerShell strings needing expansion.) - Closes #11764 + Closes #11999 -Patrick Monnerat (30 Aug 2023) +- cmake: fix `HAVE_LDAP_SSL`, `HAVE_LDAP_URL_PARSE` on non-Windows -- os400: handle CURL_TEMP_PRINTF() while building bind source + - set `HAVE_LDAP_URL_PARSE` if `ldap_url_parse` function exists. + Before this patch we set it based it on the presence of `stricmp`, + which correctly enabled it on e.g. Windows, but was inaccurate for + other platforms. - Closes #11547 + - always set `HAVE_LDAP_SSL` if an LDAP backend is detected and + LDAPS is not explicitly disabled. This mimics autotools behaviour. + Previously we set it only for Windows LDAP. After this fix, LDAPS is + correctly enabled in default macOS builds. -- os400: build test servers + - enable LDAP[S] for a CMake macOS CI job. Target OS X 10.9 (Mavericks) + to avoid deprecation warnings for LDAP API. - Also fix a non-compliant main prototype in disabled.c. + - always detect `HAVE_LDAP_SSL_H`, even with LDAPS explicitly disabled. + This doesn't make much sense, but let's do it to sync behaviour with + autotools. - Closes #11547 + - fix benign typo in variable name. -- tests: fix compilation error for os400 + Ref: #11964 (effort to sync cmake detections with autotools) - OS400 uses BSD 4.3 setsockopt() prototype by default: this does not - define parameter as const, resulting in an error if actual parameter is - const. Remove the const keyword from the actual parameter cast: this - works in all conditions, even if the formal parameter uses it. + Closes #12006 - Closes #11547 +- autotools: restore `HAVE_IOCTL_*` detections -- os400: make programs and command name configurable + This restores `CURL_CHECK_FUNC_IOCTL` detection. I deleted it in + 4d73854462f30948acab12984b611e9e33ee41e6 and + c3456652a0c72d1845d08df9769667db7e159949 (2022-08), because the + `HAVE_IOCTL` result it generated was unused in the source. But, + I did miss the fact that this had two dependent checks: + `CURL_CHECK_FUNC_IOCTL_FIONBIO`, + `CURL_CHECK_FUNC_IOCTL_SIOCGIFADDR` that we do actually need: + `HAVE_IOCTL_FIONBIO`, `HAVE_IOCTL_SIOCGIFADDR`. - Closes #11547 + Regression from 4d73854462f30948acab12984b611e9e33ee41e6 -- os400: move build configuration parameters to a separate script + Ref: #11964 (effort to sync cmake detections with autotools) - They can then easily be overriden in a script named "config400.override" - that is not part of the distribution. + Closes #12008 - Closes #11547 +Daniel Stenberg (2 Oct 2023) -- os400: implement CLI tool +- RELEASE-PROCEDURE.md: updated coming release dates - This is provided as a QADRT (ascii) program, a link to it in the IFS and - a minimal CL command. +- RELEASE-NOTES: synced - Closes #11547 +Viktor Szakats (1 Oct 2023) -Matthias Gatto (30 Aug 2023) +- cmake: pre-cache `HAVE_POLL_FINE` on Windows -- lib: fix aws-sigv4 having date header twice in some cases + Windows doesn't support `poll()`, so we can safely skip checking for + fine poll. - When the user was providing the header X-XXX-Date, the header was - re-added during signature computation, and we had it twice in the - request. + Closes #12003 - Reported-by: apparentorder@users.noreply.github.com +- gha: bump actions to latest versions - Signed-off-by: Matthias Gatto + - actions@checkout@v4 (from v3 and v2) - Fixes: https://github.com/curl/curl/issues/11738 - Closes: https://github.com/curl/curl/pull/11754 + - fsfe/reuse-action@v2 (from v1) -Jay Satiro (30 Aug 2023) + Closes #12000 -- multi: remove 'processing: ' debug message +Stefan Eissing (30 Sep 2023) - - Remove debug message added by e024d566. +- h2: testcase and fix for pausing h2 streams - Closes https://github.com/curl/curl/pull/11759 + - refs #11982 where it was noted that paused transfers may + close successfully without delivering the complete data + - made sample poc into tests/http/client/h2-pausing.c and + added test_02_27 to reproduce -- ftp: fix temp write of ipv6 address + Closes #11989 + Fixes #11982 + Reported-by: Harry Sintonen - - During the check to differentiate between a port and IPv6 address - without brackets, write the binary IPv6 address to an in6_addr. +Viktor Szakats (30 Sep 2023) - Prior to this change the binary IPv6 address was erroneously written to - a sockaddr_in6 'sa6' when it should have been written to its in6_addr - member 'sin6_addr'. There's no fallout because no members of 'sa6' are - accessed before it is later overwritten. +- cmake: validate `CURL_DEFAULT_SSL_BACKEND` config value - Closes https://github.com/curl/curl/pull/11747 + Before this patch CMake builds accepted any value and it was used at + runtime as-is. This patch make sure that the selected default backend + is also enabled in the build. It also enforces a full lowercase value. -- tool: change some fopen failures from warnings to errors + This improves reproducibility and brings CMake in sync with autotools + which already worked like described above. - - Error on missing input file for --data, --data-binary, - --data-urlencode, --header, --variable, --write-out. + Follow-up to 26c7feb8b9d51a57fab3325571b4bbfa03b11af0 #11774 - Prior to this change if a user of the curl tool specified an input file - for one of the above options and that file could not be opened then it - would be treated as zero length data instead of an error. For example, a - POST using `--data @filenametypo` would cause a zero length POST which - is probably not what the user intended. + Closes #11998 - Closes https://github.com/curl/curl/pull/11677 +- autotools: adjust `CURL_CA_PATH` value to CMake -- hostip: fix typo + autotools was using the same value as CMake, but with an ending + slash. Delete the ending slash to match configurations. -Davide Masserut (29 Aug 2023) + Ref: #11964 (effort to sync cmake detections with autotools) -- tool: avoid including leading spaces in the Location hyperlink + Closes #11997 - Co-authored-by: Dan Fandrich +- cmake: detect `sys/wait.h` and `netinet/udp.h` - Closes #11735 + Ref: #11964 (effort to sync cmake detections with autotools) -Daniel Stenberg (29 Aug 2023) + Closes #11996 -- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline +Daniel Stenberg (30 Sep 2023) - Closes #11757 +- lib: provide and use Curl_hexencode -- connect: stop halving the remaining timeout when less than 600 ms left + Generates a lower case ASCII hex output from a binary input. - When curl wants to connect to a host, it always has a TIMEOUT. The - maximum time it is allowed to spend until a connect is confirmed. + Closes #11990 - curl will try to connect to each of the IP adresses returned for the - host. Two loops, one for each IP family. +- configure: check for the capath by default - During the connect loop, while curl has more than one IP address left to - try within a single address family, curl has traditionally allowed (time - left/2) for *this* connect attempt. This, to not get stuck on the - initial addresses in case the timeout but still allow later addresses to - get attempted. + ... if the chosen TLS backend supports it: OpenSSL, GnuTLS, mbedTLS or wolfSS + L - This has the downside that when users set a very short timeout and the - host has a large number of IP addresses, the effective result might be - that every attempt gets a little too short time. + cmake: synced - This change stop doing the divided-by-two if the total time left is - below a threshold. This threshold is 600 milliseconds. + Assisted-by: Viktor Szakats + Closes #11987 - Closes #11693 +- wolfssl: ignore errors in CA path -- asyn-ares: reduce timeout to 2000ms + The default wolfSSL_CTX_load_verify_locations() function is quite picky + with the certificates it loads and will for example return error if just + one of the certs has expired. - When UDP packets get lost this makes for slightly faster retries. This - lower timeout is used by @c-ares itself by default starting next - release. + With the *_ex() function and its WOLFSSL_LOAD_FLAG_IGNORE_ERR flag, it + behaves more similar to what OpenSSL does by default. - Closes #11753 + Even the set of default certs on my Debian unstable has several expired + ones. -John Bampton (29 Aug 2023) + Assisted-by: Juliusz Sosinowicz + Assisted-by: Michael Osipov -- misc: remove duplicate words + Closes #11987 - Closes #11740 +- create-dirs.d: clarify it also uses --output-dirs -Daniel Stenberg (29 Aug 2023) + Reported-by: Robert Simpson + Fixes #11991 + Closes #11995 -- RELEASE-NOTES: synced +Viktor Szakats (30 Sep 2023) -- wolfSSL: avoid the OpenSSL compat API when not needed +- appveyor: fix yamlint issues, indent - ... and instead call wolfSSL functions directly. + Also: + - use double quotes in all batch if statements. - Closes #11752 + Closes #11994 -Viktor Szakats (28 Aug 2023) +- cmake: detect `HAVE_CLOCK_GETTIME_MONOTONIC_RAW` -- lib: fix null ptr derefs and uninitialized vars (h2/h3) + Based on existing autotools logic. - Fixing compiler warnings with gcc 13.2.0 in unity builds. + Ref: #11964 (effort to sync cmake detections with autotools) - Assisted-by: Jay Satiro - Assisted-by: Stefan Eissing - Closes #11739 + Closes #11981 -Jay Satiro (28 Aug 2023) +- cmake: detect `HAVE_GETADDRINFO_THREADSAFE` -- secureserver.pl: fix stunnel version parsing + Based on existing autotools logic. - - Allow the stunnel minor-version version part to be zero. + autotools checks for old versions of the allowlisted target OSes and + disables this feature when seeing them. In CMake we assume we're running + on newer systems and enable regardless of OS version. - Prior to this change with the stunnel version scheme of . - if either part was 0 then version parsing would fail, causing - secureserver.pl to fail with error "No stunnel", causing tests that use - the SSL protocol to be skipped. As a practical matter this bug can only - be caused by a minor-version part of 0, since the major-version part is - always greater than 0. + autotools always runs all 3 probes for non-fast-tracked systems and + enables this feature if any one of them was successful. To save + configuration time, CMake stops at the first successful check. - Closes https://github.com/curl/curl/pull/11722 + OpenBSD is not fast-tracked and then gets blocklisted as a generic BSD + system. I haven't double-checked if this is correct, but looks odd. -- secureserver.pl: fix stunnel path quoting + Ref: #11964 (effort to sync cmake detections with autotools) - - Store the stunnel path in the private variable $stunnel unquoted and - instead quote it in the command strings. + Closes #11979 - Prior to this change the quoted stunnel path was passed to perl's file - operators which cannot handle quoted paths. For example: +- cmake: fix `HAVE_WRITABLE_ARGV` detection - $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; - if(-x $stunnel or -x "$stunnel") - # false even if path exists and is executable + Move detection before the creation of detection results in + `curl_config.h`. - Our other test scripts written in perl, unlike this one, use servers.pm - which has a global $stunnel variable with the path stored unquoted and - therefore those scripts don't have this problem. + Ref: #11964 (effort to sync cmake detections with autotools) - Closes https://github.com/curl/curl/pull/11721 + Closes #11978 -Daniel Stenberg (28 Aug 2023) +- appveyor: minor improvements -- altsvc: accept and parse IPv6 addresses in response headers + - run `curl -V` after builds to see if they run and with what features. + Except for one job where a CRT DLL is missing. And ARM64 which should + fail, but is silently not launched instead. - Store numerical IPv6 addresses in the alt-svc file with the brackets - present. + - copy libcurl DLL next to curl tool and tests binaries in shared mode. + This makes it possible to run the tests. (We don't run tests after + these builds yet.) - Verify with test 437 and 438 + - list the DLLs and EXEs present after the builds. - Fixes #11737 - Reported-by: oliverpool on github - Closes #11743 + - add `DEBUG` variable for CMake builds to allow disabling it, for + testing non-debug builds. (currently enabled for all) -- libtest: use curl_free() to free libcurl allocated data + - add commented lines that dump CMake configuration logs for debugging + build/auto-detection issues. - In several test programs. These mistakes are not detected or a problem - as long as memdebug.h is included, as that provides the debug wrappers - for all memory functions in the same style libcurl internals do it, - which makes curl_free and free effectively the same call. + - add gcc version to jobs where missing. - Reported-by: Nicholas Nethercote - Closes #11746 + - switch a job to the native MSYS2 mingw-w64 toolchain. This adds gcc 9 + to the build mix. -Jay Satiro (28 Aug 2023) + - make `SHARED=OFF` and `OPENSSL=OFF` defaults global. -- disable.d: explain --disable not implemented prior to 7.50.0 + - delete a duplicate backslash. - Option -q/--disable was added in 5.0 but only -q was actually - implemented. Later --disable was implemented in e200034 (precedes - 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). + Closes #11976 - Reported-by: pszlazak@users.noreply.github.com +- configure: replace adhoc domain with `localhost` in tests - Fixes https://github.com/curl/curl/issues/11710 - Closes #11712 + Reviewed-by: Daniel Stenberg + Closes #11988 -Nicholas Nethercote (28 Aug 2023) +- tidy-up: use more example domains -- hyper: fix ownership problems + Also make use of the example TLD: + https://en.wikipedia.org/wiki/.example - Some of these changes come from comparing `Curl_http` and - `start_CONNECT`, which are similar, and adding things to them that are - present in one and missing in another. + Reviewed-by: Daniel Stenberg + Closes #11992 - The most important changes: - - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the - happy path. - - In `start_CONNECT`, add a missing `hyper_request_free` on the error - path. - - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. - - In `bodysend`, remove an unnecessary `hyper_body_free` on a different - error path that would cause a double-free. - https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html - says of `hyper_request_set_body`: "This takes ownership of the - hyper_body *, you must not use it or free it after setting it on the - request." This is true even if `hyper_request_set_body` returns an - error; I confirmed this by looking at the hyper source code. +Dan Fandrich (29 Sep 2023) - Other changes are minor but make things slightly nicer. +- runtests: display the test status if tests appear hung - Closes #11745 + It sometimes happens that a test hangs during a test run and never + returns. The test harness will wait indefinitely for the results and on + CI servers the CI job will eventually be killed after an hour or two. + At the end of a test run, if results haven't come in within a couple of + minutes, display the status of all test runners and what tests they're + running to help in debugging the problem. -Daniel Stenberg (28 Aug 2023) + This feature is really only kick in with parallel testing enabled, which + is fine because without parallel testing it's usually easy to tell what + test has hung. -- multi.h: the 'revents' field of curl_waitfd is supported + Closes #11980 - Since 6d30f8ebed34e7276 +- github/labeler: remove workaround for labeler - Reported-by: Nicolás Ojeda Bär - Ref: #11748 - Closes #11749 + This was added due to what seemed to be a bug regarding the sync-labels: + config option, but it looks like it wasn't necessary. -Gerome Fournier (27 Aug 2023) + Follow-up to b2b0534e7 -- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() +Viktor Szakats (29 Sep 2023) - Closes #11742 +- docs: upgrade an URL to HTTPS in `BINDINGS.md` [ci skip] -Daniel Stenberg (27 Aug 2023) +Daniel Stenberg (29 Sep 2023) -- docs: mention critical files in same directories as curl saves +- docs: replace made up domains with example.com - ... cannot be fully protected. Don't do it. + in FAQ and MANUAL.md - Co-authored-by: Jay Satiro - Reported-by: Harry Sintonen - Fixes #11530 - Closes #11701 + - example.com was made for this purpose. -John Hawthorn (26 Aug 2023) + - reduces the risk that one of those domains suddenly start hosting + something nasty and we provide links to them -- OpenSSL: clear error queue after SSL_shutdown + Closes #11986 - We've seen errors left in the OpenSSL error queue (specifically, - "shutdown while in init") by adding some logging it revealed that the - source was this file. +Michael Osipov (29 Sep 2023) - Since we call SSL_read and SSL_shutdown here, but don't check the return - code for an error, we should clear the OpenSSL error queue in case one - was raised. +- acinclude.m4: Document proper system truststore on FreeBSD - This didn't affect curl because we call ERR_clear_error before every - write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when - libcurl is used in a process with other OpenSSL users, they may detect - an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their - own. + The default system truststore on FreeBSD has been /etc/ssl/certs for many + years now. It is managed canonically through certctl(8) and contains hashed + symlinks for OpenSSL and other TLS providers. + The previous ones require security/ca_root_nss which might not be installed o + r + will not contain any custom CA certificates. - Co-authored-by: Satana de Sant'Ana + Closes #11985 - Closes #11736 +Daniel Stenberg (29 Sep 2023) -Alexander Kanavin (25 Aug 2023) +- FAQ: How do I upgrade curl.exe in Windows? -- tests: update cookie expiry dates to far in the future + This is a growing question, better answer it here to get somewhere to + point users to. - This allows testing Y2038 with system time set to after that, so that - actual Y2038 issues can be exposed, and not masked by expiry errors. + Closes #11984 - Fixes #11576 - Closes #11610 +Viktor Szakats (28 Sep 2023) -John Bampton (25 Aug 2023) +- cmake: pre-cache `HAVE_BASENAME` for mingw-w64 and MSVC -- misc: fix spelling + `basename` is present in mingw-w64, missing from MSVC. Pre-cache + accordingly to make configure faster. - Closes #11733 + Notice that `basename` has a bug so we later disable it even with + mingw-w64: + https://github.com/curl/curl/blob/781242ffa44a9f9b95b6da5ac5a1bf6372ec6257/li + b/curl_setup.h#L820-L825 -Daniel Stenberg (25 Aug 2023) + Closes #11974 -- cmdline-opts/page-header: clarify stronger that !opt == URL +Daniel Stenberg (28 Sep 2023) - Everything provided on the command line that is not an option (or an - argument to an option) is treated as a URL. +- cmake: add missing checks - Closes #11734 + - check for arc4random. To make rand.c use it accordingly. + - check for fcntl + - fix fseek detection + - add SIZEOF_CURL_SOCKET_T + - fix USE_UNIX_SOCKETS + - define HAVE_SNPRINTF to 1 + - check for fnmatch + - check for sched_yield + - remove HAVE_GETPPID duplicate from curl_config.h + - add HAVE_SENDMSG -- tests/runner: fix %else handling + Ref: #11964 - Getting the show state proper for %else and %endif did not properly work - in nested cases. + Co-authored-by: Viktor Szakats + Closes #11973 - Follow-up to 3d089c41ea9 +- configure: remove unused checks - Closes #11731 + - for sys/uio.h + - for fork + - for connect -Nicholas Nethercote (25 Aug 2023) + Ref: #11964 -- docs: Remove mention of #10803 from `KNOWN_BUGS`. + Closes #11973 - Because the leaks have been fixed. +- lib: remove TIME_WITH_SYS_TIME -- c-hyper: fix another memory leak in `Curl_http`. + It is not used in any code anywhere. - There is a `hyper_clientconn_free` call on the happy path, but not one - on the error path. This commit adds one. + Ref: #11964 + Closes #11975 - Fixes the second memory leak reported by Valgrind in #10803. +- docs: update curl man page references - Fixes #10803 - Closes #11729 + Detected by the manpage-syntax update -- c-hyper: fix a memory leak in `Curl_http`. + Closes #11963 - A request created with `hyper_request_new` must be consumed by either - `hyper_clientconn_send` or `hyper_request_free`. +- manpage-syntax: verify curl man page references - This is not terrifically clear from the hyper docs -- - `hyper_request_free` is documented only with "Free an HTTP request if - not going to send it on a client" -- but a perusal of the hyper code - confirms it. + 1. References to curl symbols are now checked that they indeed exist as + man pages. This for \f references as well as the names referenced in the + SEE ALSO section. - This commit adds a `hyper_request_free` to the `error:` path in - `Curl_http` so that the request is consumed when an error occurs after - the request is created but before it is sent. + Allowlist curl.1 since it is not always built in builds - Fixes the first memory leak reported by Valgrind in #10803. + 2. References to curl symbols that lack section now causes warning, since tha + t + will prevent them from getting linked properly - Closes #11729 + 3. Check for "bare" references to curl functions and warn, they should be + references -Daniel Stenberg (25 Aug 2023) + Closes #11963 -- RELEASE-NOTES: synced +- cmake: add check for suseconds_t + + And fix the HAVE_LONGLONG define + + Ref: #11964 + Closes #11977 -John Bampton (25 Aug 2023) +Viktor Szakats (28 Sep 2023) -- misc: spellfixes +- tidy-up: whitespace fixes - Closes #11730 + Closes #11972 -Daniel Stenberg (25 Aug 2023) +- cmake: detect TLS-SRP in OpenSSL/wolfSSL/GnuTLS -- tests: add support for nested %if conditions + With new option `CURL_DISABLE_SRP=ON` to force-disable it. + To match existing option and detection logic in autotools. - Provides more flexiblity to test cases. + Also: + - fix detecting GnuTLS. + We assume `nettle` as a GnuTLS dependency. + - add CMake GnuTLS CI job. + - bump AppVeyor CMake OpenSSL MSVC job to OpenSSL 1.1.1 (from 1.0.2) + TLS-SRP fails to detect with 1.0.2 due to an OpenSSL header bug. + - fix compiler warning when building with GnuTLS and disabled TLS-SRP. + - fix comment typos, whitespace. - Also warn and bail out if there is an '%else' or %endif' without a - preceeding '%if'. + Ref: #11964 - Ref: #11610 - Closes #11728 + Closes #11967 -- time-cond.d: mention what happens on a missing file +- tool: use our own stderr variable - Closes #11727 + Earlier this year we changed our own stderr variable to use the standard + name `stderr` (to avoid bugs where someone is using `stderr` instead of + the curl-tool specific variable). This solution needed to override the + standard `stderr` symbol via the preprocessor. This in turn didn't play + well with unity builds and caused curl tool to crash or stay silent due + to an uninitialized stderr. This was a hard to find issue, fixed by + manually breaking out one file from the unity sources. -Christian Hesse (24 Aug 2023) + To avoid two these two tricks, this patch implements a different + solution: Restore using our own local variable for our stderr output and + leave `stderr` as-is. To avoid using `stderr` by mistake, add a + `checksrc` rule (based on logic we already used in lib for `strerror`) + that detects any `stderr` use in `src` and points to using our own + variable instead: `tool_stderr`. -- docs/cmdline-opts: match the current output + Follow-up to 06133d3e9b8aeb9e9ca0b3370c246bdfbfc8619e + Follow-up to 2f17a9b654121dd1ecf4fc043c6d08a9da3522db - The release date has been added in output, reflect that in documentation. + Closes #11958 - Closes #11723 +Loïc Yhuel (28 Sep 2023) -Daniel Stenberg (24 Aug 2023) +- connect: only start the happy eyeballs timer when needed -- lib: minor comment corrections + The timeout is only used when there is a second address family, for the + delayed eyeballer. -- docs: rewrite to present tense + Closes #11939 - ... instead of using future tense. +Daniel Stenberg (28 Sep 2023) - + numerous cleanups and improvements - + stick to "reuse" not "re-use" - + fewer contractions +- tool_operate: free 'gateway' correctly - Closes #11713 + Pointed out by Coverity. The fix in 93885cf3a8d4e was incomplete. -- urlapi: setting a blank URL ("") is not an ok URL + Also removed repeated wording in IPFS related error messages. - Test it in 1560 - Fixes #11714 - Reported-by: ad0p on github - Closes #11715 + Closes #11969 -- spelling: use 'reuse' not 're-use' in code and elsewhere +Stefan Eissing (28 Sep 2023) - Unify the spelling as both versions were previously used intermittently +- lib: move handling of `data->req.writer_stack` into Curl_client_write() - Closes #11717 + - move definitions from content_encoding.h to sendf.h + - move create/cleanup/add code into sendf.c + - installed content_encoding writers will always be called + on Curl_client_write(CLIENTWRITE_BODY) + - Curl_client_cleanup() frees writers and tempbuffers from + paused transfers, irregardless of protocol -Michael Osipov (23 Aug 2023) + Closes #11908 -- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC +Loïc Yhuel (28 Sep 2023) - HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the - default one. Use "long long" in 32 bit mode and just "long" in 64 bit - mode. +- multi: round the timeout up to prevent early wakeups - Closes #11718 + Curl_timediff rounds down to the millisecond, so curl_multi_perform can + be called too early, then we get a timeout of 0 and call it again. -Dan Fandrich (22 Aug 2023) + The code already handled the case of timeouts which expired less than + 1ms in the future. By rounding up, we make sure we will never ask the + platform to wake up too early. -- tests: don't call HTTP errors OK in test cases + Closes #11938 - Some HTTP errors codes were accompanied by the text OK, which causes - some cognitive dissonance when reading them. +Daniel Stenberg (28 Sep 2023) -- http: close the connection after a late 417 is received +- RELEASE-NOTES: spell out that IPFS is via gateway - In this situation, only part of the data has been sent before aborting - so the connection is no longer usable. +- RELEASE-NOTES: synced - Assisted-by: Jay Satiro - Fixes #11678 - Closes #11679 +- tool_operate: avoid strlen() -1 on zero length content from file -- runtests: slightly increase the longest log file displayed + Follow-up to 65b563a96a226649ba12cb1e - The new limit provides enough space for a 64 KiB data block to be logged - in a trace file, plus a few lines at the start and end for context. This - happens to be the amount of data sent at a time in a PUT request. + Closes #11959 -- tests: add delay command to the HTTP server +- tool_operate: fix memory mixups - This adds a delay after client connect. + Switch to plain getenv() from curl_getenv() to avoid the allocation and + having to keep track of which free() or curl_free() that need to be + used. -Daniel Stenberg (22 Aug 2023) + Coverity found issues and a memory leak. -- cirrus: install everthing with pkg, avoid pip + Follow-up to 65b563a96a226649ba12cb1e - Assisted-by: Sevan Janiyan + Closes #11959 - Closes #11711 +Viktor Szakats (27 Sep 2023) -- curl_url*.3: update function descriptions +- curl-functions.m4: fixup recent bad edits - - expand and clarify several descriptions - - avoid using future tense all over + Follow-up to 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Closes #11708 + Closes #11966 -- RELEASE-NOTES: synced +Daniel Stenberg (27 Sep 2023) -Stefan Eissing (21 Aug 2023) +- curl-functions.m4: fix include line -- CI/cirrus: disable python install on FreeBSD + This made the getaddrinfo detection fail, but we did not spot it in the + CI because it graciously falled back to using legacy functions instead! - - python cryptography package does not build build FreeBSD - - install just mentions "error" - - this gets the build and the main test suite going again + Follow-up to 96c29900bcec (#11940) - Closes #11705 + Closes #11965 -- test2600: fix flakiness on low cpu +- inet_ntop: add typecast to silence Coverity - - refs #11355 where failures to to low cpu resources in CI - are reported - - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations - to test cases - - trigger Curl_expire() in test filter to allow re-checks before - the usual 1second interval + CID 1024653: Integer handling issues (SIGN_EXTENSION) - Closes #11690 + Suspicious implicit sign extension: "src[i]" with type "unsigned char + const" (8 bits, unsigned) is promoted in "src[i] << (1 - i % 2 << 3)" to + type "int" (32 bits, signed), then sign-extended to type "unsigned long" + (64 bits, unsigned). If "src[i] << (1 - i % 2 << 3)" is greater than + 0x7FFFFFFF, the upper bits of the result will all be 1. -Maksim Sciepanienka (20 Aug 2023) + 111 words[i/2] |= (src[i] << ((1 - (i % 2)) << 3)); -- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf + The value will not be greater than 0x7FFFFFFF so this still cannot + happen. - Closes #11698 + Also, switch to ints here instead of longs. The values stored are 16 bit + so at least no need to use 64 bit variables. Also, longs are 32 bit on + some platforms so this logic still needs to work with 32 bits. -Daniel Stenberg (20 Aug 2023) + Closes #11960 -- test687/688: two more basic --xattr tests +- docs: adapt SEE ALSO sections to new requirements - Closes #11697 + To please manpage-syntax.pl used by test 1173 -- cmdline-opts/docs: mentioned the negative option part + Closes #11957 - ... for --no-alpn and --no-buffer in the same style done for other --no- - options: +- manpage-syntax.pl: verify SEE ALSO syntax - "Note that this is the negated option name documented." + - Enforce a single reference per .BR line + - Skip the quotes around the section number for example (3) + - Insist on trailing commas on all lines except the last + - Error on comma on the last SEE ALSO entry - Closes #11695 + - List the entries alpha-sorted, not enforced just recommended -Emanuele Torre (19 Aug 2023) + Closes #11957 -- tool/var: also error when expansion result starts with NUL +- connect: expire the timeout when trying next - Expansions whose output starts with NUL were being expanded to the empty - string, and not being recognised as values that contain a NUL byte, and - should error. + ... so that it gets called again immediately and can continue trying + addresses to connect to. Otherwise it might unnecessarily wait for a + while there. - Closes #11694 + Fixes #11920 + Reported-by: Loïc Yhuel + Closes #11935 -Daniel Stenberg (19 Aug 2023) +- http: remove wrong comment for http_should_fail -- tests: add 'large-time' as a testable feature + Reported-by: Christian Schmitz + Ref: #11936 + Closes #11941 - This allows test cases to require this feature to run and to be used in - %if conditions. +Dan Fandrich (26 Sep 2023) - Large here means larger than 32 bits. Ie does not suffer from y2038. +- tool_setopt: remove unused function tool_setopt_flags - Closes #11696 + This function is identical to tool_setopt_bitmask except that it treats + the argument as unsigned. -- tests/Makefile: add check-translatable-options.pl to tarball + Closes #11943 - Used in test 1544 +Viktor Szakats (26 Sep 2023) - Follow-up to ae806395abc8c +- cmake: add feature checks for `memrchr` and `getifaddrs` -- gen.pl: fix a long version generation mistake + - `HAVE_MEMRCHR` for `memrchr`. + - `HAVE_GETIFADDRS` for `getifaddrs`. + This was present in `lib/curl_config.h.cmake` but missed the detection + logic. - Too excessive escaping made the parsing not find the correct long names - later and instead add "wrong" links. + To match existing autotools feature checks. - Follow-up to 439ff2052e219 + Closes #11954 - Reported-by: Lukas Tribus - Fixes #11688 - Closes #11689 +- cmake: move global headers to specific checks -- lib: move mimepost data from ->req.p.http to ->state + Before this patch we added standard headers unconditionally to the + global list of headers used for feature checks. This is unnecessary + and also doesn't help CMake 'Generate' performance. This patch moves + these headers to each feature check where they are actually needed. + Stop using `stddef.h`, as it seems unnecessary. - When the legacy CURLOPT_HTTPPOST option is used, it gets converted into - the modem mimpost struct at first use. This data is (now) kept for the - entire transfer and not only per single HTTP request. This re-enables - rewind in the beginning of the second request instead of in end of the - first, as brought by 1b39731. + I've used autotools' `m4/curl-functions.m4` to figure out these + dependencies. - The request struct is per-request data only. + Also delete checking for the C89 standard header `time.h`, that I + missed in the earlier commit. - Extend test 650 to verify. + Ref: 96c29900bcec32dd6bc8e9857c8871ff4b8b8ed9 #11940 - Fixes #11680 - Reported-by: yushicheng7788 on github - Closes #11682 + Closes #11951 -Patrick Monnerat (17 Aug 2023) +- src/mkhelp: make generated code pass `checksrc` -- os400: do not check translatable options at build time + Closes #11955 - Now that there is a test for this, the build time check is not needed - anymore. +- tests: show which curl tool `runtests.pl` is using - Closes #11650 + To help debugging when there is issue finding or running it. -- test1554: check translatable string options in OS400 wrapper + Closes #11953 - This test runs a perl script that checks all string options are properly - translated by the OS400 character code conversion wrapper. It also - verifies these options are listed in alphanumeric order in the wrapper - switch statement. +- CI/azure: make `MAKEFLAGS` global to parallelize all jobs - Closes #11650 + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17528 (before) + https://dev.azure.com/daniel0244/curl/_build/results?buildId=17545 (after, wi + th -j3) -Daniel Stenberg (17 Aug 2023) + Closes #11952 -- unit3200: skip testing if function is not present +- CI/azure: migrate old mingw MSYS1 jobs to MSYS2 - Fake a successful run since we have no easy mechanism to skip this test - for this advanced condition. + Also delete an accidental variable reference. -- unit2600: fix build warning if built without verbose messages + Follow-up to 38029101e2d78ba125732b3bab6ec267b80a0e72 -- test1608: make it build and get skipped without shuffle DNS support + Closes #11945 -- lib: --disable-bindlocal builds curl without local binding support +Daniel Stenberg (26 Sep 2023) -- test1304: build and skip without netrc support +- docs: add see also curl_multi_get_handles to some man pages -- lib: build fixups when built with most things disabled + Assisted-by: Jay Satiro - Closes #11687 + Closes #11942 -- workflows/macos.yml: disable zstd and alt-svc in the http-only build +Viktor Szakats (26 Sep 2023) - Closes #11683 +- cmake: assume `_fseeki64` and no `fseeko` on Windows -Stefan Eissing (17 Aug 2023) + `_fseeki64` is present in mingw-w64 1.0 (2011-09-26) headers, and + at least Watcom C 1.9 (2010) headers and MSVS 2008 [1]. -- bearssl: handshake fix, provide proper get_select_socks() implementation + `fseeko` is not present in any of these. - - bring bearssl handshake times down from +200ms down to other TLS backends - - vtls: improve generic get_select_socks() implementation - - tests: provide Apache with a suitable ssl session cache + (mingw-w64 1.0 also offers `fseeko64`.) - Closes #11675 + [1] https://github.com/curl/curl/pull/11944#issuecomment-1734995004 -- tests: TLS session sharing test + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 - - test TLS session sharing with special test client - - expect failure with wolfSSL - - disable flaky wolfSSL test_02_07b + Closes #11950 - Closes #11675 +- build: delete checks for C89 standard headers -Daniel Stenberg (17 Aug 2023) + Delete checks and guards for standard C89 headers and assume these are + available: `stdio.h`, `string.h`, `time.h`, `setjmp.h`, `stdlib.h`, + `stddef.h`, `signal.h`. -- CURLOPT_*TIMEOUT*: extend and clarify + Some of these we already used unconditionally, some others we only used + for feature checks. - Closes #11686 + Follow-up to 9c7165e96a3a9a2d0b7059c87c699b5ca8cdae93 #11918 (for `stdio.h` i + n CMake) -- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails + Closes #11940 - And document it. Only return out of memory when it actually is a memory - problem. +Stefan Eissing (26 Sep 2023) - Pointed-out-by: Jacob Mealey - Closes #11674 +- multiif.h: remove Curl_multi_dump declaration -Mathew Benson (17 Aug 2023) + Follow-up to d850eea2 which removed the Curl_multi_dump definition. -- cmake: add GnuTLS option + Closes https://github.com/curl/curl/pull/11946 - - Option to use GNUTLS was missing. Hence was not able to use GNUTLS - with ngtcp2 for http3. +Jay Satiro (26 Sep 2023) - Closes #11685 +- config-win32: define HAVE__FSEEKI64 -Daniel Stenberg (16 Aug 2023) + Follow-up to 9c7165e9 which added an fseeko wrapper to the lib that + calls _fseeki64 if it is available. -- RELEASE-NOTES: synced + Closes https://github.com/curl/curl/pull/11944 -- http: remove the p_pragma struct field +- docs: explain how PINNEDPUBLICKEY is independent of VERIFYPEER - unused since 40e8b4e52 (2008) + - Explain that peer verification via CURLOPT_PINNEDPUBLICKEY takes place + even if peer verification via CURLOPT_SSL_VERIFYPEER is turned off. - Closes #11681 + The behavior is verified by test2048. -Jay Satiro (16 Aug 2023) + Bug: https://github.com/curl/curl/issues/2935#issuecomment-418371872 + Reported-by: claudiusaiz@users.noreply.github.com -- CURLINFO_CERTINFO.3: better explain curl_certinfo struct + Bug: https://github.com/curl/curl/discussions/11910 + Reported-by: Hakan Sunay Halil - Closes https://github.com/curl/curl/pull/11666 + Closes https://github.com/curl/curl/pull/11930 -- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation +Stefan Eissing (26 Sep 2023) - - Remove the out-of-date SSL backend list supported by - CURLOPT_SSL_CTX_FUNCTION. +- openssl: improve ssl shutdown handling - It makes more sense to just refer to that document instead of having - a separate list that has to be kept in sync. + - If SSL shutdown is not finished then make an additional call to + SSL_read to gather additional tracing. - Closes https://github.com/curl/curl/pull/11665 + - Fix http2 and h2-proxy filters to forward do_close() calls to the next + filter. -- write-out.d: clarify %{time_starttransfer} + For example h2 and SSL shutdown before and after this change: - sync it up with CURLINFO_STARTTRANSFER_TIME_T + Before: -Daniel Stenberg (15 Aug 2023) + Curl_conn_close -> cf_hc_close -> Curl_conn_cf_discard_chain -> + ssl_cf_destroy -- transfer: don't set TIMER_STARTTRANSFER on first send + After: - The time stamp is for measuring the first *received* byte + Curl_conn_close -> cf_hc_close -> cf_h2_close -> cf_setup_close -> + ssl_cf_close - Fixes #11669 - Reported-by: JazJas on github - Closes #11670 + Note that currently the tracing does not show output on the connection + closure handle. Refer to discussion in #11878. -trrui-huawei (15 Aug 2023) + Ref: https://github.com/curl/curl/discussions/11878 -- quiche: enable quiche to handle timeout events + Closes https://github.com/curl/curl/pull/11858 - In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` - interface for the application to invoke upon timer - expiration. Therefore, invoking the `on_timeout` function of the - Connection is crucial to ensure seamless functionality of quiche with - timeout events. +Loïc Yhuel (26 Sep 2023) - Closes #11654 +- multi: fix small timeouts -- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s + Since Curl_timediff rounds down to the millisecond, timeouts which + expire in less than 1ms are considered as outdated and removed from the + list. We can use Curl_timediff_us instead, big timeouts could saturate + but this is not an issue. - Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. + Closes #11937 -Daniel Stenberg (15 Aug 2023) +Viktor Szakats (25 Sep 2023) -- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang +- cmake: fix stderr initialization in unity builds - Closes #9580 + Before this patch, in certain build configurations the curl tool may + not have displayed anything (debug, macOS), or crashed at startup + (debug, Windows). -- imap: add a check for failing strdup() + Follow-up to 3f8fc25720900b14b7432f4bd93407ca15311719 + Necessary after 2f17a9b654121dd1ecf4fc043c6d08a9da3522db -- imap: remove the only sscanf() call in the IMAP code + Closes #11929 - Avoids the use of a stack buffer. +- cmake: fix missing `zlib.h` when compiling `libcurltool` - Closes #11673 + Came up while testing debug/testing build for Windows. I'm not sure why + it didn't come up in earlier tests with similar config. + `tool_hugehelp.c` might indeed require `zlib.h` and without linking + `CURL_LIBS` to the `curltool` target, CMake doesn't seem to add detected + dependency headers to the compiler command. -- imap: use a dynbuf in imap_atom + ``` + [ 25%] Building C object src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + cd .../curl/bld-cmake-llvm-x64/src && /usr/local/opt/llvm/bin/clang + --target=x86_64-w64-mingw32 --sysroot=/usr/local/opt/mingw-w64/toolchain-x8 + 6_64 + -DCURLDEBUG -DCURL_STATICLIB -DHAVE_CONFIG_H -DUNICODE -DUNITTESTS -D_UNICO + DE + -I.../curl/include -I.../curl/lib -I.../curl/bld-cmake-llvm-x64/lib + -I.../curl/bld-cmake-llvm-x64/include -I.../curl/src -Wno-unused-command-li + ne-argument + -D_UCRT -DDEBUGBUILD -DHAS_ALPN -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static + -libgcc + -lucrt [...] -O3 -DNDEBUG -municode -MD + -MT src/CMakeFiles/curltool.dir/tool_hugehelp.c.obj + -MF CMakeFiles/curltool.dir/tool_hugehelp.c.obj.d + -o CMakeFiles/curltool.dir/tool_hugehelp.c.obj -c .../curl/bld-cmake-llvm-x + 64/src/tool_hugehelp.c + .../curl/bld-cmake-llvm-x64/src/tool_hugehelp.c:6:10: fatal error: 'zlib.h' f + ile not found + 6 | #include + | ^~~~~~~~ + ``` - Avoid a calculation + malloc. Build the output in a dynbuf. + Follow-up to 39e7c22bb459c2e818f079984989a26a09741860 - Closes #11672 + Closes #11927 -Marin Hannache (14 Aug 2023) +- cmake: fix duplicate symbols when linking tests -- http: do not require a user name when using CURLAUTH_NEGOTIATE + The linker resolves this automatically in non-unity builds. In unity + builds the linker cannot drop a single object with the duplicates, + resulting in these errors. The root issue is that we started including + certain objects both via both libcurlu and libcurltool libs. + + Regression from 39e7c22bb459c2e818f079984989a26a09741860 + + Windows errors: + ``` + [ 3%] Linking C executable unit1303.exe + [ 3%] Building C object tests/server/CMakeFiles/rtspd.dir/__/__/lib/curl_mul + tibyte.c.obj + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_UTF8_to_wch + ar': + C:/projects/curl/lib/curl_multibyte.c:44: multiple definition of `curlx_conve + rt_UTF8_to_wchar' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:44: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_convert_wchar_to_UT + F8': + C:/projects/curl/lib/curl_multibyte.c:66: multiple definition of `curlx_conve + rt_wchar_to_UTF8' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:66: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_open': + C:/projects/curl/lib/curl_multibyte.c:92: multiple definition of `curlx_win32 + _open' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:92: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_fopen': + C:/projects/curl/lib/curl_multibyte.c:120: multiple definition of `curlx_win3 + 2_fopen' + ../../src/libcurltool-d.a(unity_0.c.obj):C:/projects/curl/lib/curl_multibyte. + c:120: first defined here + ../../lib/libcurlu-d.a(unity_0.c.obj): In function `curlx_win32_stat': + [...] + ``` + Ref: https://ci.appveyor.com/project/curlorg/curl/builds/48110107/job/nvlhpt9 + aa4ehny5q#L247 - In order to get Negotiate (SPNEGO) authentication to work in HTTP you - used to be required to provide a (fake) user name (this concerned both - curl and the lib) because the code wrongly only considered - authentication if there was a user name provided, as in: + macOS errors: + ``` + [ 56%] Linking C executable unit1302 + duplicate symbol '_curlx_sotouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_sitouz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + duplicate symbol '_curlx_uztosz' in: + ../../lib/libcurlu.a(unity_0_c.c.o) + ../../src/libcurltool.a(unity_0_c.c.o) + [...] + ``` + with config: + ``` + -DCMAKE_UNITY_BUILD=ON \ + -DENABLE_DEBUG=ON -DBUILD_TESTING=ON -DCMAKE_C_FLAGS=-DDEBUGBUILD \ + -DBUILD_SHARED_LIBS=ON \ + -DBUILD_STATIC_LIBS=OFF + ``` - curl -u : --negotiate https://example.com/ + Closes #11926 - This commit leverages the `struct auth` want member to figure out if the - user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of - setting a user name both in curl and the lib. +- cmake: lib `CURL_STATICLIB` fixes (Windows) - Signed-off-by: Marin Hannache - Reported-by: Enrico Scholz - Fixes https://sourceforge.net/p/curl/bugs/440/ - Fixes #1161 - Closes #9047 + - always define `CURL_STATICLIB` when building libcurl for Windows. -Viktor Szakats (13 Aug 2023) + This disables `__declspec(dllexport)` for exported libcurl symbols. + In normal mode (hide symbols) these exported symbols are specified + via `libcurl.def`. When not hiding symbols, all symbols are exported + by default. -- build: streamline non-UWP wincrypt detections + Regression from 1199308dbc902c52be67fc805c72dd2582520d30 - - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build - and disable our non-UWP-compatible use the Windows crypto API. This - allows to drop two dynamic feature checks. + Fixes #11844 - `WINDOWS_STORE` is true when invoking CMake with - `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. + - fix to omit `libcurl.def` when not hiding private symbols. - Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html + Regression from 2ebc74c36a19a1700af394c16855ce144d9878e3 - - with autotools, drop the separate feature check for `wincrypt.h`. On - one hand this header has been present for long (even Borland C 5.5 had - it from year 2000), on the other we used the check result solely to - enable another check for certain crypto functions. This fails anyway - with the header not present. We save one dynamic feature check at the - configure stage. + - fix `ENABLED_DEBUG=ON` + shared curl tool Windows builds by also + omitting `libcurl.def` in this case, and exporting all symbols + instead. This ensures that a shared curl tool can access all debug + functions which are not normally exported from libcurl DLL. - Reviewed-by: Marcel Raad - Closes #11657 + - delete `INTERFACE_COMPILE_DEFINITIONS "CURL_STATICLIB"` for "objects" + target. -Nicholas Nethercote (13 Aug 2023) + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 -- docs/HYPER.md: update hyper build instructions + - delete duplicate `BUILDING_LIBCURL` definitions. - Nightly Rust and `-Z unstable-options` are not needed. + - fix `HIDES_CURL_PRIVATE_SYMBOLS` to not overwrite earlier build settings. - The instructions here now match the hyper docs exactly: - https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 - 6f88 + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 - Closes #11662 + Closes #11914 -Daniel Stenberg (13 Aug 2023) +Daniel Stenberg (25 Sep 2023) - RELEASE-NOTES: synced -- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name - - Asssisted-by: Jay Satiro - Closes #11655 - -- spellcheck: adapt to backslashed minuses - - As the curl.1 has more backslashed minus, the cleanup sed lines xneed to - adapt. - - Adjusted some docs slighly. - - Follow-up to 439ff2052e +Dan Fandrich (25 Sep 2023) - Closes #11663 +- tests: fix log directory path in IPFS tests -- gen: escape more minus + Hard-coding the log directory name fails with parallel tests. - Detected since it was still hard to search for option names using dashes - in the middle in the man page. + Follow-up to 65b563a96 - Closes #11660 + Ref: #8805 -- cookie-jar.d: enphasize that this option is ONLY writing cookies +Daniel Stenberg (25 Sep 2023) - Reported-by: Dan Jacobson - Tweaked-by: Jay Satiro - Ref: #11642 - Closes #11661 +- curl_multi_get_handles: get easy handles from a multi handle -Nicholas Nethercote (11 Aug 2023) + Closes #11750 -- docs/HYPER.md: document a workaround for a link error +Stefan Eissing (25 Sep 2023) - Closes #11653 +- http: h1/h2 proxy unification -Jay Satiro (11 Aug 2023) + - use shared code for setting up the CONNECT request + when tunneling, used in HTTP/1.x and HTTP/2 proxying + - eliminate use of Curl_buffer_send() and other manipulations + of `data->req` or `data->state.ulbuf` -- schannel: verify hostname independent of verify cert + Closes #11808 - Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off - and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the - hostname in schannel code. +Natanael Copa (25 Sep 2023) - This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and - verify hostname". We discussed a fix several years ago in #3285 but it - went stale. +- lib: use wrapper for curl_mime_data fseek callback - Assisted-by: Daniel Stenberg + fseek uses long offset which does not match with curl_off_t. This leads + to undefined behavior when calling the callback and caused failure on + arm 32 bit. - Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html - Reported-by: Martin Galvan + Use a wrapper to solve this and use fseeko which uses off_t instead of + long. - Ref: https://github.com/curl/curl/pull/3285 + Thanks to the nice people at Libera IRC #musl for helping finding this + out. - Fixes https://github.com/curl/curl/issues/3284 - Closes https://github.com/curl/curl/pull/10056 + Fixes #11882 + Fixes #11900 + Closes #11918 -Daniel Stenberg (11 Aug 2023) +- configure: sort AC_CHECK_FUNCS -- curl_quiche: remove superfluous NULL check + No functional changes. - 'stream' is always non-NULL at this point +Daniel Stenberg (25 Sep 2023) - Pointed out by Coverity +- warnless: remove unused functions - Closes #11656 + Previously put there for use with the intel compiler -- curl/urlapi.h: tiny typo + Closes #11932 -- github/labeler: make HYPER.md set Hyper and not TLS +- GHA/linux: run singleuse to detect single-use global functions -- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 + Use --unit for configure --enable-debug builds - 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need - to specify version changes for earlier releases in the generated output. + Closes #11932 - This ups the limit from the previous 7.30.0 (Apr 12 2013) +- singleuse: add scan for use in other source codes - This hides roughly 35 "added in" mentions. + This should reduce false-positive to almost zero. Checks for presence in + unit tests if --unit is specified, which is intended for debug builds + where unit testing is enabled. - Closes #11651 + Closes #11932 -Jay Satiro (10 Aug 2023) +- multi: remove Curl_multi_dump -- bug_report: require reporters to specify curl and os versions + A debug-only function that is basically never used. Removed to ease the + use of the singleuse script to detect non-static functions not used + outside the file where it is defined. - - Change curl version and os sections from single-line input to - multi-line textarea. + Closes #11931 - - Require curl version and os sections to be filled out before report - can be submitted. +Viktor Szakats (24 Sep 2023) - Closes https://github.com/curl/curl/pull/11636 +- tests: fix compiler warnings -Daniel Stenberg (9 Aug 2023) + Seen with llvm 17 on Windows x64. -- gen.pl: replace all single quotes with aq + ``` + .../curl/tests/server/rtspd.c:136:13: warning: no previous extern declaration + for non-static variable 'logdir' [-Wmissing-variable-declarations] + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:136:7: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 136 | const char *logdir = "log"; + | ^ + .../curl/tests/server/rtspd.c:137:6: warning: no previous extern declaration + for non-static variable 'loglockfile' [-Wmissing-variable-declarations] + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/rtspd.c:137:1: note: declare 'static' if the variable i + s not intended to be used outside of this translation unit + 137 | char loglockfile[256]; + | ^ + .../curl/tests/server/fake_ntlm.c:43:13: warning: no previous extern declarat + ion for non-static variable 'logdir' [-Wmissing-variable-declarations] + 43 | const char *logdir = "log"; + | ^ + .../curl/tests/server/fake_ntlm.c:43:7: note: declare 'static' if the variabl + e is not intended to be used outside of this translation unit + 43 | const char *logdir = "log"; + | ^ + .../curl/src/tool_doswin.c:350:8: warning: possible misuse of comma operator + here [-Wcomma] + 350 | ++d, ++s; + | ^ + .../curl/src/tool_doswin.c:350:5: note: cast expression to void to silence wa + rning + 350 | ++d, ++s; + | ^~~ + | (void)( ) + ``` - - this prevents man from using a unicode sequence for them - - which then allows search to work properly + ``` + .../curl/tests/libtest/lib540.c:146:27: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 146 | int itimeout = (L > (long)INT_MAX) ? INT_MAX : (int)L; + | ~ ^ ~~~~~~~~~~~~~ + 1 warning generated. - Closes #11645 + .../curl/tests/libtest/libntlmconnect.c:195:31: warning: result of comparison + 'long' > 2147483647 is always false [-Wtautological-type-limit-compare] + 195 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. -Viktor Szakats (9 Aug 2023) + .../curl/tests/libtest/lib591.c:117:31: warning: result of comparison 'long' + > 2147483647 is always false [-Wtautological-type-limit-compare] + 117 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + .../curl/tests/libtest/lib597.c:99:31: warning: result of comparison 'long' > + 2147483647 is always false [-Wtautological-type-limit-compare] + 99 | int itimeout = (timeout > (long)INT_MAX) ? INT_MAX : (int)timeo + ut; + | ~~~~~~~ ^ ~~~~~~~~~~~~~ + 1 warning generated. + ``` -- cmake: fix to use variable for the curl namespace + Seen on macOS Intel: + ``` + .../curl/tests/server/sws.c:440:64: warning: field precision should have type + 'int', but argument has type 'size_t' (aka 'unsigned long') [-Wformat] + msnprintf(logbuf, sizeof(logbuf), "Got request: %s %.*s HTTP/%d.%d" + , + ~~^~ + 1 warning generated. + ``` - Replace (wrong) literal with a variable to specify the curl - namespace. + Closes #11925 - Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 +Jay Satiro (24 Sep 2023) - Reported-by: balikalina on Github - Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 - 20d30#r123923098 - Closes #11629 +- url: fix netrc info message -- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms + - Fix netrc info message to use the generic ".netrc" filename if the + user did not specify a netrc location. - 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing - libcurl objects for shared and static targets. + - Update --netrc doc to add that recent versions of curl on Windows + prefer .netrc over _netrc. - The above automatically enabled for Windows builds, with an option to - disable with `SHARE_LIB_OBJECT=OFF`. + Before: + * Couldn't find host google.com in the (nil) file; using defaults - This patch extend this feature to all platforms as a manual option. - You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects - are built in PIC mode, meaning the static lib will also have PIC code. + After: + * Couldn't find host google.com in the .netrc file; using defaults - [EXPERIMENTAL] + Closes https://github.com/curl/curl/pull/11904 - Closes #11627 +Dan Fandrich (23 Sep 2023) -- cmake: assume `wldap32` availability on Windows +- wolfssh: do cleanup in Curl_ssh_cleanup - This system library first shipped with Windows ME, available as an extra - install for some older releases (according to [1]). The import library - was present already in old MinGW 3.4.2 (year 2007). + Closes: #11921 - Drop the feature check and its associated `HAVE_WLDAP32` variable. +Daniel Stenberg (24 Sep 2023) - To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` - CMake option, like before. +- tool_listhelp: regenerated - [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html + Polished the --ipfs-gateway description - Reviewed-by: Jay Satiro - Closes #11624 + Fixed the --trace-config description -Daniel Stenberg (9 Aug 2023) + The script also fixed some other small mistakes -- page-header: move up a URL paragraph from GLOBBING to URL + Closes #11923 -- variable.d: output the function names table style +Viktor Szakats (23 Sep 2023) - Also correct the url function name in the header +- Makefile.mk: always set `CURL_STATICLIB` for lib (Windows) - Closes #11641 + Also fix to export all symbols in Windows debug builds, making + `-debug-dyn` builds work with `-DCURL_STATICLIB` set. -- haproxy-clientip.d: remove backticks + Ref: https://github.com/curl/curl/pull/11914 (same for CMake) - This is not markdown + Closes #11924 - Follow-up to 0a75964d0d94a4 +Daniel Stenberg (23 Sep 2023) - Closes #11639 +- quic: set ciphers/curves the same way regular TLS does -- RELEASE-NOTES: synced + for OpenSSL/BoringSSL -- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens + Fixes #11796 + Reported-by: Karthikdasari0423 on github + Assisted-by: Jay Satiro + Closes #11836 - Reported-by: FC Stegerman - Fixes #11635 - Closes #11637 +- test457: verify --max-filesize with chunked encoding -- cmdline-opts/page-header: reorder, clean up +- lib: let the max filesize option stop too big transfers too - - removed some unnecessary blurb to focus - - moved up the more important URL details - - put "globbing" into its own subtitle and moved down a little - - mention the online man page in the version section + Previously it would only stop them from getting started if the size is + known to be too big then. - Closes #11638 + Update the libcurl and curl docs accordingly. -- c-hyper: adjust the hyper to curlcode conversion + Fixes #11810 + Reported-by: Elliot Killick + Assisted-by: Jay Satiro + Closes #11820 - Closes #11621 +Viktor Szakats (23 Sep 2023) -- test2306: make it use a persistent connection +- mingw: delete support for legacy mingw.org toolchain - + enable verbose already from the start + Drop support for "old" / "legacy" / "classic" / "v1" / "mingw32" MinGW: + https://en.wikipedia.org/wiki/MinGW, https://osdn.net/projects/mingw/ + Its homepage used to be http://mingw.org/ [no HTTPS], and broken now. + It supported the x86 CPU only and used a old Windows API header and + implib set, often causing issues. It also misses most modern Windows + features, offering old versions of both binutils and gcc (no llvm/clang + support). It was last updated 2 years ago. - Closes #11621 + curl now relies on toolchains based on the mingw-w64 project: + https://www.mingw-w64.org/ https://sourceforge.net/projects/mingw-w64/ + https://www.msys2.org/ https://github.com/msys2/msys2 + https://github.com/mstorsjo/llvm-mingw + (Also available via Linux and macOS package managers.) -eppesuig (8 Aug 2023) + Closes #11625 -- list-only.d: mention SFTP as supported protocol +Mark Gaiser (23 Sep 2023) - Closes #11628 +- curl: add support for the IPFS protocols: -Daniel Stenberg (8 Aug 2023) + - ipfs:// + - ipns:// -- request.d: use .TP for protocol "labels" + This allows you tu use ipfs in curl like: + curl ipfs:// + and + curl ipns:// - To render the section nicer in man page. + For more information consult the readme at: + https://curl.se/docs/ipfs.html - Closes #11630 + Closes #8805 -- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP +Daniel Stenberg (23 Sep 2023) - ... as documented. +- bufq: remove Curl_bufq_skip_and_shift (unused) - Update test 3201 and 3202 accordingly. + Closes #11915 - Reported-by: Markus Sommer - Fixes #11619 - Closes #11626 +- scripts/singleuse.pl: add curl_global_trace -- page-footer: QLOGDIR works with ngtcp2 and quiche +Viktor Szakats (22 Sep 2023) - It previously said "both" backends which is confusing as we currently - have three... +- cmake: fix unity symbol collisions in h2 builds - Closes #11631 + Regression from 331b89a319d0067fa1e6441719307cfef9c7960f -Stefan Eissing (8 Aug 2023) + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + Closes #11912 -- http3: quiche, handshake optimization, trace cleanup +Daniel Stenberg (22 Sep 2023) - - load x509 store after clienthello - - cleanup of tracing +- RELEASE-NOTES: synced - Closes #11618 +Dan Fandrich (21 Sep 2023) -Daniel Stenberg (8 Aug 2023) +- github/labeler: improve the match patterns -- ngtcp2: remove dead code + This includes new rules for setting the appleOS and logging labels and + matches on some example files. Also, enable dot mode for wildcard + matches in the .github directory. - 'result' is always zero (CURLE_OK) at this point +Daniel Stenberg (21 Sep 2023) - Detected by Coverity +- upload-file.d: describe the file name slash/backslash handling - Closes #11622 + Closes #11911 -Viktor Szakats (8 Aug 2023) +Jakub Jelen (21 Sep 2023) -- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` +- libssh: cap SFTP packet size sent - OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the - popular forks (yet). Use the macro itself to detect its presence, - replacing the hard-wired fork-specific conditions. + Due to libssh limitations - This way the feature will enable automatically when forks implement it, - while also shorter and possibly requiring less future maintenance. + Signed-off-by: Jakub Jelen - Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 + Closes #11804 - Reviewed-by: Jay Satiro - Closes #11617 +Daniel Stenberg (21 Sep 2023) -- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 +- curl.h: mark CURLSSLBACKEND_NSS as deprecated since 8.3.0 - LibreSSL 3.4.1 (2021-10-14) added support for - `SSL_CTX_set_ciphersuites`. + Closes #11905 - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt +- mailmap: unify Michael Osipov under a single email - Reviewed-by: Jay Satiro - Closes #11616 +Ted Lyngmo (21 Sep 2023) -- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 +- docs: use CURLSSLBACKEND_NONE - LibreSSL 3.5.0 (2022-02-24) added support for - `SSL_CTX_set_keylog_callback`. + [ssl] use CURLSSLBACKEND_NONE instead of (curl_sslbackend)-1 in + documentation and examples. - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt + Signed-off-by: Ted Lyngmo - Reviewed-by: Jay Satiro - Closes #11615 + Closes #11909 -- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks +Dan Fandrich (21 Sep 2023) - - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is - also not used by curl, but it is by its optional dependency `librtmp`. - Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This - library has been available since the early days of Windows. +- github/labeler: give the sync-labels config item a default value - - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present - since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and - other existing logic already assumed this lib being present, so delete - the check and replace the detection variable with `WIN32` and always - add `ws2_32` on Windows. + This shouldn't be necessary and is likely a bug with this beta version + of the labeller. - Closes #11612 + Also, fix the negative matches for the documentation label. -Daniel Gustafsson (8 Aug 2023) + Follow-up to dd12b452a + Closes #11907 -- crypto: ensure crypto initialization works +- github/labeler: fix up more the labeler config format - Make sure that context initialization during hash setup works to avoid - going forward with the risk of a null pointer dereference. + The new version didn't like the workaround we had for a bug in the + previous labeler version, and it should no longer be needed. - Reported-by: Philippe Antoine on HackerOne - Assisted-by: Jay Satiro - Assisted-by: Daniel Stenberg + Follow-up to dd12b452a + Closes #11906 - Closes #11614 +- github/labeler: fix indenting to try to appease labeller -Viktor Szakats (7 Aug 2023) + Follow-up to dd12b452a -- openssl: switch to modern init for LibreSSL 2.7.0+ +Jay Satiro (21 Sep 2023) - LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, - `OPENSSL_init_ssl()` function and deprecated the old, manual init - method, as seen in OpenSSL 1.1.0. Switch to the modern method when - available. +- libssh2: fix error message on failed pubkey-from-file - Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt + - If libssh2_userauth_publickey_fromfile_ex returns -1 then show error + message "SSH public key authentication failed: Reason unknown (-1)". - Reviewed-by: Daniel Stenberg - Closes #11611 + When libssh2_userauth_publickey_fromfile_ex returns -1 it does so as a + generic error and therefore doesn't set an error message. AFAICT that is + not documented behavior. -Daniel Stenberg (7 Aug 2023) + Prior to this change libcurl retrieved the last set error message which + would be from a previous function failing. That resulted in misleading + auth failed error messages in verbose mode. -- gskit: remove + Bug: https://github.com/curl/curl/issues/11837#issue-1891827355 + Reported-by: consulion@users.noreply.github.com - We remove support for building curl with gskit. + Closes https://github.com/curl/curl/pull/11881 - - This is a niche TLS library, only running on some IBM systems - - no regular curl contributors use this backend - - no CI builds use or verify this backend - - gskit, or the curl adaption for it, lacks many modern TLS features - making it an inferior solution - - build breakages in this code take weeks or more to get detected - - fixing gskit code is mostly done "flying blind" +Stefan Eissing (21 Sep 2023) - This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has - been mentioned on the curl-library mailing list. +- pytest: exclude test_03_goaway in CI runs due to timing dependency - It could be brought back, this is not a ban. Given proper effort and - will, gskit support is welcome back into the curl TLS backend family. + Closes #11860 - Closes #11460 +- lib: disambiguate Curl_client_write flag semantics -- RELEASE-NOTES: synced + - use CLIENTWRITE_BODY *only* when data is actually body data + - add CLIENTWRITE_INFO for meta data that is *not* a HEADER + - debug assertions that BODY/INFO/HEADER is not used mixed + - move `data->set.include_header` check into Curl_client_write + so protocol handlers no longer have to care + - add special in FTP for `data->set.include_header` for historic, + backward compatible reasons + - move unpausing of client writes from easy.c to sendf.c, so that + code is in one place and can forward flags correctly -Dan Fandrich (7 Aug 2023) + Closes #11885 -- THANKS-filter: add a name typo +Patrick Monnerat (21 Sep 2023) -Stefan Eissing (7 Aug 2023) +- tftpd: always use curl's own tftp.h -- http3/ngtcp2: shorten handshake, trace cleanup + Using the system's provided arpa/tftp.h and optimizing, GCC 12 detects + and reports a stringop-overread warning: - - shorten handshake timing by delayed x509 store load (OpenSSL) - as we do for HTTP/2 - - cleanup of trace output, align with HTTP/2 output + tftpd.c: In function ‘write_behind.isra’: + tftpd.c:485:12: warning: ‘write’ reading between 1 and 2147483647 bytes f + rom a region of size 0 [-Wstringop-overread] + 485 | return write(test->ofile, writebuf, count); + | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + In file included from tftpd.c:71: + /usr/include/arpa/tftp.h:58:30: note: source object ‘tu_data’ of size 0 + 58 | char tu_data[0]; /* data or error stri + ng */ + | ^~~~~~~ - Closes #11609 + This occurs because writebuf points to this field and the latter + cannot be considered as being of dynamic length because it is not + the last field in the structure. Thus it is bound to its declared + size. -Daniel Stenberg (7 Aug 2023) + This commit always uses curl's own version of tftp.h where the + target field is last in its structure, effectively avoiding the + warning. -- headers: accept leading whitespaces on first response header + As HAVE_ARPA_TFTP_H is not used anymore, cmake/configure checks for + arpa/tftp.h are removed. - This is a bad header fold but since the popular browsers accept this - violation, so does curl now. Unless built with hyper. + Closes #11897 - Add test 1473 to verify and adjust test 2306. +Dan Fandrich (20 Sep 2023) - Reported-by: junsik on github - Fixes #11605 - Closes #11607 +- test1474: make precheck more robust on non-Solaris systems -- include/curl/mprintf.h: add __attribute__ for the prototypes + If uname -r returns something odd, perl could return an error code and + the test would be erroneously skipped. The qx// syntax avoid this. - - if gcc or clang is used - - if __STDC_VERSION__ >= 199901L, which means greater than C90 - - if not using mingw - - if CURL_NO_FMT_CHECKS is not defined + Followup to 08f9b2148 - Closes #11589 +- github/labeler: switch to the 5 beta version -- tests: fix bad printf format flags in test code + This version adds an important feature that will allow more PRs to be + labelled. Rather than being limited to labeling PRs with files that + match a single glob, it can now label them if multiple changed files + match any one of a number of globs. -- tests: fix header scan tools for attribute edits in mprintf.h +Daniel Stenberg (20 Sep 2023) -- cf-socket: log successful interface bind +- lib: enable hmac for digest as well - When the setsockopt SO_BINDTODEVICE operation succeeds, output that in - the verbose output. + Previously a build that disabled NTLM and aws-sigv4 would fail to build + since the hmac was disabled, but it is also needed for digest auth. - Ref: #11599 - Closes #11608 + Follow-up to e92edfbef64448ef -- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled + Fixes #11890 + Reported-by: Aleksander Mazur + Closes #11896 - Ref: #11457 - Closes #11606 +- idn: if idn2_check_version returns NULL, return error -- CURLOPT_SSL_VERIFYPEER.3: add two more see also options + ... this avoids a NULL dereference for this unusual case. - CURLINFO_CAINFO and CURLINFO_CAPATH + Reported-by: s0urc3_ on hackerone + Closes #11898 - Closes #11603 +- http: fix CURL_DISABLE_BEARER_AUTH breakage -- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice + When bearer auth was disabled, the if/else logic got wrong and caused + problems. - Closes #11007 + Follow-up to e92edfbef64448ef461 + Fixes #11892 + Reported-by: Aleksander Mazur + Closes #11895 -Graham Campbell (6 Aug 2023) +Michael Osipov (20 Sep 2023) -- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 +- wolfssl: allow capath with CURLOPT_CAINFO_BLOB - Closes #11585 + Remain consistent with OpenSSL. While CAfile is nulled as documented + with CURLOPT_CAINFO_BLOB, CApath remains intact. -Daniel Stenberg (6 Aug 2023) + Closes #11886 -- TODO: add *5* entries for aws-sigv4 +- wolfssl: use ssl_cafile/ssl_capath variables consistent with openssl.c - Closes #7559 - Closes #8107 - Closes #8810 - Closes #9717 - Closes #10129 + Closes #11886 -- TODO: LDAP Certificate-Based Authentication +Dan Fandrich (19 Sep 2023) - Closes #9641 +- test1474: disable test on NetBSD, OpenBSD and Solaris 10 -Stefan Eissing (6 Aug 2023) + These kernels only send a fraction of the requested amount of the first + large block, invalidating the assumptions of the test and causing it to + fail. -- http2: cleanup trace messages + Assisted-by: Christian Weisgerber + Ref: https://curl.se/mail/lib-2023-09/0021.html + Closes #11888 - - more compact format with bracketed stream id - - all frames traced in and out +Ryan Schmidt (20 Sep 2023) - Closes #11592 +- cmake, configure: also link with CoreServices -Daniel Stenberg (6 Aug 2023) + When linking with CoreFoundation, also link with CoreServices which is + apparently required to avoid an NSInvalidArgumentException in software + linking with libcurl on macOS Sonoma 14 and later. -- tests/tftpd+mqttd: make variables static to silence picky warnings + Fixes #11893 + Closes #11894 - Closes #11594 +Marc Hoersken (19 Sep 2023) -- docs/cmdline: remove repeated working for negotiate + ntlm +- CI/azure: remove pip, wheel, cryptography, pyopenssl and impacket - The extra wording is added automatically by the gen.pl tool + These dependencies are now already included in the Docker image. - Closes #11597 + Ref: https://github.com/mback2k/curl-docker-winbuildenv/commit/2607a31bcab544 + b41d15606e97f38cf312c1ce56 -- docs/cmdline: add small "warning" to verbose options + Closes #11889 - "Note that verbose output of curl activities and network traffic might - contain sensitive data, including user names, credentials or secret data - content. Be aware and be careful when sharing trace logs with others." +Daniel Stenberg (19 Sep 2023) - Closes #11596 +- wolfssl: if CURLOPT_CAINFO_BLOB is set, ignore the CA files + + Ref: #11883 + Reported-by: Michael Osipov + Closes #11884 - RELEASE-NOTES: synced -- pingpong: don't use *bump_headersize +- test3103: CURLOPT_COOKIELIST test - We use that for HTTP(S) only. +- cookie: set ->running in cookie_init even if data is NULL - Follow-up to 3ee79c1674fd6 + This is a regression introduced in b1b326ec500 (shipped in curl 8.1.0) - Closes #11590 + Test 3103 verifies. -- urldata: remove spurious parenthesis to unbreak no-proxy build + Fixes #11875 + Reported-by: wangp on github + Closes #11876 - Follow-up to e12b39e13382 +- test498: total header size for all redirects is larger than accepted - Closes #11591 +- http: use per-request counter to check too large headers -- easy: don't call Curl_trc_opt() in disabled-verbose builds + Not the counter that accumulates all headers over all redirects. - Follow-up to e12b39e133822c6a0 + Follow-up to 3ee79c1674fd6 - Closes #11588 + Do a second check for 20 times the limit for the accumulated size for + all headers. -- http: use %u for printfing int + Fixes #11871 + Reported-by: Joshix-1 on github + Closes #11872 - Follow-up to 3ee79c1674fd6f99e8efca5 +Jay Satiro (18 Sep 2023) - Closes #11587 +- THANKS: add Eric Murphy -Goro FUJI (3 Aug 2023) + He reported #11850 (quiche build error) but I forgot to add a + 'reported-by' entry in the fix 267e14f1. -- vquic: show stringified messages for errno +Daniel Stenberg (18 Sep 2023) - Closes #11584 +- h2-proxy: remove left-over mistake in drain_tunnel() -Stefan Eissing (3 Aug 2023) + Left-over from 331b89a319 -- trace: make tracing available in non-debug builds + Reported-by: 南宫雪珊 - Add --trace-config to curl + Closes https://github.com/curl/curl/pull/11877 - Add curl_global_trace() to libcurl +vvb2060 (18 Sep 2023) - Closes #11421 +- lib: failf/infof compiler warnings -Daniel Stenberg (3 Aug 2023) + Closes #11874 -- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" +Daniel Stenberg (17 Sep 2023) - See also https://github.com/curl/curl/pull/7507 +- rand: fix 'alnum': array is too small to include a terminating null character -- TODO: add "WebSocket read callback" + It was that small on purpose, but this change now adds the null byte to + avoid the error. + + Follow-up to 3aa3cc9b052353b1 + + Reported-by: Dan Fandrich + Ref: #11838 + Closes #11870 - remove "Upgrade to websockets" as we already have this +Mathias Fuchs (16 Sep 2023) - Closes #11402 +- cmake: fix the help text to the static build option in CMakeLists.txt -- test497: verify rejecting too large incoming headers + Closes #11843 -- http: return error when receiving too large header set +John Haugabook (16 Sep 2023) - To avoid abuse. The limit is set to 300 KB for the accumulated size of - all received HTTP headers for a single response. Incomplete research - suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to - 1MB. +- MANUAL.md: change domain to example.com - Closes #11582 + Closes #11866 -Stefan Eissing (3 Aug 2023) +Daniel Stenberg (16 Sep 2023) -- http2: upgrade tests and add fix for non-existing stream +- doh: inherit DEBUGFUNCTION/DATA - - check in h2 filter recv that stream actually exists - and return error if not - - add test for parallel, extreme h2 upgrades that fail if - connections get reused before fully switched - - add h2 upgrade upload test just for completeness + When creating new transfers for doing DoH, they now inherit the debug + settings from the initiating transfer, so that the application can + redirect and handle the verbose output correctly even for the DoH + transfers. - Closes #11563 + Reported-by: calvin2021y on github + Fixes #11864 + Closes #11869 -Viktor Szakats (3 Aug 2023) +Dan Fandrich (16 Sep 2023) -- tests: ensure `libcurl.def` contains all exports +- http_aws_sigv4: fix sorting with empty parts - Add `test1279` to verify that `libcurl.def` lists all exported API - functions found in libcurl headers. + When comparing with an empty part, the non-empty one is always + considered greater-than. Previously, the two would be considered equal + which would randomly place empty parts amongst non-empty ones. This + showed as a test 439 failure on Solaris as it uses a different + implementation of qsort() that compares parts differently. - Also: + Fixes #11855 + Closes #11868 - - extend test suite XML `stdout` tag with the `loadfile` attribute. +- CI: ignore the "flaky" and "timing-dependent" test results - - fix `tests/extern-scan.pl` and `test1135` to include websocket API. + CI builds will now run these tests, but will ignore the results if they + fail. The relevant tests are ones that are sensitive to timing or + have edge conditions that make them more likely to fail on CI servers, + which are often heavily overloaded and slow. - - use all headers (sorted) in `test1135` instead of a manual list. + This change only adds two additional tests to be ignored, since the + others already had the flaky keyword. - - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. + Closes #11865 - - add `libcurl.def` to the auto-labeler GHA task. +- runtests: eliminate a warning on old perl versions - Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 + The warning "Use of implicit split to @_ is deprecated" showed between + perl versions about 5.8 through 5.11. - Closes #11570 +- tests: log the test result code after each libtest -Daniel Stenberg (2 Aug 2023) + This makes it easier to determine the test status. Also, capitalize + FAILURE and ABORT messages in log lines to make them easier to spot. -- url: change default value for CURLOPT_MAXREDIRS to 30 +Harry Sintonen (16 Sep 2023) - It was previously unlimited by default, but that's not a sensible - default. While changing this has a remote risk of breaking an existing - use case, I figure it is more likely to actually save users from loops. +- misc: better random strings - Closes #11581 + Generate alphanumerical random strings. -- lib: fix a few *printf() flag mistakes + Prior this change curl used to create random hex strings. This was + mostly okay, but having alphanumerical random strings is better: The + strings have more entropy in the same space. - Reported-by: Gisle Vanem - Ref: #11574 - Closes #11579 + The MIME multipart boundary used to be mere 64-bits of randomness due + to being 16 hex chars. With these changes the boundary is 22 + alphanumerical chars, or little over 130 bits of randomness. -Samuel Chiang (2 Aug 2023) + Closes #11838 -- openssl: make aws-lc version support OCSP +Daniel Stenberg (15 Sep 2023) - And bump version in CI +- cookie: reduce variable scope, add const - Closes #11568 +- cookie: do not store the expire or max-age strings -Daniel Stenberg (2 Aug 2023) + Convert it to an expire time at once and save memory. -- tool: make the length argument an int for printf()-.* flags + Closes #11862 - Closes #11578 +- cookie: remove unnecessary struct fields -- tool_operate: fix memory leak when SSL_CERT_DIR is used + Plus: reduce the hash table size from 256 to 63. It seems unlikely to + make much of a speed difference for most use cases but saves 1.5KB of + data per instance. - Detected by Coverity + Closes #11862 - Follow-up to 29bce9857a12b6cfa726a5 +- RELEASE-NOTES: synced - Closes #11577 + Bumped to 8.4.0, the next presumed version -- tool/var: free memory on OOM +Dan Fandrich (14 Sep 2023) - Coverity detected this memory leak in OOM situation +- test2600: remove special case handling for USE_ALARM_TIMEOUT - Follow-up to 2e160c9c652504e + This was originally added to handle platforms that supported only 1 + second granularity in connect timeouts, but after some recent changes + the test currently permafails on several Windows platforms. - Closes #11575 + The need for this special-case was removed in commit 8627416, which + increased the connect timeout in all cases to well above 1 second. -Viktor Szakats (2 Aug 2023) + Fixes #11767 + Closes #11849 -- gha: bump libressl and mbedtls versions +Daniel Stenberg (14 Sep 2023) - Closes #11573 +- SECURITY-PROCESS.md. call it vulnerability disclosure policy -Jay Satiro (2 Aug 2023) + SECURITY-PROCESS.md -> VULN-DISCLOSURE-POLICY.md -- schannel: fix user-set legacy algorithms in Windows 10 & 11 + This a name commonly used for a document like this. This name helps + users find it. - - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then - use the SCHANNEL_CRED legacy structure to pass the list to Schannel. + Closes #11852 - - If the user set both a legacy algorithm list and a TLS 1.3 cipher list - then abort. +Junho Choi (14 Sep 2023) - Although MS doesn't document it, Schannel will not negotiate TLS 1.3 - when SCHANNEL_CRED is used. That means setting a legacy algorithm list - limits the user to earlier versions of TLS. +- quiche: fix build error with --with-ca-fallback - Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would - ignore legacy algorithms in Windows 10 1809 and later. + - Fix build error when curl is built with --with-quiche + and --with-ca-fallback. - Reported-by: zhihaoy@users.noreply.github.com + - Add --with-ca-fallback to the quiche CI job. - Fixes https://github.com/curl/curl/pull/10741 - Closes https://github.com/curl/curl/pull/10746 + Fixes https://github.com/curl/curl/issues/11850 + Closes https://github.com/curl/curl/pull/11847 -Daniel Stenberg (2 Aug 2023) +Jay Satiro (14 Sep 2023) -- variable.d: setting a variable again overwrites it +- escape: replace Curl_isunreserved with ISUNRESERVED - Reported-by: Niall McGee - Bug: https://twitter.com/niallmcgee/status/1686523075423322113 - Closes #11571 + - Use the ALLCAPS version of the macro so that it is clear a macro is + being called that evaluates the variable multiple times. -Jay Satiro (2 Aug 2023) + - Also capitalize macro isurlpuntcs => ISURLPUNTCS since it evaluates + a variable multiple times. -- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting + This is a follow-up to 291d225a which changed Curl_isunreserved into an + alias macro for ISUNRESERVED. The problem is the former is not easily + identified as a macro by the caller, which could lead to a bug. - - Re-wrap CURLSSLOPT_ALLOW_BEAST description. + For example, ISUNRESERVED(*foo++) is easily identifiable as wrong but + Curl_isunreserved(*foo++) is not even though they both are the same. -Daniel Stenberg (2 Aug 2023) + Closes https://github.com/curl/curl/pull/11846 -- RELEASE-NOTES: synced +Dan Fandrich (13 Sep 2023) -- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set +- tests: increase the default server logs lock timeout - Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not - used, thus unnecessarily asking for addresses that will not be used. + This timeout is used to wait for the server to finish writing its logs + before checking them against the expected values. An overloaded machine + could take more than the two seconds previously allocated, so increase + the timeout to 5 seconds. - Reported-by: Joseph Tharayil - Fixes #11564 - Closes #11565 + Ref: #11328 + Closes #11834 -- docs: link to the website versions instead of markdowns +- tests: increase TEST_HANG_TIMEOUT in two tests - ... to make the links work when the markdown is converted to webpages on - https://curl.se + These tests had a 5 second timeout compared to 60 seconds for all other + tests. Make these consistent with the others for more reliability on + heavily-loaded machines. - Reported-by: Maurício Meneghini Fauth - Fixes https://github.com/curl/curl-www/issues/272 - Closes #11569 + Ref: #11328 -Viktor Szakats (1 Aug 2023) +- test1056: disable on Windows -- cmake: cache more config and delete unused ones + This test relies on the IPv6 scope field being ignored when connecting to + ipv6-localhost (i.e. [::1%259999] is treated as [::1]). Maybe this is a bit + dodgy, but it works on all our test platforms except Windows. This + test was disabled manually on all Windows CI builds already, so instead + add an incompatible feature and precheck so it's skipped on Windows + everywhere automatically. - - cache more Windows config results for faster initialization. +- test587: add a slight delay after test - - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. + This test is designed to connect to the server, then immediately send a + few bytes and disconnect. In some situations, such as on a loaded + server, this doesn't give the server enough time to write its lock file + before its existence is checked. The test harness then fails to find the + server's input log file (because it hasn't been written yet) and fails + the test. By adding a short delay after the test, the HTTP server has + enough time to write its lock file which gives itself more time to write + its remaining files. - - delete dead references to `sys/utsname.h`. + Ref: #11328 - Closes #11551 +- tests: stop overriding the lock timeout -- egd: delete feature detection and related source code + These tests reduce the server lock wait timeout which can increase + flakiness on loaded machines. Since this is merely an optimization, + eliminate them in favour of reliability. - EGD is Entropy Gathering Daemon, a socket-based entropy source supported - by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it - a while ago. + Ref: #11328 - Its detection in CMake was broken all along because OpenSSL libs were - not linked at the point of feature check. +- tests: add some --expect100-timeout to reduce timing dependencies - Delete detection from both cmake and autotools, along with the related - source snippet, and the `--with-egd-socket=` `./configure` option. + These tests can fail when the test machine is so slow that the test HTTP + server didn't get a chance to complete before the client's one second + 100-continue timeout triggered. Increase that 1 second to 999 seconds so + this situation doesn't happen. - Closes #11556 + Ref: #11328 -Stefan Eissing (1 Aug 2023) +- test661: return from test early in case of curl error -- tests: fix h3 server check and parallel instances +- tests: add the timing-dependent keyword on several tests - - fix check for availability of nghttpx server - - add `tcp` frontend config for same port as quic, as - without this, port 3000 is bound which clashes for parallel - testing + These are ones likely to fail on heavily-loaded machines that alter the + normal test timing. Most of these tests already had the flaky keyword + since this condition makes them more likely to fail on CI. - Closes #11553 +- test1592: greatly increase the maximum test timeout -Daniel Stenberg (1 Aug 2023) + It was too short to be reliable on heavily loaded CI machines, and + as a fail-safe only, it didn't need to be short. -- docs/cmdline-opts: spellfixes, typos and polish + Ref: #11328 - To make them accepted by the spell checker +- test: minor test cleanups - Closes #11562 + Remove an obsolete block of code in tests 2032 & 576. + Add a comment in test 1474. -- CI/spellcheck: build curl.1 and spellcheck it +- tests: quadruple the %FTPTIME2 and %FTPTIME3 timeouts - Added acceptable words + This gives more of a margin for error when running on overloaded CI + servers. - Closes #11562 + Ref: #11328 -Alexander Jaeger (1 Aug 2023) +- tests: improve SLOWDOWN test reliability by reducing sent data -- misc: fix various typos + These tests are run in SLOWDOWN mode which adds a 10 msec delay after + each character output, which means it takes at least 1.6 seconds (and + 320 kernel calls) just to get through the long welcome banner. On an + overloaded system, this can end up taking much more than 1.6 seconds, + and even more than the 7 or 16 second curl timeout that the tests rely + on, causing them to fail. Reducing the size of the welcome banner drops + the total number of characters sent before the transfer starts by more + than half, which reduces the opportunity for test-breaking slowdowns by + the same amount. - Closes #11561 + Ref: #11328 -Daniel Stenberg (1 Aug 2023) +- test650: fix an end tag typo -- http2: avoid too early connection re-use/multiplexing +Jay Satiro (13 Sep 2023) - HTTP/1 connections that are upgraded to HTTP/2 should not be picked up - for reuse and multiplexing by other handles until the 101 switching - process is completed. +- tool_cb_wrt: fix debug assertion - Lots-of-debgging-by: Stefan Eissing - Reported-by: Richard W.M. Jones - Bug: https://curl.se/mail/lib-2023-07/0045.html - Closes #11557 + - Fix off-by-one out-of-bounds array index in Windows debug assertion. -- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" + Bug: https://github.com/curl/curl/commit/af3f4e41#r127212213 + Reported-by: Gisle Vanem - This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. +Daniel Stenberg (13 Sep 2023) - It's a user error for supplying incomplete information to the build system. +- ctype: add ISUNRESERVED() - Reported-by: Ryan Schmidt - Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 + ... and make Curl_isunreserved() use that macro instead of providing a + separate funtion for the purpose. -Viktor Szakats (1 Aug 2023) + Closes #11840 -- cmake: add support for single libcurl compilation pass +Version 8.3.0 (13 Sep 2023) - Before this patch CMake builds used two separate compilation passes to - build the shared and static libcurl respectively. This patch allows to - reduce that to a single pass if the target platform and build settings - allow it. +Daniel Stenberg (13 Sep 2023) - This reduces CMake build times when building both static and shared - libcurl at the same time, making these dual builds an almost zero-cost - option. +- RELEASE-NOTES: syn ced - Enable this feature for Windows builds, where the difference between the - two passes was the use of `__declspec(dllexport)` attribute for exported - API functions for the shared builds. This patch replaces this method - with the use of `libcurl.def` at DLL link time. + curl 8.3.0 release - Also update `Makefile.mk` to use `libcurl.def` to export libcurl API - symbols on Windows. This simplifies (or fixes) this build method (e.g. - in curl-for-win, which generated a `libcurl.def` from `.h` files using - an elaborate set of transformations). +- THANKS: contributors from 8.3.0 - `libcurl.def` has the maintenance cost of keeping the list of public - libcurl API symbols up-to-date. This list seldom changes, so the cost - is low. +Thorsten Klein (12 Sep 2023) - Closes #11546 +- cmake: set SIZEOF_LONG_LONG in curl_config.h -- cmake: detect `SSL_set0_wbio` in OpenSSL + in order to support 32bit builds regarding wolfssl CTC_SETTINGS - Present in OpenSSL 1.1.0 and BoringSSL. - Missing from LibreSSL 3.8.0. + Closes #11839 - Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 +Jay Satiro (12 Sep 2023) - While here, also fix `RAND_egd()` detection which was broken, likely all - along. This feature is probably broken with CMake builds and also - requires a sufficiently obsolete OpenSSL version, so this part of the - update was not tested. +- curl_ngtcp2: fix error message - Closes #11555 +- http_aws_sigv4: handle no-value user header entries -- cmake: fixup H2 duplicate symbols for unity builds + - Handle user headers in format 'name:' and 'name;' with no value. - Closes #11550 + The former is used when the user wants to remove an internal libcurl + header and the latter is used when the user actually wants to send a + no-value header in the format 'name:' (note the semi-colon is converted + by libcurl to a colon). -Pablo Busse (1 Aug 2023) + Prior to this change the AWS header import code did not special case + either of those and the generated AWS SignedHeaders would be incorrect. -- openssl: Support async cert verify callback + Reported-by: apparentorder@users.noreply.github.com - - Update the OpenSSL connect state machine to handle - SSL_ERROR_WANT_RETRY_VERIFY. + Ref: https://curl.se/docs/manpage.html#-H - This allows libcurl users that are using custom certificate validation - to suspend processing while waiting for external I/O during certificate - validation. + Fixes https://github.com/curl/curl/issues/11664 + Closes https://github.com/curl/curl/pull/11668 - Closes https://github.com/curl/curl/pull/11499 +Dan Fandrich (11 Sep 2023) -Jay Satiro (1 Aug 2023) +- CI: run pytest with the -v option -- tool_cb_wrt: fix invalid unicode for windows console + This lists of the test cases being run so it can be tracked over time. - - Suppress an incomplete UTF-8 sequence at the end of the buffer. + Closes #11824 - - Attempt to reconstruct incomplete UTF-8 sequence from prior call(s) - in current call. +Daniel Stenberg (11 Sep 2023) - Prior to this change, in Windows console UTF-8 sequences split between - two or more calls to the write callback would cause invalid "replacement - characters" U+FFFD to be printed instead of the actual Unicode - character. This is because in Windows only UTF-16 encoded characters are - printed to the console, therefore we convert the UTF-8 contents to - UTF-16, which cannot be done with partial UTF-8 sequences. +- HTTP3: the msquic backend is not functional - Reported-by: Maksim Arhipov + I ask that we do not submit bugs for this backend just yet as we know it + does not fully work. - Fixes https://github.com/curl/curl/issues/9841 - Closes https://github.com/curl/curl/pull/10890 + Closes #11831 + Closes #11819 -Daniel Stenberg (1 Aug 2023) +- aws_sigv4: the query canon code miscounted URL encoded input -- sectransp: prevent CFRelease() of NULL + Added some extra ampersands to test 439 to verify "blank" query parts - When SecCertificateCopyCommonName() returns NULL, the common_name - pointer remains set to NULL which apparently when calling CFRelease() on - (sometimes?) crashes. + Follow-up to fc76a24c53b08cdf - Reported-by: Guillaume Algis - Fixes #9194 - Closes #11554 + Closes #11829 -Jay Satiro (1 Aug 2023) +vvb2060 (11 Sep 2023) -- vtls: clarify "ALPN: offers" message +- quic: don't set SNI if hostname is an IP address - Before: - * ALPN: offers h2,http/1.1 + We already do this for TLS connections. - After: - * ALPN: curl offers h2,http/1.1 + RFC 6066 says: Literal IPv4 and IPv6 addresses are not permitted in + "HostName". - Bug: https://curl.se/mail/lib-2023-07/0041.html - Reported-by: Richard W.M. Jones - Closes #11544 + Ref: https://www.rfc-editor.org/rfc/rfc6066#section-3 -Daniel Stenberg (1 Aug 2023) + Fixes https://github.com/curl/curl/issues/11827 + Closes https://github.com/curl/curl/pull/11828 -- urlapi: make sure zoneid is also duplicated in curl_url_dup +Daniel Stenberg (10 Sep 2023) - Add several curl_url_dup() tests to the general lib1560 test. +- RELEASE-NOTES: synced - Reported-by: Rutger Broekhoff - Bug: https://curl.se/mail/lib-2023-07/0047.html - Closes #11549 +Benoit Pierre (10 Sep 2023) -Sergey (1 Aug 2023) +- configure: fix `HAVE_TIME_T_UNSIGNED` check -- urlapi: fix heap buffer overflow + The syntax was incorrect (need a proper main body), and the test + condition was wrong (resulting in a signed `time_t` detected as + unsigned). - `u->path = Curl_memdup(path, pathlen + 1);` accesses bytes after the null-ter - minator. + Closes #11825 - ``` - ==2676==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x04d48c75 a - t pc 0x0112708a bp 0x006fb7e0 sp 0x006fb3c4 - READ of size 78 at 0x04d48c75 thread T0 - #0 0x1127089 in __asan_wrap_memcpy D:\a\_work\1\s\src\vctools\asan\llvm\c - ompiler-rt\lib\sanitizer_common\sanitizer_common_interceptors.inc:840 - #1 0x1891a0e in Curl_memdup C:\actions-runner\_work\client\client\third_p - arty\curl\lib\strdup.c:97 - #2 0x18db4b0 in parseurl C:\actions-runner\_work\client\client\third_part - y\curl\lib\urlapi.c:1297 - #3 0x18db819 in parseurl_and_replace C:\actions-runner\_work\client\clien - t\third_party\curl\lib\urlapi.c:1342 - #4 0x18d6e39 in curl_url_set C:\actions-runner\_work\client\client\third_ - party\curl\lib\urlapi.c:1790 - #5 0x1877d3e in parseurlandfillconn C:\actions-runner\_work\client\client - \third_party\curl\lib\url.c:1768 - #6 0x1871acf in create_conn C:\actions-runner\_work\client\client\third_p - arty\curl\lib\url.c:3403 - #7 0x186d8dc in Curl_connect C:\actions-runner\_work\client\client\third_ - party\curl\lib\url.c:3888 - #8 0x1856b78 in multi_runsingle C:\actions-runner\_work\client\client\thi - rd_party\curl\lib\multi.c:1982 - #9 0x18531e3 in curl_multi_perform C:\actions-runner\_work\client\client\ - third_party\curl\lib\multi.c:2756 - ``` +Daniel Stenberg (9 Sep 2023) - Closes #11560 +- THANKS-filter: pszlazak on github -Daniel Stenberg (31 Jul 2023) +pszlazak (9 Sep 2023) -- curl: make %output{} in -w specify a file to write to +- include.d: explain headers not printed with --fail before 7.75.0 - It can be used multiple times. Use %output{>>name} to append. + Prior to 7.75.0 response headers were not printed if -f/--fail was used + and an error was reported by server. This was fixed in ab525c0 + (precedes 7.75.0). - Add docs. Test 990 and 991 verify. + Closes #11822 - Idea: #11400 - Suggested-by: ed0d2b2ce19451f2 - Closes #11416 +Daniel Stenberg (8 Sep 2023) -- RELEASE-NOTES: synced +- http_aws_sigv4: skip the op if the query pair is zero bytes -- tool: add "variable" support + Follow-up to fc76a24c53b08cdf - Add support for command line variables. Set variables with --variable - name=content or --variable name@file (where "file" can be stdin if set - to a single dash (-)). + Spotted by OSS-Fuzz - Variable content is expanded in option parameters using "{{name}}" - (without the quotes) if the option name is prefixed with - "--expand-". This gets the contents of the variable "name" inserted, or - a blank if the name does not exist as a variable. Insert "{{" verbatim - in the string by prefixing it with a backslash, like "\\{{". + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62175 + Closes #11823 - Import an environment variable with --variable %name. It makes curl exit - with an error if the environment variable is not set. It can also rather - get a default value if the variable does not exist, using =content or - @file like shown above. +- cmdline-docs: use present tense, not future - Example: get the USER environment variable into the URL: + + some smaller cleanups - --variable %USER - --expand-url = "https://example.com/api/{{USER}}/method" + Closes #11821 - When expanding variables, curl supports a set of functions that can make - the variable contents more convenient to use. It can trim leading and - trailing white space with "trim", output the contents as a JSON quoted - string with "json", URL encode it with "url" and base 64 encode it with - "b64". To apply functions to a variable expansion, add them colon - separated to the right side of the variable. They are then performed in - a left to right order. +- cmdline-docs: make sure to phrase it as "added in ...." - Example: get the contents of a file called $HOME/.secret into a variable - called "fix". Make sure that the content is trimmed and percent-encoded - sent as POST data: + References to things that were added or changed in a specific version + should be specified as "(added in [version]) for two reasons: - --variable %HOME=/home/default - --expand-variable fix@{{HOME}}/.secret - --expand-data "{{fix:trim:url}}" - https://example.com/ + 1 - consistency - Documented. Many new test cases. + 2 - to allow gen.pl to strip them out if deemed referring to too old + versions - Co-brainstormed-by: Emanuele Torre - Assisted-by: Jat Satiro - Closes #11346 + Closes #11821 -- KNOWN_BUGS: cygwin: make install installs curl-config.1 twice +Jay Satiro (8 Sep 2023) - Closes #8839 +- docs: mark --ssl-revoke-best-effort as Schannel specific -- KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14 + Closes https://github.com/curl/curl/pull/11760 - Closes #11215 +Nathan Moinvaziri (8 Sep 2023) -- KNOWN_BUGS: cmake outputs: no version information available +- schannel: fix ordering of cert chain info - Closes #11158 + - Use CERT_CONTEXT's pbCertEncoded to determine chain order. -- KNOWN_BUGS: APOP authentication fails on POP3 + CERT_CONTEXT from SECPKG_ATTR_REMOTE_CERT_CONTEXT contains + end-entity/server certificate in pbCertEncoded. We can use this pointer + to determine the order of certificates when enumerating hCertStore using + CertEnumCertificatesInStore. - Closes #10073 + This change is to help ensure that the ordering of the certificate chain + requested by the user via CURLINFO_CERTINFO has the same ordering on all + versions of Windows. -- KNOWN_BUGS: hyper is slow + Prior to this change Schannel certificate order was reversed in 8986df80 + but that was later reverted in f540a39b when it was discovered that + Windows 11 22H2 does the reversal on its own. - Closes #11203 + Ref: https://github.com/curl/curl/issues/9706 -Patrick Monnerat (31 Jul 2023) + Closes https://github.com/curl/curl/pull/11632 -- configure, cmake, lib: more form api deprecation +Chris Talbot (8 Sep 2023) - Introduce a --enable-form-api configure option to control its inclusion - in builds. The condition name defined for it is CURL_DISABLE_FORM_API. +- digest: Use hostname to generate spn instead of realm - Form api code is dependent of MIME: configure and CMake handle this - dependency automatically: CMake by making it a dependent option - explicitly, configure by inheriting the MIME value by default and - rejecting explicit incompatible values. + In https://www.rfc-editor.org/rfc/rfc2831#section-2.1.2 - "form-api" is now a new hidden test feature. + digest-uri-value should be serv-type "/" host , where host is: - Update libcurl modules to respect this option and adjust tests - accordingly. + The DNS host name or IP address for the service requested. The + DNS host name must be the fully-qualified canonical name of the + host. The DNS host name is the preferred form; see notes on server + processing of the digest-uri. - Closes #9621 + Realm may not be the host, so we must specify the host explicitly. -Daniel Stenberg (31 Jul 2023) + Note this change only affects the non-SSPI digest code. The digest code + used by SSPI builds already uses the hostname to generate the spn. -- mailmap: add Derzsi Dániel + Ref: https://github.com/curl/curl/issues/11369 -Derzsi Dániel (31 Jul 2023) + Closes https://github.com/curl/curl/pull/11395 -- wolfssl: support loading system CA certificates +Daniel Stenberg (7 Sep 2023) - Closes #11452 +- docs: remove use of the word 'very' -Viktor Szakats (30 Jul 2023) + It is mostly superfluous. proselint would complain. -- nss: delete more NSS references + Closes #11818 - Fix the distcheck CI failure and delete more NSS references. +- curl_multi_remove_handle.3: clarify what happens with connection - Follow-up to 7c8bae0d9c9b2dfeeb008b9a316117d7b9675175 + Closes #11817 - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Closes #11548 +- RELEASE-NOTES: synced -Daniel Stenberg (29 Jul 2023) +- test439: verify query canonization for aws-sigv4 -- nss: remove support for this TLS library +- tool_operate: make aws-sigv4 not require TLS to be used - Closes #11459 + Maybe not used too often, but we want it for testing and it should work. -Ryan Schmidt (29 Jul 2023) +- http_aws_sigv4: canonicalize the query -- macOS: fix target detection more + Percent encoding needs to be done using uppercase, and most + non-alphanumerical must be percent-encoded. - Now SCDynamicStoreCopyProxies is called (and the required frameworks are - linked in) on all versions of macOS and only on macOS. Fixes crash due - to undefined symbol when built with the macOS 10.11 SDK or earlier. + Fixes #11794 + Reported-by: John Walker + Closes #11806 - CURL_OSX_CALL_COPYPROXIES is renamed to CURL_MACOS_CALL_COPYPROXIES and - is now only defined when SCDynamicStoreCopyProxies will actually be - called. Previously, it was defined when ENABLE_IPV6 was not defined but - SCDynamicStoreCopyProxies is not called in that case. +Wyatt O'Day (7 Sep 2023) - TARGET_OS_OSX is only defined in the macOS 10.12 SDK and later and only - when dynamic targets are enabled. TARGET_OS_MAC is always defined but - means any Mac OS or derivative including macOS, iOS, tvOS, and watchOS. - TARGET_OS_IPHONE means any Darwin OS other than macOS. +- lib: add ability to disable auths individually - Follow-up to c73b2f82 + Both with configure and cmake - Fixes #11502 - Closes #11516 + Closes #11490 -Daniel Stenberg (29 Jul 2023) +Stefan Eissing (7 Sep 2023) -- tool_operate: allow SSL_CERT_FILE and SSL_CERT_DIR +- ngtcp2: fix handling of large requests - ... used at once. + - requests >64K are send in parts to the filter + - fix parsing of the request to assemble it correctly + from several sends + - open a QUIC stream only when the complete request has + been collected - Reported-by: Gabriel Corona - Fixes #11325 - Closes #11531 + Closes #11815 -Thomas M. DuBuisson (29 Jul 2023) +- openssl: when CURLOPT_SSL_CTX_FUNCTION is registered, init x509 store before -- CI: remove Lift's configuration + - we delay loading the x509 store to shorten the handshake time. + However an application callback installed via CURLOPT_SSL_CTX_FUNCTION + may need to have the store loaded and try to manipulate it. + - load the x509 store before invoking the app callback - The Lift tool is being retired. Their site reads: + Fixes #11800 + Reported-by: guoxinvmware on github + Cloes #11805 - "Sonatype Lift will be retiring on Sep 12, 2023, with its analysis - stopping on Aug 12, 2023." +Daniel Stenberg (7 Sep 2023) - Closes #11541 +- krb5: fix "implicit conversion loses integer precision" warnings -Nathan Moinvaziri (29 Jul 2023) + conversions to/from enum and unsigned chars -- Revert "schannel: reverse the order of certinfo insertions" + Closes #11814 - This reverts commit 8986df802db9b5338d9d50a54232ebae4dbcf6dd. +Stefan Eissing (7 Sep 2023) - Windows does not guarantee a particular certificate ordering, even - though TLS may have its own ordering/relationship guarantees. Recent - versions of Windows 11 reversed the ordering of ceritifcates returned by - CertEnumCertificatesInStore, therefore this commit no longer works as - initially intended. libcurl makes no guarantees about certificate - ordering if the operating system can't. +- pytest: improvements - Ref: https://github.com/curl/curl/issues/9706 + - set CURL_CI for pytest runs in CI environments + - exclude timing sensitive tests from CI runs + - for failed results, list only the log and stat of + the failed transfer - Closes https://github.com/curl/curl/pull/11536 + - fix type in http.c comment -wangzhikun (29 Jul 2023) + Closes #11812 -- winbuild: improve check for static zlib +- CI: move on to ngtcp2 v0.19.1 - - Check for zlib static library name zlibstatic.lib. + Closes #11809 - zlib's static library has a different name depending on how it was - built. zlibstatic.lib is output by cmake. zlibstat.lib is output by - their pre-generated Visual Studio project files (in the contrib - directory) and defines ZLIB_WINAPI (ie it's meant to use stdcall - instead of cdecl if you end up exporting the zlib functions). +Dan Fandrich (5 Sep 2023) - Prior to this change the makefile only checked for the latter. +- CI: run Circle macOS builds on x86 for now - Closes https://github.com/curl/curl/pull/11521 + The ARM machines aren't ready for us and requesting them now causes + warnings e-mails to be sent to some PR pushers. -Daniel Stenberg (29 Jul 2023) + Ref: #11771 -- configure: use the pkg-config --libs-only-l flag for libssh2 +Viktor Szakats (5 Sep 2023) - ... instead of --libs, as that one also returns -L flags. +- http3: adjust cast for ngtcp2 v0.19.0 - Reported-by: Wilhelm von Thiele - Fixes #11538 - Closes #11539 + ngtcp2 v0.19.0 made size of `ecn` member of `ngtcp2_pkt_info` + an `uint8_t` (was: `uint32_t`). Adjust our local cast accordingly. -Viktor Szakats (29 Jul 2023) + Fixes: + ``` + ./curl/lib/vquic/curl_ngtcp2.c:1912:12: warning: implicit conversion loses in + teger precision: 'uint32_t' (aka 'unsigned int') to 'uint8_t' (aka 'unsigned + char') [-Wimplicit-int-conversion] + pi.ecn = (uint32_t)ecn; + ~ ^~~~~~~~~~~~~ + ``` -- cmake: support building static and shared libcurl in one go + Also bump ngtcp2, nghttp3 and nghttp2 to their latest versions in our + docs and CI. - This patch adds the ability to build a static and shared libcurl library - in a single build session. It also adds an option to select which one to - use when building the curl executable. + Ref: https://github.com/ngtcp2/ngtcp2/commit/80447281bbc94af53f8aa7a4cfc19175 + 782894a3 + Ref: https://github.com/ngtcp2/ngtcp2/pull/877 + Closes #11798 - New build options: - - `BUILD_STATIC_LIBS`. Default: `OFF`. - Enabled automatically if `BUILD_SHARED_LIBS` is `OFF`. - - `BUILD_STATIC_CURL`. Default: `OFF`. - Requires `BUILD_STATIC_LIBS` enabled. - Enabled automatically if building static libcurl only. - - `STATIC_LIB_SUFFIX`. Default: empty. - - `IMPORT_LIB_SUFFIX`. Default: `_imp` if implib filename would collide - with static lib name (typically with MSVC) in Windows builds. - Otherwise empty. +Stefan Eissing (5 Sep 2023) - Also: +- http: fix sending of large requests - - Stop setting the `CURL_STATICLIB` macro via `curl_config.h`, and pass - it directly to the compiler. This also allows to delete a condition - from `tests/server/CMakeLists.txt`. + - refs #11342 where errors with git https interactions + were observed + - problem was caused by 1st sends of size larger than 64KB + which resulted in later retries of 64KB only + - limit sending of 1st block to 64KB + - adjust h2/h3 filters to cope with parsing the HTTP/1.1 + formatted request in chunks - - Complete a TODO by following the logic used in autotools (also for - `LIBCURL_NO_SHARED`), and set `-DCURL_STATICLIB` in `Cflags:` of - `libcurl.pc` for _static-only_ curl builds. + - introducing Curl_nwrite() as companion to Curl_write() + for the many cases where the sockindex is already known - - Convert an existing CI test to build both shared and static libcurl. + Fixes #11342 (again) + Closes #11803 - Closes #11505 +- pytest: fix check for slow_network skips to only apply when intended -Stefan Eissing (28 Jul 2023) + Closes #11801 -- CI/awslc: add cache for build awslc library +Daniel Stenberg (5 Sep 2023) - Closes #11535 +- curl_url_get/set.3: add missing semicolon in SYNOPSIS -- GHA/linux.yml: add caching +- CURLOPT_URL.3: explain curl_url_set() uses the same parser - Closes #11532 +- CURLOPT_URL.3: add two URL API calls in the see-also section -Daniel Stenberg (27 Jul 2023) +Dan Fandrich (4 Sep 2023) -- RELEASE-NOTES: synced +- CI: add a 32-bit i686 Linux build - Bump working version to 8.3.0 + This is done by cross-compiling under regular x86_64 Linux. Since the + kernel offers backwards compatibility, the binaries can be tested as + normal. -- url: remove infof() output for "still name resolving" + Closes #11799 - The message does not help and might get spewed a lot during times. +- tests: fix a type warning on 32-bit x86 - Reported-by: yushicheng7788 on github - Fixes #11394 - Closes #11529 +Viktor Szakats (4 Sep 2023) -- KNOWN_BUGS: cygwin: "WARNING: UNPROTECTED PRIVATE KEY FILE!" +- tests: delete stray `.orig` file - Closes #11244 + Follow-up to 331b89a319d0067fa1e6441719307cfef9c7960f + Closes #11797 -Stefan Eissing (27 Jul 2023) +Daniel Stenberg (4 Sep 2023) -- CI: quiche updates +- RELEASE-NOTES: synced - - remove quiche from standard `linux` workflow - - add mod_h2 caching to quiche workflow - - rename quiche to quiche-linux - - move version definitions into env section +Viktor Szakats (4 Sep 2023) - Closes #11528 +- lib: silence compiler warning in inet_ntop6 -- http2: disable asssertion blocking OSSFuzz testing + ``` + ./curl/lib/inet_ntop.c:121:21: warning: possible misuse of comma operator her + e [-Wcomma] + cur.base = i, cur.len = 1; + ^ + ./curl/lib/inet_ntop.c:121:9: note: cast expression to void to silence warnin + g + cur.base = i, cur.len = 1; + ^~~~~~~~~~~~ + (void)( ) + ``` - - not clear how this triggers and it blocks OSSFuzz testing other - things. Since we handle the case with an error return, disabling the - assertion for now seems the best way forward. + Closes #11790 - Fixes #11500 - Closes #11519 +Daniel Stenberg (4 Sep 2023) -- http2: fix in h2 proxy tunnel: progress in ingress on sending +- transfer: also stop the sending on closed connection - - depending on what is tunneled, the proxy may never get invoked for - receiving data explicitly. Not progressing ingress may lead to stalls - due to missed WINDOW_UPDATEs. + Previously this cleared the receiving bit only but in some cases it is + also still sending (like a request-body) when disconnected and neither + direction can continue then. - CI: - - add a chache for building mod_h2 + Fixes #11769 + Reported-by: Oleg Jukovec + Closes #11795 - Closes #11527 +John Bampton (4 Sep 2023) -- CI ngtcp2+quictls: use nghttpx cache as in quiche build +- docs: change `sub-domain` to `subdomain` -Jay Satiro (27 Jul 2023) + https://en.wikipedia.org/wiki/Subdomain -- bearssl: don't load CA certs when peer verification is disabled + Closes #11793 - We already do this for other SSL backends. +Stefan Eissing (4 Sep 2023) - Bug: https://github.com/curl/curl/pull/11457#issuecomment-1644587473 - Reported-by: kyled-dell@users.noreply.github.com +- multi: more efficient pollfd count for poll - Closes https://github.com/curl/curl/pull/11497 + - do not use separate pollfds for sockets that have POLLIN+POLLOUT -Daniel Stenberg (26 Jul 2023) + Closes #11792 -- easy: remove #ifdefs to make code easier on the eye +- http2: polish things around POST - Closes #11525 + - added test cases for various code paths + - fixed handling of blocked write when stream had + been closed inbetween attempts + - re-enabled DEBUGASSERT on send with smaller data size -Stefan Eissing (26 Jul 2023) + - in debug builds, environment variables can be set to simulate a slow + network when sending data. cf-socket.c and vquic.c support + * CURL_DBG_SOCK_WBLOCK: percentage of send() calls that should be + answered with a EAGAIN. TCP/UNIX sockets. + This is chosen randomly. + * CURL_DBG_SOCK_WPARTIAL: percentage of data that shall be written + to the network. TCP/UNIX sockets. + Example: 80 means a send with 1000 bytes would only send 800 + This is applied to every send. + * CURL_DBG_QUIC_WBLOCK: percentage of send() calls that should be + answered with EAGAIN. QUIC only. + This is chosen randomly. -- GHA: adding quiche workflow + Closes #11756 - - adding separate quiche workflow to also build nghttpx server for testing +Daniel Stenberg (4 Sep 2023) - Closes #11517 +- docs: add curl_global_trace to some SEE ALSO sections -Version 8.2.1 (26 Jul 2023) + Closes #11791 -Daniel Stenberg (26 Jul 2023) +- os400: fix checksrc nits -- RELEASE-NOTES: synced + Closes #11789 - curl 8.2.1 release +Nicholas Nethercote (3 Sep 2023) -- THANKS: add contributors from 8.2.1 +- hyper: remove `hyptransfer->endtask` -- docs: provide more see also for cipher options + `Curl_hyper_stream` needs to distinguish between two kinds of + `HYPER_TASK_EMPTY` tasks: (a) the `foreach` tasks it creates itself, and + (b) background tasks that hyper produces. It does this by recording the + address of any `foreach` task in `hyptransfer->endtask` before pushing + it into the executor, and then comparing that against the address of + tasks later polled out of the executor. - More cross references. Hide nroff errors. + This works right now, but there is no guarantee from hyper that the + addresses are stable. `hyper_executor_push` says "The executor takes + ownership of the task, which should not be accessed again unless + returned back to the user with `hyper_executor_poll`". That wording is a + bit ambiguous but with my Rust programmer's hat on I read it as meaning + the task returned with `hyper_executor_poll` may be conceptually the + same as a task that was pushed, but that there are no other guarantees + and comparing addresses is a bad idea. - Closes #11513 + This commit instead uses `hyper_task_set_userdata` to mark the `foreach` + task with a `USERDATA_RESP_BODY` value which can then be checked for, + removing the need for `hyptransfer->endtask`. This makes the code look + more like that hyper C API examples, which use userdata for every task + and never look at task addresses. -- docs: mark two TLS options for TLS, not SSL + Closes #11779 - Closes #11514 +Dave Cottlehuber (3 Sep 2023) -Brad Harder (25 Jul 2023) +- ws: fix spelling mistakes in examples and tests -- curl_multi_wait.3: fix arg quoting to doc macro .BR + Closes #11784 - Closes #11511 +Daniel Stenberg (3 Sep 2023) -Daniel Stenberg (24 Jul 2023) +- tool_filetime: make -z work with file dates before 1970 -- RELEASE-NOTES: synced + Fixes #11785 + Reported-by: Harry Sintonen + Closes #11786 -Viktor Szakats (24 Jul 2023) +Dan Fandrich (1 Sep 2023) -- cmake: update ngtcp2 detection +- build: fix portability of mancheck and checksrc targets - Replace `OpenSSL` with `quictls` to follow the same change - in the v0.17.0 ngtcp2 release. + At least FreeBSD preserves cwd across makefile lines, so rules + consisting of more than one "cd X; do_something" must be explicitly run + in a subshell to avoid this. This problem caused the Cirrus FreeBSD + build to fail when parallel make jobs were enabled. - Follow-up to e0093b4b732f6495b0fb1cd6747cbfedcdcf63ed +- CI: adjust labeler match patterns for new & obsolete files - Closes #11508 +- configure: trust pkg-config when it's used for zlib -Stefan Eissing (24 Jul 2023) + The library flags retrieved from pkg-config were later thrown out and + harded-coded, which negates the whole reason to use pkg-config. + Also, previously, the assumption was made that --libs-only-l and + --libs-only-L are the full decomposition of --libs, which is untrue and + would not allow linking against a static zlib. The new approach is + better in that it uses --libs, although only if --libs-only-l returns + nothing. -- http: VLH, very large header test and fixes + Bug: https://curl.se/mail/lib-2023-08/0081.html + Reported-by: Randall + Closes #11778 - - adding tests using very large passwords in auth - - fixes general http sending to treat h3 like h2, and - not like http1.1 - - eliminate H2_HEADER max definitions and use the commmon - DYN_HTTP_REQUEST everywhere, different limits do not help - - fix http2 handling of requests denied by nghttp2 on send - to immediately report the refused stream +Stefan Eissing (1 Sep 2023) - Closes #11509 +- CI/ngtcp2: clear wolfssl for when cache is ignored -Andrei Rybak (23 Jul 2023) + Closes #11783 -- CONTRIBUTE: drop mention of copyright year ranges +Daniel Stenberg (1 Sep 2023) - Year ranges in copyrights were dropped in commits [1] and [2]. - Verification of year ranges in copyrights was dropped from script - 'scripts/copyright.pl' in commit [3]. However, the corresponding - passages in file 'docs/CONTRIBUTE.md' weren't updated. +- RELEASE-NOTES: synced - Drop mentions of copyright year ranges from 'docs/CONTRIBUTE.md'. +Nicholas Nethercote (1 Sep 2023) - [1] 2bc1d775f (copyright: update all copyright lines and remove year - ranges, 2023-01-02) - [2] c46761bd8 (tests/http: remove year ranges from copyrights, - 2023-03-14) - [3] 0e293bacb (copyright.pl: cease doing year verifications, 2023-01-28) +- hyper: fix a progress upload counter bug - Closes #11504 + `Curl_pgrsSetUploadCounter` should be a passed a total count, not an + increment. -- CONTRIBUTE: fix syntax in commit message description + This changes the failing diff for test 579 with hyper from this: + ``` + Progress callback called with UL 0 out of 0[LF] + -Progress callback called with UL 8 out of 0[LF] + -Progress callback called with UL 16 out of 0[LF] + -Progress callback called with UL 26 out of 0[LF] + -Progress callback called with UL 61 out of 0[LF] + -Progress callback called with UL 66 out of 0[LF] + +Progress callback called with UL 29 out of 0[LF] + ``` + to this: + ``` + Progress callback called with UL 0 out of 0[LF] + -Progress callback called with UL 8 out of 0[LF] + -Progress callback called with UL 16 out of 0[LF] + -Progress callback called with UL 26 out of 0[LF] + -Progress callback called with UL 61 out of 0[LF] + -Progress callback called with UL 66 out of 0[LF] + +Progress callback called with UL 40 out of 0[LF] + ``` + Presumably a step in the right direction. - File 'docs/CONTRIBUTE.md' includes a description of how one should write - commit messages in the curl project. Different possible parts of the - message are enclosed in square brackets. One exception is the section - describing how the curl project doesn't use "Signed-off-by" commit - trailers [1], which is enclosed in an opening curly brace paired with a - closing square bracket. + Closes #11780 - Fix the enclosing square brackets in description of "Signed-off-by" - trailers in commit messages in file 'docs/CONTRIBUTE.md'. +Daniel Stenberg (1 Sep 2023) - [1] See description of option '--signoff' in Git documentation: - https://git-scm.com/docs/git-commit +- awssiv4: avoid freeing the date pointer on error - Closes #11504 + Since it was not allocated, don't free it even if it was wrong syntax -Daniel Stenberg (23 Jul 2023) + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=61908 -- src/mkhelp: strip off escape sequences + Follow-up to b137634ba3adb - At some point the nroff command stopped stripping off escape sequences, - so then this script needs to do the job instead. + Closes #11782 - Reported-by: VictorVG on github - Fixes #11501 - Closes #11503 +Stefan Eissing (1 Sep 2023) -- KNOWN_BUGS: building for old macOS fails with gcc +- CI: ngtcp2-linux: use separate caches for tls libraries - Closes #11441 + allow ever changing master for wolfssl -Jacob Hoffman-Andrews (22 Jul 2023) + Closes #11766 -- rustls: update rustls-ffi 0.10.0 +- replace `master` as wolfssl-version with recent commit - This brings in version 0.21.0 of the upstream rustls implementation, - which notable includes support for IP address certificates. +- wolfssl, use master again in CI - Closes #10865 + - with the shared session update fix landed in master, it + is time to use that in our CI again -Brad Harder (22 Jul 2023) +Nicholas Nethercote (31 Aug 2023) -- websocket: rename arguments/variables to match docs +- tests: fix formatting errors in `FILEFORMAT.md`. - Pedantry/semantic-alignment between functions, docs, comments with - respect to websocket protocol code; No functional change intended. + Without the surrounding backticks, these tags get swallowed when the + markdown is rendered. - * "totalsize", "framesize" becomes "fragsize" (we deal in frame fragments). + Closes #11777 - * "sendflags" becomes "flags" +Viktor Szakats (31 Aug 2023) - * use canonical CURL *handle +- cmake: add support for `CURL_DEFAULT_SSL_BACKEND` - Closes #11493 + Allow overriding the default TLS backend via a CMake setting. -Jan Macku (21 Jul 2023) + E.g.: + `cmake [...] -DCURL_DEFAULT_SSL_BACKEND=mbedtls` -- bug_report: use issue forms instead of markdown template + Accepted values: bearssl, gnutls, mbedtls, openssl, rustls, + schannel, secure-transport, wolfssl - Issue forms allow you to define web-like input forms using YAML - syntax. It allows you to guide the reporter to get the required - information. + The passed string is baked into the curl/libcurl binaries. + The value is case-insensitive. - Signed-off-by: Jan Macku - Closes #11474 + We added a similar option to autotools in 2017 via + c7170e20d0a18ec8a514b4daa53bcdbb4dcb3a05. -Daniel Stenberg (21 Jul 2023) + TODO: Convert to lowercase to improve reproducibility. -- TODO: Obey Retry-After in redirects + Closes #11774 - (remove "Set custom client ip when using haproxy protocol" which was - shipped in 8.2.0) +- sectransp: fix compiler warnings - Mentioned-by: Yair Lenga - Closes #11447 + https://github.com/curl/curl-for-win/actions/runs/6037489221/job/16381860220# + step:3:11046 + ``` + /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:2435:1 + 4: warning: unused variable 'success' [-Wunused-variable] + OSStatus success; + ^ + /Users/runner/work/curl-for-win/curl-for-win/curl/lib/vtls/sectransp.c:3300:4 + 4: warning: unused parameter 'sha256len' [-Wunused-parameter] + size_t sha256len) + ^ + ``` -- RELEASE-NOTES: synced + Closes #11773 -Oliver Roberts (21 Jul 2023) +- tidy-up: mostly whitespace nits -- amissl: fix AmiSSL v5 detection + - delete completed TODO from `./CMakeLists.txt`. + - convert a C++ comment to C89 in `./CMake/CurlTests.c`. + - delete duplicate EOLs from EOF. + - add missing EOL at EOF. + - delete whitespace at EOL (except from expected test results). + - convert tabs to spaces. + - convert CRLF EOLs to LF in GHA yaml. + - text casing fixes in `./CMakeLists.txt`. + - fix a codespell typo in `packages/OS400/initscript.sh`. - Due to changes in the AmiSSL SDK, the detection needed adjusting. + Closes #11772 - Closes #11477 +Dan Fandrich (31 Aug 2023) -Alois Klink (21 Jul 2023) +- CI: remove Windows builds from Cirrus, without replacement -- unittest/makefile: remove unneeded unit1621_LDADD + If we don't do this, all coverage on Cirrus will cease in a few days. By + removing the Windows builds, the FreeBSD one should still continue + as before. The Windows builds will need be moved to another service to + maintain test coverage. - The `unit1621_LDADD` variable has the exact same value as the `LDADD` - flag in `Makefile.am`, except without `@LDFLAGS@ @LIBCURL_LIBS@`. + Closes #11771 - This was originally added by [98e6629][], but I can't see any reason - why it exists, so we should remove it to clean things up. +- CI: switch macOS ARM build from Cirrus to Circle CI - [98e6629]: https://github.com/curl/curl/commit/98e6629154044e4ab1ee7cff8351c7 - ebcb131e88 + Cirrus is drastically reducing their free tier on Sept. 1, so they will + no longer perform all these builds for us. All but one build has been + moved, with the LibreSSL one being dropped because of linking problems + on Circle. - Closes #11494 + One important note about this change is that Circle CI is currently + directing all these builds to x86_64 hardware, despite them requesting + ARM. This is because ARM nodes are scheduled to be available on the + free tier only in December. This reduces our architectural diversity + until then but it should automatically come back once those machines are + enabled. -- unittest/makefile: remove unneeded unit1394_LDADD +- CI: use the right variable for BSD make - These custom `unit1394_LDADD` and similar automake overrides are no - longer neded. They were originally added by added by [8dac7be][] for - metalink support, but are no longer after [265b14d][] removed metalink. + BSD uses MAKEFLAGS instead of MAKE_FLAGS so it wasn't doing parallel + builds before. - [8dac7be]: https://github.com/curl/curl/commit/8dac7be438512a8725d3c71e9139bd - fdcac1ed8c - [265b14d]: https://github.com/curl/curl/commit/265b14d6b37c4298bd5556fabcbc37 - d36f911693 +- CI: drop the FreeBSD 12.X build - Closes #11494 + Cirrus' new free tier won't let us have many builds, so drop the + nonessential ones. The FreeBSD 13.X build will still give us the most + relevant FreeBSD coverage. -- cmake: add `libcurlu`/`libcurltool` for unit tests +- CI: move the Alpine build from Cirrus to GHA - Add a `libcurlu`/`libcurltool` static library that is compiled only for - unit tests. We use `EXCLUDE_FROM_ALL` to make sure that they're not - built by default, they're only built if unit tests are built. + Cirrus is reducing their free tier to next to nothing, so we must move + builds elsewhere. - These libraries allow us to compile every unit test with CMake. +Stefan Eissing (30 Aug 2023) - Closes #11446 +- test_07_upload.py: fix test_07_34 curl args -Daniel Stenberg (21 Jul 2023) + - Pass correct filename to --data-binary. -- test979: test -u with redirect to (the same) absolute host + Prior to this change --data-binary was passed an incorrect filename due + to a missing separator in the arguments list. Since aacbeae7 curl will + error on incorrect filenames for POST. - Verifies #11492 + Fixes https://github.com/curl/curl/issues/11761 + Closes https://github.com/curl/curl/pull/11763 -- transfer: do not clear the credentials on redirect to absolute URL +Nicholas Nethercote (30 Aug 2023) - Makes test 979 work. Regression shipped in 8.2.0 from commit - dd4d1a26959f63a2c +- tests: document which tests fail due to hyper's lack of trailer support. - Fixes #11486 - Reported-by: Cloudogu Siebels - Closes #11492 + Closes #11762 -Jon Rumsey (20 Jul 2023) +- docs: removing "pausing transfers" from HYPER.md. -- os400: correct EXPECTED_STRING_LASTZEROTERMINATED + It's a reference to #8600, which was fixed by #9070. - Correct EXPECTED_STRING_LASTZEROTERMINATED to account for - CURLOPT_HAPROXY_CLIENT_IP which requires EBCDIC to ASCII conversion when - passed into curl_easy_setopt(). + Closes #11764 - Closes #11476 +Patrick Monnerat (30 Aug 2023) -Oliver Roberts (20 Jul 2023) +- os400: handle CURL_TEMP_PRINTF() while building bind source -- amissl: add missing signal.h include + Closes #11547 - In some environments, signal.h is already included, but not in others - which cause compilation to fail, so explictly include it. +- os400: build test servers - Closes #11478 + Also fix a non-compliant main prototype in disabled.c. -- amigaos: fix sys/mbuf.h m_len macro clash + Closes #11547 - The updated Curl_http_req_make and Curl_http_req_make2 functions spawned - a parameter called m_len. The AmigaOS networking headers, derived from - NetBSD, contain "#define m_len m_hdr.mh_len" which clashes with - this. Since we do not actually use mbuf, force the include file to be - ignored, removing the clash. +- tests: fix compilation error for os400 - Closes #11479 + OS400 uses BSD 4.3 setsockopt() prototype by default: this does not + define parameter as const, resulting in an error if actual parameter is + const. Remove the const keyword from the actual parameter cast: this + works in all conditions, even if the formal parameter uses it. -Daniel Stenberg (20 Jul 2023) + Closes #11547 -- socks: print ipv6 address within brackets +- os400: make programs and command name configurable - Fixes #11483 - Closes #11484 + Closes #11547 -Christian Schmitz (20 Jul 2023) +- os400: move build configuration parameters to a separate script -- libcurl-errors.3: add CURLUE_OK + They can then easily be overriden in a script named "config400.override" + that is not part of the distribution. - Closes #11488 + Closes #11547 -Oliver Roberts (20 Jul 2023) +- os400: implement CLI tool -- cfilters: rename close/connect functions to avoid clashes + This is provided as a QADRT (ascii) program, a link to it in the IFS and + a minimal CL command. - Rename `close` and `connect` in `struct Curl_cftype` for - consistency and to avoid clashes with macros of the same name - (the standard AmigaOS networking connect() function is implemented - via a macro). + Closes #11547 - Closes #11491 +Matthias Gatto (30 Aug 2023) -Stefan Eissing (20 Jul 2023) +- lib: fix aws-sigv4 having date header twice in some cases -- http2: fix regression on upload EOF handling + When the user was providing the header X-XXX-Date, the header was + re-added during signature computation, and we had it twice in the + request. - - a regression introduced by c9ec85121110d7cbbbed2990024222c8f5b8afe5 - where optimization of small POST bodies leads to a new code path - for such uploads that did not trigger the "done sending" event - - add triggering this event for early "upload_done" situations + Reported-by: apparentorder@users.noreply.github.com - Fixes #11485 - Closes #11487 - Reported-by: Aleksander Mazur + Signed-off-by: Matthias Gatto -Daniel Stenberg (19 Jul 2023) + Fixes: https://github.com/curl/curl/issues/11738 + Closes: https://github.com/curl/curl/pull/11754 -- configure: check for nghttp2_session_get_stream_local_window_size +Jay Satiro (30 Aug 2023) - The http2 code uses it now. Introduced in nghttp2 1.15.0 (Sep 2016) +- multi: remove 'processing: ' debug message - Fixes #11470 - Reported-by: Paul Howarth - Closes #11473 + - Remove debug message added by e024d566. -Stefan Eissing (19 Jul 2023) + Closes https://github.com/curl/curl/pull/11759 -- quiche: fix segfault and other things +- ftp: fix temp write of ipv6 address - - refs #11449 where a segfault is reported when IP Eyeballing did - not immediately connect but made several attempts - - The transfer initiating the eyeballing was initialized too early, - leadding to references to the filter instance that was then - replaced in the subsequent eyeball attempts. That led to a use - after free in the buffer handling for the transfer - - transfers are initiated now more lazy (like in the ngtcp2 filter), - when the stream is actually opened - - suppress reporting on quiche event errors for "other" transfers - than the current one to not fail a transfer due to faults in - another one. - - revert recent return value handling for quiche_h3_recv_body() - to not indicate an error but an EAGAIN situation. We wish quiche - would document what functions return. + - During the check to differentiate between a port and IPv6 address + without brackets, write the binary IPv6 address to an in6_addr. - Fixes #11449 - Closes #11469 - Reported-by: ウさん + Prior to this change the binary IPv6 address was erroneously written to + a sockaddr_in6 'sa6' when it should have been written to its in6_addr + member 'sin6_addr'. There's no fallout because no members of 'sa6' are + accessed before it is later overwritten. -Daniel Stenberg (19 Jul 2023) + Closes https://github.com/curl/curl/pull/11747 -- hostip: return IPv6 first for localhost resolves +- tool: change some fopen failures from warnings to errors - Fixes #11465 - Reported-by: Chilledheart on github - Closes #11466 + - Error on missing input file for --data, --data-binary, + --data-urlencode, --header, --variable, --write-out. -Harry Sintonen (19 Jul 2023) + Prior to this change if a user of the curl tool specified an input file + for one of the above options and that file could not be opened then it + would be treated as zero length data instead of an error. For example, a + POST using `--data @filenametypo` would cause a zero length POST which + is probably not what the user intended. -- tool: fix tool_seek_cb build when SIZEOF_CURL_OFF_T > SIZEOF_OFF_T + Closes https://github.com/curl/curl/pull/11677 - - a variable was renamed, and some use of it wasn't. this fixes the - build. +- hostip: fix typo - Closes #11468 +Davide Masserut (29 Aug 2023) -Stefan Eissing (19 Jul 2023) +- tool: avoid including leading spaces in the Location hyperlink -- quiche: fix lookup of transfer at multi + Co-authored-by: Dan Fandrich - - refs #11449 where weirdness in quiche multi connection tranfers was - observed - - fixes lookup of transfer for a quiche event to take the connection - into account - - formerly, a transfer with the same stream_id, but on another connection - could be found + Closes #11735 - Closes #11462 +Daniel Stenberg (29 Aug 2023) -Daniel Stenberg (19 Jul 2023) +- SECURITY-PROCESS.md: not a sec issue: Tricking user to run a cmdline -- RELEASE-NOTES: synced + Closes #11757 - bump to 8.2.1 +- connect: stop halving the remaining timeout when less than 600 ms left -John Haugabook (19 Jul 2023) + When curl wants to connect to a host, it always has a TIMEOUT. The + maximum time it is allowed to spend until a connect is confirmed. -- ciphers.d: put URL in first column + curl will try to connect to each of the IP adresses returned for the + host. Two loops, one for each IP family. - This makes the URL turn into a link properly when "webified". + During the connect loop, while curl has more than one IP address left to + try within a single address family, curl has traditionally allowed (time + left/2) for *this* connect attempt. This, to not get stuck on the + initial addresses in case the timeout but still allow later addresses to + get attempted. - Fixes https://github.com/curl/curl-www/issues/270 - Closes #11464 + This has the downside that when users set a very short timeout and the + host has a large number of IP addresses, the effective result might be + that every attempt gets a little too short time. -Version 8.2.0 (19 Jul 2023) + This change stop doing the divided-by-two if the total time left is + below a threshold. This threshold is 600 milliseconds. -Daniel Stenberg (19 Jul 2023) + Closes #11693 -- RELEASE-NOTES: synced +- asyn-ares: reduce timeout to 2000ms - 8.2.0 release + When UDP packets get lost this makes for slightly faster retries. This + lower timeout is used by @c-ares itself by default starting next + release. -- THANKS-filter: strip out "GitHub" + Closes #11753 -- THANKS: add contributors from 8.2.0 +John Bampton (29 Aug 2023) -- RELEASE-PROCEDURE.md: adjust the release dates +- misc: remove duplicate words -Stefan Eissing (17 Jul 2023) + Closes #11740 -- quiche: fix defects found in latest coverity report +Daniel Stenberg (29 Aug 2023) - Closes #11455 +- RELEASE-NOTES: synced -Daniel Stenberg (17 Jul 2023) +- wolfSSL: avoid the OpenSSL compat API when not needed -- quiche: avoid NULL deref in debug logging + ... and instead call wolfSSL functions directly. - Coverity reported "Dereference after null check" + Closes #11752 - If stream is NULL and the function exits, the logging must not deref it. +Viktor Szakats (28 Aug 2023) - Closes #11454 +- lib: fix null ptr derefs and uninitialized vars (h2/h3) -Stefan Eissing (17 Jul 2023) + Fixing compiler warnings with gcc 13.2.0 in unity builds. -- http2: treat initial SETTINGS as a WINDOW_UPDATE + Assisted-by: Jay Satiro + Assisted-by: Stefan Eissing + Closes #11739 - - refs #11426 where spurious stalls on large POST requests - are reported - - the issue seems to involve the following - * first stream on connection adds up to 64KB of POST - data, which is the max default HTTP/2 stream window size - transfer is set to HOLD - * initial SETTINGS from server arrive, enlarging the stream - window. But no WINDOW_UPDATE is received. - * curl stalls - - the fix un-HOLDs a stream on receiving SETTINGS, not - relying on a WINDOW_UPDATE from lazy servers +Jay Satiro (28 Aug 2023) - Closes #11450 +- secureserver.pl: fix stunnel version parsing -Daniel Stenberg (17 Jul 2023) + - Allow the stunnel minor-version version part to be zero. -- ngtcp2: assigning timeout, but value is overwritten before used + Prior to this change with the stunnel version scheme of . + if either part was 0 then version parsing would fail, causing + secureserver.pl to fail with error "No stunnel", causing tests that use + the SSL protocol to be skipped. As a practical matter this bug can only + be caused by a minor-version part of 0, since the major-version part is + always greater than 0. - Reported by Coverity + Closes https://github.com/curl/curl/pull/11722 - Closes #11453 +- secureserver.pl: fix stunnel path quoting -- krb5: add typecast to please Coverity + - Store the stunnel path in the private variable $stunnel unquoted and + instead quote it in the command strings. -Derzsi Dániel (16 Jul 2023) + Prior to this change the quoted stunnel path was passed to perl's file + operators which cannot handle quoted paths. For example: -- wolfssl: support setting CA certificates as blob + $stunnel = "\"/C/Program Files (x86)/stunnel/bin/tstunnel\""; + if(-x $stunnel or -x "$stunnel") + # false even if path exists and is executable - Closes #11445 + Our other test scripts written in perl, unlike this one, use servers.pm + which has a global $stunnel variable with the path stored unquoted and + therefore those scripts don't have this problem. -- wolfssl: detect when TLS 1.2 support is not built into wolfssl + Closes https://github.com/curl/curl/pull/11721 - Closes #11444 +Daniel Stenberg (28 Aug 2023) -Graham Campbell (15 Jul 2023) +- altsvc: accept and parse IPv6 addresses in response headers -- CI: bump nghttp2 from 1.55.0 to 1.55.1 + Store numerical IPv6 addresses in the alt-svc file with the brackets + present. - Closes #11442 + Verify with test 437 and 438 -Daniel Stenberg (15 Jul 2023) + Fixes #11737 + Reported-by: oliverpool on github + Closes #11743 -- curl: return error when asked to use an unsupported HTTP version +- libtest: use curl_free() to free libcurl allocated data - When one of the following options are used but the libcurl in use does - not support it: + In several test programs. These mistakes are not detected or a problem + as long as memdebug.h is included, as that provides the debug wrappers + for all memory functions in the same style libcurl internals do it, + which makes curl_free and free effectively the same call. - --http2 - --http2-prior-knowledge - --proxy-http2 + Reported-by: Nicholas Nethercote + Closes #11746 - Closes #11440 +Jay Satiro (28 Aug 2023) -Chris Paulson-Ellis (14 Jul 2023) +- disable.d: explain --disable not implemented prior to 7.50.0 -- cf-socket: don't bypass fclosesocket callback if cancelled before connect + Option -q/--disable was added in 5.0 but only -q was actually + implemented. Later --disable was implemented in e200034 (precedes + 7.49.0), but incorrectly, and fixed in 6dbc23c (precedes 7.50.0). - After upgrading to 8.1.2 from 7.84.0, I found that sockets were being - closed without calling the fclosesocket callback if a request was - cancelled after the associated socket was created, but before the socket - was connected. This lead to an imbalance of fopensocket & fclosesocket - callbacks, causing problems with a custom event loop integration using - the multi-API. + Reported-by: pszlazak@users.noreply.github.com - This was caused by cf_socket_close() calling sclose() directly instead - of calling socket_close() if the socket was not active. For regular TCP - client connections, the socket is activated by cf_socket_active(), which - is only called when the socket completes the connect. + Fixes https://github.com/curl/curl/issues/11710 + Closes #11712 - As far as I can tell, this issue has existed since 7.88.0. That is, - since the code in question was introduced by: - commit 71b7e0161032927cdfb4e75ea40f65b8898b3956 - Author: Stefan Eissing - Date: Fri Dec 30 09:14:55 2022 +0100 +Nicholas Nethercote (28 Aug 2023) - lib: connect/h2/h3 refactor +- hyper: fix ownership problems - Closes #11439 + Some of these changes come from comparing `Curl_http` and + `start_CONNECT`, which are similar, and adding things to them that are + present in one and missing in another. -Daniel Stenberg (13 Jul 2023) + The most important changes: + - In `start_CONNECT`, add a missing `hyper_clientconn_free` call on the + happy path. + - In `start_CONNECT`, add a missing `hyper_request_free` on the error + path. + - In `bodysend`, add a missing `hyper_body_free` on an early-exit path. + - In `bodysend`, remove an unnecessary `hyper_body_free` on a different + error path that would cause a double-free. + https://docs.rs/hyper/latest/hyper/ffi/fn.hyper_request_set_body.html + says of `hyper_request_set_body`: "This takes ownership of the + hyper_body *, you must not use it or free it after setting it on the + request." This is true even if `hyper_request_set_body` returns an + error; I confirmed this by looking at the hyper source code. -- tool_parsecfg: accept line lengths up to 10M + Other changes are minor but make things slightly nicer. - Bumped from 100K set in 47dd957daff9 + Closes #11745 - Reported-by: Antoine du Hamel - Fixes #11431 - Closes #11435 +Daniel Stenberg (28 Aug 2023) -Stefan Eissing (13 Jul 2023) +- multi.h: the 'revents' field of curl_waitfd is supported -- CI: brew fix for openssl in default path + Since 6d30f8ebed34e7276 - If brew install/update links openssl into /usr/local, it will be found - before anything we add with `-isystem path` to CPP/LDLFAGS. Get rid of - that by unlinking the keg. + Reported-by: Nicolás Ojeda Bär + Ref: #11748 + Closes #11749 - Fixes #11413 - Closes #11436 +Gerome Fournier (27 Aug 2023) -Daniel Stenberg (13 Jul 2023) +- tool_paramhlp: improve str2num(): avoid unnecessary call to strlen() -- RELEASE-NOTES: synced + Closes #11742 -Ondřej Koláček (13 Jul 2023) +Daniel Stenberg (27 Aug 2023) -- sectransp: fix EOF handling +- docs: mention critical files in same directories as curl saves - Regression since the large refactor from 2022 + ... cannot be fully protected. Don't do it. - Closes #11427 + Co-authored-by: Jay Satiro + Reported-by: Harry Sintonen + Fixes #11530 + Closes #11701 -Daniel Stenberg (13 Jul 2023) +John Hawthorn (26 Aug 2023) -- checksrc: quote the file name to work with "funny" letters +- OpenSSL: clear error queue after SSL_shutdown - Closes #11437 + We've seen errors left in the OpenSSL error queue (specifically, + "shutdown while in init") by adding some logging it revealed that the + source was this file. -Karthikdasari0423 (13 Jul 2023) + Since we call SSL_read and SSL_shutdown here, but don't check the return + code for an error, we should clear the OpenSSL error queue in case one + was raised. -- HTTP3.md: ngtcp2 updated to v0.17.0 and nghttp3 to v0.13.0 + This didn't affect curl because we call ERR_clear_error before every + write operation (a0dd9df9ab35528eb9eb669e741a5df4b1fb833c), but when + libcurl is used in a process with other OpenSSL users, they may detect + an OpenSSL error pushed by libcurl's SSL_shutdown as if it was their + own. - Follow-up to e0093b4b732f6 + Co-authored-by: Satana de Sant'Ana - Closes #11433 + Closes #11736 -Daniel Stenberg (13 Jul 2023) +Alexander Kanavin (25 Aug 2023) -- CURLOPT_MIMEPOST.3: clarify what setting to NULL means +- tests: update cookie expiry dates to far in the future - Follow-up to e08382a208d4e480 + This allows testing Y2038 with system time set to after that, so that + actual Y2038 issues can be exposed, and not masked by expiry errors. - Closes #11430 + Fixes #11576 + Closes #11610 -Tatsuhiro Tsujikawa (12 Jul 2023) +John Bampton (25 Aug 2023) -- ngtcp2: build with 0.17.0 and nghttp3 0.13.0 +- misc: fix spelling - - ngtcp2_crypto_openssl was renamed to ngtcp2_crypto_quictls. + Closes #11733 - Closes #11428 +Daniel Stenberg (25 Aug 2023) -- CI: Bump ngtcp2, nghttp3, and nghttp2 +- cmdline-opts/page-header: clarify stronger that !opt == URL - Closes #11428 + Everything provided on the command line that is not an option (or an + argument to an option) is treated as a URL. -James Fuller (11 Jul 2023) + Closes #11734 -- example/maxconnects: set maxconnect example +- tests/runner: fix %else handling - Closes #11343 + Getting the show state proper for %else and %endif did not properly work + in nested cases. -Pontakorn Prasertsuk (11 Jul 2023) + Follow-up to 3d089c41ea9 -- http2: send HEADER & DATA together if possible + Closes #11731 - Closes #11420 +Nicholas Nethercote (25 Aug 2023) -Daniel Stenberg (11 Jul 2023) +- docs: Remove mention of #10803 from `KNOWN_BUGS`. -- CI: use wolfSSL 5.6.3 in builds + Because the leaks have been fixed. - No using master anymore +- c-hyper: fix another memory leak in `Curl_http`. - Closes #11424 + There is a `hyper_clientconn_free` call on the happy path, but not one + on the error path. This commit adds one. -SaltyMilk (11 Jul 2023) + Fixes the second memory leak reported by Valgrind in #10803. -- fopen: optimize + Fixes #10803 + Closes #11729 - Closes #11419 +- c-hyper: fix a memory leak in `Curl_http`. -Daniel Stenberg (11 Jul 2023) + A request created with `hyper_request_new` must be consumed by either + `hyper_clientconn_send` or `hyper_request_free`. -- cmake: make use of snprintf + This is not terrifically clear from the hyper docs -- + `hyper_request_free` is documented only with "Free an HTTP request if + not going to send it on a client" -- but a perusal of the hyper code + confirms it. - Follow-up to 935b1bd4544a23a91d68 + This commit adds a `hyper_request_free` to the `error:` path in + `Curl_http` so that the request is consumed when an error occurs after + the request is created but before it is sent. - Closes #11423 + Fixes the first memory leak reported by Valgrind in #10803. -Stefan Eissing (11 Jul 2023) + Closes #11729 -- macOS: fix taget detection +Daniel Stenberg (25 Aug 2023) - - TARGET_OS_OSX is not always defined on macOS - - this leads to missing symbol Curl_macos_init() - - TargetConditionals.h seems to define these only when - dynamic targets are enabled (somewhere?) - - this PR fixes that on my macOS 13.4.1 - - I have no clue why CI builds worked without it +- RELEASE-NOTES: synced - Follow-up to c7308592fb8ba213fc2c1 - Closes #11417 +John Bampton (25 Aug 2023) -Stan Hu (9 Jul 2023) +- misc: spellfixes -- hostip.c: Move macOS-specific calls into global init call + Closes #11730 - https://github.com/curl/curl/pull/7121 introduced a macOS system call - to `SCDynamicStoreCopyProxies`, which is invoked every time an IP - address needs to be resolved. +Daniel Stenberg (25 Aug 2023) - However, this system call is not thread-safe, and macOS will kill the - process if the system call is run first in a fork. To make it possible - for the parent process to call this once and prevent the crash, only - invoke this system call in the global initialization routine. +- tests: add support for nested %if conditions - In addition, this change is beneficial because it: + Provides more flexiblity to test cases. - 1. Avoids extra macOS system calls for every IP lookup. - 2. Consolidates macOS-specific initialization in a separate file. + Also warn and bail out if there is an '%else' or %endif' without a + preceeding '%if'. - Fixes #11252 - Closes #11254 + Ref: #11610 + Closes #11728 -Daniel Stenberg (9 Jul 2023) +- time-cond.d: mention what happens on a missing file -- docs: use a space after RFC when spelling out RFC numbers + Closes #11727 - Closes #11382 +Christian Hesse (24 Aug 2023) -Margu (9 Jul 2023) +- docs/cmdline-opts: match the current output -- imap-append.c: update to make it more likely to work + The release date has been added in output, reflect that in documentation. - Fixes #10300 - Closes #11397 + Closes #11723 -Emanuele Torre (9 Jul 2023) +Daniel Stenberg (24 Aug 2023) -- tool_writeout_json: fix encoding of control characters +- lib: minor comment corrections - Control characters without a special escape sequence e.g. %00 or %06 - were being encoded as "u0006" instead of "\u0006". +- docs: rewrite to present tense - Ref: https://github.com/curl/trurl/pull/214#discussion_r1257487858 - Closes #11414 + ... instead of using future tense. -Stefan Eissing (9 Jul 2023) + + numerous cleanups and improvements + + stick to "reuse" not "re-use" + + fewer contractions -- http3/ngtcp2: upload EAGAIN handling + Closes #11713 - - refs #11389 where IDLE timeouts on upload are reported - - reword ngtcp2 expiry handling to apply to both send+recv - calls into the filter - - EAGAIN uploads similar to the recent changes in HTTP/2, e.g. - report success only when send data was ACKed. - - HOLD sending of EAGAINed uploads to avoid cpu busy loops - - rename internal function for consistency with HTTP/2 - implementation +- urlapi: setting a blank URL ("") is not an ok URL - Fixes #11389 - Closes #11390 + Test it in 1560 + Fixes #11714 + Reported-by: ad0p on github + Closes #11715 -Brian Nixon (9 Jul 2023) +- spelling: use 'reuse' not 're-use' in code and elsewhere -- tool_easysrc.h: correct `easysrc_perform` for `CURL_DISABLE_LIBCURL_OPTION` + Unify the spelling as both versions were previously used intermittently - Closes #11398 + Closes #11717 -Daniel Stenberg (9 Jul 2023) +Michael Osipov (23 Aug 2023) -- RELEASE-NOTES: synced +- system.h: add CURL_OFF_T definitions on HP-UX with HP aCC -- transfer: clear credentials when redirecting to absolute URL + HP-UX on IA64 provides two modes: 32 and 64 bit while 32 bit being the + default one. Use "long long" in 32 bit mode and just "long" in 64 bit + mode. - Make sure the user and password for the second request is taken from the - redirected-to URL. + Closes #11718 - Add test case 899 to verify. +Dan Fandrich (22 Aug 2023) - Reported-by: James Lucas - Fixes #11410 - Closes #11412 +- tests: don't call HTTP errors OK in test cases -Stefan Eissing (8 Jul 2023) + Some HTTP errors codes were accompanied by the text OK, which causes + some cognitive dissonance when reading them. -- hyper: fix EOF handling on input +- http: close the connection after a late 417 is received - We ran out of disc space due to an infinite loop with debug logging + In this situation, only part of the data has been sent before aborting + so the connection is no longer usable. - Fixes #11377 - Closes #11385 - Reported-by: Dan Fandrich + Assisted-by: Jay Satiro + Fixes #11678 + Closes #11679 -- http2: raise header limitations above and beyond +- runtests: slightly increase the longest log file displayed - - not quite to infinity - - rewrote the implementation of our internal HTTP/1.x request - parsing to work with very large lines using dynbufs. - - new default limit is `DYN_HTTP_REQUEST`, aka 1MB, which - is also the limit of curl's general HTTP request processing. + The new limit provides enough space for a 64 KiB data block to be logged + in a trace file, plus a few lines at the start and end for context. This + happens to be the amount of data sent at a time in a PUT request. - Fixes #11405 - Closes #11407 +- tests: add delay command to the HTTP server -Juan Cruz Viotti (8 Jul 2023) + This adds a delay after client connect. -- curl_easy_nextheader.3: add missing open parenthesis examples +Daniel Stenberg (22 Aug 2023) - Closes #11409 - Signed-off-by: Juan Cruz Viotti +- cirrus: install everthing with pkg, avoid pip -Dan Fandrich (7 Jul 2023) + Assisted-by: Sevan Janiyan -- CI: enable verbose test output on pytest + Closes #11711 - This shows individual pass/fail status on tests and makes this output - consistent with other jobs' pytest invocations. +- curl_url*.3: update function descriptions -Stefan Eissing (28 Jun 2023) + - expand and clarify several descriptions + - avoid using future tense all over -- http2: fix crash in handling stream weights + Closes #11708 - - Delay the priority handling until the stream has been opened. +- RELEASE-NOTES: synced - - Add test2404 to reproduce and verify. +Stefan Eissing (21 Aug 2023) - Weights may change "on the run", which is why there are checks in - general egress handling. These must not trigger when the stream has not - been opened yet. +- CI/cirrus: disable python install on FreeBSD - Reported-by: jbgoog@users.noreply.github.com + - python cryptography package does not build build FreeBSD + - install just mentions "error" + - this gets the build and the main test suite going again - Fixes https://github.com/curl/curl/issues/11379 - Closes https://github.com/curl/curl/pull/11384 + Closes #11705 -- tests/http: Add mod_h2 directive `H2ProxyRequests` +- test2600: fix flakiness on low cpu - master of mod_h2 now requires H2ProxyRequests directives for forward - proxying with HTTP/2 to work. + - refs #11355 where failures to to low cpu resources in CI + are reported + - vastly extend CURLOPT_CONNECTTIMEOUT_MS and max durations + to test cases + - trigger Curl_expire() in test filter to allow re-checks before + the usual 1second interval - Ref: https://github.com/icing/mod_h2/commit/3897a7086 + Closes #11690 - Closes https://github.com/curl/curl/pull/11392 +Maksim Sciepanienka (20 Aug 2023) -Dan Fandrich (28 Jun 2023) +- tool_urlglob: use the correct format specifier for curl_off_t in msnprintf -- CI: make Appveyor job names unique + Closes #11698 - Two otherwise identical mingw-w64 jobs now have their differing compiler - versions mentioned in their names. +Daniel Stenberg (20 Aug 2023) -Sheshadri.V (25 Jun 2023) +- test687/688: two more basic --xattr tests -- curl.h: include for vxworks + Closes #11697 - Closes #11356 +- cmdline-opts/docs: mentioned the negative option part -Dan Fandrich (24 Jun 2023) + ... for --no-alpn and --no-buffer in the same style done for other --no- + options: -- CI: enable parallel make in more builds + "Note that this is the negated option name documented." - Most CI services provide at least two cores, so enable parallel make - jobs to take advantage of that for builds. Some dependencies aren't safe - to build in parallel so leave those as-is. Also, rename a few - workflows to eliminate duplicate names and provide a better idea what - they're about. + Closes #11695 -- CI: don't install impacket if tests are not run +Emanuele Torre (19 Aug 2023) - It just wastes time and bandwidth and isn't even used. +- tool/var: also error when expansion result starts with NUL -divinity76 (24 Jun 2023) + Expansions whose output starts with NUL were being expanded to the empty + string, and not being recognised as values that contain a NUL byte, and + should error. -- configure: the --without forms of the options are also gone + Closes #11694 - --without-darwin-ssl and --without-metalink +Daniel Stenberg (19 Aug 2023) - Closes #11378 +- tests: add 'large-time' as a testable feature -Daniel Stenberg (23 Jun 2023) + This allows test cases to require this feature to run and to be used in + %if conditions. -- configure: add check for ldap_init_fd + Large here means larger than 32 bits. Ie does not suffer from y2038. - ... as otherwise the configure script will say it is OpenLDAP in the - summary, but not set the USE_OPENLDAP define, therefor not using the - intended OpenLDAP code paths. + Closes #11696 - Regression since 4d7385446 (7.85.0) - Fixes #11372 - Closes #11374 - Reported-by: vlkl-sap on github +- tests/Makefile: add check-translatable-options.pl to tarball -Michał Petryka (23 Jun 2023) + Used in test 1544 -- cmake: stop CMake from quietly ignoring missing Brotli + Follow-up to ae806395abc8c - The CMake project was set to `QUIET` for Brotli instead of - `REQUIRED`. This makes builds unexpectedly ignore missing Brotli even - when `CURL_BROTLI` is enabled. +- gen.pl: fix a long version generation mistake - Closes #11376 + Too excessive escaping made the parsing not find the correct long names + later and instead add "wrong" links. -Emanuele Torre (22 Jun 2023) + Follow-up to 439ff2052e219 -- docs: add more .IP after .RE to fix indentation of generate paragraphs + Reported-by: Lukas Tribus + Fixes #11688 + Closes #11689 - follow-up from 099f41e097c030077b8ec078f2c2d4038d31353b +- lib: move mimepost data from ->req.p.http to ->state - I just thought of checking all the other files with .RE, and I found 6 - other files that were missing .IP at the end. + When the legacy CURLOPT_HTTPPOST option is used, it gets converted into + the modem mimpost struct at first use. This data is (now) kept for the + entire transfer and not only per single HTTP request. This re-enables + rewind in the beginning of the second request instead of in end of the + first, as brought by 1b39731. - Closes #11375 + The request struct is per-request data only. -Stefan Eissing (22 Jun 2023) + Extend test 650 to verify. -- http2: h2 and h2-PROXY connection alive check fixes + Fixes #11680 + Reported-by: yushicheng7788 on github + Closes #11682 - - fix HTTP/2 check to not declare a connection dead when - the read attempt results in EAGAIN - - add H2-PROXY alive check as for HTTP/2 that was missing - and is needed - - add attach/detach around Curl_conn_is_alive() and remove - these in filter methods - - add checks for number of connections used in some test_10 - proxy tunneling tests +Patrick Monnerat (17 Aug 2023) - Closes #11368 +- os400: do not check translatable options at build time -- http2: error stream resets with code CURLE_HTTP2_STREAM + Now that there is a test for this, the build time check is not needed + anymore. - - refs #11357, where it was reported that HTTP/1.1 downgrades - no longer works - - fixed with suggested change - - added test_05_03 and a new handler in the curltest module - to reproduce that downgrades work + Closes #11650 - Fixes #11357 - Closes #11362 - Reported-by: Jay Satiro +- test1554: check translatable string options in OS400 wrapper -Daniel Stenberg (22 Jun 2023) + This test runs a perl script that checks all string options are properly + translated by the OS400 character code conversion wrapper. It also + verifies these options are listed in alphanumeric order in the wrapper + switch statement. -- connect-timeout.d: mention that the DNS lookup is included + Closes #11650 - Closes #11370 +Daniel Stenberg (17 Aug 2023) -Emanuele Torre (22 Jun 2023) +- unit3200: skip testing if function is not present -- quote.d: fix indentation of generated paragraphs + Fake a successful run since we have no easy mechanism to skip this test + for this advanced condition. - quote.d was missing a .IP at the end which caused the paragraphs - generated for See-also, Multi, and Example to not be indented correctly. +- unit2600: fix build warning if built without verbose messages - I also remove a redundant "This option can be used multiple times.", and - replaced .IP "item" with .TP .B "item" to make more clear which lines - are part of the list of commands and which aren't. +- test1608: make it build and get skipped without shuffle DNS support - Closes #11371 +- lib: --disable-bindlocal builds curl without local binding support -Paul Wise (22 Jun 2023) +- test1304: build and skip without netrc support -- checksrc: modernise perl file open +- lib: build fixups when built with most things disabled - Use regular variables and separate file open modes from filenames. + Closes #11687 - Suggested by perlcritic +- workflows/macos.yml: disable zstd and alt-svc in the http-only build - Copied from https://github.com/curl/trurl/commit/f2784a9240f47ee28a845 + Closes #11683 - Closes #11358 +Stefan Eissing (17 Aug 2023) -Dan Fandrich (21 Jun 2023) +- bearssl: handshake fix, provide proper get_select_socks() implementation -- runtests: work around a perl without SIGUSR1 + - bring bearssl handshake times down from +200ms down to other TLS backends + - vtls: improve generic get_select_socks() implementation + - tests: provide Apache with a suitable ssl session cache - At least msys2 perl v5.32.1 doesn't seem to define this signal. Since - this signal is only used for debugging, just ignore if setting it fails. + Closes #11675 - Reported-by: Marcel Raad - Fixes #11350 - Closes #11366 +- tests: TLS session sharing test -- runtests: include missing valgrind package + - test TLS session sharing with special test client + - expect failure with wolfSSL + - disable flaky wolfSSL test_02_07b - use valgrind was missing which caused torture tests with valgrind - enabled to fail. + Closes #11675 - Reported-by: Daniel Stenberg - Fixes #11364 - Closes #11365 +Daniel Stenberg (17 Aug 2023) -- runtests: use more consistent failure lines +- CURLOPT_*TIMEOUT*: extend and clarify - After a test failure log a consistent log message to make it easier to - parse the log file. Also, log a consistent message with "ignored" for - failures that cause the test to be not considered at all. These should - perhaps be counted in the skipped category, but this commit does not - change that behaviour. + Closes #11686 -- runtests: consistently write the test check summary block +- urlapi: return CURLUE_BAD_HOSTNAME if puny2idn encoding fails - The memory check character was erroneously omitted if the memory - checking file was not available for some reason, making the block of - characters an inconsistent length. + And document it. Only return out of memory when it actually is a memory + problem. -- test2600: fix the description + Pointed-out-by: Jacob Mealey + Closes #11674 - It looks like it was cut-and-pasted. +Mathew Benson (17 Aug 2023) - Closes #11354 +- cmake: add GnuTLS option -Daniel Stenberg (21 Jun 2023) + - Option to use GNUTLS was missing. Hence was not able to use GNUTLS + with ngtcp2 for http3. -- TODO: "Support HTTP/2 for HTTP(S) proxies" *done* + Closes #11685 -humbleacolyte (21 Jun 2023) +Daniel Stenberg (16 Aug 2023) -- cf-socket: move ctx declaration under HAVE_GETPEERNAME +- RELEASE-NOTES: synced - Closes #11352 +- http: remove the p_pragma struct field -Daniel Stenberg (20 Jun 2023) + unused since 40e8b4e52 (2008) -- RELEASE-NOTES: synced + Closes #11681 -- example/connect-to: show CURLOPT_CONNECT_TO +Jay Satiro (16 Aug 2023) - Closes #11340 +- CURLINFO_CERTINFO.3: better explain curl_certinfo struct -Stefan Eissing (20 Jun 2023) + Closes https://github.com/curl/curl/pull/11666 -- hyper: unslow +- CURLINFO_TLS_SSL_PTR.3: clarify a recommendation - - refs #11203 where hyper was reported as being slow - - fixes hyper_executor_poll to loop until it is out of - tasks as advised by @seanmonstar in https://github.com/hyperium/hyper/issue - s/3237 - - added a fix in hyper io handling for detecting EAGAIN - - added some debug logs to see IO results - - pytest http/1.1 test cases pass - - pytest h2 test cases fail on connection reuse. HTTP/2 - connection reuse does not seem to work. Hyper submits - a request on a reused connection, curl's IO works and - thereafter hyper declares `Hyper: [1] operation was canceled: connection cl - osed` - on stderr without any error being logged before. + - Remove the out-of-date SSL backend list supported by + CURLOPT_SSL_CTX_FUNCTION. - Fixes #11203 - Reported-by: Gisle Vanem - Advised-by: Sean McArthur - Closes #11344 + It makes more sense to just refer to that document instead of having + a separate list that has to be kept in sync. -- HTTP/2: upload handling fixes + Closes https://github.com/curl/curl/pull/11665 - - fixes #11242 where 100% CPU on uploads was reported - - fixes possible stalls on last part of a request body when - that information could not be fully send on the connection - due to an EAGAIN - - applies the same EGAIN handling to HTTP/2 proxying +- write-out.d: clarify %{time_starttransfer} - Reported-by: Sergey Alirzaev - Fixed #11242 - Closes #11342 + sync it up with CURLINFO_STARTTRANSFER_TIME_T -Daniel Stenberg (20 Jun 2023) +Daniel Stenberg (15 Aug 2023) -- example/opensslthreadlock: remove +- transfer: don't set TIMER_STARTTRANSFER on first send - This shows how to setup OpenSSL mutex callbacks, but this is not - necessary since OpenSSL 1.1.0 - meaning that no currently supported - OpenSSL version requires this anymore + The time stamp is for measuring the first *received* byte - Closes #11341 + Fixes #11669 + Reported-by: JazJas on github + Closes #11670 -Dan Fandrich (19 Jun 2023) +trrui-huawei (15 Aug 2023) -- libtest: display the times after a test timeout error +- quiche: enable quiche to handle timeout events - This is to help with test failure debugging. + In parallel with ngtcp2, quiche also offers the `quiche_conn_on_timeout` + interface for the application to invoke upon timer + expiration. Therefore, invoking the `on_timeout` function of the + Connection is crucial to ensure seamless functionality of quiche with + timeout events. - Ref: #11328 - Closes #11329 + Closes #11654 -- test2600: bump a test timeout +- quiche: adjust quiche `QUIC_IDLE_TIMEOUT` to 60s - Case 1 failed at least once on GHA by going 30 msec too long. + Set the `QUIC_IDLE_TIMEOUT` parameter to match ngtcp2 for consistency. - Ref: #11328 +Daniel Stenberg (15 Aug 2023) -- runtests: better detect and handle pipe errors in the controller +- KNOWN_BUGS: LDAPS requests to ActiveDirectory server hang - Errors reading and writing to the pipes are now better detected and - propagated up to the main test loop so it can be cleanly shut down. Such - errors are usually due to a runner dying so it doesn't make much sense - to try to continue the test run. + Closes #9580 -- runtests: cleanly abort the runner if the controller dies +- imap: add a check for failing strdup() - If the controller dies unexpectedly, have the runner stop its servers - and exit cleanly. Otherwise, the orphaned servers will stay running in - the background. +- imap: remove the only sscanf() call in the IMAP code -- runtests: improve error logging + Avoids the use of a stack buffer. - Give more information about test harness error conditions to help figure - out what might be wrong. Print some internal test state when SIGUSR1 is - sent to runtests.pl. + Closes #11673 - Ref: #11328 +- imap: use a dynbuf in imap_atom -- runtests: better handle ^C during slow tests + Avoid a calculation + malloc. Build the output in a dynbuf. - Since the SIGINT handler now just sets a flag that must be checked in the - main controller loop, make sure that runs periodically. Rather than - blocking on a response from a test runner near the end of the test run, - add a short timeout to allow it. + Closes #11672 -- runtests: rename server command file +Marin Hannache (14 Aug 2023) - The name ftpserver.cmd was historical and has been used for more than - ftp for many years now. Rename it to plain server.cmd to reduce - confusion. +- http: do not require a user name when using CURLAUTH_NEGOTIATE -- tests: improve reliability of TFTP tests + In order to get Negotiate (SPNEGO) authentication to work in HTTP you + used to be required to provide a (fake) user name (this concerned both + curl and the lib) because the code wrongly only considered + authentication if there was a user name provided, as in: - Stop checking the timeout used by the client under test (for most - tests). The timeout will change if the TFTP test server is slow (such as - happens on an overprovisioned CI server) because the client will retry - and reduce its timeout, and the actual value is not important for most - tests. + curl -u : --negotiate https://example.com/ - test285 is changed a different way, by increasing the connect timeout. - This improves test coverage by allowing the changed timeout value to be - checked, but improves reliability with a carefully-chosen timeout that - not only allows twice the time to respond as before, but also allows - several retries before the client will change its timeout value. + This commit leverages the `struct auth` want member to figure out if the + user enabled CURLAUTH_NEGOTIATE, effectively removing the requirement of + setting a user name both in curl and the lib. - Ref: #11328 + Signed-off-by: Marin Hannache + Reported-by: Enrico Scholz + Fixes https://sourceforge.net/p/curl/bugs/440/ + Fixes #1161 + Closes #9047 -Daniel Stenberg (19 Jun 2023) +Viktor Szakats (13 Aug 2023) -- cf-socket: skip getpeername()/getsockname for TFTP +- build: streamline non-UWP wincrypt detections - Since the socket is not connected then the call fails. When the call - fails, failf() is called to write an error message that is then - surviving and is returned when the *real* error occurs later. The - earlier, incorrect, error therefore hides the actual error message. + - with CMake, use the variable `WINDOWS_STORE` to detect an UWP build + and disable our non-UWP-compatible use the Windows crypto API. This + allows to drop two dynamic feature checks. - This could be seen in stderr for test 1007 + `WINDOWS_STORE` is true when invoking CMake with + `CMAKE_SYSTEM_NAME` == `WindowsStore`. Introduced in CMake v3.1. - Test 1007 has now been extended to verify the stderr message. + Ref: https://cmake.org/cmake/help/latest/variable/WINDOWS_STORE.html - Closes #11332 + - with autotools, drop the separate feature check for `wincrypt.h`. On + one hand this header has been present for long (even Borland C 5.5 had + it from year 2000), on the other we used the check result solely to + enable another check for certain crypto functions. This fails anyway + with the header not present. We save one dynamic feature check at the + configure stage. -- example/crawler: make it use a few more options + Reviewed-by: Marcel Raad + Closes #11657 - For show, but reasonable +Nicholas Nethercote (13 Aug 2023) -- libcurl-ws.3: mention raw mode +- docs/HYPER.md: update hyper build instructions - Closes #11339 + Nightly Rust and `-Z unstable-options` are not needed. -- example/default-scheme: set the default scheme for schemeless URLs + The instructions here now match the hyper docs exactly: + https://github.com/hyperium/hyper/commit/bd7928f3dd6a8461f0f0fdf7ee0fd95c2f15 + 6f88 - Closes #11338 + Closes #11662 -- example/hsts-preload: show one way to HSTS preload +Daniel Stenberg (13 Aug 2023) - Closes #11337 +- RELEASE-NOTES: synced -- examples/http-options: show how to send "OPTIONS *" +- urlapi: CURLU_PUNY2IDN - convert from punycode to IDN name - With CURLOPT_REQUEST_TARGET. + Asssisted-by: Jay Satiro + Closes #11655 - Also add use of CURLOPT_QUICK_EXIT to show. +- spellcheck: adapt to backslashed minuses - Closes #11333 + As the curl.1 has more backslashed minus, the cleanup sed lines xneed to + adapt. -- examples: make use of CURLOPT_(REDIR_|)PROTOCOLS_STR + Adjusted some docs slighly. - To show how to use them + Follow-up to 439ff2052e - Closes #11334 + Closes #11663 -- examples/smtp-mime: use CURLOPT_MAIL_RCPT_ALLOWFAILS +- gen: escape more minus - For show + Detected since it was still hard to search for option names using dashes + in the middle in the man page. - Closes #11335 + Closes #11660 -- http: rectify the outgoing Cookie: header field size check +- cookie-jar.d: enphasize that this option is ONLY writing cookies - Previously it would count the size of the entire outgoing request and - not just the size of only the Cookie: header field - which was the - intention. + Reported-by: Dan Jacobson + Tweaked-by: Jay Satiro + Ref: #11642 + Closes #11661 - This could make the check be off by several hundred bytes in some cases. +Nicholas Nethercote (11 Aug 2023) - Closes #11331 +- docs/HYPER.md: document a workaround for a link error -Jay Satiro (17 Jun 2023) + Closes #11653 -- lib: fix some format specifiers +Jay Satiro (11 Aug 2023) - - Use CURL_FORMAT_CURL_OFF_T where %zd was erroneously used for some - curl_off_t variables. +- schannel: verify hostname independent of verify cert - - Use %zu where %zd was erroneously used for some size_t variables. + Prior to this change when CURLOPT_SSL_VERIFYPEER (verifypeer) was off + and CURLOPT_SSL_VERIFYHOST (verifyhost) was on we did not verify the + hostname in schannel code. - Prior to this change some of the Windows CI tests were failing because - in Windows 32-bit targets have a 32-bit size_t and a 64-bit curl_off_t. - When %zd was used for some curl_off_t variables then only the lower - 32-bits was read and the upper 32-bits would be read for part or all of - the next specifier. + This fixes KNOWN_BUG 2.8 "Schannel disable CURLOPT_SSL_VERIFYPEER and + verify hostname". We discussed a fix several years ago in #3285 but it + went stale. - Fixes https://github.com/curl/curl/issues/11327 - Closes https://github.com/curl/curl/pull/11321 + Assisted-by: Daniel Stenberg -Marcel Raad (16 Jun 2023) + Bug: https://curl.haxx.se/mail/lib-2018-10/0113.html + Reported-by: Martin Galvan -- test427: add `cookies` feature and keyword + Ref: https://github.com/curl/curl/pull/3285 - This test doesn't work with `--disable-cookies`. + Fixes https://github.com/curl/curl/issues/3284 + Closes https://github.com/curl/curl/pull/10056 - Closes https://github.com/curl/curl/pull/11320 +Daniel Stenberg (11 Aug 2023) -Chris Talbot (15 Jun 2023) +- curl_quiche: remove superfluous NULL check -- imap: Provide method to disable SASL if it is advertised + 'stream' is always non-NULL at this point - - Implement AUTH=+LOGIN for CURLOPT_LOGIN_OPTIONS to prefer plaintext - LOGIN over SASL auth. + Pointed out by Coverity - Prior to this change there was no method to be able to fall back to - LOGIN if an IMAP server advertises SASL capabilities. However, this may - be desirable for e.g. a misconfigured server. + Closes #11656 - Per: https://www.ietf.org/rfc/rfc5092.html#section-3.2 +- curl/urlapi.h: tiny typo - ";AUTH=" looks to be the correct way to specify what - authenication method to use, regardless of SASL or not. +- github/labeler: make HYPER.md set Hyper and not TLS - Closes https://github.com/curl/curl/pull/10041 +- docs/cmdline-opts/gen.pl: hide "added in" before 7.50.0 -Daniel Stenberg (15 Jun 2023) + 7.50.0 shipped on Jul 21 2016, over seven years ago. We no longer need + to specify version changes for earlier releases in the generated output. -- RELEASE-NOTES: synced + This ups the limit from the previous 7.30.0 (Apr 12 2013) -- examples/multi-debugcallback.c: avoid the bool typedef + This hides roughly 35 "added in" mentions. - Apparently this cannot be done in c23 + Closes #11651 - Reported-by: Cristian Rodríguez - Fixes #11299 - Closes #11319 +Jay Satiro (10 Aug 2023) -- docs/libcurl/libcurl.3: cleanups and improvements +- bug_report: require reporters to specify curl and os versions - Closes #11317 + - Change curl version and os sections from single-line input to + multi-line textarea. -- libcurl-ws.3: fix typo + - Require curl version and os sections to be filled out before report + can be submitted. -- curl_ws_*.3: enhance + Closes https://github.com/curl/curl/pull/11636 - - all: SEE ALSO the libcurl-ws man page - - send: add example and return value information - - meta: mention that the returned data is read-only +Daniel Stenberg (9 Aug 2023) - Closes #11318 +- gen.pl: replace all single quotes with aq -- docs/libcurl/libcurl-ws.3: see also CURLOPT_WS_OPTIONS + - this prevents man from using a unicode sequence for them + - which then allows search to work properly -- docs/libcurl/libcurl-ws.3: minor polish + Closes #11645 -- libcurl-ws.3. WebSocket API overview +Viktor Szakats (9 Aug 2023) - Closes #11314 +- cmake: fix to use variable for the curl namespace -- libcurl-url.3: also mention CURLUPART_ZONEID + Replace (wrong) literal with a variable to specify the curl + namespace. - ... and sort the two part-using lists alphabetically + Follow-up to 1199308dbc902c52be67fc805c72dd2582520d30 #11505 -Marcel Raad (14 Jun 2023) + Reported-by: balikalina on Github + Fixes https://github.com/curl/curl/commit/1199308dbc902c52be67fc805c72dd25825 + 20d30#r123923098 + Closes #11629 -- fopen: fix conversion warning on 32-bit Android +- cmake: allow `SHARE_LIB_OBJECT=ON` on all platforms - When building for 32-bit ARM or x86 Android, `st_mode` is defined as - `unsigned int` instead of `mode_t`, resulting in a - -Wimplicit-int-conversion clang warning because `mode_t` is - `unsigned short`. Add a cast to silence the warning. + 2ebc74c36a19a1700af394c16855ce144d9878e3 #11546 introduced sharing + libcurl objects for shared and static targets. - Ref: https://android.googlesource.com/platform/bionic/+/refs/tags/ndk-r25c/li - bc/include/sys/stat.h#86 - Closes https://github.com/curl/curl/pull/11313 + The above automatically enabled for Windows builds, with an option to + disable with `SHARE_LIB_OBJECT=OFF`. -- http2: fix variable type + This patch extend this feature to all platforms as a manual option. + You can enable it by setting `SHARE_LIB_OBJECT=ON`. Then shared objects + are built in PIC mode, meaning the static lib will also have PIC code. - `max_recv_speed` is `curl_off_t`, so using `size_t` might result in - -Wconversion GCC warnings for 32-bit `size_t`. Visible in the NetBSD - ARM autobuilds. + [EXPERIMENTAL] - Closes https://github.com/curl/curl/pull/11312 + Closes #11627 -Daniel Stenberg (13 Jun 2023) +- cmake: assume `wldap32` availability on Windows -- vtls: fix potentially uninitialized local variable warnings + This system library first shipped with Windows ME, available as an extra + install for some older releases (according to [1]). The import library + was present already in old MinGW 3.4.2 (year 2007). - Follow-up from a4a5e438ae533c + Drop the feature check and its associated `HAVE_WLDAP32` variable. - Closes #11310 + To manually disable `wldap32`, you can use the `USE_WIN32_LDAP=OFF` + CMake option, like before. -- timeval: use CLOCK_MONOTONIC_RAW if available + [1]: https://dlcdn.apache.org/httpd/binaries/win32/LEGACY.html - Reported-by: Harry Sintonen - Ref: #11288 - Closes #11291 + Reviewed-by: Jay Satiro + Closes #11624 -Stefan Eissing (12 Jun 2023) +Daniel Stenberg (9 Aug 2023) -- tool: add curl command line option `--trace-ids` +- page-header: move up a URL paragraph from GLOBBING to URL - - added and documented --trace-ids to prepend (after the timestamp) - the transfer and connection identifiers to each verbose log line - - format is [n-m] with `n` being the transfer id and `m` being the - connection id. In case there is not valid connection id, print 'x'. - - Log calls with a handle that has no transfer id yet, are written - without any ids. +- variable.d: output the function names table style - Closes #11185 + Also correct the url function name in the header -- lib: add CURLINFO_CONN_ID and CURLINFO_XFER_ID + Closes #11641 - - add an `id` long to Curl_easy, -1 on init - - once added to a multi (or its own multi), it gets - a non-negative number assigned by the connection cache - - `id` is unique among all transfers using the same - cache until reaching LONG_MAX where it will wrap - around. So, not unique eternally. - - CURLINFO_CONN_ID returns the connection id attached to - data or, if none present, data->state.lastconnect_id - - variables and type declared in tool for write out +- haproxy-clientip.d: remove backticks - Closes #11185 + This is not markdown -Daniel Stenberg (12 Jun 2023) + Follow-up to 0a75964d0d94a4 -- CURLOPT_INFILESIZE.3: mention -1 triggers chunked + Closes #11639 - Ref: #11300 - Closes #11304 +- RELEASE-NOTES: synced -Philip Heiduck (12 Jun 2023) +- gen.pl: escape all dashes (ascii minus) to avoid unicode hyphens -- CI: openssl-3.0.9+quic + Reported-by: FC Stegerman + Fixes #11635 + Closes #11637 - Closes #11296 +- cmdline-opts/page-header: reorder, clean up -Karthikdasari0423 (12 Jun 2023) + - removed some unnecessary blurb to focus + - moved up the more important URL details + - put "globbing" into its own subtitle and moved down a little + - mention the online man page in the version section -- HTTP3.md: update openssl version + Closes #11638 - Closes #11297 +- c-hyper: adjust the hyper to curlcode conversion -Daniel Stenberg (12 Jun 2023) + Closes #11621 -- vtls: avoid memory leak if sha256 call fails +- test2306: make it use a persistent connection - ... in the pinned public key handling function. + + enable verbose already from the start - Reported-by: lizhuang0630 on github - Fixes #11306 - Closes #11307 + Closes #11621 -- examples/ipv6: disable on win32 +eppesuig (8 Aug 2023) - I can't make if_nametoindex() work there +- list-only.d: mention SFTP as supported protocol - Follow-up to c23dc42f3997acf23 + Closes #11628 - Closes #11305 +Daniel Stenberg (8 Aug 2023) -- tool_operate: allow cookie lines up to 8200 bytes +- request.d: use .TP for protocol "labels" - Since this option might set multiple cookies in the same line, it does - not make total sense to cap this at 4096 bytes, which is the limit for a - single cookie name or value. + To render the section nicer in man page. - Closes #11303 + Closes #11630 -- test427: verify sending more cookies than fit in a 8190 bytes line +- cf-haproxy: make CURLOPT_HAPROXY_CLIENT_IP set the *source* IP - curl will then only populate the header with cookies that fit, dropping - ones that otherwise would have been sent + ... as documented. - Ref: https://curl.se/mail/lib-2023-06/0020.html + Update test 3201 and 3202 accordingly. - Closes #11303 + Reported-by: Markus Sommer + Fixes #11619 + Closes #11626 -- testutil: allow multiple %-operators on the same line +- page-footer: QLOGDIR works with ngtcp2 and quiche - Closes #11303 + It previously said "both" backends which is confusing as we currently + have three... -Oleg Jukovec (12 Jun 2023) + Closes #11631 -- docs: update CURLOPT_UPLOAD.3 +Stefan Eissing (8 Aug 2023) - The behavior of CURLOPT_UPLOAD differs from what is described in the - documentation. The option automatically adds the 'Transfer-Encoding: - chunked' header if the upload size is unknown. +- http3: quiche, handshake optimization, trace cleanup - Closes #11300 + - load x509 store after clienthello + - cleanup of tracing -Daniel Stenberg (12 Jun 2023) + Closes #11618 -- RELEASE-NOTES: synced +Daniel Stenberg (8 Aug 2023) -- CURLOPT_AWS_SIGV4.3: remove unused variable from example +- ngtcp2: remove dead code - Closes #11302 + 'result' is always zero (CURLE_OK) at this point -- examples/https.c: use CURLOPT_CA_CACHE_TIMEOUT + Detected by Coverity - for demonstration purposes + Closes #11622 - Closes #11290 +Viktor Szakats (8 Aug 2023) -- example/ipv6: feature CURLOPT_ADDRESS_SCOPE in use +- openssl: auto-detect `SSL_R_TLSV13_ALERT_CERTIFICATE_REQUIRED` - Closes #11282 + OpenSSL 1.1.1 defines this macro, but no ealier version, or any of the + popular forks (yet). Use the macro itself to detect its presence, + replacing the hard-wired fork-specific conditions. -Karthikdasari0423 (10 Jun 2023) + This way the feature will enable automatically when forks implement it, + while also shorter and possibly requiring less future maintenance. -- docs: Update HTTP3.md for newer ngtcp2 and nghttp3 + Follow-up to 94241a9e78397a2aaf89a213e6ada61e7de7ee02 #6721 - Follow-up to fb9b9b58 + Reviewed-by: Jay Satiro + Closes #11617 - Ref: #11184 - Closes #11295 +- openssl: use `SSL_CTX_set_ciphersuites` with LibreSSL 3.4.1 -Dan Fandrich (10 Jun 2023) + LibreSSL 3.4.1 (2021-10-14) added support for + `SSL_CTX_set_ciphersuites`. -- docs: update the supported ngtcp2 and nghttp3 versions + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1-relnotes.txt - Follow-up to cae9d10b + Reviewed-by: Jay Satiro + Closes #11616 - Ref: #11184 - Closes #11294 +- openssl: use `SSL_CTX_set_keylog_callback` with LibreSSL 3.5.0 -- tests: fix error messages & handling around sockets + LibreSSL 3.5.0 (2022-02-24) added support for + `SSL_CTX_set_keylog_callback`. - The wrong error code was checked on Windows on UNIX socket failures, - which could have caused all UNIX sockets to be reported as having - errored and the tests therefore skipped. Also, a useless error message - was displayed on socket errors in many test servers on Windows because - strerror() doesn't work on WinSock error codes; perror() is overridden - there to work on all errors and is used instead. + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.5.0-relnotes.txt - Ref #11258 - Closes #11265 + Reviewed-by: Jay Satiro + Closes #11615 -Daniel Stenberg (9 Jun 2023) +- cmake: drop `HAVE_LIBWINMM` and `HAVE_LIBWS2_32` feature checks -- CURLOPT_SSH_PRIVATE_KEYFILE.3: expand on the file search + - `HAVE_LIBWINMM` was detected but unused. The `winmm` system library is + also not used by curl, but it is by its optional dependency `librtmp`. + Change the logic to always add `winmm` when `USE_LIBRTMP` is set. This + library has been available since the early days of Windows. - Reported-by: atjg on github - Ref: #11287 - Closes #11289 + - `HAVE_LIBWS2_32` detected `ws2_32` lib on Windows. This lib is present + since Windows 95 OSR2 (AFAIR). Winsock1 already wasn't supported and + other existing logic already assumed this lib being present, so delete + the check and replace the detection variable with `WIN32` and always + add `ws2_32` on Windows. -Stefan Eissing (9 Jun 2023) + Closes #11612 -- ngtcp2: use ever increasing timestamp in io +Daniel Gustafsson (8 Aug 2023) - - ngtcp2 v0.16.0 asserts that timestamps passed to its function - will only ever increase. - - Use a context shared between ingress/egress operations that - uses a shared timestamp, regularly updated during calls. +- crypto: ensure crypto initialization works - Closes #11288 + Make sure that context initialization during hash setup works to avoid + going forward with the risk of a null pointer dereference. -Daniel Stenberg (9 Jun 2023) + Reported-by: Philippe Antoine on HackerOne + Assisted-by: Jay Satiro + Assisted-by: Daniel Stenberg -- GHA: use nghttp2 1.54.0 for the ngtcp2 jobs + Closes #11614 -Philip Heiduck (9 Jun 2023) +Viktor Szakats (7 Aug 2023) -- GHA: ngtcp2: use 0.16.0 and nghttp3 0.12.0 +- openssl: switch to modern init for LibreSSL 2.7.0+ -Daniel Stenberg (9 Jun 2023) + LibreSSL 2.7.0 (2018-03-21) introduced automatic initialization, + `OPENSSL_init_ssl()` function and deprecated the old, manual init + method, as seen in OpenSSL 1.1.0. Switch to the modern method when + available. -- ngtcp2: build with 0.16.0 and nghttp3 0.12.0 + Ref: https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-2.7.0-relnotes.txt - - moved to qlog_write - - crypto => encryption - - CRYPTO => ENCRYPTION - - removed "_is_" - - ngtcp2_conn_shutdown_stream_read and - ngtcp2_conn_shutdown_stream_write got flag arguments - - the nghttp3_callbacks struct got a recv_settings callback + Reviewed-by: Daniel Stenberg + Closes #11611 - Closes #11184 +Daniel Stenberg (7 Aug 2023) -- example/http2-download: set CURLOPT_BUFFERSIZE +- gskit: remove - Primarily because no other example sets it, and remove the disabling of - the certificate check because we should not recommend that. + We remove support for building curl with gskit. - Closes #11284 + - This is a niche TLS library, only running on some IBM systems + - no regular curl contributors use this backend + - no CI builds use or verify this backend + - gskit, or the curl adaption for it, lacks many modern TLS features + making it an inferior solution + - build breakages in this code take weeks or more to get detected + - fixing gskit code is mostly done "flying blind" -- example/crawler: also set CURLOPT_AUTOREFERER + This removal has been advertized in DEPRECATED in Jan 2, 2023 and it has + been mentioned on the curl-library mailing list. - Could make sense, and it was not used in any example before. + It could be brought back, this is not a ban. Given proper effort and + will, gskit support is welcome back into the curl TLS backend family. - Closes #11283 + Closes #11460 -Wyatt OʼDay (9 Jun 2023) +- RELEASE-NOTES: synced -- tls13-ciphers.d: include Schannel +Dan Fandrich (7 Aug 2023) - Closes #11271 +- THANKS-filter: add a name typo -Daniel Stenberg (9 Jun 2023) +Stefan Eissing (7 Aug 2023) -- curl_pushheader_byname/bynum.3: document in their own man pages +- http3/ngtcp2: shorten handshake, trace cleanup - These two functions were added in 7.44.0 when CURLMOPT_PUSHFUNCTION was - introduced but always lived a life in the shadows, embedded in the - CURLMOPT_PUSHFUNCTION man page. Until now. + - shorten handshake timing by delayed x509 store load (OpenSSL) + as we do for HTTP/2 + - cleanup of trace output, align with HTTP/2 output - It makes better sense and gives more visibility to document them in - their own stand-alone man pages. + Closes #11609 - Closes #11286 +Daniel Stenberg (7 Aug 2023) -- curl_mprintf.3: minor fix of the example +- headers: accept leading whitespaces on first response header -- curl_url_set: enforce the max string length check for all parts + This is a bad header fold but since the popular browsers accept this + violation, so does curl now. Unless built with hyper. - Update the docs and test 1559 accordingly + Add test 1473 to verify and adjust test 2306. - Closes #11273 + Reported-by: junsik on github + Fixes #11605 + Closes #11607 -- examples/ftpuploadresume.c: add use of CURLOPT_ACCEPTTIMEOUT_MS +- include/curl/mprintf.h: add __attribute__ for the prototypes - For show + - if gcc or clang is used + - if __STDC_VERSION__ >= 199901L, which means greater than C90 + - if not using mingw + - if CURL_NO_FMT_CHECKS is not defined - Closes #11277 + Closes #11589 -- examples/unixsocket.c: example using CURLOPT_UNIX_SOCKET_PATH +- tests: fix bad printf format flags in test code - and alternatively CURLOPT_ABSTRACT_UNIX_SOCKET +- tests: fix header scan tools for attribute edits in mprintf.h - Closes #11276 +- cf-socket: log successful interface bind -Anssi Kolehmainen (8 Jun 2023) + When the setsockopt SO_BINDTODEVICE operation succeeds, output that in + the verbose output. -- docs: fix missing parameter names in examples + Ref: #11599 + Closes #11608 - Closes #11278 +- CURLOPT_SSL_VERIFYPEER.3: mention it does not load CA certs when disabled -Daniel Stenberg (8 Jun 2023) + Ref: #11457 + Closes #11606 -- urlapi: have *set(PATH) prepend a slash if one is missing +- CURLOPT_SSL_VERIFYPEER.3: add two more see also options - Previously the code would just do that for the path when extracting the - full URL, which made a subsequent curl_url_get() of the path to - (unexpectedly) still return it without the leading path. + CURLINFO_CAINFO and CURLINFO_CAPATH - Amend lib1560 to verify this. Clarify the curl_url_set() docs about it. + Closes #11603 - Bug: https://curl.se/mail/lib-2023-06/0015.html - Closes #11272 - Reported-by: Pedro Henrique +- KNOWN_BUGS: aws-sigv4 does not behave well with AWS VPC Lattice -Dan Fandrich (7 Jun 2023) + Closes #11007 -- runtests; give each server a unique log lock file +Graham Campbell (6 Aug 2023) - Logs are written by several servers and all of them must be finished - writing before the test results can be determined. This means each - server must have its own lock file rather than sharing a single one, - which is how it was done up to now. Previously, the first server to - complete a test would clear the lock before the other server was done, - which caused flaky tests. +- CI: use openssl 3.0.10+quic, nghttp3 0.14.0, ngtcp2 0.18.0 - Lock files are now all found in their own directory, so counting locks - equals counting the files in that directory. The result is that the - proxy logs are now reliably written which actually changes the expected - output for two tests. + Closes #11585 - Fixes #11231 - Closes #11259 +Daniel Stenberg (6 Aug 2023) -- runtests: make test file directories in log/N +- TODO: add *5* entries for aws-sigv4 - Test files in subdirectories were not created after parallel test log - directories were moved down a level due to a now-bad comparison. + Closes #7559 + Closes #8107 + Closes #8810 + Closes #9717 + Closes #10129 - Follow-up to 92d7dd39 +- TODO: LDAP Certificate-Based Authentication - Ref #11264 - Closes #11267 + Closes #9641 -Daniel Stenberg (7 Jun 2023) +Stefan Eissing (6 Aug 2023) -- ws: make the curl_ws_meta() return pointer a const +- http2: cleanup trace messages - The returned info is read-only for the user. + - more compact format with bracketed stream id + - all frames traced in and out - Closes #11261 + Closes #11592 -- RELEASE-NOTES: synced +Daniel Stenberg (6 Aug 2023) -- runtests: move parallel log dirs from logN to log/N +- tests/tftpd+mqttd: make variables static to silence picky warnings - Having several hundreds of them in there gets annoying. + Closes #11594 - Closes #11264 +- docs/cmdline: remove repeated working for negotiate + ntlm -Dan Fandrich (7 Jun 2023) + The extra wording is added automatically by the gen.pl tool -- test447: move the test file into %LOGDIR + Closes #11597 -Viktor Szakats (7 Jun 2023) +- docs/cmdline: add small "warning" to verbose options -- cmake: add support for "unity" builds + "Note that verbose output of curl activities and network traffic might + contain sensitive data, including user names, credentials or secret data + content. Be aware and be careful when sharing trace logs with others." - Aka "jumbo" or "amalgamation" builds. It means to compile all sources - per target as a single C source. This is experimental. + Closes #11596 - You can enable it by passing `-DCMAKE_UNITY_BUILD=ON` to cmake. - It requires CMake 3.16 or newer. +- RELEASE-NOTES: synced - It makes builds (much) faster, allows for better optimizations and tends - to promote less ambiguous code. +- pingpong: don't use *bump_headersize - Also add a new AppVeyor CI job and convert an existing one to use - "unity" mode (one MSVC, one MinGW), and enable it for one macOS CI job. + We use that for HTTP(S) only. - Fix related issues: - - add missing include guard to `easy_lock.h`. - - rename static variables and functions (and a macro) with names reused - across sources, or shadowed by local variables. - - add an `#undef` after use. - - add a missing `#undef` before use. - - move internal definitions from `ftp.h` to `ftp.c`. - - `curl_memory.h` fixes to make it work when included repeatedly. - - stop building/linking curlx bits twice for a static-mode curl tool. - These caused doubly defined symbols in unity builds. - - silence missing extern declarations compiler warning for ` _CRT_glob`. - - fix extern declarations for `tool_freq` and `tool_isVistaOrGreater`. - - fix colliding static symbols in debug mode: `debugtime()` and - `statename`. - - rename `ssl_backend_data` structure to unique names for each - TLS-backend, along with the `ssl_connect_data` struct member - referencing them. This required adding casts for each access. - - add workaround for missing `[P]UNICODE_STRING` types in certain Windows - builds when compiling `lib/ldap.c`. To support "unity" builds, we had - to enable `SCHANNEL_USE_BLACKLISTS` for Schannel (a Windows - `schannel.h` option) _globally_. This caused an indirect inclusion of - Windows `schannel.h` from `ldap.c` via `winldap.h` to have it enabled - as well. This requires `[P]UNICODE_STRING` types, which is apperantly - not defined automatically (as seen with both MSVS and mingw-w64). - This patch includes `` to fix it. - Ref: https://github.com/curl/curl/runs/13987772013 - Ref: https://dev.azure.com/daniel0244/curl/_build/results?buildId=15827&vie - w=logs&jobId=2c9f582d-e278-56b6-4354-f38a4d851906&j=2c9f582d-e278-56b6-4354-f - 38a4d851906&t=90509b00-34fa-5a81-35d7-5ed9569d331c - - tweak unity builds to compile `lib/memdebug.c` separately in memory - trace builds to avoid PP confusion. - - force-disable unity for test programs. - - do not compile and link libcurl sources to libtests _twice_ when libcurl - is built in static mode. + Follow-up to 3ee79c1674fd6 - KNOWN ISSUES: - - running tests with unity builds may fail in cases. - - some build configurations/env may not compile in unity mode. E.g.: - https://ci.appveyor.com/project/curlorg/curl/builds/47230972/job/51wfesgnfu - auwl8q#L250 + Closes #11590 - Ref: https://github.com/libssh2/libssh2/issues/1034 - Ref: https://cmake.org/cmake/help/latest/prop_tgt/UNITY_BUILD.html - Ref: https://en.wikipedia.org/wiki/Unity_build +- urldata: remove spurious parenthesis to unbreak no-proxy build - Closes #11095 + Follow-up to e12b39e13382 -Daniel Stenberg (7 Jun 2023) + Closes #11591 -- examples/websocket.c: websocket example using CONNECT_ONLY +- easy: don't call Curl_trc_opt() in disabled-verbose builds - Closes #11262 + Follow-up to e12b39e133822c6a0 -- websocket-cb: example doing WebSocket download using callback + Closes #11588 - Very basic +- http: use %u for printfing int - Closes #11260 + Follow-up to 3ee79c1674fd6f99e8efca5 -- test/.gitignore: ignore log* + Closes #11587 -Dan Fandrich (5 Jun 2023) +Goro FUJI (3 Aug 2023) -- runtests: document the -j parallel testing option +- vquic: show stringified messages for errno - Reported-by: Daniel Stenberg - Ref: #10818 - Closes #11255 + Closes #11584 -- runtests: create multiple test runners when requested +Stefan Eissing (3 Aug 2023) - Parallel testing is enabled by using a nonzero value for the -j option - to runtests.pl. Performant values seem to be about 7*num CPU cores, or - 1.3*num CPU cores if Valgrind is in use. +- trace: make tracing available in non-debug builds - Flaky tests due to improper log locking (bug #11231) are exacerbated - while parallel testing, so it is not enabled by default yet. + Add --trace-config to curl - Fixes #10818 - Closes #11246 + Add curl_global_trace() to libcurl -- runtests: handle repeating tests in multiprocess mode + Closes #11421 - Such as what happens with the --repeat option. Some functions are - changed to pass the runner ID instead of relying on the non-unique test - number. +Daniel Stenberg (3 Aug 2023) - Ref: #10818 +- TODO: remove "Support intermediate & root pinning for PINNEDPUBLICKEY" -- runtests: buffer logmsg while running singletest() + See also https://github.com/curl/curl/pull/7507 - This allows all messages relating to a single test case to be displayed - together at the end of the test. +- TODO: add "WebSocket read callback" - Ref: #10818 + remove "Upgrade to websockets" as we already have this -- runtests: call initserverconfig() in the runner + Closes #11402 - This must be done so variables pick up the runner's unique $LOGDIR. +- test497: verify rejecting too large incoming headers - Ref: #10818 +- http: return error when receiving too large header set -- runtests: use a per-runner random seed + To avoid abuse. The limit is set to 300 KB for the accumulated size of + all received HTTP headers for a single response. Incomplete research + suggests that Chrome uses a 256-300 KB limit, while Firefox allows up to + 1MB. - Each runner needs a unique random seed to reduce the chance of port - number collisions. The new scheme uses a consistent per-runner source of - randomness which results in deterministic behaviour, as it did before. + Closes #11582 - Ref: #10818 +Stefan Eissing (3 Aug 2023) -- runtests: complete main test loop refactor for multiple runners +- http2: upgrade tests and add fix for non-existing stream - The main test loop is now able to handle multiple runners, or no - additional runner processes at all. At most one process is still - created, however. + - check in h2 filter recv that stream actually exists + and return error if not + - add test for parallel, extreme h2 upgrades that fail if + connections get reused before fully switched + - add h2 upgrade upload test just for completeness - Ref: #10818 + Closes #11563 -- runtests: prepare main test loop for multiple runners +Viktor Szakats (3 Aug 2023) - Some variables are expanded to arrays and hashes so that multiple - runners can be used for running tests. +- tests: ensure `libcurl.def` contains all exports - Ref: #10818 + Add `test1279` to verify that `libcurl.def` lists all exported API + functions found in libcurl headers. -Stefan Eissing (5 Jun 2023) + Also: -- bufq: make write/pass methods more robust + - extend test suite XML `stdout` tag with the `loadfile` attribute. - - related to #11242 where curl enters busy loop when - sending http2 data to the server + - fix `tests/extern-scan.pl` and `test1135` to include websocket API. - Closes #11247 + - use all headers (sorted) in `test1135` instead of a manual list. -Boris Verkhovskiy (5 Jun 2023) + - add options `--sort`, `--heading=` to `tests/extern-scan.pl`. -- tool_getparam: fix comment + - add `libcurl.def` to the auto-labeler GHA task. - Closes #11253 + Follow-up to 2ebc74c36a19a1700af394c16855ce144d9878e3 -Raito Bezarius (5 Jun 2023) + Closes #11570 -- haproxy: add --haproxy-clientip flag to spoof client IPs +Daniel Stenberg (2 Aug 2023) - CURLOPT_HAPROXY_CLIENT_IP in the library +- url: change default value for CURLOPT_MAXREDIRS to 30 - Closes #10779 + It was previously unlimited by default, but that's not a sensible + default. While changing this has a remote risk of breaking an existing + use case, I figure it is more likely to actually save users from loops. -Daniel Stenberg (5 Jun 2023) + Closes #11581 -- curl: add --ca-native and --proxy-ca-native +- lib: fix a few *printf() flag mistakes - These are two boolean options to ask curl to use the native OS's CA - store when verifying TLS servers. For peers and for proxies - respectively. + Reported-by: Gisle Vanem + Ref: #11574 + Closes #11579 - They currently only have an effect for curl on Windows when built to use - OpenSSL for TLS. +Samuel Chiang (2 Aug 2023) - Closes #11049 +- openssl: make aws-lc version support OCSP -Viktor Szakats (5 Jun 2023) + And bump version in CI -- build: drop unused/redundant `HAVE_WINLDAP_H` + Closes #11568 - Sources did not use it. Autotools used it when checking for the - `winldap` library, which is redundant. +Daniel Stenberg (2 Aug 2023) - With CMake, detection was broken: - ``` - Run Build Command(s):/usr/local/Cellar/cmake/3.26.3/bin/cmake -E env VERBOSE= - 1 /usr/bin/make -f Makefile cmTC_2d8fe/fast && /Library/Developer/CommandLine - Tools/usr/bin/make -f CMakeFiles/cmTC_2d8fe.dir/build.make CMakeFiles/cmTC_2 - d8fe.dir/build - Building C object CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj - /usr/local/opt/llvm/bin/clang --target=x86_64-w64-mingw32 --sysroot=/usr/loca - l/opt/mingw-w64/toolchain-x86_64 -D_WINSOCKAPI_="" -I/my/quictls/x64-ucrt/usr - /include -I/my/zlib/x64-ucrt/usr/include -I/my/brotli/x64-ucrt/usr/include -W - no-unused-command-line-argument -D_UCRT -DCURL_HIDDEN_SYMBOLS -DHAVE_SSL_SE - T0_WBIO -DHAS_ALPN -DNGHTTP2_STATICLIB -DNGHTTP3_STATICLIB -DNGTCP2_STATICLIB - -DUSE_MANUAL=1 -fuse-ld=lld -Wl,-s -static-libgcc -lucrt -Wextra -Wall -p - edantic -Wbad-function-cast -Wconversion -Winline -Wmissing-declarations -Wmi - ssing-prototypes -Wnested-externs -Wno-long-long -Wno-multichar -Wpointer-ari - th -Wshadow -Wsign-compare -Wundef -Wunused -Wwrite-strings -Wcast-align -Wde - claration-after-statement -Wempty-body -Wendif-labels -Wfloat-equal -Wignored - -qualifiers -Wno-format-nonliteral -Wno-sign-conversion -Wno-system-headers - - Wstrict-prototypes -Wtype-limits -Wvla -Wshift-sign-overflow -Wshorten-64-to- - 32 -Wdouble-promotion -Wenum-conversion -Wunused-const-variable -Wcomma -Wmis - sing-variable-declarations -Wassign-enum -Wextra-semi-stmt -MD -MT CMakeFile - s/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -MF CMakeFiles/cmTC_2d8fe.dir/HAVE_WINL - DAP_H.c.obj.d -o CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj -c /my/curl/b - ld-cmake-llvm-x64-shared/CMakeFiles/CMakeScratch/TryCompile-3JP6dR/HAVE_WINLD - AP_H.c - In file included from /my/curl/bld-cmake-llvm-x64-shared/CMakeFiles/CMakeScra - tch/TryCompile-3JP6dR/HAVE_WINLDAP_H.c:2: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/winldap.h:17: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/schnlsp.h:9: - In file included from /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mi - ngw32/include/schannel.h:10: - /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt - .h:5041:254: error: unknown type name 'PSYSTEMTIME' - WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ - NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, - DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER - pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT - ENSIONS pExtensions); - - - - ^ - /usr/local/opt/mingw-w64/toolchain-x86_64/x86_64-w64-mingw32/include/wincrypt - .h:5041:278: error: unknown type name 'PSYSTEMTIME' - WINIMPM PCCERT_CONTEXT WINAPI CertCreateSelfSignCertificate (HCRYPTPROV_OR_ - NCRYPT_KEY_HANDLE hCryptProvOrNCryptKey, PCERT_NAME_BLOB pSubjectIssuerBlob, - DWORD dwFlags, PCRYPT_KEY_PROV_INFO pKeyProvInfo, PCRYPT_ALGORITHM_IDENTIFIER - pSignatureAlgorithm, PSYSTEMTIME pStartTime, PSYSTEMTIME pEndTime, PCERT_EXT - ENSIONS pExtensions); - - - - ^ - 2 errors generated. - make[1]: *** [CMakeFiles/cmTC_2d8fe.dir/HAVE_WINLDAP_H.c.obj] Error 1 - make: *** [cmTC_2d8fe/fast] Error 2 - exitCode: 2 - ``` +- tool: make the length argument an int for printf()-.* flags - Cherry-picked from #11095 88e4a21ff70ccef391cf99c8165281ff81374503 - Reviewed-by: Daniel Stenberg - Closes #11245 + Closes #11578 -Daniel Stenberg (5 Jun 2023) +- tool_operate: fix memory leak when SSL_CERT_DIR is used -- urlapi: scheme starts with alpha + Detected by Coverity - Add multiple tests to lib1560 to verify + Follow-up to 29bce9857a12b6cfa726a5 - Fixes #11249 - Reported-by: ad0p on github - Closes #11250 + Closes #11577 -- RELEASE-NOTES: synced +- tool/var: free memory on OOM -- CURLOPT_MAIL_RCPT_ALLOWFAILS: replace CURLOPT_MAIL_RCPT_ALLLOWFAILS + Coverity detected this memory leak in OOM situation - Deprecate the name using three Ls and prefer the name with two. + Follow-up to 2e160c9c652504e - Replaces #10047 - Closes #11218 + Closes #11575 -- tests/servers: generate temp names in /tmp for unix domain sockets +Viktor Szakats (2 Aug 2023) - ... instead of putting them in the regular pid directories because - systems generally have strict length requirements for the path name to - be shorter than 107 bytes and we easily hit that boundary otherwise. +- gha: bump libressl and mbedtls versions - The new concept generates two random names: one for the socks daemon and - one for http. + Closes #11573 - Reported-by: Andy Fiddaman - Fixes #11152 - Closes #11166 +Jay Satiro (2 Aug 2023) -Stefan Eissing (2 Jun 2023) +- schannel: fix user-set legacy algorithms in Windows 10 & 11 -- http2: better support for --limit-rate + - If the user set a legacy algorithm list (CURLOPT_SSL_CIPHER_LIST) then + use the SCHANNEL_CRED legacy structure to pass the list to Schannel. - - leave transfer loop when --limit-rate is in effect and has - been received - - adjust stream window size to --limit-rate plus some slack - to make the server observe the pacing we want - - add test case to confirm behaviour + - If the user set both a legacy algorithm list and a TLS 1.3 cipher list + then abort. - Closes #11115 + Although MS doesn't document it, Schannel will not negotiate TLS 1.3 + when SCHANNEL_CRED is used. That means setting a legacy algorithm list + limits the user to earlier versions of TLS. -- curl_log: evaluate log statement only when transfer is verbose + Prior to this change, since 8beff435 (precedes 7.85.0), libcurl would + ignore legacy algorithms in Windows 10 1809 and later. - Closes #11238 + Reported-by: zhihaoy@users.noreply.github.com -Daniel Stenberg (2 Jun 2023) + Fixes https://github.com/curl/curl/pull/10741 + Closes https://github.com/curl/curl/pull/10746 -- libssh2: provide error message when setting host key type fails +Daniel Stenberg (2 Aug 2023) - Ref: https://curl.se/mail/archive-2023-06/0001.html +- variable.d: setting a variable again overwrites it - Closes #11240 + Reported-by: Niall McGee + Bug: https://twitter.com/niallmcgee/status/1686523075423322113 + Closes #11571 -Igor Todorovski (2 Jun 2023) +Jay Satiro (2 Aug 2023) -- system.h: remove __IBMC__/__IBMCPP__ guards and apply to all z/OS compiles +- CURLOPT_PROXY_SSL_OPTIONS.3: sync formatting - Closes #11241 + - Re-wrap CURLSSLOPT_ALLOW_BEAST description. -Daniel Stenberg (2 Jun 2023) +Daniel Stenberg (2 Aug 2023) -- docs/SECURITY-PROCESS.md: link to example of previous critical flaw +- RELEASE-NOTES: synced -Mark Seuffert (2 Jun 2023) +- resolve: use PF_INET6 family lookups when CURL_IPRESOLVE_V6 is set -- README.md: updated link to opencollective + Previously it would always do PF_UNSPEC if CURL_IPRESOLVE_V4 is not + used, thus unnecessarily asking for addresses that will not be used. - Closes #11232 + Reported-by: Joseph Tharayil + Fixes #11564 + Closes #11565 -Daniel Stenberg (1 Jun 2023) +- docs: link to the website versions instead of markdowns -- libssh2: use custom memory functions + ... to make the links work when the markdown is converted to webpages on + https://curl.se - Because of how libssh2_userauth_keyboard_interactive_ex() works: the - libcurl callback allocates memory that is later free()d by libssh2, we - must set the custom memory functions. + Reported-by: Maurício Meneghini Fauth + Fixes https://github.com/curl/curl-www/issues/272 + Closes #11569 - Reverts 8b5f100db388ee60118c08aa28 +Viktor Szakats (1 Aug 2023) - Ref: https://github.com/libssh2/libssh2/issues/1078 - Closes #11235 +- cmake: cache more config and delete unused ones -- test447: test PUTting a file that grows + - cache more Windows config results for faster initialization. - ... and have curl trim the end when it reaches the expected total amount - of bytes instead of over-sending. + - delete unused config macros `HAVE_SYS_UTSNAME_H`, `HAVE_SSL_H`. - Reported-by: JustAnotherArchivist on github - Closes #11223 + - delete dead references to `sys/utsname.h`. -- curl: count uploaded data to stop at the originally given size + Closes #11551 - Closes #11223 - Fixes #11222 - Reported-by: JustAnotherArchivist on github +- egd: delete feature detection and related source code -- tool: remove exclamation marks from error/warning messages + EGD is Entropy Gathering Daemon, a socket-based entropy source supported + by pre-OpenSSL v1.1 versions and now deprecated. curl also deprecated it + a while ago. -- tool: use errorf() for error output + Its detection in CMake was broken all along because OpenSSL libs were + not linked at the point of feature check. - Convert a number of fprintf() calls. + Delete detection from both cmake and autotools, along with the related + source snippet, and the `--with-egd-socket=` `./configure` option. -- tool: remove newlines from all helpf/notef/warnf/errorf calls + Closes #11556 - Make voutf() always add one. +Stefan Eissing (1 Aug 2023) - Closes #11226 +- tests: fix h3 server check and parallel instances -- tests/servers.pm: pick unused port number with a server socket + - fix check for availability of nghttpx server + - add `tcp` frontend config for same port as quic, as + without this, port 3000 is bound which clashes for parallel + testing - This change replaces the previous method of picking a port number at - random to try to start servers on, then retrying up to ten times with - new random numbers each time, with a function that creates a server - socket on port zero, thereby getting a suitable random port set by the - kernel. That server socket is then closed and that port number is used - to setup the actual test server on. + Closes #11553 - There is a risk that *another* server can be started on the machine in - the time gap, but the server verification feature will detect that. +Daniel Stenberg (1 Aug 2023) - Closes #11220 +- docs/cmdline-opts: spellfixes, typos and polish -- RELEASE-NOTES: synced + To make them accepted by the spell checker - bump to 8.2.0 + Closes #11562 -Alejandro R. Sedeño (31 May 2023) +- CI/spellcheck: build curl.1 and spellcheck it -- configure: fix run-compiler for old /bin/sh + Added acceptable words - If you try to assign and export on the same line on some older /bin/sh - implementations, it complains: + Closes #11562 - ``` - $ export "NAME=value" - NAME=value: is not an identifier - ``` +Alexander Jaeger (1 Aug 2023) - This commit rewrites run-compiler's assignments and exports to work with - old /bin/sh, splitting assignment and export into two separate - statements, and only quote the value. So now we have: +- misc: fix various typos - ``` - NAME="value" - export NAME - ``` + Closes #11561 - While we're here, make the same change to the two supporting - assign+export lines preceeding the script to be consistent with how - exports work throughout the rest of configure.ac. +Daniel Stenberg (1 Aug 2023) - Closes #11228 +- http2: avoid too early connection re-use/multiplexing -Philip Heiduck (31 May 2023) + HTTP/1 connections that are upgraded to HTTP/2 should not be picked up + for reuse and multiplexing by other handles until the 101 switching + process is completed. -- circleci: install impacket & wolfssl 5.6.0 + Lots-of-debgging-by: Stefan Eissing + Reported-by: Richard W.M. Jones + Bug: https://curl.se/mail/lib-2023-07/0045.html + Closes #11557 - Closes #11221 +- Revert "KNOWN_BUGS: build for iOS simulator on macOS 13.2 with Xcode 14" -Daniel Stenberg (31 May 2023) + This reverts commit 2e8a3d7cb73c85a9aa151e263315f8a496dbb9d4. -- tool_urlglob: use curl_off_t instead of longs + It's a user error for supplying incomplete information to the build system. - To handle more globs better (especially on Windows) + Reported-by: Ryan Schmidt + Ref: https://github.com/curl/curl/issues/11215#issuecomment-1658729367 - Closes #11224 +Viktor Szakats (1 Aug 2023) -Dan Fandrich (30 May 2023) +- cmake: add support for single libcurl compilation pass -- scripts: Fix GHA matrix job detection in cijobs.pl + Before this patch CMake builds used two separate compilation passes to + build the shared and static libcurl respectively. This patch allows to + reduce that to a single pass if the target platform and build settings + allow it. - The parsing is pretty brittle and it broke detecting some jobs at some - point. Also, detect if Windows is used in GHA. + This reduces CMake build times when building both static and shared + libcurl at the same time, making these dual builds an almost zero-cost + option. -- runtests: abort test run after failure without -a + Enable this feature for Windows builds, where the difference between the + two passes was the use of `__declspec(dllexport)` attribute for exported + API functions for the shared builds. This patch replaces this method + with the use of `libcurl.def` at DLL link time. - This was broken in a recent refactor and test runs would not stop. + Also update `Makefile.mk` to use `libcurl.def` to export libcurl API + symbols on Windows. This simplifies (or fixes) this build method (e.g. + in curl-for-win, which generated a `libcurl.def` from `.h` files using + an elaborate set of transformations). - Follow-up to d4a1b5b6 + `libcurl.def` has the maintenance cost of keeping the list of public + libcurl API symbols up-to-date. This list seldom changes, so the cost + is low. - Reported-by: Daniel Stenberg - Fixes #11225 - Closes #11227 + Closes #11546 -Version 8.1.2 (30 May 2023) +- cmake: detect `SSL_set0_wbio` in OpenSSL -Daniel Stenberg (30 May 2023) + Present in OpenSSL 1.1.0 and BoringSSL. + Missing from LibreSSL 3.8.0. -- RELEASE-NOTES: synced + Follow-up to f39472ea9f4f4e12cfbc0500c4580a8d52ce4a59 - 8.1.2 release + While here, also fix `RAND_egd()` detection which was broken, likely all + along. This feature is probably broken with CMake builds and also + requires a sufficiently obsolete OpenSSL version, so this part of the + update was not tested. -- THANKS: contributors from 8.1.2 + Closes #11555 diff --git a/libs/libcurl/docs/COPYING b/libs/libcurl/docs/COPYING index 5cf733b096..5f6dc0aef2 100644 --- a/libs/libcurl/docs/COPYING +++ b/libs/libcurl/docs/COPYING @@ -1,6 +1,6 @@ COPYRIGHT AND PERMISSION NOTICE -Copyright (c) 1996 - 2023, Daniel Stenberg, , and many +Copyright (c) 1996 - 2024, Daniel Stenberg, , and many contributors, see the THANKS file. All rights reserved. diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index d7e05f07ef..0c1d0c7589 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -92,6 +92,7 @@ Alex Samorukov Alex Suykov Alex Vinnik Alex Xu +Alexander Bartel Alexander Beedie Alexander Chuykov Alexander Dyagilev @@ -206,6 +207,7 @@ Andy Stamp Andy Tsouladze Angus Mackay anio on github +annalee anon00000000 on github anshnd on github Anssi Kolehmainen @@ -290,6 +292,7 @@ Basuke Suzuki baumanj on github bdry on github beckenc on github +Ben Ben Boeckel Ben Darnell Ben Fritz @@ -373,6 +376,7 @@ Brandon Dong Brandon Wang BratSinot on github Brendan Jurd +Brennan Kinney Brent Beardsley Brian Akins Brian Bergeron @@ -404,11 +408,13 @@ Bryan Henderson Bryan Kemp bsammon on github bsergean on github +bubbleguuum on github Bubu on github buzo-ffm on github bxac on github Bylon2 on github Byrial Jensen +Cajus Pollmeier Caleb Raitto Calvin Buckley calvin2021y on github @@ -441,6 +447,7 @@ Cering on github Cesar Eduardo Barros Chad Monroe Chandrakant Bagul +Chara White Charles Cazabon Charles Kerr Charles Romestant @@ -465,6 +472,7 @@ Chris Maltby Chris Mumford Chris Paulson-Ellis Chris Roberts +Chris Sauer Chris Smowton Chris Talbot Chris Young @@ -681,6 +689,7 @@ Denis Laxalde Denis Ollier Dennis Clarke Dennis Felsing +dependabot[bot] Derek Higgins Derzsi Dániel Desmond O. Chang @@ -943,6 +952,7 @@ FuccDucc on github Fujii Hironori fullincome on github fundawang on github +Gabe Gabriel Corona Gabriel Kuri Gabriel Simmer @@ -1055,6 +1065,7 @@ Hannes Magnusson Hanno Böck Hanno Kranzhoff Hans Steegers +Hans-Christian Egtvedt Hans-Christian Noren Egtvedt Hans-Jurgen May Hao Wu @@ -1066,6 +1077,7 @@ Harry Sarson Harry Sintonen Harshal Pradhan Hauke Duden +Haydar Alaidrus Hayden Roche He Qin Heikki Korpela @@ -1084,6 +1096,7 @@ Henry Ludemann Henry Roeland Herve Amblard HexTheDragon +hgdagon on github Hide Ishikawa Hidemoto Nakada highmtworks on github @@ -1117,6 +1130,7 @@ Ian Lynagh Ian Spence Ian Turner Ian Wilkes +iAroc on github iconoclasthero icy17 on github Ignacio Vazquez-Abrams @@ -1155,6 +1169,7 @@ Ishan SinghLevett Ithubg on github Ivan Avdeev Ivan Tsybulin +ivanfywang IvanoG on github Ivo Bellin Salarin iz8mbw on github @@ -1214,6 +1229,7 @@ Jan Venekamp Jan Verbeek Jan-Piet Mens JanB on github +janko-js on github Janne Blomqvist Janne Johansson Jared Jennings @@ -1235,6 +1251,7 @@ Javier Navarro Javier Sixto Jay Austin Jay Dommaschk +Jay Wu Jayesh A Shah Jaz Fresh JazJas on github @@ -1289,6 +1306,7 @@ Jerry Krinock Jerry Wu Jes Badwal Jesper Jensen +Jess Lowe Jesse Chisholm Jesse Noller Jesse Tan @@ -1600,6 +1618,7 @@ Lawrence Gripper Lawrence Matthews Lawrence Wagerfield Leah Neukirchen +Lealem Amedie Leandro Coutinho Legoff Vincent Lehel Bernadt @@ -1623,6 +1642,7 @@ LigH-de on github lijian996 on github Lijo Antony lilongyan-huawei on github +Lin Sun Linas Vepstas Lindley French Ling Thio @@ -1739,6 +1759,7 @@ Mark Davies Mark Dodgson Mark Gaiser Mark Hamilton +Mark Huang Mark Incley Mark Itzcovitz Mark Karpeles @@ -1747,6 +1768,7 @@ Mark Nottingham Mark Roszko Mark Salisbury Mark Seuffert +Mark Sinkovics Mark Snelling Mark Swaanenburg Mark Tully @@ -1835,6 +1857,7 @@ Matthias Naegler Mattias Fornander Matus Uzak Maurice Barnum +Mauricio Scheffer Mauro Iorio Mauro Rappa Maurício Meneghini Fauth @@ -1958,6 +1981,7 @@ Mohamed Lrhazi Mohamed Osama Mohammad AlSaleh Mohammad Hasbini +Mohammadreza Hendiani Mohammed Naser Mohun Biswas momala454 on github @@ -2109,6 +2133,7 @@ Oskar Liljeblad Oskar Sigvardsson Oumph on github ovidiu-benea on github +Ozan Cansel P R Schaffner Pablo Busse Palo Markovic @@ -2261,6 +2286,7 @@ Prithvi MK privetryan on github Priyanka Shah ProceduralMan on github +promptfuzz_ on hackerone Pronyushkin Petr Przemysław Tomaszewski pszemus on github @@ -2330,6 +2356,7 @@ Renaud Lehoux Rene Bernhardt Rene Rebe Reuven Wachtfogel +RevaliQaQ on github Reza Arbab Rianov Viacheslav Ricardo Cadime @@ -2355,6 +2382,7 @@ Richard Gorton Richard Gray Richard Hosking Richard Hsu +Richard Levitte Richard Marion Richard Michael Richard Moore @@ -2511,6 +2539,7 @@ Saul good Saurav Babu sayrer on github SBKarr on github +Scarlett McAllister Scott Bailey Scott Barrett Scott Cantor @@ -2556,6 +2585,7 @@ SerusDev on github Seshubabu Pasam Seth Mos Sevan Janiyan +sfan5 on github Sgharat on github Sh Diao Shachaf Ben-Kiki @@ -2705,6 +2735,7 @@ Taneli Vähäkangas Tanguy Fautre Taras Kushnir tarek112 on github +Tatsuhiko Miyagawa Tatsuhiro Tsujikawa tawmoto on github tbugfinder on github @@ -2717,6 +2748,7 @@ thanhchungbtc on github The Infinnovation team TheAssassin on github TheKnarf on github +Theo Theodore Dubois therealhirudo on github Thiago Suchorski @@ -2724,6 +2756,7 @@ tholin on github Thomas Bouzerar Thomas Braun Thomas Danielsson +Thomas Ferguson Thomas Gamper Thomas Glanzmann Thomas Guillem @@ -2961,6 +2994,7 @@ x2018 on github Xavier Bouchoux XhmikosR on github XhstormR on github +Xi Ruoyao Xiang Xiao Xiangbin Li xianghongai on github @@ -2972,6 +3006,7 @@ xtonik on github xwxbug on github Xì Gà Yaakov Selkowitz +Yadhu Krishna M Yair Lenga Yang Tse Yaobin Wen @@ -2979,10 +3014,12 @@ Yarram Sunil Yasuharu Yamada Yasuhiro Matsumoto Yechiel Kalmenson +Yedaya Katsman Yehezkel Horowitz Yehoshua Hershberg ygthien on github Yi Huang +Yifei Kong Yiming Jing Yingwei Liu yiyuaner on github @@ -3010,6 +3047,8 @@ Zachary Seguin Zdenek Pavlas Zekun Ni zelinchen on github +zengwei +zengwei2000 Zenju on github Zero King Zespre Schmidt -- cgit v1.2.3