From d7b105a2240a2c1dcd789cec21cc985c709747eb Mon Sep 17 00:00:00 2001 From: dartraiden Date: Sun, 1 Nov 2020 23:02:54 +0300 Subject: libcurl: update to 7.73.0 --- libs/libcurl/docs/CHANGES | 10230 ++++++++++++++++++++++---------------------- 1 file changed, 5091 insertions(+), 5139 deletions(-) (limited to 'libs/libcurl/docs/CHANGES') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 51a99f4d52..b5bc305951 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,7495 +6,7447 @@ Changelog -Version 7.72.0 (19 Aug 2020) +Version 7.73.0 (14 Oct 2020) -Daniel Stenberg (19 Aug 2020) +Daniel Stenberg (14 Oct 2020) - RELEASE-NOTES: synced - The curl 7.72.0 release + for 7.73.0 -- THANKS: add names from curl 7.72.0 release +- THANKS: from 7.73.0 and .mailmap fixes -Jay Satiro (18 Aug 2020) -- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions +- mailmap: fixups of some contributors + +- projects/build-wolfssl.bat: fix the copyright year range + +Marc Hoersken (14 Oct 2020) +- [Sergei Nikulov brought this change] + + CI/tests: fix invocation of tests for CMake builds - Reported-by: plujon@users.noreply.github.com + Update appveyor.yml to set env variable TFLAGS and run tests + Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) + Move testdeps build to build step (per review comments) - Closes https://github.com/curl/curl/issues/5488 + Reviewed-by: Marc Hörsken + + Closes #6066 + Fixes #6052 -Daniel Stenberg (17 Aug 2020) -- Curl_easy: remember last connection by id, not by pointer +- tests/server/util.c: fix support for Windows Unicode builds - CVE-2020-8231 + Detected via #6066 + Closes #6070 + +Daniel Stenberg (13 Oct 2020) +- [Jay Satiro brought this change] + + strerror: Revert to local codepage for Windows error string - Bug: https://curl.haxx.se/docs/CVE-2020-8231.html + - Change get_winapi_error() to return the error string in the local + codepage instead of UTF-8 encoding. - Reported-by: Marc Aldorasi - Closes #5824 + Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it + also changed the error string's encoding from local codepage to UTF-8. + + We return the local codepage version of the error string because if it + is output to the user's terminal it will likely be with functions which + expect the local codepage (eg fprintf, failf, infof). + + This is essentially a partial revert of bed5f84. The support for xbox + remains but the error string is reverted back to local codepage. + + Ref: https://github.com/curl/curl/pull/6005 + + Reviewed-by: Marcel Raad + Closes #6065 -- examples/rtsp.c: correct the copyright year +Marc Hoersken (13 Oct 2020) +- CI/tests: use verification curl for test reporting APIs + + Avoid using our own, potentially installed, curl for + the test reporting APIs in case it is broken. + + Reviewed-by: Daniel Stenberg + + Preparation for #6049 + Closes #6063 -- RELEASE-PROCEDURE.md: add more future release dates +Viktor Szakats (12 Oct 2020) +- windows: fix comparison of mismatched types warning + + clang 10, mingw-w64: + ``` + vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') + [-Wsign-compare] + if(GetLastError() != CRYPT_E_NOT_FOUND) + ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ + ``` + + Approved-by: Daniel Stenberg + Closes #6062 -- [H3RSKO brought this change] +Daniel Stenberg (11 Oct 2020) +- [Viktor Szakats brought this change] - docs: change "web site" to "website" + src/Makefile.m32: fix undefined curlx_dyn_* errors - According to wikipedia: - - While "web site" was the original spelling, this variant has become - rarely used, and "website" has become the standard spelling + by linking `lib/dynbuf.c` when building a static curl binary. + Previously this source file was only included when building + a dynamic curl binary. This was likely possibly because no + functions from the `src/Makefile.inc` / `CURLX_CFILES` sources + were actually required for a curl tool build. This has + recently changed with the introduction of `curlx_dyn_*()` + memory functions and their use by the tool sources. - Closes #5822 + Closes #6060 -- [Bevan Weiss brought this change] +- HISTORY: curl verifies SSL certs by default since version 7.10 - CMake: don't complain about missing nroff +Marc Hoersken (8 Oct 2020) +- runtests.pl: use $LIBDIR variable instead of hardcoded path - The curl_nroff_check() was always being called, and complaining if - *NROFF wasn't found, even when not making the manual. + Reviewed-by: Daniel Stenberg + Closes #6051 + +Daniel Stenberg (7 Oct 2020) +- checksrc: detect // comments on column 0 - Only check for nroff (and complain) if actually making the manual + Spotted while working on #6045 - Closes #5817 + Closes #6048 -- [Brian Inglis brought this change] +- [Frederik Wedel-Heinen brought this change] - libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin + mbedtls: add missing header when defining MBEDTLS_DEBUG - copy the LDFLAGS approach for adding same option with `libhostname` in - `libtest/Makefile.am`: + Closes #6045 + +- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long - - init `libstubgss_la_LDFLAGS_EXTRA` variable, - - add option to variable inside conditional, - - use variable in `libstubgss_la_LDFLAGS` + Previously, it would pass on a define (int) which could make libcurl + read junk as a value - which prevented the CURLOPT_IPRESOLVE option to + "take". This could then make test 2100 do two DoH requests instead of + one! - Fixes #5819 - Closes #5820 + Fixes #6042 + Closes #6043 -- docs: clarify MAX_SEND/RECV_SPEED functionality +- RELEASE-NOTES: synced + +- scripts/release-notes.pl: don't "embed" $ in format string for printf() - ... in particular what happens if the maximum speed limit is set to a - value that's smaller than the transfer buffer size in use. + ... since they might contain %-codes that mess up the output! + +Jay Satiro (5 Oct 2020) +- [M.R.T brought this change] + + build-wolfssl: fix build with Visual Studio 2019 - Reported-by: Tomas Berger - Fixes #5788 - Closes #5813 + Closes https://github.com/curl/curl/pull/6033 -- test1140: compare stdout +Daniel Stenberg (4 Oct 2020) +- runtests: add %repeat[]% for test files - To make problems more immediately obvious when tests fail. + ... and use this new keywords in all the test files larger than 50K to reduce + their sizes and make them a lot easier to read and understand. - Closes #5814 + Closes #6040 -- asyn-ares: correct some bad comments +- [Emil Engler brought this change] + + --help: move two options from the misc category - Closes #5812 + The cmdline opts delegation and suppress-connect-headers + fit better into auth and proxy rather than misc. + + Follow-up to aa8777f63febc + Closes #6038 -- [Emil Engler brought this change] +- [Samanta Navarro brought this change] - docs: Add video link to docs/CONTRIBUTE.md + docs/opts: fix typos in two manual pages - Closes #5811 + Closes #6039 -- curl-config: ignore REQUIRE_LIB_DEPS in --libs output +- ldap: reduce the amount of #ifdefs needed - Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is - not considered for the --libs output. + Closes #6035 + +- runtests: provide curl's version string as %VERSION for tests - Reported-by: ramsay-jones on github - Assisted-by: Brian Inglis and Ken Brown - Fixes #5793 - Closes #5808 + ... so that we can check HTTP requests for User-Agent: curl/%VERSION + + Update 600+ test cases accordingly. + + Closes #6037 -- copyright: update/correct the year range on a few files +- checksrc: warn on space after exclamation mark + + Closes #6034 -- scripts/copyright.pl: ignore .muse files +- test1465: verify --libcurl with binary POST data -- [Emil Engler brought this change] +- runtests: allow generating a binary sequence from hex - multi: Remove 10-year old out-commented code - - The code hasn't been touched since 2010-08-18 +- tool_setopt: escape binary data to hex, not octal + +- curl: make --libcurl show binary posts correctly - Closes #5805 + Reported-by: Stephan Mühlstrasser + Fixes #6031 + Closes #6032 -- KNOWN_BUGS: A shared connection cache is not thread-safe +Jay Satiro (1 Oct 2020) +- strerror: fix null deref on winapi out-of-memory - Closes #4915 - Closes #5802 + Follow-up to bed5f84 from several days ago. + + Ref: https://github.com/curl/curl/pull/6005 -- CONTRIBUTE: extend git commit message description +Daniel Stenberg (1 Oct 2020) +- [Kamil Dudka brought this change] + + vtls: deduplicate some DISABLE_PROXY ifdefs - In particular how the first line works. + ... in the code of gtls, nss, and openssl - Closes #5803 + Closes #5735 - RELEASE-NOTES: synced -- [Stefan Yohansson brought this change] +- [Emil Engler brought this change] - transfer: move retrycount from connect struct to easy handle + TODO: Add OpenBSD libtool notice - This flag was applied to the connection struct that is released on - retry. These changes move the retry counter into Curl_easy struct that - lives across retries and retains the new connection. + See #5862 + Closes #6030 + +- tests/unit/README: convert to markdown - Reported-by: Cherish98 on github - Fixes #5794 - Closes #5800 + ... and add to dist! + + Closes #6028 -- libssh2: s/ssherr/sftperr/ +- tests/README: convert to markdown - The debug output used ssherr instead of sftperr which not only outputs - the wrong error code but also casues a warning on Windows. + Closes #6028 + +- include/README: convert to markdown - Follow-up to 7370b4e39f1 + Closes #6028 + +- examples/README: convert to markdown - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 - Closes #5799 + Closes #6028 -- ftp: don't do ssl_shutdown instead of ssl_close +- configure: don't say HTTPS-proxy is enabled when disabled! - The shutdown function is for downgrading a connection from TLS to plain, - and this is not requested here. + Reported-by: Kamil Dudka + Reviewed-by: Kamil Dudka + Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 + Closes #6029 + +Daniel Gustafsson (30 Sep 2020) +- src: Consistently spell whitespace without whitespace - Have ssl_close reset the TLS connection state. + Whitespace is spelled without a space between white and space, so + make sure to consistently spell it that way across the codebase. - This partially reverts commit f002c850d98d + Closes #6023 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler + +- MANUAL: update examples to resolve without redirects - Reported-by: Rasmus Melchior Jacobsen - Reported-by: Denis Goleshchikhin - Fixes #5797 + www.netscape.com is redirecting to a cookie consent form on Aol, and + cool.haxx.se isn't responding to FTP anymore. Replace with examples + that resolves in case users try out the commands when reading the + manual. + + Closes #6024 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -Marc Hoersken (9 Aug 2020) -- CI/azure: fix test outcome values and use latest API version +Daniel Stenberg (30 Sep 2020) +- HISTORY: add some 2020 events + +- sectransp: make it build with --disable-proxy - This makes sure that tests ignored or skipped are not shown - just in the category "Other", but with their correct state. + Follow-up from #5466 and f3d501dc678d80 + Reported-by: Javier Navarro + Fixes #6025 + Closes #6026 + +- ECH: renamed from ESNI in docs and configure - Closes #5796 + Encrypted Client Hello (ECH) is the current name. + + Closes #6022 -- CI/azure: show runtime stats to investigate slowness +- configure: use "no" instead of "disabled" for the end summary - Also avoid naming conflict of TFLAGS env and tflags variables. + ... for consistency but also to make them more distinctly stand out next + to the "enabled" lines. + +- TODO: SSH over HTTPS proxy with more backends - Closes #5776 + ... as right now only the libssh2 backend supports it. -Daniel Stenberg (8 Aug 2020) -- TLS naming: fix more Winssl and Darwinssl leftovers +- libssh2: handle the SSH protocols done over HTTPS proxy - The CMake option is now called CMAKE_USE_SCHANNEL + Reported-by: Robin Douine + Fixes #4295 + Closes #6021 + +- [Emil Engler brought this change] + + memdebug: remove 9 year old unused debug function - The winbuild flag is USE_SCHANNEL + There used to be a way to have memdebug fill allocated memory. 9 years + later this has no value there (valgrind and ASAN etc are way better). If + people need to know about it they can have a look at VCS logs. - The CI jobs and build scripts only use the new names and the new name - options + Closes #5973 + +- sendf: move Curl_sendf to dict.c and make it static - Tests now require 'Schannel' (when necessary) + ... as the only remaining user of that function. Also fix gopher.c to + instead use Curl_write() - Closes #5795 + Closes #6020 -- smtp_parse_address: handle blank input string properly +- ROADMAP: updates and cleanups - Closes #5792 + Fix the HSTS PR + + Remove DoT, thread-safe init and hard-coded localhost. I feel very + little interest for these with users so I downgrade them to plain "TODO" + entries again. -- runtests: run the DICT server on a random port number +- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root - Removed support for -b (base port number) + This matches what is returned in other TLS backends in the same + situation. - Closes #5783 + Reviewed-by: Jay Satiro + Reviewed-by: Emil Engler + Follow-up to 5a3efb1 + Reported-by: iammrtau on github + Fixes #6003 + Closes #6018 - RELEASE-NOTES: synced -- runtests: move the TELNET server to a dynamic port +- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL - Rename the port variable to TELNETPORT to better match the existing - pattern. + Added test 348 to verify. Added a 'STOR' command to the test FTP + server to enable test 348. Documented the command in FILEFORMAT.md - Closes #5785 + Reported-by: Duncan Wilcox + Fixes #6016 + Closes #6017 -- ngtcp2: adapt to error code rename +- pause: only trigger a reread if the unpause sticks - Closes #5786 + As an unpause might itself get paused again and then triggering another + reread doesn't help. + + Follow-up from e040146f22608fd9 (shipped since 7.69.1) + + Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html + Patch-by: Kunal Chandarana + Fixes #5988 + Closes #6013 -- runtests: move the smbserver to use a dynamic port number +- test163[12]: require http to be built-in to run - Closes #5782 + ... as speaking over an HTTPS proxy implies http! + + Closes #6014 -- runtests: run the http2 tests on a random port number +- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define - Closes #5779 + Closes #6012 -- gtls: survive not being able to get name/issuer +- [Javier Blazquez brought this change] + + strerror: honor Unicode API choice on Windows - Closes #5778 + Closes #6005 -- runtests: move the gnutls-serv tests to a dynamic port +- imap: make imap_send use dynbuf for the send buffer management - Affects test 320, 321, 322 and 324. + Reuses the buffer and thereby reduces number of mallocs over a transfer. - Closes #5778 + Closes #6010 -- runtests: support dynamicly base64 encoded sections in tests +- Curl_send: return error when pre_receive_plain can't malloc - This allows us to make test cases to use base64 at run-time and still - use and verify information determined at run-time, such as the IMAP test - server's port number in test 842. + ... will probably trigger some false DEAD CODE positives on non-windows + code analyzers for the conditional code. - This change makes 12 tests run again that basically never ran since we - moved to dynamic port numbers. + Closes #6011 + +- ftp: separate FTPS from FTP over "HTTPS proxy" - ftpserver.pl is adjusted to load test instructions and test number from - the preprocessed test file. + When using HTTPS proxy, SSL is used but not in the view of the FTP + protocol handler itself so separate the connection's use of SSL from the + FTP control connection's sue. - FILEFORMAT.md now documents the new base64 encoding syntax. + Reported-by: Mingtao Yang + Fixes #5523 + Closes #6006 + +Dan Fandrich (23 Sep 2020) +- tests/data: Fix some mismatched XML tags in test cases - Reported-by: Marcel Raad - Fixes #5761 - Closes #5775 + This allows these test files to pass xmllint. -- curl.1: add a few missing valid exit codes +Daniel Stenberg (23 Sep 2020) +- pingpong: use a dynbuf for the *_pp_sendf() function - 93 - 96 can be returned as well. + ... reuses the same dynamic buffer instead of doing repeated malloc/free + cycles. - Closes #5777 + Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls + after this change in my test setup (132 => 125), curl 7.72.0 needed 140 + calls for this. + + Test case 103 makes 9 less allocations now (130). Down from 149 in + 7.72.0. + + Closes #6004 -- TODO: Use multiple parallel transfers for a single download +- dynbuf: add Curl_dyn_vaddf - Closes #5774 + Closes #6004 -- TODO: Set the modification date on an uploaded file +- dynbuf: make *addf() not require extra mallocs - Closes #5768 + ... by introducing a printf() function that appends directly into a + dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if + the buffer is already big enough it can just printf directly into it. + + Since this less-malloc version requires tthe use of a library internal + printf function, we only provide this version when building libcurl and + not for the dynbuf code that is used when building the curl tool. + + Closes #5998 -- [Thomas M. DuBuisson brought this change] +- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport + + Closes #5403 - CI: Add muse CI config +- pingpong: remove a malloc per Curl_pp_vsendf call - Closes #5772 + This typically makes 7-9 fewer mallocs per FTP transfer. + + Closes #5997 -- [Thomas M. DuBuisson brought this change] +- symbian: drop support + + The OS is deprecated. I see no traces of anyone having actually built + curl for Symbian after 2012. + + The public headers are unmodified. + + Closes #5989 - travis/script.sh: fix use of `-n' with unquoted envvar +- RELEASE-NOTES: synced + +- curl_krb5.h: rename from krb5.h - Shellcheck tells us "-n doesn't work with unquoted arguments. quote or - use [[ ]]." + Follow-up from f4873ebd0be32cf - And testing shows: + Turns out some older openssl installations go bananas otherwise. + Reported-by: Tom van der Woerdt + Fixes #5995 + Closes #5996 + +- test1297: verify GOT_NOTHING with http proxy tunnel + +- http_proxy: do not count proxy headers in the header bytecount - ``` - docker run --rm -it ubuntu bash - root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" - I ran - root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" - root@fe85ce156856:/# - ``` + ... as that counter is subsequently used to detect if nothing was + returned from the peer. This made curl return CURLE_OK when it should + have returned CURLE_GOT_NOTHING. - Closes #5773 + Fixes #5992 + Reported-by: Tom van der Woerdt + Closes #5994 -- h2: repair trailer handling +- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument - The previous h2 trailer fix in 54a2b63 was wrong and caused a - regression: it cannot deal with trailers immediately when read since - they may be read off the connection by the wrong 'data' owner. + Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the + option is, yeah, not known. Clarified this in the setopt man page too. - This change reverts the logic back to gathering all trailers into a - single buffer, like before 54a2b63. + Closes #5993 + +- krb5: merged security.c and krb specific FTP functions in here - Reported-by: Tadej Vengust - Fixes #5663 - Closes #5769 - -Viktor Szakats (3 Aug 2020) -- windows: disable Unix Sockets for old mingw + These two files were always tightly connected and it was hard to + understand what went into which. This also allows us to make the + ftpsend() function static (moved from ftp.c). - Classic mingw and 10y+ old versions of mingw-w64 don't ship with - Windows headers having the typedef necessary for Unix Sockets - support, so try detecting these environments to disable this - feature. + Removed security.c + Renamed curl_sec.h to krb5.h - Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ + Closes #5987 + +- Curl_handler: add 'family' to each protocol - Reviewed-by: Daniel Stenberg + Makes get_protocol_family() faster and it moves the knowledge about the + "families" to each protocol handler, where it belongs. - Fixes #5674 - Closes #5758 + Closes #5986 -Marcel Raad (3 Aug 2020) -- test1908: treat file as text +- parsedate: tune the date to epoch conversion - Fixes the line endings on Windows. + By avoiding an unnecessary error check and the temp use of the tm + struct, the time2epoch conversion function gets a little bit faster. + When repeating test 517, the updated version is perhaps 1% faster (on + one particular build on one particular architecture). - Closes https://github.com/curl/curl/pull/5767 + Closes #5985 -- TrackMemory tests: ignore realloc and free in getenv.c +- cmake: remove scary warning - These are only called for WIN32. + Remove the text saying - Closes https://github.com/curl/curl/pull/5767 + "the curl cmake build system is poorly maintained. Be aware" + + ... not because anything changed just now, but to encourage users to use + it and subsequently improve it. + + Closes #5984 -Daniel Stenberg (3 Aug 2020) -- tests/FILEFORMAT.md: mention %HTTP2PORT +- docs/MQTT: remove outdated paaragraphs -- RELEASE-NOTES: synced +- docs/MQTT: not experimental anymore + + Follow-up to e37e4468688d8f -- tlsv1.3.d. only for TLS-using connections +- docs/RESOURCES: remove - ... and rephrase that "not all" TLS backends support it. + This document is not maintained and rather than trying to refresh it, + let's kill it. A more up-to-date document with relevant RFCs is this + page on the curl website: https://curl.haxx.se/rfc/ - Closes #5764 + Closes #5980 -- tls-max.d: this option is only for TLS-using connections +- docs/TheArtOfHttpScripting: convert to markdown - Ref: #5763 - Closes #5764 - -Marcel Raad (2 Aug 2020) -- [Cameron Cawley brought this change] - - tool_doswin: Simplify Windows version detection + Makes it easier to browse on github etc. Offers (better) links. - Closes https://github.com/curl/curl/pull/5754 - -- [Cameron Cawley brought this change] + It should be noted that this document is already mostly outdated and + "Everything curl" at https://ec.haxx.se/ is a better resource and + tutorial. + + Closes #5981 - win32: Add Curl_verify_windows_version() to curlx +- BUGS: convert document to markdown - Closes https://github.com/curl/curl/pull/5754 + Closes #5979 -- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL +- --help: strdup the category - This makes the tests that require the OpenSSL feature also run for - those two compatible libraries. + ... since it is converted and the original pointer is freed on Windows + unicode handling. - Closes https://github.com/curl/curl/pull/5762 + Follow-up to aa8777f63febc + Fixes #5977 + Closes #5978 + Reported-by: xwxbug on github -Daniel Stenberg (1 Aug 2020) -- multi: Condition 'extrawait' is always true +- CHECKSRC: document two missing warnings + +- RELEASE-NOTES: synced + +- ftp: avoid risk of reading uninitialized integers - Reported by Codacy. + If the received PASV response doesn't match the expected pattern, we + could end up reading uninitialized integers for IP address and port + number. - Reviewed-by: Marcel Raad - Closes #5759 + Issue pointed out by muse.dev + Closes #5972 -Marcel Raad (1 Aug 2020) -- openssl: fix build with LibreSSL < 2.9.1 +- [Quentin Balland brought this change] + + easy_reset: clear retry counter - `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were - introduced in LibreSSL 2.9.1 [0]. + Closes #5975 + Fixes #5974 + +- ftp: get rid of the PPSENDF macro - [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef + The use of such a macro hides some of what's actually going on to the + reader and is generally disapproved of in the project. - Closes https://github.com/curl/curl/pull/5757 - -Daniel Stenberg (1 Aug 2020) -- [Marc Aldorasi brought this change] + Closes #5971 - multi_remove_handle: close unused connect-only connections +- man pages: switch to https://example.com URLs - Previously any connect-only connections in a multi handle would be kept - alive until the multi handle was closed. Since these connections cannot - be re-used, they can be marked for closure when the associated easy - handle is removed from the multi handle. + Since HTTPS is "the new normal", this update changes a lot of man page + examples to use https://example.com instead of the previous "http://..." - Closes #5749 + Closes #5969 -- checksrc: invoke script with -D to find .checksrc proper +- github: remove the duplicate "Security vulnerability" entry - Without the -D command line option, checksrc.pl won't know which - directory to load the ".checksrc" file from when building out of the - source tree. + ... since github adds an entry automatically by itself. - Reported-by: Marcel Raad - Fixes #5715 - Closes #5755 + Closes #5970 -- [Carlo Marcelo Arenas Belón brought this change] +- [Emil Engler brought this change] - buildconf: retire ares buildconf invocation + github: use new issue template feature - no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 + This helps us to avoid getting feature requests as well as security + bugs reported into the issue tracker. + + Closes #5936 -- [Carlo Marcelo Arenas Belón brought this change] +- [Emil Engler brought this change] - buildconf: excempt defunct reference to ACLOCAL_FLAGS + urlapi: use more Curl_safefree - retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as - the name is generic enough that it might be in use and relied upon from - the environment. - -- [Carlo Marcelo Arenas Belón brought this change] + Closes #5968 - buildconf: avoid array concatenation in die() +Marc Hoersken (17 Sep 2020) +- multi: align WinSock mask variables in Curl_multi_wait - reported as error SC2145[1] by shellcheck, but not expected to cause - any behavioural differences otherwise. + Also skip pre-checking sockets to set timeout_ms to 0 + after the first socket has been detected to be ready. - [1] https://github.com/koalaman/shellcheck/wiki/SC2145 + Reviewed-by: rcombs on github + Reviewed-by: Daniel Stenberg - Closes #5701 + Follow up to #5886 -- travis: add ppc64le and s390x builds +- multi: reuse WinSock events variable in Curl_multi_wait - Closes #5752 - -Marc Hoersken (31 Jul 2020) -- connect: remove redundant message about connect failure + Since the struct is quite large (1 long and 10 ints) we + declare it once at the beginning of the function instead + of multiple times inside loops to avoid stack movements. + Reviewed-by: Viktor Szakats Reviewed-by: Daniel Stenberg - Closes #5708 + Closes #5886 -- tests/sshserver.pl: fix compatibility with OpenSSH for Windows +Daniel Stenberg (16 Sep 2020) +- TODO: dynamically decide to use socketpair - Follow up to #5721 - -- CI/azure: install libssh2 for use with msys2-based builds + Suggested-by: Anders Bakken - This enables building and running the SFTP tests. - Unfortunately OpenSSH for Windows does not support SCP (yet). + Closes #4829 + +- TODO: add PR reference for native IDN support on macOS - Reviewed-by: Daniel Stenberg + As there was work started on this that never got completed. - Closes #5721 + Closes #5371 -- CI/azure: increase Windows job timeout once again +- tool_help.h: update copyright year range - Avoid aborted jobs due to performance issues on Azure DevOps. + Follow-up from aa8777f63febca + +- CI/azure: disable test 571 in the msys2 builds - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro + It's just too flaky there - Closes #5738 + Reviewed-by: Marc Hoersken + Closes #5954 -Jay Satiro (30 Jul 2020) -- TODO: Schannel: 'Add option to allow abrupt server closure' +- tool_writeout: protect fputs() from NULL - We should offer an option to allow abrupt server closures (server closes - SSL transfer without sending a known termination point such as length of - transfer or close_notify alert). Abrupt server closures are usually - because of misconfigured or very old servers. + When the code was changed to do fputs() instead of fprintf() it got + sensitive for NULL pointers; add checks for that. - Closes https://github.com/curl/curl/issues/4427 - -- url: fix CURLU and location following + Follow-up from 0c1e767e83ec66 - Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was - incorrectly used for the location follow, resulting in infinite requests - to the original location. + Closes #5963 + +- test3015: verify stdout "as text" - Reported-by: sspiri@users.noreply.github.com + Follow-up from 0c1e767e83e to please win32 tests - Fixes https://github.com/curl/curl/issues/5709 - Closes https://github.com/curl/curl/pull/5713 + Closes #5962 -Daniel Stenberg (30 Jul 2020) -- RELEASE-NOTES: synced +- travis: use libressl v3.1.4 instead of master + + ... as their git master seems too fragile to use (and 3.2.1 which is the + latest has a build failure). + + Closes #5964 -- [divinity76 brought this change] +- tests/FILEFORMAT: document type=shell for - docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions +- tests/FILEFORMAT: document nonewline support for - it helps make it obvious that most developers don't have to care about - the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 - years old, November 4 2009) + The one in , that creates files. - Closes #5744 + Follow-up from b83947c8df7 -Jay Satiro (29 Jul 2020) -- tool_cb_wrt: fix outfile mode flags for Windows +- [anio brought this change] + + tool_writeout: add new writeout variable, %{num_headers} - - Use S_IREAD and S_IWRITE mode permission flags to create the file - on Windows instead of S_IRUSR, S_IWUSR, etc. + This variable gives the number of headers. - Windows only accepts a combination of S_IREAD and S_IWRITE. It does not - acknowledge other combinations, for which it may generate an assertion. + Closes #5947 + +- tool_urlglob: fix compiler warning "unreachable code" - This is a follow-up to 81b4e99 from yesterday, which improved the - existing file check with -J. + (On Windows builds.) - Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks - Ref: https://github.com/curl/curl/pull/5731 + Follow-up to 70a3b003d9 + +- [Gergely Nagy brought this change] + + vtls: deduplicate client certificates in ssl_config_data - Closes https://github.com/curl/curl/pull/5742 + Closes #5629 -Daniel Stenberg (28 Jul 2020) -- checksrc: ban gmtime/localtime +- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND - They're not thread-safe so they should not be used in libcurl code. + This is primarily interesting for cases where CURLOPT_NOBODY is set as + previously curl would not return an error for this case. - Explictly enabled when deemed necessary and in examples and tests + MDTM getting 550 now also returns this error (it returned + CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for + missing files across protocols and specific FTP commands. - Reviewed-by: Nicolas Sterchele - Closes #5732 + libcurl already returns error on a 550 as a MDTM response (when + CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would + happen subsequently anyway since the RETR command would fail. + + Add test 1913 and 1914 to verify. Updated several tests accordingly due + to the updated SIZE behavior. + + Reported-by: Tomas Berger + Fixes #5953 + Closes #5957 -- transfer: fix data_pending for builds with both h2 and h3 enabled +- curl: make checkpasswd use dynbuf - Closes #5734 + Closes #5952 -- curl_multi_setopt: fix compiler warning "result is always false" +- curl: make glob_match_url use dynbuf - On systems with 32 bit long the expression is always false. Avoid - the warning. + Closes #5952 + +- curl: make file2memory use dynbuf - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 - Closes #5736 + Closes #5952 -- curl: improve the existing file check with -J +- curl: make file2string use dynbuf - Previously a file that isn't user-readable but is user-writable would - not be properly avoided and would get overwritten. + Closes #5952 + +- [Antarpreet Singh brought this change] + + imap: set cselect_bits to CURL_CSELECT_IN initially - Reported-by: BrumBrum on hackerone - Assisted-by: Jay Satiro - Bug: https://hackerone.com/reports/926638 - Closes #5731 + ... when continuing a transfer from a FETCH response. + + When the size of the file was small enough that the entirety of the + transfer happens in a single go and schannel buffers holds the entire + data. However, it wasn't completely read in Curl_pp_readresp since a + line break was found before that could happen. So, by the time we are in + imap_state_fetch_resp - there's data in buffers that needs to be read + via Curl_read but nothing to read from the socket. After we setup a + transfer (Curl_setup_transfer), curl just waits on the socket state to + change - which doesn't happen since no new data ever comes. + + Closes #5961 -- [Jonathan Nieder brought this change] +- RELEASE-NOTES: synced - multi: update comment to say easyp list is linear +- test434: test -K use in a single line without newline - Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, - 2013-08-02), the easy handle list is not circular but ends with - ->next pointing to NULL. + Closes #5946 + +- runtests: allow creating files without newlines - Reported-by: Masaya Suzuki - Closes #5737 + Closes #5946 -- CURLOPT_NOBODY.3: fix the syntax for referring to options +- curl: use curlx_dynbuf for realloc when loading config files - As test 1140 fails otherwise! + ... fixes an integer overflow at the same time. - Follow-up to e1bac81cc815 + Reported-by: ihsinme on github + Assisted-by: Jay Satiro + + Closes #5946 -- ngtcp2: store address in sockaddr_storage +- dynbuf: provide curlx_ names for reuse by the curl tool - Reported-by: Tatsuhiro Tsujikawa - Closes #5733 + Closes #5946 -- CURLOPT_NOBODY.3: clarify what setting to 0 means +- dynbuf: make sure Curl_dyn_tail() zero terminates - ... and mention that HTTP with other methods than HEAD might get a body and - there's no option available to stop that. + Closes #5959 + +- tests: add test1912 to the dist - Closes #5729 + Follow-up to 70984ce1be4cab6c -- setopt: unset NOBODY switches to GET if still HEAD +- docs/LICENSE-MIXING: remove - Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented - action but before 7.71.0 that used to switch back to GET and with this - change (assuming the method is still set to HEAD) this behavior is - brought back. + This document is not maintained and I feel that it doesn't provide much + value to users anymore (if it ever did). - Reported-by: causal-agent on github - Fixes #5725 - Closes #5728 + Closes #5955 -- [Ehren Bendler brought this change] +- [Laramie Leavitt brought this change] - configure: cleanup wolfssl + pkg-config conflicts when cross compiling. + http: consolidate nghttp2_session_mem_recv() call paths - Also choose a different wolfSSL function to test for NTLM support. + Previously there were several locations that called + nghttp2_session_mem_recv and handled responses slightly differently. + Those have been converted to call the existing + h2_process_pending_input() function. - Fixes #5605 - Closes #5682 - -- configure: show zstd "no" in summary when built without it + Moved the end-of-session check to h2_process_pending_input() since the + only place the end-of-session state can change is after nghttp2 + processes additional input frames. - Reported-by: Marc Hörsken - Fixes #5720 - Closes #5730 - -- quiche: handle calling disconnect twice + This will likely fix the fuzzing error. While I don't have a root cause + the out-of-bounds read seems like a use after free, so moving the + nghttp2_session_check_request_allowed() call to a location with a + guaranteed nghttp2 session seems reasonable. - Reported-by: lilongyan-huawei on github - Fixes #5726 - Closes #5727 + Also updated a few nghttp2 callsites to include error messages and added + a few additional error checks. + + Closes #5648 -- [Nicolas Sterchele brought this change] +- HISTORY: mention alt-svc added in 2019 + + ... and make 1996 the first year subtitle - getinfo: reset retry-after value in initinfo +- base64: also build for pop3 and imap - - Avoid re-using retry_after value from preceding request - - Add libtest 3010 to verify + Follow-up to the fix in 20417a13fb8f83 - Reported-by: joey-l-us on github - Fixes #5661 - Closes #5672 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5948 -Marcel Raad (27 Jul 2020) -- WIN32: stop forcing narrow-character API +- base64: enable in build with SMTP - Except where the results are only used for character output. - getenv is not touched because it's part of the public API, and having - it return UTF-8 instead of ANSI would be a breaking change. + The oauth2 support is used with SMTP and it uses base64 functions. - Fixes https://github.com/curl/curl/issues/5658 - Fixes https://github.com/curl/curl/issues/5712 - Closes https://github.com/curl/curl/pull/5718 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5938 -Jay Satiro (27 Jul 2020) -- [Tobias Stoeckmann brought this change] +- curl_mime_headers.3: fix the example's use of curl_slist_append + + Reported-by: sofaboss on github + Fixes #5942 + Closes #5943 - mprintf: Fix stack overflows +- lib583: fix enum mixup - Stack overflows can occur with precisions for integers and floats. + grrr the previous follow-up to 17fcdf6a31 was wrong + +- libtest: fix build errors - Proof of concepts: - - curl_mprintf("%d, %.*1$d", 500, 1); - - curl_mprintf("%d, %+0500.*1$f", 500, 1); + Follow-up from 17fcdf6a310d4c8076 + +- lib: fix -Wassign-enum warnings - Ideally, compile with -fsanitize=address which makes this undefined - behavior a bit more defined for debug purposes. + configure --enable-debug now enables -Wassign-enum with clang, + identifying several enum "abuses" also fixed. - The format strings are valid. The overflows occur due to invalid - arguments. If these arguments are variables with contents controlled - by an attacker, the function's stack can be corrupted. + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 - Also see CVE-2016-9586 which partially fixed the float aspect. + Closes #5929 + +- RELEASE-NOTES: synced + +- [Diven Qi brought this change] + + url: use blank credentials when using proxy w/o username and password - Signed-off-by: Tobias Stoeckmann + Fixes proxy regression brought in commit ad829b21ae (7.71.0) - Closes https://github.com/curl/curl/pull/5722 + Fixed #5911 + Closes #5914 -- [Tobias Stoeckmann brought this change] +- travis: add a build using libressl (from git master) + + The v3.2.1 tag (latest release atm) results in a broken build. + + Closes #5932 - mprintf: Fix dollar string handling +- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 - Verify that specified parameters are in range. If parameters are too - large, fail early on and avoid out of boundary accesses. + Unfortunately, this option is not detecting the same issues as clang's + -Wassign-enum flag, but should still be useful to detect future + mistakes. - Also do not read behind boundaries of illegal format strings. + Closes #5930 + +- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - These are defensive measures since it is expected that format strings - are well-formed. Format strings should not be modifiable by user - input due to possible generic format string attacks. + If the error reason from the lib is + SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return + CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. - Closes https://github.com/curl/curl/pull/5722 + This unifies the libcurl return code and makes libressl run test 313 + (CRL testing) fine. + + Closes #5934 -Daniel Stenberg (26 Jul 2020) -- ntlm: free target_info before (re-)malloc +- FAQ: refreshed some very old language + +- cmake: make HTTP_ONLY also disable MQTT - OSS-Fuzz found a way this could get called again with the pointer still - pointing to a malloc'ed memory, leading to a leak. + ... and alphasort the order of disabling protocols to make it easier to + browse. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 + Closes #5931 + +- libtest: remove lib1541 leftovers - Closes #5724 + Caused automake errors. + + Follow-up to 8ca54a03ea08a -Marcel Raad (26 Jul 2020) -- CI/macos: set minimum macOS version +- tests/libtests: remove test 1900 and 2033 - This enables some deprecation warnings. - Previously, autotools defaulted to 10.8. + We already remove the test files, now remove the libtest codes as well. - Closes https://github.com/curl/curl/pull/5723 + Follow-up to e50a877df74 -Daniel Stenberg (26 Jul 2020) -- RELEASE-NOTES: synced +Marc Hoersken (7 Sep 2020) +- CI/azure: add test number to title for display in analytics + + To ease identification of tests the test number is added to + the test case title in order to have it on the Azure DevOps + Analytics pages and reports which currently do not show it. + + Bump test case revision to make Azure DevOps update titles. + + Closes #5927 -Marcel Raad (25 Jul 2020) -- CI/macos: enable warnings as errors for CMake builds +Daniel Stenberg (6 Sep 2020) +- altsvc: clone setting in curl_easy_duphandle - Closes https://github.com/curl/curl/pull/5716 + The cache content is not duplicated, like other caches, but the setting + and specified file name are. + + Test 1908 is extended to verify this somewhat. Since the duplicated + handle gets the same file name, the test unfortunately overwrites the + same file twice (with different contents) which makes it hard to check + automatically. + + Closes #5923 -- CMake: fix test for warning suppressions +- test1541: remove since it is a known bug - GCC doesn't warn for unknown `-Wno-` options, except if there are other - warnings or errors [0]. This was problematic with `CURL_WERROR` as that - warning-as-error cannot be suppressed. Notably, this always happened - with `-Wno-pedantic-ms-format` when not targeting Windows. So test for - the positive form of the warning instead, which should always result in - a diagnostic if unknown. + A shared connection cache is not thread-safe is a known issue. Stop + testing this until we believe this issue is addressed. Reduces + occasional test failures we don't care about. - [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html + The test code in lib1541.c is left in git to allow us to restore it when + we get to fix this. - Closes https://github.com/curl/curl/pull/5714 + Closes #5922 -Jay Satiro (23 Jul 2020) -- curl.h: update CURLINFO_LASTONE +- tests: remove pipelining tests - CURLINFO_LASTONE should have been updated when - CURLINFO_EFFECTIVE_METHOD was added. + Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were + previously disabled. - Reported-by: xwxbug@users.noreply.github.com + The Pipelining code was removed from curl in commit 2f44e94efb3df8e, + April 2019. - Fixes https://github.com/curl/curl/issues/5711 + Closes #5921 -Marc Hoersken (22 Jul 2020) -- CI/azure: unconditionally enable warnings-as-errors with autotools +- curl: retry delays in parallel mode no longer sleeps blocking - Reviewed-by: Marcel Raad + The previous sleep for retries would block all other concurrent + transfers. Starting now, the retry will instead be properly marked to + not get restarted until after the delay time but other transfers can + still continue in the mean time. - Follow up to #5694 - Closes #5706 + Closes #5917 -Marcel Raad (21 Jul 2020) -- doh: remove redundant cast +- curl:parallel_transfers: make sure retry readds the transfer - Closes https://github.com/curl/curl/pull/5704 + Reported-by: htasta on github + Fixes #5905 + Closes #5917 -- CI/macos: unconditionally enable warnings-as-errors with autotools +- build: drop support for building with Watcom - Previously, warnings were only visible in the output for most jobs. + These files are not maintained, they seem to have no users, Watcom + compilers look like not having users nor releases anymore. - Closes https://github.com/curl/curl/pull/5694 + Closes #5918 -- util: silence conversion warnings +- winbuild/rundebug.cmd: remove - timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might - be a 64-bit integer. This is the case when building for recent macOS - versions, for example. Just treat tv_usec as an int, which should - hopefully always be sufficient on systems with - `HAVE_CLOCK_GETTIME_MONOTONIC`. + Seems to have been added by mistake? Not included in dists. - Closes https://github.com/curl/curl/pull/5695 + Closes #5919 -- md(4|5): don't use deprecated macOS functions +- curl: in retry output don't call all problems "transient" - They are marked as deprecated for -mmacosx-version-min >= 10.15, - which might result in warnings-as-errors. + ... because when --retry-all-errors is used, the error isn't necessarily + transient at all. - Closes https://github.com/curl/curl/pull/5695 + Closes #5916 -Daniel Stenberg (18 Jul 2020) -- strdup: remove the odd strlen check +- easygetopt: pass a valid enum to avoid compiler warning - It confuses code analyzers with its use of -1 for unsigned value. Also, - a check that's not normally used in strdup() code - and not necessary. + "integer constant not in range of enumerated type 'CURLoption'" - Closes #5697 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 + + Closes #5915 -- [Alessandro Ghedini brought this change] +- [Emil Engler brought this change] - travis: update quiche builds for new boringssl layout + tests: Add tests for new --help - This is required after https://github.com/cloudflare/quiche/pull/593 - moved BoringSSL around slightly. + This commit is a part of "--help me if you can" - This also means that Go is not needed to build BoringSSL anymore (the - one provided by quiche anyway). + Closes #5680 + +- [Emil Engler brought this change] + + tool: update --help with categories - Closes #5691 + This commit is a part of "--help me if you can" + + Closes #5680 -Marcel Raad (17 Jul 2020) -- configure: allow disabling warnings +- [Emil Engler brought this change] + + docs: add categories to all cmdline opts - When using `--enable-warnings`, it was not possible to disable warnings - via CFLAGS that got explicitly enabled. Now warnings are not enabled - anymore if they are explicitly disabled (or enabled) in CFLAGS. This - works for at least GCC, clang, and TCC as they have corresponding - `-Wno-` options for every warning. + Adapted gen.pl with 'listcats' - Closes https://github.com/curl/curl/pull/5689 + This commit is a part of "--help me if you can" + + Closes #5680 -Daniel Stenberg (16 Jul 2020) -- ngtcp2: adjust to recent sockaddr updates +- RELEASE-NOTES: synced + +- [ihsinme brought this change] + + connect.c: remove superfluous 'else' in Curl_getconnectinfo - Closes #5690 + Closes #5912 -- page-header: provide protocol details in the curl.1 man page +- [Samuel Marks brought this change] + + CMake: remove explicit `CMAKE_ANSI_CFLAGS` - Add protocol and version specific information about all protocols curl - supports. + This variable was removed from cmake in commit + https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later + CMake commit removes the variable from the tests, claiming that it was + removed in CMake 2.6 - Fixes #5679 - Reported-by: tbugfinder on github - Closes #5686 + Reviewed-By: Peter Wu + Closes #5439 -Daniel Gustafsson (16 Jul 2020) -- docs: Update a few leftover mentions of DarwinSSL +- [cbe brought this change] + + libssh2: pass on the error from ssh_force_knownhost_key_type - Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the - more correct/common name Secure Transport, but a few mentions in the docs - remained. + Closes #5909 + +- scripts/delta: add diffstat summary - Closes #5688 - Reviewed-by: Daniel Stenberg + ... and make output more table-like -Daniel Stenberg (16 Jul 2020) -- file2memory: use a define instead of -1 unsigned value +- [Martin Bašti brought this change] + + http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set - ... to use the maximum value for 'size_t' when detecting integer overflow. - Changed the limit to max/4 as already that seems unreasonably large. + ... in case NO_PROXY takes an effect - Codacy didn't like the previous approach. + Without this patch, the following command crashes: - Closes #5683 + $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ + git clone https://github.com/curl/curl.git + + Minimal libcurl-based reproducer: + + #include + + int main() { + CURL *curl = curl_easy_init(); + if(curl) { + CURLcode ret; + curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); + curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); + /* set the proxy type */ + curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); + curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); + curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); + ret = curl_easy_perform(curl); + curl_easy_cleanup(curl); + return ret; + } + return -1; + } + + Assisted-by: Kamil Dudka + Bug: https://bugzilla.redhat.com/1873327 + Closes #5902 -- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream +- travis: add a CI job with openssl3 (from git master) - ... by adding support for a new dedicated return code. + Closes #5908 + +- openssl: avoid error conditions when importing native CA - Suggested-by: Jonathan Cardoso - Assisted-by: Erik Johansson - URL: https://curl.haxx.se/mail/lib-2020-06/0099.html - Closes #5636 + The code section that is OpenSSL 3+ specific now uses the same logic as + is used in the version < 3 section. It caused a compiler error without + it. + + Closes #5907 -- [Baruch Siach brought this change] +- setopt: avoid curl_ on local variable + + Closes #5906 - nss: fix build with disabled proxy support +- mqtt.c: avoid curl_ prefix on local variable - Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is - defined. + Closes #5906 + +- wildcard: strip "curl_" prefix from private symbols - Closes #5667 + Closes #5906 -- test1139: make it display the difference on test failures +- vtls: make it 'struct Curl_ssl_session' + + Use uppercase C for internal symbols. + + Closes #5906 -- test1119: verify stdout in the test +- curl_threads: make it 'struct Curl_actual_call' - So that failures will be displayed in the terminal, as it makes test failures - visually displayed easier and faster. + Internal names should not be prefixed "curl_" - Closes #5644 + Closes #5906 -- curl: add %{method} to the -w variables +- schannel: make it 'struct Curl_schannel*' - Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. + As internal global names should use captical C. - Added test 1197 to verify. + Closes #5906 -- CURLINFO_EFFECTIVE_METHOD: added +- hash: make it 'struct Curl_hash' - Provide the HTTP method that was used on the latest request, which might - be relevant for users when there was one or more redirects involved. + As internal global names should use captical C. - Closes #5511 + Closes #5906 -Viktor Szakats (14 Jul 2020) -- windows: add unicode to feature list +- llist: make it "struct Curl_llist" - Reviewed-by: Marcel Raad - Reviewed-by: Marc Hörsken + As internal global names should use captical C. - Closes #5491 + Closes #5906 -Daniel Stenberg (14 Jul 2020) -- multi: remove two checks always true +Marc Hoersken (2 Sep 2020) +- telnet.c: depend on static requirement of WinSock version 2 - Detected by Codacy - Closes #5676 + Drop dynamic loading of ws2_32.dll and instead rely on the + imported version which is now required to be at least 2.2. + + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats + + Closes #5854 -Marc Hoersken (13 Jul 2020) -- workflows: limit what branches to run CodeQL on +- win32: drop support for WinSock version 1, require version 2 - Align CodeQL action with existing CI actions: - - Update branch filter to avoid duplicate CI runs. - - Shorten workflow name due to informative job name. + IPv6, telnet and now also the multi API require WinSock + version 2 which is available starting with Windows 95. + Therefore we think it is time to drop support for version 1. + + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats - Closes #5660 + Follow up to #5634 + Closes #5854 -- appveyor: collect libcurl.dll variants with prefix or suffix +- select: align poll emulation to return all relevant events - On some platforms libcurl is build with a platform-specific - prefix and/or a version number suffix. + The poll emulation via select already consumes POLLRDNORM, + POLLWRNORM and POLLRDBAND as input events. Therefore it + should also return them as output events if signaled. + + Also fix indentation in input event handling block. Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5659 + Replaces #5852 + Closes #5883 -Daniel Stenberg (12 Jul 2020) -- [ihsinme brought this change] - - socks: use size_t for size variable +- CI/azure: MQTT is now enabled by default - Use the unsigned type (size_t) in the arithmetic of pointers. In this - context, the signed type (ssize_t) is used unnecessarily. + Reviewed-by: Daniel Stenberg - Authored-by: ihsinme on github - Closes #5654 + Follow up to #5858 + Closes #5903 -- RELEASE-NOTES: synced +Daniel Stenberg (2 Sep 2020) +- copyright.pl: ignore buildconf + +- test971: show test mismatches "inline" + +- lib/Makefile.am: bump VERSIONINFO due to new functions - ... and bumped to 7.72.0 as the next release version number + ... we're generally bad at this, but we are adding new functions for + this release. + + Closes #5899 -- [Gilles Vollant brought this change] +- optiontable: use DEBUGBUILD + + Follow-up to commit 6e18568ba38 (#5877) - content_encoding: add zstd decoding support +- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 - include zstd curl patch for Makefile.m32 from vszakats - and include Add CMake support for zstd from Peter Wu + If there are more than two items in the list, use commas for all but the + last separator which is set to 'and'. Reads better. - Helped-by: Viktor Szakats - Helped-by: Peter Wu - Closes #5453 + Closes #5898 -- asyn.h: remove the Curl_resolver_getsock define +- curl.1: add see also no-progress-meter on two spots - - not used - - used the wrong number of arguments - - confused the Codeacy code analyzer + Ref: #5894 - Closes #5647 + Closes #5897 -- [Nicolas Sterchele brought this change] +- RELEASE-NOTES: synced - configure.ac: Sort features name in summary +- mqtt: enable by default - - Same as protocols + No longer considered experimental. - Closes #5656 + Closes #5858 -- [Matthias Naegler brought this change] +- [Michael Baentsch brought this change] - cmake: fix windows xp build + tls: add CURLOPT_SSL_EC_CURVES and --curves - Reviewed-by: Marcel Raad - Closes #5662 + Closes #5892 -- ngtcp2: update to modified qlog callback prototype - - Closes #5675 +- url: remove funny embedded comments in Curl_disonnect calls -- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle +- [Chris Paulson-Ellis brought this change] + + conn: check for connection being dead before reuse - Added test case 674 to reproduce and verify the bug report. + Prevents incorrect reuse of an HTTP connection that has been prematurely + shutdown() by the server. - Fixes #5665 - Reported-by: NobodyXu on github - Closes #5673 - -- [Baruch Siach brought this change] + Partial revert of 755083d00deb16 + + Fixes #5884 + Closes #5893 - bearssl: fix build with disabled proxy support +Marc Hoersken (29 Aug 2020) +- buildconf: exec autoreconf to avoid additional process - Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is - defined. + Also make buildconf exit with the return code of autoreconf. - Reviewed-by: Nicolas Sterchele - Closes #5666 - -- RELEASE-NOTES: synced + Reviewed-by: Daniel Stenberg + + Follow up to #5853 + Closes #5890 -Jay Satiro (11 Jul 2020) -- [Carlo Marcelo Arenas Belón brought this change] +- CI/azure: no longer ignore results of test 1013 + + Follow up to #5771 + Closes #5889 - cirrus-ci: upgrade 11-STABLE to 11.4 +- docs: add description about CI platforms to CONTRIBUTE.md - Meant to be the last of the 11 series and so make sure that all - other references reflect all 11 versions so they can be retired - together later. + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro - Closes https://github.com/curl/curl/pull/5668 + Closes #5882 -- [Filip Salomonsson brought this change] - - CURLINFO_CERTINFO.3: fix typo +Daniel Stenberg (29 Aug 2020) +- tests/getpart: use MIME::Base64 instead of home-cooked - Closes https://github.com/curl/curl/pull/5655 - -Daniel Stenberg (4 Jul 2020) -- http2: only do the *done() cleanups for HTTP + Since we already use the base64 package since a while back, we can just + as well switch to that here too. - Follow-up to ef86daf4d3 + It also happens to use the exact same function name, which otherwise + causes a run-time warning. - Closes #5650 - Fixes #5646 - -- [Alex Kiernan brought this change] + Reported-by: Marc Hörsken + Fixes #5885 + Closes #5887 - gnutls: repair the build with `CURL_DISABLE_PROXY` +Marcel Raad (29 Aug 2020) +- ntlm: fix condition for curl_ntlm_core usage - `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` - if `CURL_DISABLE_PROXY` is enabled. Repair the build with that - configuration. + `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES + backend is fine, but was excluded before. - Signed-off-by: Alex Kiernan - Closes #5645 - -Alex Kiernan (3 Jul 2020) -- gnutls: Fetch backend when using proxy + This also fixes test 1013 as the condition for SMB support in + configure.ac didn't match the condition in the source code. Now it + does. - Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") - Signed-off-by: Alex Kiernan + Fixes https://github.com/curl/curl/issues/1262 + Closes https://github.com/curl/curl/pull/5771 -Daniel Stenberg (3 Jul 2020) -- [Laramie Leavitt brought this change] - - http2: close the http2 connection when no more requests may be sent +- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode - Well-behaving HTTP2 servers send two GOAWAY messages. The first - message is a warning that indicates that the server is going to - stop accepting streams. The second one actually closes the stream. + The Schannel builds are the most useful to verify as they make the most + use of the Windows API. Classic MinGW doesn't support Unicode at all, + only MinGW-w64 and MSVC do. - nghttp2 reports this state (and the other state of no more stream - identifiers) via the call nghttp2_session_check_request_allowed(). - In this state the client should not create more streams on the - session (tcp connection), and in curl this means that the server - has requested that the connection is closed. + Closes https://github.com/curl/curl/pull/5843 + +- CMake: add option to enable Unicode on Windows - It would be also be possible to put the connclose() call into the - on_http2_frame_recv() function that triggers on the GOAWAY message. + As already existing for winbuild. - This fixes a bug seen when the client sees the following sequence of - frames: + Closes https://github.com/curl/curl/pull/5843 + +Marc Hoersken (29 Aug 2020) +- select: simplify return code handling for poll and select - // advisory GOAWAY - HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] - ... some additional frames + poll and select already return -1 on error according to POSIX, + so there is no need to perform a <0 to -1 conversion in code. - // final GOAWAY - HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] + Also we can just use one check with <= 0 on the return code. - Before this change, curl will attempt to reuse the connection even - after the last stream, will encounter this error: + Assisted-by: Daniel Stenberg + Reviewed-by: Jay Satiro - * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] - * Re-using existing connection! (#0) with host localhost - * Connected to localhost (::1) port 10443 (#0) - * Using Stream ID: 9 (easy handle 0x5595f0a72e30) - > GET /index.html?5 HTTP/2 - > Host: localhost:10443 - > user-agent: curl/7.68.0 - > accept: */* - > - * stopped the pause stream! - * Connection #0 to host localhost left intact - curl: (16) Error in the HTTP2 framing layer + Replaces #5852 + Closes #5880 + +Daniel Stenberg (28 Aug 2020) +- RELEASE-NOTES: synced + +- [Jeroen Ooms brought this change] + + tests: add test1912 with typechecks - This error may posion the connection cache, causing future requests - which resolve to the same curl connection to go through the same error - path. + Validates that gcc-typecheck macros match the new option type API. - Closes #5643 + Closes #5873 -- ftpserver: don't verify SMTP MAIL FROM names +- easyoptions: provide debug function when DEBUGBUILD - Rely on tests asking the names to get refused instead - test servers - should be as dumb as possible. Edited test 914, 955 and 959 accordingly. + ... not CURLDEBUG as they're not always set in conjunction. - Closes #5639 + Follow-up to 6ebe63fac23f38df + + Fixes #5877 + Closes #5878 -- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated +Marc Hoersken (28 Aug 2020) +- sockfilt: handle FD_CLOSE winsock event on write socket - This came up in #5640. It make sense to clarify this in the docs! + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Reminded-by: Kamil Dudka - Closes #5642 + Follow up to #5867 + Closes #5879 -Kamil Dudka (3 Jul 2020) -- tool_getparam: make --krb option work again +- multi: handle connection state winsock events - It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Bug: https://bugzilla.redhat.com/1833193 - Closes #5640 - -Daniel Stenberg (2 Jul 2020) -- [Jeremy Maitin-Shepard brought this change] + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + + Follow up to #5634 + Closes #5867 - http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages +Daniel Stenberg (28 Aug 2020) +- Curl_pgrsTime - return new time to avoid timeout integer overflow - Confusingly, nghttp2 has two different error code enums: + Setting a timeout to INT_MAX could cause an immediate error to get + returned as timeout because of an overflow when different values of + 'now' were used. - - nghttp2_error, to be used with nghttp2_strerror - - nghttp2_error_code, to be used with nghttp2_http2_strerror + This is primarily fixed by having Curl_pgrsTime() return the "now" when + TIMER_STARTSINGLE is set so that the parent function will continue using + that time. - Closes #5641 + Reported-by: Ionuț-Francisc Oancea + Fixes #5583 + Closes #5847 -Marcel Raad (2 Jul 2020) -- url: silence MSVC warning +- TLS: fix SRP detection by using the proper #ifdefs - Since commit f3d501dc678, if proxy support is disabled, MSVC warns: - url.c : warning C4701: potentially uninitialized local variable - 'hostaddr' used - url.c : error C4703: potentially uninitialized local pointer variable - 'hostaddr' used + USE_TLS_SRP will be true if *any* selected TLS backend can use SRP - That could actually only happen if both `conn->bits.proxy` and - `CURL_DISABLE_PROXY` were enabled. - Initialize it to NULL to silence the warning. + HAVE_OPENSSL_SRP is defined when OpenSSL can use it - Closes https://github.com/curl/curl/pull/5638 - -Daniel Stenberg (1 Jul 2020) -- RELEASE-NOTES: synced - -Version 7.71.1 (30 Jun 2020) + HAVE_GNUTLS_SRP is defined when GnuTLS can use it + + Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is + set if at least one of the supported backends offers SRP. + + Reported-by: Stefan Strogin + Fixes #5865 + Closes #5870 -Daniel Stenberg (30 Jun 2020) -- RELEASE-NOTES: curl 7.71.1 +- [Dan Kenigsberg brought this change] -- THANKS: add contributors to 7.71.1 + docs: SSLCERTS: fix English syntax + + Signed-off-by: Dan Kenigsberg + + Closes #5876 -- scripts/copyright.pl: skip .dcignore +- [Alessandro Ghedini brought this change] -- Revert "multi: implement wait using winsock events" + docs: non-existing macros in man pages - This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. + As reported by man(1) when invoked as: - That commit (from #5397) introduced a regression in 7.71.0. + man --warnings -E UTF-8 -l -Tutf8 -Z >/dev/null - Reported-by: tmkk on github - Fixes #5631 - Closes #5632 + Closes #5846 -- TODO: Add flag to specify download directory +- [Alessandro Ghedini brought this change] -- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection + curl.1: fix typo invokved -> invoked + + Closes #5846 -- cirrus-ci: disable FreeBSD 13 (again) +- buildconf: invoke 'autoreconf -fi' instead - It has been failing for a good while again. This time we better leave it - disabled until we have more reason to believe it behaves. + The custom script isn't necessary anymore - but remains for simplicity + and just invokes autoreconf. - Closes #5628 + Closes #5853 -- ngtcp2: sync with current master +- [Emil Engler brought this change] + + lib: make Curl_gethostname accept a const pointer - ngtcp2 added two new callbacks + The address of that variable never gets changed, only the data in it so + why not make it a "char * const"? - Reported-by: Lucien Zürcher - Fixes #5624 - Closes #5627 + Closes #5866 -- examples/multithread.c: call curl_global_cleanup() +- docs/libcurl: update "Added in" version for curl_easy_option* - Reported-by: qiandu2006 on github - Fixes #5622 - Closes #5623 + Follow-up to 6ebe63fac23f38 -- vtls: compare cert blob when finding a connection to reuse +- scripts: improve the "get latest curl release tag" logic - Reported-by: Gergely Nagy - Fixes #5617 - Closes #5619 - -- RELEASE-NOTES: synced + ... by insiting on it matching "^curl-". -- terminology: call them null-terminated strings +- configure: added --disable-get-easy-options - Updated terminology in docs, comments and phrases to refer to C strings - as "null-terminated". Done to unify with how most other C oriented docs - refer of them and what users in general seem to prefer (based on a - single highly unscientific poll on twitter). + To allow disabling of the curl_easy_option APIs in a build. - Reported-by: coinhubs on github - Fixes #5598 - Closes #5608 + Closes #5365 -- http: fix proxy auth with blank password +- options: API for meta-data about easy options - Regression in 7.71.0 + const struct curl_easyoption *curl_easy_option_by_name(const char *name); - Added test case 346 to verify. + const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); - Reported-by: Kristoffer Gleditsch - Fixes #5613 - Closes #5616 + const struct curl_easyoption * + curl_easy_option_next(const struct curl_easyoption *prev); + + The purpose is to provide detailed enough information to allow for + example libcurl bindings to get option information at run-time about + what easy options that exist and what arguments they expect. + + Assisted-by: Jeroen Ooms + Closes #5365 -- .dcignore: ignore tests and docs directories +- [Eric Curtin brought this change] + + HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 - This is a config file for deepcode.ai, a static code analyzer. + Closes #5871 -Jay Satiro (26 Jun 2020) -- tool_cb_hdr: Fix etag warning output and return code +- RELEASE-NOTES: synced + +Jay Satiro (26 Aug 2020) +- openssl: Fix wincrypt symbols conflict with BoringSSL - - Return 'failure' on failure, to follow the existing style. + OpenSSL undefines the conflicting symbols but BoringSSL does not so we + must do it ourselves. - - Put Warning: and the warning message on the same line. + Reported-by: Samuel Tranchet + Assisted-by: Javier Blazquez - Ref: https://github.com/curl/curl/issues/5610 + Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 + Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 - Closes https://github.com/curl/curl/pull/5612 + Fixes https://github.com/curl/curl/issues/5669 + Closes https://github.com/curl/curl/pull/5857 -Daniel Stenberg (26 Jun 2020) -- CURLOPT_READFUNCTION.3: provide the upload data size up front +Daniel Stenberg (26 Aug 2020) +- socketpair: allow CURL_DISABLE_SOCKETPAIR - Assisted-by: Jay Satiro - Closes #5607 - -- test1539: do a HTTP 1.0 POST without a set size (fails) + ... to completely disable the use of socketpair - Attempt to reproduce #5593. Test case 1514 is very similar but uses - HTTP/1.1 and thus switches to chunked. + Closes #5850 + +- curl_get_line: build only if cookies or alt-svc are enabled - Closes #5595 + Closes #5851 -- [Baruch Siach brought this change] +- [fullincome brought this change] - mbedtls: fix build with disabled proxy support - - Don't reference fields that do not exist. Fixes build failure: + schannel: fix memory leak when using get_cert_location - vtls/mbedtls.c: In function 'mbed_connect_step1': - vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' + The get_cert_location function allocates memory only on success. + Previously get_cert_location was able to allocate memory and return + error. It wasn't obvious and in this case the memory wasn't + released. - Closes #5615 + Fixes #5855 + Closes #5860 -- codeql-analysis.yml: fix the 'languages' setting +- [Emil Engler brought this change] + + git: ignore libtests in 3XXX area - It needs a 'with:' in front of it. + Currently the file tests/libtest/lib3010 is not getting + ignored by git. This fixes it by adding the 3XXX area to + the according .gitignore file. + + Closes #5859 -GitHub (26 Jun 2020) -- [Daniel Stenberg brought this change] +- [Emil Engler brought this change] - gtihub: codeql-analysis.yml + doh: add error message for DOH_DNS_NAME_TOO_LONG - enables code security scanning with github actions + When this error code was introduced in b6a53fff6c1d07e8a9, it was + forgotten to be added in the errors array and doh_strerror function. + + Closes #5863 -Daniel Stenberg (25 Jun 2020) -- tests: verify newline in username and password for HTTP +- ngtcp2: adapt to the new pkt_info arguments - test 1296 is a simply command line test + Guidance-by: Tatsuhiro Tsujikawa - test 1910 is a libcurl test including a redirect + Closes #5864 -- url: allow user + password to contain "control codes" for HTTP(S) +- winbuild/README.md: make visible - Reported-by: Jon Johnson Jr - Fixes #5582 - Closes #5592 + Follow-up to be753add31c2d8c -- escape: make the URL decode able to reject only %00 bytes +- winbuild: convert the instruction text to README.md - ... or all "control codes" or nothing. + Closes #5861 + +- lib1560: verify "redirect" to double-slash leading URL - Assisted-by: Nicolas Sterchele + Closes #5849 -- http2: set the correct URL in pushed transfers +Marc Hoersken (25 Aug 2020) +- multi: expand pre-check for socket readiness - ...previously CURLINFO_EFFECTIVE_URL would report the URL of the - original "mother transfer", not the actually pushed resource. + Check readiness of all sockets before waiting on them + to avoid locking in case the one-time event FD_WRITE + was already consumed by a previous wait operation. - Reported-by: Jonathan Cardoso Machado - Fixes #5589 - Closes #5591 + More information about WinSock network events: + https://docs.microsoft.com/en-us/windows/win32/api/ + winsock2/nf-winsock2-wsaeventselect#return-value + + Closes #5634 -Jay Satiro (25 Jun 2020) -- [Javier Blazquez brought this change] +- [rcombs brought this change] - openssl: Fix compilation on Windows when ngtcp2 is enabled + multi: implement wait using winsock events - - Include wincrypt before OpenSSL includes so that the latter can - properly handle any conflicts between the two. + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. - Closes https://github.com/curl/curl/pull/5606 - -Daniel Stenberg (25 Jun 2020) -- test543: extended to verify zero length input + A previous version of this patch failed to account for how in WinSock, + FD_WRITE is set only once when writing becomes possible and not again + until after a send has failed due to the buffer filling. This contrasts + to how FD_READ and FD_OOB continue to be set until the conditions they + refer to no longer apply. This meant that if a user wrote some data to + a socket, but not enough data to completely fill its send buffer, then + waited on that socket to become writable, we'd erroneously stall until + their configured timeout rather than returning immediately. - As was reported in #5601 - -- escape: zero length input should return a zero length output + This version of the patch addresses that issue by checking each socket + we're waiting on to become writable with select() before the wait, and + zeroing the timeout if it's already writable. - Regression added in 7.71.0. + Assisted-by: Marc Hörsken + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Tested-by: Gergely Nagy + Tested-by: Rasmus Melchior Jacobsen + Tested-by: Tomas Berger - Fixes #5601 - Reported-by: Kristoffer Gleditsch - Closes #5602 + Replaces #5397 + Reverts #5632 + Closes #5634 -- Curl_inet_ntop: always check the return code +- select: reduce duplication of Curl_poll in Curl_socket_check - Reported-by: Siva Sivaraman - Fixes #5412 - Closes #5597 - -- sendf: improve the message on client write errors + Change Curl_socket_check to use select-fallback in Curl_poll + instead of implementing it in Curl_socket_check and Curl_poll. - Replace "Failed writing body (X != Y)" with - "Failure writing output to destination". Possibly slightly less cryptic. + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Reported-by: coinhubs on github - Fixes #5594 - Closes #5596 - -- RELEASE-NOTES: synced - -- curlver: start working on 7.71.1 - -- [Denis Baručić brought this change] + Replaces #5262 and #5492 + Closes #5707 - DYNBUF.md: fix a typo: trail => tail +- select: fix poll-based check not detecting connect failure - Closes #5599 - -Version 7.71.0 (23 Jun 2020) - -Daniel Stenberg (23 Jun 2020) -- RELEASE-NOTES: curl 7.71.0 release - -- THANKS: curl 7.71.0 additions - -- url: make sure pushed streams get an allocated download buffer + This commit changes Curl_socket_check to use POLLPRI to + check for connect failure on the write socket, because + POLLPRI maps to fds_err. This is in line with select(2). - Follow-up to c4e6968127e876b0 + The select-based socket check correctly checks for connect + failures by adding the write socket also to fds_err. - When a new transfer is created, as a resuly of an acknowledged push, - that transfer needs a download buffer allocated. + The poll-based implementation (which internally can itself + fallback to select again) did not previously check for + connect failure by using POLLPRI with the write socket. - Closes #5590 - -Jay Satiro (22 Jun 2020) -- openssl: Don't ignore CA paths when using Windows CA store + See the follow up commit to this for more information. - This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does - not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default - locations. Instead the CA store can now be used at the same time. + This commit makes sure connect failures can be detected + and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. - The change is due to the impending release. The issue is still being - discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and - is now documented as experimental. + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Ref: bc052cc (parent commit) - Ref: https://github.com/curl/curl/issues/5585 + Replaces #5509 + Prepares #5707 -- tool_operate: Don't use Windows CA store as a fallback - - Background: - - 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate - store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides - CURLOPT_CAINFO if both are set. The curl tool will fall back to - CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set - via CURLOPT_CAINFO. - - Problem: +- select.h: make socket validation macros test for INVALID_SOCKET - libcurl may be built with hardcoded paths to a certificate bundle or - directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are - ignored. - - Solution: - - A solution is still being discussed but since there's an impending - release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + With Winsock the valid range is [0..INVALID_SOCKET-1] according to + https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 - Ref: https://github.com/curl/curl/issues/5585 - -- openssl: Fix CA fallback logic for OpenSSL 3.0 build - - Prior to this change I assume a build error would occur when - CURL_CA_FALLBACK was used. + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5587 - -Daniel Stenberg (22 Jun 2020) -- copyright: update mismatched copyright years - -- test1460: verify that -Ji is not ok + Closes #5760 -- tool_getparam: -i is not OK if -J is used +Daniel Stenberg (24 Aug 2020) +- docs: --output-dir is added in 7.73.0, nothing else - Reported-by: sn on hackerone - Bug: https://curl.haxx.se/docs/CVE-2020-8177.html - -- [Peter Wu brought this change] + Follow-up to 5620d2cc78c0 - CMake: ignore INTERFACE_LIBRARY targets for pkg-config file +- curl: add --output-dir - Reviewed-by: Marcel Raad - Fixes #5512 - Closes #5517 - -- [Valentyn Korniienko brought this change] - - multibyte: Fixed access-> waccess to file for Windows Plarform + Works with --create-dirs and with -J - Reviewed-by: Marcel Raad - Closes #5580 - -- altsvc: bump to h3-29 + Add test 3008, 3009, 3011, 3012 and 3013 to verify. - Closes #5584 + Closes #5637 -- urlglob: treat literal IPv6 addresses with zone IDs as a host name +- configure: fix pkg-config detecting wolfssl - ... and not as a "glob". Now done by passing the supposed host to the - URL parser which supposedly will do a better job at identifying "real" - numerical IPv6 addresses. + When amending the include path with "/wolfssl", this now properly strips + off all whitespace from the path variable! Previously this would lead to + pkg-config builds creating bad command lines. - Reported-by: puckipedia on github - Fixes #5576 - Closes #5579 + Closes #5848 -- test1179: verify error message for non-existing cmdline option +- [Michael Musset brought this change] -- tool_getparam: repair the error message for unknown flag + sftp: add the option CURLKHSTAT_FINE_REPLACE - Follow-up to 9e5669f3880674 - Detected by Coverity CID 1464582 ("Logically dead code") + Replace the old fingerprint of the host with a new. - Closes #5577 + Closes #5685 -- FILEFORMAT: describe verify/stderr +- RELEASE-NOTES: synced + + The next release is now to become 7.73.0 -- connect: improve happy eyeballs handling +- checksrc: verify do-while and spaces between the braces - For QUIC but also for regular TCP when the second family runs out of IPs - with a failure while the first family is still trying to connect. + Updated mprintf.c to comply - Separated the timeout handling for IPv4 and IPv6 connections when they - both have a number of addresses to iterate over. - -- ngtcp2: never call fprintf() in lib code in release version + Closes #5845 -- ngtcp2: fix happy eyeballs quic connect crash +- curl: support XDG_CONFIG_HOME to find .curlrc - Reported-by: Peter Wu - Fixes #5565 - Closes #5568 - -- select: remove the unused ELAPSED_MS() macro + Added test433 to verify. Updated documentation. - Closes #5573 - -Marc Hoersken (17 Jun 2020) -- [rcombs brought this change] + Reviewed-by: Jay Satiro + Suggested-by: Eli Schwartz + Fixes #5829 + Closes #5837 - multi: implement wait using winsock events +- etag: save and use the full received contents - This avoids using a pair of TCP ports to provide wakeup functionality - for every multi instance on Windows, where socketpair() is emulated - using a TCP socket on loopback which could in turn lead to socket - resource exhaustion. + ... which makes it support weak tags and non-standard etags too! - Reviewed-by: Gergely Nagy - Reviewed-by: Marc Hörsken + Added test case 347 to verify blank incoming ETag: - Closes #5397 + Fixes #5610 + Closes #5833 -Daniel Stenberg (17 Jun 2020) -- manpage: add three missing environment variables +- setopt: if the buffer exists, refuse the new BUFFERSIZE - CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE + The buffer only exists during transfer and then we shouldn't change the + size (the setopt is not documented to work then). - Closes #5571 + Reported-by: Harry Sintonen + Closes #5842 -- RELEASE-NOTES: synced +- [COFFEETALES brought this change] -- configure: for wolfSSL, check for the DES func needed for NTLM + sftp: add new quote commands 'atime' and 'mtime' - Also adds pkg-config support for the wolfSSL detection. - -- [Ruurd Beerstra brought this change] + Closes #5810 - ntlm: enable NTLM support with wolfSSL - - When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* - functions that OpenSSL has. This change take advantage of that. +- CURLE_PROXY: new error code - Co-authored-by: Daniel Stenberg - Closes #5556 - Fixes #5548 - -- http: move header storage to Curl_easy from connectdata + Failures clearly returned from a (SOCKS) proxy now causes this return + code. Previously the situation was not very clear as what would be + returned and when. - Since the connection can be used by many independent requests (using - HTTP/2 or HTTP/3), things like user-agent and other transfer-specific - data MUST NOT be kept connection oriented as it could lead to requests - getting the wrong string for their requests. This struct data was - lingering like this due to old HTTP1 legacy thinking where it didn't - mattered.. + In addition: when this error code is returned, an application can use + CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then + returns a value from the new 'CURLproxycode' enum. - Fixes #5566 - Closes #5567 + Closes #5770 -- CODE_REVIEW.md: how to do code reviews in curl +- runtests: make cleardir() erase dot files too - Assisted-by: Daniel Gustafsson - Assisted-by: Rich Salz - Assisted-by: Hugo van Kemenade - Assisted-by: James Fuller - Assisted-by: Marc Hörsken - Assisted-by: Jay Satiro + Because test cases might use dot files. - Closes #5555 + Closes #5838 -- altsvc: remove the num field from the altsvc struct +- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses - It was superfluous since we have the list.size alredy + Also: the current behavior is now documented in the curl.1 and + CURLOPT_NOPROXY.3 man pages. - Reported-by: Jay Satiro - Fixes #5553 - Closes #5563 + Reported-by: Andrew Barnes + Closes #5745 + Closes #5841 -- version.d: expanded and alpha-sorted +Viktor Szakats (22 Aug 2020) +- Makefile.m32: add ability to override zstd libs [ci skip] - Added a few missing features not previously mentioned. Ordered them - alphabetically. + Similarly to brotli, where this was already possible. + E.g. it allows to link zstd statically to libcurl.dll. - Closes #5558 - -- ABI.md: rename to .md and polish the markdown + Ref: https://github.com/curl/curl-for-win/issues/12 + Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 - Closes #5562 + Closes https://github.com/curl/curl/pull/5840 -- HELP-US: add a section for "smaller tasks" +Daniel Stenberg (21 Aug 2020) +- runtests: avoid 'fail to start' repeated messages in attempt loops - The point of this section is to meet the CII Best Practices gold level - critera: + Closes #5834 + +- runtests: clear pid variables when failing to start a server - "The project MUST clearly identify small tasks that can be performed by - new or casual contributors" + ... as otherwise the parent doesn't detect the failure and believe it + actually worked to start. - Closes #5560 + Reported-by: Christian Weisgerber + Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html + Closes #5834 -- TODO: retry on the redirected-to URL +- TODO: Virtual external sockets - Closes #5462 - -- mailmap: Nicolas Sterchele + Closes #5835 -- [Nicolas Sterchele brought this change] +- [Don J Olmstead brought this change] - TODO: remove 19.3 section title - - Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on - curl documentation website + dist: add missing CMake Find modules to the distribution - Closes #5561 - -- [Martin V brought this change] + Closes #5836 - test1560: avoid possibly negative association in wording +- RELEASE-NOTES: synced - Closes #5549 + ... and version bumped to 7.72.1 -- share: don't set the share flag it something fails +- tls: provide the CApath verbose log on its own line - When asking for a specific feature to be shared in the share object, - that bit was previously set unconditionally even if the shared feature - failed or otherwise wouldn't work. + ... not newline separated from the previous line. This makes it output + asterisk prefixed properly like other verbose putput! - Closes #5554 + Reported-by: jmdavitt on github + Fixes #5826 + Closes #5827 -- buildconf: remove -print from the find command that removes files - - It's just too annoying and unnecessary to get a long list of files shown +Version 7.72.0 (19 Aug 2020) +Daniel Stenberg (19 Aug 2020) - RELEASE-NOTES: synced + + The curl 7.72.0 release -- wording: avoid blacklist/whitelist stereotypes +- THANKS: add names from curl 7.72.0 release + +Jay Satiro (18 Aug 2020) +- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions - Instead of discussing if there's value or meaning (implied or not) in - the colors, let's use words without the same possibly negative - associations. + Reported-by: plujon@users.noreply.github.com - Closes #5546 + Closes https://github.com/curl/curl/issues/5488 -Jay Satiro (9 Jun 2020) -- tool_getparam: fix memory leak in parse_args +Daniel Stenberg (17 Aug 2020) +- Curl_easy: remember last connection by id, not by pointer - Prior to this change in Windows Unicode builds most parsed options would - not be freed. + CVE-2020-8231 - Found using _CrtDumpMemoryLeaks(). + Bug: https://curl.haxx.se/docs/CVE-2020-8231.html - Ref: https://github.com/curl/curl/issues/5545 + Reported-by: Marc Aldorasi + Closes #5824 -Daniel Stenberg (8 Jun 2020) -- socks: detect connection close during handshake +- examples/rtsp.c: correct the copyright year + +- RELEASE-PROCEDURE.md: add more future release dates + +- [H3RSKO brought this change] + + docs: change "web site" to "website" - The SOCKS4/5 state machines weren't properly terminated when the proxy - connection got closed, leading to a busy-loop. + According to wikipedia: - Reported-By: zloi-user on github - Fixes #5532 - Closes #5542 + While "web site" was the original spelling, this variant has become + rarely used, and "website" has become the standard spelling + + Closes #5822 -- [James Fuller brought this change] +- [Bevan Weiss brought this change] - multi: add defensive check on data->multi->num_alive + CMake: don't complain about missing nroff - Closes #5540 - -- Curl_addrinfo: use one malloc instead of three + The curl_nroff_check() was always being called, and complaining if + *NROFF wasn't found, even when not making the manual. - To reduce the amount of allocations needed for creating a Curl_addrinfo - struct, make a single larger malloc instead of three separate smaller - ones. + Only check for nroff (and complain) if actually making the manual - Closes #5533 + Closes #5817 -- [Alessandro Ghedini brought this change] +- [Brian Inglis brought this change] - quiche: update SSLKEYLOGFILE support + libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin - quiche now requires the application to explicitly set the keylog path - for each connection, rather than reading the environment variable - itself. + copy the LDFLAGS approach for adding same option with `libhostname` in + `libtest/Makefile.am`: - Closes #5541 + - init `libstubgss_la_LDFLAGS_EXTRA` variable, + - add option to variable inside conditional, + - use variable in `libstubgss_la_LDFLAGS` + + Fixes #5819 + Closes #5820 -- tests: add two simple tests for --login-options +- docs: clarify MAX_SEND/RECV_SPEED functionality - Test 895 and 896 - as a follow-up to a3e972313b + ... in particular what happens if the maximum speed limit is set to a + value that's smaller than the transfer buffer size in use. - Closes #5539 + Reported-by: Tomas Berger + Fixes #5788 + Closes #5813 -- ngtcp2: update with recent API changes +- test1140: compare stdout - Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. + To make problems more immediately obvious when tests fail. - Assisted-by: Tatsuhiro Tsujikawa - Closes #5538 - -- [James Fuller brought this change] + Closes #5814 - socks: remove unreachable breaks in socks.c and mime.c +- asyn-ares: correct some bad comments - Closes #5537 + Closes #5812 -- tool_cfgable: free login_options at exit - - Memory leak - Reported-by: Geeknik Labs - Fixes #5535 - Closes #5536 +- [Emil Engler brought this change] -- libssh2: keep sftp errors as 'unsigned long' - - Remove weird work-around for storing the SFTP errors as int instead of - the "unsigned long" that libssh2 actually returns for SFTP errors. + docs: Add video link to docs/CONTRIBUTE.md - Closes #5534 + Closes #5811 -Marc Hoersken (6 Jun 2020) -- timeouts: move ms timeouts to timediff_t from int and long - - Now that all functions in select.[ch] take timediff_t instead - of the limited int or long, we can remove type conversions - and related preprocessor checks to silence compiler warnings. +- curl-config: ignore REQUIRE_LIB_DEPS in --libs output - Avoiding conversions from time_t was already done in 842f73de. + Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is + not considered for the --libs output. - Based upon #5262 - Supersedes #5214, #5220 and #5221 - Follow up to #5343 and #5479 - Closes #5490 + Reported-by: ramsay-jones on github + Assisted-by: Brian Inglis and Ken Brown + Fixes #5793 + Closes #5808 -Daniel Stenberg (6 Jun 2020) -- [François Rigault brought this change] +- copyright: update/correct the year range on a few files - openssl: set FLAG_TRUSTED_FIRST unconditionally - - On some systems, openssl 1.0.x is still the default, but it has been - patched to contain all the recent security fixes. As a result of this - patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be - defined, while the previous behavior of openssl to not look at trusted - chains first, remains. +- scripts/copyright.pl: ignore .muse files + +- [Emil Engler brought this change] + + multi: Remove 10-year old out-commented code - Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to - probe for the behavior of openssl based on the existence ofmacros. + The code hasn't been touched since 2010-08-18 - Closes #5530 + Closes #5805 -- server/util: fix logmsg format using curl_off_t argument +- KNOWN_BUGS: A shared connection cache is not thread-safe - ... this caused segfaults on armv7. + Closes #4915 + Closes #5802 + +- CONTRIBUTE: extend git commit message description - Regression added in dd0365d560aea5a (7.70.0) + In particular how the first line works. - Reviewed-by: Jay Satiro - Closes #5529 + Closes #5803 - RELEASE-NOTES: synced -- [Cherish98 brought this change] +- [Stefan Yohansson brought this change] - socks: fix expected length of SOCKS5 reply + transfer: move retrycount from connect struct to easy handle - Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the - reply ATYP is X'01'. This resulted in erroneously expecting more bytes - when the request length is greater than the reply length (e.g., when - remotely resolving the hostname). + This flag was applied to the connection struct that is released on + retry. These changes move the retry counter into Curl_easy struct that + lives across retries and retains the new connection. - Closes #5527 + Reported-by: Cherish98 on github + Fixes #5794 + Closes #5800 -Marc Hoersken (5 Jun 2020) -- .gitignore: add directory containing the stats repo +- libssh2: s/ssherr/sftperr/ - Since the new curl/stats repository is designed to be - checked out into the curl repository working tree as stats/ - it should be on the ignore list to aid in commit staging. - -Daniel Stenberg (5 Jun 2020) -- [Adnan Khan brought this change] - - HTTP3.md: clarify cargo build directory + The debug output used ssherr instead of sftperr which not only outputs + the wrong error code but also casues a warning on Windows. - Cargo needs to be called from within the 'quiche' directory. + Follow-up to 7370b4e39f1 - Closes #5522 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 + Closes #5799 -- user-agent.d: spell out what happens given a blank argument +- ftp: don't do ssl_shutdown instead of ssl_close - Closes #5525 - -- trailers: switch h1-trailer logic to use dynbuf + The shutdown function is for downgrading a connection from TLS to plain, + and this is not requested here. - In the continued effort to remove "manual" realloc schemes. + Have ssl_close reset the TLS connection state. - Closes #5524 - -- CURLINFO_ACTIVESOCKET.3: clarify the description + This partially reverts commit f002c850d98d - Reported-by: Jay Satiro - Fixes #5299 - Closes #5520 - -- mailmap: Don J Olmstead + Reported-by: Rasmus Melchior Jacobsen + Reported-by: Denis Goleshchikhin + Fixes #5797 -- configure: only strip first -L from LDFLAGS - - In the logic that works out if a given OpenSSL path works, it stripped - off a possibly leading -L flag using an incorrect sed pattern which - would remove all instances of -L in the string, including if the path - itself contained that two-letter sequence! +Marc Hoersken (9 Aug 2020) +- CI/azure: fix test outcome values and use latest API version - The same pattern was used and is now updated in multiple places. Now it - only removes -L if it starts the strings. + This makes sure that tests ignored or skipped are not shown + just in the category "Other", but with their correct state. - Reported-by: Mohamed Osama - Fixes #5519 - Closes #5521 + Closes #5796 -Peter Wu (4 Jun 2020) -- quiche: advertise draft 28 support +- CI/azure: show runtime stats to investigate slowness - Fix the verbose message while at it, quiche currently supports draft - 27 and draft 28 simultaneously. + Also avoid naming conflict of TFLAGS env and tflags variables. - Closes #5518 + Closes #5776 -Daniel Stenberg (4 Jun 2020) -- KNOWN_BUGS: RTSP authentication breaks without redirect support +Daniel Stenberg (8 Aug 2020) +- TLS naming: fix more Winssl and Darwinssl leftovers - Closes #4750 - -Jay Satiro (4 Jun 2020) -- projects: Add crypt32.lib to dependencies for all OpenSSL configs + The CMake option is now called CMAKE_USE_SCHANNEL - Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO - need crypt32. + The winbuild flag is USE_SCHANNEL - Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. + The CI jobs and build scripts only use the new names and the new name + options - The changes that are in this commit were made by script. + Tests now require 'Schannel' (when necessary) - Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 + Closes #5795 + +- smtp_parse_address: handle blank input string properly - Closes https://github.com/curl/curl/pull/5516 + Closes #5792 -Marc Hoersken (3 Jun 2020) -- CI/macos: fix 'is already installed' errors by using bundle +- runtests: run the DICT server on a random port number - Avoid failing CI builds due to nghttp2 being already installed. + Removed support for -b (base port number) - Closes #5513 - -Daniel Stenberg (3 Jun 2020) -- altsvc: fix 'dsthost' may be used uninitialized in this function + Closes #5783 - RELEASE-NOTES: synced -- urldata: let the HTTP method be in the set.* struct - - When the method is updated inside libcurl we must still not change the - method as set by the user as then repeated transfers with that same - handle might not execute the same operation anymore! - - This fixes the libcurl part of #5462 +- runtests: move the TELNET server to a dynamic port - Test 1633 added to verify. + Rename the port variable to TELNETPORT to better match the existing + pattern. - Closes #5499 + Closes #5785 -- hostip: fix the memory-leak introduced in 67d2802 +- ngtcp2: adapt to error code rename - Fixes #5503 - Closes #5504 + Closes #5786 -- test970: make it require proxy support - - This test verifies the -w %json output and the test case includes a full - generated "blob". If there's no proxy support built into libcurl, it - will return an error for proxy related info variables and they will not - be included in the json, thus causing a mismatch and this test fails. +- runtests: move the smbserver to use a dynamic port number - Reported-by: Marc Hörsken - Fixes #5501 - Closes #5502 - -- [Radoslav Georgiev brought this change] + Closes #5782 - examples/http2-down/upload: add error checks - - If `index.html` does not exist in the directory from which the example - is invoked, the fopen(upload, "rb") invocation in `setup` would fail, - returning NULL. This value is subsequently passed as the FILE* argument - of the `fread` invocation in the `read_callback` function, which is the - actual cause of the crash (apparently `fread` assumes that argument to - be non-null). - - In addition, mitigate some possible crashes of similar origin. +- runtests: run the http2 tests on a random port number - Closes #5463 - -- [kotoriのねこ brought this change] + Closes #5779 - examples/ephiperfifo: turn off interval when setting timerfd +- gtls: survive not being able to get name/issuer - Reported-by: therealhirudo on github - Fixes #5485 - Closes #5497 - -- [Saleem Abdulrasool brought this change] + Closes #5778 - vtls: repair the build with `CURL_DISABLE_PROXY` - - `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` - is enabled. Repair the build with that configuration. +- runtests: move the gnutls-serv tests to a dynamic port - Follow-up to f3d501dc67 + Affects test 320, 321, 322 and 324. - Closes #5498 + Closes #5778 -- transfer: remove k->str NULL check - - "Null-checking k->str suggests that it may be null, but it has already - been dereferenced on all paths leading to the check" - and it can't - legally be NULL at this point. Remove check. - - Detected by Coverity CID 1463884 +- runtests: support dynamicly base64 encoded sections in tests - Closes #5495 - -Marc Hoersken (1 Jun 2020) -- select: always use Sleep in Curl_wait_ms on Win32 + This allows us to make test cases to use base64 at run-time and still + use and verify information determined at run-time, such as the IMAP test + server's port number in test 842. - Since Win32 almost always will also have USE_WINSOCK, - we can reduce complexity and always use Sleep there. + This change makes 12 tests run again that basically never ran since we + moved to dynamic port numbers. - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ftpserver.pl is adjusted to load test instructions and test number from + the preprocessed test file. - Follow up to #5343 - Closes #5489 - -Daniel Stenberg (31 May 2020) -- conncache: download buffer needs +1 size for trailing zero + FILEFORMAT.md now documents the new base64 encoding syntax. - Follow-up to c4e6968127e - Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 + Reported-by: Marcel Raad + Fixes #5761 + Closes #5775 -Marc Hoersken (31 May 2020) -- azure: use matrix strategy to avoid configuration redundancy - - This also includes the following changes: - - - Use the same timeout for all jobs on Linux (60 minutes) - and Windows (90 minutes) - - Use CLI stable apt-get install -y instead of apt install - which warns about that and run apt-get update first - - Enable MQTT for Windows msys2 builds instead of - legacy msys1 builds - - Add ./configure --prefix parameter to the msys2 builds - - The MSYSTEM environment variable is now preset inside - the container images for the msys2 builds +- curl.1: add a few missing valid exit codes - Note: on Azure Pipelines the matrix strategy is basically - just a simple list of job copies and not really a matrix. + 93 - 96 can be returned as well. - Closes #5468 + Closes #5777 -Daniel Stenberg (30 May 2020) -- build: disable more code/data when built without proxy support - - Added build to travis to verify +- TODO: Use multiple parallel transfers for a single download - Closes #5466 + Closes #5774 -- url: alloc the download buffer at transfer start - - ... and free it as soon as the transfer is done. It removes the extra - alloc when a new size is set with setopt() and reduces memory for unused - easy handles. - - In addition: the closure_handle now doesn't use an allocated buffer at - all but the smallest supported size as a stack based one. +- TODO: Set the modification date on an uploaded file - Closes #5472 + Closes #5768 -- timeouts: change millisecond timeouts to timediff_t from time_t - - For millisecond timers we like timediff_t better. Also, time_t can be - unsigned so returning a negative value doesn't work then. +- [Thomas M. DuBuisson brought this change] + + CI: Add muse CI config - Closes #5479 + Closes #5772 -Marc Hoersken (30 May 2020) -- select: add overflow checks for timeval conversions +- [Thomas M. DuBuisson brought this change] + + travis/script.sh: fix use of `-n' with unquoted envvar - Using time_t and suseconds_t if suseconds_t is available, - long on Windows (maybe others in the future) and int elsewhere. + Shellcheck tells us "-n doesn't work with unquoted arguments. quote or + use [[ ]]." - Also handle case of ULONG_MAX being greater or equal to INFINITE. + And testing shows: - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ``` + docker run --rm -it ubuntu bash + root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" + I ran + root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" + root@fe85ce156856:/# + ``` - Part of #5343 + Closes #5773 -- select: use timediff_t instead of time_t and int for timeout_ms +- h2: repair trailer handling - Make all functions in select.[ch] take timeout_ms as timediff_t - which should always be large enough and signed on all platforms - to take all possible timeout values and avoid type conversions. + The previous h2 trailer fix in 54a2b63 was wrong and caused a + regression: it cannot deal with trailers immediately when read since + they may be read off the connection by the wrong 'data' owner. - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + This change reverts the logic back to gathering all trailers into a + single buffer, like before 54a2b63. - Replaces #5107 and partially #5262 - Related to #5240 and #5286 - Closes #5343 + Reported-by: Tadej Vengust + Fixes #5663 + Closes #5769 -- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' +Viktor Szakats (3 Aug 2020) +- windows: disable Unix Sockets for old mingw - GCC 10 warns about this with warning: implicit conversion - from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] + Classic mingw and 10y+ old versions of mingw-w64 don't ship with + Windows headers having the typedef necessary for Unix Sockets + support, so try detecting these environments to disable this + feature. - Since 'expected_result' is not really of type 'CURLcode' and - it is not exposed in any way, we can just use 'SANITIZEcode'. + Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #5476 + Fixes #5674 + Closes #5758 -- tests/libtest: fix undefined reference to 'curlx_win32_fopen' - - Since curl_setup.h now makes use of curlx_win32_fopen for Win32 - builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, - we need to include the relevant files for tests using fopen, - because the libtest sources are also including curl_setup.h +Marcel Raad (3 Aug 2020) +- test1908: treat file as text - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg + Fixes the line endings on Windows. - Follow up to #3784 (ffdddb45d9) - Closes #5475 + Closes https://github.com/curl/curl/pull/5767 -- appveyor: add non-debug plain autotools-based build - - This should enable us to catch linking issues with the - testsuite early, like the one described/fixed in #5475. +- TrackMemory tests: ignore realloc and free in getenv.c - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad + These are only called for WIN32. - Closes #5477 + Closes https://github.com/curl/curl/pull/5767 + +Daniel Stenberg (3 Aug 2020) +- tests/FILEFORMAT.md: mention %HTTP2PORT -Daniel Stenberg (29 May 2020) - RELEASE-NOTES: synced -- Revert "buildconf: use find -execdir" +- tlsv1.3.d. only for TLS-using connections - This partially reverts commit c712009838f44211958854de431315586995bc61. + ... and rephrase that "not all" TLS backends support it. - Keep the ares_ files removed but bring back the older way to run find, - to make it work with busybox's find, as apparently that's being used. + Closes #5764 + +- tls-max.d: this option is only for TLS-using connections - Reported-by: Max Peal - Fixes #5483 - Closes #5484 + Ref: #5763 + Closes #5764 -- server/sws: fix asan warning on use of uninitialized variable +Marcel Raad (2 Aug 2020) +- [Cameron Cawley brought this change] -- libssh2: improved error output for wrong quote syntax - - Reported-by: Werner Stolz + tool_doswin: Simplify Windows version detection - Closes #5474 + Closes https://github.com/curl/curl/pull/5754 -- mk-lib1521: generate code for testing BLOB options as well - - Follow-up to cac5374298b3 - - Closes #5478 +- [Cameron Cawley brought this change] -- configure: repair the check if argv can be written to - - Due to bad escaping of the test code, the test wouldn't build and thus - result in a negative test result, which would lead to the unconditional - assumption that overwriting the arguments doesn't work and thus curl - would never hide credentials given in the command line, even when it - would otherwise be possible. - - Regression from commit 2d4c2152c (7.60.0) + win32: Add Curl_verify_windows_version() to curlx - Reported-by: huzunhao on github - Fixes #5470 - Closes #5471 + Closes https://github.com/curl/curl/pull/5754 -Peter Wu (28 May 2020) -- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - - Otherwise the build might fail due to missing source files, as - demonstrated by the recent keylog.c addition on an existing build dir. +- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL - Closes #5469 - -Daniel Stenberg (28 May 2020) -- urldata: fix comments: Curl_done() is called multi_done() now + This makes the tests that require the OpenSSL feature also run for + those two compatible libraries. - ... since 575e885db + Closes https://github.com/curl/curl/pull/5762 -Peter Wu (27 May 2020) -- ngtcp2: use common key log routine for better thread-safety +Daniel Stenberg (1 Aug 2020) +- multi: Condition 'extrawait' is always true - Tested with ngtcp2 built against the OpenSSL library. Additionally - tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + Reported by Codacy. - The TLS backend (independent of QUIC) may or may not already have opened - the keylog file before. Therefore Curl_tls_keylog_open is always called - to ensure the file is open. + Reviewed-by: Marcel Raad + Closes #5759 -- wolfssl: add SSLKEYLOGFILE support +Marcel Raad (1 Aug 2020) +- openssl: fix build with LibreSSL < 2.9.1 - Tested following the same curl and tshark commands as in commit - "vtls: Extract and simplify key log file handling from OpenSSL" using - WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with - `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. + `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were + introduced in LibreSSL 2.9.1 [0]. - Full support for this feature requires certain wolfSSL build options, - see "Availability note" in lib/vtls/wolfssl.c for details. + [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef - Closes #5327 + Closes https://github.com/curl/curl/pull/5757 -- vtls: Extract and simplify key log file handling from OpenSSL - - Create a set of routines for TLS key log file handling to enable reuse - with other TLS backends. Simplify the OpenSSL backend as follows: - - - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. - - Do not perform dynamic memory allocation when preparing a log entry. - Unless the TLS specifications change we can suffice with a reasonable - fixed-size buffer. - - Simplify state tracking when SSL_CTX_set_keylog_callback is - unavailable. My original sslkeylog.c code included this tracking in - order to handle multiple calls to SSL_connect and detect new keys - after renegotiation (via SSL_read/SSL_write). For curl however we can - be sure that a single master secret eventually becomes available - after SSL_connect, so a simple flag is sufficient. An alternative to - the flag is examining SSL_state(), but this seems more complex and is - not pursued. Capturing keys after server renegotiation was already - unsupported in curl and remains unsupported. - - Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f - (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) - against an OpenSSL 1.1.1f server configured with: +Daniel Stenberg (1 Aug 2020) +- [Marc Aldorasi brought this change] + + multi_remove_handle: close unused connect-only connections - # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 - openssl s_server -www -tls1 - # Likewise, but fail the server handshake. - openssl s_server -www -tls1 -Verify 2 - # TLS 1.3 test. No need to test the failing server handshake. - openssl s_server -www -tls1_3 + Previously any connect-only connections in a multi handle would be kept + alive until the multi handle was closed. Since these connections cannot + be re-used, they can be marked for closure when the associated easy + handle is removed from the multi handle. - Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly - written using Wireshark. For the first and third case, expect four - matches per connection (decrypted Server Finished, Client Finished, HTTP - Request, HTTP Response). For the second case where the handshake fails, - expect a decrypted Server Finished only. + Closes #5749 + +- checksrc: invoke script with -D to find .checksrc proper - tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ - -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ - -dtls.port==4433,http -ohttp.desegment_body:FALSE \ - -Y 'tls.handshake.verify_data or http' + Without the -D command line option, checksrc.pl won't know which + directory to load the ".checksrc" file from when building out of the + source tree. - A single connection can easily be identified via the `tcp.stream` field. - -Daniel Stenberg (27 May 2020) -- FILEFORMAT: add more features that tests can depend on + Reported-by: Marcel Raad + Fixes #5715 + Closes #5755 -- [Michael Kaufmann brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - transfer: close connection after excess data has been read - - For HTTP 1.x, it's a protocol error when the server sends more bytes - than announced. If this happens, don't reuse the connection, because the - start position of the next response is undefined. + buildconf: retire ares buildconf invocation - Closes #5440 + no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 -- [Estanislau Augé-Pujadas brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - Revert "ssh: ignore timeouts during disconnect" - - This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in - curl 7.54.1. + buildconf: excempt defunct reference to ACLOCAL_FLAGS - Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html - Closes #5465 + retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as + the name is generic enough that it might be in use and relied upon from + the environment. -- urldata: connect related booleans live in struct ConnectBits - - And remove a few unused booleans! - - Closes #5461 +- [Carlo Marcelo Arenas Belón brought this change] -- hostip: on macOS avoid DoH when given a numerical IP address + buildconf: avoid array concatenation in die() - When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that - numerical IP addresses still need to get "resolved" - but not with DoH. + reported as error SC2145[1] by shellcheck, but not expected to cause + any behavioural differences otherwise. - Reported-by: Viktor Szakats - Fixes #5454 - Closes #5459 - -- ngtcp2: cleanup memory when failing to connect + [1] https://github.com/koalaman/shellcheck/wiki/SC2145 - Reported-by: Peter Wu - Fixes #5447 (the ngtcp2 side of it) - Closes #5451 + Closes #5701 -- quiche: clean up memory properly when failing to connect +- travis: add ppc64le and s390x builds - Addresses the quiche side of #5447 - Reported-by: Peter Wu - Closes #5450 - -- cleanup: use a single space after equals sign in assignments + Closes #5752 -- url: accept "any length" credentials for proxy auth - - They're only limited to the maximum string input restrictions, not to - 256 bytes. +Marc Hoersken (31 Jul 2020) +- connect: remove redundant message about connect failure - Added test 1178 to verify + Reviewed-by: Daniel Stenberg - Reported-by: Will Roberts - Fixes #5448 - Closes #5449 - -- [Maksim Stsepanenka brought this change] + Closes #5708 - test1167: fixes in badsymbols.pl +- tests/sshserver.pl: fix compatibility with OpenSSH for Windows - Closes #5442 + Follow up to #5721 -- altsvc: fix parser for lines ending with CRLF +- CI/azure: install libssh2 for use with msys2-based builds - Fixed the alt-svc parser to treat a newline as end of line. + This enables building and running the SFTP tests. + Unfortunately OpenSSH for Windows does not support SCP (yet). - The unit tests in test 1654 were done without CRLF and thus didn't quite - match the real world. Now they use CRLF as well. + Reviewed-by: Daniel Stenberg - Reported-by: Peter Wu - Assisted-by: Peter Wu - Assisted-by: Jay Satiro - Fixes #5445 - Closes #5446 + Closes #5721 -Viktor Szakats (25 May 2020) -- all: fix codespell errors +- CI/azure: increase Windows job timeout once again + + Avoid aborted jobs due to performance issues on Azure DevOps. - Reviewed-by: Jay Satiro Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5452 + Reviewed-by: Jay Satiro + + Closes #5738 -Peter Wu (25 May 2020) -- ngtcp2: fix build with current ngtcp2 master implementing draft 28 +Jay Satiro (30 Jul 2020) +- TODO: Schannel: 'Add option to allow abrupt server closure' - Based on client.cc changes from ngtcp2. Tested with current git master, - ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. + We should offer an option to allow abrupt server closures (server closes + SSL transfer without sending a known termination point such as length of + transfer or close_notify alert). Abrupt server closures are usually + because of misconfigured or very old servers. - Fixes #5444 - Closes #5443 + Closes https://github.com/curl/curl/issues/4427 -Daniel Stenberg (25 May 2020) -- RELEASE-NOTES: synced +- url: fix CURLU and location following - moved the new setopts up to a "change" + Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was + incorrectly used for the location follow, resulting in infinite requests + to the original location. + + Reported-by: sspiri@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/5709 + Closes https://github.com/curl/curl/pull/5713 +Daniel Stenberg (30 Jul 2020) - RELEASE-NOTES: synced -- copyright: updated year ranges out of sync - - ... and whitelisted a few more files in the the copyright.pl script. - -- [Gilles Vollant brought this change] +- [divinity76 brought this change] - setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency + docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions - Closes #5431 - -- curl: remove -J "informational" written on stdout + it helps make it obvious that most developers don't have to care about + the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 + years old, November 4 2009) - curl would previously show "curl: Saved to filename 'name from header'" - if -J was used and a name was picked from the Content-Disposition - header. That output could interfer with other stdout output, such as -w. + Closes #5744 + +Jay Satiro (29 Jul 2020) +- tool_cb_wrt: fix outfile mode flags for Windows - This commit removes that output line. - Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html - Reported-by: Коваленко Анатолий Викторович - Closes #5435 - -Peter Wu (22 May 2020) -- travis: simplify quiche build instructions wrt boringssl + - Use S_IREAD and S_IWRITE mode permission flags to create the file + on Windows instead of S_IRUSR, S_IWUSR, etc. - quiche builds boringssl as static library, reuse that instead of - building another shared library. + Windows only accepts a combination of S_IREAD and S_IWRITE. It does not + acknowledge other combinations, for which it may generate an assertion. - Closes #5438 - -- configure: fix pthread check with static boringssl + This is a follow-up to 81b4e99 from yesterday, which improved the + existing file check with -J. - A shared boringssl/OpenSSL library requires -lcrypto only for linking. - A static build additionally requires `-ldl -lpthread`. In the latter - case `-lpthread` is added to LIBS which prevented `-pthread` from being - added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. + Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks + Ref: https://github.com/curl/curl/pull/5731 + + Closes https://github.com/curl/curl/pull/5742 -Daniel Stenberg (22 May 2020) -- Revert "sendf: make failf() use the mvsnprintf() return code" +Daniel Stenberg (28 Jul 2020) +- checksrc: ban gmtime/localtime - This reverts commit 74623551f306990e70c7c5515b88972005604a74. + They're not thread-safe so they should not be used in libcurl code. - Instead mark the function call with (void). Getting the return code and - using it instead triggered Coverity warning CID 1463596 because - snprintf() can return a negative value... + Explictly enabled when deemed necessary and in examples and tests - Closes #5441 + Reviewed-by: Nicolas Sterchele + Closes #5732 -- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' +- transfer: fix data_pending for builds with both h2 and h3 enabled - Reported-by: Billyzou0741326 on github - Fixes #5432 - Closes #5436 + Closes #5734 -- tests/server/util.h: add extern to silence compiler warning +- curl_multi_setopt: fix compiler warning "result is always false" - Follow-up from a3b0699d5c1 + On systems with 32 bit long the expression is always false. Avoid + the warning. + + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 + Closes #5736 -- typecheck-gcc.h: fix the OFF_T check +- curl: improve the existing file check with -J - The option number also needs to be less than CURLOPTTYPE_BLOB. + Previously a file that isn't user-readable but is user-writable would + not be properly avoided and would get overwritten. - Follow-up to cac5374298 - Reported-by: Jeroen Ooms - Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 + Reported-by: BrumBrum on hackerone + Assisted-by: Jay Satiro + Bug: https://hackerone.com/reports/926638 + Closes #5731 -- TODO: --dry-run - - Closes #5426 +- [Jonathan Nieder brought this change] -- TODO: Ratelimit or wait between serial requests + multi: update comment to say easyp list is linear - Closes #5406 - -- tool_paramhlp: fixup C89 mistake + Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, + 2013-08-02), the easy handle list is not circular but ends with + ->next pointing to NULL. - Follow-up to c5f0a9db22. - -- [Siva Sivaraman brought this change] + Reported-by: Masaya Suzuki + Closes #5737 - tool_paramhlp: fixed potentially uninitialized strtol() variable +- CURLOPT_NOBODY.3: fix the syntax for referring to options - Seems highly unlikely to actually be possible, but better safe than - sorry. + As test 1140 fails otherwise! - Closes #5417 + Follow-up to e1bac81cc815 -- [Siva Sivaraman brought this change] +- ngtcp2: store address in sockaddr_storage + + Reported-by: Tatsuhiro Tsujikawa + Closes #5733 - tool_operate: fixed potentially uninitialized variables +- CURLOPT_NOBODY.3: clarify what setting to 0 means - ... in curl_easy_getinfo() calls. They're harmless but clearing the - variables makes the code safer and comforts the reader. + ... and mention that HTTP with other methods than HEAD might get a body and + there's no option available to stop that. - Closes #5416 + Closes #5729 -- sha256: move assign to the declaration line +- setopt: unset NOBODY switches to GET if still HEAD - Follow-up to fae30656. Should've been squashed with that commit... + Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented + action but before 7.71.0 that used to switch back to GET and with this + change (assuming the method is still set to HEAD) this behavior is + brought back. + + Reported-by: causal-agent on github + Fixes #5725 + Closes #5728 -- [Siva Sivaraman brought this change] +- [Ehren Bendler brought this change] - sha256: fixed potentially uninitialized variable + configure: cleanup wolfssl + pkg-config conflicts when cross compiling. - Closes #5414 + Also choose a different wolfSSL function to test for NTLM support. + + Fixes #5605 + Closes #5682 -- sendf: make failf() use the mvsnprintf() return code +- configure: show zstd "no" in summary when built without it - ... and avoid a strlen() call. Fixes a MonocleAI warning. + Reported-by: Marc Hörsken + Fixes #5720 + Closes #5730 + +- quiche: handle calling disconnect twice - Reported-by: MonocleAI - Fixes #5413 - Closes #5420 + Reported-by: lilongyan-huawei on github + Fixes #5726 + Closes #5727 -- hostip: make Curl_printable_address not return anything +- [Nicolas Sterchele brought this change] + + getinfo: reset retry-after value in initinfo - It was not used much anyway and instead we let it store a blank buffer - in case of failure. + - Avoid re-using retry_after value from preceding request + - Add libtest 3010 to verify - Reported-by: MonocleAI - Fixes #5411 - Closes #5418 + Reported-by: joey-l-us on github + Fixes #5661 + Closes #5672 -- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) +Marcel Raad (27 Jul 2020) +- WIN32: stop forcing narrow-character API - They're done on purpose, make that visible in the code. - Reported-by: MonocleAI - Fixes #5412 - Closes #549 - -- TODO: forbid TLS post-handshake auth and do TLS record padding + Except where the results are only used for character output. + getenv is not touched because it's part of the public API, and having + it return UTF-8 instead of ANSI would be a breaking change. - Closes #5396 - Closes #5398 + Fixes https://github.com/curl/curl/issues/5658 + Fixes https://github.com/curl/curl/issues/5712 + Closes https://github.com/curl/curl/pull/5718 -- RELEASE-NOTES: synced +Jay Satiro (27 Jul 2020) +- [Tobias Stoeckmann brought this change] -- dynbuf: return NULL when there's no buffer length + mprintf: Fix stack overflows - ... as returning a "" is not a good idea as the string is supposed to be - allocated and returning a const string will cause issues. + Stack overflows can occur with precisions for integers and floats. - Reported-by: Brian Carpenter - Follow-up to ed35d6590e72c - Closes #5405 - -Peter Wu (16 May 2020) -- travis: upgrade to bionic, clang-9, improve readability + Proof of concepts: + - curl_mprintf("%d, %.*1$d", 500, 1); + - curl_mprintf("%d, %+0500.*1$f", 500, 1); - Changes, partially to reduce build failures from external dependencies: - - Upgrade Ubuntu and drop unnecessary third-party repos. - - Properly clone apt config to ensure retries. - - Upgrade to clang-9 from the standard repos. - - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey - fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use - focal everywhere yet since Travis CI has not documented this option. - In focal, python-impacket (Py2.7) has been removed, leaving only - python3-impacket. Since it is only needed for SMB tests and not SSH, - skip it for the libssh job since it might need more work. - - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. + Ideally, compile with -fsanitize=address which makes this undefined + behavior a bit more defined for debug purposes. - Non-functional cleanups: - - Simplify test matrix, drop redundant os and compiler keys. - - Deprecation fixes: remove sudo, rename matrix -> jobs. - - Every job has an 'env' key, put this key first in a list item. + The format strings are valid. The overflows occur due to invalid + arguments. If these arguments are variables with contents controlled + by an attacker, the function's stack can be corrupted. - Closes #5370 + Also see CVE-2016-9586 which partially fixed the float aspect. + + Signed-off-by: Tobias Stoeckmann + + Closes https://github.com/curl/curl/pull/5722 -- travis: whitespace-only changes for consistency +- [Tobias Stoeckmann brought this change] + + mprintf: Fix dollar string handling - Automatically apply a consistent indentation with: + Verify that specified parameters are in range. If parameters are too + large, fail early on and avoid out of boundary accesses. - python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' + Also do not read behind boundaries of illegal format strings. - followed by manually re-indenting three comments. + These are defensive measures since it is expected that format strings + are well-formed. Format strings should not be modifiable by user + input due to possible generic format string attacks. - Closes #5370 + Closes https://github.com/curl/curl/pull/5722 -- CMake: add libssh build support +Daniel Stenberg (26 Jul 2020) +- ntlm: free target_info before (re-)malloc - Closes #5372 - -Daniel Stenberg (15 May 2020) -- KNOWN_BUGS: wolfssh: publickey auth doesn't work + OSS-Fuzz found a way this could get called again with the pointer still + pointing to a malloc'ed memory, leading to a leak. - Closes #4820 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 + + Closes #5724 -- KNOWN_BUGS: OS400 port requires deprecated IBM library +Marcel Raad (26 Jul 2020) +- CI/macos: set minimum macOS version - Closes #5176 + This enables some deprecation warnings. + Previously, autotools defaulted to 10.8. + + Closes https://github.com/curl/curl/pull/5723 -- [Vyron Tsingaras brought this change] +Daniel Stenberg (26 Jul 2020) +- RELEASE-NOTES: synced - http2: keep trying to send pending frames after req.upload_done +Marcel Raad (25 Jul 2020) +- CI/macos: enable warnings as errors for CMake builds - Fixes #1410 - Closes #5401 + Closes https://github.com/curl/curl/pull/5716 -- [Gilles Vollant brought this change] - - setopt: support certificate options in memory with struct curl_blob - - This change introduces a generic way to provide binary data in setopt - options, called BLOBs. +- CMake: fix test for warning suppressions - This change introduces these new setopts: + GCC doesn't warn for unknown `-Wno-` options, except if there are other + warnings or errors [0]. This was problematic with `CURL_WERROR` as that + warning-as-error cannot be suppressed. Notably, this always happened + with `-Wno-pedantic-ms-format` when not targeting Windows. So test for + the positive form of the warning instead, which should always result in + a diagnostic if unknown. - CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, - CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. + [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html - Reviewed-by: Daniel Stenberg - Closes #5357 + Closes https://github.com/curl/curl/pull/5714 -- source cleanup: remove all custom typedef structs +Jay Satiro (23 Jul 2020) +- curl.h: update CURLINFO_LASTONE - - Stick to a single unified way to use structs - - Make checksrc complain on 'typedef struct {' - - Allow them in tests, public headers and examples + CURLINFO_LASTONE should have been updated when + CURLINFO_EFFECTIVE_METHOD was added. - - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually - typedef different types/structs depending on build conditions. + Reported-by: xwxbug@users.noreply.github.com - Closes #5338 - -- travis: remove the .checksrc fiddling + Fixes https://github.com/curl/curl/issues/5711 -- ftp: make domore_getsock() return the secondary socket properly +Marc Hoersken (22 Jul 2020) +- CI/azure: unconditionally enable warnings-as-errors with autotools - Previously, after PASV and immediately after the data connection has - connected, the function would only return the control socket to wait for - which then made the data connection simply timeout and not get polled - correctly. This become obvious when running test 1631 and 1632 event- - based. - -- test1632: verify FTP through HTTPS-proxy with connection re-use + Reviewed-by: Marcel Raad + + Follow up to #5694 + Closes #5706 -- test1631: verify FTP download through HTTPS-proxy +Marcel Raad (21 Jul 2020) +- doh: remove redundant cast + + Closes https://github.com/curl/curl/pull/5704 -- sws: as last resort, get test number from server cmd file +- CI/macos: unconditionally enable warnings-as-errors with autotools - If it can't be found in the request. Also support --cmdfile to set it to - a custom file name. + Previously, warnings were only visible in the output for most jobs. - runtests.pl always writes this file with the test number in it since a - while back. + Closes https://github.com/curl/curl/pull/5694 -- ftp: shut down the secondary connection properly when SSL is used +- util: silence conversion warnings - Reported-by: Neal Poole - Fixes #5340 - Closes #5385 + timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might + be a 64-bit integer. This is the case when building for recent macOS + versions, for example. Just treat tv_usec as an int, which should + hopefully always be sufficient on systems with + `HAVE_CLOCK_GETTIME_MONOTONIC`. + + Closes https://github.com/curl/curl/pull/5695 -Marcel Raad (14 May 2020) -- KNOWN_BUGS: adapt 5.5 to recent changes +- md(4|5): don't use deprecated macOS functions - It only applies to non-Unicode builds now. - Also merge 5.10 into it as it's effectively a duplicate. + They are marked as deprecated for -mmacosx-version-min >= 10.15, + which might result in warnings-as-errors. - Closes https://github.com/curl/curl/pull/3784 + Closes https://github.com/curl/curl/pull/5695 -- curl_setup: support Unicode functions to open files on Windows +Daniel Stenberg (18 Jul 2020) +- strdup: remove the odd strlen check - Use them only if `_UNICODE` is defined, in which case command-line - arguments have been converted to UTF-8. + It confuses code analyzers with its use of -1 for unsigned value. Also, + a check that's not normally used in strdup() code - and not necessary. - Closes https://github.com/curl/curl/pull/3784 + Closes #5697 -- tool: support UTF-16 command line on Windows - - - use `wmain` instead of `main` when `_UNICODE` is defined [0] - - define `argv_item_t` as `wchar_t *` in this case - - use the curl_multibyte gear to convert the command-line arguments to - UTF-8 +- [Alessandro Ghedini brought this change] + + travis: update quiche builds for new boringssl layout - This makes it possible to pass parameters with characters outside of - the current locale on Windows, which is required for some tests, e.g. - the IDN tests. Out of the box, this currently only works with the - Visual Studio project files, which default to Unicode, and winbuild - with the `ENABLE_UNICODE` option. + This is required after https://github.com/cloudflare/quiche/pull/593 + moved BoringSSL around slightly. - [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + This also means that Go is not needed to build BoringSSL anymore (the + one provided by quiche anyway). - Ref: https://github.com/curl/curl/issues/3747 - Closes https://github.com/curl/curl/pull/3784 + Closes #5691 -- curl_multibyte: add to curlx +Marcel Raad (17 Jul 2020) +- configure: allow disabling warnings - This will also be needed in the tool and tests. + When using `--enable-warnings`, it was not possible to disable warnings + via CFLAGS that got explicitly enabled. Now warnings are not enabled + anymore if they are explicitly disabled (or enabled) in CFLAGS. This + works for at least GCC, clang, and TCC as they have corresponding + `-Wno-` options for every warning. - Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 - Closes https://github.com/curl/curl/pull/3784 + Closes https://github.com/curl/curl/pull/5689 -Daniel Stenberg (14 May 2020) -- url: make the updated credentials URL-encoded in the URL +Daniel Stenberg (16 Jul 2020) +- ngtcp2: adjust to recent sockaddr updates - Found-by: Gregory Jefferis - Reported-by: Jeroen Ooms - Added test 1168 to verify. Bug spotted when doing a redirect. - Bug: https://github.com/jeroen/curl/issues/224 - Closes #5400 + Closes #5690 -- tests: add https-proxy support to the test suite +- page-header: provide protocol details in the curl.1 man page - Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like - HTTP proxy but with a full TLS connection to the proxy. + Add protocol and version specific information about all protocols curl + supports. - Closes #5399 - -- mailmap: James Fuller - -- [Major_Tom brought this change] + Fixes #5679 + Reported-by: tbugfinder on github + Closes #5686 - vauth/cleartext: fix theoretical integer overflow - - Fix theoretical integer overflow in Curl_auth_create_plain_message. +Daniel Gustafsson (16 Jul 2020) +- docs: Update a few leftover mentions of DarwinSSL - The security impact of the overflow was discussed on hackerone. We - agreed this is more of a theoretical vulnerability, as the integer - overflow would only be triggerable on systems using 32-bits size_t with - over 4GB of available memory space for the process. + Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the + more correct/common name Secure Transport, but a few mentions in the docs + remained. - Closes #5391 + Closes #5688 + Reviewed-by: Daniel Stenberg -Jay Satiro (13 May 2020) -- curl.1: Quote globbed URLs +Daniel Stenberg (16 Jul 2020) +- file2memory: use a define instead of -1 unsigned value - - Quote the globbing example URLs that contain characters [] {} since - otherwise they may be interpreted as shell metacharacters. + ... to use the maximum value for 'size_t' when detecting integer overflow. + Changed the limit to max/4 as already that seems unreasonably large. - Bug: https://github.com/curl/curl/issues/5388 - Reported-by: John Simpson + Codacy didn't like the previous approach. - Closes https://github.com/curl/curl/pull/5394 + Closes #5683 -Daniel Stenberg (14 May 2020) -- checksrc: enhance the ASTERISKSPACE and update code accordingly - - Fine: "struct hello *world" +- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream - Not fine: "struct hello* world" (and variations) + ... by adding support for a new dedicated return code. - Closes #5386 + Suggested-by: Jonathan Cardoso + Assisted-by: Erik Johansson + URL: https://curl.haxx.se/mail/lib-2020-06/0099.html + Closes #5636 -- docs/options-in-versions: which version added each cmdline option +- [Baruch Siach brought this change] + + nss: fix build with disabled proxy support - Added test 971 to verify that the list is in sync with the files in - cmdline-opts. The check also verifies that .d-files that uses Added: - specify the same version number as the options-in-versions file does. + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - Closes #5381 + Closes #5667 -- docs: unify protocol lists +- test1139: make it display the difference on test failures + +- test1119: verify stdout in the test - We boast support for 25 transfer protocols. Make sure the lists are - consistent + So that failures will be displayed in the terminal, as it makes test failures + visually displayed easier and faster. - Closes #5384 + Closes #5644 -- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN +- curl: add %{method} to the -w variables - ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. + Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. - Reported-by: Michael Kaufmann - Fixes #5374 - Closes #5376 + Added test 1197 to verify. -- tls13-ciphers.d: shorten the Arg +- CURLINFO_EFFECTIVE_METHOD: added + + Provide the HTTP method that was used on the latest request, which might + be relevant for users when there was one or more redirects involved. + + Closes #5511 -- sasl-authzid.d: add Arg: and shorten the desc +Viktor Szakats (14 Jul 2020) +- windows: add unicode to feature list + + Reviewed-by: Marcel Raad + Reviewed-by: Marc Hörsken + + Closes #5491 -- cert-type.d: mention the available types in the desc +Daniel Stenberg (14 Jul 2020) +- multi: remove two checks always true + + Detected by Codacy + Closes #5676 -- tool: shorten 3 --help descriptions +Marc Hoersken (13 Jul 2020) +- workflows: limit what branches to run CodeQL on - --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort + Align CodeQL action with existing CI actions: + - Update branch filter to avoid duplicate CI runs. + - Shorten workflow name due to informative job name. - gen.pl already warned about these lines but we didn't listen + Reviewed-by: Daniel Stenberg - Closes #5379 + Closes #5660 -- configure: the wolfssh backend does not provide SCP +- appveyor: collect libcurl.dll variants with prefix or suffix - Closes #5387 - -- RELEASE-NOTES: synced - -- url: reject too long input when parsing credentials + On some platforms libcurl is build with a platform-specific + prefix and/or a version number suffix. - Since input passed to libcurl with CURLOPT_USERPWD and - CURLOPT_PROXYUSERPWD circumvents the regular string length check we have - in Curl_setstropt(), the input length limit is enforced in - Curl_parse_login_details too, separately. + Assisted-by: Jay Satiro - Reported-by: Thomas Bouzerar - Closes #5383 + Closes #5659 -- list-only.d: this option existed already in 4.0 +Daniel Stenberg (12 Jul 2020) +- [ihsinme brought this change] -Jay Satiro (12 May 2020) -- retry-all-errors.d: Shorten the summary line + socks: use size_t for size variable - Follow-up to b995bb5 from a few moments ago. + Use the unsigned type (size_t) in the arithmetic of pointers. In this + context, the signed type (ssize_t) is used unnecessarily. - Reported-by: Daniel Stenberg + Authored-by: ihsinme on github + Closes #5654 + +- RELEASE-NOTES: synced - Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 + ... and bumped to 7.72.0 as the next release version number -- [denzor brought this change] +- [Gilles Vollant brought this change] - easy: fix dangling pointer on easy_perform fail + content_encoding: add zstd decoding support - Closes https://github.com/curl/curl/pull/5363 + include zstd curl patch for Makefile.m32 from vszakats + and include Add CMake support for zstd from Peter Wu + + Helped-by: Viktor Szakats + Helped-by: Peter Wu + Closes #5453 -- tool: Add option --retry-all-errors to retry on any error +- asyn.h: remove the Curl_resolver_getsock define - The "sledgehammer" of retrying. + - not used + - used the wrong number of arguments + - confused the Codeacy code analyzer - Closes https://github.com/curl/curl/pull/5185 + Closes #5647 -Daniel Stenberg (12 May 2020) -- [James Le Cuirot brought this change] +- [Nicolas Sterchele brought this change] - libcurl.pc: Merge Libs.private into Libs for static-only builds - - A project being built entirely statically will call pkg-config with - --static, which utilises the Libs.private field. Conversely it will - not use --static when not being built entirely statically, even if - there is only a static build of libcurl available. This will most - likely cause the build to fail due to underlinking unless we merge the - Libs fields. - - Consider that this is what the Meson build system does when it - generates pkg-config files. + configure.ac: Sort features name in summary - I have also reflected this in the --libs argument of curl-config even - though REQUIRE_LIB_DEPS always seems to be "yes" anyway. + - Same as protocols - Closes #5373 + Closes #5656 -- [Peter Wu brought this change] +- [Matthias Naegler brought this change] - CMake: fix runtests.pl with CMake, add new test targets - - * runtests.pl: - - Fix out-of-tree build under CMake when srcdir is not set. Default - srcdir to the location of runtests.pl. - - Add a hack to allow CMake to use the TFLAGS option as documented - in tests/README and used in scripts/travis/script.sh. - * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie - support (no one should care, it is already EOL.). - * Remove CTest since it defines its own 'test' target with no tests - since all unittests are already broken and not built by default. - * Add new test targets based on the options from Makefile.am. Since - new test targets are rarely added, I opted for duplicating the - runtests.pl options as opposed to creating a new Makefile.inc file. - Use top-level target names (test-x) instead of x-test since that is - used by CI and others. + cmake: fix windows xp build - Closes #5358 + Reviewed-by: Marcel Raad + Closes #5662 -- [Peter Wu brought this change] +- ngtcp2: update to modified qlog callback prototype + + Closes #5675 - CMake: do not build test programs by default +- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle - The default target should only build libcurl and curl. Add a dedicated - 'testdeps' target which will be used later when running tests. Note that - unittests are currently broken in CMake and already excluded. + Added test case 674 to reproduce and verify the bug report. - Closes #5368 + Fixes #5665 + Reported-by: NobodyXu on github + Closes #5673 -- FILEFORMAT: moved up the variables section and further polished +- [Baruch Siach brought this change] -- runtests: remove ftp2 support, not used + bearssl: fix build with disabled proxy support - We once supported two separate ftp instances in the test suite. Has not - been used the last decade. + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - Closes #5375 + Reviewed-by: Nicolas Sterchele + Closes #5666 -- url: sort the protocol schemes in rough popularity order - - When looking for a protocol match among supported schemes, check the - most "popular" schemes first. It has zero functionality difference and - for all practical purposes a speed difference will not be measureable - but it still think it makes sense to put the least likely matches last. - - "Popularity" based on the 2019 user survey. - - Closes #5377 +- RELEASE-NOTES: synced -Marc Hoersken (11 May 2020) -- test1238: avoid tftpd being busy for tests shortly following - - The tftpd server may still be busy if the total timeout of - 25 seconds has not been reached or no sread error was received - during or after the execution of the timeout test 1238. - - Once the next TFTP test comes around (eg. 1242 or 1243), - those will fail because the tftpd server is still waiting - on data from curl due to the UDP protocol being stateless - and having no connection close. On Linux this error may not - happen, because ICMP errors generated due to a swrite error - can also be returned async on the next sread call instead. - - Therefore we will now just kill the tftpd server after test - 1238 to make sure that the following tests are not affected. +Jay Satiro (11 Jul 2020) +- [Carlo Marcelo Arenas Belón brought this change] + + cirrus-ci: upgrade 11-STABLE to 11.4 - This enables us to no longer ignore tests 1242, 1243, 2002 - and 2003 on the CI platforms CirrusCI and AppVeyor. + Meant to be the last of the 11 series and so make sure that all + other references reflect all 11 versions so they can be retired + together later. - Assisted-by: Peter Wu - Closes #5364 + Closes https://github.com/curl/curl/pull/5668 -Daniel Stenberg (11 May 2020) -- write-out.d: added "response_code" +- [Filip Salomonsson brought this change] -- KNOWN_BUGS: Build with staticly built dependency + CURLINFO_CERTINFO.3: fix typo - I rewrote the item 5.4 to be more generic about static dependencies. + Closes https://github.com/curl/curl/pull/5655 -- ROADMAP: remove old entries - - MQTT - the start has already landed +Daniel Stenberg (4 Jul 2020) +- http2: only do the *done() cleanups for HTTP - tiny-curl - also mostly landed and is a continuous work + Follow-up to ef86daf4d3 - make menuconfig - basically no interest from users, not pushing there + Closes #5650 + Fixes #5646 -- [Peter Wu brought this change] +- [Alex Kiernan brought this change] - travis: Add ngtcp2 and quiche tests for CMake + gnutls: repair the build with `CURL_DISABLE_PROXY` - To avoid an explosion of jobs, extend the existing CMake tests with - ngtcp2 and quiche support. macOS was previously moved to GitHub actions, - so the non-Linux case can be dropped. - -- [Peter Wu brought this change] + `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` + if `CURL_DISABLE_PROXY` is enabled. Repair the build with that + configuration. + + Signed-off-by: Alex Kiernan + Closes #5645 - CMake: add ENABLE_ALT_SVC option +Alex Kiernan (3 Jul 2020) +- gnutls: Fetch backend when using proxy - Tested alt-svc with quiche. While at it, add missing MultiSSL reporting - (not tested). + Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") + Signed-off-by: Alex Kiernan -- [Peter Wu brought this change] +Daniel Stenberg (3 Jul 2020) +- [Laramie Leavitt brought this change] - CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) + http2: close the http2 connection when no more requests may be sent - Add three new CMake Find modules (using the curl license, but I grant - others the right to apply the CMake BSD license instead). + Well-behaving HTTP2 servers send two GOAWAY messages. The first + message is a warning that indicates that the server is going to + stop accepting streams. The second one actually closes the stream. - This CMake config is simpler than the autotools one because it assumes - ngtcp2 and nghttp3 to be used together. Another difference is that this - CMake config checks whether QUIC is actually supported by the TLS - library (patched OpenSSL or boringssl) since this can be a common - configuration mistake that could result in build errors later. + nghttp2 reports this state (and the other state of no more stream + identifiers) via the call nghttp2_session_check_request_allowed(). + In this state the client should not create more streams on the + session (tcp connection), and in curl this means that the server + has requested that the connection is closed. - Unlike autotools, CMake does not warn you that the features are - experimental. The user is supposed to already know that and read the - documentation. It requires a very special build environment anyway. + It would be also be possible to put the connclose() call into the + on_http2_frame_recv() function that triggers on the GOAWAY message. - Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from - current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure - out which features (libldap-2.4, libssh2) to disable due to conflicts - with boringssl. + This fixes a bug seen when the client sees the following sequence of + frames: - Closes #5359 - -Marc Hoersken (10 May 2020) -- tests/server/tftpd.c: fix include and enhance debug logging + // advisory GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] + ... some additional frames - setjmp.h should only be included if HAVE_SETJMP_H is defined. + // final GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] - Add additional log statements to see wether reads and writes - are blocking or finishing before an alarm signal is received. + Before this change, curl will attempt to reuse the connection even + after the last stream, will encounter this error: - Assisted-by: Peter Wu - Part of #5364 - -Daniel Stenberg (10 May 2020) -- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present + * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] + * Re-using existing connection! (#0) with host localhost + * Connected to localhost (::1) port 10443 (#0) + * Using Stream ID: 9 (easy handle 0x5595f0a72e30) + > GET /index.html?5 HTTP/2 + > Host: localhost:10443 + > user-agent: curl/7.68.0 + > accept: */* + > + * stopped the pause stream! + * Connection #0 to host localhost left intact + curl: (16) Error in the HTTP2 framing layer - Reported-by: Marcel Raad - Follow-up to 148534db5 - Fixes #5367 - Closes #5369 - -Marc Hoersken (9 May 2020) -- appveyor: update comments to be clear about toolchain + This error may posion the connection cache, causing future requests + which resolve to the same curl connection to go through the same error + path. - - CMake-based MSYS builds use mingw-w64 to cross-compile. - - autotools-based builds are compiled using msys2-devel. + Closes #5643 + +- ftpserver: don't verify SMTP MAIL FROM names - The difference is that the later ones are not cross-compiled - to Windows and instead require the msys2 runtime to be present. + Rely on tests asking the names to get refused instead - test servers + should be as dumb as possible. Edited test 914, 955 and 959 accordingly. - At the moment only the Azure Pipelines CI builds actually - run autotools-based cross-compilation builds for Windows. + Closes #5639 -- TODO: update regarding missing Schannel features +- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated - Some aspects have already been implemented over the years. + This came up in #5640. It make sense to clarify this in the docs! - 15.1 Client certificates are now supported: + Reminded-by: Kamil Dudka + Closes #5642 + +Kamil Dudka (3 Jul 2020) +- tool_getparam: make --krb option work again - - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 - - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 + It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. - 15.2 Ciphers can now be specified through: + Bug: https://bugzilla.redhat.com/1833193 + Closes #5640 + +Daniel Stenberg (2 Jul 2020) +- [Jeremy Maitin-Shepard brought this change] + + http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages - - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 + Confusingly, nghttp2 has two different error code enums: - Reviewed-by: Daniel Stenberg and Marcel Raad - Closes #5358 + - nghttp2_error, to be used with nghttp2_strerror + - nghttp2_error_code, to be used with nghttp2_http2_strerror + + Closes #5641 -Daniel Stenberg (8 May 2020) -- checksrc: close the .checksrc file handle when done reading +Marcel Raad (2 Jul 2020) +- url: silence MSVC warning + + Since commit f3d501dc678, if proxy support is disabled, MSVC warns: + url.c : warning C4701: potentially uninitialized local variable + 'hostaddr' used + url.c : error C4703: potentially uninitialized local pointer variable + 'hostaddr' used + + That could actually only happen if both `conn->bits.proxy` and + `CURL_DISABLE_PROXY` were enabled. + Initialize it to NULL to silence the warning. + + Closes https://github.com/curl/curl/pull/5638 +Daniel Stenberg (1 Jul 2020) - RELEASE-NOTES: synced - - And bumped next version to 7.71.0 -- [Gilles Vollant brought this change] +Version 7.71.1 (30 Jun 2020) - CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) - - Closes #4346 +Daniel Stenberg (30 Jun 2020) +- RELEASE-NOTES: curl 7.71.1 -- TODO: native IDN support on macOS +- THANKS: add contributors to 7.71.1 -- urlapi: accept :: as a valid IPv6 address +- scripts/copyright.pl: skip .dcignore + +- Revert "multi: implement wait using winsock events" - Text 1560 is extended to verify. + This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. - Reported-by: Pavel Volgarev - Fixes #5344 - Closes #5351 + That commit (from #5397) introduced a regression in 7.71.0. + + Reported-by: tmkk on github + Fixes #5631 + Closes #5632 -- THANKS-filter: Peter Wang +- TODO: Add flag to specify download directory -- [Peter Wang brought this change] +- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection - *_sspi: fix bad uses of CURLE_NOT_BUILT_IN +- cirrus-ci: disable FreeBSD 13 (again) - Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other - instances of QuerySecurityPackageInfo failing, as in - commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. + It has been failing for a good while again. This time we better leave it + disabled until we have more reason to believe it behaves. - Closes #5355 - -- docs/HTTP3: add qlog to the quiche build instruction + Closes #5628 -- ngtcp2: introduce qlog support - - If the QLOGDIR environment variable is set, enable qlogging. +- ngtcp2: sync with current master - ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for - QUIC functions that are backend independent. + ngtcp2 added two new callbacks - Closes #5353 + Reported-by: Lucien Zürcher + Fixes #5624 + Closes #5627 -- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN - - That return code is reserved for build-time conditional code not being - present while this was a regular run-time error from a Windows API. +- examples/multithread.c: call curl_global_cleanup() - Reported-by: wangp on github - Fixes #5349 - Closes #5350 + Reported-by: qiandu2006 on github + Fixes #5622 + Closes #5623 -- runtests: show elapsed test time with higher precision (ms) +- vtls: compare cert blob when finding a connection to reuse + + Reported-by: Gergely Nagy + Fixes #5617 + Closes #5619 - RELEASE-NOTES: synced -- http2: simplify and clean up trailer handling +- terminology: call them null-terminated strings - Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in - ed35d6590e72. This should make the trailer handling more straight forward and - hopefully less error-prone. + Updated terminology in docs, comments and phrases to refer to C strings + as "null-terminated". Done to unify with how most other C oriented docs + refer of them and what users in general seem to prefer (based on a + single highly unscientific poll on twitter). - Deliver the trailer header to the callback already at receive-time. No - longer caches the trailers to get delivered at end of stream. + Reported-by: coinhubs on github + Fixes #5598 + Closes #5608 + +- http: fix proxy auth with blank password - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 - Closes #5348 + Regression in 7.71.0 + + Added test case 346 to verify. + + Reported-by: Kristoffer Gleditsch + Fixes #5613 + Closes #5616 -Marc Hoersken (7 May 2020) -- appveyor: disable test 1139 instead of ignoring it +- .dcignore: ignore tests and docs directories - Spending time on manpage checking makes no sense - for these builds due to lacking manpage support. + This is a config file for deepcode.ai, a static code analyzer. -- appveyor: disable flaky test 1501 and ignore broken 1056 +Jay Satiro (26 Jun 2020) +- tool_cb_hdr: Fix etag warning output and return code - Test 1501 is flaky on Windows CI due to being time sensitive - and the testsuite relying on taskkill.exe to check for the - existance of processes which can take to much time itself. + - Return 'failure' on failure, to follow the existing style. - Test 1056 is broken in autotools-based Windows builds due - to scope ID support missing in these builds at the moment. - -- test613.pl: make tests 613 and 614 work with OpenSSH for Windows + - Put Warning: and the warning message on the same line. - OpenSSH for Windows shows group and other/world permissions as *, - because those concepts do not exist on Windows. It also does not - show the current or parent directory, so we just ignore those. + Ref: https://github.com/curl/curl/issues/5610 - Reviewed-by: Daniel Stenberg - Closes #5328 + Closes https://github.com/curl/curl/pull/5612 -Daniel Stenberg (6 May 2020) -- runtests: set +x mode again +Daniel Stenberg (26 Jun 2020) +- CURLOPT_READFUNCTION.3: provide the upload data size up front + + Assisted-by: Jay Satiro + Closes #5607 -- libssh2: convert over to use dynbuf +- test1539: do a HTTP 1.0 POST without a set size (fails) - In my very basic test that lists sftp://127.0.0.1/tmp/, this patched - code makes 161 allocations compared to 194 in git master. A 17% - reduction. + Attempt to reproduce #5593. Test case 1514 is very similar but uses + HTTP/1.1 and thus switches to chunked. - Closes #5336 + Closes #5595 -- travis: add "qlog" as feature in the quiche build +- [Baruch Siach brought this change] -- quiche: enable qlog output + mbedtls: fix build with disabled proxy support - quiche has the potential to log qlog files. To enable this, you must - build quiche with the qlog feature enabled `cargo build --features - qlog`. curl then passes a file descriptor to quiche, which takes - ownership of the file. The FD transfer only works on UNIX. + Don't reference fields that do not exist. Fixes build failure: - The convention is to enable logging when the QLOGDIR environment is - set. This should be a path to a folder where files are written with the - naming template .qlog. + vtls/mbedtls.c: In function 'mbed_connect_step1': + vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' - Co-authored-by: Lucas Pardue - Replaces #5337 - Closes #5341 + Closes #5615 -- urldata.h: remove #define HEADERSIZE, not used anymore +- codeql-analysis.yml: fix the 'languages' setting - Follow-up to ed35d6590e72c + It needs a 'with:' in front of it. -- ngtcp2: convert to dynbuf +GitHub (26 Jun 2020) +- [Daniel Stenberg brought this change] + + gtihub: codeql-analysis.yml - Closes #5335 + enables code security scanning with github actions -- connect: make happy eyeballs work for QUIC (again) +Daniel Stenberg (25 Jun 2020) +- tests: verify newline in username and password for HTTP - Follow-up from dbd16c3e256c6c (regression in 7.70.0) + test 1296 is a simply command line test - Closes #5334 - -- connect: add two asserts to clue code analyzers in a little + test 1910 is a libcurl test including a redirect -- http_proxy: ported to use dynbuf instead of a static size buffer +- url: allow user + password to contain "control codes" for HTTP(S) - Removes a 16K static buffer from the easy handle. Simplifies the code. + Reported-by: Jon Johnson Jr + Fixes #5582 + Closes #5592 -- dynbuf: introduce internal generic dynamic buffer functions +- escape: make the URL decode able to reject only %00 bytes - A common set of functions instead of many separate implementations for - creating buffers that can grow when appending data to them. Existing - functionality has been ported over. + ... or all "control codes" or nothing. - In my early basic testing, the total number of allocations seem at - roughly the same amount as before, possibly a few less. + Assisted-by: Nicolas Sterchele + +- http2: set the correct URL in pushed transfers - See docs/DYNBUF.md for a description of the API. + ...previously CURLINFO_EFFECTIVE_URL would report the URL of the + original "mother transfer", not the actually pushed resource. - Closes #5300 + Reported-by: Jonathan Cardoso Machado + Fixes #5589 + Closes #5591 -- runtests: remove sleep calls - - Remove many one second sleeps that were done *after* each newly started - test server already has been verified. They should not have any purpose - there. - - Closes #5323 +Jay Satiro (25 Jun 2020) +- [Javier Blazquez brought this change] -- asyn-*: remove support for never-used NULL entry pointers + openssl: Fix compilation on Windows when ngtcp2 is enabled - ... and instead convert those to asserts to make sure they are truly - never NULL. + - Include wincrypt before OpenSSL includes so that the latter can + properly handle any conflicts between the two. - Closes #5324 + Closes https://github.com/curl/curl/pull/5606 -- [Emil Engler brought this change] +Daniel Stenberg (25 Jun 2020) +- test543: extended to verify zero length input + + As was reported in #5601 - doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax +- escape: zero length input should return a zero length output - Closes #5325 + Regression added in 7.71.0. + + Fixes #5601 + Reported-by: Kristoffer Gleditsch + Closes #5602 -Jay Satiro (2 May 2020) -- asyn-thread: fix cppcheck warning +- Curl_inet_ntop: always check the return code - - Check for NULL entry parameter before attempting to deref entry in - Curl_resolver_is_resolved, like is already done in asyn-ares. + Reported-by: Siva Sivaraman + Fixes #5412 + Closes #5597 + +- sendf: improve the message on client write errors - This is to silence cppcheck which does not seem to understand that - asyn-ares and asyn-thread have separate Curl_resolver_is_resolved - and those units are mutually exclusive. Prior to this change it warned - of a scenario where asyn-thread's Curl_resolver_is_resolved is called - with a NULL entry from asyn-ares, but that couldn't happen. + Replace "Failed writing body (X != Y)" with + "Failure writing output to destination". Possibly slightly less cryptic. - Reported-by: rl1987@users.noreply.github.com + Reported-by: coinhubs on github + Fixes #5594 + Closes #5596 + +- RELEASE-NOTES: synced + +- curlver: start working on 7.71.1 + +- [Denis Baručić brought this change] + + DYNBUF.md: fix a typo: trail => tail - Fixes https://github.com/curl/curl/issues/5326 + Closes #5599 -- select: fix overflow protection in Curl_socket_check +Version 7.71.0 (23 Jun 2020) + +Daniel Stenberg (23 Jun 2020) +- RELEASE-NOTES: curl 7.71.0 release + +- THANKS: curl 7.71.0 additions + +- url: make sure pushed streams get an allocated download buffer - Follow-up to a96c752 which changed the timeout_ms type from time_t to - timediff_t. + Follow-up to c4e6968127e876b0 - Ref: https://github.com/curl/curl/pull/5240 + When a new transfer is created, as a resuly of an acknowledged push, + that transfer needs a download buffer allocated. - Closes https://github.com/curl/curl/pull/5286 + Closes #5590 -Marc Hoersken (2 May 2020) -- sockfilt: make select_ws stop waiting on exit signal event +Jay Satiro (22 Jun 2020) +- openssl: Don't ignore CA paths when using Windows CA store - This makes sure that select_ws behaves similar to real select - which stops waiting on a signal handler being triggered. + This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does + not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default + locations. Instead the CA store can now be used at the same time. - This makes it possible to gracefully stop sockfilt.exe on - Windows with taskkill /IM sockfilt.exe (without /F force flag). + The change is due to the impending release. The issue is still being + discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and + is now documented as experimental. - Reviewed-by: Jay Satiro - Part of #5260 + Ref: bc052cc (parent commit) + Ref: https://github.com/curl/curl/issues/5585 -- tests/server/util.[ch]: add exit event to stop waiting on Windows +- tool_operate: Don't use Windows CA store as a fallback - This commit adds a global exit event to the test servers that - Windows-specific wait routines can use to get triggered if the - program was signaled to be terminated, eg. select_ws in sockfilt.c + Background: - The exit event will be managed by the signal handling code and is - set to not reset automatically to support multiple wait routines. + 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate + store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides + CURLOPT_CAINFO if both are set. The curl tool will fall back to + CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set + via CURLOPT_CAINFO. - Reviewed-by: Jay Satiro - Closes #5260 - -- tests/server/util.c: fix thread handle not being closed + Problem: - Reviewed-by: Jay Satiro - Part of #5260 - -- tests/server/util.c: use raise instead of calling signal handler + libcurl may be built with hardcoded paths to a certificate bundle or + directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are + ignored. - Use raise to trigger signal handler instead of calling it - directly and causing potential unexpected control flow. + Solution: - Reviewed-by: Jay Satiro - Part of #5260 + A solution is still being discussed but since there's an impending + release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + + Ref: https://github.com/curl/curl/issues/5585 -- tests: add support for SSH server variant specific transfer paths +- openssl: Fix CA fallback logic for OpenSSL 3.0 build - OpenSSH for Windows requires paths in the format of /C:/ - instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ + Prior to this change I assume a build error would occur when + CURL_CA_FALLBACK was used. - Reviewed-by: Daniel Stenberg - Closes #5298 + Closes https://github.com/curl/curl/pull/5587 -Daniel Stenberg (2 May 2020) -- RELEASE-NOTES: synced +Daniel Stenberg (22 Jun 2020) +- copyright: update mismatched copyright years -- libssh2: set the expected total size in SCP upload init +- test1460: verify that -Ji is not ok + +- tool_getparam: -i is not OK if -J is used - ... as otherwise the progress callback gets called without that - information, making the progress meter have less info. + Reported-by: sn on hackerone + Bug: https://curl.haxx.se/docs/CVE-2020-8177.html + +- [Peter Wu brought this change] + + CMake: ignore INTERFACE_LIBRARY targets for pkg-config file - Reported-by: Murugan Balraj - Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html - Closes #5317 + Reviewed-by: Marcel Raad + Fixes #5512 + Closes #5517 -- runtests: make the logmsg from the ssh server only show in verbose +- [Valentyn Korniienko brought this change] -- tests: make test 1248 + 1249 use %NOLISTENPORT + multibyte: Fixed access-> waccess to file for Windows Plarform - ... instead of a port of a non-running server so that it works - stand-alone. + Reviewed-by: Marcel Raad + Closes #5580 + +- altsvc: bump to h3-29 - Closes #5318 + Closes #5584 -- examples: remove asiohiper.cpp +- urlglob: treat literal IPv6 addresses with zone IDs as a host name - This example has repeatedly been reported to contain bugs, and as users - copy and paste code from this into production, I now deem it better to - not provide the example at all. + ... and not as a "glob". Now done by passing the supposed host to the + URL parser which supposedly will do a better job at identifying "real" + numerical IPv6 addresses. - Closes #5090 - Closes #5322 + Reported-by: puckipedia on github + Fixes #5576 + Closes #5579 -- [Emil Engler brought this change] +- test1179: verify error message for non-existing cmdline option - doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 +- tool_getparam: repair the error message for unknown flag - Closes #5320 + Follow-up to 9e5669f3880674 + Detected by Coverity CID 1464582 ("Logically dead code") + + Closes #5577 -- [Emil Engler brought this change] +- FILEFORMAT: describe verify/stderr - KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" +- connect: improve happy eyeballs handling - It was fixed in 9a2cbf3 + For QUIC but also for regular TCP when the second family runs out of IPs + with a failure while the first family is still trying to connect. - Closes #5319 + Separated the timeout handling for IPv4 and IPv6 connections when they + both have a number of addresses to iterate over. -- cirrus: disable SFTP and SCP tests +- ngtcp2: never call fprintf() in lib code in release version + +- ngtcp2: fix happy eyeballs quic connect crash - ... as we can't seem to start the sshd server on it. Those problems - existed before d1239b50bececd (running the SSH server on a random port), - but they're more noticable now since there are more failed attempts in - the logs. + Reported-by: Peter Wu + Fixes #5565 + Closes #5568 + +- select: remove the unused ELAPSED_MS() macro - Closes #5315 + Closes #5573 -- [Emil Engler brought this change] +Marc Hoersken (17 Jun 2020) +- [rcombs brought this change] - runtests: fix typo in the existence of disabled tests checker + multi: implement wait using winsock events - Closes #5316 + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. + + Reviewed-by: Gergely Nagy + Reviewed-by: Marc Hörsken + + Closes #5397 -Dan Fandrich (30 Apr 2020) -- test75: Remove precheck test +Daniel Stenberg (17 Jun 2020) +- manpage: add three missing environment variables - This has not been needed since commit 9fa42bed and often prevents it - from running at all with dynamic test ports. + CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE + + Closes #5571 -- tests: Stop referring to server ports when they're not used +- RELEASE-NOTES: synced + +- configure: for wolfSSL, check for the DES func needed for NTLM - Several tests referred to specific server ports even when the test - didn't actually use that server or specify that it's needed. In such - cases, the test harness substitutes the text "[not running]" as the port - number which causes many such tests to fail due to the inability to - parse the URL. These tests are changed to use %NOLISTENPORT which will - always be substituted correctly. - -Daniel Stenberg (30 Apr 2020) -- [Emil Engler brought this change] + Also adds pkg-config support for the wolfSSL detection. - GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT - - Closes #5287 +- [Ruurd Beerstra brought this change] -- conncache: various concept cleanups - - More connection cache accesses are protected by locks. - - CONNCACHE_* is a beter prefix for the connection cache lock macros. - - Curl_attach_connnection: now called as soon as there's a connection - struct available and before the connection is added to the connection - cache. + ntlm: enable NTLM support with wolfSSL - Curl_disconnect: now assumes that the connection is already removed from - the connection cache. + When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* + functions that OpenSSL has. This change take advantage of that. - Ref: #4915 - Closes #5009 + Co-authored-by: Daniel Stenberg + Closes #5556 + Fixes #5548 -- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports +- http: move header storage to Curl_easy from connectdata - As stunnel is an external tool and it has no specific option to export - the actually used port number when asked to listen to 0, runtests - instead iterates over ten randomly picked high number ports and sticks - to the first one stunnel can listen to. + Since the connection can be used by many independent requests (using + HTTP/2 or HTTP/3), things like user-agent and other transfer-specific + data MUST NOT be kept connection oriented as it could lead to requests + getting the wrong string for their requests. This struct data was + lingering like this due to old HTTP1 legacy thinking where it didn't + mattered.. - Closes #5267 + Fixes #5566 + Closes #5567 -- tests: pick a random port number for SSH +- CODE_REVIEW.md: how to do code reviews in curl - Since sshd doesn't have such an option by itself, we iterate over a - series of random ports until one works. + Assisted-by: Daniel Gustafsson + Assisted-by: Rich Salz + Assisted-by: Hugo van Kemenade + Assisted-by: James Fuller + Assisted-by: Marc Hörsken + Assisted-by: Jay Satiro - Closes #5273 - -- [Rikard Falkeborn brought this change] + Closes #5555 - libtest/cmake: Remove commented code +- altsvc: remove the num field from the altsvc struct - These were commented out in e9dd0998706a when Makefile.inc was included - instead. 11 years have passed since then and the commented code is of - course very outdated. Remove it to avoid confusion. + It was superfluous since we have the list.size alredy - Closes #5311 + Reported-by: Jay Satiro + Fixes #5553 + Closes #5563 -- schannel: source code reindent +- version.d: expanded and alpha-sorted - White space edits only. Conform better to standard curl source code - indenting style. + Added a few missing features not previously mentioned. Ordered them + alphabetically. - Closes #5305 + Closes #5558 -Kamil Dudka (29 Apr 2020) -- test1177: look for curl.h in source directory - - If we use a separate build directory, there is no copy of the header. +- ABI.md: rename to .md and polish the markdown - Closes #5310 + Closes #5562 -- tests: look for preprocessed tests in build directory +- HELP-US: add a section for "smaller tasks" - ... which is not always the same directory as source directory + The point of this section is to meet the CII Best Practices gold level + critera: - Closes #5310 - -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: synced + "The project MUST clearly identify small tasks that can be performed by + new or casual contributors" - ... and bumped curlver.h to 7.70.1 + Closes #5560 -Version 7.70.0 (29 Apr 2020) +- TODO: retry on the redirected-to URL + + Closes #5462 -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: 7.70.0 +- mailmap: Nicolas Sterchele -- THANKS: synced with the 7.70.0 release +- [Nicolas Sterchele brought this change] -- headers: copyright range fix + TODO: remove 19.3 section title + + Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on + curl documentation website + + Closes #5561 -- [Rikard Falkeborn brought this change] +- [Martin V brought this change] - doh: Constify some input pointers + test1560: avoid possibly negative association in wording - Closes #5306 + Closes #5549 -- nss: check for PK11_CreateDigestContext() returning NULL +- share: don't set the share flag it something fails - ... to avoid crashes! + When asking for a specific feature to be shared in the share object, + that bit was previously set unconditionally even if the shared feature + failed or otherwise wouldn't work. - Reported-by: Hao Wu - Fixes #5302 - Closes #5303 + Closes #5554 -- travis: bump the wolfssl CI build to use 4.4.0 +- buildconf: remove -print from the find command that removes files - Closes #5301 + It's just too annoying and unnecessary to get a long list of files shown -- copyright updates: adjust year ranges +- RELEASE-NOTES: synced -Marc Hoersken (26 Apr 2020) -- CI: do not include */ci branches in PR builds +- wording: avoid blacklist/whitelist stereotypes - Align Azure Pipelines with GitHub Actions. - -Daniel Stenberg (25 Apr 2020) -- runtests: check for the disabled tests relative srcdir + Instead of discussing if there's value or meaning (implied or not) in + the colors, let's use words without the same possibly negative + associations. - To make it work correctly for out-of-tree builds. + Closes #5546 + +Jay Satiro (9 Jun 2020) +- tool_getparam: fix memory leak in parse_args - Follow-up to 75e8feb6fb08b + Prior to this change in Windows Unicode builds most parsed options would + not be freed. - Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 - Reported-by: Marcel Raad - Closes #5297 - -- runtests: revert commenting out a line I did for debugging + Found using _CrtDumpMemoryLeaks(). - Follow-up to 11091cd4d. It was not meant to be pushed! + Ref: https://github.com/curl/curl/issues/5545 -- smtp: set auth correctly +Daniel Stenberg (8 Jun 2020) +- socks: detect connection close during handshake - Regression since 7.69.0 and 68fb25fa3fcff. + The SOCKS4/5 state machines weren't properly terminated when the proxy + connection got closed, leading to a busy-loop. - The code wrongly assigned 'from' instead of 'auth' which probably was a - copy and paste mistake from other code, leading to that auth could - remain NULL and later cause an error to be returned. + Reported-By: zloi-user on github + Fixes #5532 + Closes #5542 + +- [James Fuller brought this change] + + multi: add defensive check on data->multi->num_alive - Assisted-by: Eric Sauvageau - Fixes #5294 - Closes #5295 + Closes #5540 -Marcel Raad (25 Apr 2020) -- lib: clean up whitespace +- Curl_addrinfo: use one malloc instead of three - This fixes CodeFactor warnings. + To reduce the amount of allocations needed for creating a Curl_addrinfo + struct, make a single larger malloc instead of three separate smaller + ones. + + Closes #5533 -Daniel Stenberg (25 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] +- [Alessandro Ghedini brought this change] - libssh: avoid options override by configuration files - - Previously, options set explicitly through command line options could be - overridden by the configuration files parsed automatically when - ssh_connect() was called. + quiche: update SSLKEYLOGFILE support - By calling ssh_options_parse_config() explicitly, the configuration - files are parsed before setting the options, avoiding the options - override. Once the configuration files are parsed, the automatic - configuration parsing is not executed. + quiche now requires the application to explicitly set the keylog path + for each connection, rather than reading the environment variable + itself. - Fixes #4972 - Closes #5283 - Signed-off-by: Anderson Toshiyuki Sasaki + Closes #5541 -- runtests: when mentions http, kill http/2 too +- tests: add two simple tests for --login-options - Since the http2 test server is a mere proxy that needs to know about the - dynamic port the HTTP server is using, it too needs to get restarted - when the http server is killed. + Test 895 and 896 - as a follow-up to a3e972313b - A regression caused by 80d6515. + Closes #5539 + +- ngtcp2: update with recent API changes - Fixes #5289 - Closes #5291 + Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. + + Assisted-by: Tatsuhiro Tsujikawa + Closes #5538 -- [Yuri Slobodyanyuk brought this change] +- [James Fuller brought this change] - docs: fix two typos + socks: remove unreachable breaks in socks.c and mime.c - Closes #5292 - -- [Emil Engler brought this change] + Closes #5537 - tests/git: ignore mqttd and port files +- tool_cfgable: free login_options at exit - Closes #5290 + Memory leak + Reported-by: Geeknik Labs + Fixes #5535 + Closes #5536 -- tests: make runtests check that disabled tests exists +- libssh2: keep sftp errors as 'unsigned long' - ... and error out if so. Removed '536' from DISABLED as there is no such - test file. + Remove weird work-around for storing the SFTP errors as int instead of + the "unsigned long" that libssh2 actually returns for SFTP errors. - Closes #5288 - -- test1154: set a proper name + Closes #5534 -- select: make Curl_socket_check take timediff_t timeout +Marc Hoersken (6 Jun 2020) +- timeouts: move ms timeouts to timediff_t from int and long - Coverity found CID 1461718: + Now that all functions in select.[ch] take timediff_t instead + of the limited int or long, we can remove type conversions + and related preprocessor checks to silence compiler warnings. - Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > - 9223372036854775807L" is always false regardless of the values of its - operands. This occurs as the logical second operand of "||". + Avoiding conversions from time_t was already done in 842f73de. - Closes #5240 - -- [i-ky brought this change] + Based upon #5262 + Supersedes #5214, #5220 and #5221 + Follow up to #5343 and #5479 + Closes #5490 - libcurl-multi.3: added missing full stop - - Closes #5285 +Daniel Stenberg (6 Jun 2020) +- [François Rigault brought this change] -Jay Satiro (22 Apr 2020) -- transfer: Switch PUT to GET/HEAD on 303 redirect + openssl: set FLAG_TRUSTED_FIRST unconditionally - Prior to this change if there was a 303 reply to a PUT request then - the subsequent request to respond to that redirect would also be a PUT. - It was determined that was most likely incorrect based on the language - of the RFCs. Basically 303 means "see other" resource, which implies it - is most likely not the same resource, therefore we should not try to PUT - to that different resource. + On some systems, openssl 1.0.x is still the default, but it has been + patched to contain all the recent security fixes. As a result of this + patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be + defined, while the previous behavior of openssl to not look at trusted + chains first, remains. - Refer to the discussions in #5237 and #5248 for more information. + Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to + probe for the behavior of openssl based on the existence ofmacros. - Fixes https://github.com/curl/curl/issues/5237 - Closes https://github.com/curl/curl/pull/5248 + Closes #5530 -Daniel Stenberg (22 Apr 2020) -- lib/mk-ca-bundle: skip empty certs +- server/util: fix logmsg format using curl_off_t argument - Reviewed-by: Emil Engler - Reported-by: Ashwin Metpalli - Fixes #5278 - Closes #5280 - -- version: skip idn2_check_version() check and add precaution + ... this caused segfaults on armv7. - A gcc-10's -fanalyze complaint made me spot and do these improvements. + Regression added in dd0365d560aea5a (7.70.0) - Closes #5281 + Reviewed-by: Jay Satiro + Closes #5529 - RELEASE-NOTES: synced -- [Brian Bergeron brought this change] +- [Cherish98 brought this change] - curl.h: update comment typo - - "routines with be invoked" -> "routines will be invoked" + socks: fix expected length of SOCKS5 reply - Closes #5279 - -- [Emil Engler brought this change] - - GnuTLS: Don't skip really long certificate fields + Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the + reply ATYP is X'01'. This resulted in erroneously expecting more bytes + when the request length is greater than the reply length (e.g., when + remotely resolving the hostname). - Closes #5271 + Closes #5527 -- gnutls: bump lowest supported version to 3.1.10 - - GnuTLS 3.1.10 added new functions we want to use. That version was - released on Mar 22, 2013. Removing support for older versions also - greatly simplifies the code. +Marc Hoersken (5 Jun 2020) +- .gitignore: add directory containing the stats repo - Ref: #5271 - Closes #5276 + Since the new curl/stats repository is designed to be + checked out into the curl repository working tree as stats/ + it should be on the ignore list to aid in commit staging. -- mqtt: make NOSTATE get within the debug name array +Daniel Stenberg (5 Jun 2020) +- [Adnan Khan brought this change] -- tests: run the RTSP test server on a dynamic port number + HTTP3.md: clarify cargo build directory - To avoid port collisions. + Cargo needs to be called from within the 'quiche' directory. - Closes #5272 + Closes #5522 -- tests: add %NOLISTENPORT and use it - - The purpose with this variable is to provide a port number that is - reasonably likely to not have a listener on the local host so that tests - can try connect failures against it. It uses port 47 - "reserved" - according to IANA. - - Updated six tests to use it instead of the previous different ports. +- user-agent.d: spell out what happens given a blank argument - Assisted-by: Emil Engler - Closes #5270 + Closes #5525 -- mqtt: remove code with no purpose +- trailers: switch h1-trailer logic to use dynbuf - Detected by Coverity. CID 1462319. + In the continued effort to remove "manual" realloc schemes. - "The same code is executed when the condition result is true or false, - because the code in the if-then branch and after the if statement is - identical." + Closes #5524 + +- CURLINFO_ACTIVESOCKET.3: clarify the description - Closes #5275 + Reported-by: Jay Satiro + Fixes #5299 + Closes #5520 -- mqtt: fix Curl_read() error handling while reading remaining length +- mailmap: Don J Olmstead + +- configure: only strip first -L from LDFLAGS - Detected by Coverity. CID 1462320. + In the logic that works out if a given OpenSSL path works, it stripped + off a possibly leading -L flag using an incorrect sed pattern which + would remove all instances of -L in the string, including if the path + itself contained that two-letter sequence! - Closes #5274 - -- server/tftpd: fix compiler warning + The same pattern was used and is now updated in multiple places. Now it + only removes -L if it starts the strings. - Follow-up from 369ce38ac1d - Reported-by: Marc Hörsken + Reported-by: Mohamed Osama + Fixes #5519 + Closes #5521 -- http: free memory when Alt-Used header creation fails due to OOM +Peter Wu (4 Jun 2020) +- quiche: advertise draft 28 support - Reported-by: James Fuller - Fixes #5268 - Closes #5269 - -Daniel Gustafsson (20 Apr 2020) -- lib: fix typos in comments and errormessages + Fix the verbose message while at it, quiche currently supports draft + 27 and draft 28 simultaneously. - This fixes a few randomly spotted typos in recently merged code, most - notably one in a userfacing errormessage the schannel code. + Closes #5518 -Daniel Stenberg (20 Apr 2020) -- tests: run the SOCKS test server on a dynamic port number +Daniel Stenberg (4 Jun 2020) +- KNOWN_BUGS: RTSP authentication breaks without redirect support - Closes #5266 - -- [Johannes Schindelin brought this change] + Closes #4750 - multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - - When cURL is compiled with support for multiple SSL backends, it is - possible to configure an SSL backend via `curl_global_sslset()`, but - only *before* `curl_global_init()` was called. +Jay Satiro (4 Jun 2020) +- projects: Add crypt32.lib to dependencies for all OpenSSL configs - If another SSL backend should be used after that, a user might be - tempted to call `curl_global_cleanup()` to start over. However, we did - not foresee that use case and forgot to reset the SSL backend in that - cleanup. + Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO + need crypt32. - Let's allow that use case. + Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. - Fixes #5255 - Closes #5257 - Reported-by: davidedec on github - Signed-off-by: Johannes Schindelin - -- tests: run the TFTP test server on a dynamic port number + The changes that are in this commit were made by script. - Picking a dynamic unused port is better than a fixed to avoid the - collision risk. + Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 - Closes #5265 + Closes https://github.com/curl/curl/pull/5516 -- mqtt: improve the state machine - - To handle PUBLISH before SUBACK and more. +Marc Hoersken (3 Jun 2020) +- CI/macos: fix 'is already installed' errors by using bundle - Updated the existing tests and added three new ones. + Avoid failing CI builds due to nghttp2 being already installed. - Reported-by: Christoph Krey - Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html - Closes #5246 + Closes #5513 -- runtests: always put test number in servercmd file +Daniel Stenberg (3 Jun 2020) +- altsvc: fix 'dsthost' may be used uninitialized in this function - RELEASE-NOTES: synced -- release-notes.pl: fix parsing typo - -James Fuller (20 Apr 2020) -- ensure all references to ports are replaced by vars - -- add more alt-svc test coverage +- urldata: let the HTTP method be in the set.* struct + + When the method is updated inside libcurl we must still not change the + method as set by the user as then repeated transfers with that same + handle might not execute the same operation anymore! + + This fixes the libcurl part of #5462 + + Test 1633 added to verify. + + Closes #5499 -Daniel Stenberg (20 Apr 2020) -- test1247: use http server to get the port number set +- hostip: fix the memory-leak introduced in 67d2802 - Follow-up to 0f5db7b263f + Fixes #5503 + Closes #5504 -- runtests: use a unix domain socket path with the pid in the name +- test970: make it require proxy support - To make it impossible for test cases to access the file name without - using the proper variable for the purpose. + This test verifies the -w %json output and the test case includes a full + generated "blob". If there's no proxy support built into libcurl, it + will return an error for proxy related info variables and they will not + be included in the json, thus causing a mismatch and this test fails. - Closes #5264 + Reported-by: Marc Hörsken + Fixes #5501 + Closes #5502 -Daniel Gustafsson (19 Apr 2020) -- [Tom brought this change] +- [Radoslav Georgiev brought this change] - src: Remove C99 constructs to ensure C89 compliance + examples/http2-down/upload: add error checks - This fixes the error: 'for' loop initial declaration used outside C99 - mode by declaring the loop increment variable in the beginning of the - block instead of inside the for loop. + If `index.html` does not exist in the directory from which the example + is invoked, the fopen(upload, "rb") invocation in `setup` would fail, + returning NULL. This value is subsequently passed as the FILE* argument + of the `fread` invocation in the `read_callback` function, which is the + actual cause of the crash (apparently `fread` assumes that argument to + be non-null). - Fixes #5254 - Reviewed-by: Daniel Gustafsson - -Daniel Stenberg (19 Apr 2020) -- runtests: dummy init the ports variables to avoid warnings + In addition, mitigate some possible crashes of similar origin. - ... and generate something that can help debug test cases. + Closes #5463 -- [Patrick Monnerat brought this change] - - mime: properly check Content-Type even if it has parameters - - New test 669 checks this fix is effective. - - Fixes #5256 - Closes #5258 - Reported-by: thanhchungbtc on github +- [kotoriのねこ brought this change] -- tests/FILEFORMAT: converted to markdown and extended + examples/ephiperfifo: turn off interval when setting timerfd - Closes #5261 - -- test1245: make it work with dynamic FTP server port - -- test1055: make it work with dynamic FTP port + Reported-by: therealhirudo on github + Fixes #5485 + Closes #5497 -- test1028: make it run on dynamic FTP server port +- [Saleem Abdulrasool brought this change] -- tests: move pingpong server to dynamic listening port + vtls: repair the build with `CURL_DISABLE_PROXY` - FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic - ports + `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` + is enabled. Repair the build with that configuration. - Test 842-845 are unfortunately a bit hard to move over to this concept - right now and require "default port" still... - -- test1056: work with dynamic HTTP ipv6 port + Follow-up to f3d501dc67 + + Closes #5498 -- test1448: work with dynamic HTTP server port +- transfer: remove k->str NULL check + + "Null-checking k->str suggests that it may be null, but it has already + been dereferenced on all paths leading to the check" - and it can't + legally be NULL at this point. Remove check. + + Detected by Coverity CID 1463884 + + Closes #5495 -- tests: introduce preprocessed test cases +Marc Hoersken (1 Jun 2020) +- select: always use Sleep in Curl_wait_ms on Win32 - The runtests script now always performs variable replacement on the - entire test source file before the test gets executed, and saves the - updated version in a temporary file (log/test[num]) so that all test - case readers/servers can use that version (if present) and thus enjoy - the powers of test case variable substitution. + Since Win32 almost always will also have USE_WINSOCK, + we can reduce complexity and always use Sleep there. - This is necessary to allow complete port number freedom. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Test 309 is updated to work with a non-fixed port number thanks to this. + Follow up to #5343 + Closes #5489 -- tests: make 2006-2010 handle different port number lengths +Daniel Stenberg (31 May 2020) +- conncache: download buffer needs +1 size for trailing zero + + Follow-up to c4e6968127e + Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 -- tests: run the sws server on "any port" +Marc Hoersken (31 May 2020) +- azure: use matrix strategy to avoid configuration redundancy - Makes the test servers for HTTP and Gopher pop up on a currently unused - port and runtests adapts to that! + This also includes the following changes: - Closes #5247 - -Marc Hoersken (18 Apr 2020) -- sockfilt: tidy variable naming and data structure in select_ws + - Use the same timeout for all jobs on Linux (60 minutes) + and Windows (90 minutes) + - Use CLI stable apt-get install -y instead of apt install + which warns about that and run apt-get update first + - Enable MQTT for Windows msys2 builds instead of + legacy msys1 builds + - Add ./configure --prefix parameter to the msys2 builds + - The MSYSTEM environment variable is now preset inside + the container images for the msys2 builds - This commit does not introduce any logical changes to the code. + Note: on Azure Pipelines the matrix strategy is basically + just a simple list of job copies and not really a matrix. - Reviewed-by: Jay Satiro and Marcel Raad - Closes #5238 - -Daniel Stenberg (17 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] + Closes #5468 - libssh: Use new ECDSA key types to check known hosts +Daniel Stenberg (30 May 2020) +- build: disable more code/data when built without proxy support - From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA - keys depending on the curve. + Added build to travis to verify - Signed-off-by: Anderson Toshiyuki Sasaki - Fixes #5252 - Closes #5253 + Closes #5466 -Marcel Raad (17 Apr 2020) -- appveyor: add Unicode winbuild jobs +- url: alloc the download buffer at transfer start - These are cheap as they don't build tests. + ... and free it as soon as the transfer is done. It removes the extra + alloc when a new size is set with setopt() and reduces memory for unused + easy handles. - Closes https://github.com/curl/curl/pull/5063 + In addition: the closure_handle now doesn't use an allocated buffer at + all but the smallest supported size as a stack based one. + + Closes #5472 -Daniel Stenberg (16 Apr 2020) -- mqttd: s/errno/SOCKERRNO +- timeouts: change millisecond timeouts to timediff_t from time_t - To behave proper on Windows - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 - Closes #5241 + For millisecond timers we like timediff_t better. Also, time_t can be + unsigned so returning a negative value doesn't work then. + + Closes #5479 -- buildconf: use find -execdir instead, remove -print and the ares files +Marc Hoersken (30 May 2020) +- select: add overflow checks for timeval conversions - Follow-up to 1e41bec96a6e + Using time_t and suseconds_t if suseconds_t is available, + long on Windows (maybe others in the future) and int elsewhere. - Suggested-by: Marc Hörsken + Also handle case of ULONG_MAX being greater or equal to INFINITE. + + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Part of #5343 -- [Alexander V. Tikhonov brought this change] +- select: use timediff_t instead of time_t and int for timeout_ms + + Make all functions in select.[ch] take timeout_ms as timediff_t + which should always be large enough and signed on all platforms + to take all possible timeout values and avoid type conversions. + + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Replaces #5107 and partially #5262 + Related to #5240 and #5286 + Closes #5343 - buildconf: avoid using tempfile when removing files +- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' - Closes #5213 + GCC 10 warns about this with warning: implicit conversion + from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] + + Since 'expected_result' is not really of type 'CURLcode' and + it is not exposed in any way, we can just use 'SANITIZEcode'. + + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + + Closes #5476 -- copyright: bump the copyright year range +- tests/libtest: fix undefined reference to 'curlx_win32_fopen' + + Since curl_setup.h now makes use of curlx_win32_fopen for Win32 + builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, + we need to include the relevant files for tests using fopen, + because the libtest sources are also including curl_setup.h + + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + + Follow up to #3784 (ffdddb45d9) + Closes #5475 -- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords +- appveyor: add non-debug plain autotools-based build + + This should enable us to catch linking issues with the + testsuite early, like the one described/fixed in #5475. + + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + + Closes #5477 -- [JP Mens brought this change] +Daniel Stenberg (29 May 2020) +- RELEASE-NOTES: synced - docs/MQTT: replace confusing 80 by 75 +- Revert "buildconf: use find -execdir" - I was a bit surprised by the `80`: first thought: what's HTTP doing - here? ;) + This partially reverts commit c712009838f44211958854de431315586995bc61. - Closes #5236 + Keep the ares_ files removed but bring back the older way to run find, + to make it work with busybox's find, as apparently that's being used. + + Reported-by: Max Peal + Fixes #5483 + Closes #5484 -- [Brad King brought this change] +- server/sws: fix asan warning on use of uninitialized variable - cmake: Avoid MSVC C4273 warnings in send/recv checks +- libssh2: improved error output for wrong quote syntax - We use `check_c_source_compiles` to check possible send/recv signatures - by reproducing the forward declarations from system headers. On Windows - the `winsock2.h` header adds dll linkage settings to its forward - declaration. If ours does not match the compiler warns: + Reported-by: Werner Stolz - warning C4273: 'recv': inconsistent dll linkage + Closes #5474 + +- mk-lib1521: generate code for testing BLOB options as well - Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so - that our linkage is consistent with that from `winsock2.h`. + Follow-up to cac5374298b3 - Fixes #4764 - Closes #5232 + Closes #5478 -Jay Satiro (14 Apr 2020) -- KNOWN_BUGS: Add entry 'Blocking socket operations' +- configure: repair the check if argv can be written to - - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of - known blocking operations. + Due to bad escaping of the test code, the test wouldn't build and thus + result in a negative test result, which would lead to the unconditional + assumption that overwriting the arguments doesn't work and thus curl + would never hide credentials given in the command line, even when it + would otherwise be possible. - - New known bugs entry 'Blocking socket operations in non-blocking API' - that directs to the TODO's list of known blocking operations. + Regression from commit 2d4c2152c (7.60.0) - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 + Reported-by: huzunhao on github + Fixes #5470 + Closes #5471 + +Peter Wu (28 May 2020) +- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - Reported-by: Marc Hoersken + Otherwise the build might fail due to missing source files, as + demonstrated by the recent keylog.c addition on an existing build dir. - Closes https://github.com/curl/curl/pull/5216 + Closes #5469 -Marc Hoersken (14 Apr 2020) -- test2043: use revoked.badssl.com instead of revoked.grc.com +Daniel Stenberg (28 May 2020) +- urldata: fix comments: Curl_done() is called multi_done() now - The certificate of revoked.grc.com has expired on 2020-04-13. + ... since 575e885db + +Peter Wu (27 May 2020) +- ngtcp2: use common key log routine for better thread-safety - Reviewed-by: Jay Satiro + Tested with ngtcp2 built against the OpenSSL library. Additionally + tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). - Closes #5233 + The TLS backend (independent of QUIC) may or may not already have opened + the keylog file before. Therefore Curl_tls_keylog_open is always called + to ensure the file is open. -- sockfilt: fix broken pipe on Windows to be ready in select_ws +- wolfssl: add SSLKEYLOGFILE support - Closes #5228 - -Daniel Stenberg (14 Apr 2020) -- RELEASE-NOTES: synced + Tested following the same curl and tshark commands as in commit + "vtls: Extract and simplify key log file handling from OpenSSL" using + WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with + `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. + + Full support for this feature requires certain wolfSSL build options, + see "Availability note" in lib/vtls/wolfssl.c for details. + + Closes #5327 -- scripts/release-notes: fix duplicate output header +- vtls: Extract and simplify key log file handling from OpenSSL + + Create a set of routines for TLS key log file handling to enable reuse + with other TLS backends. Simplify the OpenSSL backend as follows: + + - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. + - Do not perform dynamic memory allocation when preparing a log entry. + Unless the TLS specifications change we can suffice with a reasonable + fixed-size buffer. + - Simplify state tracking when SSL_CTX_set_keylog_callback is + unavailable. My original sslkeylog.c code included this tracking in + order to handle multiple calls to SSL_connect and detect new keys + after renegotiation (via SSL_read/SSL_write). For curl however we can + be sure that a single master secret eventually becomes available + after SSL_connect, so a simple flag is sufficient. An alternative to + the flag is examining SSL_state(), but this seems more complex and is + not pursued. Capturing keys after server renegotiation was already + unsupported in curl and remains unsupported. + + Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f + (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) + against an OpenSSL 1.1.1f server configured with: + + # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 + openssl s_server -www -tls1 + # Likewise, but fail the server handshake. + openssl s_server -www -tls1 -Verify 2 + # TLS 1.3 test. No need to test the failing server handshake. + openssl s_server -www -tls1_3 + + Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly + written using Wireshark. For the first and third case, expect four + matches per connection (decrypted Server Finished, Client Finished, HTTP + Request, HTTP Response). For the second case where the handshake fails, + expect a decrypted Server Finished only. + + tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ + -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ + -dtls.port==4433,http -ohttp.desegment_body:FALSE \ + -Y 'tls.handshake.verify_data or http' + + A single connection can easily be identified via the `tcp.stream` field. -- github/workflow: enable MQTT in the macOS debug build +Daniel Stenberg (27 May 2020) +- FILEFORMAT: add more features that tests can depend on -- azure: add mqtt support to one of the Windows builds +- [Michael Kaufmann brought this change] -- travis: add mqtt job on Linux + transfer: close connection after excess data has been read + + For HTTP 1.x, it's a protocol error when the server sends more bytes + than announced. If this happens, don't reuse the connection, because the + start position of the next response is undefined. + + Closes #5440 -- tests: add four MQTT tests 1190 - 1193 +- [Estanislau Augé-Pujadas brought this change] -- tests: add the mqtt test server mqttd + Revert "ssh: ignore timeouts during disconnect" + + This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in + curl 7.54.1. + + Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html + Closes #5465 -- tests: support hex encoded data and mqtt server +- urldata: connect related booleans live in struct ConnectBits - The mqtt server is started using a "random" port. + And remove a few unused booleans! + + Closes #5461 -- [Björn Stenberg brought this change] +- hostip: on macOS avoid DoH when given a numerical IP address + + When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that + numerical IP addresses still need to get "resolved" - but not with DoH. + + Reported-by: Viktor Szakats + Fixes #5454 + Closes #5459 - mqtt: add new experimental protocol +- ngtcp2: cleanup memory when failing to connect - Closes #5173 + Reported-by: Peter Wu + Fixes #5447 (the ngtcp2 side of it) + Closes #5451 -- TODO: Consider convenience options for JSON and XML? +- quiche: clean up memory properly when failing to connect - Closes #5203 + Addresses the quiche side of #5447 + Reported-by: Peter Wu + Closes #5450 -- tool: do not declare functions with Curl_ prefix +- cleanup: use a single space after equals sign in assignments + +- url: accept "any length" credentials for proxy auth - To avoid collision risks with private libcurl symbols when linked with - static versions (or just versions not hiding internal symbols). + They're only limited to the maximum string input restrictions, not to + 256 bytes. - Reported-by: hydra3333 on github - Fixes #5219 - Closes #5234 + Added test 1178 to verify + + Reported-by: Will Roberts + Fixes #5448 + Closes #5449 -- [Nathaniel R. Lewis brought this change] +- [Maksim Stsepanenka brought this change] - cmake: add aliases so exported target names are available in tree + test1167: fixes in badsymbols.pl - Reviewed-by: Brad King - Closes #5206 + Closes #5442 -- version: increase buffer space for ssl version output +- altsvc: fix parser for lines ending with CRLF - To avoid it getting truncated, especially when several SSL backends are - built-in. + Fixed the alt-svc parser to treat a newline as end of line. - Reported-by: Gisle Vanem - Fixes #5222 - Closes #5226 + The unit tests in test 1654 were done without CRLF and thus didn't quite + match the real world. Now they use CRLF as well. + + Reported-by: Peter Wu + Assisted-by: Peter Wu + Assisted-by: Jay Satiro + Fixes #5445 + Closes #5446 -Marc Hoersken (13 Apr 2020) -- cirrus: no longer ignore test 504 which is working again +Viktor Szakats (25 May 2020) +- all: fix codespell errors - The test is working again, because TCP blackholing is disabled. + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes https://github.com/curl/curl/pull/5452 -- appveyor: completely disable tests that fail to timeout early +Peter Wu (25 May 2020) +- ngtcp2: fix build with current ngtcp2 master implementing draft 28 - The tests changed from ignored to disabled are tests that are - about connecting to non-listening socket. On AppVeyor these - tests are not reliable, because for some unknown reason the - connect is not timing out before the test time limit is reached. + Based on client.cc changes from ngtcp2. Tested with current git master, + ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. + + Fixes #5444 + Closes #5443 -Daniel Stenberg (13 Apr 2020) -- test1908: avoid using fixed port number in test data +Daniel Stenberg (25 May 2020) +- RELEASE-NOTES: synced - Closes #5225 + moved the new setopts up to a "change" -Jay Satiro (12 Apr 2020) -- [Andrew Kurushin brought this change] +- RELEASE-NOTES: synced - schannel: Fix blocking timeout logic +- copyright: updated year ranges out of sync - - Fix schannel_send for the case when no timeout was set. + ... and whitelisted a few more files in the the copyright.pl script. + +- [Gilles Vollant brought this change] + + setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency - Prior to this change schannel would error if the socket was not ready - to send data and no timeout was set. + Closes #5431 + +- curl: remove -J "informational" written on stdout + + curl would previously show "curl: Saved to filename 'name from header'" + if -J was used and a name was picked from the Content-Disposition + header. That output could interfer with other stdout output, such as -w. + + This commit removes that output line. + Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html + Reported-by: Коваленко Анатолий Викторович + Closes #5435 + +Peter Wu (22 May 2020) +- travis: simplify quiche build instructions wrt boringssl - This commit is similar to parent commit 89dc6e0 which recently made the - same change for SOCKS, for the same reason. Basically it was not well - understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms - but actually means no timeout. + quiche builds boringssl as static library, reuse that instead of + building another shared library. - Fixes https://github.com/curl/curl/issues/5177 - Closes https://github.com/curl/curl/pull/5221 + Closes #5438 -- socks: Fix blocking timeout logic +- configure: fix pthread check with static boringssl - - Document in Curl_timeleft's comment block that returning 0 signals no - timeout (ie there's infinite time left). + A shared boringssl/OpenSSL library requires -lcrypto only for linking. + A static build additionally requires `-ldl -lpthread`. In the latter + case `-lpthread` is added to LIBS which prevented `-pthread` from being + added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. + +Daniel Stenberg (22 May 2020) +- Revert "sendf: make failf() use the mvsnprintf() return code" - - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. + This reverts commit 74623551f306990e70c7c5515b88972005604a74. - Prior to this change if the timeout had a value of 0 and that was passed - to SOCKET_READABLE it would return right away instead of blocking. That - was likely because it was not well understood that when Curl_timeleft - returns 0 it is not a timeout of 0 ms but actually means no timeout. + Instead mark the function call with (void). Getting the return code and + using it instead triggered Coverity warning CID 1463596 because + snprintf() can return a negative value... - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + Closes #5441 + +- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' - Closes https://github.com/curl/curl/pull/5220 + Reported-by: Billyzou0741326 on github + Fixes #5432 + Closes #5436 -- [Marc Hoersken brought this change] +- tests/server/util.h: add extern to silence compiler warning + + Follow-up from a3b0699d5c1 - gopher: check remaining time left during write busy loop +- typecheck-gcc.h: fix the OFF_T check - Prior to this change gopher's blocking code would block forever, - ignoring any set timeout value. + The option number also needs to be less than CURLOPTTYPE_BLOB. - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Follow-up to cac5374298 + Reported-by: Jeroen Ooms + Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 + +- TODO: --dry-run - Similar to #5220 and #5221 - Closes #5214 + Closes #5426 -Daniel Stenberg (13 Apr 2020) -- [Dirkjan Bussink brought this change] +- TODO: Ratelimit or wait between serial requests + + Closes #5406 - gnutls: ensure TLS 1.3 when SRP isn't requested +- tool_paramhlp: fixup C89 mistake - When SRP is requested in the priority string, GnuTLS will disable - support for TLS 1.3. Before this change, curl would always add +SRP to - the priority list, effectively always disabling TLS 1.3 support. + Follow-up to c5f0a9db22. + +- [Siva Sivaraman brought this change] + + tool_paramhlp: fixed potentially uninitialized strtol() variable - With this change, +SRP is only added to the priority list when SRP - authentication is also requested. This also allows updating the error - handling here to not have to retry without SRP. This is because SRP is - only added when requested and in that case a retry is not needed. + Seems highly unlikely to actually be possible, but better safe than + sorry. - Closes #5223 + Closes #5417 -Marc Hoersken (12 Apr 2020) -- tests/server: add hidden window to gracefully handle WM_CLOSE - - Forward Window events as signals to existing signal event handler. +- [Siva Sivaraman brought this change] -- tests/server: add CTRL event handler for Win32 consoles + tool_operate: fixed potentially uninitialized variables - Forward CTRL events as signals to existing signal event handler. + ... in curl_easy_getinfo() calls. They're harmless but clearing the + variables makes the code safer and comforts the reader. + + Closes #5416 -- tests/server: move all signal handling routines to util.[ch] +- sha256: move assign to the declaration line - Avoid code duplication to prepare for portability enhancements. + Follow-up to fae30656. Should've been squashed with that commit... -Daniel Stenberg (12 Apr 2020) -- compressed.d: stress that the headers are not modified +- [Siva Sivaraman brought this change] + + sha256: fixed potentially uninitialized variable - Suggested-by: Michael Osipov - Assisted-by: Jay Satiro - Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 - Closes #5217 + Closes #5414 -Marc Hoersken (11 Apr 2020) -- tests/server/util.c: use curl_off_t instead of long for pid +- sendf: make failf() use the mvsnprintf() return code - Avoid potential overflow of huge PIDs on Windows. + ... and avoid a strlen() call. Fixes a MonocleAI warning. - Related to #5188 - Assisted-by: Marcel Raad + Reported-by: MonocleAI + Fixes #5413 + Closes #5420 -- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows - - Since the Windows versions of both programs would write Windows - PIDs to their pidfiles which we cannot handle, we need to use - our known perl.exe Cygwin/msys PID together with exec() in order - to tie the spawned processes to the existance of our perl.exe +- hostip: make Curl_printable_address not return anything - The perl.exe that is executing secureserver.pl and sshserver.pl - has a Cygwin/msys PID, because it is started inside Cygwin/msys. + It was not used much anyway and instead we let it store a blank buffer + in case of failure. - Related to #5188 + Reported-by: MonocleAI + Fixes #5411 + Closes #5418 -- tests: add Windows compatible pidwait like pidkill and pidterm +- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) - Related to #5188 + They're done on purpose, make that visible in the code. + Reported-by: MonocleAI + Fixes #5412 + Closes #549 -- tests: fix conflict between Cygwin/msys and Windows PIDs - - Add 65536 to Windows PIDs to allow Windows specific treatment - by having disjunct ranges for Cygwin/msys and Windows PIDs. - - See also: - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=b5e1003722cb14235c4f166be72c09acdffc62ea - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe +- TODO: forbid TLS post-handshake auth and do TLS record padding - Replaces #5178 - Closes #5188 + Closes #5396 + Closes #5398 -Daniel Stenberg (11 Apr 2020) - RELEASE-NOTES: synced -- release-notes.pl: detect the start of the references in cleanup mode - -- Revert "file: on Windows, refuse paths that start with \\" - - This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. +- dynbuf: return NULL when there's no buffer length - Reminded-by: Chris Roberts - Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html + ... as returning a "" is not a good idea as the string is supposed to be + allocated and returning a const string will cause issues. - Closes #5215 + Reported-by: Brian Carpenter + Follow-up to ed35d6590e72c + Closes #5405 -Jay Satiro (11 Apr 2020) -- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE +Peter Wu (16 May 2020) +- travis: upgrade to bionic, clang-9, improve readability - - If loss of data may occur converting a timediff_t to time_t and - the time value is > TIME_T_MAX then treat it as TIME_T_MAX. + Changes, partially to reduce build failures from external dependencies: + - Upgrade Ubuntu and drop unnecessary third-party repos. + - Properly clone apt config to ensure retries. + - Upgrade to clang-9 from the standard repos. + - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey + fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use + focal everywhere yet since Travis CI has not documented this option. + In focal, python-impacket (Py2.7) has been removed, leaving only + python3-impacket. Since it is only needed for SMB tests and not SSH, + skip it for the libssh job since it might need more work. + - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. - This is a follow-up to 8843678 which removed the (time_t) typecast - from the macros so that conversion warnings could be identified. + Non-functional cleanups: + - Simplify test matrix, drop redundant os and compiler keys. + - Deprecation fixes: remove sudo, rename matrix -> jobs. + - Every job has an 'env' key, put this key first in a list item. - Closes https://github.com/curl/curl/pull/5199 + Closes #5370 -- test1148: tolerate progress updates better (again) +- travis: whitespace-only changes for consistency - - Ignore intermediate progress updates. + Automatically apply a consistent indentation with: - - Support locales that use a character other than period as decimal - separator (eg 100,0%). + python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' - test1148 checks that the progress finishes at 100% and has the right - bar width. Prior to this change the test assumed that the only progress - reported for such a quick transfer was 100%, however in rare instances - (like in the CI where transfer time can slow considerably) there may be - intermediate updates. For example, below is stderrlog1148 from a failed - CI run with explicit \r and \n added (it is one line; broken up so that - it's easier to understand). + followed by manually re-indenting three comments. - \r - \r################################## 48.3% - \r######################################################################## 100.0% - \n + Closes #5370 + +- CMake: add libssh build support - Closes https://github.com/curl/curl/pull/5194 + Closes #5372 -Marc Hoersken (10 Apr 2020) -- sshserver.pl: use cached Win32 environment check variable +Daniel Stenberg (15 May 2020) +- KNOWN_BUGS: wolfssh: publickey auth doesn't work + + Closes #4820 -- appveyor: partially revert 3413a110 to keep build without proxy +- KNOWN_BUGS: OS400 port requires deprecated IBM library - Ref: #5211 and #4526 - Reported-by: Marcel Raad + Closes #5176 -- appveyor: ignore failing 'connect to non-listening proxy' tests +- [Vyron Tsingaras brought this change] + + http2: keep trying to send pending frames after req.upload_done - Closes #5211 + Fixes #1410 + Closes #5401 -- CI/macos: convert CRLF to LF and align indentation +- [Gilles Vollant brought this change] -Daniel Stenberg (9 Apr 2020) -- url: allow non-HTTPS altsvc-matching for debug builds + setopt: support certificate options in memory with struct curl_blob - This is already partly supported but this part was missing. - Reported-by: James Fuller + This change introduces a generic way to provide binary data in setopt + options, called BLOBs. - Closes #5205 - -- server/resolve: remove AI_CANONNAME to make macos tell the truth + This change introduces these new setopts: - With this bit set, my mac successfully resolves "ip6-localhost" when in - fact there is no such host known to my machine! That in turn made test - 241 wrongly execute and fail. + CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, + CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. - Closes #5202 + Reviewed-by: Daniel Stenberg + Closes #5357 -- runtests: fix warning about using an undefined variable +- source cleanup: remove all custom typedef structs - Follow-up from 4d939ef6ceb2db1 + - Stick to a single unified way to use structs + - Make checksrc complain on 'typedef struct {' + - Allow them in tests, public headers and examples + + - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually + typedef different types/structs depending on build conditions. + + Closes #5338 -- release-notes: fix the initial reference list output +- travis: remove the .checksrc fiddling -- github actions: run when pushed to master or */ci + PRs - - Avoid double-builds when using "local" branches for PRs. For both macos - and fuzz jobs. +- ftp: make domore_getsock() return the secondary socket properly - Closes #5201 + Previously, after PASV and immediately after the data connection has + connected, the function would only return the control socket to wait for + which then made the data connection simply timeout and not get polled + correctly. This become obvious when running test 1631 and 1632 event- + based. -- runtests: provide nicer errormsg when protocol "dump" file is empty +- test1632: verify FTP through HTTPS-proxy with connection re-use -- [Gilles Vollant brought this change] +- test1631: verify FTP download through HTTPS-proxy - schannel: support .P12 or .PFX client certificates +- sws: as last resort, get test number from server cmd file - Used with curl command line option like this: --cert - : --cert-type p12 + If it can't be found in the request. Also support --cmdfile to set it to + a custom file name. - Closes #5193 + runtests.pl always writes this file with the test number in it since a + while back. -- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - - test1294: "split request" being when the entire request isn't sent in - the first go, and the remainder is sent in the PERFORM state. A GET - request is otherwise not sending anything during PERFORM. - - test1295: same kind of split but with POST +- ftp: shut down the secondary connection properly when SSL is used - Closes #5197 + Reported-by: Neal Poole + Fixes #5340 + Closes #5385 -- http: don't consider upload done if the request isn't completely sent off +Marcel Raad (14 May 2020) +- KNOWN_BUGS: adapt 5.5 to recent changes - Fixes #4919 - Closes #5197 - -- http: allow Curl_add_buffer_send() to do a short first send by force + It only applies to non-Unicode builds now. + Also merge 5.10 into it as it's effectively a duplicate. - In a debug build, settting the environment variable "CURL_SMALLREQSEND" - will make the first HTTP request send not send more bytes than the set - amount, thus ending up verifying that the logic for handling a split - HTTP request send works correctly. + Closes https://github.com/curl/curl/pull/3784 -- connect: store connection info for QUIC connections +- curl_setup: support Unicode functions to open files on Windows - Restores the --head functionality to the curl utility which extracts - 'protocol' that is stored that way. + Use them only if `_UNICODE` is defined, in which case command-line + arguments have been converted to UTF-8. - Reported-by: James Fuller - Fixes #5196 - Closes #5198 + Closes https://github.com/curl/curl/pull/3784 -- tests/README: update the port numbers list +- tool: support UTF-16 command line on Windows - Since the pipelining server is long gone. - Reported-by: James Fuller - -- select: remove typecast from SOCKET_WRITABLE/READABLE macros + - use `wmain` instead of `main` when `_UNICODE` is defined [0] + - define `argv_item_t` as `wchar_t *` in this case + - use the curl_multibyte gear to convert the command-line arguments to + UTF-8 - So that they don't hide conversions-by-mistake + This makes it possible to pass parameters with characters outside of + the current locale on Windows, which is required for some tests, e.g. + the IDN tests. Out of the box, this currently only works with the + Visual Studio project files, which default to Unicode, and winbuild + with the `ENABLE_UNICODE` option. - Reviewed-by: Jay Satiro - Closes #5190 - -- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also + [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 - Closes #5192 - -- release-notes: output trailing references sorted numerically - -- cleanup: correct copyright year range on a few files + Ref: https://github.com/curl/curl/issues/3747 + Closes https://github.com/curl/curl/pull/3784 -- configure: remove use of -vec-report0 from CFLAGS with icc +- curl_multibyte: add to curlx - ... as it apparently isn't (always) supported. - Reported-by: Alain Miniussi - Fixes #5096 - Closes #5191 - -- warnless: remove code block for icc that didn't work + This will also be needed in the tool and tests. - Reported-by: Alain Miniussi - Fixes #5096 + Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 + Closes https://github.com/curl/curl/pull/3784 -Marc Hoersken (6 Apr 2020) -- dist: add missing setup-win32.h +Daniel Stenberg (14 May 2020) +- url: make the updated credentials URL-encoded in the URL - Follow up to d820224b8b - -Daniel Stenberg (6 Apr 2020) -- RELEASE-NOTES: synced + Found-by: Gregory Jefferis + Reported-by: Jeroen Ooms + Added test 1168 to verify. Bug spotted when doing a redirect. + Bug: https://github.com/jeroen/curl/issues/224 + Closes #5400 -- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance +- tests: add https-proxy support to the test suite - This script helps putting entries in the RELEASE-NOTES using a coherent - style and sorting with a minimal human editing effort - as long as the - first line in the commit message is good enough! There's a short howto - at the top of the file. + Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like + HTTP proxy but with a full TLS connection to the proxy. + + Closes #5399 -- [Dennis Felsing brought this change] +- mailmap: James Fuller - configure: don't check for Security.framework when cross-compiling +- [Major_Tom brought this change] + + vauth/cleartext: fix theoretical integer overflow - Since it checks for the local file, not the cross-compiled one. + Fix theoretical integer overflow in Curl_auth_create_plain_message. - Closes #5189 - -- TODO: Option to make -Z merge lined based outputs on stdout + The security impact of the overflow was discussed on hackerone. We + agreed this is more of a theoretical vulnerability, as the integer + overflow would only be triggerable on systems using 32-bits size_t with + over 4GB of available memory space for the process. - Closes #5175 + Closes #5391 -- lib: never define CURL_CA_BUNDLE with a getenv +Jay Satiro (13 May 2020) +- curl.1: Quote globbed URLs - - it breaks the build (since 6de756c9b1de34b7a1) - - it's not documented and not consistent across platforms - - the curl tool does that getenv magic + - Quote the globbing example URLs that contain characters [] {} since + otherwise they may be interpreted as shell metacharacters. - Bug: https://github.com/curl/curl/commit/6de756c#r38127030 - Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/issues/5388 + Reported-by: John Simpson - Closes #5187 - -Marc Hoersken (5 Apr 2020) -- lib670: use the same Win32 API check as all other lib tests + Closes https://github.com/curl/curl/pull/5394 -- appveyor: use random test server ports based upon APPVEYOR_API_URL +Daniel Stenberg (14 May 2020) +- checksrc: enhance the ASTERISKSPACE and update code accordingly - Avoid conflicts of test server ports with AppVeyor API on localhost. + Fine: "struct hello *world" - Closes #5034 - -- appveyor: sort builds by type and add two new variants + Not fine: "struct hello* world" (and variations) - Related to #5034 and #5063 + Closes #5386 -- appveyor: show failed tests in log even if test is ignored +- docs/options-in-versions: which version added each cmdline option - And print API response with newline only if there is one - -- appveyor: turn disabled tests into ignored result tests - -Daniel Stenberg (5 Apr 2020) -- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" + Added test 971 to verify that the list is in sync with the files in + cmdline-opts. The check also verifies that .d-files that uses Added: + specify the same version number as the options-in-versions file does. - Fixed with #5170 (commit 23a870f2fd041278) + Closes #5381 -- test1566: verify --etag-compare that gets a 304 back +- docs: unify protocol lists - Verifies the fix in #5183 + We boast support for 25 transfer protocols. Make sure the lists are + consistent - Closes #5186 - -- [Kwon-Young Choi brought this change] + Closes #5384 - CURLINFO_CONDITION_UNMET: return true for 304 http status code +- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN - In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the - output file if the server did not transfered a file based on time - condition. In the same manner, getting a 304 HTTP response back from the - server, for example after passing a custom If-Match-* header, also - fulfill this condition. + ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. - Fixes #5181 - Closes #5183 + Reported-by: Michael Kaufmann + Fixes #5374 + Closes #5376 -- [Kwon-Young Choi brought this change] +- tls13-ciphers.d: shorten the Arg - curl: allow both --etag-compare and --etag-save with same file name - - This change inverse the order of processing for the --etag-compare and - --etag-save option to process first --etag-compare. This in turn allows - to use the same file name to compare and save an etag. - - The original behavior of not failing if the etag file does not exists is - conserved. - - Fixes #5179 - Closes #5180 +- sasl-authzid.d: add Arg: and shorten the desc -Viktor Szakats (4 Apr 2020) -- windows: enable UnixSockets with all build toolchains +- cert-type.d: mention the available types in the desc + +- tool: shorten 3 --help descriptions - Extend existing unix socket support in Windows builds to be - enabled for all toolchain vendors or versions. (Previously - it was only supported with certain MSVC versions + more recent - Windows 10 SDKs) + --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort - Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ - Ref: https://github.com/curl/curl/issues/5162 - Closes: https://github.com/curl/curl/pull/5170 - -Daniel Stenberg (4 Apr 2020) -- KNOWN_BUGS: Store TLS context per transfer instead of per connection + gen.pl already warned about these lines but we didn't listen - Closes #5102 - -Marc Hoersken (3 Apr 2020) -- sockfilt: remove redundancy in timeout handling + Closes #5379 + +- configure: the wolfssh backend does not provide SCP - And update other logmsg output in select_ws on Windows. + Closes #5387 -- sockfilt: fix handling of ready closed sockets on Windows +- RELEASE-NOTES: synced + +- url: reject too long input when parsing credentials - Replace the incomplete workaround regarding FD_CLOSE - only signalling once by instead doing a pre-check with - standard select and storing the result for later use. + Since input passed to libcurl with CURLOPT_USERPWD and + CURLOPT_PROXYUSERPWD circumvents the regular string length check we have + in Curl_setstropt(), the input length limit is enforced in + Curl_parse_login_details too, separately. - select keeps triggering on closed sockets on Windows while - WSAEventSelect fires only once with data still available. - By doing the pre-check we do not run in a deadlock - due to waiting forever for another FD_CLOSE event. + Reported-by: Thomas Bouzerar + Closes #5383 -- sockfilt: fix race-condition of waiting threads and event handling +- list-only.d: this option existed already in 4.0 + +Jay Satiro (12 May 2020) +- retry-all-errors.d: Shorten the summary line - Fix race-condition of waiting threads finishing while events are - already being processed which lead to invalid or skipped events. + Follow-up to b995bb5 from a few moments ago. - Use mutex to check for one event at a time or do post-processing. - In addition to mutex-based locking use specific event as signal. + Reported-by: Daniel Stenberg - Closes #5156 + Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 -Daniel Stenberg (2 Apr 2020) -- [Leo Neat brought this change] +- [denzor brought this change] - CI-fuzz: increase fuzz time to 40 minutes + easy: fix dangling pointer on easy_perform fail - Closes #5174 + Closes https://github.com/curl/curl/pull/5363 -Marc Hoersken (2 Apr 2020) -- CI: increase Azure Pipelines timeouts due to performance issues +- tool: Add option --retry-all-errors to retry on any error - The current demand on Azure negatively impacts the CI performance. - -- runtests.pl: log host OS as detected by Perl environment + The "sledgehammer" of retrying. + + Closes https://github.com/curl/curl/pull/5185 -- ftpserver.pl: log before and after data connection is closed +Daniel Stenberg (12 May 2020) +- [James Le Cuirot brought this change] -Daniel Stenberg (1 Apr 2020) -- RELEASE-NOTES: synced + libcurl.pc: Merge Libs.private into Libs for static-only builds + + A project being built entirely statically will call pkg-config with + --static, which utilises the Libs.private field. Conversely it will + not use --static when not being built entirely statically, even if + there is only a static build of libcurl available. This will most + likely cause the build to fail due to underlinking unless we merge the + Libs fields. + + Consider that this is what the Meson build system does when it + generates pkg-config files. + + I have also reflected this in the --libs argument of curl-config even + though REQUIRE_LIB_DEPS always seems to be "yes" anyway. + + Closes #5373 -- RELEASE-PROCEDURE.md: run the copyright.pl script! +- [Peter Wu brought this change] -- vquic/ngtcp2.h: update copyright year range + CMake: fix runtests.pl with CMake, add new test targets - Follow-up to 0736ee73d346a52 + * runtests.pl: + - Fix out-of-tree build under CMake when srcdir is not set. Default + srcdir to the location of runtests.pl. + - Add a hack to allow CMake to use the TFLAGS option as documented + in tests/README and used in scripts/travis/script.sh. + * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie + support (no one should care, it is already EOL.). + * Remove CTest since it defines its own 'test' target with no tests + since all unittests are already broken and not built by default. + * Add new test targets based on the options from Makefile.am. Since + new test targets are rarely added, I opted for duplicating the + runtests.pl options as opposed to creating a new Makefile.inc file. + Use top-level target names (test-x) instead of x-test since that is + used by CI and others. + + Closes #5358 -- [Daiki Ueno brought this change] +- [Peter Wu brought this change] - CI: add build with ngtcp2 + gnutls on Travis CI + CMake: do not build test programs by default + + The default target should only build libcurl and curl. Add a dedicated + 'testdeps' target which will be used later when running tests. Note that + unittests are currently broken in CMake and already excluded. + + Closes #5368 -- [Daiki Ueno brought this change] +- FILEFORMAT: moved up the variables section and further polished - vquic: add support for GnuTLS backend of ngtcp2 +- runtests: remove ftp2 support, not used - Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile - time. Therefore OpenSSL support needs to be explicitly disabled. + We once supported two separate ftp instances in the test suite. Has not + been used the last decade. - Signed-off-by: Daiki Ueno - Closes #5148 - -- [Gisle Vanem brought this change] + Closes #5375 - examples/sessioninfo.c: add include to fix compiler warning +- url: sort the protocol schemes in rough popularity order - Fixes #5171 + When looking for a protocol match among supported schemes, check the + most "popular" schemes first. It has zero functionality difference and + for all practical purposes a speed difference will not be measureable + but it still think it makes sense to put the least likely matches last. + + "Popularity" based on the 2019 user survey. + + Closes #5377 -- misc: copyright year updates +Marc Hoersken (11 May 2020) +- test1238: avoid tftpd being busy for tests shortly following - Follow-up to 7a71965e9 + The tftpd server may still be busy if the total timeout of + 25 seconds has not been reached or no sread error was received + during or after the execution of the timeout test 1238. + + Once the next TFTP test comes around (eg. 1242 or 1243), + those will fail because the tftpd server is still waiting + on data from curl due to the UDP protocol being stateless + and having no connection close. On Linux this error may not + happen, because ICMP errors generated due to a swrite error + can also be returned async on the next sread call instead. + + Therefore we will now just kill the tftpd server after test + 1238 to make sure that the following tests are not affected. + + This enables us to no longer ignore tests 1242, 1243, 2002 + and 2003 on the CI platforms CirrusCI and AppVeyor. + + Assisted-by: Peter Wu + Closes #5364 -- [Harry Sintonen brought this change] +Daniel Stenberg (11 May 2020) +- write-out.d: added "response_code" - build: fixed build for systems with select() in unistd.h +- KNOWN_BUGS: Build with staticly built dependency - Closes #5169 + I rewrote the item 5.4 to be more generic about static dependencies. -- memdebug: don't log free(NULL) +- ROADMAP: remove old entries - ... it serves no purpose and fills up the log. - -- cleanup: insert newline after if() conditions + MQTT - the start has already landed - Our code style mandates we put the conditional block on a separate - line. These mistakes are now detected by the updated checksrc. - -- checksrc: warn on obvious conditional blocks on the same line as if() + tiny-curl - also mostly landed and is a continuous work - Closes #5164 + make menuconfig - basically no interest from users, not pushing there -- [Roger Orr brought this change] +- [Peter Wu brought this change] - cmake: add CMAKE_MSVC_RUNTIME_LIBRARY + travis: Add ngtcp2 and quiche tests for CMake - Fixes #5165 - Closes #5167 + To avoid an explosion of jobs, extend the existing CMake tests with + ngtcp2 and quiche support. macOS was previously moved to GitHub actions, + so the non-Linux case can be dropped. -- [Daiki Ueno brought this change] +- [Peter Wu brought this change] - ngtcp2: update to git master for the key installation API change - - This updates the ngtcp2 OpenSSL backend to follow the API change in - commit 32e703164 of ngtcp2. - - Notable changes are: - - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace - ngtcp2_crypto_derive_and_install_key - - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key - has been removed + CMake: add ENABLE_ALT_SVC option - Fixes #5166 - Closes #5168 + Tested alt-svc with quiche. While at it, add missing MultiSSL reporting + (not tested). -- [Cyrus brought this change] +- [Peter Wu brought this change] - SECURITY.md: minor rephrase + CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) - Closes #5158 - -- output.d: quote the URL when globbing + Add three new CMake Find modules (using the curl license, but I grant + others the right to apply the CMake BSD license instead). - Some shells do globbing of their own unless the URL is quoted, so maybe - encourage this. + This CMake config is simpler than the autotools one because it assumes + ngtcp2 and nghttp3 to be used together. Another difference is that this + CMake config checks whether QUIC is actually supported by the TLS + library (patched OpenSSL or boringssl) since this can be a common + configuration mistake that could result in build errors later. - Co-authored-by: Jay Satiro - Closes #5160 - -- dist: add tests/version-scan.pl to tarball + Unlike autotools, CMake does not warn you that the features are + experimental. The user is supposed to already know that and read the + documentation. It requires a very special build environment anyway. - ... used in test 1177. + Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from + current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure + out which features (libldap-2.4, libssh2) to disable due to conflicts + with boringssl. - Follow-up to a97d826f6de3 - -- test1177: verify that all the CURL_VERSION_ bits are documented + Closes #5359 -- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented +Marc Hoersken (10 May 2020) +- tests/server/tftpd.c: fix include and enhance debug logging - Considered experimental and therefore we can do this. + setjmp.h should only be included if HAVE_SETJMP_H is defined. - Closes #5157 + Add additional log statements to see wether reads and writes + are blocking or finishing before an alarm signal is received. + + Assisted-by: Peter Wu + Part of #5364 -- KNOWN_BUGS: DoH doesn't inherit all transfer options +Daniel Stenberg (10 May 2020) +- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present - Closes #4578 - Closes #4579 + Reported-by: Marcel Raad + Follow-up to 148534db5 + Fixes #5367 + Closes #5369 -- KNOWN_BUGS: DoH leaks memory after followlocation +Marc Hoersken (9 May 2020) +- appveyor: update comments to be clear about toolchain + + - CMake-based MSYS builds use mingw-w64 to cross-compile. + - autotools-based builds are compiled using msys2-devel. + + The difference is that the later ones are not cross-compiled + to Windows and instead require the msys2 runtime to be present. - Closes #4592 + At the moment only the Azure Pipelines CI builds actually + run autotools-based cross-compilation builds for Windows. -- KNOWN_BUGS: "FTPS needs session reuse" +- TODO: update regarding missing Schannel features - Closes #4654 - -- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed - -- TODO: Set custom client ip when using haproxy protocol + Some aspects have already been implemented over the years. - Closes #5125 - -Michael Kaufmann (27 Mar 2020) -- writeout_json: Fix data type issues + 15.1 Client certificates are now supported: - Load long values correctly (e.g. for http_code). + - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 + - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 - Use curl_off_t (not long) for: - - size_download (CURLINFO_SIZE_DOWNLOAD_T) - - size_upload (CURLINFO_SIZE_UPLOAD_T) + 15.2 Ciphers can now be specified through: - The unit for these values is bytes/second, not microseconds: - - speed_download (CURLINFO_SPEED_DOWNLOAD_T) - - speed_upload (CURLINFO_SPEED_UPLOAD_T) + - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 - Fixes #5131 - Closes #5152 + Reviewed-by: Daniel Stenberg and Marcel Raad + Closes #5358 -Daniel Stenberg (27 Mar 2020) -- mailmap: fixup a few author names/fields - - Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and - Svyatoslav Mishyn +Daniel Stenberg (8 May 2020) +- checksrc: close the .checksrc file handle when done reading -- version: add 'cainfo' and 'capath' to version info struct - - Suggested-by: Timothe Litt - URL: https://curl.haxx.se/mail/lib-2020-03/0090.html - Reviewed-by: Jay Satiro +- RELEASE-NOTES: synced - Closes #5150 + And bumped next version to 7.71.0 -- RELEASE-NOTES: synced +- [Gilles Vollant brought this change] -Jay Satiro (26 Mar 2020) -- SSLCERTS.md: Fix example code for setting CA cert file + CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) - Prior to this change the documentation erroneously said use - CURLOPT_CAPATH to set a CA cert file. + Closes #4346 + +- TODO: native IDN support on macOS + +- urlapi: accept :: as a valid IPv6 address - Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html - Reported-by: Timothe Litt + Text 1560 is extended to verify. - Closes https://github.com/curl/curl/pull/5151 + Reported-by: Pavel Volgarev + Fixes #5344 + Closes #5351 -Marc Hoersken (26 Mar 2020) -- sockfilt: add logmsg output to select_ws_wait_thread on Windows +- THANKS-filter: Peter Wang + +- [Peter Wang brought this change] + + *_sspi: fix bad uses of CURLE_NOT_BUILT_IN - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other + instances of QuerySecurityPackageInfo failing, as in + commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. - Closes #5086 + Closes #5355 -Daniel Stenberg (26 Mar 2020) -- docs/make: generate curl.1 from listed files only +- docs/HTTP3: add qlog to the quiche build instruction + +- ngtcp2: introduce qlog support - Previously it rendered the page from files matching "*.d" in the correct - directory, which worked fine in git builds when the files were added but - made it easy to forget adding the files to the dist. + If the QLOGDIR environment variable is set, enable qlogging. - Now, only man page sections listed in DPAGES in Makefile.inc will be - used, thus "forcing" us to update this to get the man page right and get - it included in the dist at the same time. + ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for + QUIC functions that are backend independent. - Ref: #5146 - Closes #5149 + Closes #5353 -- openssl: adapt to functions marked as deprecated since version 3 - - OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES - functions we use. - - Fix the MD4 and SSL_CTX_load_verify_locations warnings. +- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN - In configure, detect OpenSSL v3 and if so, inhibit the deprecation - warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and - until we rewrite the code to use non-deprecated functions we better - ignore these warnings as they don't help us. + That return code is reserved for build-time conditional code not being + present while this was a regular run-time error from a Windows API. - Closes #5139 + Reported-by: wangp on github + Fixes #5349 + Closes #5350 -- dist: add mail-rcpt-allowfails.d to the tarball +- runtests: show elapsed test time with higher precision (ms) + +- RELEASE-NOTES: synced + +- http2: simplify and clean up trailer handling - Reported-by: Maksim Stsepanenka - Reviewed-by: Jat Satiro + Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in + ed35d6590e72. This should make the trailer handling more straight forward and + hopefully less error-prone. - Closes #5146 - -- travis: update the ngtcp2 build to use the latest OpenSSL patch + Deliver the trailer header to the callback already at receive-time. No + longer caches the trailers to get delivered at end of stream. - ... which also makes it OpenSSL 1.1.1d based and not v3. + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 + Closes #5348 -Marc Hoersken (24 Mar 2020) -- CI: remove default Ubuntu build from GitHub Actions +Marc Hoersken (7 May 2020) +- appveyor: disable test 1139 instead of ignoring it - We are already running a very similar Ubuntu build on Travis CI. - The macOS variant of this default build is kept on Github Actions. + Spending time on manpage checking makes no sense + for these builds due to lacking manpage support. -- CI: bring GitHub Actions fuzzing job in line with macOS jobs +- appveyor: disable flaky test 1501 and ignore broken 1056 - Update YAML formatting, job naming and triggers. + Test 1501 is flaky on Windows CI due to being time sensitive + and the testsuite relying on taskkill.exe to check for the + existance of processes which can take to much time itself. + + Test 1056 is broken in autotools-based Windows builds due + to scope ID support missing in these builds at the moment. -- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions +- test613.pl: make tests 613 and 614 work with OpenSSH for Windows - Reduce workload on Azure Pipelines and Travis CI while - consolidating macOS jobs onto less utilized GitHub Actions. + OpenSSH for Windows shows group and other/world permissions as *, + because those concepts do not exist on Windows. It also does not + show the current or parent directory, so we just ignore those. Reviewed-by: Daniel Stenberg - - Closes #5124 + Closes #5328 -Daniel Stenberg (24 Mar 2020) -- config: remove all defines of HAVE_DES_H +Daniel Stenberg (6 May 2020) +- runtests: set +x mode again + +- libssh2: convert over to use dynbuf - As there's no code using it. + In my very basic test that lists sftp://127.0.0.1/tmp/, this patched + code makes 161 allocations compared to 194 in git master. A 17% + reduction. - Closes #5144 + Closes #5336 -- copyright: fix out-of-date copyright ranges and missing headers +- travis: add "qlog" as feature in the quiche build + +- quiche: enable qlog output - Reported by the new script 'scripts/copyright.pl'. The script has a - regex whitelist for the files that don't need copyright headers. + quiche has the potential to log qlog files. To enable this, you must + build quiche with the qlog feature enabled `cargo build --features + qlog`. curl then passes a file descriptor to quiche, which takes + ownership of the file. The FD transfer only works on UNIX. - Removed three (mostly usesless) README files from docs/ + The convention is to enable logging when the QLOGDIR environment is + set. This should be a path to a folder where files are written with the + naming template .qlog. - Closes #5141 + Co-authored-by: Lucas Pardue + Replaces #5337 + Closes #5341 -- packages: add OS400/chkstrings.c to the dist +- urldata.h: remove #define HEADERSIZE, not used anymore - Reported-by: Jon Rumsey - Fixes #5142 - Closes #5143 + Follow-up to ed35d6590e72c -- [Clément Notin brought this change] +- ngtcp2: convert to dynbuf + + Closes #5335 - nghttp2: 1.12.0 required +- connect: make happy eyeballs work for QUIC (again) - since nghttp2_session_set_local_window_size is needed + Follow-up from dbd16c3e256c6c (regression in 7.70.0) - Closes #5140 - -- RELEASE-NOTES: synced + Closes #5334 -- [Calvin Buckley brought this change] +- connect: add two asserts to clue code analyzers in a little - OS400: Update strings for ccsid-ifier - - Fixes build. +- http_proxy: ported to use dynbuf instead of a static size buffer - Closes #5132 + Removes a 16K static buffer from the easy handle. Simplifies the code. -- cirrus: make freebsd ignore the tests instead of skipping - - To allow us to see in the CI logs how they actually behave +- dynbuf: introduce internal generic dynamic buffer functions - Closes #5091 - -- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 - -- Revert "cirrus-ci: disable the FreeBSD 13 builds" + A common set of functions instead of many separate implementations for + creating buffers that can grow when appending data to them. Existing + functionality has been ported over. - This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. - -- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override + In my early basic testing, the total number of allocations seem at + roughly the same amount as before, possibly a few less. - To let debug-builds return fake values, like in test 970. + See docs/DYNBUF.md for a description of the API. - Ref: #5131 - Closes #5136 + Closes #5300 -- test970: improve the test +- runtests: remove sleep calls - - send more data to make problems more obvious - - don't start the data with minus, it makes diffs harder to read - - skip the headers in the stdout comparison - - save to a file name to also verify 'filename_effective' + Remove many one second sleeps that were done *after* each newly started + test server already has been verified. They should not have any purpose + there. - Ref: #5131 + Closes #5323 -- CURLINFO_NUM_CONNECTS: improve accuracy - - The counter was not bumped in all cases correctly. +- asyn-*: remove support for never-used NULL entry pointers - Reported-by: Marcel Raad - Ref: #5131 - Closes #5135 + ... and instead convert those to asserts to make sure they are truly + never NULL. + + Closes #5324 -- TODO: Use "random" ports for the test servers +- [Emil Engler brought this change] -- lib/curl_setup: adjust the copyright year range + doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax - Follow-up from d820224b8 + Closes #5325 -Jay Satiro (21 Mar 2020) -- curl_setup: define _WIN32_WINNT_[OS] symbols +Jay Satiro (2 May 2020) +- asyn-thread: fix cppcheck warning - .. because not all Windows build systems have those symbols, and even - those that do may be missing newer symbols (eg the Windows 7 SDK does - not define _WIN32_WINNT_WIN10). + - Check for NULL entry parameter before attempting to deref entry in + Curl_resolver_is_resolved, like is already done in asyn-ares. - Those symbols are used in build-time logic to decide which API to use - and prior to this change if the symbols were missing it would have - resulted in deprecated API being used when more recent functions were - available (eg GetVersionEx used instead of VerifyVersionInfo). + This is to silence cppcheck which does not seem to understand that + asyn-ares and asyn-thread have separate Curl_resolver_is_resolved + and those units are mutually exclusive. Prior to this change it warned + of a scenario where asyn-thread's Curl_resolver_is_resolved is called + with a NULL entry from asyn-ares, but that couldn't happen. - Reported-by: FuccDucc@users.noreply.github.com + Reported-by: rl1987@users.noreply.github.com - Probably fixes https://github.com/curl/curl/issues/4995 - Closes https://github.com/curl/curl/pull/5057 - -- [Ross Burton brought this change] + Fixes https://github.com/curl/curl/issues/5326 - curl-functions.m4: remove inappropriate AC_REQUIRE +- select: fix overflow protection in Curl_socket_check - AC_REQUIRE means "if this macro hasn't been executed already, execute - it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) - isn't correct at that will execute AC_RUN_IFELSE without any arguments. + Follow-up to a96c752 which changed the timeout_ms type from time_t to + timediff_t. - With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, - AC_RUN_IFELSE without a default value when cross-compiling is fatal. - The result is that curl with autoconf 2.70 cannot cross-compile. + Ref: https://github.com/curl/curl/pull/5240 - Fixes https://github.com/curl/curl/issues/5126 - Closes https://github.com/curl/curl/pull/5130 + Closes https://github.com/curl/curl/pull/5286 -Marc Hoersken (20 Mar 2020) -- ci/tests: fix Azure Pipelines not running Windows containers +Marc Hoersken (2 May 2020) +- sockfilt: make select_ws stop waiting on exit signal event - Workaround posted here: microsoft/azure-pipelines-agent#2864 + This makes sure that select_ws behaves similar to real select + which stops waiting on a signal handler being triggered. - Assisted-by: Simon Chalifoux - Assisted-by: Tommy Petty + This makes it possible to gracefully stop sockfilt.exe on + Windows with taskkill /IM sockfilt.exe (without /F force flag). - Fixes #5117 - Closes #5129 + Reviewed-by: Jay Satiro + Part of #5260 -Daniel Stenberg (20 Mar 2020) -- tests: add test 430, 431 and 432 to verify the --config fix +- tests/server/util.[ch]: add exit event to stop waiting on Windows - Verify the fixes in 4e0b4fee4 - -- [Rici Lake brought this change] - - cmdline: fix handling of OperationConfig linked list (--next) + This commit adds a global exit event to the test servers that + Windows-specific wait routines can use to get triggered if the + program was signaled to be terminated, eg. select_ws in sockfilt.c - Ensures that -K/--config inserts new items at the end of the list - instead of overwriting the second item, and that after a -K/--config - option has been parsed, the option parser's view of the current config - is update. + The exit event will be managed by the signal handling code and is + set to not reset automatically to support multiple wait routines. - Fixes #5120 - Closes #5123 - -Marc Hoersken (20 Mar 2020) -- test2100: fix static port instead of dynamic value being used - -- test970: fix static ip:port instead of dynamic values being used + Reviewed-by: Jay Satiro + Closes #5260 -Daniel Stenberg (19 Mar 2020) -- secure transport: remove the BACKEND define kludge +- tests/server/util.c: fix thread handle not being closed - Closes #5122 + Reviewed-by: Jay Satiro + Part of #5260 -- mbedtls: remove the BACKEND define kludge +- tests/server/util.c: use raise instead of calling signal handler + + Use raise to trigger signal handler instead of calling it + directly and causing potential unexpected control flow. + + Reviewed-by: Jay Satiro + Part of #5260 -- bearssl: remove the BACKEND define kludge +- tests: add support for SSH server variant specific transfer paths + + OpenSSH for Windows requires paths in the format of /C:/ + instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ + + Reviewed-by: Daniel Stenberg + Closes #5298 -- wolfssl: remove the BACKEND define kludge +Daniel Stenberg (2 May 2020) +- RELEASE-NOTES: synced -- nss: remove the BACKEND define kludge +- libssh2: set the expected total size in SCP upload init + + ... as otherwise the progress callback gets called without that + information, making the progress meter have less info. + + Reported-by: Murugan Balraj + Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html + Closes #5317 -- gnutls: remove the BACKEND define kludge +- runtests: make the logmsg from the ssh server only show in verbose -- openssl: remove the BACKEND define kludge +- tests: make test 1248 + 1249 use %NOLISTENPORT - Use a proper variable instead to make it easier to use a debugger and - read the code. + ... instead of a port of a non-running server so that it works + stand-alone. + + Closes #5318 -Marc Hoersken (19 Mar 2020) -- tests: make Python-based servers compatible with Python 2 and 3 +- examples: remove asiohiper.cpp - Update smbserver.py and negtelnetserver.py to be compatible with - Python 3 while staying backwards-compatible to support Python 2. + This example has repeatedly been reported to contain bugs, and as users + copy and paste code from this into production, I now deem it better to + not provide the example at all. - Fix string encoding and handling of echoed and transferred data. + Closes #5090 + Closes #5322 + +- [Emil Engler brought this change] + + doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 - Tested with both Python 2.7.17 and Python 3.7.7 + Closes #5320 + +- [Emil Engler brought this change] + + KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" - Reported-by: Daniel Stenberg - Assisted-by: Kamil Dudka - Reviewed-by: Marcel Raad + It was fixed in 9a2cbf3 - Fixes #5104 - Closes #5110 + Closes #5319 -Daniel Stenberg (18 Mar 2020) -- writeout_json: use curl_off_t printf() option for the time output +- cirrus: disable SFTP and SCP tests - Follow-up to: 04c03416e68fd635a15 + ... as we can't seem to start the sshd server on it. Those problems + existed before d1239b50bececd (running the SSH server on a random port), + but they're more noticable now since there are more failed attempts in + the logs. - Closes #5115 + Closes #5315 -- RELEASE-NOTES: synced - - Uh, I missed this in 1a46b218db +- [Emil Engler brought this change] -- RELEASE-NOTES: synced + runtests: fix typo in the existence of disabled tests checker - ... and bumped curlver.h to 7.70.0 + Closes #5316 -Jay Satiro (18 Mar 2020) -- http2: Fix erroneous debug message that h2 connection closed - - Prior to this change in libcurl debug builds http2 stream closure was - erroneously referred to as connection closure. - - Before: - * nread <= 0, server closed connection, bailing - - After: - * nread == 0, stream closed, bailing +Dan Fandrich (30 Apr 2020) +- test75: Remove precheck test - Closes https://github.com/curl/curl/pull/5118 + This has not been needed since commit 9fa42bed and often prevents it + from running at all with dynamic test ports. -Daniel Stenberg (18 Mar 2020) -- tool_setopt: correct the copyright year range +- tests: Stop referring to server ports when they're not used - Follow-up to 5450428491 + Several tests referred to specific server ports even when the test + didn't actually use that server or specify that it's needed. In such + cases, the test harness substitutes the text "[not running]" as the port + number which causes many such tests to fail due to the inability to + parse the URL. These tests are changed to use %NOLISTENPORT which will + always be substituted correctly. -Jay Satiro (18 Mar 2020) -- [Johannes Schindelin brought this change] +Daniel Stenberg (30 Apr 2020) +- [Emil Engler brought this change] - schannel: add "best effort" revocation check option - - - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and - --ssl-revoke-best-effort to allow a "best effort" revocation check. + GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT - A best effort revocation check ignores errors that the revocation check - was unable to take place. The reasoning is described in detail below and - discussed further in the PR. + Closes #5287 + +- conncache: various concept cleanups - --- + More connection cache accesses are protected by locks. - When running e.g. with Fiddler, the schannel backend fails with an - unhelpful error message: + CONNCACHE_* is a beter prefix for the connection cache lock macros. - Unknown error (0x80092012) - The revocation function was unable - to check revocation for the certificate. + Curl_attach_connnection: now called as soon as there's a connection + struct available and before the connection is added to the connection + cache. - Sadly, many enterprise users who are stuck behind MITM proxies suffer - the very same problem. + Curl_disconnect: now assumes that the connection is already removed from + the connection cache. - This has been discussed in plenty of issues: - https://github.com/curl/curl/issues/3727, - https://github.com/curl/curl/issues/264, for example. + Ref: #4915 + Closes #5009 + +- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports - In the latter, a Microsoft Edge developer even made the case that the - common behavior is to ignore issues when a certificate has no recorded - distribution point for revocation lists, or when the server is offline. - This is also known as "best effort" strategy and addresses the Fiddler - issue. + As stunnel is an external tool and it has no specific option to export + the actually used port number when asked to listen to 0, runtests + instead iterates over ten randomly picked high number ports and sticks + to the first one stunnel can listen to. - Unfortunately, this strategy was not chosen as the default for schannel - (and is therefore a backend-specific behavior: OpenSSL seems to happily - ignore the offline servers and missing distribution points). + Closes #5267 + +- tests: pick a random port number for SSH - To maintain backward-compatibility, we therefore add a new flag - (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option - (`--ssl-revoke-best-effort`) to select the new behavior. + Since sshd doesn't have such an option by itself, we iterate over a + series of random ports until one works. - Due to the many related issues Git for Windows and GitHub Desktop, the - plan is to make this behavior the default in these software packages. + Closes #5273 + +- [Rikard Falkeborn brought this change] + + libtest/cmake: Remove commented code - The test 2070 was added to verify this behavior, adapted from 310. + These were commented out in e9dd0998706a when Makefile.inc was included + instead. 11 years have passed since then and the commented code is of + course very outdated. Remove it to avoid confusion. - Based-on-work-by: georgeok - Co-authored-by: Markus Olsson - Signed-off-by: Johannes Schindelin + Closes #5311 + +- schannel: source code reindent - Closes https://github.com/curl/curl/pull/4981 + White space edits only. Conform better to standard curl source code + indenting style. + + Closes #5305 -- multi: Improve parameter check for curl_multi_remove_handle +Kamil Dudka (29 Apr 2020) +- test1177: look for curl.h in source directory - - If an easy handle is owned by a multi different from the one specified - then return CURLM_BAD_EASY_HANDLE. + If we use a separate build directory, there is no copy of the header. - Prior to this change I assume user error could cause corruption. + Closes #5310 + +- tests: look for preprocessed tests in build directory - Closes https://github.com/curl/curl/pull/5116 + ... which is not always the same directory as source directory + + Closes #5310 -Viktor Szakats (17 Mar 2020) -- windows: suppress UI in all CryptAcquireContext() calls +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: synced - Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters - Reviewed-by: Marc Hörsken - Closes https://github.com/curl/curl/pull/5088 + ... and bumped curlver.h to 7.70.1 -Daniel Stenberg (17 Mar 2020) -- writeout_json: add missing comma to fix the HTTP version +Version 7.70.0 (29 Apr 2020) + +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: 7.70.0 + +- THANKS: synced with the 7.70.0 release + +- headers: copyright range fix + +- [Rikard Falkeborn brought this change] + + doh: Constify some input pointers - Follow-up to 04c03416e68fd635a15 + Closes #5306 -- test 970: verify --write-out '%{json}' +- nss: check for PK11_CreateDigestContext() returning NULL - Makes curl_easy_getinfo() of "variable" numerical content instead return - the number set in the env variable `CURL_TIME`. + ... to avoid crashes! - Makes curl_version() of "variable" textual content. This guarantees a - stable version string which can be tested against. Environment variable - `CURL_VERSION` defines the content. + Reported-by: Hao Wu + Fixes #5302 + Closes #5303 + +- travis: bump the wolfssl CI build to use 4.4.0 - Assisted-by: Mathias Gumz + Closes #5301 -- [Mathias Gumz brought this change] +- copyright updates: adjust year ranges - writeout: support to generate JSON output - - This commit adds support to generate JSON via the writeout feature: - - -w "%{json}" +Marc Hoersken (26 Apr 2020) +- CI: do not include */ci branches in PR builds - It leverages the existing infrastructure as much as possible. Thus, - generating the JSON on STDERR is possible by: + Align Azure Pipelines with GitHub Actions. + +Daniel Stenberg (25 Apr 2020) +- runtests: check for the disabled tests relative srcdir - -w "%{stderr}%{json}" + To make it work correctly for out-of-tree builds. - This implements a variant of - https://github.com/curl/curl/wiki/JSON#--write-out-json. + Follow-up to 75e8feb6fb08b - Closes #4870 + Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 + Reported-by: Marcel Raad + Closes #5297 -- CI: stop ignoring 323, it is disabled +- runtests: revert commenting out a line I did for debugging + + Follow-up to 11091cd4d. It was not meant to be pushed! -- DISABLED: disable test 323 +- smtp: set auth correctly - The test uses SRP to "a server not supporting it" but modern stunnel - versions will silently accept it and remain happy. The test is therefore - faulty. + Regression since 7.69.0 and 68fb25fa3fcff. - I haven't figured out how to make stunnel explicitly reject SRP-using - connects. + The code wrongly assigned 'from' instead of 'auth' which probably was a + copy and paste mistake from other code, leading to that auth could + remain NULL and later cause an error to be returned. - Reported-by: Marc Hörsken - Fixes #5105 - Closes #5113 + Assisted-by: Eric Sauvageau + Fixes #5294 + Closes #5295 -Marc Hoersken (17 Mar 2020) -- ci/tests: increase timeouts for torture builds on Azure Pipelines - - For some reason the torture builds have slowed down recently. +Marcel Raad (25 Apr 2020) +- lib: clean up whitespace - Reported-by: Daniel Stenberg + This fixes CodeFactor warnings. -Daniel Stenberg (16 Mar 2020) -- cmake: add support for building with wolfSSL +Daniel Stenberg (25 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] + + libssh: avoid options override by configuration files - My working build cmdline: + Previously, options set explicitly through command line options could be + overridden by the configuration files parsed automatically when + ssh_connect() was called. - $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . + By calling ssh_options_parse_config() explicitly, the configuration + files are parsed before setting the options, avoiding the options + override. Once the configuration files are parsed, the automatic + configuration parsing is not executed. - Assisted-by: Brad King - Closes #5095 + Fixes #4972 + Closes #5283 + Signed-off-by: Anderson Toshiyuki Sasaki -- tool_operate: fix add_parallel_transfers when more are in queue - - Trying to return early from the function if no new transfers were added - would break the "morep" argument and cause issues. This could lead to - zero content "transfers" (within quotes since they would never be - started) when parallel-max was reduced. +- runtests: when mentions http, kill http/2 too - Reported-by: Gavin Wong - Analyzed-by: Jay Satiro - Fixes #4937 - Closes #5112 - -- vtls: free ssl_config leftovers on out-of-memory + Since the http2 test server is a mere proxy that needs to know about the + dynamic port the HTTP server is using, it too needs to get restarted + when the http server is killed. - Torture testing 2034 and 2037 found this. + A regression caused by 80d6515. - Reported-by: Marc Hörsken - Fixes #5108 - Closes #5109 + Fixes #5289 + Closes #5291 -Marc Hoersken (16 Mar 2020) -- ci/tests: fix Azure Pipelines not running for pull requests - - Closes #5111 +- [Yuri Slobodyanyuk brought this change] -Daniel Stenberg (15 Mar 2020) -- gskit: update the copyright year range + docs: fix two typos - Follow-up from 083603c63a3 + Closes #5292 -Marc Hoersken (15 Mar 2020) -- gskit: use our internal select wrapper for portability - - Follow up to c52b342 - Closes #5106 +- [Emil Engler brought this change] -- tests: fix verification of stdout in test 1452 due to newline + tests/git: ignore mqttd and port files - Fixes test1452:41:1: error: missing tag before + Closes #5290 -- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI +- tests: make runtests check that disabled tests exists - Also force the package index/cache to be updated before installing. + ... and error out if so. Removed '536' from DISABLED as there is no such + test file. - Closes #5103 + Closes #5288 -- tests/README: add note about manually installing python-impacket - - Follow up to 4be2560 +- test1154: set a proper name -Daniel Stenberg (15 Mar 2020) -- transfer: cap retries of "dead connections" to 5 +- select: make Curl_socket_check take timediff_t timeout - When libcurl retries a connection due to it being "seemingly dead" or by - REFUSED_STREAM, it will now only do it up five times before giving up, - to avoid never-ending loops. + Coverity found CID 1461718: - Reported-by: Dima Tisnek - Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html - Closes #5074 - -- TODO: TLS-PSK with OpenSSL + Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > + 9223372036854775807L" is always false regardless of the values of its + operands. This occurs as the logical second operand of "||". - Closes #5081 + Closes #5240 -Marc Hoersken (15 Mar 2020) -- select: add 'timeout_ms' wrap-around precaution to Curl_select +- [i-ky brought this change] -- select: fix 'pending_ms' is assigned a value that is never used + libcurl-multi.3: added missing full stop - Detected by Codacy + Closes #5285 -- select: move duplicate select preparation code into Curl_select +Jay Satiro (22 Apr 2020) +- transfer: Switch PUT to GET/HEAD on 303 redirect - Reviewed by Daniel Stenberg - Reviewed by Marcel Raad - Closes #5078 + Prior to this change if there was a 303 reply to a PUT request then + the subsequent request to respond to that redirect would also be a PUT. + It was determined that was most likely incorrect based on the language + of the RFCs. Basically 303 means "see other" resource, which implies it + is most likely not the same resource, therefore we should not try to PUT + to that different resource. + + Refer to the discussions in #5237 and #5248 for more information. + + Fixes https://github.com/curl/curl/issues/5237 + Closes https://github.com/curl/curl/pull/5248 + +Daniel Stenberg (22 Apr 2020) +- lib/mk-ca-bundle: skip empty certs + + Reviewed-by: Emil Engler + Reported-by: Ashwin Metpalli + Fixes #5278 + Closes #5280 -Daniel Stenberg (15 Mar 2020) -- connect: happy eyeballs cleanup - - Make sure each separate index in connn->tempaddr[] is used for a fixed - family (and only that family) during the connection process. +- version: skip idn2_check_version() check and add precaution - If family one takes a long time and family two fails immediately, the - previous logic could misbehave and retry the same family two address - repeatedly. + A gcc-10's -fanalyze complaint made me spot and do these improvements. - Reported-by: Paul Vixie - Reported-by: Jay Satiro - Fixes #5083 - Fixes #4954 - Closes #5089 + Closes #5281 -Marc Hoersken (15 Mar 2020) -- ci/tests: fix and align setting TFLAGS for make test-nonflaky +- RELEASE-NOTES: synced -- ci/tests: install test suite dependencies stunnel and impacket +- [Brian Bergeron brought this change] -- tests: remove python_dependencies for smbserver from our tree - - Users of the SMB tests will have to install impacket manually. + curl.h: update comment typo - Reasoning: our in-tree version of impacket was quite outdated - and only compatible with Python 2 which is already end-of-life. - Upgrading to Python 3 and a compatible impacket version would - require to import additional Python-only and CPython-extension - dependencies. This would have hindered portability enormously. + "routines with be invoked" -> "routines will be invoked" - Closes #5094 + Closes #5279 -Jay Satiro (14 Mar 2020) -- Makefile.m32: Improve windres parameter compatibility - - - s/COFF/coff/ - - Some versions of windres do not recognize uppercase COFF as a valid - way to specify the COFF output format. - - Reported-by: Steven Penny - - Fixes https://github.com/curl/curl/issues/5099 - Closes https://github.com/curl/curl/pull/5101 +- [Emil Engler brought this change] -- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares + GnuTLS: Don't skip really long certificate fields - - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in - curl_easy_duphandle. + Closes #5271 + +- gnutls: bump lowest supported version to 3.1.10 - Prior to this change if c-ares was used as the resolver backend and - either it was too old or libcurl was built without IPv6 support then - some of our resolver functions could return CURLE_NOT_BUILT_IN to - curl_easy_duphandle causing it to fail. + GnuTLS 3.1.10 added new functions we want to use. That version was + released on Mar 22, 2013. Removing support for older versions also + greatly simplifies the code. - Caused by c8f086b which shipped in 7.69.1. + Ref: #5271 + Closes #5276 + +- mqtt: make NOSTATE get within the debug name array + +- tests: run the RTSP test server on a dynamic port number - Reported-by: Karl Chen + To avoid port collisions. - Fixes https://github.com/curl/curl/issues/5097 - Closes https://github.com/curl/curl/pull/5100 + Closes #5272 -Daniel Stenberg (13 Mar 2020) -- docs: add warnings about FILE: URLs on Windows +- tests: add %NOLISTENPORT and use it - - --url man page section - - libcurl-security.3 gets the full text - - CURLOPT_URL.3 + The purpose with this variable is to provide a port number that is + reasonably likely to not have a listener on the local host so that tests + can try connect failures against it. It uses port 47 - "reserved" + according to IANA. - Reported-by: Tim Sedlmeyer + Updated six tests to use it instead of the previous different ports. + + Assisted-by: Emil Engler + Closes #5270 -- server/getpart: make the "XML-parser" stricter +- mqtt: remove code with no purpose - When extracting a
and there's no before -
, this now outputs an error and returns a wrong string to - make users spot the mistake. + Detected by Coverity. CID 1462319. - Ref: #5070 - Closes #5071 - -Marc Hoersken (13 Mar 2020) -- impacket: some more Python 3 code compatibility updates + "The same code is executed when the condition result is true or false, + because the code in the if-then branch and after the if statement is + identical." - This makes smbserver load on Python 3, but still not work completely. + Closes #5275 -- smbserver: pin Python version to 2 since we are not yet 3 compatible +- mqtt: fix Curl_read() error handling while reading remaining length - Even though the existing code can be fixed to run on Python 3, the - tests will fail due to the Unicode transition the protocol is invalid. + Detected by Coverity. CID 1462320. - Follow up to ee63837 - Closes #5085 - -Daniel Stenberg (12 Mar 2020) -- [Viktor Szakats brought this change] + Closes #5274 - cleanup: fix some text/comment typos +- server/tftpd: fix compiler warning - Closes #5087 + Follow-up from 369ce38ac1d + Reported-by: Marc Hörsken -Marc Hoersken (12 Mar 2020) -- smbserver: fix Python version specific ConfigParser import +- http: free memory when Alt-Used header creation fails due to OOM - Follow up to ee63837 and 8c7c4a6 - Fixes #5077 + Reported-by: James Fuller + Fixes #5268 + Closes #5269 -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: synced +Daniel Gustafsson (20 Apr 2020) +- lib: fix typos in comments and errormessages - bumped to 7.69.2 + This fixes a few randomly spotted typos in recently merged code, most + notably one in a userfacing errormessage the schannel code. -Dan Fandrich (11 Mar 2020) -- tests/data: Fix some XML formatting issues in test cases +Daniel Stenberg (20 Apr 2020) +- tests: run the SOCKS test server on a dynamic port number - This allows these test files to pass xmllint. + Closes #5266 -Daniel Stenberg (11 Mar 2020) -- [Muhammad Herdiansyah brought this change] +- [Johannes Schindelin brought this change] - Makefile: run the cd commands in a subshell + multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - In bmake, if the directory is changed (with cd or anything else), bmake - won't return to the "root directory" on the next command (in the same - Makefile rule). This commit runs the cd command in a subshell so it - would work in bmake. + When cURL is compiled with support for multiple SSL backends, it is + possible to configure an SSL backend via `curl_global_sslset()`, but + only *before* `curl_global_init()` was called. - Closes #5073 - -- configure: convert -I to -isystem as a last step + If another SSL backend should be used after that, a user might be + tempted to call `curl_global_cleanup()` to start over. However, we did + not foresee that use case and forgot to reset the SSL backend in that + cleanup. - As all the -I uses in CFLAGS at that point are for system headers and - third party libraries this helps us remove/ignore warnings on those! + Let's allow that use case. - Closes #5060 + Fixes #5255 + Closes #5257 + Reported-by: davidedec on github + Signed-off-by: Johannes Schindelin -- configure: fix -pedantic-errors for GCC 5 and later - - If --enable-werror is used. +- tests: run the TFTP test server on a dynamic port number - Follow-up to d5c0351055d5709da which added it too early in the configure - script before $compiler_num was set correctly and thus this option was - never used. + Picking a dynamic unused port is better than a fixed to avoid the + collision risk. - Reported-by: Stepan Efremov - Fixes #5067 - Closes #5068 + Closes #5265 -- configure: document 'compiler_num' for gcc +- mqtt: improve the state machine - The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + - MINOR and ignores the patch version, and since gcc version 7 it only - sets it to MAJOR*100. + To handle PUBLISH before SUBACK and more. - Reported-by: Stepan Efremov - Ref: #5067 - Closes #5069 + Updated the existing tests and added three new ones. + + Reported-by: Christoph Krey + Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html + Closes #5246 -Version 7.69.1 (11 Mar 2020) +- runtests: always put test number in servercmd file -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: 7.69.1 +- RELEASE-NOTES: synced -- THANKS: from the 7.69.1 release +- release-notes.pl: fix parsing typo -- [Marc Hoersken brought this change] +James Fuller (20 Apr 2020) +- ensure all references to ports are replaced by vars - test1129: fix invalid case of closing XML-tag and Content-Length +- add more alt-svc test coverage + +Daniel Stenberg (20 Apr 2020) +- test1247: use http server to get the port number set - Fixes #5070 - Closes #5072 + Follow-up to 0f5db7b263f -Marc Hoersken (10 Mar 2020) -- tests/data: fix static ip instead of dynamic value being used +- runtests: use a unix domain socket path with the pid in the name - Follow up to 94ced8e + To make it impossible for test cases to access the file name without + using the proper variable for the purpose. + + Closes #5264 -- tests/data: fix static ip:port instead of dynamic values being used +Daniel Gustafsson (19 Apr 2020) +- [Mipsters on github brought this change] + + src: Remove C99 constructs to ensure C89 compliance - Closes #5065 + This fixes the error: 'for' loop initial declaration used outside C99 + mode by declaring the loop increment variable in the beginning of the + block instead of inside the for loop. + + Fixes #5254 + Reviewed-by: Daniel Gustafsson -- tests/server: fix missing use of exe_ext helper function +Daniel Stenberg (19 Apr 2020) +- runtests: dummy init the ports variables to avoid warnings - Follow up to 9819984 and 3dce984 - Reviewed-By: Daniel Stenberg - Closes #5064 + ... and generate something that can help debug test cases. -- runtests: log minimal and maximal used port numbers +- [Patrick Monnerat brought this change] + + mime: properly check Content-Type even if it has parameters + + New test 669 checks this fix is effective. + + Fixes #5256 + Closes #5258 + Reported-by: thanhchungbtc on github + +- tests/FILEFORMAT: converted to markdown and extended + + Closes #5261 + +- test1245: make it work with dynamic FTP server port + +- test1055: make it work with dynamic FTP port -Daniel Stenberg (9 Mar 2020) -- [James Fuller brought this change] +- test1028: make it run on dynamic FTP server port - sftp: fix segfault regression introduced by #4747 +- tests: move pingpong server to dynamic listening port - This fix adds a defensive check for the case where the char *name in - struct libssh2_knownhost is NULL + FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic + ports - Fixes #5041 - Closes #5062 + Test 842-845 are unfortunately a bit hard to move over to this concept + right now and require "default port" still... -- RELEASE-NOTES: synced +- test1056: work with dynamic HTTP ipv6 port -- socks4: fix host resolve regression - - 1. The socks4 state machine was broken in the host resolving phase +- test1448: work with dynamic HTTP server port + +- tests: introduce preprocessed test cases - 2. The code now insists on IPv4-only when using SOCKS4 as the protocol - only supports that. + The runtests script now always performs variable replacement on the + entire test source file before the test gets executed, and saves the + updated version in a temporary file (log/test[num]) so that all test + case readers/servers can use that version (if present) and thus enjoy + the powers of test case variable substitution. - Regression from #4907 and 4a4b63d, shipped in 7.69.0 + This is necessary to allow complete port number freedom. - Reported-by: amishmm on github - Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 - Closes #5061 + Test 309 is updated to work with a non-fixed port number thanks to this. -- [Patrick Monnerat brought this change] +- tests: make 2006-2010 handle different port number lengths - silly web server: silent a compilation warning +- tests: run the sws server on "any port" - Recent gcc warns when byte count of strncpy() equals the destination - buffer size. Since the destination buffer is previously cleared and - the source string is always shorter, reducing the byte count by one - silents the warning without affecting the result. + Makes the test servers for HTTP and Gopher pop up on a currently unused + port and runtests adapts to that! - Closes #5059 - -- [Patrick Monnerat brought this change] + Closes #5247 - cookie: get_top_domain() sets zero length for null domains +Marc Hoersken (18 Apr 2020) +- sockfilt: tidy variable naming and data structure in select_ws - This silents a compilation warning with gcc -O3. + This commit does not introduce any logical changes to the code. + + Reviewed-by: Jay Satiro and Marcel Raad + Closes #5238 -- [Patrick Monnerat brought this change] +Daniel Stenberg (17 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] - test 1560: avoid valgrind false positives + libssh: Use new ECDSA key types to check known hosts - When using maximum code optimization level (-O3), valgrind wrongly - detects uses of uninitialized values in strcmp(). + From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA + keys depending on the curve. - Preset buffers with all zeroes to avoid that. + Signed-off-by: Anderson Toshiyuki Sasaki + Fixes #5252 + Closes #5253 -Steve Holme (8 Mar 2020) -- sha256: Added WinCrypt implementation +Marcel Raad (17 Apr 2020) +- appveyor: add Unicode winbuild jobs - Closed #5030 - -- sha256: Added SecureTransport implementation + These are cheap as they don't build tests. + + Closes https://github.com/curl/curl/pull/5063 -Daniel Stenberg (7 Mar 2020) -- lib1564: reduce number of mid-wait wakeup calls +Daniel Stenberg (16 Apr 2020) +- mqttd: s/errno/SOCKERRNO - This test does A LOT of *wakeup() calls and then calls curl_multi_poll() - twice. The first *poll() is then expected to return early and the second - not - as the first is supposed to drain the socketpair pipe. + To behave proper on Windows + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 + Closes #5241 + +- buildconf: use find -execdir instead, remove -print and the ares files - It turns out however that when given "excessive" amounts of writes to - the pipe, some operating systems (the Solaris based are known) will - return EAGAIN before the pipe is drained, which in our test case causes - the second *poll() call to also abort early. + Follow-up to 1e41bec96a6e - This change attempts to avoid the OS-specific behaviors in the test by - reducing the amount of wakeup calls from 1234567 to 10. + Suggested-by: Marc Hörsken + +- [Alexander V. Tikhonov brought this change] + + buildconf: avoid using tempfile when removing files - Reported-by: Andy Fiddaman - Fixes #5037 - Closes #5058 + Closes #5213 -- [Patrick Monnerat brought this change] +- copyright: bump the copyright year range - mime: fix the binary encoder to handle large data properly +- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords + +- [JP Mens brought this change] + + docs/MQTT: replace confusing 80 by 75 - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. + I was a bit surprised by the `80`: first thought: what's HTTP doing + here? ;) - Fixes #4860 - Closes #4833 - Reported-by: RuurdBeerstra on github + Closes #5236 -- [Patrick Monnerat brought this change] +- [Brad King brought this change] - mime: do not perform more than one read in a row + cmake: Avoid MSVC C4273 warnings in send/recv checks - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. + We use `check_c_source_compiles` to check possible send/recv signatures + by reproducing the forward declarations from system headers. On Windows + the `winsock2.h` header adds dll linkage settings to its forward + declaration. If ours does not match the compiler warns: - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 667 checks mimepost using single-byte read callback with encoder. - New test 668 checks the end of part data early detection. + warning C4273: 'recv': inconsistent dll linkage - Fixes #4826 - Reported-by: MrdUkk on github - -- [Patrick Monnerat brought this change] + Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so + that our linkage is consistent with that from `winsock2.h`. + + Fixes #4764 + Closes #5232 - mime: latch last read callback status. +Jay Satiro (14 Apr 2020) +- KNOWN_BUGS: Add entry 'Blocking socket operations' - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. + - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of + known blocking operations. - Fixes #4813 - Reported-by: MrdUkk on github - -Marc Hoersken (7 Mar 2020) -- runtests: fix missing use of exe_ext helper function - -Daniel Stenberg (7 Mar 2020) -- [Ernst Sjöstrand brought this change] + - New known bugs entry 'Blocking socket operations in non-blocking API' + that directs to the TODO's list of known blocking operations. + + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 + + Reported-by: Marc Hoersken + + Closes https://github.com/curl/curl/pull/5216 - ares: store dns parameters for duphandle +Marc Hoersken (14 Apr 2020) +- test2043: use revoked.badssl.com instead of revoked.grc.com - With c-ares the dns parameters lives in ares_channel. Store them in the - curl handle and set them again in easy_duphandle. + The certificate of revoked.grc.com has expired on 2020-04-13. - Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. + Reviewed-by: Jay Satiro - Fixes #4893 - Closes #5020 - Signed-off-by: Ernst Sjöstrand + Closes #5233 -- version: make curl_version* thread-safe without using global context +- sockfilt: fix broken pipe on Windows to be ready in select_ws - Closes #5010 + Closes #5228 +Daniel Stenberg (14 Apr 2020) - RELEASE-NOTES: synced -Marc Hoersken (7 Mar 2020) -- tests: use native Sleep function as fallback on Windows - - Reviewed-By: Daniel Stenberg - Closes #5054 +- scripts/release-notes: fix duplicate output header -- perl: align order and completeness of Windows OS checks +- github/workflow: enable MQTT in the macOS debug build -Daniel Stenberg (7 Mar 2020) -- tool_cb_see: set correct copyright year range - - Follow-up to a39e5bfb9 +- azure: add mqtt support to one of the Windows builds -Marc Hoersken (7 Mar 2020) -- seek: fix fallback for missing ftruncate on Windows - - This fixes test 198 on versions of MinGW-w64 without ftruncate - - Reviewed-By: Daniel Stenberg - Reviewed-By: Marcel Raad - Closes #5055 +- travis: add mqtt job on Linux -- config-win32: Windows does not have ftruncate +- tests: add four MQTT tests 1190 - 1193 -Daniel Stenberg (7 Mar 2020) -- pause: force a connection (re-)check after unpausing +- tests: add the mqtt test server mqttd + +- tests: support hex encoded data and mqtt server - There might be data available that was already read off the socket, for - example in the TLS layer. + The mqtt server is started using a "random" port. + +- [Björn Stenberg brought this change] + + mqtt: add new experimental protocol - Reported-by: Anders Berg - Fixes #4966 - Closes #5049 + Closes #5173 -- socks5: switch state properly when the resolve is done +- TODO: Consider convenience options for JSON and XML? - Regression from 4a4b63d (and #4907) - Reported-by: vitaha85 on github - Fixes #5053 - Closes #5056 + Closes #5203 -Jay Satiro (7 Mar 2020) -- libssh: Fix matching user-specified MD5 hex key +- tool: do not declare functions with Curl_ prefix - Prior to this change a match would never be successful because it - was mistakenly coded to compare binary data from libssh to a - user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). + To avoid collision risks with private libcurl symbols when linked with + static versions (or just versions not hiding internal symbols). - Reported-by: fds242@users.noreply.github.com + Reported-by: hydra3333 on github + Fixes #5219 + Closes #5234 + +- [Nathaniel R. Lewis brought this change] + + cmake: add aliases so exported target names are available in tree - Fixes https://github.com/curl/curl/issues/4971 - Closes https://github.com/curl/curl/pull/4974 + Reviewed-by: Brad King + Closes #5206 -Daniel Stenberg (6 Mar 2020) -- pause: bail out on bad input +- version: increase buffer space for ssl version output - A NULL easy handle or an easy handle without an associated connection - cannot be paused or unpaused. + To avoid it getting truncated, especially when several SSL backends are + built-in. - Closes #5050 + Reported-by: Gisle Vanem + Fixes #5222 + Closes #5226 -Steve Holme (6 Mar 2020) -- unit1612: fixed the inclusion and compilation of the HMAC unit test - - Follow up to 3f74e5e6 to fix: +Marc Hoersken (13 Apr 2020) +- cirrus: no longer ignore test 504 which is working again - - A typo in Makefile.inc where unit1611 was used instead - - Some compilation issues in unit1612.c + The test is working again, because TCP blackholing is disabled. + +- appveyor: completely disable tests that fail to timeout early - Closes #5024 + The tests changed from ignored to disabled are tests that are + about connecting to non-listening socket. On AppVeyor these + tests are not reliable, because for some unknown reason the + connect is not timing out before the test time limit is reached. -Daniel Stenberg (6 Mar 2020) -- pause: return early for calls that don't change pause state +Daniel Stenberg (13 Apr 2020) +- test1908: avoid using fixed port number in test data - Reviewed-by: Patrick Monnerat - Ref: #4833 - Closes #5026 + Closes #5225 -Jay Satiro (6 Mar 2020) -- curl_share_setopt.3: Note sharing cookies doesn't enable the engine +Jay Satiro (12 Apr 2020) +- [Andrew Kurushin brought this change] + + schannel: Fix blocking timeout logic - Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused - CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. + - Fix schannel_send for the case when no timeout was set. - Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html - Reported-by: Felipe Gasper + Prior to this change schannel would error if the socket was not ready + to send data and no timeout was set. - Closes https://github.com/curl/curl/pull/5048 + This commit is similar to parent commit 89dc6e0 which recently made the + same change for SOCKS, for the same reason. Basically it was not well + understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms + but actually means no timeout. + + Fixes https://github.com/curl/curl/issues/5177 + Closes https://github.com/curl/curl/pull/5221 -- multi: skip EINTR check on wakeup socket if it was closed +- socks: Fix blocking timeout logic - - Don't check errno on wakeup socket if sread returned 0 since sread - doesn't set errno in that case. + - Document in Curl_timeleft's comment block that returning 0 signals no + timeout (ie there's infinite time left). - This is a follow-up to cf7760a from several days ago which fixed - Curl_multi_wait to stop busy looping sread on the non-blocking wakeup - socket if it was closed (ie sread returns 0). Due to a logic error it - was still possible to busy loop in that case if errno == EINTR. + - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. - Closes https://github.com/curl/curl/pull/5047 + Prior to this change if the timeout had a value of 0 and that was passed + to SOCKET_READABLE it would return right away instead of blocking. That + was likely because it was not well understood that when Curl_timeleft + returns 0 it is not a timeout of 0 ms but actually means no timeout. + + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + + Closes https://github.com/curl/curl/pull/5220 -Daniel Stenberg (6 Mar 2020) -- transfer: set correct copyright year range +- [Marc Hoersken brought this change] -- urldata: remove the 'stream_was_rewound' connectdata struct member + gopher: check remaining time left during write busy loop - ... as it is never set anywhere. + Prior to this change gopher's blocking code would block forever, + ignoring any set timeout value. - Follow-up to 2f44e94ef - Closes #5046 + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Similar to #5220 and #5221 + Closes #5214 -- Revert "pause: force-drain the transfer on unpause" +Daniel Stenberg (13 Apr 2020) +- [Dirkjan Bussink brought this change] + + gnutls: ensure TLS 1.3 when SRP isn't requested - This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) + When SRP is requested in the priority string, GnuTLS will disable + support for TLS 1.3. Before this change, curl would always add +SRP to + the priority list, effectively always disabling TLS 1.3 support. - Clearly that didn't solve the problem correctly. + With this change, +SRP is only added to the priority list when SRP + authentication is also requested. This also allows updating the error + handling here to not have to retry without SRP. This is because SRP is + only added when requested and in that case a retry is not needed. - Reported-by: Christopher Reid - Reopens #4966 - Fixes #5044 + Closes #5223 -- RELEASE-NOTES: synced +Marc Hoersken (12 Apr 2020) +- tests/server: add hidden window to gracefully handle WM_CLOSE - and bumped curlver.h + Forward Window events as signals to existing signal event handler. -- MANUAL: update a dict-using command line +- tests/server: add CTRL event handler for Win32 consoles - The 'web1913' database is now invalid, use 'gcide' instead. + Forward CTRL events as signals to existing signal event handler. -- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS +- tests/server: move all signal handling routines to util.[ch] - Closes #3841 + Avoid code duplication to prepare for portability enhancements. -- polarssl: remove more references and mentions +Daniel Stenberg (12 Apr 2020) +- compressed.d: stress that the headers are not modified + Suggested-by: Michael Osipov Assisted-by: Jay Satiro - Follow-up to 6357a19ff29dac04 - Closes #5036 - -Marc Hoersken (4 Mar 2020) -- tests: wrap ignored test failures in braces - -- tests: align some Windows sleep defines with each other + Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 + Closes #5217 -- tests: try to make sleeping portable by avoiding select +Marc Hoersken (11 Apr 2020) +- tests/server/util.c: use curl_off_t instead of long for pid - select does not support just waiting on Windows: - https://perldoc.perl.org/perlport.html#select + Avoid potential overflow of huge PIDs on Windows. - Reviewed-By: Daniel Stenberg - Closes #5035 + Related to #5188 + Assisted-by: Marcel Raad -Daniel Stenberg (4 Mar 2020) -- runtests.1: rephrase how to specify what tests to run +- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows - Also mention the new tilde-prefixed way to ignore test results. + Since the Windows versions of both programs would write Windows + PIDs to their pidfiles which we cannot handle, we need to use + our known perl.exe Cygwin/msys PID together with exec() in order + to tie the spawned processes to the existance of our perl.exe - Reviewed-By: Marc Hoersken - Closes #5033 + The perl.exe that is executing secureserver.pl and sshserver.pl + has a Cygwin/msys PID, because it is started inside Cygwin/msys. + + Related to #5188 -- cirrus-ci: disable the FreeBSD 13 builds +- tests: add Windows compatible pidwait like pidkill and pidterm - FreeBSD 13.0 is apparently close to a year away from a stable release - and has proven to cause intermittent builds failures recently. + Related to #5188 + +- tests: fix conflict between Cygwin/msys and Windows PIDs - Assisted-by: Dan Fandrich - Assisted-by: Fedor Korotkov - Fixes #5028 - Closes #5029 + Add 65536 to Windows PIDs to allow Windows specific treatment + by having disjunct ranges for Cygwin/msys and Windows PIDs. + + See also: + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=b5e1003722cb14235c4f166be72c09acdffc62ea + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe + + Replaces #5178 + Closes #5188 -Version 7.69.0 (4 Mar 2020) +Daniel Stenberg (11 Apr 2020) +- RELEASE-NOTES: synced -Daniel Stenberg (4 Mar 2020) -- RELEASE-NOTES: 7.69.0 +- release-notes.pl: detect the start of the references in cleanup mode -- THANKS: from 7.69.0 +- Revert "file: on Windows, refuse paths that start with \\" - Now sorted case insensitive - -Marc Hoersken (3 Mar 2020) -- ci/tests: fix escaping of testnames and disable proxy for CI APIs + This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. - Follow up to ada581f and c0d8b96 - Closes #5031 - -Jay Satiro (3 Mar 2020) -- cmake: Show HTTPS-proxy in the features output + Reminded-by: Chris Roberts + Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html - - Show HTTPS-proxy in the features output for those backends that - support it: OpenSSL, GnuTLS and NSS. + Closes #5215 + +Jay Satiro (11 Apr 2020) +- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE - Prior to this change HTTPS-proxy was missing from the cmake features - output even if curl was built with it. Only cmake output was affected. - Both the library and tool correctly reported the feature. + - If loss of data may occur converting a timediff_t to time_t and + the time value is > TIME_T_MAX then treat it as TIME_T_MAX. - Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html - Reported-by: David Lopes + This is a follow-up to 8843678 which removed the (time_t) typecast + from the macros so that conversion warnings could be identified. - Closes https://github.com/curl/curl/pull/5025 + Closes https://github.com/curl/curl/pull/5199 -Marc Hoersken (3 Mar 2020) -- ci/tests: Make it possible to still run but ignore failing tests +- test1148: tolerate progress updates better (again) - This enables the development of a solution for the failing tests by - running them on CI while ignoring their result for the overall status. + - Ignore intermediate progress updates. - Closes #4994 - -- README.md: add Azure DevOps Pipelines build status badge - -- ci/tests: Move CI test result creation above environment setup + - Support locales that use a character other than period as decimal + separator (eg 100,0%). - This avoids using our test servers as proxy to the AppVeyor API. + test1148 checks that the progress finishes at 100% and has the right + bar width. Prior to this change the test assumed that the only progress + reported for such a quick transfer was 100%, however in rare instances + (like in the CI where transfer time can slow considerably) there may be + intermediate updates. For example, below is stderrlog1148 from a failed + CI run with explicit \r and \n added (it is one line; broken up so that + it's easier to understand). + + \r + \r################################## 48.3% + \r######################################################################## 100.0% + \n - Closes #5022 + Closes https://github.com/curl/curl/pull/5194 -- ci/tests: Send test results to AppVeyor for status overview - - Closes #5021 +Marc Hoersken (10 Apr 2020) +- sshserver.pl: use cached Win32 environment check variable -Daniel Stenberg (3 Mar 2020) -- Revert "sha256: Added SecureTransport implementation" - - This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) - - That commit broke test 1610 on macos builds without TLS. +- appveyor: partially revert 3413a110 to keep build without proxy - Closes #5027 + Ref: #5211 and #4526 + Reported-by: Marcel Raad -- dist: include tests/azure.pm in the tarball +- appveyor: ignore failing 'connect to non-listening proxy' tests - Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 - Reported-by: Marcel Raad + Closes #5211 -Steve Holme (3 Mar 2020) -- configure.ac: Disable metalink if mbedTLS is specified +- CI/macos: convert CRLF to LF and align indentation + +Daniel Stenberg (9 Apr 2020) +- url: allow non-HTTPS altsvc-matching for debug builds - Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as - being one of the backends that metalink needs to be disabled for, I - seem to have included it in the list of allowed SSL/TLS backends in - comnfigure.ac :( + This is already partly supported but this part was missing. + Reported-by: James Fuller - Closes #5013 + Closes #5205 -- sha256: Tidy up following recent changes +- server/resolve: remove AI_CANONNAME to make macos tell the truth - Reviewed-by: Daniel Stenberg - Closes #4956 + With this bit set, my mac successfully resolves "ip6-localhost" when in + fact there is no such host known to my machine! That in turn made test + 241 wrongly execute and fail. + + Closes #5202 -- sha256: Added WinCrypt implementation +- runtests: fix warning about using an undefined variable + + Follow-up from 4d939ef6ceb2db1 -- sha256: Added SecureTransport implementation +- release-notes: fix the initial reference list output -- sha256: Added mbedtls implementation +- github actions: run when pushed to master or */ci + PRs + + Avoid double-builds when using "local" branches for PRs. For both macos + and fuzz jobs. + + Closes #5201 -- sha256: Added GNU TLS gcrypt implementation +- runtests: provide nicer errormsg when protocol "dump" file is empty -- sha256: Added GNU TLS Nettle implementation +- [Gilles Vollant brought this change] -Jay Satiro (2 Mar 2020) -- curl_escape.3: Add a link to curl_free + schannel: support .P12 or .PFX client certificates - Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 + Used with curl command line option like this: --cert + : --cert-type p12 + + Closes #5193 -- curl_getenv.3: Fix the memory handling description +- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - - Tell the user to call curl_free() to free the pointer returned by - curl_getenv(). + test1294: "split request" being when the entire request isn't sent in + the first go, and the remainder is sent in the PERFORM state. A GET + request is otherwise not sending anything during PERFORM. - Prior to this change the user was directed to call free(), but that - would not work in cases where the library and application use separate C - runtimes and therefore have separate heap memory management. + test1295: same kind of split but with POST - Closes https://github.com/curl/curl/pull/5016 - -Daniel Stenberg (2 Mar 2020) -- [Nick Zitzmann brought this change] + Closes #5197 - md4: use init/update/final functions in Secure Transport - - We can use CC_MD4_Init/Update/Final without having to allocate memory - directly. +- http: don't consider upload done if the request isn't completely sent off - Closes #4979 + Fixes #4919 + Closes #5197 -Marc Hoersken (2 Mar 2020) -- ci/tests: some MacOS builds randomly take longer than 20min +- http: allow Curl_add_buffer_send() to do a short first send by force + + In a debug build, settting the environment variable "CURL_SMALLREQSEND" + will make the first HTTP request send not send more bytes than the set + amount, thus ending up verifying that the logic for handling a split + HTTP request send works correctly. -Daniel Stenberg (2 Mar 2020) -- multi_wait: stop loop when sread() returns zero +- connect: store connection info for QUIC connections - It's unclear why it would ever return zero here, but this change fixes - Robert's problem and it shouldn't loop forever... + Restores the --head functionality to the curl utility which extracts + 'protocol' that is stored that way. - Reported-by: Robert Dunaj - Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html - Closes #5019 + Reported-by: James Fuller + Fixes #5196 + Closes #5198 -- http: mark POSTs with no body as "upload done" from the start - - As we have logic that checks if we get a >= 400 reponse code back before - the upload is done, which then got confused since it wasn't "done" but - yet there was no data to send! +- tests/README: update the port numbers list - Reported-by: IvanoG on github - Fixes #4996 - Closes #5002 + Since the pipelining server is long gone. + Reported-by: James Fuller -- tests: disable 962, 963 and 964 on Windows +- select: remove typecast from SOCKET_WRITABLE/READABLE macros - These tests are also doing UTF-8 SMTP. + So that they don't hide conversions-by-mistake - Follow-up to df207d2dd93b9e73 - -Marc Hoersken (2 Mar 2020) -- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer + Reviewed-by: Jay Satiro + Closes #5190 -Daniel Stenberg (2 Mar 2020) -- configure: bump the AC_COPYRIGHT year range +- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also + + Closes #5192 -- [Steve Holme brought this change] +- release-notes: output trailing references sorted numerically - tests: disable SMTP UTF-8 tests on Windows - - Fixes #4988 - Closes #4992 +- cleanup: correct copyright year range on a few files -- formdata/mime: copyright year range update +- configure: remove use of -vec-report0 from CFLAGS with icc - Due to the merge/revert cycle + ... as it apparently isn't (always) supported. + Reported-by: Alain Miniussi + Fixes #5096 + Closes #5191 -- Revert "mime: latch last read callback status." - - This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. +- warnless: remove code block for icc that didn't work - Fixes #5014 - Closes #5015 - Reopens #4833 + Reported-by: Alain Miniussi + Fixes #5096 -- Revert "mime: do not perform more than one read in a row" +Marc Hoersken (6 Apr 2020) +- dist: add missing setup-win32.h - This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. + Follow up to d820224b8b -- Revert "mime: fix the binary encoder to handle large data properly" - - This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. +Daniel Stenberg (6 Apr 2020) +- RELEASE-NOTES: synced -- altsvc: both h3 backends now speak h3-27 +- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance - ... also updated the HTTP3 build description for ngtcp2 accordingly. + This script helps putting entries in the RELEASE-NOTES using a coherent + style and sorting with a minimal human editing effort - as long as the + first line in the commit message is good enough! There's a short howto + at the top of the file. -- [Patrick Monnerat brought this change] +- [Dennis Felsing brought this change] - mime: fix the binary encoder to handle large data properly + configure: don't check for Security.framework when cross-compiling - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. + Since it checks for the local file, not the cross-compiled one. - Fixes #4860 - Reported-by: RuurdBeerstra on github + Closes #5189 -- [Patrick Monnerat brought this change] +- TODO: Option to make -Z merge lined based outputs on stdout + + Closes #5175 - mime: do not perform more than one read in a row +- lib: never define CURL_CA_BUNDLE with a getenv - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. + - it breaks the build (since 6de756c9b1de34b7a1) + - it's not documented and not consistent across platforms + - the curl tool does that getenv magic - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 664 checks mimepost using single-byte read callback with encoder. - New test 665 checks the end of part data early detection. + Bug: https://github.com/curl/curl/commit/6de756c#r38127030 + Reported-by: Gisle Vanem - Fixes #4826 - Reported-by: MrdUkk on github - -- [Patrick Monnerat brought this change] + Closes #5187 - mime: latch last read callback status. - - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. - - Fixes #4813 - Reported-by: MrdUkk on github - Closes #4833 +Marc Hoersken (5 Apr 2020) +- lib670: use the same Win32 API check as all other lib tests -Steve Holme (1 Mar 2020) -- unit1651: Fixed conversion compilation warning +- appveyor: use random test server ports based upon APPVEYOR_API_URL - 371:17: warning: conversion to 'unsigned char' from 'int' may alter its - value [-Wconversion] + Avoid conflicts of test server ports with AppVeyor API on localhost. - Closes #5008 + Closes #5034 -- configure.ac: Disable metalink support if an incompatible SSL/TLS specified - - tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, - The Win32 Crypto library and Apple's Common Crypto library. - - If an TLS backend such as mbedTLS or WolfSSL is specified then the - following error is given during compilation along, with a load of - unresolved extern errors: - - Can't compile METALINK support without a crypto library. +- appveyor: sort builds by type and add two new variants - Reviewed-by: Daniel Stenberg - Closes #5006 + Related to #5034 and #5063 -Marc Hoersken (1 Mar 2020) -- ci/tests: Update Azure DevOps pipeline job display names +- appveyor: show failed tests in log even if test is ignored - Make the configure step more descriptive and align others. + And print API response with newline only if there is one -- ci/tests: Fix typo in previous commit 597cf2 +- appveyor: turn disabled tests into ignored result tests -- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available +Daniel Stenberg (5 Apr 2020) +- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" - For security reasons the access token is not available to PR builds. - Therefore we should not try to use the DevOps API with an empty token. + Fixed with #5170 (commit 23a870f2fd041278) -Daniel Stenberg (1 Mar 2020) -- build: remove all HAVE_OPENSSL_ENGINE_H defines +- test1566: verify --etag-compare that gets a 304 back - ... as there's nothing in the code that actually uses the define! The - last reference was removed in 38203f158. + Verifies the fix in #5183 - Closes #5007 + Closes #5186 -Jay Satiro (29 Feb 2020) -- [Rolf Eike Beer brought this change] +- [Kwon-Young Choi brought this change] - CMake: clean up and improve build procedures - - - remove check for unsupported old CMake versions - - - do not link to c-ares library twice - - - modernize custom Find modules - - - FindLibSSH2: - - pass version to FPHSA to show it in the output - - use LIBSSH2_VERSION define to extract the version number in - one shot. This variable exists in the header for 10 years. - - remove unneeded code + CURLINFO_CONDITION_UNMET: return true for 304 http status code - - FindNGHTTP2.cmake: - - drop needless FPHSA argument - - mark found variables as advanced + In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the + output file if the server did not transfered a file based on time + condition. In the same manner, getting a 304 HTTP response back from the + server, for example after passing a custom If-Match-* header, also + fulfill this condition. - - FindNSS.cmake: - - show version number + Fixes #5181 + Closes #5183 + +- [Kwon-Young Choi brought this change] + + curl: allow both --etag-compare and --etag-save with same file name - - FindCARES.cmake: - - drop default paths - - use FPHSA instead of checking things by hand + This change inverse the order of processing for the --etag-compare and + --etag-save option to process first --etag-compare. This in turn allows + to use the same file name to compare and save an etag. - - remove needless explict variable dereference + The original behavior of not failing if the etag file does not exists is + conserved. - - simplify count_true() + Fixes #5179 + Closes #5180 + +Viktor Szakats (4 Apr 2020) +- windows: enable UnixSockets with all build toolchains - - allow all policies up to version 3.16 to be set to NEW + Extend existing unix socket support in Windows builds to be + enabled for all toolchain vendors or versions. (Previously + it was only supported with certain MSVC versions + more recent + Windows 10 SDKs) - - do not rerun check for -Wstrict-aliasing=3 every time + Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ + Ref: https://github.com/curl/curl/issues/5162 + Closes: https://github.com/curl/curl/pull/5170 + +Daniel Stenberg (4 Apr 2020) +- KNOWN_BUGS: Store TLS context per transfer instead of per connection - In contrast to every other compiler flag this has a = in it, which CMake - can't have in a variable name. + Closes #5102 + +Marc Hoersken (3 Apr 2020) +- sockfilt: remove redundancy in timeout handling - - only read the interesting strings from curlver.h + And update other logmsg output in select_ws on Windows. + +- sockfilt: fix handling of ready closed sockets on Windows - Reviewed-by: Peter Wu + Replace the incomplete workaround regarding FD_CLOSE + only signalling once by instead doing a pre-check with + standard select and storing the result for later use. - Closes https://github.com/curl/curl/pull/4975 + select keeps triggering on closed sockets on Windows while + WSAEventSelect fires only once with data still available. + By doing the pre-check we do not run in a deadlock + due to waiting forever for another FD_CLOSE event. -- runtests: fix output to command log +- sockfilt: fix race-condition of waiting threads and event handling - - Record only the command of the most recently ran test in the command - log. + Fix race-condition of waiting threads finishing while events are + already being processed which lead to invalid or skipped events. - This is a follow-up to 02988b7 from several weeks ago which fixed - writing to the command log, however it saved all commands for all tests - instead of just the most recently ran test as we would now expect. + Use mutex to check for one event at a time or do post-processing. + In addition to mutex-based locking use specific event as signal. - Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 - Closes https://github.com/curl/curl/pull/5001 + Closes #5156 -Steve Holme (1 Mar 2020) -- polarssl: Additional removal +Daniel Stenberg (2 Apr 2020) +- [Leo Neat brought this change] + + CI-fuzz: increase fuzz time to 40 minutes - Follow up to 6357a19f. + Closes #5174 + +Marc Hoersken (2 Apr 2020) +- CI: increase Azure Pipelines timeouts due to performance issues - Reviewed-by: Daniel Stenberg - Closes #5004 + The current demand on Azure negatively impacts the CI performance. -- [Jonathan Cardoso Machado brought this change] +- runtests.pl: log host OS as detected by Perl environment - docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always - - Reviewed-by: Steve Holme - Closes #5005 +- ftpserver.pl: log before and after data connection is closed -- md5: Added implementation for mbedTLS - - Reviewed-by: Jay Satiro - Closes #4980 +Daniel Stenberg (1 Apr 2020) +- RELEASE-NOTES: synced -- md5: Use pointer notation for array parameters in GnuTLS implementation +- RELEASE-PROCEDURE.md: run the copyright.pl script! -- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 +- vquic/ngtcp2.h: update copyright year range - Closes #4983 + Follow-up to 0736ee73d346a52 -Marc Hoersken (29 Feb 2020) -- ci/tests: Send test results to Azure DevOps for reporting +- [Daiki Ueno brought this change] -Daniel Stenberg (29 Feb 2020) -- pause: force-drain the transfer on unpause - - ... since the socket might not actually be readable anymore when for - example the data is already buffered in the TLS layer. - - Fixes #4966 - Reported-by: Anders Berg - Closes #5000 + CI: add build with ngtcp2 + gnutls on Travis CI -- TODO: curl --proxycommand - - Suggested-by: Kristian Mide - Closes #4941 +- [Daiki Ueno brought this change] -- smtp: overwriting 'from' leaks memory - - Detected by Coverity. CID 1418139. + vquic: add support for GnuTLS backend of ngtcp2 - Also, make sure to return error if the new 'from' allocation fails. + Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile + time. Therefore OpenSSL support needs to be explicitly disabled. - Closes #4997 + Signed-off-by: Daiki Ueno + Closes #5148 -- CIfuzz: switch off 'dry_run' mode - - Follow-up from #4960: now make it fail if it detects problems. - - Closes #4998 +- [Gisle Vanem brought this change] -Marc Hoersken (28 Feb 2020) -- ci/tests: Increase timeouts of Windows builds due to new tests + examples/sessioninfo.c: add include to fix compiler warning - Recently added tests increased their runtime above the limit of 60min. - -- ci/tests: align Azure Pipeline job names with each other + Fixes #5171 -- ci/tests: Add Windows builds via Azure Pipelines using Docker +- misc: copyright year updates + + Follow-up to 7a71965e9 -- tests: fix Python 3 compatibility of smbserver.py +- [Harry Sintonen brought this change] -Daniel Stenberg (27 Feb 2020) -- runtests: restore the command log - - The log file with all command lines for the invoked command lines is now - called logs/commands.log + build: fixed build for systems with select() in unistd.h - Fixes #4911 - Closes #4989 + Closes #5169 -- smtp: fix memory leak on exit path - - Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' - going out of scope leaks the storage it points to" +- memdebug: don't log free(NULL) - Closes #4990 + ... it serves no purpose and fills up the log. -Steve Holme (27 Feb 2020) -- gtls: Fixed compilation when using GnuTLS < 3.5.0 +- cleanup: insert newline after if() conditions - Reverts the functionality from 41fcb4f when compiling with GnuTLS older - than 3.5.0. + Our code style mandates we put the conditional block on a separate + line. These mistakes are now detected by the updated checksrc. + +- checksrc: warn on obvious conditional blocks on the same line as if() - Reviewed-by: Daniel Stenberg - Closes #4984 + Closes #5164 -- RELEASE-NOTES: Corrected the link to issue #4892 +- [Roger Orr brought this change] -Daniel Stenberg (27 Feb 2020) -- Curl_is_ASCII_name: handle a NULL argument - - Make the function tolerate a NULL pointer input to avoid dereferencing - that pointer. + cmake: add CMAKE_MSVC_RUNTIME_LIBRARY - Follow-up to efce3ea5a85126d - Detected by OSS-Fuzz - Reviewed-By: Steve Holme - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 - Fixes #4985 - Closes #4986 + Fixes #5165 + Closes #5167 -- RELEASE-NOTES: synced +- [Daiki Ueno brought this change] -- http2: make pausing/unpausing set/clear local stream window + ngtcp2: update to git master for the key installation API change - This reduces the HTTP/2 window size to 32 MB since libcurl might have to - buffer up to this amount of data in memory and yet we don't want it set - lower to potentially impact tranfer performance on high speed networks. + This updates the ngtcp2 OpenSSL backend to follow the API change in + commit 32e703164 of ngtcp2. - Requires nghttp2 commit b3f85e2daa629 - (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end - up in the next release after 1.40.0. + Notable changes are: + - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace + ngtcp2_crypto_derive_and_install_key + - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key + has been removed - Fixes #4939 - Closes #4940 + Fixes #5166 + Closes #5168 -- [Anderson Toshiyuki Sasaki brought this change] +- [Cyrus brought this change] - libssh: improve known hosts handling - - Previously, it was not possible to get a known hosts file entry due to - the lack of an API. ssh_session_get_known_hosts_entry(), introduced in - libssh-0.9.0, allows libcurl to obtain such information and behave the - same as when compiled with libssh2. + SECURITY.md: minor rephrase - This also tries to avoid the usage of deprecated functions when the - replacements are available. The behaviour will not change if versions - older than libssh-0.8.0 are used. + Closes #5158 + +- output.d: quote the URL when globbing - Signed-off-by: Anderson Toshiyuki Sasaki + Some shells do globbing of their own unless the URL is quoted, so maybe + encourage this. - Fixes #4953 - Closes #4962 + Co-authored-by: Jay Satiro + Closes #5160 -Steve Holme (27 Feb 2020) -- tests: Automatically deduce the tool name from the test case for unit tests - - It is still possible to override the executable to run during the test, - using the tag, but this patch removes the requirement that the - tag must be present for unit tests. +- dist: add tests/version-scan.pl to tarball - It also removes the possibility of human error when existing test cases - are used as the basis for new tests, as recently witnessed in 81c37124. + ... used in test 1177. - Reviewed-by: Daniel Stenberg - Closes #4976 + Follow-up to a97d826f6de3 -- test1323: Added the missing 'unit test' feature requirement in the test case +- test1177: verify that all the CURL_VERSION_ bits are documented -Daniel Stenberg (26 Feb 2020) -- cookie: remove unnecessary check for 'out != 0' +- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented - ... as it will always be non-NULL at this point. + Considered experimental and therefore we can do this. - Detected by Coverity: CID 1459009 + Closes #5157 -- http: added 417 response treatment - - When doing a request with a body + Expect: 100-continue and the server - responds with a 417, the same request will be retried immediately - without the Expect: header. - - Added test 357 to verify. - - Also added a control instruction to tell the sws test server to not read - the request body if Expect: is present, which the new test 357 uses. +- KNOWN_BUGS: DoH doesn't inherit all transfer options - Reported-by: bramus on github - Fixes #4949 - Closes #4964 + Closes #4578 + Closes #4579 -Steve Holme (26 Feb 2020) -- smtp: Tidy up, following recent changes, to maintain the coding style +- KNOWN_BUGS: DoH leaks memory after followlocation - Closes #4892 + Closes #4592 -- smtp: Support the SMTPUTF8 extension for the EXPN command +- KNOWN_BUGS: "FTPS needs session reuse" - Simply notify the server we support the SMTPUTF8 extension if it does. + Closes #4654 -- smtp: Support the SMTPUTF8 extension in the VRFY command +- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed -- smtp: Support the SMTPUTF8 extension in the RCPT TO command +- TODO: Set custom client ip when using haproxy protocol - Note: The RCPT TO command isn't required to advertise to the server that - it contains UTF-8 characters, instead the server is told that a mail may - contain UTF-8 in any envelope command via the MAIL command. + Closes #5125 -- smtp: Support the SMTPUTF8 extension in the MAIL command +Michael Kaufmann (27 Mar 2020) +- writeout_json: Fix data type issues + + Load long values correctly (e.g. for http_code). + + Use curl_off_t (not long) for: + - size_download (CURLINFO_SIZE_DOWNLOAD_T) + - size_upload (CURLINFO_SIZE_UPLOAD_T) - Support the SMTPUTF8 extension when sending mailbox information in the - MAIL command (FROM and AUTH parameters). Non-ASCII domain names will - be ACE encoded, if IDN is supported, whilst non-ASCII characters in - the local address part are passed to the server. + The unit for these values is bytes/second, not microseconds: + - speed_download (CURLINFO_SPEED_DOWNLOAD_T) + - speed_upload (CURLINFO_SPEED_UPLOAD_T) - Reported-by: ygthien on github - Fixes #4828 + Fixes #5131 + Closes #5152 -- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 +Daniel Stenberg (27 Mar 2020) +- mailmap: fixup a few author names/fields + + Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and + Svyatoslav Mishyn -- smtp: Support UTF-8 based host names in the VRFY command +- version: add 'cainfo' and 'capath' to version info struct + + Suggested-by: Timothe Litt + URL: https://curl.haxx.se/mail/lib-2020-03/0090.html + Reviewed-by: Jay Satiro + + Closes #5150 -- smtp: Support UTF-8 based host names in the RCPT TO command +- RELEASE-NOTES: synced -- smtp: Support UTF-8 based host names in the MAIL command +Jay Satiro (26 Mar 2020) +- SSLCERTS.md: Fix example code for setting CA cert file - Non-ASCII host names will be ACE encoded if IDN is supported. - -- url: Make the IDN conversion functions available to others + Prior to this change the documentation erroneously said use + CURLOPT_CAPATH to set a CA cert file. + + Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html + Reported-by: Timothe Litt + + Closes https://github.com/curl/curl/pull/5151 -- smtp: Added UTF-8 mailbox tests to verify existing behaviour +Marc Hoersken (26 Mar 2020) +- sockfilt: add logmsg output to select_ws_wait_thread on Windows + + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Closes #5086 -- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case +Daniel Stenberg (26 Mar 2020) +- docs/make: generate curl.1 from listed files only + + Previously it rendered the page from files matching "*.d" in the correct + directory, which worked fine in git builds when the files were added but + made it easy to forget adding the files to the dist. + + Now, only man page sections listed in DPAGES in Makefile.inc will be + used, thus "forcing" us to update this to get the man page right and get + it included in the dist at the same time. + + Ref: #5146 + Closes #5149 -- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() +- openssl: adapt to functions marked as deprecated since version 3 - The dot character between the host and the tld was not being escaped, - which meant it specified a match of 'any' character rather than an - explicit dot separator. + OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES + functions we use. - Additionally removed the dot character from the host name as it allowed - the following to be specified as a valid address in our test cases: + Fix the MD4 and SSL_CTX_load_verify_locations warnings. - + In configure, detect OpenSSL v3 and if so, inhibit the deprecation + warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and + until we rewrite the code to use non-deprecated functions we better + ignore these warnings as they don't help us. - Both are typos from 98f7ca7 and 8880f84 :( + Closes #5139 + +- dist: add mail-rcpt-allowfails.d to the tarball - I can't remember whether my intention was to allow sub-domains to be - specified in the host or not with these additional dots, but by placing - it outside of the host means it can only be specified once per domain - and by placing a + after the new grouping support for sub-domains is - kept. + Reported-by: Maksim Stsepanenka + Reviewed-by: Jat Satiro - Closes #4912 + Closes #5146 -- hmac: Added a unit test for the HMAC hash generation +- travis: update the ngtcp2 build to use the latest OpenSSL patch - Closes #4973 - -- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function + ... which also makes it OpenSSL 1.1.1d based and not v3. -- tests: Added a unit test for MD4 digest generation +Marc Hoersken (24 Mar 2020) +- CI: remove default Ubuntu build from GitHub Actions - Closes #4970 + We are already running a very similar Ubuntu build on Travis CI. + The macOS variant of this default build is kept on Github Actions. -- md4: Use const for the length input parameter +- CI: bring GitHub Actions fuzzing job in line with macOS jobs - This keeps the interface the same as md5 and sha256. + Update YAML formatting, job naming and triggers. -- test1610: Fixed the link to the unit test +- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions - Typo from 81c37124. - -- ntlm: Removed the dependency on the TLS libaries when using MD5 + Reduce workload on Azure Pipelines and Travis CI while + consolidating macOS jobs onto less utilized GitHub Actions. - As we have our own MD5 implementation use the MD5 wrapper to remove the - TLS dependency. + Reviewed-by: Daniel Stenberg - Closes #4967 - -- md5/sha256: Updated the functions to allow non-string data to be hashed + Closes #5124 -- digest: Corrected the name of the local HTTP digest function +Daniel Stenberg (24 Mar 2020) +- config: remove all defines of HAVE_DES_H + + As there's no code using it. - Follow up to 2b5b37cb. Local static functions do not require the Curl - prefix. + Closes #5144 -- tests: Added a unit test for SHA256 digest generation +- copyright: fix out-of-date copyright ranges and missing headers + + Reported by the new script 'scripts/copyright.pl'. The script has a + regex whitelist for the files that don't need copyright headers. - Follow up to 2b5b37c. + Removed three (mostly usesless) README files from docs/ - Closes #4968 + Closes #5141 -- md4: Fixed compilation issues when using GNU TLS gcrypt +- packages: add OS400/chkstrings.c to the dist - * Don't include 'struct' in the gcrypt MD4_CTX typedef - * The call to gcry_md_read() should use a dereferenced ctx - * The call to gcry_md_close() should use a dereferenced ctx + Reported-by: Jon Rumsey + Fixes #5142 + Closes #5143 + +- [Clément Notin brought this change] + + nghttp2: 1.12.0 required - Additional minor whitespace issue in the USE_WIN32_CRYPTO code. + since nghttp2_session_set_local_window_size is needed - Closes #4959 + Closes #5140 -Daniel Stenberg (21 Feb 2020) - RELEASE-NOTES: synced -- http2: now require nghttp2 >= 1.12.0 - - To simplify our code and since earlier versions lack important function - calls libcurl needs to function correctly. - - nghttp2 1.12.0 was relased on June 26, 2016. - - Closes #4961 +- [Calvin Buckley brought this change] -- gtls: fix the copyright year + OS400: Update strings for ccsid-ifier - Follow-up from 41fcb4f609 - -- [jethrogb brought this change] + Fixes build. + + Closes #5132 - GnuTLS: Always send client cert +- cirrus: make freebsd ignore the tests instead of skipping - TLS servers may request a certificate from the client. This request - includes a list of 0 or more acceptable issuer DNs. The client may use - this list to determine which certificate to send. GnuTLS's default - behavior is to not send a client certificate if there is no - match. However, OpenSSL's default behavior is to send the configured - certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL - behavior. + To allow us to see in the CI logs how they actually behave - Authored-by: jethrogb on github - Fixes #1411 - Closes #4958 + Closes #5091 -- [Leo Neat brought this change] +- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 - github action: add CIFuzz +- Revert "cirrus-ci: disable the FreeBSD 13 builds" - Closes #4960 + This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. -- cleanup: comment typos +- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override - Spotted by 'codespell' + To let debug-builds return fake values, like in test 970. - Closes #4957 + Ref: #5131 + Closes #5136 -Steve Holme (20 Feb 2020) -- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions +- test970: improve the test - Whilst lib\md4.c used this pre-processor, lib\md5.c and - src\tool_metalink.c did not and simply relied on the WIN32 - pre-processor directive. + - send more data to make problems more obvious + - don't start the data with minus, it makes diffs harder to read + - skip the headers in the stdout comparison + - save to a file name to also verify 'filename_effective' - Reviewed-by: Marcel Raad - Closes #4955 + Ref: #5131 -Daniel Stenberg (19 Feb 2020) -- connect: remove some spurious infof() calls +- CURLINFO_NUM_CONNECTS: improve accuracy - As they were added primarily for debugging, they provide little use for - users. + The counter was not bumped in all cases correctly. - Closes #4951 + Reported-by: Marcel Raad + Ref: #5131 + Closes #5135 -- HTTP-COOKIES: mention that a trailing newline is required - - ... so that we know we got the whole and not a partial line. - - Also, changed the formatting of the fields away from a table again since - the table format requires a github-markdown tool version that we don't - run on the web server atm. - - Reported-by: Sunny Bean - Fixes #4946 - Closes #4947 +- TODO: Use "random" ports for the test servers -- nit: Copyright year out of date +- lib/curl_setup: adjust the copyright year range - Follow-up to 1fc0617dcc + Follow-up from d820224b8 -Jay Satiro (18 Feb 2020) -- tool_util: Improve Windows version of tvnow() - - - Change tool_util.c tvnow() for Windows to match more closely to - timeval.c Curl_now(). - - - Create a win32 init function for the tool, since some initialization - is required for the tvnow() changes. +Jay Satiro (21 Mar 2020) +- curl_setup: define _WIN32_WINNT_[OS] symbols - Prior to this change the monotonic time function used by curl in Windows - was determined at build-time and not runtime. That was a problem because - when curl was built targeted for compatibility with old versions of - Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps - every 49.7 days that Windows has been running. + .. because not all Windows build systems have those symbols, and even + those that do may be missing newer symbols (eg the Windows 7 SDK does + not define _WIN32_WINNT_WIN10). - This change makes curl behave similar to libcurl's tvnow function, which - determines at runtime whether the OS is Vista+ and if so calls - QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used - because it has higher resolution than the more obvious candidate - GetTickCount64). The changes to tvnow are basically a copy and paste but - the types in some cases are different. + Those symbols are used in build-time logic to decide which API to use + and prior to this change if the symbols were missing it would have + resulted in deprecated API being used when more recent functions were + available (eg GetVersionEx used instead of VerifyVersionInfo). - Ref: https://github.com/curl/curl/issues/3309 + Reported-by: FuccDucc@users.noreply.github.com - Closes https://github.com/curl/curl/pull/4847 + Probably fixes https://github.com/curl/curl/issues/4995 + Closes https://github.com/curl/curl/pull/5057 -Daniel Stenberg (18 Feb 2020) -- SOCKS: fix typo in printf formatting - - Follow-up to 4a4b63daa - - Reported-by: Peter Piekarski - Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 +- [Ross Burton brought this change] -- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section + curl-functions.m4: remove inappropriate AC_REQUIRE + + AC_REQUIRE means "if this macro hasn't been executed already, execute + it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) + isn't correct at that will execute AC_RUN_IFELSE without any arguments. - to be in sync with the description above + With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, + AC_RUN_IFELSE without a default value when cross-compiling is fatal. + The result is that curl with autoconf 2.70 cannot cross-compile. - Reported-by: Joonas Kuorilehto - Fixes #4943 - Closes #4945 - -- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" + Fixes https://github.com/curl/curl/issues/5126 + Closes https://github.com/curl/curl/pull/5130 -- altsvc: make saving the cache an atomic operation +Marc Hoersken (20 Mar 2020) +- ci/tests: fix Azure Pipelines not running Windows containers - ... by writing the file to temp name then rename to the final when done. + Workaround posted here: microsoft/azure-pipelines-agent#2864 - Assisted-by: Jay Satiro - Fixes #4936 - Closes #4942 - -- rename: a new file for Curl_rename() + Assisted-by: Simon Chalifoux + Assisted-by: Tommy Petty - And make the cookie save function use it. + Fixes #5117 + Closes #5129 -- cookies: make saving atomic with a rename - - Saves the file as "[filename].[8 random hex digits].tmp" and renames - away the extension when done. +Daniel Stenberg (20 Mar 2020) +- tests: add test 430, 431 and 432 to verify the --config fix - Co-authored-by: Jay Satiro - Reported-by: Mike Frysinger - Fixes #4914 - Closes #4926 + Verify the fixes in 4e0b4fee4 -- RELEASE-NOTES: synced +- [Rici Lake brought this change] -- socks: make the connect phase non-blocking + cmdline: fix handling of OperationConfig linked list (--next) - Removes two entries from KNOWN_BUGS. + Ensures that -K/--config inserts new items at the end of the list + instead of overwriting the second item, and that after a -K/--config + option has been parsed, the option parser's view of the current config + is update. - Closes #4907 + Fixes #5120 + Closes #5123 -- multi: if Curl_readwrite sets 'comeback' use expire, not loop - - Otherwise, a very fast single transfer ricks starving out other - concurrent transfers. - - Closes #4927 +Marc Hoersken (20 Mar 2020) +- test2100: fix static port instead of dynamic value being used -- ftp: convert 'sock_accepted' to a plain boolean - - This was an array indexed with sockindex but it was only ever used for - the secondary socket. - - Closes #4929 +- test970: fix static ip:port instead of dynamic values being used -Jay Satiro (15 Feb 2020) -- CURLINFO_COOKIELIST.3: Fix example - - Prior to this change the example would try to import cookies from stdin, - which wasn't what was intended. - - Reported-by: 3dyd@users.noreply.github.com +Daniel Stenberg (19 Mar 2020) +- secure transport: remove the BACKEND define kludge - Fixes https://github.com/curl/curl/issues/4930 + Closes #5122 -Daniel Stenberg (14 Feb 2020) -- TODO: Paged searches on LDAP server - - Closes #4452 +- mbedtls: remove the BACKEND define kludge -- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS - - Closes #4108 +- bearssl: remove the BACKEND define kludge -- azure: disable brotli on the macos debug-builds - - Because of: - - brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] - const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], - - Closes #4925 +- wolfssl: remove the BACKEND define kludge -Steve Holme (13 Feb 2020) -- tool_home: Fix the copyright year being out of date - - Follow up to 9dc350b6. +- nss: remove the BACKEND define kludge -Jay Satiro (12 Feb 2020) -- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() - - - Deduplicate GetEnv() code. - - - On Windows change ultimate call to use Windows API - GetEnvironmentVariable() instead of C runtime getenv(). - - Prior to this change both libcurl and the tool had their own GetEnv - which over time diverged. Now the tool's GetEnv is a wrapper around - curl_getenv (libcurl API function which is itself a wrapper around - libcurl's GetEnv). - - Furthermore this change fixes a bug in that Windows API - GetEnvironmentVariable() is called instead of C runtime getenv() to get - the environment variable since some changes aren't always visible to the - latter. - - Reported-by: Christoph M. Becker - - Fixes https://github.com/curl/curl/issues/4774 - Closes https://github.com/curl/curl/pull/4863 +- gnutls: remove the BACKEND define kludge -Daniel Stenberg (12 Feb 2020) -- strerror.h: Copyright year out of date +- openssl: remove the BACKEND define kludge - Follow-up to 1c4fa67e8a8fcf6 + Use a proper variable instead to make it easier to use a debugger and + read the code. -Jay Satiro (12 Feb 2020) -- strerror: Increase STRERROR_LEN 128 -> 256 - - STRERROR_LEN is the constant used throughout the library to set the size - of the buffer on the stack that the curl strerror functions write to. +Marc Hoersken (19 Mar 2020) +- tests: make Python-based servers compatible with Python 2 and 3 - Prior to this change some extended length Windows error messages could - be truncated. + Update smbserver.py and negtelnetserver.py to be compatible with + Python 3 while staying backwards-compatible to support Python 2. - Closes https://github.com/curl/curl/pull/4920 - -- multi: fix outdated comment + Fix string encoding and handling of echoed and transferred data. - - Do not say that conn->data is "cleared" by multi_done(). + Tested with both Python 2.7.17 and Python 3.7.7 - If the connection is in use then multi_done assigns another easy handle - still using the connection to conn->data, therefore in that case it is - not cleared. + Reported-by: Daniel Stenberg + Assisted-by: Kamil Dudka + Reviewed-by: Marcel Raad - Closes https://github.com/curl/curl/pull/4901 + Fixes #5104 + Closes #5110 -- easy: remove dead code - - multi is already assigned to data->multi by curl_multi_add_handle. +Daniel Stenberg (18 Mar 2020) +- writeout_json: use curl_off_t printf() option for the time output - Closes https://github.com/curl/curl/pull/4900 - -Daniel Stenberg (12 Feb 2020) -- create-dirs.d: mention the mode + Follow-up to: 04c03416e68fd635a15 - Reported-by: Dan Jacobson - Fixes #4766 - Closes #4916 + Closes #5115 -- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording +- RELEASE-NOTES: synced - Assisted-by: Jay Satiro - Reported-by: Craig Andrews - Fixes #4909 - Closes #4910 + Uh, I missed this in 1a46b218db - RELEASE-NOTES: synced - -Steve Holme (9 Feb 2020) -- smtp: Simplify the MAIL command and avoid a duplication of send strings - - This avoids the duplication of strings when the optional AUTH and SIZE - parameters are required. It also assists with the modifications that - are part of #4892. - Closes #4903 + ... and bumped curlver.h to 7.70.0 -Daniel Stenberg (9 Feb 2020) -- altsvc: keep a copy of the file name to survive handle reset - - The alt-svc cache survives a call to curl_easy_reset fine, but the file - name to use for saving the cache was cleared. Now the alt-svc cache has - a copy of the file name to survive handle resets. +Jay Satiro (18 Mar 2020) +- http2: Fix erroneous debug message that h2 connection closed - Added test 1908 to verify. + Prior to this change in libcurl debug builds http2 stream closure was + erroneously referred to as connection closure. - Reported-by: Craig Andrews - Fixes #4898 - Closes #4902 - -Steve Holme (9 Feb 2020) -- url: Include the failure reason when curl_win32_idn_to_ascii() fails + Before: + * nread <= 0, server closed connection, bailing - Provide the failure reason in the failf() info just as we do for the - libidn2 version of code. + After: + * nread == 0, stream closed, bailing - Closes #4899 - -Jay Satiro (9 Feb 2020) -- asyn-thread: remove dead code - -Daniel Stenberg (8 Feb 2020) -- [Emil Engler brought this change] + Closes https://github.com/curl/curl/pull/5118 - github: Instructions to post "uname -a" on Unix systems in issues +Daniel Stenberg (18 Mar 2020) +- tool_setopt: correct the copyright year range - Closes #4896 + Follow-up to 5450428491 -- [Cristian Greco brought this change] +Jay Satiro (18 Mar 2020) +- [Johannes Schindelin brought this change] - configure.ac: fix comments about --with-quiche - - A simple s/nghttp3/quiche in some comments of --with-quiche. - Looks like a copy-paste error from --with-nghttp3. + schannel: add "best effort" revocation check option - Closes #4897 - -Steve Holme (7 Feb 2020) -- checksrc.bat: Fix not being able to run script from the main curl directory + - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and + --ssl-revoke-best-effort to allow a "best effort" revocation check. - If the script was ran from the main curl directory rather then the - projects directory then the script would simply exit without error: + A best effort revocation check ignores errors that the revocation check + was unable to take place. The reasoning is described in detail below and + discussed further in the PR. - C:\url> projects\checksrc.bat + --- - The user would either need to change to the projects directory, - explicitly specify the current working directory, or perform a - oneline hacky workaround: + When running e.g. with Fiddler, the schannel backend fails with an + unhelpful error message: - C:\url> cd projects - C:\url\projects> checksrc.bat + Unknown error (0x80092012) - The revocation function was unable + to check revocation for the certificate. - C:\url> checksrc.bat %cd% + Sadly, many enterprise users who are stuck behind MITM proxies suffer + the very same problem. - C:\url> pushd projects & checksrc.bat & popd + This has been discussed in plenty of issues: + https://github.com/curl/curl/issues/3727, + https://github.com/curl/curl/issues/264, for example. - Closes #4894 - -Daniel Stenberg (7 Feb 2020) -- [Pierre-Yves Bigourdan brought this change] - - digest: Do not quote algorithm in HTTP authorisation + In the latter, a Microsoft Edge developer even made the case that the + common behavior is to ignore issues when a certificate has no recorded + distribution point for revocation lists, or when the server is offline. + This is also known as "best effort" strategy and addresses the Fiddler + issue. - RFC 7616 section 3.4 (The Authorization Header Field) states that "For - historical reasons, a sender MUST NOT generate the quoted string syntax - for the following parameters: algorithm, qop, and nc". This removes the - quoting for the algorithm parameter. + Unfortunately, this strategy was not chosen as the default for schannel + (and is therefore a backend-specific behavior: OpenSSL seems to happily + ignore the offline servers and missing distribution points). - Reviewed-by: Steve Holme - Closes #4890 - -- ftp: remove the duplicated user/password struct fields + To maintain backward-compatibility, we therefore add a new flag + (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option + (`--ssl-revoke-best-effort`) to select the new behavior. - Closes #4887 - -- ftp: remove superfluous checking for crlf in user or pwd + Due to the many related issues Git for Windows and GitHub Desktop, the + plan is to make this behavior the default in these software packages. - ... as this is already done much earlier in the URL parser. + The test 2070 was added to verify this behavior, adapted from 310. - Also add test case 894 that verifies that pop3 with an encodedd CR in - the user name is rejected. + Based-on-work-by: georgeok + Co-authored-by: Markus Olsson + Signed-off-by: Johannes Schindelin - Closes #4887 + Closes https://github.com/curl/curl/pull/4981 -Steve Holme (6 Feb 2020) -- ntlm_wb: Use Curl_socketpair() for greater portability +- multi: Improve parameter check for curl_multi_remove_handle - Reported-by: Daniel Stenberg - Closes #4886 - -Daniel Stenberg (5 Feb 2020) -- [Frank Gevaerts brought this change] - - contributors: Also include people who contributed to curl-www + - If an easy handle is owned by a multi different from the one specified + then return CURLM_BAD_EASY_HANDLE. - Closes #4884 + Prior to this change I assume user error could cause corruption. + + Closes https://github.com/curl/curl/pull/5116 -- [Frank Gevaerts brought this change] +Viktor Szakats (17 Mar 2020) +- windows: suppress UI in all CryptAcquireContext() calls + + Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters + Reviewed-by: Marc Hörsken + Closes https://github.com/curl/curl/pull/5088 - contrithanks: Use the most recent tag by default - - (similar to 5296abe) +Daniel Stenberg (17 Mar 2020) +- writeout_json: add missing comma to fix the HTTP version - Closes #4883 + Follow-up to 04c03416e68fd635a15 -- scripts: use last set tag if none given +- test 970: verify --write-out '%{json}' - Makes 'delta' and 'contributors.sh' easier to use. + Makes curl_easy_getinfo() of "variable" numerical content instead return + the number set in the env variable `CURL_TIME`. - Make the delta script invoke contrithanks to get current number of - contributors instead of counting THANKS, for accuracy. + Makes curl_version() of "variable" textual content. This guarantees a + stable version string which can be tested against. Environment variable + `CURL_VERSION` defines the content. - Closes #4881 + Assisted-by: Mathias Gumz -- ftp: shrink temp buffers used for PORT - - These two stack based buffers only need to be 46 + 66 bytes instead of - 256 + 1024. - - Closes #4880 +- [Mathias Gumz brought this change] -- curl: error on --alt-svc use w/o support + writeout: support to generate JSON output - Make the tool check for alt-svc support at run-time and return error - accordingly if not present when the option is used. + This commit adds support to generate JSON via the writeout feature: - Reported-by: Harry Sintonen - Closes #4878 - -- docs/HTTP3: add --enable-alt-svc to curl's configure - -- RELEASE-PROCEDURE: feature win is closed post-release a few days + -w "%{json}" - We've tried to uphold this already but let's make it official by - publicly stating this is the way we do it. + It leverages the existing infrastructure as much as possible. Thus, + generating the JSON on STDERR is possible by: - Closes #4877 - -- altsvc: set h3 version at a common single spot + -w "%{stderr}%{json}" - ... and move the #ifdefs out of the functions. Addresses the fact they - were different before this change. + This implements a variant of + https://github.com/curl/curl/wiki/JSON#--write-out-json. - Reported-by: Harry Sintonen - Closes #4876 + Closes #4870 -- [Harry Sintonen brought this change] +- CI: stop ignoring 323, it is disabled - altsvc: improved header parser - - - Fixed the flag parsing to apply to specific alternative entry only, as - per RFC. The earlier code would also get totally confused by - multiprotocol header, parsing flags from the wrong part of the header. - - - Fixed the parser terminating on unknown protocols, instead of skipping - them. +- DISABLED: disable test 323 - - Fixed a busyloop when protocol-id was present without an equal sign. + The test uses SRP to "a server not supporting it" but modern stunnel + versions will silently accept it and remain happy. The test is therefore + faulty. - Closes #4875 - -- [Harry Sintonen brought this change] - - ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 - -- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 + I haven't figured out how to make stunnel explicitly reject SRP-using + connects. - Reported-by: James Fuller + Reported-by: Marc Hörsken + Fixes #5105 + Closes #5113 -Steve Holme (4 Feb 2020) -- ntlm: Pass the Curl_easy structure to the private winbind functions +Marc Hoersken (17 Mar 2020) +- ci/tests: increase timeouts for torture builds on Azure Pipelines - ...rather than the full conndata structure. - -Daniel Stenberg (4 Feb 2020) -- RELEASE-NOTES: synced - -- tool_operhlp: Copyright year out of date, should be 2020 + For some reason the torture builds have slowed down recently. - Follow-up from 2bc373740a3 - -- [Orgad Shaneh brought this change] + Reported-by: Daniel Stenberg - curl: avoid using strlen for testing if a string is empty +Daniel Stenberg (16 Mar 2020) +- cmake: add support for building with wolfSSL - Closes #4873 - -Steve Holme (3 Feb 2020) -- ntlm: Ensure the HTTP header data is not stored in the challenge/response - -Marcel Raad (3 Feb 2020) -- openssl: remove redundant assignment + My working build cmdline: - Fixes a scan-build failure on Bionic. + $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . - Closes https://github.com/curl/curl/pull/4872 + Assisted-by: Brad King + Closes #5095 -- travis: update non-OpenSSL Linux jobs to Bionic - - For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is - failing with "curl returned 52, when expecting 35". +- tool_operate: fix add_parallel_transfers when more are in queue - Closes https://github.com/curl/curl/pull/4872 - -Dan Fandrich (3 Feb 2020) -- cirrus: Add some missing semicolons + Trying to return early from the function if no new transfers were added + would break the "morep" argument and cause issues. This could lead to + zero content "transfers" (within quotes since they would never be + started) when parallel-max was reduced. - Newlines aren't preserved in this section so they're needed to separate - commands. The exports luckily worked anyway as a single long line, but - erroneously exported a variable called "export" - [skip ci] - -Daniel Gustafsson (2 Feb 2020) -- [Pedro Monreal brought this change] + Reported-by: Gavin Wong + Analyzed-by: Jay Satiro + Fixes #4937 + Closes #5112 - cleanup: fix typos and wording in docs and comments +- vtls: free ssl_config leftovers on out-of-memory - Closes #4869 - Reviewed-by: Emil Engler and Daniel Gustafsson - -Steve Holme (2 Feb 2020) -- ntlm: Move the winbind data into the NTLM data structure + Torture testing 2034 and 2037 found this. - To assist with adding winbind support to the SASL NTLM authentication, - move the winbind specific data out of conndata into ntlmdata. + Reported-by: Marc Hörsken + Fixes #5108 + Closes #5109 -Daniel Stenberg (30 Jan 2020) -- quiche: Copyright year out of date +Marc Hoersken (16 Mar 2020) +- ci/tests: fix Azure Pipelines not running for pull requests - Follow-up to 7fc63d72333a + Closes #5111 -- altsvc: use h3-25 +Daniel Stenberg (15 Mar 2020) +- gskit: update the copyright year range - Closes #4868 - -- [Alessandro Ghedini brought this change] + Follow-up from 083603c63a3 - quiche: update to draft-25 +Marc Hoersken (15 Mar 2020) +- gskit: use our internal select wrapper for portability - Closes #4867 + Follow up to c52b342 + Closes #5106 -- ngtcp2: update to git master and its draft-25 support +- tests: fix verification of stdout in test 1452 due to newline - Closes #4865 + Fixes test1452:41:1: error: missing tag before -- cookie: check __Secure- and __Host- case sensitively - - While most keywords in cookies are case insensitive, these prefixes are - specified explicitly to get checked "with a case-sensitive match". +- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI - (From the 6265bis document in progress) + Also force the package index/cache to be updated before installing. - Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 - Closes #4864 - -- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header + Closes #5103 -- oauth2-bearer.d: works for HTTP too +- tests/README: add note about manually installing python-impacket - Reported-by: Mischa Salle - Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html - Closes #4862 + Follow up to 4be2560 -- multi_done: if multiplexed, make conn->data point to another transfer +Daniel Stenberg (15 Mar 2020) +- transfer: cap retries of "dead connections" to 5 - ... since the current transfer is being killed. Setting to NULL is - wrong, leaving it pointing to 'data' is wrong since that handle might be - about to get freed. + When libcurl retries a connection due to it being "seemingly dead" or by + REFUSED_STREAM, it will now only do it up five times before giving up, + to avoid never-ending loops. - Fixes #4845 - Closes #4858 - Reported-by: dmitrmax on github + Reported-by: Dima Tisnek + Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html + Closes #5074 -- location.d: the method change is from POST to GET only - - Not from generic non-GET to GET. +- TODO: TLS-PSK with OpenSSL - Reported-by: Andrius Merkys - Ref: #4859 - Closes #4861 + Closes #5081 -- urlapi: guess scheme correct even with credentials given - - In the "scheme-less" parsing case, we need to strip off credentials - first before we guess scheme based on the host name! - - Assisted-by: Jay Satiro - Fixes #4856 - Closes #4857 +Marc Hoersken (15 Mar 2020) +- select: add 'timeout_ms' wrap-around precaution to Curl_select -- global_init: move the IPv6 works status bool to multi handle - - Previously it was stored in a global state which contributed to - curl_global_init's thread unsafety. This boolean is now instead figured - out in curl_multi_init() and stored in the multi handle. Less effective, - but thread safe. +- select: fix 'pending_ms' is assigned a value that is never used - Closes #4851 - -- [Jay Satiro brought this change] + Detected by Codacy - README: mention that the docs is in docs/ +- select: move duplicate select preparation code into Curl_select - Reported-by: Austin Green - Fixes #4830 - Closes #4853 + Reviewed by Daniel Stenberg + Reviewed by Marcel Raad + Closes #5078 -- curl.h: define CURL_WIN32 on windows +Daniel Stenberg (15 Mar 2020) +- connect: happy eyeballs cleanup - ... so that the subsequent logic below can use a single known define to know - when built on Windows (as we don't define WIN32 anymore). + Make sure each separate index in connn->tempaddr[] is used for a fixed + family (and only that family) during the connection process. - Follow-up to 1adebe7886ddf20b + If family one takes a long time and family two fails immediately, the + previous logic could misbehave and retry the same family two address + repeatedly. - Reported-by: crazydef on github - Assisted-by: Marcel Raad - Fixes #4854 - Closes #4855 + Reported-by: Paul Vixie + Reported-by: Jay Satiro + Fixes #5083 + Fixes #4954 + Closes #5089 -- RELEASE-NOTES: synced +Marc Hoersken (15 Mar 2020) +- ci/tests: fix and align setting TFLAGS for make test-nonflaky -- [Jon Rumsey brought this change] +- ci/tests: install test suite dependencies stunnel and impacket - urldata: do string enums without #ifdefs for build scripts +- tests: remove python_dependencies for smbserver from our tree + + Users of the SMB tests will have to install impacket manually. - ... and check for inconsistencies for OS400 at build time with the new - chkstrings tool. + Reasoning: our in-tree version of impacket was quite outdated + and only compatible with Python 2 which is already end-of-life. + Upgrading to Python 3 and a compatible impacket version would + require to import additional Python-only and CPython-extension + dependencies. This would have hindered portability enormously. - Closes #4822 + Closes #5094 -- curl: make the -# spaceship bar not wrap the line +Jay Satiro (14 Mar 2020) +- Makefile.m32: Improve windres parameter compatibility + + - s/COFF/coff/ - The fixed-point math made us lose precision and thus a too high index - value could be used for outputting the hashtags which could overwrite - the newline. + Some versions of windres do not recognize uppercase COFF as a valid + way to specify the COFF output format. - The fix increases the precision in the sine table (*100) and the - associated position math. + Reported-by: Steven Penny - Reported-by: Andrew Potter - Fixes #4849 - Closes #4850 + Fixes https://github.com/curl/curl/issues/5099 + Closes https://github.com/curl/curl/pull/5101 -- global_init: assume the EINTR bit by default +- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares + + - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in + curl_easy_duphandle. - - Removed from global_init since it isn't thread-safe. The symbol will - still remain to not break compiles, it just won't have any effect going - forward. + Prior to this change if c-ares was used as the resolver backend and + either it was too old or libcurl was built without IPv6 support then + some of our resolver functions could return CURLE_NOT_BUILT_IN to + curl_easy_duphandle causing it to fail. - - make the internals NOT loop on EINTR (the opposite from previously). - It only risks returning from the select/poll/wait functions early, and that - should be risk-free. + Caused by c8f086b which shipped in 7.69.1. - Closes #4840 - -- [Peter Piekarski brought this change] - - conn: do not reuse connection if SOCKS proxy credentials differ + Reported-by: Karl Chen - Closes #4835 + Fixes https://github.com/curl/curl/issues/5097 + Closes https://github.com/curl/curl/pull/5100 -- llist: removed unused Curl_llist_move() +Daniel Stenberg (13 Mar 2020) +- docs: add warnings about FILE: URLs on Windows - (and the corresponding unit test) + - --url man page section + - libcurl-security.3 gets the full text + - CURLOPT_URL.3 - Closes #4842 - -- conncache: removed unused Curl_conncache_bundle_size() + Reported-by: Tim Sedlmeyer -- strcase: turn Curl_raw_tolower into static +- server/getpart: make the "XML-parser" stricter - Only ever used from within this file. - -- singleuse.pl: support new API functions, fix curl_dbg_ handling - -- wolfssh: make it init properly via Curl_ssh_init() + When extracting a
and there's no before +
, this now outputs an error and returns a wrong string to + make users spot the mistake. - Closes #4846 - -- [Aron Rotteveel brought this change] + Ref: #5070 + Closes #5071 - form.d: fix two minor typos +Marc Hoersken (13 Mar 2020) +- impacket: some more Python 3 code compatibility updates - Closes #4843 + This makes smbserver load on Python 3, but still not work completely. -- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields +- smbserver: pin Python version to 2 since we are not yet 3 compatible - Avoid "reparsing" the content and instead deliver more exactly what is - provided in the certificate and avoid truncating the data after 512 - bytes as done previously. This no longer removes embedded newlines. + Even though the existing code can be fixed to run on Python 3, the + tests will fail due to the Unicode transition the protocol is invalid. - Fixes #4837 - Reported-by: bnfp on github - Closes #4841 + Follow up to ee63837 + Closes #5085 -Jay Satiro (23 Jan 2020) -- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 - - - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. - - Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the - CURLOPT_PROXY_SSL_OPTIONS description. +Daniel Stenberg (12 Mar 2020) +- [Viktor Szakats brought this change] -Daniel Stenberg (22 Jan 2020) -- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER + cleanup: fix some text/comment typos - For now, no cert in the bundle actually sets a date there... + Closes #5087 + +Marc Hoersken (12 Mar 2020) +- smbserver: fix Python version specific ConfigParser import - Co-Authored-by: Jay Satiro - Reported-by: Christian Heimes - Fixes #4834 - Closes #4836 + Follow up to ee63837 and 8c7c4a6 + Fixes #5077 +Daniel Stenberg (11 Mar 2020) - RELEASE-NOTES: synced - -- [Pavel Volgarev brought this change] - - smtp: Allow RCPT TO command to fail for some recipients - - Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. - - Verified with the new tests 3002-3007 - Closes #4816 + bumped to 7.69.2 -- copyright: fix year ranges +Dan Fandrich (11 Mar 2020) +- tests/data: Fix some XML formatting issues in test cases - follow-up from dea17b519d (one of these days I'll learn to check before - I push) + This allows these test files to pass xmllint. -- [nao brought this change] +Daniel Stenberg (11 Mar 2020) +- [Muhammad Herdiansyah brought this change] - http: move "oauth_bearer" from connectdata to Curl_easy + Makefile: run the cd commands in a subshell - Fixes the bug where oauth_bearer gets deallocated when we re-use a - connection. + In bmake, if the directory is changed (with cd or anything else), bmake + won't return to the "root directory" on the next command (in the same + Makefile rule). This commit runs the cd command in a subshell so it + would work in bmake. - Closes #4824 - -- [Emil Engler brought this change] + Closes #5073 - curl: Let -D merge headers in one file again +- configure: convert -I to -isystem as a last step + + As all the -I uses in CFLAGS at that point are for system headers and + third party libraries this helps us remove/ignore warnings on those! - Closes #4762 - Fixes #4753 + Closes #5060 -- data.d: remove "Multiple files can also be specified" +- configure: fix -pedantic-errors for GCC 5 and later - It is superfluous and could even be misleading. + If --enable-werror is used. + + Follow-up to d5c0351055d5709da which added it too early in the configure + script before $compiler_num was set correctly and thus this option was + never used. - Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html - Reported-by: Mike Norton - Closes #4832 + Reported-by: Stepan Efremov + Fixes #5067 + Closes #5068 -Marcel Raad (20 Jan 2020) -- CMake: support specifying the target Windows version +- configure: document 'compiler_num' for gcc - Previously, it was only possible to set it to Windows Vista or XP by - setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. - Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to - explicitly set the target Windows version. `ENABLE_INET_PTON` is - ignored in this case. + The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + + MINOR and ignores the patch version, and since gcc version 7 it only + sets it to MAJOR*100. - Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 - Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 - Closes https://github.com/curl/curl/pull/4815 + Reported-by: Stepan Efremov + Ref: #5067 + Closes #5069 + +Version 7.69.1 (11 Mar 2020) + +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: 7.69.1 -Daniel Stenberg (20 Jan 2020) -- http.h: Copyright year out of date, should be 2020 - - Follow-up to 7ff9222ced8c +- THANKS: from the 7.69.1 release -- [加藤郁之 brought this change] +- [Marc Hoersken brought this change] - HTTP: increase EXPECT_100_THRESHOLD to 1Mb - - Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html + test1129: fix invalid case of closing XML-tag and Content-Length - Closes #4814 + Fixes #5070 + Closes #5072 -- ROADMAP: thread-safe `curl_global_init()` +Marc Hoersken (10 Mar 2020) +- tests/data: fix static ip instead of dynamic value being used - I'd like to see this happen. - -- RELEASE-NOTES: synced + Follow up to 94ced8e -- wolfssl: use the wc-prefixed symbol alternatives - - The symbols without wc_ prefix are not always provided. - - Ref: https://github.com/wolfSSL/wolfssl/issues/2744 +- tests/data: fix static ip:port instead of dynamic values being used - Closes #4827 + Closes #5065 -- polarssl: removed - - As detailed in DEPRECATE.md, the polarssl support is now removed after - having been disabled for 6 months and nobody has missed it. - - The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix - instead of the former 'polarssl' and the common functions that - previously were shared between mbedtls and polarssl and contained the - name 'polarssl' have now all been renamed to instead say 'mbedtls'. +- tests/server: fix missing use of exe_ext helper function - Closes #4825 + Follow up to 9819984 and 3dce984 + Reviewed-By: Daniel Stenberg + Closes #5064 -Marcel Raad (16 Jan 2020) -- libssh2: fix variable type - - This led to a conversion warning on 64-bit MinGW, which has 32-bit - `long` but 64-bit `size_t`. - - Closes https://github.com/curl/curl/pull/4823 +- runtests: log minimal and maximal used port numbers + +Daniel Stenberg (9 Mar 2020) +- [James Fuller brought this change] -Daniel Stenberg (16 Jan 2020) -- curl:progressbarinit: ignore column width from terminals < 20 + sftp: fix segfault regression introduced by #4747 - To avoid division by zero - or other issues. + This fix adds a defensive check for the case where the char *name in + struct libssh2_knownhost is NULL - Reported-by: Daniel Marjamäki - Closes #4818 - -- wolfssh: set the password correctly for PASSWORD auth + Fixes #5041 + Closes #5062 -- wolfssh: remove fprintf() calls (and uses of __func__) +- RELEASE-NOTES: synced -Marcel Raad (14 Jan 2020) -- CMake: use check_symbol_exists also for inet_pton +- socks4: fix host resolve regression - It doesn't make much sense to only check if the function can be linked - when it's not declared in any header and that is treated as an error. - With the correct target Windows version set, the function is declared - in ws2tcpip.h and the comment above the modified block is invalid. + 1. The socks4 state machine was broken in the host resolving phase - Also, move the definition of `_WIN32_WINNT` up to before all symbol - availability checks so that we don't have to care which ones must be - done after it. + 2. The code now insists on IPv4-only when using SOCKS4 as the protocol + only supports that. - Tested with Visual Studio 2019 and current MinGW-w64. + Regression from #4907 and 4a4b63d, shipped in 7.69.0 - Closes https://github.com/curl/curl/pull/4808 + Reported-by: amishmm on github + Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 + Closes #5061 -Jay Satiro (13 Jan 2020) -- schannel_verify: Fix alt names manual verify for UNICODE builds - - Follow-up to 29e40a6 from two days ago, which added that feature for - Windows 7 and earlier. The bug only occurred in same. - - Ref: https://github.com/curl/curl/pull/4761 +- [Patrick Monnerat brought this change] -Daniel Stenberg (13 Jan 2020) -- HTTP-COOKIES.md: describe the cookie file format + silly web server: silent a compilation warning - ... and refer to that file from from CURLOPT_COOKIEFILE.3 and - CURLOPT_COOKIELIST.3 + Recent gcc warns when byte count of strncpy() equals the destination + buffer size. Since the destination buffer is previously cleared and + the source string is always shorter, reducing the byte count by one + silents the warning without affecting the result. - Assisted-by: Jay Satiro - Reported-by: bsammon on github - Fixes #4805 - Closes #4806 + Closes #5059 -- [Tobias Hieta brought this change] +- [Patrick Monnerat brought this change] - CMake: Add support for CMAKE_LTO option. - - This enables Link Time Optimization. LTO is a proven technique for - optimizing across compilation units. + cookie: get_top_domain() sets zero length for null domains - Closes #4799 + This silents a compilation warning with gcc -O3. -- RELEASE-NOTES: synced +- [Patrick Monnerat brought this change] -- ConnectionExists: respect the max_concurrent_streams limits - - A regression made the code use 'multiplexed' as a boolean instead of the - counter it is intended to be. This made curl try to "over-populate" - connections with new streams. - - This regression came with 41fcdf71a1, shipped in curl 7.65.0. + test 1560: avoid valgrind false positives - Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same - check. + When using maximum code optimization level (-O3), valgrind wrongly + detects uses of uninitialized values in strcmp(). - Reported-by: Kunal Ekawde - Fixes #4779 - Closes #4784 + Preset buffers with all zeroes to avoid that. -- curl: make #0 not output the full URL - - It was not intended nor documented! - - Added test 1176 to verify. - - Reported-by: vshmuk on hackerone +Steve Holme (8 Mar 2020) +- sha256: Added WinCrypt implementation - Closes #4812 + Closed #5030 -- wolfSSH: new SSH backend - - Adds support for SFTP (not SCP) using WolfSSH. - - Closes #4231 +- sha256: Added SecureTransport implementation -- curl: remove 'config' field from OutStruct - - As it was just unnecessary duplicated information already stored in the - 'per_transfer' struct and that's around mostly anyway. +Daniel Stenberg (7 Mar 2020) +- lib1564: reduce number of mid-wait wakeup calls - The duplicated pointer caused problems when the code flow was aborted - before the dupe was filled in and could cause a NULL pointer access. + This test does A LOT of *wakeup() calls and then calls curl_multi_poll() + twice. The first *poll() is then expected to return early and the second + not - as the first is supposed to drain the socketpair pipe. - Reported-by: Brian Carpenter - Fixes #4807 - Closes #4810 - -- misc: Copyright year out of date, should be 2020 + It turns out however that when given "excessive" amounts of writes to + the pipe, some operating systems (the Solaris based are known) will + return EAGAIN before the pipe is drained, which in our test case causes + the second *poll() call to also abort early. - Follow-up to recent commits + This change attempts to avoid the OS-specific behaviors in the test by + reducing the amount of wakeup calls from 1234567 to 10. - [skip ci] + Reported-by: Andy Fiddaman + Fixes #5037 + Closes #5058 -Jay Satiro (11 Jan 2020) -- [Santino Keupp brought this change] +- [Patrick Monnerat brought this change] - libssh2: add support for forcing a hostkey type - - - Allow forcing the host's key type found in the known_hosts file. + mime: fix the binary encoder to handle large data properly - Currently, curl (with libssh2) does not take keys from your known_hosts - file into account when talking to a server. With this patch the - known_hosts file will be searched for an entry matching the hostname - and, if found, libssh2 will be told to claim this key type from the - server. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes https://github.com/curl/curl/pull/4747 + Fixes #4860 + Closes #4833 + Reported-by: RuurdBeerstra on github -- [Nicolas Guillier brought this change] +- [Patrick Monnerat brought this change] - cmake: Improve libssh2 check on Windows - - - Add "libssh2" name to FindLibSSH2 library search. + mime: do not perform more than one read in a row - On Windows systems, libSSH2 CMake installation may name the library - "LibSSH2". + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Prior to this change cmake only checked for name "ssh2". On Linux that - works fine because it will prepend the "lib", but it doesn't do that on - Windows. + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 667 checks mimepost using single-byte read callback with encoder. + New test 668 checks the end of part data early detection. - Closes https://github.com/curl/curl/pull/4804 + Fixes #4826 + Reported-by: MrdUkk on github -- [Faizur Rahman brought this change] +- [Patrick Monnerat brought this change] - schannel: Make CURLOPT_CAINFO work better on Windows 7 - - - Support hostname verification via alternative names (SAN) in the - peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. - - CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a - result CertGetNameString doesn't quite work on those versions of - Windows. This change provides an alternative solution for - CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier - versions of Windows. - - Prior to this change many certificates failed the hostname validation - when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates - now represent multiple hostnames and rely on the alternative names field - exclusively to represent their hostnames. + mime: latch last read callback status. - Reported-by: Jeroen Ooms + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Fixes https://github.com/curl/curl/issues/3711 - Closes https://github.com/curl/curl/pull/4761 + Fixes #4813 + Reported-by: MrdUkk on github -- [Emil Engler brought this change] +Marc Hoersken (7 Mar 2020) +- runtests: fix missing use of exe_ext helper function - ngtcp2: Add an error code for QUIC connection errors - - - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection - errors. - - Prior to this change CURLE_FAILED_INIT was used, but that was not - correct. - - Closes https://github.com/curl/curl/pull/4754 +Daniel Stenberg (7 Mar 2020) +- [Ernst Sjöstrand brought this change] -- multi: Change curl_multi_wait/poll to error on negative timeout - - - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when - curl_multi_wait/poll is passed timeout param < 0. + ares: store dns parameters for duphandle - Prior to this change passing a negative value to curl_multi_wait/poll - such as -1 could cause the function to wait forever. + With c-ares the dns parameters lives in ares_channel. Store them in the + curl handle and set them again in easy_duphandle. - Reported-by: hamstergene@users.noreply.github.com + Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. - Fixes https://github.com/curl/curl/issues/4763 + Fixes #4893 + Closes #5020 + Signed-off-by: Ernst Sjöstrand + +- version: make curl_version* thread-safe without using global context - Closes https://github.com/curl/curl/pull/4765 + Closes #5010 -- [Marc Aldorasi brought this change] +- RELEASE-NOTES: synced - cmake: Enable SMB for Windows builds - - - Define USE_WIN32_CRYPTO by default. This enables SMB. - - - Show whether SMB is enabled in the "Enabled features" output. - - - Fix mingw compiler warning for call to CryptHashData by casting away - const param. mingw CryptHashData prototype is wrong. +Marc Hoersken (7 Mar 2020) +- tests: use native Sleep function as fallback on Windows - Closes https://github.com/curl/curl/pull/4717 + Reviewed-By: Daniel Stenberg + Closes #5054 -- vtls: Refactor Curl_multissl_version to make the code clearer - - Reported-by: Johannes Schindelin - - Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 - - Closes https://github.com/curl/curl/pull/4803 +- perl: align order and completeness of Windows OS checks -Daniel Stenberg (10 Jan 2020) -- fix: Copyright year out of date, should be 2020 +Daniel Stenberg (7 Mar 2020) +- tool_cb_see: set correct copyright year range - Follow-up to 875314ed0bf3b + Follow-up to a39e5bfb9 -Marcel Raad (10 Jan 2020) -- hostip: move code to resolve IP address literals to `Curl_resolv` - - The code was duplicated in the various resolver backends. - - Also, it was called after the call to `Curl_ipvalid`, which matters in - case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This - caused test 1048 to fail on classic MinGW. - - The code ignores `conn->ip_version` as done previously in the - individual resolver backends. +Marc Hoersken (7 Mar 2020) +- seek: fix fallback for missing ftruncate on Windows - Move the call to the `resolver_start` callback up to appease test 655, - which wants it to be called also for literal addresses. + This fixes test 198 on versions of MinGW-w64 without ftruncate - Closes https://github.com/curl/curl/pull/4798 + Reviewed-By: Daniel Stenberg + Reviewed-By: Marcel Raad + Closes #5055 -Daniel Stenberg (9 Jan 2020) -- scripts/delta: adapt to new public header layout +- config-win32: Windows does not have ftruncate -- test1167: verify global symbols in public headers are curl prefixed +Daniel Stenberg (7 Mar 2020) +- pause: force a connection (re-)check after unpausing - ... using the new badsymbols.pl perl script + There might be data available that was already read off the socket, for + example in the TLS layer. - Fixes #4793 - Closes #4794 - -- libtest/mk-lib1521: adapt to new public header layout + Reported-by: Anders Berg + Fixes #4966 + Closes #5049 -- include: remove non-curl prefixed defines +- socks5: switch state properly when the resolve is done - ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ - enums. + Regression from 4a4b63d (and #4907) + Reported-by: vitaha85 on github + Fixes #5053 + Closes #5056 -- curl.h: remove WIN32 define +Jay Satiro (7 Mar 2020) +- libssh: Fix matching user-specified MD5 hex key - It isn't our job to define this in a public header - and it defines a - name outside of our naming scope. - -- tool_dirhie.c: fix the copyright year range + Prior to this change a match would never be successful because it + was mistakenly coded to compare binary data from libssh to a + user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). - Follow-up to: 4027bd72d9 - -- bump: work towards 7.69.0 is started + Reported-by: fds242@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/4971 + Closes https://github.com/curl/curl/pull/4974 -Jay Satiro (9 Jan 2020) -- tool_dirhie: Allow directory traversal during creation +Daniel Stenberg (6 Mar 2020) +- pause: bail out on bad input - - When creating a directory hierarchy do not error when mkdir fails due - to error EACCESS (13) "access denied". + A NULL easy handle or an easy handle without an associated connection + cannot be paused or unpaused. - Some file systems allow for directory traversal; in this case that it - should be possible to create child directories when permission to the - parent directory is restricted. + Closes #5050 + +Steve Holme (6 Mar 2020) +- unit1612: fixed the inclusion and compilation of the HMAC unit test - This is a regression caused by me in f16bed0 (precedes curl-7_61_1). - Basically I had assumed that if a directory already existed it would - fail only with error EEXIST, and not error EACCES. The latter may - happen if the directory exists but has certain restricted permissions. + Follow up to 3f74e5e6 to fix: - Reported-by: mbeifuss@users.noreply.github.com + - A typo in Makefile.inc where unit1611 was used instead + - Some compilation issues in unit1612.c - Fixes https://github.com/curl/curl/issues/4796 - Closes https://github.com/curl/curl/pull/4797 + Closes #5024 -Daniel Stenberg (9 Jan 2020) -- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers +Daniel Stenberg (6 Mar 2020) +- pause: return early for calls that don't change pause state - Closes #4080 + Reviewed-by: Patrick Monnerat + Ref: #4833 + Closes #5026 -- docs/RELEASE-PROCEDURE.md: pushed some release dates +Jay Satiro (6 Mar 2020) +- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html - -- runtests: make random seed fixed for a month + Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused + CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. - When using randomized features of runtests (-R and --shallow) it is - useful to have a fixed random seed to make sure for example extra - commits in a branch or a rebase won't change the seed that would make - repeated runs work differently. + Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html + Reported-by: Felipe Gasper - As it is also useful to change seed sometimes, the default seed is now - determined based on the current month (and first line curl -V - output). When the month changes, so will the random seed. + Closes https://github.com/curl/curl/pull/5048 + +- multi: skip EINTR check on wakeup socket if it was closed - The specific seed is also shown in the standard test suite top header - and it can be set explictly with the new --seed=[num] option so that the - exact order of a previous run can be achieved. + - Don't check errno on wakeup socket if sread returned 0 since sread + doesn't set errno in that case. - Closes #4734 - -- RELEASE-PROCEDURE.md: fix next release date (Feb 26) + This is a follow-up to cf7760a from several days ago which fixed + Curl_multi_wait to stop busy looping sread on the non-blocking wakeup + socket if it was closed (ie sread returns 0). Due to a logic error it + was still possible to busy loop in that case if errno == EINTR. - [skip ci] - -Version 7.68.0 (8 Jan 2020) - -Daniel Stenberg (8 Jan 2020) -- RELEASE-NOTES: 7.68.0 + Closes https://github.com/curl/curl/pull/5047 -- THANKS: updated with names from the 7.68.0 release +Daniel Stenberg (6 Mar 2020) +- transfer: set correct copyright year range -- RELEASE-PROCEDURE: add four future release dates +- urldata: remove the 'stream_was_rewound' connectdata struct member - and remove four past release dates + ... as it is never set anywhere. - [skip ci] + Follow-up to 2f44e94ef + Closes #5046 -Marcel Raad (6 Jan 2020) -- TrackMemory tests: always remove CR before LF +- Revert "pause: force-drain the transfer on unpause" + + This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) - It was removed for output containing ' =' via `s/ =.*//`. With classic - MinGW, this made lines with `free()` end with CRLF, but lines with e.g. - `malloc()` end with only LF. The tests expect LF only. + Clearly that didn't solve the problem correctly. - Closes https://github.com/curl/curl/pull/4788 + Reported-by: Christopher Reid + Reopens #4966 + Fixes #5044 -Daniel Stenberg (6 Jan 2020) -- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header - - ... to the private multihhandle.h. It is not for public use and it - wasn't prefixed correctly anyway! +- RELEASE-NOTES: synced - Closes #4790 + and bumped curlver.h -- file: fix copyright year range +- MANUAL: update a dict-using command line - Follow-up to 1b71bc532bd + The 'web1913' database is now invalid, use 'gcide' instead. -- curl -w: handle a blank input file correctly - - Previously it would end up with an uninitialized memory buffer that - would lead to a crash or junk getting output. - - Added test 1271 to verify. +- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - Reported-by: Brian Carpenter - Closes #4786 + Closes #3841 -- file: on Windows, refuse paths that start with \\ - - ... as that might cause an unexpected SMB connection to a given host - name. +- polarssl: remove more references and mentions - Reported-by: Fernando Muñoz - CVE-2019-15601 - Bug: https://curl.haxx.se/docs/CVE-2019-15601.html - -Jay Satiro (6 Jan 2020) -- CURLOPT_READFUNCTION.3: fix fopen params in example + Assisted-by: Jay Satiro + Follow-up to 6357a19ff29dac04 + Closes #5036 -- CURLOPT_READFUNCTION.3: fix variable name in example - - Reported-by: Paul Joyce - - Fixes https://github.com/curl/curl/issues/4787 +Marc Hoersken (4 Mar 2020) +- tests: wrap ignored test failures in braces -Daniel Stenberg (5 Jan 2020) -- curl:getparameter return error for --http3 if libcurl doesn't support - - Closes #4785 +- tests: align some Windows sleep defines with each other -- docs: mention CURL_MAX_INPUT_LENGTH restrictions - - ... for curl_easy_setopt() and curl_url_set(). +- tests: try to make sleeping portable by avoiding select - [skip ci] + select does not support just waiting on Windows: + https://perldoc.perl.org/perlport.html#select - Closes #4783 + Reviewed-By: Daniel Stenberg + Closes #5035 -- curl: properly free mimepost data - - ... as it could otherwise leak memory when a transfer failed. +Daniel Stenberg (4 Mar 2020) +- runtests.1: rephrase how to specify what tests to run - Added test 1293 to verify. + Also mention the new tilde-prefixed way to ignore test results. - Reported-by: Brian Carpenter - Fixes #4781 - Closes #4782 + Reviewed-By: Marc Hoersken + Closes #5033 -- curl: cleanup multi handle on failure - - ... to fix memory leak in error path. +- cirrus-ci: disable the FreeBSD 13 builds - Fixes #4772 - Closes #4780 - Reported-by: Brian Carpenter - -Marcel Raad (3 Jan 2020) -- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` + FreeBSD 13.0 is apparently close to a year away from a stable release + and has proven to cause intermittent builds failures recently. - Closes https://github.com/curl/curl/pull/4775 + Assisted-by: Dan Fandrich + Assisted-by: Fedor Korotkov + Fixes #5028 + Closes #5029 -Daniel Stenberg (3 Jan 2020) -- COPYING: it's 2020! - - [skip ci] +Version 7.69.0 (4 Mar 2020) -Jay Satiro (3 Jan 2020) -- [Marc Aldorasi brought this change] +Daniel Stenberg (4 Mar 2020) +- RELEASE-NOTES: 7.69.0 - tests: Fix bounce requests with truncated writes - - Prior to this change the swsbounce check in service_connection could - fail because prevtestno and prevpartno were not set, which would cause - the wrong response data to be sent to some tests and cause them to fail. +- THANKS: from 7.69.0 - Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 + Now sorted case insensitive -Marcel Raad (31 Dec 2019) -- tool: make a few char pointers point to const char instead - - These are read-only. +Marc Hoersken (3 Mar 2020) +- ci/tests: fix escaping of testnames and disable proxy for CI APIs - Closes https://github.com/curl/curl/pull/4771 + Follow up to ada581f and c0d8b96 + Closes #5031 -Jay Satiro (31 Dec 2019) -- tests: Change NTLM tests to require SSL +Jay Satiro (3 Mar 2020) +- cmake: Show HTTPS-proxy in the features output - Prior to this change tests that required NTLM feature did not require - SSL feature. + - Show HTTPS-proxy in the features output for those backends that + support it: OpenSSL, GnuTLS and NSS. - There are pending changes to cmake builds that will allow enabling NTLM - in non-SSL builds in Windows. In that case the NTLM auth strings created - are different from what is expected by the NTLM tests and they fail: + Prior to this change HTTPS-proxy was missing from the cmake features + output even if curl was built with it. Only cmake output was affected. + Both the library and tool correctly reported the feature. - "The issue with NTLM is that previous non-SSL builds would not enable - NTLM and so the NTLM tests would be skipped." + Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html + Reported-by: David Lopes - Assisted-by: marc-groundctl@users.noreply.github.com + Closes https://github.com/curl/curl/pull/5025 + +Marc Hoersken (3 Mar 2020) +- ci/tests: Make it possible to still run but ignore failing tests - Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 + This enables the development of a solution for the failing tests by + running them on CI while ignoring their result for the overall status. - Closes https://github.com/curl/curl/pull/4768 + Closes #4994 -- [Michael Forney brought this change] +- README.md: add Azure DevOps Pipelines build status badge - bearssl: Improve I/O handling - - Factor out common I/O loop as bearssl_run_until, which reads/writes TLS - records until the desired engine state is reached. This is now used for - the handshake, read, write, and close. - - Match OpenSSL SSL_write behavior, and don't return the number of bytes - written until the corresponding records have been completely flushed - across the socket. This involves keeping track of the length of data - buffered into the TLS engine, and assumes that when CURLE_AGAIN is - returned, the write function will be called again with the same data - and length arguments. This is the same requirement of SSL_write. +- ci/tests: Move CI test result creation above environment setup - Handle TLS close notify as EOF when reading by returning 0. + This avoids using our test servers as proxy to the AppVeyor API. - Closes https://github.com/curl/curl/pull/4748 + Closes #5022 -- travis: Fix error detection +- ci/tests: Send test results to AppVeyor for status overview - - Stop using inline shell scripts for before_script and script sections. + Closes #5021 + +Daniel Stenberg (3 Mar 2020) +- Revert "sha256: Added SecureTransport implementation" - Prior to this change Travis could ignore errors from commands in inline - scripts. I don't understand how or why it happens. This is a workaround. + This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) - Assisted-by: Simon Warta + That commit broke test 1610 on macos builds without TLS. - Ref: https://github.com/travis-ci/travis-ci/issues/1066 + Closes #5027 + +- dist: include tests/azure.pm in the tarball - Fixes https://github.com/curl/curl/issues/3730 - Closes https://github.com/curl/curl/pull/3755 + Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 + Reported-by: Marcel Raad -- tool_operate: fix mem leak when failed config parse +Steve Holme (3 Mar 2020) +- configure.ac: Disable metalink if mbedTLS is specified - Found by fuzzing the config file. + Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as + being one of the backends that metalink needs to be disabled for, I + seem to have included it in the list of allowed SSL/TLS backends in + comnfigure.ac :( - Reported-by: Geeknik Labs + Closes #5013 + +- sha256: Tidy up following recent changes - Fixes https://github.com/curl/curl/issues/4767 + Reviewed-by: Daniel Stenberg + Closes #4956 -- [Xiang Xiao brought this change] +- sha256: Added WinCrypt implementation - lib: remove erroneous +x file permission on some c files - - Modified by commit eb9a604 accidentally. - - Closes https://github.com/curl/curl/pull/4756 +- sha256: Added SecureTransport implementation -- [Xiang Xiao brought this change] +- sha256: Added mbedtls implementation - lib: fix warnings found when porting to NuttX - - - Undefine DEBUGASSERT in curl_setup_once.h in case it was already - defined as a system macro. - - - Don't compile write32_le in curl_endian unless - CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. - - - Include in socketpair.c. - - Closes https://github.com/curl/curl/pull/4756 +- sha256: Added GNU TLS gcrypt implementation -- os400: Add missing CURLE error constants - - Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 - Reported-by: Emil Engler +- sha256: Added GNU TLS Nettle implementation -- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 - - For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback - is passed two `size_t` parameters which, when multiplied, designate the - number of bytes of data passed in. In practice, CURL always sets the - first parameter (`size`) to 1. - - This practice is also enshrined in documentation and cannot be changed - in future. The documentation states that the default callback is - `fwrite`, which means `fwrite` must be a suitable function for this - purpose. However, the documentation also states that the callback must - return the number of *bytes* it successfully handled, whereas ISO C - `fwrite` returns the number of items (each of size `size`) which it - wrote. The only way these numbers can be equal is if `size` is 1. - - Since `size` is 1 and can never be changed in future anyway, document - that fact explicitly and let users rely on it. - - Reported-by: Frank Gevaerts - Commit-message-by: Christopher Head - - Ref: https://github.com/curl/curl/pull/2787 +Jay Satiro (2 Mar 2020) +- curl_escape.3: Add a link to curl_free - Fixes https://github.com/curl/curl/issues/4758 + Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 -- examples/postinmemory.c: Call curl_global_cleanup always - - Prior to this change curl_global_cleanup was not called if - curl_easy_init failed. +- curl_getenv.3: Fix the memory handling description - Reported-by: kouzhudong@users.noreply.github.com + - Tell the user to call curl_free() to free the pointer returned by + curl_getenv(). - Fixes https://github.com/curl/curl/issues/4751 - -Daniel Stenberg (21 Dec 2019) -- url2file.c: fix copyright year + Prior to this change the user was directed to call free(), but that + would not work in cases where the library and application use separate C + runtimes and therefore have separate heap memory management. - Follow-up to 525787269599b5 + Closes https://github.com/curl/curl/pull/5016 -- [Rickard Hallerbäck brought this change] +Daniel Stenberg (2 Mar 2020) +- [Nick Zitzmann brought this change] - examples/url2file.c: corrected a comment + md4: use init/update/final functions in Secure Transport - The comment was confusing and suggested that setting CURLOPT_NOPROGRESS - to 0L would both enable and disable debug output at the same time, like - a Schrödinger's cat of CURLOPTs. + We can use CC_MD4_Init/Update/Final without having to allocate memory + directly. - Closes #4745 - -- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 + Closes #4979 -- RELEASE-NOTES: synced +Marc Hoersken (2 Mar 2020) +- ci/tests: some MacOS builds randomly take longer than 20min -Jay Satiro (20 Dec 2019) -- ngtcp2: Support the latest update key callback type - - - Remove our cb_update_key in favor of ngtcp2's new - ngtcp2_crypto_update_key_cb which does the same thing. - - Several days ago the ngtcp2_update_key callback function prototype was - changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to - fix up our cb_update_key for that change they also added - ngtcp2_crypto_update_key_cb which does the same thing so we'll use that - instead. +Daniel Stenberg (2 Mar 2020) +- multi_wait: stop loop when sread() returns zero - Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c + It's unclear why it would ever return zero here, but this change fixes + Robert's problem and it shouldn't loop forever... - Closes https://github.com/curl/curl/pull/4735 + Reported-by: Robert Dunaj + Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html + Closes #5019 -Daniel Stenberg (19 Dec 2019) -- sws: search for "Testno:" header uncondtionally if no testno - - Even if the initial request line wasn't found. With the fix to 1455, the - test number is now detected correctly. +- http: mark POSTs with no body as "upload done" from the start - (Problem found when running tests in random order.) + As we have logic that checks if we get a >= 400 reponse code back before + the upload is done, which then got confused since it wasn't "done" but + yet there was no data to send! - Closes #4744 + Reported-by: IvanoG on github + Fixes #4996 + Closes #5002 -- tests: set LC_ALL in more tests +- tests: disable 962, 963 and 964 on Windows - Follow-up to 23208e330ac0c21 + These tests are also doing UTF-8 SMTP. - Closes #4743 + Follow-up to df207d2dd93b9e73 -- test165: set LC_ALL=en_US.UTF-8 too - - On my current Debian Unstable with libidn2 2.2.0, I get an error if - LC_ALL is set to blank. Then curl errors out with: +Marc Hoersken (2 Mar 2020) +- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer + +Daniel Stenberg (2 Mar 2020) +- configure: bump the AC_COPYRIGHT year range + +- [Steve Holme brought this change] + + tests: disable SMTP UTF-8 tests on Windows - curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 + Fixes #4988 + Closes #4992 + +- formdata/mime: copyright year range update - Closes #4738 + Due to the merge/revert cycle -- curl.h: add two defines for the "pre ISO C" case +- Revert "mime: latch last read callback status." - Without this fix, this caused a compilation failure on AIX with IBM xlc - 13.1.3 compiler. + This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. - Reported-by: Ram Krushna Mishra - Fixes #4739 - Closes #4740 + Fixes #5014 + Closes #5015 + Reopens #4833 -- create_conn: prefer multiplexing to using new connections +- Revert "mime: do not perform more than one read in a row" - ... as it would previously prefer new connections rather than - multiplexing in most conditions! The (now removed) code was a leftover - from the Pipelining code that was translated wrongly into a - multiplex-only world. + This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. + +- Revert "mime: fix the binary encoder to handle large data properly" - Reported-by: Kunal Ekawde - Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html - Closes #4732 + This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. -- test1456: remove the use of a fixed local port +- altsvc: both h3 backends now speak h3-27 - Fixup the test to instead not compare the port number. It sometimes - caused problems like this: + ... also updated the HTTP3 build description for ngtcp2 accordingly. + +- [Patrick Monnerat brought this change] + + mime: fix the binary encoder to handle large data properly - "curl: (45) bind failed with errno 98: Address already in use" + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes #4733 + Fixes #4860 + Reported-by: RuurdBeerstra on github -Jay Satiro (18 Dec 2019) -- CURLOPT_QUOTE.3: fix typos +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man - pages would not compile because a variable name was incorrect. + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Reported-by: Bylon2@users.noreply.github.com + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 664 checks mimepost using single-byte read callback with encoder. + New test 665 checks the end of part data early detection. - Fixes https://github.com/curl/curl/issues/4736 + Fixes #4826 + Reported-by: MrdUkk on github -- [Gisle Vanem brought this change] +- [Patrick Monnerat brought this change] - strerror: Fix compiler warning "empty expression" - - - Remove the final semi-colon in the SEC2TXT() macro definition. + mime: latch last read callback status. - Before: #define SEC2TXT(sec) case sec: txt = #sec; break; + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - After: #define SEC2TXT(sec) case sec: txt = #sec; break + Fixes #4813 + Reported-by: MrdUkk on github + Closes #4833 + +Steve Holme (1 Mar 2020) +- unit1651: Fixed conversion compilation warning - Prior to this change SEC2TXT(foo); would generate break;; which caused - the empty expression warning. + 371:17: warning: conversion to 'unsigned char' from 'int' may alter its + value [-Wconversion] - Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 + Closes #5008 -Daniel Stenberg (18 Dec 2019) -- curl/parseconfig: use curl_free() to free memory allocated by libcurl +- configure.ac: Disable metalink support if an incompatible SSL/TLS specified - Reported-by: bxac on github - Fixes #4730 - Closes #4731 - -- curl/parseconfig: fix mem-leak + tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, + The Win32 Crypto library and Apple's Common Crypto library. - When looping, first trying '.curlrc' and then '_curlrc', the function - would not free the first string. + If an TLS backend such as mbedTLS or WolfSSL is specified then the + following error is given during compilation along, with a load of + unresolved extern errors: - Closes #4731 - -- CURLOPT_URL.3: "curl supports SMB version 1 (only)" + Can't compile METALINK support without a crypto library. - [skip ci] + Reviewed-by: Daniel Stenberg + Closes #5006 -- test1270: a basic -w redirect_url test +Marc Hoersken (1 Mar 2020) +- ci/tests: Update Azure DevOps pipeline job display names - Closes #4728 + Make the configure step more descriptive and align others. -- HISTORY: the SMB(S) support landed in 2014 +- ci/tests: Fix typo in previous commit 597cf2 -- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore - - It is covered by USE_OPENSSL_ENGINE now. - - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 +- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available - Closes #4725 + For security reasons the access token is not available to PR builds. + Therefore we should not try to use the DevOps API with an empty token. -- cgit v1.2.3