From e5193f7cfc63bed18795b195a1b86a959638a5bf Mon Sep 17 00:00:00 2001 From: dartraiden Date: Thu, 20 Aug 2020 16:00:06 +0300 Subject: libcurl: update to 7.72.0 --- libs/libcurl/docs/CHANGES | 10210 ++++++++++++++++++++++---------------------- 1 file changed, 5124 insertions(+), 5086 deletions(-) (limited to 'libs/libcurl/docs/CHANGES') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 5ee0cc6964..51a99f4d52 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,7457 +6,7495 @@ Changelog -Version 7.71.1 (30 Jun 2020) +Version 7.72.0 (19 Aug 2020) -Daniel Stenberg (30 Jun 2020) -- RELEASE-NOTES: curl 7.71.1 +Daniel Stenberg (19 Aug 2020) +- RELEASE-NOTES: synced + + The curl 7.72.0 release -- THANKS: add contributors to 7.71.1 +- THANKS: add names from curl 7.72.0 release -- scripts/copyright.pl: skip .dcignore +Jay Satiro (18 Aug 2020) +- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions + + Reported-by: plujon@users.noreply.github.com + + Closes https://github.com/curl/curl/issues/5488 -- Revert "multi: implement wait using winsock events" +Daniel Stenberg (17 Aug 2020) +- Curl_easy: remember last connection by id, not by pointer - This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. + CVE-2020-8231 - That commit (from #5397) introduced a regression in 7.71.0. + Bug: https://curl.haxx.se/docs/CVE-2020-8231.html - Reported-by: tmkk on github - Fixes #5631 - Closes #5632 + Reported-by: Marc Aldorasi + Closes #5824 -- TODO: Add flag to specify download directory +- examples/rtsp.c: correct the copyright year -- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection +- RELEASE-PROCEDURE.md: add more future release dates -- cirrus-ci: disable FreeBSD 13 (again) +- [H3RSKO brought this change] + + docs: change "web site" to "website" - It has been failing for a good while again. This time we better leave it - disabled until we have more reason to believe it behaves. + According to wikipedia: - Closes #5628 + While "web site" was the original spelling, this variant has become + rarely used, and "website" has become the standard spelling + + Closes #5822 -- ngtcp2: sync with current master +- [Bevan Weiss brought this change] + + CMake: don't complain about missing nroff - ngtcp2 added two new callbacks + The curl_nroff_check() was always being called, and complaining if + *NROFF wasn't found, even when not making the manual. - Reported-by: Lucien Zürcher - Fixes #5624 - Closes #5627 - -- examples/multithread.c: call curl_global_cleanup() + Only check for nroff (and complain) if actually making the manual - Reported-by: qiandu2006 on github - Fixes #5622 - Closes #5623 + Closes #5817 -- vtls: compare cert blob when finding a connection to reuse - - Reported-by: Gergely Nagy - Fixes #5617 - Closes #5619 +- [Brian Inglis brought this change] -- RELEASE-NOTES: synced + libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin + + copy the LDFLAGS approach for adding same option with `libhostname` in + `libtest/Makefile.am`: + + - init `libstubgss_la_LDFLAGS_EXTRA` variable, + - add option to variable inside conditional, + - use variable in `libstubgss_la_LDFLAGS` + + Fixes #5819 + Closes #5820 -- terminology: call them null-terminated strings +- docs: clarify MAX_SEND/RECV_SPEED functionality - Updated terminology in docs, comments and phrases to refer to C strings - as "null-terminated". Done to unify with how most other C oriented docs - refer of them and what users in general seem to prefer (based on a - single highly unscientific poll on twitter). + ... in particular what happens if the maximum speed limit is set to a + value that's smaller than the transfer buffer size in use. - Reported-by: coinhubs on github - Fixes #5598 - Closes #5608 + Reported-by: Tomas Berger + Fixes #5788 + Closes #5813 -- http: fix proxy auth with blank password +- test1140: compare stdout - Regression in 7.71.0 + To make problems more immediately obvious when tests fail. - Added test case 346 to verify. + Closes #5814 + +- asyn-ares: correct some bad comments - Reported-by: Kristoffer Gleditsch - Fixes #5613 - Closes #5616 + Closes #5812 -- .dcignore: ignore tests and docs directories +- [Emil Engler brought this change] + + docs: Add video link to docs/CONTRIBUTE.md - This is a config file for deepcode.ai, a static code analyzer. + Closes #5811 -Jay Satiro (26 Jun 2020) -- tool_cb_hdr: Fix etag warning output and return code +- curl-config: ignore REQUIRE_LIB_DEPS in --libs output - - Return 'failure' on failure, to follow the existing style. + Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is + not considered for the --libs output. - - Put Warning: and the warning message on the same line. + Reported-by: ramsay-jones on github + Assisted-by: Brian Inglis and Ken Brown + Fixes #5793 + Closes #5808 + +- copyright: update/correct the year range on a few files + +- scripts/copyright.pl: ignore .muse files + +- [Emil Engler brought this change] + + multi: Remove 10-year old out-commented code - Ref: https://github.com/curl/curl/issues/5610 + The code hasn't been touched since 2010-08-18 - Closes https://github.com/curl/curl/pull/5612 + Closes #5805 -Daniel Stenberg (26 Jun 2020) -- CURLOPT_READFUNCTION.3: provide the upload data size up front +- KNOWN_BUGS: A shared connection cache is not thread-safe - Assisted-by: Jay Satiro - Closes #5607 + Closes #4915 + Closes #5802 -- test1539: do a HTTP 1.0 POST without a set size (fails) +- CONTRIBUTE: extend git commit message description - Attempt to reproduce #5593. Test case 1514 is very similar but uses - HTTP/1.1 and thus switches to chunked. + In particular how the first line works. - Closes #5595 + Closes #5803 -- [Baruch Siach brought this change] +- RELEASE-NOTES: synced - mbedtls: fix build with disabled proxy support - - Don't reference fields that do not exist. Fixes build failure: +- [Stefan Yohansson brought this change] + + transfer: move retrycount from connect struct to easy handle - vtls/mbedtls.c: In function 'mbed_connect_step1': - vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' + This flag was applied to the connection struct that is released on + retry. These changes move the retry counter into Curl_easy struct that + lives across retries and retains the new connection. - Closes #5615 + Reported-by: Cherish98 on github + Fixes #5794 + Closes #5800 -- codeql-analysis.yml: fix the 'languages' setting +- libssh2: s/ssherr/sftperr/ - It needs a 'with:' in front of it. - -GitHub (26 Jun 2020) -- [Daniel Stenberg brought this change] + The debug output used ssherr instead of sftperr which not only outputs + the wrong error code but also casues a warning on Windows. + + Follow-up to 7370b4e39f1 + + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 + Closes #5799 - gtihub: codeql-analysis.yml +- ftp: don't do ssl_shutdown instead of ssl_close - enables code security scanning with github actions + The shutdown function is for downgrading a connection from TLS to plain, + and this is not requested here. + + Have ssl_close reset the TLS connection state. + + This partially reverts commit f002c850d98d + + Reported-by: Rasmus Melchior Jacobsen + Reported-by: Denis Goleshchikhin + Fixes #5797 -Daniel Stenberg (25 Jun 2020) -- tests: verify newline in username and password for HTTP +Marc Hoersken (9 Aug 2020) +- CI/azure: fix test outcome values and use latest API version - test 1296 is a simply command line test + This makes sure that tests ignored or skipped are not shown + just in the category "Other", but with their correct state. - test 1910 is a libcurl test including a redirect + Closes #5796 -- url: allow user + password to contain "control codes" for HTTP(S) +- CI/azure: show runtime stats to investigate slowness - Reported-by: Jon Johnson Jr - Fixes #5582 - Closes #5592 + Also avoid naming conflict of TFLAGS env and tflags variables. + + Closes #5776 -- escape: make the URL decode able to reject only %00 bytes +Daniel Stenberg (8 Aug 2020) +- TLS naming: fix more Winssl and Darwinssl leftovers - ... or all "control codes" or nothing. + The CMake option is now called CMAKE_USE_SCHANNEL - Assisted-by: Nicolas Sterchele + The winbuild flag is USE_SCHANNEL + + The CI jobs and build scripts only use the new names and the new name + options + + Tests now require 'Schannel' (when necessary) + + Closes #5795 -- http2: set the correct URL in pushed transfers +- smtp_parse_address: handle blank input string properly - ...previously CURLINFO_EFFECTIVE_URL would report the URL of the - original "mother transfer", not the actually pushed resource. + Closes #5792 + +- runtests: run the DICT server on a random port number - Reported-by: Jonathan Cardoso Machado - Fixes #5589 - Closes #5591 + Removed support for -b (base port number) + + Closes #5783 -Jay Satiro (25 Jun 2020) -- [Javier Blazquez brought this change] +- RELEASE-NOTES: synced - openssl: Fix compilation on Windows when ngtcp2 is enabled +- runtests: move the TELNET server to a dynamic port - - Include wincrypt before OpenSSL includes so that the latter can - properly handle any conflicts between the two. + Rename the port variable to TELNETPORT to better match the existing + pattern. - Closes https://github.com/curl/curl/pull/5606 + Closes #5785 -Daniel Stenberg (25 Jun 2020) -- test543: extended to verify zero length input +- ngtcp2: adapt to error code rename - As was reported in #5601 + Closes #5786 -- escape: zero length input should return a zero length output +- runtests: move the smbserver to use a dynamic port number - Regression added in 7.71.0. + Closes #5782 + +- runtests: run the http2 tests on a random port number - Fixes #5601 - Reported-by: Kristoffer Gleditsch - Closes #5602 + Closes #5779 -- Curl_inet_ntop: always check the return code +- gtls: survive not being able to get name/issuer - Reported-by: Siva Sivaraman - Fixes #5412 - Closes #5597 + Closes #5778 -- sendf: improve the message on client write errors +- runtests: move the gnutls-serv tests to a dynamic port - Replace "Failed writing body (X != Y)" with - "Failure writing output to destination". Possibly slightly less cryptic. + Affects test 320, 321, 322 and 324. - Reported-by: coinhubs on github - Fixes #5594 - Closes #5596 + Closes #5778 -- RELEASE-NOTES: synced +- runtests: support dynamicly base64 encoded sections in tests + + This allows us to make test cases to use base64 at run-time and still + use and verify information determined at run-time, such as the IMAP test + server's port number in test 842. + + This change makes 12 tests run again that basically never ran since we + moved to dynamic port numbers. + + ftpserver.pl is adjusted to load test instructions and test number from + the preprocessed test file. + + FILEFORMAT.md now documents the new base64 encoding syntax. + + Reported-by: Marcel Raad + Fixes #5761 + Closes #5775 -- curlver: start working on 7.71.1 +- curl.1: add a few missing valid exit codes + + 93 - 96 can be returned as well. + + Closes #5777 -- [Denis Baručić brought this change] +- TODO: Use multiple parallel transfers for a single download + + Closes #5774 - DYNBUF.md: fix a typo: trail => tail +- TODO: Set the modification date on an uploaded file - Closes #5599 + Closes #5768 -Version 7.71.0 (23 Jun 2020) +- [Thomas M. DuBuisson brought this change] -Daniel Stenberg (23 Jun 2020) -- RELEASE-NOTES: curl 7.71.0 release + CI: Add muse CI config + + Closes #5772 -- THANKS: curl 7.71.0 additions +- [Thomas M. DuBuisson brought this change] -- url: make sure pushed streams get an allocated download buffer + travis/script.sh: fix use of `-n' with unquoted envvar - Follow-up to c4e6968127e876b0 + Shellcheck tells us "-n doesn't work with unquoted arguments. quote or + use [[ ]]." - When a new transfer is created, as a resuly of an acknowledged push, - that transfer needs a download buffer allocated. + And testing shows: - Closes #5590 + ``` + docker run --rm -it ubuntu bash + root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" + I ran + root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" + root@fe85ce156856:/# + ``` + + Closes #5773 -Jay Satiro (22 Jun 2020) -- openssl: Don't ignore CA paths when using Windows CA store +- h2: repair trailer handling - This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does - not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default - locations. Instead the CA store can now be used at the same time. + The previous h2 trailer fix in 54a2b63 was wrong and caused a + regression: it cannot deal with trailers immediately when read since + they may be read off the connection by the wrong 'data' owner. - The change is due to the impending release. The issue is still being - discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and - is now documented as experimental. + This change reverts the logic back to gathering all trailers into a + single buffer, like before 54a2b63. - Ref: bc052cc (parent commit) - Ref: https://github.com/curl/curl/issues/5585 + Reported-by: Tadej Vengust + Fixes #5663 + Closes #5769 -- tool_operate: Don't use Windows CA store as a fallback +Viktor Szakats (3 Aug 2020) +- windows: disable Unix Sockets for old mingw - Background: + Classic mingw and 10y+ old versions of mingw-w64 don't ship with + Windows headers having the typedef necessary for Unix Sockets + support, so try detecting these environments to disable this + feature. - 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate - store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides - CURLOPT_CAINFO if both are set. The curl tool will fall back to - CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set - via CURLOPT_CAINFO. + Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ - Problem: + Reviewed-by: Daniel Stenberg - libcurl may be built with hardcoded paths to a certificate bundle or - directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are - ignored. + Fixes #5674 + Closes #5758 + +Marcel Raad (3 Aug 2020) +- test1908: treat file as text - Solution: + Fixes the line endings on Windows. - A solution is still being discussed but since there's an impending - release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + Closes https://github.com/curl/curl/pull/5767 + +- TrackMemory tests: ignore realloc and free in getenv.c - Ref: https://github.com/curl/curl/issues/5585 + These are only called for WIN32. + + Closes https://github.com/curl/curl/pull/5767 -- openssl: Fix CA fallback logic for OpenSSL 3.0 build +Daniel Stenberg (3 Aug 2020) +- tests/FILEFORMAT.md: mention %HTTP2PORT + +- RELEASE-NOTES: synced + +- tlsv1.3.d. only for TLS-using connections - Prior to this change I assume a build error would occur when - CURL_CA_FALLBACK was used. + ... and rephrase that "not all" TLS backends support it. - Closes https://github.com/curl/curl/pull/5587 + Closes #5764 -Daniel Stenberg (22 Jun 2020) -- copyright: update mismatched copyright years +- tls-max.d: this option is only for TLS-using connections + + Ref: #5763 + Closes #5764 -- test1460: verify that -Ji is not ok +Marcel Raad (2 Aug 2020) +- [Cameron Cawley brought this change] -- tool_getparam: -i is not OK if -J is used + tool_doswin: Simplify Windows version detection - Reported-by: sn on hackerone - Bug: https://curl.haxx.se/docs/CVE-2020-8177.html + Closes https://github.com/curl/curl/pull/5754 -- [Peter Wu brought this change] +- [Cameron Cawley brought this change] - CMake: ignore INTERFACE_LIBRARY targets for pkg-config file + win32: Add Curl_verify_windows_version() to curlx - Reviewed-by: Marcel Raad - Fixes #5512 - Closes #5517 + Closes https://github.com/curl/curl/pull/5754 -- [Valentyn Korniienko brought this change] +- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL + + This makes the tests that require the OpenSSL feature also run for + those two compatible libraries. + + Closes https://github.com/curl/curl/pull/5762 - multibyte: Fixed access-> waccess to file for Windows Plarform +Daniel Stenberg (1 Aug 2020) +- multi: Condition 'extrawait' is always true + + Reported by Codacy. Reviewed-by: Marcel Raad - Closes #5580 + Closes #5759 -- altsvc: bump to h3-29 +Marcel Raad (1 Aug 2020) +- openssl: fix build with LibreSSL < 2.9.1 - Closes #5584 - -- urlglob: treat literal IPv6 addresses with zone IDs as a host name + `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were + introduced in LibreSSL 2.9.1 [0]. - ... and not as a "glob". Now done by passing the supposed host to the - URL parser which supposedly will do a better job at identifying "real" - numerical IPv6 addresses. + [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef - Reported-by: puckipedia on github - Fixes #5576 - Closes #5579 + Closes https://github.com/curl/curl/pull/5757 -- test1179: verify error message for non-existing cmdline option +Daniel Stenberg (1 Aug 2020) +- [Marc Aldorasi brought this change] -- tool_getparam: repair the error message for unknown flag + multi_remove_handle: close unused connect-only connections - Follow-up to 9e5669f3880674 - Detected by Coverity CID 1464582 ("Logically dead code") + Previously any connect-only connections in a multi handle would be kept + alive until the multi handle was closed. Since these connections cannot + be re-used, they can be marked for closure when the associated easy + handle is removed from the multi handle. - Closes #5577 - -- FILEFORMAT: describe verify/stderr + Closes #5749 -- connect: improve happy eyeballs handling +- checksrc: invoke script with -D to find .checksrc proper - For QUIC but also for regular TCP when the second family runs out of IPs - with a failure while the first family is still trying to connect. + Without the -D command line option, checksrc.pl won't know which + directory to load the ".checksrc" file from when building out of the + source tree. - Separated the timeout handling for IPv4 and IPv6 connections when they - both have a number of addresses to iterate over. + Reported-by: Marcel Raad + Fixes #5715 + Closes #5755 -- ngtcp2: never call fprintf() in lib code in release version +- [Carlo Marcelo Arenas Belón brought this change] -- ngtcp2: fix happy eyeballs quic connect crash + buildconf: retire ares buildconf invocation - Reported-by: Peter Wu - Fixes #5565 - Closes #5568 + no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 -- select: remove the unused ELAPSED_MS() macro +- [Carlo Marcelo Arenas Belón brought this change] + + buildconf: excempt defunct reference to ACLOCAL_FLAGS - Closes #5573 + retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as + the name is generic enough that it might be in use and relied upon from + the environment. -Marc Hoersken (17 Jun 2020) -- [rcombs brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - multi: implement wait using winsock events + buildconf: avoid array concatenation in die() - This avoids using a pair of TCP ports to provide wakeup functionality - for every multi instance on Windows, where socketpair() is emulated - using a TCP socket on loopback which could in turn lead to socket - resource exhaustion. + reported as error SC2145[1] by shellcheck, but not expected to cause + any behavioural differences otherwise. - Reviewed-by: Gergely Nagy - Reviewed-by: Marc Hörsken + [1] https://github.com/koalaman/shellcheck/wiki/SC2145 - Closes #5397 + Closes #5701 -Daniel Stenberg (17 Jun 2020) -- manpage: add three missing environment variables - - CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE +- travis: add ppc64le and s390x builds - Closes #5571 - -- RELEASE-NOTES: synced + Closes #5752 -- configure: for wolfSSL, check for the DES func needed for NTLM +Marc Hoersken (31 Jul 2020) +- connect: remove redundant message about connect failure - Also adds pkg-config support for the wolfSSL detection. + Reviewed-by: Daniel Stenberg + + Closes #5708 -- [Ruurd Beerstra brought this change] +- tests/sshserver.pl: fix compatibility with OpenSSH for Windows + + Follow up to #5721 - ntlm: enable NTLM support with wolfSSL +- CI/azure: install libssh2 for use with msys2-based builds - When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* - functions that OpenSSL has. This change take advantage of that. + This enables building and running the SFTP tests. + Unfortunately OpenSSH for Windows does not support SCP (yet). - Co-authored-by: Daniel Stenberg - Closes #5556 - Fixes #5548 + Reviewed-by: Daniel Stenberg + + Closes #5721 -- http: move header storage to Curl_easy from connectdata +- CI/azure: increase Windows job timeout once again - Since the connection can be used by many independent requests (using - HTTP/2 or HTTP/3), things like user-agent and other transfer-specific - data MUST NOT be kept connection oriented as it could lead to requests - getting the wrong string for their requests. This struct data was - lingering like this due to old HTTP1 legacy thinking where it didn't - mattered.. + Avoid aborted jobs due to performance issues on Azure DevOps. - Fixes #5566 - Closes #5567 + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + + Closes #5738 -- CODE_REVIEW.md: how to do code reviews in curl +Jay Satiro (30 Jul 2020) +- TODO: Schannel: 'Add option to allow abrupt server closure' - Assisted-by: Daniel Gustafsson - Assisted-by: Rich Salz - Assisted-by: Hugo van Kemenade - Assisted-by: James Fuller - Assisted-by: Marc Hörsken - Assisted-by: Jay Satiro + We should offer an option to allow abrupt server closures (server closes + SSL transfer without sending a known termination point such as length of + transfer or close_notify alert). Abrupt server closures are usually + because of misconfigured or very old servers. - Closes #5555 + Closes https://github.com/curl/curl/issues/4427 -- altsvc: remove the num field from the altsvc struct +- url: fix CURLU and location following - It was superfluous since we have the list.size alredy + Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was + incorrectly used for the location follow, resulting in infinite requests + to the original location. - Reported-by: Jay Satiro - Fixes #5553 - Closes #5563 + Reported-by: sspiri@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/5709 + Closes https://github.com/curl/curl/pull/5713 -- version.d: expanded and alpha-sorted +Daniel Stenberg (30 Jul 2020) +- RELEASE-NOTES: synced + +- [divinity76 brought this change] + + docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions - Added a few missing features not previously mentioned. Ordered them - alphabetically. + it helps make it obvious that most developers don't have to care about + the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 + years old, November 4 2009) - Closes #5558 + Closes #5744 -- ABI.md: rename to .md and polish the markdown +Jay Satiro (29 Jul 2020) +- tool_cb_wrt: fix outfile mode flags for Windows - Closes #5562 + - Use S_IREAD and S_IWRITE mode permission flags to create the file + on Windows instead of S_IRUSR, S_IWUSR, etc. + + Windows only accepts a combination of S_IREAD and S_IWRITE. It does not + acknowledge other combinations, for which it may generate an assertion. + + This is a follow-up to 81b4e99 from yesterday, which improved the + existing file check with -J. + + Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks + Ref: https://github.com/curl/curl/pull/5731 + + Closes https://github.com/curl/curl/pull/5742 -- HELP-US: add a section for "smaller tasks" +Daniel Stenberg (28 Jul 2020) +- checksrc: ban gmtime/localtime - The point of this section is to meet the CII Best Practices gold level - critera: + They're not thread-safe so they should not be used in libcurl code. - "The project MUST clearly identify small tasks that can be performed by - new or casual contributors" + Explictly enabled when deemed necessary and in examples and tests - Closes #5560 + Reviewed-by: Nicolas Sterchele + Closes #5732 -- TODO: retry on the redirected-to URL +- transfer: fix data_pending for builds with both h2 and h3 enabled - Closes #5462 + Closes #5734 -- mailmap: Nicolas Sterchele +- curl_multi_setopt: fix compiler warning "result is always false" + + On systems with 32 bit long the expression is always false. Avoid + the warning. + + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 + Closes #5736 -- [Nicolas Sterchele brought this change] +- curl: improve the existing file check with -J + + Previously a file that isn't user-readable but is user-writable would + not be properly avoided and would get overwritten. + + Reported-by: BrumBrum on hackerone + Assisted-by: Jay Satiro + Bug: https://hackerone.com/reports/926638 + Closes #5731 - TODO: remove 19.3 section title +- [Jonathan Nieder brought this change] + + multi: update comment to say easyp list is linear - Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on - curl documentation website + Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, + 2013-08-02), the easy handle list is not circular but ends with + ->next pointing to NULL. - Closes #5561 + Reported-by: Masaya Suzuki + Closes #5737 -- [Martin V brought this change] +- CURLOPT_NOBODY.3: fix the syntax for referring to options + + As test 1140 fails otherwise! + + Follow-up to e1bac81cc815 - test1560: avoid possibly negative association in wording +- ngtcp2: store address in sockaddr_storage - Closes #5549 + Reported-by: Tatsuhiro Tsujikawa + Closes #5733 -- share: don't set the share flag it something fails +- CURLOPT_NOBODY.3: clarify what setting to 0 means - When asking for a specific feature to be shared in the share object, - that bit was previously set unconditionally even if the shared feature - failed or otherwise wouldn't work. + ... and mention that HTTP with other methods than HEAD might get a body and + there's no option available to stop that. - Closes #5554 + Closes #5729 -- buildconf: remove -print from the find command that removes files +- setopt: unset NOBODY switches to GET if still HEAD - It's just too annoying and unnecessary to get a long list of files shown + Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented + action but before 7.71.0 that used to switch back to GET and with this + change (assuming the method is still set to HEAD) this behavior is + brought back. + + Reported-by: causal-agent on github + Fixes #5725 + Closes #5728 -- RELEASE-NOTES: synced +- [Ehren Bendler brought this change] -- wording: avoid blacklist/whitelist stereotypes + configure: cleanup wolfssl + pkg-config conflicts when cross compiling. - Instead of discussing if there's value or meaning (implied or not) in - the colors, let's use words without the same possibly negative - associations. + Also choose a different wolfSSL function to test for NTLM support. - Closes #5546 + Fixes #5605 + Closes #5682 -Jay Satiro (9 Jun 2020) -- tool_getparam: fix memory leak in parse_args +- configure: show zstd "no" in summary when built without it - Prior to this change in Windows Unicode builds most parsed options would - not be freed. + Reported-by: Marc Hörsken + Fixes #5720 + Closes #5730 + +- quiche: handle calling disconnect twice - Found using _CrtDumpMemoryLeaks(). + Reported-by: lilongyan-huawei on github + Fixes #5726 + Closes #5727 + +- [Nicolas Sterchele brought this change] + + getinfo: reset retry-after value in initinfo - Ref: https://github.com/curl/curl/issues/5545 + - Avoid re-using retry_after value from preceding request + - Add libtest 3010 to verify + + Reported-by: joey-l-us on github + Fixes #5661 + Closes #5672 -Daniel Stenberg (8 Jun 2020) -- socks: detect connection close during handshake +Marcel Raad (27 Jul 2020) +- WIN32: stop forcing narrow-character API - The SOCKS4/5 state machines weren't properly terminated when the proxy - connection got closed, leading to a busy-loop. + Except where the results are only used for character output. + getenv is not touched because it's part of the public API, and having + it return UTF-8 instead of ANSI would be a breaking change. - Reported-By: zloi-user on github - Fixes #5532 - Closes #5542 + Fixes https://github.com/curl/curl/issues/5658 + Fixes https://github.com/curl/curl/issues/5712 + Closes https://github.com/curl/curl/pull/5718 -- [James Fuller brought this change] +Jay Satiro (27 Jul 2020) +- [Tobias Stoeckmann brought this change] - multi: add defensive check on data->multi->num_alive + mprintf: Fix stack overflows - Closes #5540 - -- Curl_addrinfo: use one malloc instead of three + Stack overflows can occur with precisions for integers and floats. - To reduce the amount of allocations needed for creating a Curl_addrinfo - struct, make a single larger malloc instead of three separate smaller - ones. + Proof of concepts: + - curl_mprintf("%d, %.*1$d", 500, 1); + - curl_mprintf("%d, %+0500.*1$f", 500, 1); - Closes #5533 + Ideally, compile with -fsanitize=address which makes this undefined + behavior a bit more defined for debug purposes. + + The format strings are valid. The overflows occur due to invalid + arguments. If these arguments are variables with contents controlled + by an attacker, the function's stack can be corrupted. + + Also see CVE-2016-9586 which partially fixed the float aspect. + + Signed-off-by: Tobias Stoeckmann + + Closes https://github.com/curl/curl/pull/5722 -- [Alessandro Ghedini brought this change] +- [Tobias Stoeckmann brought this change] - quiche: update SSLKEYLOGFILE support + mprintf: Fix dollar string handling - quiche now requires the application to explicitly set the keylog path - for each connection, rather than reading the environment variable - itself. + Verify that specified parameters are in range. If parameters are too + large, fail early on and avoid out of boundary accesses. - Closes #5541 + Also do not read behind boundaries of illegal format strings. + + These are defensive measures since it is expected that format strings + are well-formed. Format strings should not be modifiable by user + input due to possible generic format string attacks. + + Closes https://github.com/curl/curl/pull/5722 -- tests: add two simple tests for --login-options +Daniel Stenberg (26 Jul 2020) +- ntlm: free target_info before (re-)malloc - Test 895 and 896 - as a follow-up to a3e972313b + OSS-Fuzz found a way this could get called again with the pointer still + pointing to a malloc'ed memory, leading to a leak. - Closes #5539 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 + + Closes #5724 -- ngtcp2: update with recent API changes +Marcel Raad (26 Jul 2020) +- CI/macos: set minimum macOS version - Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. + This enables some deprecation warnings. + Previously, autotools defaulted to 10.8. - Assisted-by: Tatsuhiro Tsujikawa - Closes #5538 + Closes https://github.com/curl/curl/pull/5723 -- [James Fuller brought this change] +Daniel Stenberg (26 Jul 2020) +- RELEASE-NOTES: synced - socks: remove unreachable breaks in socks.c and mime.c +Marcel Raad (25 Jul 2020) +- CI/macos: enable warnings as errors for CMake builds - Closes #5537 + Closes https://github.com/curl/curl/pull/5716 -- tool_cfgable: free login_options at exit +- CMake: fix test for warning suppressions - Memory leak - Reported-by: Geeknik Labs - Fixes #5535 - Closes #5536 - -- libssh2: keep sftp errors as 'unsigned long' + GCC doesn't warn for unknown `-Wno-` options, except if there are other + warnings or errors [0]. This was problematic with `CURL_WERROR` as that + warning-as-error cannot be suppressed. Notably, this always happened + with `-Wno-pedantic-ms-format` when not targeting Windows. So test for + the positive form of the warning instead, which should always result in + a diagnostic if unknown. - Remove weird work-around for storing the SFTP errors as int instead of - the "unsigned long" that libssh2 actually returns for SFTP errors. + [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html - Closes #5534 + Closes https://github.com/curl/curl/pull/5714 -Marc Hoersken (6 Jun 2020) -- timeouts: move ms timeouts to timediff_t from int and long +Jay Satiro (23 Jul 2020) +- curl.h: update CURLINFO_LASTONE - Now that all functions in select.[ch] take timediff_t instead - of the limited int or long, we can remove type conversions - and related preprocessor checks to silence compiler warnings. + CURLINFO_LASTONE should have been updated when + CURLINFO_EFFECTIVE_METHOD was added. - Avoiding conversions from time_t was already done in 842f73de. + Reported-by: xwxbug@users.noreply.github.com - Based upon #5262 - Supersedes #5214, #5220 and #5221 - Follow up to #5343 and #5479 - Closes #5490 + Fixes https://github.com/curl/curl/issues/5711 -Daniel Stenberg (6 Jun 2020) -- [François Rigault brought this change] +Marc Hoersken (22 Jul 2020) +- CI/azure: unconditionally enable warnings-as-errors with autotools + + Reviewed-by: Marcel Raad + + Follow up to #5694 + Closes #5706 - openssl: set FLAG_TRUSTED_FIRST unconditionally +Marcel Raad (21 Jul 2020) +- doh: remove redundant cast - On some systems, openssl 1.0.x is still the default, but it has been - patched to contain all the recent security fixes. As a result of this - patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be - defined, while the previous behavior of openssl to not look at trusted - chains first, remains. + Closes https://github.com/curl/curl/pull/5704 + +- CI/macos: unconditionally enable warnings-as-errors with autotools - Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to - probe for the behavior of openssl based on the existence ofmacros. + Previously, warnings were only visible in the output for most jobs. - Closes #5530 + Closes https://github.com/curl/curl/pull/5694 -- server/util: fix logmsg format using curl_off_t argument +- util: silence conversion warnings - ... this caused segfaults on armv7. + timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might + be a 64-bit integer. This is the case when building for recent macOS + versions, for example. Just treat tv_usec as an int, which should + hopefully always be sufficient on systems with + `HAVE_CLOCK_GETTIME_MONOTONIC`. - Regression added in dd0365d560aea5a (7.70.0) + Closes https://github.com/curl/curl/pull/5695 + +- md(4|5): don't use deprecated macOS functions - Reviewed-by: Jay Satiro - Closes #5529 + They are marked as deprecated for -mmacosx-version-min >= 10.15, + which might result in warnings-as-errors. + + Closes https://github.com/curl/curl/pull/5695 -- RELEASE-NOTES: synced +Daniel Stenberg (18 Jul 2020) +- strdup: remove the odd strlen check + + It confuses code analyzers with its use of -1 for unsigned value. Also, + a check that's not normally used in strdup() code - and not necessary. + + Closes #5697 -- [Cherish98 brought this change] +- [Alessandro Ghedini brought this change] - socks: fix expected length of SOCKS5 reply + travis: update quiche builds for new boringssl layout - Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the - reply ATYP is X'01'. This resulted in erroneously expecting more bytes - when the request length is greater than the reply length (e.g., when - remotely resolving the hostname). + This is required after https://github.com/cloudflare/quiche/pull/593 + moved BoringSSL around slightly. - Closes #5527 + This also means that Go is not needed to build BoringSSL anymore (the + one provided by quiche anyway). + + Closes #5691 -Marc Hoersken (5 Jun 2020) -- .gitignore: add directory containing the stats repo +Marcel Raad (17 Jul 2020) +- configure: allow disabling warnings - Since the new curl/stats repository is designed to be - checked out into the curl repository working tree as stats/ - it should be on the ignore list to aid in commit staging. + When using `--enable-warnings`, it was not possible to disable warnings + via CFLAGS that got explicitly enabled. Now warnings are not enabled + anymore if they are explicitly disabled (or enabled) in CFLAGS. This + works for at least GCC, clang, and TCC as they have corresponding + `-Wno-` options for every warning. + + Closes https://github.com/curl/curl/pull/5689 -Daniel Stenberg (5 Jun 2020) -- [Adnan Khan brought this change] +Daniel Stenberg (16 Jul 2020) +- ngtcp2: adjust to recent sockaddr updates + + Closes #5690 - HTTP3.md: clarify cargo build directory +- page-header: provide protocol details in the curl.1 man page - Cargo needs to be called from within the 'quiche' directory. + Add protocol and version specific information about all protocols curl + supports. - Closes #5522 + Fixes #5679 + Reported-by: tbugfinder on github + Closes #5686 -- user-agent.d: spell out what happens given a blank argument +Daniel Gustafsson (16 Jul 2020) +- docs: Update a few leftover mentions of DarwinSSL - Closes #5525 + Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the + more correct/common name Secure Transport, but a few mentions in the docs + remained. + + Closes #5688 + Reviewed-by: Daniel Stenberg -- trailers: switch h1-trailer logic to use dynbuf +Daniel Stenberg (16 Jul 2020) +- file2memory: use a define instead of -1 unsigned value - In the continued effort to remove "manual" realloc schemes. + ... to use the maximum value for 'size_t' when detecting integer overflow. + Changed the limit to max/4 as already that seems unreasonably large. - Closes #5524 + Codacy didn't like the previous approach. + + Closes #5683 -- CURLINFO_ACTIVESOCKET.3: clarify the description +- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream - Reported-by: Jay Satiro - Fixes #5299 - Closes #5520 + ... by adding support for a new dedicated return code. + + Suggested-by: Jonathan Cardoso + Assisted-by: Erik Johansson + URL: https://curl.haxx.se/mail/lib-2020-06/0099.html + Closes #5636 -- mailmap: Don J Olmstead +- [Baruch Siach brought this change] -- configure: only strip first -L from LDFLAGS + nss: fix build with disabled proxy support - In the logic that works out if a given OpenSSL path works, it stripped - off a possibly leading -L flag using an incorrect sed pattern which - would remove all instances of -L in the string, including if the path - itself contained that two-letter sequence! + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - The same pattern was used and is now updated in multiple places. Now it - only removes -L if it starts the strings. - - Reported-by: Mohamed Osama - Fixes #5519 - Closes #5521 + Closes #5667 -Peter Wu (4 Jun 2020) -- quiche: advertise draft 28 support +- test1139: make it display the difference on test failures + +- test1119: verify stdout in the test - Fix the verbose message while at it, quiche currently supports draft - 27 and draft 28 simultaneously. + So that failures will be displayed in the terminal, as it makes test failures + visually displayed easier and faster. - Closes #5518 + Closes #5644 -Daniel Stenberg (4 Jun 2020) -- KNOWN_BUGS: RTSP authentication breaks without redirect support +- curl: add %{method} to the -w variables - Closes #4750 + Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. + + Added test 1197 to verify. -Jay Satiro (4 Jun 2020) -- projects: Add crypt32.lib to dependencies for all OpenSSL configs +- CURLINFO_EFFECTIVE_METHOD: added - Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO - need crypt32. + Provide the HTTP method that was used on the latest request, which might + be relevant for users when there was one or more redirects involved. - Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. + Closes #5511 + +Viktor Szakats (14 Jul 2020) +- windows: add unicode to feature list - The changes that are in this commit were made by script. + Reviewed-by: Marcel Raad + Reviewed-by: Marc Hörsken - Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 + Closes #5491 + +Daniel Stenberg (14 Jul 2020) +- multi: remove two checks always true - Closes https://github.com/curl/curl/pull/5516 + Detected by Codacy + Closes #5676 -Marc Hoersken (3 Jun 2020) -- CI/macos: fix 'is already installed' errors by using bundle +Marc Hoersken (13 Jul 2020) +- workflows: limit what branches to run CodeQL on - Avoid failing CI builds due to nghttp2 being already installed. + Align CodeQL action with existing CI actions: + - Update branch filter to avoid duplicate CI runs. + - Shorten workflow name due to informative job name. - Closes #5513 + Reviewed-by: Daniel Stenberg + + Closes #5660 -Daniel Stenberg (3 Jun 2020) -- altsvc: fix 'dsthost' may be used uninitialized in this function +- appveyor: collect libcurl.dll variants with prefix or suffix + + On some platforms libcurl is build with a platform-specific + prefix and/or a version number suffix. + + Assisted-by: Jay Satiro + + Closes #5659 -- RELEASE-NOTES: synced +Daniel Stenberg (12 Jul 2020) +- [ihsinme brought this change] -- urldata: let the HTTP method be in the set.* struct - - When the method is updated inside libcurl we must still not change the - method as set by the user as then repeated transfers with that same - handle might not execute the same operation anymore! + socks: use size_t for size variable - This fixes the libcurl part of #5462 + Use the unsigned type (size_t) in the arithmetic of pointers. In this + context, the signed type (ssize_t) is used unnecessarily. - Test 1633 added to verify. + Authored-by: ihsinme on github + Closes #5654 + +- RELEASE-NOTES: synced - Closes #5499 + ... and bumped to 7.72.0 as the next release version number -- hostip: fix the memory-leak introduced in 67d2802 +- [Gilles Vollant brought this change] + + content_encoding: add zstd decoding support - Fixes #5503 - Closes #5504 + include zstd curl patch for Makefile.m32 from vszakats + and include Add CMake support for zstd from Peter Wu + + Helped-by: Viktor Szakats + Helped-by: Peter Wu + Closes #5453 -- test970: make it require proxy support +- asyn.h: remove the Curl_resolver_getsock define - This test verifies the -w %json output and the test case includes a full - generated "blob". If there's no proxy support built into libcurl, it - will return an error for proxy related info variables and they will not - be included in the json, thus causing a mismatch and this test fails. + - not used + - used the wrong number of arguments + - confused the Codeacy code analyzer - Reported-by: Marc Hörsken - Fixes #5501 - Closes #5502 + Closes #5647 -- [Radoslav Georgiev brought this change] +- [Nicolas Sterchele brought this change] - examples/http2-down/upload: add error checks - - If `index.html` does not exist in the directory from which the example - is invoked, the fopen(upload, "rb") invocation in `setup` would fail, - returning NULL. This value is subsequently passed as the FILE* argument - of the `fread` invocation in the `read_callback` function, which is the - actual cause of the crash (apparently `fread` assumes that argument to - be non-null). + configure.ac: Sort features name in summary - In addition, mitigate some possible crashes of similar origin. + - Same as protocols - Closes #5463 + Closes #5656 -- [kotoriのねこ brought this change] +- [Matthias Naegler brought this change] - examples/ephiperfifo: turn off interval when setting timerfd + cmake: fix windows xp build - Reported-by: therealhirudo on github - Fixes #5485 - Closes #5497 - -- [Saleem Abdulrasool brought this change] + Reviewed-by: Marcel Raad + Closes #5662 - vtls: repair the build with `CURL_DISABLE_PROXY` +- ngtcp2: update to modified qlog callback prototype - `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` - is enabled. Repair the build with that configuration. + Closes #5675 + +- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle - Follow-up to f3d501dc67 + Added test case 674 to reproduce and verify the bug report. - Closes #5498 + Fixes #5665 + Reported-by: NobodyXu on github + Closes #5673 -- transfer: remove k->str NULL check - - "Null-checking k->str suggests that it may be null, but it has already - been dereferenced on all paths leading to the check" - and it can't - legally be NULL at this point. Remove check. +- [Baruch Siach brought this change] + + bearssl: fix build with disabled proxy support - Detected by Coverity CID 1463884 + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - Closes #5495 + Reviewed-by: Nicolas Sterchele + Closes #5666 -Marc Hoersken (1 Jun 2020) -- select: always use Sleep in Curl_wait_ms on Win32 - - Since Win32 almost always will also have USE_WINSOCK, - we can reduce complexity and always use Sleep there. +- RELEASE-NOTES: synced + +Jay Satiro (11 Jul 2020) +- [Carlo Marcelo Arenas Belón brought this change] + + cirrus-ci: upgrade 11-STABLE to 11.4 - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Meant to be the last of the 11 series and so make sure that all + other references reflect all 11 versions so they can be retired + together later. - Follow up to #5343 - Closes #5489 + Closes https://github.com/curl/curl/pull/5668 -Daniel Stenberg (31 May 2020) -- conncache: download buffer needs +1 size for trailing zero +- [Filip Salomonsson brought this change] + + CURLINFO_CERTINFO.3: fix typo - Follow-up to c4e6968127e - Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 + Closes https://github.com/curl/curl/pull/5655 -Marc Hoersken (31 May 2020) -- azure: use matrix strategy to avoid configuration redundancy +Daniel Stenberg (4 Jul 2020) +- http2: only do the *done() cleanups for HTTP - This also includes the following changes: + Follow-up to ef86daf4d3 - - Use the same timeout for all jobs on Linux (60 minutes) - and Windows (90 minutes) - - Use CLI stable apt-get install -y instead of apt install - which warns about that and run apt-get update first - - Enable MQTT for Windows msys2 builds instead of - legacy msys1 builds - - Add ./configure --prefix parameter to the msys2 builds - - The MSYSTEM environment variable is now preset inside - the container images for the msys2 builds + Closes #5650 + Fixes #5646 + +- [Alex Kiernan brought this change] + + gnutls: repair the build with `CURL_DISABLE_PROXY` - Note: on Azure Pipelines the matrix strategy is basically - just a simple list of job copies and not really a matrix. + `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` + if `CURL_DISABLE_PROXY` is enabled. Repair the build with that + configuration. - Closes #5468 + Signed-off-by: Alex Kiernan + Closes #5645 -Daniel Stenberg (30 May 2020) -- build: disable more code/data when built without proxy support +Alex Kiernan (3 Jul 2020) +- gnutls: Fetch backend when using proxy - Added build to travis to verify - - Closes #5466 + Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") + Signed-off-by: Alex Kiernan -- url: alloc the download buffer at transfer start +Daniel Stenberg (3 Jul 2020) +- [Laramie Leavitt brought this change] + + http2: close the http2 connection when no more requests may be sent - ... and free it as soon as the transfer is done. It removes the extra - alloc when a new size is set with setopt() and reduces memory for unused - easy handles. + Well-behaving HTTP2 servers send two GOAWAY messages. The first + message is a warning that indicates that the server is going to + stop accepting streams. The second one actually closes the stream. - In addition: the closure_handle now doesn't use an allocated buffer at - all but the smallest supported size as a stack based one. + nghttp2 reports this state (and the other state of no more stream + identifiers) via the call nghttp2_session_check_request_allowed(). + In this state the client should not create more streams on the + session (tcp connection), and in curl this means that the server + has requested that the connection is closed. - Closes #5472 - -- timeouts: change millisecond timeouts to timediff_t from time_t + It would be also be possible to put the connclose() call into the + on_http2_frame_recv() function that triggers on the GOAWAY message. - For millisecond timers we like timediff_t better. Also, time_t can be - unsigned so returning a negative value doesn't work then. + This fixes a bug seen when the client sees the following sequence of + frames: - Closes #5479 - -Marc Hoersken (30 May 2020) -- select: add overflow checks for timeval conversions + // advisory GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] + ... some additional frames - Using time_t and suseconds_t if suseconds_t is available, - long on Windows (maybe others in the future) and int elsewhere. + // final GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] - Also handle case of ULONG_MAX being greater or equal to INFINITE. + Before this change, curl will attempt to reuse the connection even + after the last stream, will encounter this error: - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] + * Re-using existing connection! (#0) with host localhost + * Connected to localhost (::1) port 10443 (#0) + * Using Stream ID: 9 (easy handle 0x5595f0a72e30) + > GET /index.html?5 HTTP/2 + > Host: localhost:10443 + > user-agent: curl/7.68.0 + > accept: */* + > + * stopped the pause stream! + * Connection #0 to host localhost left intact + curl: (16) Error in the HTTP2 framing layer - Part of #5343 - -- select: use timediff_t instead of time_t and int for timeout_ms + This error may posion the connection cache, causing future requests + which resolve to the same curl connection to go through the same error + path. - Make all functions in select.[ch] take timeout_ms as timediff_t - which should always be large enough and signed on all platforms - to take all possible timeout values and avoid type conversions. + Closes #5643 + +- ftpserver: don't verify SMTP MAIL FROM names - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Rely on tests asking the names to get refused instead - test servers + should be as dumb as possible. Edited test 914, 955 and 959 accordingly. - Replaces #5107 and partially #5262 - Related to #5240 and #5286 - Closes #5343 + Closes #5639 -- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' +- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated - GCC 10 warns about this with warning: implicit conversion - from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] + This came up in #5640. It make sense to clarify this in the docs! - Since 'expected_result' is not really of type 'CURLcode' and - it is not exposed in any way, we can just use 'SANITIZEcode'. + Reminded-by: Kamil Dudka + Closes #5642 + +Kamil Dudka (3 Jul 2020) +- tool_getparam: make --krb option work again - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad + It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. - Closes #5476 + Bug: https://bugzilla.redhat.com/1833193 + Closes #5640 -- tests/libtest: fix undefined reference to 'curlx_win32_fopen' +Daniel Stenberg (2 Jul 2020) +- [Jeremy Maitin-Shepard brought this change] + + http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages - Since curl_setup.h now makes use of curlx_win32_fopen for Win32 - builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, - we need to include the relevant files for tests using fopen, - because the libtest sources are also including curl_setup.h + Confusingly, nghttp2 has two different error code enums: - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg + - nghttp2_error, to be used with nghttp2_strerror + - nghttp2_error_code, to be used with nghttp2_http2_strerror - Follow up to #3784 (ffdddb45d9) - Closes #5475 + Closes #5641 -- appveyor: add non-debug plain autotools-based build +Marcel Raad (2 Jul 2020) +- url: silence MSVC warning - This should enable us to catch linking issues with the - testsuite early, like the one described/fixed in #5475. + Since commit f3d501dc678, if proxy support is disabled, MSVC warns: + url.c : warning C4701: potentially uninitialized local variable + 'hostaddr' used + url.c : error C4703: potentially uninitialized local pointer variable + 'hostaddr' used - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad + That could actually only happen if both `conn->bits.proxy` and + `CURL_DISABLE_PROXY` were enabled. + Initialize it to NULL to silence the warning. - Closes #5477 + Closes https://github.com/curl/curl/pull/5638 -Daniel Stenberg (29 May 2020) +Daniel Stenberg (1 Jul 2020) - RELEASE-NOTES: synced -- Revert "buildconf: use find -execdir" +Version 7.71.1 (30 Jun 2020) + +Daniel Stenberg (30 Jun 2020) +- RELEASE-NOTES: curl 7.71.1 + +- THANKS: add contributors to 7.71.1 + +- scripts/copyright.pl: skip .dcignore + +- Revert "multi: implement wait using winsock events" - This partially reverts commit c712009838f44211958854de431315586995bc61. + This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. - Keep the ares_ files removed but bring back the older way to run find, - to make it work with busybox's find, as apparently that's being used. + That commit (from #5397) introduced a regression in 7.71.0. - Reported-by: Max Peal - Fixes #5483 - Closes #5484 + Reported-by: tmkk on github + Fixes #5631 + Closes #5632 -- server/sws: fix asan warning on use of uninitialized variable +- TODO: Add flag to specify download directory -- libssh2: improved error output for wrong quote syntax - - Reported-by: Werner Stolz - - Closes #5474 +- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection -- mk-lib1521: generate code for testing BLOB options as well +- cirrus-ci: disable FreeBSD 13 (again) - Follow-up to cac5374298b3 + It has been failing for a good while again. This time we better leave it + disabled until we have more reason to believe it behaves. - Closes #5478 + Closes #5628 -- configure: repair the check if argv can be written to - - Due to bad escaping of the test code, the test wouldn't build and thus - result in a negative test result, which would lead to the unconditional - assumption that overwriting the arguments doesn't work and thus curl - would never hide credentials given in the command line, even when it - would otherwise be possible. +- ngtcp2: sync with current master - Regression from commit 2d4c2152c (7.60.0) + ngtcp2 added two new callbacks - Reported-by: huzunhao on github - Fixes #5470 - Closes #5471 + Reported-by: Lucien Zürcher + Fixes #5624 + Closes #5627 -Peter Wu (28 May 2020) -- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes +- examples/multithread.c: call curl_global_cleanup() - Otherwise the build might fail due to missing source files, as - demonstrated by the recent keylog.c addition on an existing build dir. - - Closes #5469 + Reported-by: qiandu2006 on github + Fixes #5622 + Closes #5623 -Daniel Stenberg (28 May 2020) -- urldata: fix comments: Curl_done() is called multi_done() now +- vtls: compare cert blob when finding a connection to reuse - ... since 575e885db + Reported-by: Gergely Nagy + Fixes #5617 + Closes #5619 -Peter Wu (27 May 2020) -- ngtcp2: use common key log routine for better thread-safety +- RELEASE-NOTES: synced + +- terminology: call them null-terminated strings - Tested with ngtcp2 built against the OpenSSL library. Additionally - tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + Updated terminology in docs, comments and phrases to refer to C strings + as "null-terminated". Done to unify with how most other C oriented docs + refer of them and what users in general seem to prefer (based on a + single highly unscientific poll on twitter). - The TLS backend (independent of QUIC) may or may not already have opened - the keylog file before. Therefore Curl_tls_keylog_open is always called - to ensure the file is open. + Reported-by: coinhubs on github + Fixes #5598 + Closes #5608 -- wolfssl: add SSLKEYLOGFILE support +- http: fix proxy auth with blank password - Tested following the same curl and tshark commands as in commit - "vtls: Extract and simplify key log file handling from OpenSSL" using - WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with - `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. + Regression in 7.71.0 - Full support for this feature requires certain wolfSSL build options, - see "Availability note" in lib/vtls/wolfssl.c for details. + Added test case 346 to verify. - Closes #5327 + Reported-by: Kristoffer Gleditsch + Fixes #5613 + Closes #5616 -- vtls: Extract and simplify key log file handling from OpenSSL - - Create a set of routines for TLS key log file handling to enable reuse - with other TLS backends. Simplify the OpenSSL backend as follows: - - - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. - - Do not perform dynamic memory allocation when preparing a log entry. - Unless the TLS specifications change we can suffice with a reasonable - fixed-size buffer. - - Simplify state tracking when SSL_CTX_set_keylog_callback is - unavailable. My original sslkeylog.c code included this tracking in - order to handle multiple calls to SSL_connect and detect new keys - after renegotiation (via SSL_read/SSL_write). For curl however we can - be sure that a single master secret eventually becomes available - after SSL_connect, so a simple flag is sufficient. An alternative to - the flag is examining SSL_state(), but this seems more complex and is - not pursued. Capturing keys after server renegotiation was already - unsupported in curl and remains unsupported. +- .dcignore: ignore tests and docs directories - Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f - (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) - against an OpenSSL 1.1.1f server configured with: + This is a config file for deepcode.ai, a static code analyzer. + +Jay Satiro (26 Jun 2020) +- tool_cb_hdr: Fix etag warning output and return code - # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 - openssl s_server -www -tls1 - # Likewise, but fail the server handshake. - openssl s_server -www -tls1 -Verify 2 - # TLS 1.3 test. No need to test the failing server handshake. - openssl s_server -www -tls1_3 + - Return 'failure' on failure, to follow the existing style. - Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly - written using Wireshark. For the first and third case, expect four - matches per connection (decrypted Server Finished, Client Finished, HTTP - Request, HTTP Response). For the second case where the handshake fails, - expect a decrypted Server Finished only. + - Put Warning: and the warning message on the same line. - tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ - -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ - -dtls.port==4433,http -ohttp.desegment_body:FALSE \ - -Y 'tls.handshake.verify_data or http' + Ref: https://github.com/curl/curl/issues/5610 - A single connection can easily be identified via the `tcp.stream` field. - -Daniel Stenberg (27 May 2020) -- FILEFORMAT: add more features that tests can depend on + Closes https://github.com/curl/curl/pull/5612 -- [Michael Kaufmann brought this change] +Daniel Stenberg (26 Jun 2020) +- CURLOPT_READFUNCTION.3: provide the upload data size up front + + Assisted-by: Jay Satiro + Closes #5607 - transfer: close connection after excess data has been read +- test1539: do a HTTP 1.0 POST without a set size (fails) - For HTTP 1.x, it's a protocol error when the server sends more bytes - than announced. If this happens, don't reuse the connection, because the - start position of the next response is undefined. + Attempt to reproduce #5593. Test case 1514 is very similar but uses + HTTP/1.1 and thus switches to chunked. - Closes #5440 + Closes #5595 -- [Estanislau Augé-Pujadas brought this change] +- [Baruch Siach brought this change] - Revert "ssh: ignore timeouts during disconnect" + mbedtls: fix build with disabled proxy support - This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in - curl 7.54.1. + Don't reference fields that do not exist. Fixes build failure: - Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html - Closes #5465 + vtls/mbedtls.c: In function 'mbed_connect_step1': + vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' + + Closes #5615 -- urldata: connect related booleans live in struct ConnectBits +- codeql-analysis.yml: fix the 'languages' setting - And remove a few unused booleans! + It needs a 'with:' in front of it. + +GitHub (26 Jun 2020) +- [Daniel Stenberg brought this change] + + gtihub: codeql-analysis.yml - Closes #5461 + enables code security scanning with github actions -- hostip: on macOS avoid DoH when given a numerical IP address +Daniel Stenberg (25 Jun 2020) +- tests: verify newline in username and password for HTTP - When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that - numerical IP addresses still need to get "resolved" - but not with DoH. + test 1296 is a simply command line test - Reported-by: Viktor Szakats - Fixes #5454 - Closes #5459 + test 1910 is a libcurl test including a redirect -- ngtcp2: cleanup memory when failing to connect +- url: allow user + password to contain "control codes" for HTTP(S) - Reported-by: Peter Wu - Fixes #5447 (the ngtcp2 side of it) - Closes #5451 + Reported-by: Jon Johnson Jr + Fixes #5582 + Closes #5592 -- quiche: clean up memory properly when failing to connect +- escape: make the URL decode able to reject only %00 bytes - Addresses the quiche side of #5447 - Reported-by: Peter Wu - Closes #5450 - -- cleanup: use a single space after equals sign in assignments - -- url: accept "any length" credentials for proxy auth + ... or all "control codes" or nothing. - They're only limited to the maximum string input restrictions, not to - 256 bytes. + Assisted-by: Nicolas Sterchele + +- http2: set the correct URL in pushed transfers - Added test 1178 to verify + ...previously CURLINFO_EFFECTIVE_URL would report the URL of the + original "mother transfer", not the actually pushed resource. - Reported-by: Will Roberts - Fixes #5448 - Closes #5449 + Reported-by: Jonathan Cardoso Machado + Fixes #5589 + Closes #5591 -- [Maksim Stsepanenka brought this change] +Jay Satiro (25 Jun 2020) +- [Javier Blazquez brought this change] - test1167: fixes in badsymbols.pl + openssl: Fix compilation on Windows when ngtcp2 is enabled - Closes #5442 + - Include wincrypt before OpenSSL includes so that the latter can + properly handle any conflicts between the two. + + Closes https://github.com/curl/curl/pull/5606 -- altsvc: fix parser for lines ending with CRLF +Daniel Stenberg (25 Jun 2020) +- test543: extended to verify zero length input - Fixed the alt-svc parser to treat a newline as end of line. + As was reported in #5601 + +- escape: zero length input should return a zero length output - The unit tests in test 1654 were done without CRLF and thus didn't quite - match the real world. Now they use CRLF as well. + Regression added in 7.71.0. - Reported-by: Peter Wu - Assisted-by: Peter Wu - Assisted-by: Jay Satiro - Fixes #5445 - Closes #5446 + Fixes #5601 + Reported-by: Kristoffer Gleditsch + Closes #5602 -Viktor Szakats (25 May 2020) -- all: fix codespell errors +- Curl_inet_ntop: always check the return code - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5452 + Reported-by: Siva Sivaraman + Fixes #5412 + Closes #5597 -Peter Wu (25 May 2020) -- ngtcp2: fix build with current ngtcp2 master implementing draft 28 +- sendf: improve the message on client write errors - Based on client.cc changes from ngtcp2. Tested with current git master, - ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. + Replace "Failed writing body (X != Y)" with + "Failure writing output to destination". Possibly slightly less cryptic. - Fixes #5444 - Closes #5443 + Reported-by: coinhubs on github + Fixes #5594 + Closes #5596 -Daniel Stenberg (25 May 2020) - RELEASE-NOTES: synced - - moved the new setopts up to a "change" -- RELEASE-NOTES: synced +- curlver: start working on 7.71.1 -- copyright: updated year ranges out of sync +- [Denis Baručić brought this change] + + DYNBUF.md: fix a typo: trail => tail - ... and whitelisted a few more files in the the copyright.pl script. + Closes #5599 -- [Gilles Vollant brought this change] +Version 7.71.0 (23 Jun 2020) - setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency - - Closes #5431 +Daniel Stenberg (23 Jun 2020) +- RELEASE-NOTES: curl 7.71.0 release -- curl: remove -J "informational" written on stdout +- THANKS: curl 7.71.0 additions + +- url: make sure pushed streams get an allocated download buffer - curl would previously show "curl: Saved to filename 'name from header'" - if -J was used and a name was picked from the Content-Disposition - header. That output could interfer with other stdout output, such as -w. + Follow-up to c4e6968127e876b0 - This commit removes that output line. - Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html - Reported-by: Коваленко Анатолий Викторович - Closes #5435 + When a new transfer is created, as a resuly of an acknowledged push, + that transfer needs a download buffer allocated. + + Closes #5590 -Peter Wu (22 May 2020) -- travis: simplify quiche build instructions wrt boringssl +Jay Satiro (22 Jun 2020) +- openssl: Don't ignore CA paths when using Windows CA store - quiche builds boringssl as static library, reuse that instead of - building another shared library. + This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does + not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default + locations. Instead the CA store can now be used at the same time. - Closes #5438 - -- configure: fix pthread check with static boringssl + The change is due to the impending release. The issue is still being + discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and + is now documented as experimental. - A shared boringssl/OpenSSL library requires -lcrypto only for linking. - A static build additionally requires `-ldl -lpthread`. In the latter - case `-lpthread` is added to LIBS which prevented `-pthread` from being - added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. + Ref: bc052cc (parent commit) + Ref: https://github.com/curl/curl/issues/5585 -Daniel Stenberg (22 May 2020) -- Revert "sendf: make failf() use the mvsnprintf() return code" +- tool_operate: Don't use Windows CA store as a fallback - This reverts commit 74623551f306990e70c7c5515b88972005604a74. + Background: - Instead mark the function call with (void). Getting the return code and - using it instead triggered Coverity warning CID 1463596 because - snprintf() can return a negative value... + 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate + store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides + CURLOPT_CAINFO if both are set. The curl tool will fall back to + CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set + via CURLOPT_CAINFO. - Closes #5441 - -- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' + Problem: - Reported-by: Billyzou0741326 on github - Fixes #5432 - Closes #5436 - -- tests/server/util.h: add extern to silence compiler warning + libcurl may be built with hardcoded paths to a certificate bundle or + directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are + ignored. - Follow-up from a3b0699d5c1 + Solution: + + A solution is still being discussed but since there's an impending + release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + + Ref: https://github.com/curl/curl/issues/5585 -- typecheck-gcc.h: fix the OFF_T check +- openssl: Fix CA fallback logic for OpenSSL 3.0 build - The option number also needs to be less than CURLOPTTYPE_BLOB. + Prior to this change I assume a build error would occur when + CURL_CA_FALLBACK was used. - Follow-up to cac5374298 - Reported-by: Jeroen Ooms - Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 + Closes https://github.com/curl/curl/pull/5587 -- TODO: --dry-run +Daniel Stenberg (22 Jun 2020) +- copyright: update mismatched copyright years + +- test1460: verify that -Ji is not ok + +- tool_getparam: -i is not OK if -J is used - Closes #5426 + Reported-by: sn on hackerone + Bug: https://curl.haxx.se/docs/CVE-2020-8177.html -- TODO: Ratelimit or wait between serial requests +- [Peter Wu brought this change] + + CMake: ignore INTERFACE_LIBRARY targets for pkg-config file - Closes #5406 + Reviewed-by: Marcel Raad + Fixes #5512 + Closes #5517 -- tool_paramhlp: fixup C89 mistake +- [Valentyn Korniienko brought this change] + + multibyte: Fixed access-> waccess to file for Windows Plarform - Follow-up to c5f0a9db22. + Reviewed-by: Marcel Raad + Closes #5580 -- [Siva Sivaraman brought this change] +- altsvc: bump to h3-29 + + Closes #5584 - tool_paramhlp: fixed potentially uninitialized strtol() variable +- urlglob: treat literal IPv6 addresses with zone IDs as a host name - Seems highly unlikely to actually be possible, but better safe than - sorry. + ... and not as a "glob". Now done by passing the supposed host to the + URL parser which supposedly will do a better job at identifying "real" + numerical IPv6 addresses. - Closes #5417 + Reported-by: puckipedia on github + Fixes #5576 + Closes #5579 -- [Siva Sivaraman brought this change] +- test1179: verify error message for non-existing cmdline option - tool_operate: fixed potentially uninitialized variables +- tool_getparam: repair the error message for unknown flag - ... in curl_easy_getinfo() calls. They're harmless but clearing the - variables makes the code safer and comforts the reader. + Follow-up to 9e5669f3880674 + Detected by Coverity CID 1464582 ("Logically dead code") - Closes #5416 + Closes #5577 -- sha256: move assign to the declaration line +- FILEFORMAT: describe verify/stderr + +- connect: improve happy eyeballs handling - Follow-up to fae30656. Should've been squashed with that commit... + For QUIC but also for regular TCP when the second family runs out of IPs + with a failure while the first family is still trying to connect. + + Separated the timeout handling for IPv4 and IPv6 connections when they + both have a number of addresses to iterate over. -- [Siva Sivaraman brought this change] +- ngtcp2: never call fprintf() in lib code in release version - sha256: fixed potentially uninitialized variable +- ngtcp2: fix happy eyeballs quic connect crash - Closes #5414 + Reported-by: Peter Wu + Fixes #5565 + Closes #5568 -- sendf: make failf() use the mvsnprintf() return code - - ... and avoid a strlen() call. Fixes a MonocleAI warning. +- select: remove the unused ELAPSED_MS() macro - Reported-by: MonocleAI - Fixes #5413 - Closes #5420 + Closes #5573 -- hostip: make Curl_printable_address not return anything +Marc Hoersken (17 Jun 2020) +- [rcombs brought this change] + + multi: implement wait using winsock events - It was not used much anyway and instead we let it store a blank buffer - in case of failure. + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. - Reported-by: MonocleAI - Fixes #5411 - Closes #5418 - -- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) + Reviewed-by: Gergely Nagy + Reviewed-by: Marc Hörsken - They're done on purpose, make that visible in the code. - Reported-by: MonocleAI - Fixes #5412 - Closes #549 + Closes #5397 -- TODO: forbid TLS post-handshake auth and do TLS record padding +Daniel Stenberg (17 Jun 2020) +- manpage: add three missing environment variables - Closes #5396 - Closes #5398 + CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE + + Closes #5571 - RELEASE-NOTES: synced -- dynbuf: return NULL when there's no buffer length - - ... as returning a "" is not a good idea as the string is supposed to be - allocated and returning a const string will cause issues. +- configure: for wolfSSL, check for the DES func needed for NTLM - Reported-by: Brian Carpenter - Follow-up to ed35d6590e72c - Closes #5405 + Also adds pkg-config support for the wolfSSL detection. -Peter Wu (16 May 2020) -- travis: upgrade to bionic, clang-9, improve readability - - Changes, partially to reduce build failures from external dependencies: - - Upgrade Ubuntu and drop unnecessary third-party repos. - - Properly clone apt config to ensure retries. - - Upgrade to clang-9 from the standard repos. - - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey - fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use - focal everywhere yet since Travis CI has not documented this option. - In focal, python-impacket (Py2.7) has been removed, leaving only - python3-impacket. Since it is only needed for SMB tests and not SSH, - skip it for the libssh job since it might need more work. - - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. +- [Ruurd Beerstra brought this change] + + ntlm: enable NTLM support with wolfSSL - Non-functional cleanups: - - Simplify test matrix, drop redundant os and compiler keys. - - Deprecation fixes: remove sudo, rename matrix -> jobs. - - Every job has an 'env' key, put this key first in a list item. + When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* + functions that OpenSSL has. This change take advantage of that. - Closes #5370 + Co-authored-by: Daniel Stenberg + Closes #5556 + Fixes #5548 -- travis: whitespace-only changes for consistency +- http: move header storage to Curl_easy from connectdata - Automatically apply a consistent indentation with: + Since the connection can be used by many independent requests (using + HTTP/2 or HTTP/3), things like user-agent and other transfer-specific + data MUST NOT be kept connection oriented as it could lead to requests + getting the wrong string for their requests. This struct data was + lingering like this due to old HTTP1 legacy thinking where it didn't + mattered.. - python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' + Fixes #5566 + Closes #5567 + +- CODE_REVIEW.md: how to do code reviews in curl - followed by manually re-indenting three comments. + Assisted-by: Daniel Gustafsson + Assisted-by: Rich Salz + Assisted-by: Hugo van Kemenade + Assisted-by: James Fuller + Assisted-by: Marc Hörsken + Assisted-by: Jay Satiro - Closes #5370 + Closes #5555 -- CMake: add libssh build support +- altsvc: remove the num field from the altsvc struct - Closes #5372 - -Daniel Stenberg (15 May 2020) -- KNOWN_BUGS: wolfssh: publickey auth doesn't work + It was superfluous since we have the list.size alredy - Closes #4820 + Reported-by: Jay Satiro + Fixes #5553 + Closes #5563 -- KNOWN_BUGS: OS400 port requires deprecated IBM library +- version.d: expanded and alpha-sorted - Closes #5176 - -- [Vyron Tsingaras brought this change] - - http2: keep trying to send pending frames after req.upload_done + Added a few missing features not previously mentioned. Ordered them + alphabetically. - Fixes #1410 - Closes #5401 - -- [Gilles Vollant brought this change] + Closes #5558 - setopt: support certificate options in memory with struct curl_blob +- ABI.md: rename to .md and polish the markdown - This change introduces a generic way to provide binary data in setopt - options, called BLOBs. - - This change introduces these new setopts: - - CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, - CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. - - Reviewed-by: Daniel Stenberg - Closes #5357 + Closes #5562 -- source cleanup: remove all custom typedef structs +- HELP-US: add a section for "smaller tasks" - - Stick to a single unified way to use structs - - Make checksrc complain on 'typedef struct {' - - Allow them in tests, public headers and examples + The point of this section is to meet the CII Best Practices gold level + critera: - - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually - typedef different types/structs depending on build conditions. + "The project MUST clearly identify small tasks that can be performed by + new or casual contributors" - Closes #5338 - -- travis: remove the .checksrc fiddling + Closes #5560 -- ftp: make domore_getsock() return the secondary socket properly +- TODO: retry on the redirected-to URL - Previously, after PASV and immediately after the data connection has - connected, the function would only return the control socket to wait for - which then made the data connection simply timeout and not get polled - correctly. This become obvious when running test 1631 and 1632 event- - based. + Closes #5462 -- test1632: verify FTP through HTTPS-proxy with connection re-use +- mailmap: Nicolas Sterchele -- test1631: verify FTP download through HTTPS-proxy +- [Nicolas Sterchele brought this change] -- sws: as last resort, get test number from server cmd file + TODO: remove 19.3 section title - If it can't be found in the request. Also support --cmdfile to set it to - a custom file name. + Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on + curl documentation website - runtests.pl always writes this file with the test number in it since a - while back. + Closes #5561 -- ftp: shut down the secondary connection properly when SSL is used - - Reported-by: Neal Poole - Fixes #5340 - Closes #5385 +- [Martin V brought this change] -Marcel Raad (14 May 2020) -- KNOWN_BUGS: adapt 5.5 to recent changes - - It only applies to non-Unicode builds now. - Also merge 5.10 into it as it's effectively a duplicate. + test1560: avoid possibly negative association in wording - Closes https://github.com/curl/curl/pull/3784 + Closes #5549 -- curl_setup: support Unicode functions to open files on Windows +- share: don't set the share flag it something fails - Use them only if `_UNICODE` is defined, in which case command-line - arguments have been converted to UTF-8. + When asking for a specific feature to be shared in the share object, + that bit was previously set unconditionally even if the shared feature + failed or otherwise wouldn't work. - Closes https://github.com/curl/curl/pull/3784 + Closes #5554 -- tool: support UTF-16 command line on Windows - - - use `wmain` instead of `main` when `_UNICODE` is defined [0] - - define `argv_item_t` as `wchar_t *` in this case - - use the curl_multibyte gear to convert the command-line arguments to - UTF-8 +- buildconf: remove -print from the find command that removes files - This makes it possible to pass parameters with characters outside of - the current locale on Windows, which is required for some tests, e.g. - the IDN tests. Out of the box, this currently only works with the - Visual Studio project files, which default to Unicode, and winbuild - with the `ENABLE_UNICODE` option. + It's just too annoying and unnecessary to get a long list of files shown + +- RELEASE-NOTES: synced + +- wording: avoid blacklist/whitelist stereotypes - [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + Instead of discussing if there's value or meaning (implied or not) in + the colors, let's use words without the same possibly negative + associations. - Ref: https://github.com/curl/curl/issues/3747 - Closes https://github.com/curl/curl/pull/3784 + Closes #5546 -- curl_multibyte: add to curlx +Jay Satiro (9 Jun 2020) +- tool_getparam: fix memory leak in parse_args - This will also be needed in the tool and tests. + Prior to this change in Windows Unicode builds most parsed options would + not be freed. - Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 - Closes https://github.com/curl/curl/pull/3784 - -Daniel Stenberg (14 May 2020) -- url: make the updated credentials URL-encoded in the URL + Found using _CrtDumpMemoryLeaks(). - Found-by: Gregory Jefferis - Reported-by: Jeroen Ooms - Added test 1168 to verify. Bug spotted when doing a redirect. - Bug: https://github.com/jeroen/curl/issues/224 - Closes #5400 + Ref: https://github.com/curl/curl/issues/5545 -- tests: add https-proxy support to the test suite +Daniel Stenberg (8 Jun 2020) +- socks: detect connection close during handshake - Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like - HTTP proxy but with a full TLS connection to the proxy. + The SOCKS4/5 state machines weren't properly terminated when the proxy + connection got closed, leading to a busy-loop. - Closes #5399 - -- mailmap: James Fuller + Reported-By: zloi-user on github + Fixes #5532 + Closes #5542 -- [Major_Tom brought this change] +- [James Fuller brought this change] - vauth/cleartext: fix theoretical integer overflow - - Fix theoretical integer overflow in Curl_auth_create_plain_message. - - The security impact of the overflow was discussed on hackerone. We - agreed this is more of a theoretical vulnerability, as the integer - overflow would only be triggerable on systems using 32-bits size_t with - over 4GB of available memory space for the process. + multi: add defensive check on data->multi->num_alive - Closes #5391 + Closes #5540 -Jay Satiro (13 May 2020) -- curl.1: Quote globbed URLs - - - Quote the globbing example URLs that contain characters [] {} since - otherwise they may be interpreted as shell metacharacters. +- Curl_addrinfo: use one malloc instead of three - Bug: https://github.com/curl/curl/issues/5388 - Reported-by: John Simpson + To reduce the amount of allocations needed for creating a Curl_addrinfo + struct, make a single larger malloc instead of three separate smaller + ones. - Closes https://github.com/curl/curl/pull/5394 + Closes #5533 -Daniel Stenberg (14 May 2020) -- checksrc: enhance the ASTERISKSPACE and update code accordingly - - Fine: "struct hello *world" - - Not fine: "struct hello* world" (and variations) - - Closes #5386 +- [Alessandro Ghedini brought this change] -- docs/options-in-versions: which version added each cmdline option + quiche: update SSLKEYLOGFILE support - Added test 971 to verify that the list is in sync with the files in - cmdline-opts. The check also verifies that .d-files that uses Added: - specify the same version number as the options-in-versions file does. + quiche now requires the application to explicitly set the keylog path + for each connection, rather than reading the environment variable + itself. - Closes #5381 + Closes #5541 -- docs: unify protocol lists +- tests: add two simple tests for --login-options - We boast support for 25 transfer protocols. Make sure the lists are - consistent + Test 895 and 896 - as a follow-up to a3e972313b - Closes #5384 + Closes #5539 -- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN +- ngtcp2: update with recent API changes - ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. + Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. - Reported-by: Michael Kaufmann - Fixes #5374 - Closes #5376 - -- tls13-ciphers.d: shorten the Arg + Assisted-by: Tatsuhiro Tsujikawa + Closes #5538 -- sasl-authzid.d: add Arg: and shorten the desc +- [James Fuller brought this change] -- cert-type.d: mention the available types in the desc + socks: remove unreachable breaks in socks.c and mime.c + + Closes #5537 -- tool: shorten 3 --help descriptions +- tool_cfgable: free login_options at exit - --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort + Memory leak + Reported-by: Geeknik Labs + Fixes #5535 + Closes #5536 + +- libssh2: keep sftp errors as 'unsigned long' - gen.pl already warned about these lines but we didn't listen + Remove weird work-around for storing the SFTP errors as int instead of + the "unsigned long" that libssh2 actually returns for SFTP errors. - Closes #5379 + Closes #5534 -- configure: the wolfssh backend does not provide SCP +Marc Hoersken (6 Jun 2020) +- timeouts: move ms timeouts to timediff_t from int and long - Closes #5387 - -- RELEASE-NOTES: synced - -- url: reject too long input when parsing credentials + Now that all functions in select.[ch] take timediff_t instead + of the limited int or long, we can remove type conversions + and related preprocessor checks to silence compiler warnings. - Since input passed to libcurl with CURLOPT_USERPWD and - CURLOPT_PROXYUSERPWD circumvents the regular string length check we have - in Curl_setstropt(), the input length limit is enforced in - Curl_parse_login_details too, separately. + Avoiding conversions from time_t was already done in 842f73de. - Reported-by: Thomas Bouzerar - Closes #5383 + Based upon #5262 + Supersedes #5214, #5220 and #5221 + Follow up to #5343 and #5479 + Closes #5490 -- list-only.d: this option existed already in 4.0 +Daniel Stenberg (6 Jun 2020) +- [François Rigault brought this change] -Jay Satiro (12 May 2020) -- retry-all-errors.d: Shorten the summary line + openssl: set FLAG_TRUSTED_FIRST unconditionally - Follow-up to b995bb5 from a few moments ago. + On some systems, openssl 1.0.x is still the default, but it has been + patched to contain all the recent security fixes. As a result of this + patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be + defined, while the previous behavior of openssl to not look at trusted + chains first, remains. - Reported-by: Daniel Stenberg + Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to + probe for the behavior of openssl based on the existence ofmacros. - Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 - -- [denzor brought this change] + Closes #5530 - easy: fix dangling pointer on easy_perform fail +- server/util: fix logmsg format using curl_off_t argument - Closes https://github.com/curl/curl/pull/5363 - -- tool: Add option --retry-all-errors to retry on any error + ... this caused segfaults on armv7. - The "sledgehammer" of retrying. + Regression added in dd0365d560aea5a (7.70.0) - Closes https://github.com/curl/curl/pull/5185 + Reviewed-by: Jay Satiro + Closes #5529 -Daniel Stenberg (12 May 2020) -- [James Le Cuirot brought this change] +- RELEASE-NOTES: synced - libcurl.pc: Merge Libs.private into Libs for static-only builds - - A project being built entirely statically will call pkg-config with - --static, which utilises the Libs.private field. Conversely it will - not use --static when not being built entirely statically, even if - there is only a static build of libcurl available. This will most - likely cause the build to fail due to underlinking unless we merge the - Libs fields. +- [Cherish98 brought this change] + + socks: fix expected length of SOCKS5 reply - Consider that this is what the Meson build system does when it - generates pkg-config files. + Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the + reply ATYP is X'01'. This resulted in erroneously expecting more bytes + when the request length is greater than the reply length (e.g., when + remotely resolving the hostname). - I have also reflected this in the --libs argument of curl-config even - though REQUIRE_LIB_DEPS always seems to be "yes" anyway. + Closes #5527 + +Marc Hoersken (5 Jun 2020) +- .gitignore: add directory containing the stats repo - Closes #5373 + Since the new curl/stats repository is designed to be + checked out into the curl repository working tree as stats/ + it should be on the ignore list to aid in commit staging. -- [Peter Wu brought this change] +Daniel Stenberg (5 Jun 2020) +- [Adnan Khan brought this change] - CMake: fix runtests.pl with CMake, add new test targets + HTTP3.md: clarify cargo build directory - * runtests.pl: - - Fix out-of-tree build under CMake when srcdir is not set. Default - srcdir to the location of runtests.pl. - - Add a hack to allow CMake to use the TFLAGS option as documented - in tests/README and used in scripts/travis/script.sh. - * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie - support (no one should care, it is already EOL.). - * Remove CTest since it defines its own 'test' target with no tests - since all unittests are already broken and not built by default. - * Add new test targets based on the options from Makefile.am. Since - new test targets are rarely added, I opted for duplicating the - runtests.pl options as opposed to creating a new Makefile.inc file. - Use top-level target names (test-x) instead of x-test since that is - used by CI and others. + Cargo needs to be called from within the 'quiche' directory. - Closes #5358 + Closes #5522 -- [Peter Wu brought this change] +- user-agent.d: spell out what happens given a blank argument + + Closes #5525 - CMake: do not build test programs by default +- trailers: switch h1-trailer logic to use dynbuf - The default target should only build libcurl and curl. Add a dedicated - 'testdeps' target which will be used later when running tests. Note that - unittests are currently broken in CMake and already excluded. + In the continued effort to remove "manual" realloc schemes. - Closes #5368 + Closes #5524 -- FILEFORMAT: moved up the variables section and further polished +- CURLINFO_ACTIVESOCKET.3: clarify the description + + Reported-by: Jay Satiro + Fixes #5299 + Closes #5520 -- runtests: remove ftp2 support, not used +- mailmap: Don J Olmstead + +- configure: only strip first -L from LDFLAGS - We once supported two separate ftp instances in the test suite. Has not - been used the last decade. + In the logic that works out if a given OpenSSL path works, it stripped + off a possibly leading -L flag using an incorrect sed pattern which + would remove all instances of -L in the string, including if the path + itself contained that two-letter sequence! - Closes #5375 + The same pattern was used and is now updated in multiple places. Now it + only removes -L if it starts the strings. + + Reported-by: Mohamed Osama + Fixes #5519 + Closes #5521 -- url: sort the protocol schemes in rough popularity order +Peter Wu (4 Jun 2020) +- quiche: advertise draft 28 support - When looking for a protocol match among supported schemes, check the - most "popular" schemes first. It has zero functionality difference and - for all practical purposes a speed difference will not be measureable - but it still think it makes sense to put the least likely matches last. + Fix the verbose message while at it, quiche currently supports draft + 27 and draft 28 simultaneously. - "Popularity" based on the 2019 user survey. + Closes #5518 + +Daniel Stenberg (4 Jun 2020) +- KNOWN_BUGS: RTSP authentication breaks without redirect support - Closes #5377 + Closes #4750 -Marc Hoersken (11 May 2020) -- test1238: avoid tftpd being busy for tests shortly following +Jay Satiro (4 Jun 2020) +- projects: Add crypt32.lib to dependencies for all OpenSSL configs - The tftpd server may still be busy if the total timeout of - 25 seconds has not been reached or no sread error was received - during or after the execution of the timeout test 1238. + Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO + need crypt32. - Once the next TFTP test comes around (eg. 1242 or 1243), - those will fail because the tftpd server is still waiting - on data from curl due to the UDP protocol being stateless - and having no connection close. On Linux this error may not - happen, because ICMP errors generated due to a swrite error - can also be returned async on the next sread call instead. + Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. - Therefore we will now just kill the tftpd server after test - 1238 to make sure that the following tests are not affected. + The changes that are in this commit were made by script. - This enables us to no longer ignore tests 1242, 1243, 2002 - and 2003 on the CI platforms CirrusCI and AppVeyor. + Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 - Assisted-by: Peter Wu - Closes #5364 + Closes https://github.com/curl/curl/pull/5516 -Daniel Stenberg (11 May 2020) -- write-out.d: added "response_code" +Marc Hoersken (3 Jun 2020) +- CI/macos: fix 'is already installed' errors by using bundle + + Avoid failing CI builds due to nghttp2 being already installed. + + Closes #5513 -- KNOWN_BUGS: Build with staticly built dependency +Daniel Stenberg (3 Jun 2020) +- altsvc: fix 'dsthost' may be used uninitialized in this function + +- RELEASE-NOTES: synced + +- urldata: let the HTTP method be in the set.* struct - I rewrote the item 5.4 to be more generic about static dependencies. + When the method is updated inside libcurl we must still not change the + method as set by the user as then repeated transfers with that same + handle might not execute the same operation anymore! + + This fixes the libcurl part of #5462 + + Test 1633 added to verify. + + Closes #5499 -- ROADMAP: remove old entries +- hostip: fix the memory-leak introduced in 67d2802 - MQTT - the start has already landed + Fixes #5503 + Closes #5504 + +- test970: make it require proxy support - tiny-curl - also mostly landed and is a continuous work + This test verifies the -w %json output and the test case includes a full + generated "blob". If there's no proxy support built into libcurl, it + will return an error for proxy related info variables and they will not + be included in the json, thus causing a mismatch and this test fails. - make menuconfig - basically no interest from users, not pushing there + Reported-by: Marc Hörsken + Fixes #5501 + Closes #5502 -- [Peter Wu brought this change] +- [Radoslav Georgiev brought this change] - travis: Add ngtcp2 and quiche tests for CMake + examples/http2-down/upload: add error checks - To avoid an explosion of jobs, extend the existing CMake tests with - ngtcp2 and quiche support. macOS was previously moved to GitHub actions, - so the non-Linux case can be dropped. + If `index.html` does not exist in the directory from which the example + is invoked, the fopen(upload, "rb") invocation in `setup` would fail, + returning NULL. This value is subsequently passed as the FILE* argument + of the `fread` invocation in the `read_callback` function, which is the + actual cause of the crash (apparently `fread` assumes that argument to + be non-null). + + In addition, mitigate some possible crashes of similar origin. + + Closes #5463 -- [Peter Wu brought this change] +- [kotoriのねこ brought this change] - CMake: add ENABLE_ALT_SVC option + examples/ephiperfifo: turn off interval when setting timerfd - Tested alt-svc with quiche. While at it, add missing MultiSSL reporting - (not tested). + Reported-by: therealhirudo on github + Fixes #5485 + Closes #5497 -- [Peter Wu brought this change] +- [Saleem Abdulrasool brought this change] - CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) + vtls: repair the build with `CURL_DISABLE_PROXY` - Add three new CMake Find modules (using the curl license, but I grant - others the right to apply the CMake BSD license instead). + `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` + is enabled. Repair the build with that configuration. - This CMake config is simpler than the autotools one because it assumes - ngtcp2 and nghttp3 to be used together. Another difference is that this - CMake config checks whether QUIC is actually supported by the TLS - library (patched OpenSSL or boringssl) since this can be a common - configuration mistake that could result in build errors later. + Follow-up to f3d501dc67 - Unlike autotools, CMake does not warn you that the features are - experimental. The user is supposed to already know that and read the - documentation. It requires a very special build environment anyway. + Closes #5498 + +- transfer: remove k->str NULL check - Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from - current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure - out which features (libldap-2.4, libssh2) to disable due to conflicts - with boringssl. + "Null-checking k->str suggests that it may be null, but it has already + been dereferenced on all paths leading to the check" - and it can't + legally be NULL at this point. Remove check. - Closes #5359 + Detected by Coverity CID 1463884 + + Closes #5495 -Marc Hoersken (10 May 2020) -- tests/server/tftpd.c: fix include and enhance debug logging +Marc Hoersken (1 Jun 2020) +- select: always use Sleep in Curl_wait_ms on Win32 - setjmp.h should only be included if HAVE_SETJMP_H is defined. + Since Win32 almost always will also have USE_WINSOCK, + we can reduce complexity and always use Sleep there. - Add additional log statements to see wether reads and writes - are blocking or finishing before an alarm signal is received. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Assisted-by: Peter Wu - Part of #5364 + Follow up to #5343 + Closes #5489 -Daniel Stenberg (10 May 2020) -- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present +Daniel Stenberg (31 May 2020) +- conncache: download buffer needs +1 size for trailing zero - Reported-by: Marcel Raad - Follow-up to 148534db5 - Fixes #5367 - Closes #5369 + Follow-up to c4e6968127e + Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 -Marc Hoersken (9 May 2020) -- appveyor: update comments to be clear about toolchain +Marc Hoersken (31 May 2020) +- azure: use matrix strategy to avoid configuration redundancy - - CMake-based MSYS builds use mingw-w64 to cross-compile. - - autotools-based builds are compiled using msys2-devel. + This also includes the following changes: - The difference is that the later ones are not cross-compiled - to Windows and instead require the msys2 runtime to be present. + - Use the same timeout for all jobs on Linux (60 minutes) + and Windows (90 minutes) + - Use CLI stable apt-get install -y instead of apt install + which warns about that and run apt-get update first + - Enable MQTT for Windows msys2 builds instead of + legacy msys1 builds + - Add ./configure --prefix parameter to the msys2 builds + - The MSYSTEM environment variable is now preset inside + the container images for the msys2 builds - At the moment only the Azure Pipelines CI builds actually - run autotools-based cross-compilation builds for Windows. - -- TODO: update regarding missing Schannel features + Note: on Azure Pipelines the matrix strategy is basically + just a simple list of job copies and not really a matrix. - Some aspects have already been implemented over the years. + Closes #5468 + +Daniel Stenberg (30 May 2020) +- build: disable more code/data when built without proxy support - 15.1 Client certificates are now supported: + Added build to travis to verify - - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 - - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 + Closes #5466 + +- url: alloc the download buffer at transfer start - 15.2 Ciphers can now be specified through: + ... and free it as soon as the transfer is done. It removes the extra + alloc when a new size is set with setopt() and reduces memory for unused + easy handles. - - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 + In addition: the closure_handle now doesn't use an allocated buffer at + all but the smallest supported size as a stack based one. - Reviewed-by: Daniel Stenberg and Marcel Raad - Closes #5358 - -Daniel Stenberg (8 May 2020) -- checksrc: close the .checksrc file handle when done reading + Closes #5472 -- RELEASE-NOTES: synced +- timeouts: change millisecond timeouts to timediff_t from time_t - And bumped next version to 7.71.0 - -- [Gilles Vollant brought this change] - - CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) + For millisecond timers we like timediff_t better. Also, time_t can be + unsigned so returning a negative value doesn't work then. - Closes #4346 - -- TODO: native IDN support on macOS + Closes #5479 -- urlapi: accept :: as a valid IPv6 address +Marc Hoersken (30 May 2020) +- select: add overflow checks for timeval conversions - Text 1560 is extended to verify. + Using time_t and suseconds_t if suseconds_t is available, + long on Windows (maybe others in the future) and int elsewhere. - Reported-by: Pavel Volgarev - Fixes #5344 - Closes #5351 - -- THANKS-filter: Peter Wang - -- [Peter Wang brought this change] - - *_sspi: fix bad uses of CURLE_NOT_BUILT_IN + Also handle case of ULONG_MAX being greater or equal to INFINITE. - Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other - instances of QuerySecurityPackageInfo failing, as in - commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5355 - -- docs/HTTP3: add qlog to the quiche build instruction + Part of #5343 -- ngtcp2: introduce qlog support +- select: use timediff_t instead of time_t and int for timeout_ms - If the QLOGDIR environment variable is set, enable qlogging. + Make all functions in select.[ch] take timeout_ms as timediff_t + which should always be large enough and signed on all platforms + to take all possible timeout values and avoid type conversions. - ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for - QUIC functions that are backend independent. + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5353 + Replaces #5107 and partially #5262 + Related to #5240 and #5286 + Closes #5343 -- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN - - That return code is reserved for build-time conditional code not being - present while this was a regular run-time error from a Windows API. +- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' - Reported-by: wangp on github - Fixes #5349 - Closes #5350 - -- runtests: show elapsed test time with higher precision (ms) - -- RELEASE-NOTES: synced - -- http2: simplify and clean up trailer handling + GCC 10 warns about this with warning: implicit conversion + from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] - Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in - ed35d6590e72. This should make the trailer handling more straight forward and - hopefully less error-prone. + Since 'expected_result' is not really of type 'CURLcode' and + it is not exposed in any way, we can just use 'SANITIZEcode'. - Deliver the trailer header to the callback already at receive-time. No - longer caches the trailers to get delivered at end of stream. + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 - Closes #5348 + Closes #5476 -Marc Hoersken (7 May 2020) -- appveyor: disable test 1139 instead of ignoring it +- tests/libtest: fix undefined reference to 'curlx_win32_fopen' - Spending time on manpage checking makes no sense - for these builds due to lacking manpage support. - -- appveyor: disable flaky test 1501 and ignore broken 1056 + Since curl_setup.h now makes use of curlx_win32_fopen for Win32 + builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, + we need to include the relevant files for tests using fopen, + because the libtest sources are also including curl_setup.h - Test 1501 is flaky on Windows CI due to being time sensitive - and the testsuite relying on taskkill.exe to check for the - existance of processes which can take to much time itself. + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - Test 1056 is broken in autotools-based Windows builds due - to scope ID support missing in these builds at the moment. + Follow up to #3784 (ffdddb45d9) + Closes #5475 -- test613.pl: make tests 613 and 614 work with OpenSSH for Windows +- appveyor: add non-debug plain autotools-based build - OpenSSH for Windows shows group and other/world permissions as *, - because those concepts do not exist on Windows. It also does not - show the current or parent directory, so we just ignore those. + This should enable us to catch linking issues with the + testsuite early, like the one described/fixed in #5475. Reviewed-by: Daniel Stenberg - Closes #5328 + Reviewed-by: Marcel Raad + + Closes #5477 -Daniel Stenberg (6 May 2020) -- runtests: set +x mode again +Daniel Stenberg (29 May 2020) +- RELEASE-NOTES: synced -- libssh2: convert over to use dynbuf +- Revert "buildconf: use find -execdir" - In my very basic test that lists sftp://127.0.0.1/tmp/, this patched - code makes 161 allocations compared to 194 in git master. A 17% - reduction. + This partially reverts commit c712009838f44211958854de431315586995bc61. - Closes #5336 + Keep the ares_ files removed but bring back the older way to run find, + to make it work with busybox's find, as apparently that's being used. + + Reported-by: Max Peal + Fixes #5483 + Closes #5484 -- travis: add "qlog" as feature in the quiche build +- server/sws: fix asan warning on use of uninitialized variable -- quiche: enable qlog output - - quiche has the potential to log qlog files. To enable this, you must - build quiche with the qlog feature enabled `cargo build --features - qlog`. curl then passes a file descriptor to quiche, which takes - ownership of the file. The FD transfer only works on UNIX. +- libssh2: improved error output for wrong quote syntax - The convention is to enable logging when the QLOGDIR environment is - set. This should be a path to a folder where files are written with the - naming template .qlog. + Reported-by: Werner Stolz - Co-authored-by: Lucas Pardue - Replaces #5337 - Closes #5341 + Closes #5474 -- urldata.h: remove #define HEADERSIZE, not used anymore +- mk-lib1521: generate code for testing BLOB options as well - Follow-up to ed35d6590e72c - -- ngtcp2: convert to dynbuf + Follow-up to cac5374298b3 - Closes #5335 + Closes #5478 -- connect: make happy eyeballs work for QUIC (again) +- configure: repair the check if argv can be written to - Follow-up from dbd16c3e256c6c (regression in 7.70.0) + Due to bad escaping of the test code, the test wouldn't build and thus + result in a negative test result, which would lead to the unconditional + assumption that overwriting the arguments doesn't work and thus curl + would never hide credentials given in the command line, even when it + would otherwise be possible. - Closes #5334 - -- connect: add two asserts to clue code analyzers in a little - -- http_proxy: ported to use dynbuf instead of a static size buffer + Regression from commit 2d4c2152c (7.60.0) - Removes a 16K static buffer from the easy handle. Simplifies the code. + Reported-by: huzunhao on github + Fixes #5470 + Closes #5471 -- dynbuf: introduce internal generic dynamic buffer functions - - A common set of functions instead of many separate implementations for - creating buffers that can grow when appending data to them. Existing - functionality has been ported over. +Peter Wu (28 May 2020) +- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - In my early basic testing, the total number of allocations seem at - roughly the same amount as before, possibly a few less. + Otherwise the build might fail due to missing source files, as + demonstrated by the recent keylog.c addition on an existing build dir. - See docs/DYNBUF.md for a description of the API. + Closes #5469 + +Daniel Stenberg (28 May 2020) +- urldata: fix comments: Curl_done() is called multi_done() now - Closes #5300 + ... since 575e885db -- runtests: remove sleep calls +Peter Wu (27 May 2020) +- ngtcp2: use common key log routine for better thread-safety - Remove many one second sleeps that were done *after* each newly started - test server already has been verified. They should not have any purpose - there. + Tested with ngtcp2 built against the OpenSSL library. Additionally + tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). - Closes #5323 + The TLS backend (independent of QUIC) may or may not already have opened + the keylog file before. Therefore Curl_tls_keylog_open is always called + to ensure the file is open. -- asyn-*: remove support for never-used NULL entry pointers +- wolfssl: add SSLKEYLOGFILE support - ... and instead convert those to asserts to make sure they are truly - never NULL. + Tested following the same curl and tshark commands as in commit + "vtls: Extract and simplify key log file handling from OpenSSL" using + WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with + `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. - Closes #5324 - -- [Emil Engler brought this change] - - doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax + Full support for this feature requires certain wolfSSL build options, + see "Availability note" in lib/vtls/wolfssl.c for details. - Closes #5325 + Closes #5327 -Jay Satiro (2 May 2020) -- asyn-thread: fix cppcheck warning +- vtls: Extract and simplify key log file handling from OpenSSL - - Check for NULL entry parameter before attempting to deref entry in - Curl_resolver_is_resolved, like is already done in asyn-ares. + Create a set of routines for TLS key log file handling to enable reuse + with other TLS backends. Simplify the OpenSSL backend as follows: - This is to silence cppcheck which does not seem to understand that - asyn-ares and asyn-thread have separate Curl_resolver_is_resolved - and those units are mutually exclusive. Prior to this change it warned - of a scenario where asyn-thread's Curl_resolver_is_resolved is called - with a NULL entry from asyn-ares, but that couldn't happen. + - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. + - Do not perform dynamic memory allocation when preparing a log entry. + Unless the TLS specifications change we can suffice with a reasonable + fixed-size buffer. + - Simplify state tracking when SSL_CTX_set_keylog_callback is + unavailable. My original sslkeylog.c code included this tracking in + order to handle multiple calls to SSL_connect and detect new keys + after renegotiation (via SSL_read/SSL_write). For curl however we can + be sure that a single master secret eventually becomes available + after SSL_connect, so a simple flag is sufficient. An alternative to + the flag is examining SSL_state(), but this seems more complex and is + not pursued. Capturing keys after server renegotiation was already + unsupported in curl and remains unsupported. - Reported-by: rl1987@users.noreply.github.com + Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f + (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) + against an OpenSSL 1.1.1f server configured with: - Fixes https://github.com/curl/curl/issues/5326 - -- select: fix overflow protection in Curl_socket_check + # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 + openssl s_server -www -tls1 + # Likewise, but fail the server handshake. + openssl s_server -www -tls1 -Verify 2 + # TLS 1.3 test. No need to test the failing server handshake. + openssl s_server -www -tls1_3 - Follow-up to a96c752 which changed the timeout_ms type from time_t to - timediff_t. + Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly + written using Wireshark. For the first and third case, expect four + matches per connection (decrypted Server Finished, Client Finished, HTTP + Request, HTTP Response). For the second case where the handshake fails, + expect a decrypted Server Finished only. - Ref: https://github.com/curl/curl/pull/5240 + tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ + -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ + -dtls.port==4433,http -ohttp.desegment_body:FALSE \ + -Y 'tls.handshake.verify_data or http' - Closes https://github.com/curl/curl/pull/5286 + A single connection can easily be identified via the `tcp.stream` field. -Marc Hoersken (2 May 2020) -- sockfilt: make select_ws stop waiting on exit signal event - - This makes sure that select_ws behaves similar to real select - which stops waiting on a signal handler being triggered. +Daniel Stenberg (27 May 2020) +- FILEFORMAT: add more features that tests can depend on + +- [Michael Kaufmann brought this change] + + transfer: close connection after excess data has been read - This makes it possible to gracefully stop sockfilt.exe on - Windows with taskkill /IM sockfilt.exe (without /F force flag). + For HTTP 1.x, it's a protocol error when the server sends more bytes + than announced. If this happens, don't reuse the connection, because the + start position of the next response is undefined. - Reviewed-by: Jay Satiro - Part of #5260 + Closes #5440 -- tests/server/util.[ch]: add exit event to stop waiting on Windows - - This commit adds a global exit event to the test servers that - Windows-specific wait routines can use to get triggered if the - program was signaled to be terminated, eg. select_ws in sockfilt.c +- [Estanislau Augé-Pujadas brought this change] + + Revert "ssh: ignore timeouts during disconnect" - The exit event will be managed by the signal handling code and is - set to not reset automatically to support multiple wait routines. + This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in + curl 7.54.1. - Reviewed-by: Jay Satiro - Closes #5260 + Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html + Closes #5465 -- tests/server/util.c: fix thread handle not being closed +- urldata: connect related booleans live in struct ConnectBits - Reviewed-by: Jay Satiro - Part of #5260 + And remove a few unused booleans! + + Closes #5461 -- tests/server/util.c: use raise instead of calling signal handler +- hostip: on macOS avoid DoH when given a numerical IP address - Use raise to trigger signal handler instead of calling it - directly and causing potential unexpected control flow. + When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that + numerical IP addresses still need to get "resolved" - but not with DoH. - Reviewed-by: Jay Satiro - Part of #5260 + Reported-by: Viktor Szakats + Fixes #5454 + Closes #5459 -- tests: add support for SSH server variant specific transfer paths +- ngtcp2: cleanup memory when failing to connect - OpenSSH for Windows requires paths in the format of /C:/ - instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ + Reported-by: Peter Wu + Fixes #5447 (the ngtcp2 side of it) + Closes #5451 + +- quiche: clean up memory properly when failing to connect - Reviewed-by: Daniel Stenberg - Closes #5298 + Addresses the quiche side of #5447 + Reported-by: Peter Wu + Closes #5450 -Daniel Stenberg (2 May 2020) -- RELEASE-NOTES: synced +- cleanup: use a single space after equals sign in assignments -- libssh2: set the expected total size in SCP upload init +- url: accept "any length" credentials for proxy auth - ... as otherwise the progress callback gets called without that - information, making the progress meter have less info. + They're only limited to the maximum string input restrictions, not to + 256 bytes. - Reported-by: Murugan Balraj - Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html - Closes #5317 + Added test 1178 to verify + + Reported-by: Will Roberts + Fixes #5448 + Closes #5449 -- runtests: make the logmsg from the ssh server only show in verbose +- [Maksim Stsepanenka brought this change] -- tests: make test 1248 + 1249 use %NOLISTENPORT - - ... instead of a port of a non-running server so that it works - stand-alone. + test1167: fixes in badsymbols.pl - Closes #5318 + Closes #5442 -- examples: remove asiohiper.cpp +- altsvc: fix parser for lines ending with CRLF - This example has repeatedly been reported to contain bugs, and as users - copy and paste code from this into production, I now deem it better to - not provide the example at all. + Fixed the alt-svc parser to treat a newline as end of line. - Closes #5090 - Closes #5322 - -- [Emil Engler brought this change] - - doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 + The unit tests in test 1654 were done without CRLF and thus didn't quite + match the real world. Now they use CRLF as well. - Closes #5320 - -- [Emil Engler brought this change] + Reported-by: Peter Wu + Assisted-by: Peter Wu + Assisted-by: Jay Satiro + Fixes #5445 + Closes #5446 - KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" - - It was fixed in 9a2cbf3 +Viktor Szakats (25 May 2020) +- all: fix codespell errors - Closes #5319 + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes https://github.com/curl/curl/pull/5452 -- cirrus: disable SFTP and SCP tests +Peter Wu (25 May 2020) +- ngtcp2: fix build with current ngtcp2 master implementing draft 28 - ... as we can't seem to start the sshd server on it. Those problems - existed before d1239b50bececd (running the SSH server on a random port), - but they're more noticable now since there are more failed attempts in - the logs. + Based on client.cc changes from ngtcp2. Tested with current git master, + ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. - Closes #5315 - -- [Emil Engler brought this change] + Fixes #5444 + Closes #5443 - runtests: fix typo in the existence of disabled tests checker +Daniel Stenberg (25 May 2020) +- RELEASE-NOTES: synced - Closes #5316 + moved the new setopts up to a "change" -Dan Fandrich (30 Apr 2020) -- test75: Remove precheck test - - This has not been needed since commit 9fa42bed and often prevents it - from running at all with dynamic test ports. +- RELEASE-NOTES: synced -- tests: Stop referring to server ports when they're not used +- copyright: updated year ranges out of sync - Several tests referred to specific server ports even when the test - didn't actually use that server or specify that it's needed. In such - cases, the test harness substitutes the text "[not running]" as the port - number which causes many such tests to fail due to the inability to - parse the URL. These tests are changed to use %NOLISTENPORT which will - always be substituted correctly. + ... and whitelisted a few more files in the the copyright.pl script. -Daniel Stenberg (30 Apr 2020) -- [Emil Engler brought this change] +- [Gilles Vollant brought this change] - GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT + setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency - Closes #5287 + Closes #5431 -- conncache: various concept cleanups - - More connection cache accesses are protected by locks. +- curl: remove -J "informational" written on stdout - CONNCACHE_* is a beter prefix for the connection cache lock macros. + curl would previously show "curl: Saved to filename 'name from header'" + if -J was used and a name was picked from the Content-Disposition + header. That output could interfer with other stdout output, such as -w. - Curl_attach_connnection: now called as soon as there's a connection - struct available and before the connection is added to the connection - cache. + This commit removes that output line. + Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html + Reported-by: Коваленко Анатолий Викторович + Closes #5435 + +Peter Wu (22 May 2020) +- travis: simplify quiche build instructions wrt boringssl - Curl_disconnect: now assumes that the connection is already removed from - the connection cache. + quiche builds boringssl as static library, reuse that instead of + building another shared library. - Ref: #4915 - Closes #5009 + Closes #5438 -- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports +- configure: fix pthread check with static boringssl - As stunnel is an external tool and it has no specific option to export - the actually used port number when asked to listen to 0, runtests - instead iterates over ten randomly picked high number ports and sticks - to the first one stunnel can listen to. - - Closes #5267 + A shared boringssl/OpenSSL library requires -lcrypto only for linking. + A static build additionally requires `-ldl -lpthread`. In the latter + case `-lpthread` is added to LIBS which prevented `-pthread` from being + added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. -- tests: pick a random port number for SSH - - Since sshd doesn't have such an option by itself, we iterate over a - series of random ports until one works. +Daniel Stenberg (22 May 2020) +- Revert "sendf: make failf() use the mvsnprintf() return code" - Closes #5273 - -- [Rikard Falkeborn brought this change] - - libtest/cmake: Remove commented code + This reverts commit 74623551f306990e70c7c5515b88972005604a74. - These were commented out in e9dd0998706a when Makefile.inc was included - instead. 11 years have passed since then and the commented code is of - course very outdated. Remove it to avoid confusion. + Instead mark the function call with (void). Getting the return code and + using it instead triggered Coverity warning CID 1463596 because + snprintf() can return a negative value... - Closes #5311 + Closes #5441 -- schannel: source code reindent - - White space edits only. Conform better to standard curl source code - indenting style. +- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' - Closes #5305 + Reported-by: Billyzou0741326 on github + Fixes #5432 + Closes #5436 -Kamil Dudka (29 Apr 2020) -- test1177: look for curl.h in source directory - - If we use a separate build directory, there is no copy of the header. +- tests/server/util.h: add extern to silence compiler warning - Closes #5310 + Follow-up from a3b0699d5c1 -- tests: look for preprocessed tests in build directory +- typecheck-gcc.h: fix the OFF_T check - ... which is not always the same directory as source directory + The option number also needs to be less than CURLOPTTYPE_BLOB. - Closes #5310 + Follow-up to cac5374298 + Reported-by: Jeroen Ooms + Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: synced +- TODO: --dry-run - ... and bumped curlver.h to 7.70.1 - -Version 7.70.0 (29 Apr 2020) - -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: 7.70.0 + Closes #5426 -- THANKS: synced with the 7.70.0 release +- TODO: Ratelimit or wait between serial requests + + Closes #5406 -- headers: copyright range fix +- tool_paramhlp: fixup C89 mistake + + Follow-up to c5f0a9db22. -- [Rikard Falkeborn brought this change] +- [Siva Sivaraman brought this change] - doh: Constify some input pointers + tool_paramhlp: fixed potentially uninitialized strtol() variable - Closes #5306 + Seems highly unlikely to actually be possible, but better safe than + sorry. + + Closes #5417 -- nss: check for PK11_CreateDigestContext() returning NULL +- [Siva Sivaraman brought this change] + + tool_operate: fixed potentially uninitialized variables - ... to avoid crashes! + ... in curl_easy_getinfo() calls. They're harmless but clearing the + variables makes the code safer and comforts the reader. - Reported-by: Hao Wu - Fixes #5302 - Closes #5303 + Closes #5416 -- travis: bump the wolfssl CI build to use 4.4.0 +- sha256: move assign to the declaration line - Closes #5301 + Follow-up to fae30656. Should've been squashed with that commit... -- copyright updates: adjust year ranges +- [Siva Sivaraman brought this change] -Marc Hoersken (26 Apr 2020) -- CI: do not include */ci branches in PR builds + sha256: fixed potentially uninitialized variable - Align Azure Pipelines with GitHub Actions. + Closes #5414 -Daniel Stenberg (25 Apr 2020) -- runtests: check for the disabled tests relative srcdir +- sendf: make failf() use the mvsnprintf() return code - To make it work correctly for out-of-tree builds. + ... and avoid a strlen() call. Fixes a MonocleAI warning. - Follow-up to 75e8feb6fb08b + Reported-by: MonocleAI + Fixes #5413 + Closes #5420 + +- hostip: make Curl_printable_address not return anything - Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 - Reported-by: Marcel Raad - Closes #5297 + It was not used much anyway and instead we let it store a blank buffer + in case of failure. + + Reported-by: MonocleAI + Fixes #5411 + Closes #5418 -- runtests: revert commenting out a line I did for debugging +- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) - Follow-up to 11091cd4d. It was not meant to be pushed! + They're done on purpose, make that visible in the code. + Reported-by: MonocleAI + Fixes #5412 + Closes #549 -- smtp: set auth correctly +- TODO: forbid TLS post-handshake auth and do TLS record padding - Regression since 7.69.0 and 68fb25fa3fcff. + Closes #5396 + Closes #5398 + +- RELEASE-NOTES: synced + +- dynbuf: return NULL when there's no buffer length - The code wrongly assigned 'from' instead of 'auth' which probably was a - copy and paste mistake from other code, leading to that auth could - remain NULL and later cause an error to be returned. + ... as returning a "" is not a good idea as the string is supposed to be + allocated and returning a const string will cause issues. - Assisted-by: Eric Sauvageau - Fixes #5294 - Closes #5295 + Reported-by: Brian Carpenter + Follow-up to ed35d6590e72c + Closes #5405 -Marcel Raad (25 Apr 2020) -- lib: clean up whitespace +Peter Wu (16 May 2020) +- travis: upgrade to bionic, clang-9, improve readability - This fixes CodeFactor warnings. - -Daniel Stenberg (25 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] + Changes, partially to reduce build failures from external dependencies: + - Upgrade Ubuntu and drop unnecessary third-party repos. + - Properly clone apt config to ensure retries. + - Upgrade to clang-9 from the standard repos. + - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey + fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use + focal everywhere yet since Travis CI has not documented this option. + In focal, python-impacket (Py2.7) has been removed, leaving only + python3-impacket. Since it is only needed for SMB tests and not SSH, + skip it for the libssh job since it might need more work. + - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. + + Non-functional cleanups: + - Simplify test matrix, drop redundant os and compiler keys. + - Deprecation fixes: remove sudo, rename matrix -> jobs. + - Every job has an 'env' key, put this key first in a list item. + + Closes #5370 - libssh: avoid options override by configuration files +- travis: whitespace-only changes for consistency - Previously, options set explicitly through command line options could be - overridden by the configuration files parsed automatically when - ssh_connect() was called. + Automatically apply a consistent indentation with: - By calling ssh_options_parse_config() explicitly, the configuration - files are parsed before setting the options, avoiding the options - override. Once the configuration files are parsed, the automatic - configuration parsing is not executed. + python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' - Fixes #4972 - Closes #5283 - Signed-off-by: Anderson Toshiyuki Sasaki + followed by manually re-indenting three comments. + + Closes #5370 -- runtests: when mentions http, kill http/2 too +- CMake: add libssh build support - Since the http2 test server is a mere proxy that needs to know about the - dynamic port the HTTP server is using, it too needs to get restarted - when the http server is killed. + Closes #5372 + +Daniel Stenberg (15 May 2020) +- KNOWN_BUGS: wolfssh: publickey auth doesn't work - A regression caused by 80d6515. + Closes #4820 + +- KNOWN_BUGS: OS400 port requires deprecated IBM library - Fixes #5289 - Closes #5291 + Closes #5176 -- [Yuri Slobodyanyuk brought this change] +- [Vyron Tsingaras brought this change] - docs: fix two typos + http2: keep trying to send pending frames after req.upload_done - Closes #5292 + Fixes #1410 + Closes #5401 -- [Emil Engler brought this change] +- [Gilles Vollant brought this change] - tests/git: ignore mqttd and port files + setopt: support certificate options in memory with struct curl_blob - Closes #5290 - -- tests: make runtests check that disabled tests exists + This change introduces a generic way to provide binary data in setopt + options, called BLOBs. - ... and error out if so. Removed '536' from DISABLED as there is no such - test file. + This change introduces these new setopts: - Closes #5288 - -- test1154: set a proper name + CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, + CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. + + Reviewed-by: Daniel Stenberg + Closes #5357 -- select: make Curl_socket_check take timediff_t timeout +- source cleanup: remove all custom typedef structs - Coverity found CID 1461718: + - Stick to a single unified way to use structs + - Make checksrc complain on 'typedef struct {' + - Allow them in tests, public headers and examples - Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > - 9223372036854775807L" is always false regardless of the values of its - operands. This occurs as the logical second operand of "||". + - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually + typedef different types/structs depending on build conditions. - Closes #5240 + Closes #5338 -- [i-ky brought this change] +- travis: remove the .checksrc fiddling - libcurl-multi.3: added missing full stop +- ftp: make domore_getsock() return the secondary socket properly - Closes #5285 + Previously, after PASV and immediately after the data connection has + connected, the function would only return the control socket to wait for + which then made the data connection simply timeout and not get polled + correctly. This become obvious when running test 1631 and 1632 event- + based. -Jay Satiro (22 Apr 2020) -- transfer: Switch PUT to GET/HEAD on 303 redirect - - Prior to this change if there was a 303 reply to a PUT request then - the subsequent request to respond to that redirect would also be a PUT. - It was determined that was most likely incorrect based on the language - of the RFCs. Basically 303 means "see other" resource, which implies it - is most likely not the same resource, therefore we should not try to PUT - to that different resource. +- test1632: verify FTP through HTTPS-proxy with connection re-use + +- test1631: verify FTP download through HTTPS-proxy + +- sws: as last resort, get test number from server cmd file - Refer to the discussions in #5237 and #5248 for more information. + If it can't be found in the request. Also support --cmdfile to set it to + a custom file name. - Fixes https://github.com/curl/curl/issues/5237 - Closes https://github.com/curl/curl/pull/5248 + runtests.pl always writes this file with the test number in it since a + while back. -Daniel Stenberg (22 Apr 2020) -- lib/mk-ca-bundle: skip empty certs +- ftp: shut down the secondary connection properly when SSL is used - Reviewed-by: Emil Engler - Reported-by: Ashwin Metpalli - Fixes #5278 - Closes #5280 + Reported-by: Neal Poole + Fixes #5340 + Closes #5385 -- version: skip idn2_check_version() check and add precaution +Marcel Raad (14 May 2020) +- KNOWN_BUGS: adapt 5.5 to recent changes - A gcc-10's -fanalyze complaint made me spot and do these improvements. + It only applies to non-Unicode builds now. + Also merge 5.10 into it as it's effectively a duplicate. - Closes #5281 - -- RELEASE-NOTES: synced + Closes https://github.com/curl/curl/pull/3784 -- [Brian Bergeron brought this change] +- curl_setup: support Unicode functions to open files on Windows + + Use them only if `_UNICODE` is defined, in which case command-line + arguments have been converted to UTF-8. + + Closes https://github.com/curl/curl/pull/3784 - curl.h: update comment typo +- tool: support UTF-16 command line on Windows - "routines with be invoked" -> "routines will be invoked" + - use `wmain` instead of `main` when `_UNICODE` is defined [0] + - define `argv_item_t` as `wchar_t *` in this case + - use the curl_multibyte gear to convert the command-line arguments to + UTF-8 - Closes #5279 + This makes it possible to pass parameters with characters outside of + the current locale on Windows, which is required for some tests, e.g. + the IDN tests. Out of the box, this currently only works with the + Visual Studio project files, which default to Unicode, and winbuild + with the `ENABLE_UNICODE` option. + + [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + + Ref: https://github.com/curl/curl/issues/3747 + Closes https://github.com/curl/curl/pull/3784 -- [Emil Engler brought this change] +- curl_multibyte: add to curlx + + This will also be needed in the tool and tests. + + Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 + Closes https://github.com/curl/curl/pull/3784 - GnuTLS: Don't skip really long certificate fields +Daniel Stenberg (14 May 2020) +- url: make the updated credentials URL-encoded in the URL - Closes #5271 + Found-by: Gregory Jefferis + Reported-by: Jeroen Ooms + Added test 1168 to verify. Bug spotted when doing a redirect. + Bug: https://github.com/jeroen/curl/issues/224 + Closes #5400 -- gnutls: bump lowest supported version to 3.1.10 +- tests: add https-proxy support to the test suite - GnuTLS 3.1.10 added new functions we want to use. That version was - released on Mar 22, 2013. Removing support for older versions also - greatly simplifies the code. + Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like + HTTP proxy but with a full TLS connection to the proxy. - Ref: #5271 - Closes #5276 + Closes #5399 -- mqtt: make NOSTATE get within the debug name array +- mailmap: James Fuller -- tests: run the RTSP test server on a dynamic port number +- [Major_Tom brought this change] + + vauth/cleartext: fix theoretical integer overflow - To avoid port collisions. + Fix theoretical integer overflow in Curl_auth_create_plain_message. - Closes #5272 + The security impact of the overflow was discussed on hackerone. We + agreed this is more of a theoretical vulnerability, as the integer + overflow would only be triggerable on systems using 32-bits size_t with + over 4GB of available memory space for the process. + + Closes #5391 -- tests: add %NOLISTENPORT and use it +Jay Satiro (13 May 2020) +- curl.1: Quote globbed URLs - The purpose with this variable is to provide a port number that is - reasonably likely to not have a listener on the local host so that tests - can try connect failures against it. It uses port 47 - "reserved" - according to IANA. + - Quote the globbing example URLs that contain characters [] {} since + otherwise they may be interpreted as shell metacharacters. - Updated six tests to use it instead of the previous different ports. + Bug: https://github.com/curl/curl/issues/5388 + Reported-by: John Simpson - Assisted-by: Emil Engler - Closes #5270 + Closes https://github.com/curl/curl/pull/5394 -- mqtt: remove code with no purpose +Daniel Stenberg (14 May 2020) +- checksrc: enhance the ASTERISKSPACE and update code accordingly - Detected by Coverity. CID 1462319. + Fine: "struct hello *world" - "The same code is executed when the condition result is true or false, - because the code in the if-then branch and after the if statement is - identical." + Not fine: "struct hello* world" (and variations) - Closes #5275 + Closes #5386 -- mqtt: fix Curl_read() error handling while reading remaining length +- docs/options-in-versions: which version added each cmdline option - Detected by Coverity. CID 1462320. + Added test 971 to verify that the list is in sync with the files in + cmdline-opts. The check also verifies that .d-files that uses Added: + specify the same version number as the options-in-versions file does. - Closes #5274 + Closes #5381 -- server/tftpd: fix compiler warning +- docs: unify protocol lists - Follow-up from 369ce38ac1d - Reported-by: Marc Hörsken - -- http: free memory when Alt-Used header creation fails due to OOM + We boast support for 25 transfer protocols. Make sure the lists are + consistent - Reported-by: James Fuller - Fixes #5268 - Closes #5269 + Closes #5384 -Daniel Gustafsson (20 Apr 2020) -- lib: fix typos in comments and errormessages +- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN - This fixes a few randomly spotted typos in recently merged code, most - notably one in a userfacing errormessage the schannel code. - -Daniel Stenberg (20 Apr 2020) -- tests: run the SOCKS test server on a dynamic port number + ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. - Closes #5266 + Reported-by: Michael Kaufmann + Fixes #5374 + Closes #5376 -- [Johannes Schindelin brought this change] +- tls13-ciphers.d: shorten the Arg - multi-ssl: reset the SSL backend on `Curl_global_cleanup()` +- sasl-authzid.d: add Arg: and shorten the desc + +- cert-type.d: mention the available types in the desc + +- tool: shorten 3 --help descriptions - When cURL is compiled with support for multiple SSL backends, it is - possible to configure an SSL backend via `curl_global_sslset()`, but - only *before* `curl_global_init()` was called. + --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort - If another SSL backend should be used after that, a user might be - tempted to call `curl_global_cleanup()` to start over. However, we did - not foresee that use case and forgot to reset the SSL backend in that - cleanup. + gen.pl already warned about these lines but we didn't listen - Let's allow that use case. + Closes #5379 + +- configure: the wolfssh backend does not provide SCP - Fixes #5255 - Closes #5257 - Reported-by: davidedec on github - Signed-off-by: Johannes Schindelin + Closes #5387 -- tests: run the TFTP test server on a dynamic port number +- RELEASE-NOTES: synced + +- url: reject too long input when parsing credentials - Picking a dynamic unused port is better than a fixed to avoid the - collision risk. + Since input passed to libcurl with CURLOPT_USERPWD and + CURLOPT_PROXYUSERPWD circumvents the regular string length check we have + in Curl_setstropt(), the input length limit is enforced in + Curl_parse_login_details too, separately. - Closes #5265 + Reported-by: Thomas Bouzerar + Closes #5383 -- mqtt: improve the state machine +- list-only.d: this option existed already in 4.0 + +Jay Satiro (12 May 2020) +- retry-all-errors.d: Shorten the summary line - To handle PUBLISH before SUBACK and more. + Follow-up to b995bb5 from a few moments ago. - Updated the existing tests and added three new ones. + Reported-by: Daniel Stenberg - Reported-by: Christoph Krey - Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html - Closes #5246 - -- runtests: always put test number in servercmd file - -- RELEASE-NOTES: synced - -- release-notes.pl: fix parsing typo - -James Fuller (20 Apr 2020) -- ensure all references to ports are replaced by vars + Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 -- add more alt-svc test coverage +- [denzor brought this change] -Daniel Stenberg (20 Apr 2020) -- test1247: use http server to get the port number set + easy: fix dangling pointer on easy_perform fail - Follow-up to 0f5db7b263f + Closes https://github.com/curl/curl/pull/5363 -- runtests: use a unix domain socket path with the pid in the name +- tool: Add option --retry-all-errors to retry on any error - To make it impossible for test cases to access the file name without - using the proper variable for the purpose. + The "sledgehammer" of retrying. - Closes #5264 + Closes https://github.com/curl/curl/pull/5185 -Daniel Gustafsson (19 Apr 2020) -- [Tom brought this change] +Daniel Stenberg (12 May 2020) +- [James Le Cuirot brought this change] - src: Remove C99 constructs to ensure C89 compliance + libcurl.pc: Merge Libs.private into Libs for static-only builds - This fixes the error: 'for' loop initial declaration used outside C99 - mode by declaring the loop increment variable in the beginning of the - block instead of inside the for loop. + A project being built entirely statically will call pkg-config with + --static, which utilises the Libs.private field. Conversely it will + not use --static when not being built entirely statically, even if + there is only a static build of libcurl available. This will most + likely cause the build to fail due to underlinking unless we merge the + Libs fields. - Fixes #5254 - Reviewed-by: Daniel Gustafsson - -Daniel Stenberg (19 Apr 2020) -- runtests: dummy init the ports variables to avoid warnings - - ... and generate something that can help debug test cases. - -- [Patrick Monnerat brought this change] - - mime: properly check Content-Type even if it has parameters - - New test 669 checks this fix is effective. + Consider that this is what the Meson build system does when it + generates pkg-config files. - Fixes #5256 - Closes #5258 - Reported-by: thanhchungbtc on github - -- tests/FILEFORMAT: converted to markdown and extended + I have also reflected this in the --libs argument of curl-config even + though REQUIRE_LIB_DEPS always seems to be "yes" anyway. - Closes #5261 - -- test1245: make it work with dynamic FTP server port - -- test1055: make it work with dynamic FTP port + Closes #5373 -- test1028: make it run on dynamic FTP server port +- [Peter Wu brought this change] -- tests: move pingpong server to dynamic listening port + CMake: fix runtests.pl with CMake, add new test targets - FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic - ports + * runtests.pl: + - Fix out-of-tree build under CMake when srcdir is not set. Default + srcdir to the location of runtests.pl. + - Add a hack to allow CMake to use the TFLAGS option as documented + in tests/README and used in scripts/travis/script.sh. + * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie + support (no one should care, it is already EOL.). + * Remove CTest since it defines its own 'test' target with no tests + since all unittests are already broken and not built by default. + * Add new test targets based on the options from Makefile.am. Since + new test targets are rarely added, I opted for duplicating the + runtests.pl options as opposed to creating a new Makefile.inc file. + Use top-level target names (test-x) instead of x-test since that is + used by CI and others. - Test 842-845 are unfortunately a bit hard to move over to this concept - right now and require "default port" still... - -- test1056: work with dynamic HTTP ipv6 port + Closes #5358 -- test1448: work with dynamic HTTP server port +- [Peter Wu brought this change] -- tests: introduce preprocessed test cases - - The runtests script now always performs variable replacement on the - entire test source file before the test gets executed, and saves the - updated version in a temporary file (log/test[num]) so that all test - case readers/servers can use that version (if present) and thus enjoy - the powers of test case variable substitution. + CMake: do not build test programs by default - This is necessary to allow complete port number freedom. + The default target should only build libcurl and curl. Add a dedicated + 'testdeps' target which will be used later when running tests. Note that + unittests are currently broken in CMake and already excluded. - Test 309 is updated to work with a non-fixed port number thanks to this. + Closes #5368 -- tests: make 2006-2010 handle different port number lengths +- FILEFORMAT: moved up the variables section and further polished -- tests: run the sws server on "any port" +- runtests: remove ftp2 support, not used - Makes the test servers for HTTP and Gopher pop up on a currently unused - port and runtests adapts to that! + We once supported two separate ftp instances in the test suite. Has not + been used the last decade. - Closes #5247 + Closes #5375 -Marc Hoersken (18 Apr 2020) -- sockfilt: tidy variable naming and data structure in select_ws +- url: sort the protocol schemes in rough popularity order - This commit does not introduce any logical changes to the code. + When looking for a protocol match among supported schemes, check the + most "popular" schemes first. It has zero functionality difference and + for all practical purposes a speed difference will not be measureable + but it still think it makes sense to put the least likely matches last. - Reviewed-by: Jay Satiro and Marcel Raad - Closes #5238 - -Daniel Stenberg (17 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] + "Popularity" based on the 2019 user survey. + + Closes #5377 - libssh: Use new ECDSA key types to check known hosts +Marc Hoersken (11 May 2020) +- test1238: avoid tftpd being busy for tests shortly following - From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA - keys depending on the curve. + The tftpd server may still be busy if the total timeout of + 25 seconds has not been reached or no sread error was received + during or after the execution of the timeout test 1238. - Signed-off-by: Anderson Toshiyuki Sasaki - Fixes #5252 - Closes #5253 - -Marcel Raad (17 Apr 2020) -- appveyor: add Unicode winbuild jobs + Once the next TFTP test comes around (eg. 1242 or 1243), + those will fail because the tftpd server is still waiting + on data from curl due to the UDP protocol being stateless + and having no connection close. On Linux this error may not + happen, because ICMP errors generated due to a swrite error + can also be returned async on the next sread call instead. - These are cheap as they don't build tests. + Therefore we will now just kill the tftpd server after test + 1238 to make sure that the following tests are not affected. - Closes https://github.com/curl/curl/pull/5063 + This enables us to no longer ignore tests 1242, 1243, 2002 + and 2003 on the CI platforms CirrusCI and AppVeyor. + + Assisted-by: Peter Wu + Closes #5364 -Daniel Stenberg (16 Apr 2020) -- mqttd: s/errno/SOCKERRNO +Daniel Stenberg (11 May 2020) +- write-out.d: added "response_code" + +- KNOWN_BUGS: Build with staticly built dependency - To behave proper on Windows - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 - Closes #5241 + I rewrote the item 5.4 to be more generic about static dependencies. -- buildconf: use find -execdir instead, remove -print and the ares files +- ROADMAP: remove old entries - Follow-up to 1e41bec96a6e + MQTT - the start has already landed - Suggested-by: Marc Hörsken + tiny-curl - also mostly landed and is a continuous work + + make menuconfig - basically no interest from users, not pushing there -- [Alexander V. Tikhonov brought this change] +- [Peter Wu brought this change] - buildconf: avoid using tempfile when removing files + travis: Add ngtcp2 and quiche tests for CMake - Closes #5213 + To avoid an explosion of jobs, extend the existing CMake tests with + ngtcp2 and quiche support. macOS was previously moved to GitHub actions, + so the non-Linux case can be dropped. -- copyright: bump the copyright year range +- [Peter Wu brought this change] -- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords + CMake: add ENABLE_ALT_SVC option + + Tested alt-svc with quiche. While at it, add missing MultiSSL reporting + (not tested). -- [JP Mens brought this change] +- [Peter Wu brought this change] - docs/MQTT: replace confusing 80 by 75 + CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) - I was a bit surprised by the `80`: first thought: what's HTTP doing - here? ;) + Add three new CMake Find modules (using the curl license, but I grant + others the right to apply the CMake BSD license instead). - Closes #5236 - -- [Brad King brought this change] + This CMake config is simpler than the autotools one because it assumes + ngtcp2 and nghttp3 to be used together. Another difference is that this + CMake config checks whether QUIC is actually supported by the TLS + library (patched OpenSSL or boringssl) since this can be a common + configuration mistake that could result in build errors later. + + Unlike autotools, CMake does not warn you that the features are + experimental. The user is supposed to already know that and read the + documentation. It requires a very special build environment anyway. + + Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from + current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure + out which features (libldap-2.4, libssh2) to disable due to conflicts + with boringssl. + + Closes #5359 - cmake: Avoid MSVC C4273 warnings in send/recv checks +Marc Hoersken (10 May 2020) +- tests/server/tftpd.c: fix include and enhance debug logging - We use `check_c_source_compiles` to check possible send/recv signatures - by reproducing the forward declarations from system headers. On Windows - the `winsock2.h` header adds dll linkage settings to its forward - declaration. If ours does not match the compiler warns: + setjmp.h should only be included if HAVE_SETJMP_H is defined. - warning C4273: 'recv': inconsistent dll linkage + Add additional log statements to see wether reads and writes + are blocking or finishing before an alarm signal is received. - Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so - that our linkage is consistent with that from `winsock2.h`. + Assisted-by: Peter Wu + Part of #5364 + +Daniel Stenberg (10 May 2020) +- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present - Fixes #4764 - Closes #5232 + Reported-by: Marcel Raad + Follow-up to 148534db5 + Fixes #5367 + Closes #5369 -Jay Satiro (14 Apr 2020) -- KNOWN_BUGS: Add entry 'Blocking socket operations' +Marc Hoersken (9 May 2020) +- appveyor: update comments to be clear about toolchain - - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of - known blocking operations. + - CMake-based MSYS builds use mingw-w64 to cross-compile. + - autotools-based builds are compiled using msys2-devel. - - New known bugs entry 'Blocking socket operations in non-blocking API' - that directs to the TODO's list of known blocking operations. + The difference is that the later ones are not cross-compiled + to Windows and instead require the msys2 runtime to be present. - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 + At the moment only the Azure Pipelines CI builds actually + run autotools-based cross-compilation builds for Windows. + +- TODO: update regarding missing Schannel features - Reported-by: Marc Hoersken + Some aspects have already been implemented over the years. - Closes https://github.com/curl/curl/pull/5216 - -Marc Hoersken (14 Apr 2020) -- test2043: use revoked.badssl.com instead of revoked.grc.com + 15.1 Client certificates are now supported: - The certificate of revoked.grc.com has expired on 2020-04-13. + - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 + - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 - Reviewed-by: Jay Satiro + 15.2 Ciphers can now be specified through: - Closes #5233 - -- sockfilt: fix broken pipe on Windows to be ready in select_ws + - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 - Closes #5228 + Reviewed-by: Daniel Stenberg and Marcel Raad + Closes #5358 -Daniel Stenberg (14 Apr 2020) -- RELEASE-NOTES: synced +Daniel Stenberg (8 May 2020) +- checksrc: close the .checksrc file handle when done reading -- scripts/release-notes: fix duplicate output header +- RELEASE-NOTES: synced + + And bumped next version to 7.71.0 -- github/workflow: enable MQTT in the macOS debug build +- [Gilles Vollant brought this change] -- azure: add mqtt support to one of the Windows builds - -- travis: add mqtt job on Linux - -- tests: add four MQTT tests 1190 - 1193 - -- tests: add the mqtt test server mqttd - -- tests: support hex encoded data and mqtt server + CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) - The mqtt server is started using a "random" port. + Closes #4346 -- [Björn Stenberg brought this change] +- TODO: native IDN support on macOS - mqtt: add new experimental protocol +- urlapi: accept :: as a valid IPv6 address - Closes #5173 - -- TODO: Consider convenience options for JSON and XML? + Text 1560 is extended to verify. - Closes #5203 + Reported-by: Pavel Volgarev + Fixes #5344 + Closes #5351 -- tool: do not declare functions with Curl_ prefix +- THANKS-filter: Peter Wang + +- [Peter Wang brought this change] + + *_sspi: fix bad uses of CURLE_NOT_BUILT_IN - To avoid collision risks with private libcurl symbols when linked with - static versions (or just versions not hiding internal symbols). + Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other + instances of QuerySecurityPackageInfo failing, as in + commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. - Reported-by: hydra3333 on github - Fixes #5219 - Closes #5234 + Closes #5355 -- [Nathaniel R. Lewis brought this change] +- docs/HTTP3: add qlog to the quiche build instruction - cmake: add aliases so exported target names are available in tree +- ngtcp2: introduce qlog support - Reviewed-by: Brad King - Closes #5206 - -- version: increase buffer space for ssl version output + If the QLOGDIR environment variable is set, enable qlogging. - To avoid it getting truncated, especially when several SSL backends are - built-in. + ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for + QUIC functions that are backend independent. - Reported-by: Gisle Vanem - Fixes #5222 - Closes #5226 + Closes #5353 -Marc Hoersken (13 Apr 2020) -- cirrus: no longer ignore test 504 which is working again +- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN - The test is working again, because TCP blackholing is disabled. - -- appveyor: completely disable tests that fail to timeout early + That return code is reserved for build-time conditional code not being + present while this was a regular run-time error from a Windows API. - The tests changed from ignored to disabled are tests that are - about connecting to non-listening socket. On AppVeyor these - tests are not reliable, because for some unknown reason the - connect is not timing out before the test time limit is reached. + Reported-by: wangp on github + Fixes #5349 + Closes #5350 -Daniel Stenberg (13 Apr 2020) -- test1908: avoid using fixed port number in test data - - Closes #5225 +- runtests: show elapsed test time with higher precision (ms) -Jay Satiro (12 Apr 2020) -- [Andrew Kurushin brought this change] +- RELEASE-NOTES: synced - schannel: Fix blocking timeout logic - - - Fix schannel_send for the case when no timeout was set. +- http2: simplify and clean up trailer handling - Prior to this change schannel would error if the socket was not ready - to send data and no timeout was set. + Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in + ed35d6590e72. This should make the trailer handling more straight forward and + hopefully less error-prone. - This commit is similar to parent commit 89dc6e0 which recently made the - same change for SOCKS, for the same reason. Basically it was not well - understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms - but actually means no timeout. + Deliver the trailer header to the callback already at receive-time. No + longer caches the trailers to get delivered at end of stream. - Fixes https://github.com/curl/curl/issues/5177 - Closes https://github.com/curl/curl/pull/5221 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 + Closes #5348 -- socks: Fix blocking timeout logic +Marc Hoersken (7 May 2020) +- appveyor: disable test 1139 instead of ignoring it - - Document in Curl_timeleft's comment block that returning 0 signals no - timeout (ie there's infinite time left). + Spending time on manpage checking makes no sense + for these builds due to lacking manpage support. + +- appveyor: disable flaky test 1501 and ignore broken 1056 - - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. + Test 1501 is flaky on Windows CI due to being time sensitive + and the testsuite relying on taskkill.exe to check for the + existance of processes which can take to much time itself. - Prior to this change if the timeout had a value of 0 and that was passed - to SOCKET_READABLE it would return right away instead of blocking. That - was likely because it was not well understood that when Curl_timeleft - returns 0 it is not a timeout of 0 ms but actually means no timeout. + Test 1056 is broken in autotools-based Windows builds due + to scope ID support missing in these builds at the moment. + +- test613.pl: make tests 613 and 614 work with OpenSSH for Windows - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + OpenSSH for Windows shows group and other/world permissions as *, + because those concepts do not exist on Windows. It also does not + show the current or parent directory, so we just ignore those. - Closes https://github.com/curl/curl/pull/5220 + Reviewed-by: Daniel Stenberg + Closes #5328 -- [Marc Hoersken brought this change] +Daniel Stenberg (6 May 2020) +- runtests: set +x mode again - gopher: check remaining time left during write busy loop - - Prior to this change gopher's blocking code would block forever, - ignoring any set timeout value. +- libssh2: convert over to use dynbuf - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + In my very basic test that lists sftp://127.0.0.1/tmp/, this patched + code makes 161 allocations compared to 194 in git master. A 17% + reduction. - Similar to #5220 and #5221 - Closes #5214 + Closes #5336 -Daniel Stenberg (13 Apr 2020) -- [Dirkjan Bussink brought this change] +- travis: add "qlog" as feature in the quiche build - gnutls: ensure TLS 1.3 when SRP isn't requested +- quiche: enable qlog output - When SRP is requested in the priority string, GnuTLS will disable - support for TLS 1.3. Before this change, curl would always add +SRP to - the priority list, effectively always disabling TLS 1.3 support. + quiche has the potential to log qlog files. To enable this, you must + build quiche with the qlog feature enabled `cargo build --features + qlog`. curl then passes a file descriptor to quiche, which takes + ownership of the file. The FD transfer only works on UNIX. - With this change, +SRP is only added to the priority list when SRP - authentication is also requested. This also allows updating the error - handling here to not have to retry without SRP. This is because SRP is - only added when requested and in that case a retry is not needed. + The convention is to enable logging when the QLOGDIR environment is + set. This should be a path to a folder where files are written with the + naming template .qlog. - Closes #5223 + Co-authored-by: Lucas Pardue + Replaces #5337 + Closes #5341 -Marc Hoersken (12 Apr 2020) -- tests/server: add hidden window to gracefully handle WM_CLOSE +- urldata.h: remove #define HEADERSIZE, not used anymore - Forward Window events as signals to existing signal event handler. + Follow-up to ed35d6590e72c -- tests/server: add CTRL event handler for Win32 consoles +- ngtcp2: convert to dynbuf - Forward CTRL events as signals to existing signal event handler. + Closes #5335 -- tests/server: move all signal handling routines to util.[ch] +- connect: make happy eyeballs work for QUIC (again) - Avoid code duplication to prepare for portability enhancements. - -Daniel Stenberg (12 Apr 2020) -- compressed.d: stress that the headers are not modified + Follow-up from dbd16c3e256c6c (regression in 7.70.0) - Suggested-by: Michael Osipov - Assisted-by: Jay Satiro - Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 - Closes #5217 + Closes #5334 -Marc Hoersken (11 Apr 2020) -- tests/server/util.c: use curl_off_t instead of long for pid - - Avoid potential overflow of huge PIDs on Windows. +- connect: add two asserts to clue code analyzers in a little + +- http_proxy: ported to use dynbuf instead of a static size buffer - Related to #5188 - Assisted-by: Marcel Raad + Removes a 16K static buffer from the easy handle. Simplifies the code. -- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows +- dynbuf: introduce internal generic dynamic buffer functions - Since the Windows versions of both programs would write Windows - PIDs to their pidfiles which we cannot handle, we need to use - our known perl.exe Cygwin/msys PID together with exec() in order - to tie the spawned processes to the existance of our perl.exe + A common set of functions instead of many separate implementations for + creating buffers that can grow when appending data to them. Existing + functionality has been ported over. - The perl.exe that is executing secureserver.pl and sshserver.pl - has a Cygwin/msys PID, because it is started inside Cygwin/msys. + In my early basic testing, the total number of allocations seem at + roughly the same amount as before, possibly a few less. - Related to #5188 - -- tests: add Windows compatible pidwait like pidkill and pidterm + See docs/DYNBUF.md for a description of the API. - Related to #5188 + Closes #5300 -- tests: fix conflict between Cygwin/msys and Windows PIDs +- runtests: remove sleep calls - Add 65536 to Windows PIDs to allow Windows specific treatment - by having disjunct ranges for Cygwin/msys and Windows PIDs. + Remove many one second sleeps that were done *after* each newly started + test server already has been verified. They should not have any purpose + there. - See also: - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=b5e1003722cb14235c4f166be72c09acdffc62ea - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe + Closes #5323 + +- asyn-*: remove support for never-used NULL entry pointers - Replaces #5178 - Closes #5188 + ... and instead convert those to asserts to make sure they are truly + never NULL. + + Closes #5324 -Daniel Stenberg (11 Apr 2020) -- RELEASE-NOTES: synced +- [Emil Engler brought this change] -- release-notes.pl: detect the start of the references in cleanup mode + doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax + + Closes #5325 -- Revert "file: on Windows, refuse paths that start with \\" +Jay Satiro (2 May 2020) +- asyn-thread: fix cppcheck warning - This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. + - Check for NULL entry parameter before attempting to deref entry in + Curl_resolver_is_resolved, like is already done in asyn-ares. - Reminded-by: Chris Roberts - Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html + This is to silence cppcheck which does not seem to understand that + asyn-ares and asyn-thread have separate Curl_resolver_is_resolved + and those units are mutually exclusive. Prior to this change it warned + of a scenario where asyn-thread's Curl_resolver_is_resolved is called + with a NULL entry from asyn-ares, but that couldn't happen. - Closes #5215 + Reported-by: rl1987@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/5326 -Jay Satiro (11 Apr 2020) -- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE +- select: fix overflow protection in Curl_socket_check - - If loss of data may occur converting a timediff_t to time_t and - the time value is > TIME_T_MAX then treat it as TIME_T_MAX. + Follow-up to a96c752 which changed the timeout_ms type from time_t to + timediff_t. - This is a follow-up to 8843678 which removed the (time_t) typecast - from the macros so that conversion warnings could be identified. + Ref: https://github.com/curl/curl/pull/5240 - Closes https://github.com/curl/curl/pull/5199 + Closes https://github.com/curl/curl/pull/5286 -- test1148: tolerate progress updates better (again) +Marc Hoersken (2 May 2020) +- sockfilt: make select_ws stop waiting on exit signal event - - Ignore intermediate progress updates. + This makes sure that select_ws behaves similar to real select + which stops waiting on a signal handler being triggered. - - Support locales that use a character other than period as decimal - separator (eg 100,0%). + This makes it possible to gracefully stop sockfilt.exe on + Windows with taskkill /IM sockfilt.exe (without /F force flag). - test1148 checks that the progress finishes at 100% and has the right - bar width. Prior to this change the test assumed that the only progress - reported for such a quick transfer was 100%, however in rare instances - (like in the CI where transfer time can slow considerably) there may be - intermediate updates. For example, below is stderrlog1148 from a failed - CI run with explicit \r and \n added (it is one line; broken up so that - it's easier to understand). + Reviewed-by: Jay Satiro + Part of #5260 + +- tests/server/util.[ch]: add exit event to stop waiting on Windows - \r - \r################################## 48.3% - \r######################################################################## 100.0% - \n + This commit adds a global exit event to the test servers that + Windows-specific wait routines can use to get triggered if the + program was signaled to be terminated, eg. select_ws in sockfilt.c - Closes https://github.com/curl/curl/pull/5194 - -Marc Hoersken (10 Apr 2020) -- sshserver.pl: use cached Win32 environment check variable - -- appveyor: partially revert 3413a110 to keep build without proxy + The exit event will be managed by the signal handling code and is + set to not reset automatically to support multiple wait routines. - Ref: #5211 and #4526 - Reported-by: Marcel Raad + Reviewed-by: Jay Satiro + Closes #5260 -- appveyor: ignore failing 'connect to non-listening proxy' tests +- tests/server/util.c: fix thread handle not being closed - Closes #5211 - -- CI/macos: convert CRLF to LF and align indentation + Reviewed-by: Jay Satiro + Part of #5260 -Daniel Stenberg (9 Apr 2020) -- url: allow non-HTTPS altsvc-matching for debug builds +- tests/server/util.c: use raise instead of calling signal handler - This is already partly supported but this part was missing. - Reported-by: James Fuller + Use raise to trigger signal handler instead of calling it + directly and causing potential unexpected control flow. - Closes #5205 + Reviewed-by: Jay Satiro + Part of #5260 -- server/resolve: remove AI_CANONNAME to make macos tell the truth - - With this bit set, my mac successfully resolves "ip6-localhost" when in - fact there is no such host known to my machine! That in turn made test - 241 wrongly execute and fail. +- tests: add support for SSH server variant specific transfer paths - Closes #5202 - -- runtests: fix warning about using an undefined variable + OpenSSH for Windows requires paths in the format of /C:/ + instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ - Follow-up from 4d939ef6ceb2db1 + Reviewed-by: Daniel Stenberg + Closes #5298 -- release-notes: fix the initial reference list output +Daniel Stenberg (2 May 2020) +- RELEASE-NOTES: synced -- github actions: run when pushed to master or */ci + PRs +- libssh2: set the expected total size in SCP upload init - Avoid double-builds when using "local" branches for PRs. For both macos - and fuzz jobs. + ... as otherwise the progress callback gets called without that + information, making the progress meter have less info. - Closes #5201 - -- runtests: provide nicer errormsg when protocol "dump" file is empty + Reported-by: Murugan Balraj + Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html + Closes #5317 -- [Gilles Vollant brought this change] +- runtests: make the logmsg from the ssh server only show in verbose - schannel: support .P12 or .PFX client certificates +- tests: make test 1248 + 1249 use %NOLISTENPORT - Used with curl command line option like this: --cert - : --cert-type p12 + ... instead of a port of a non-running server so that it works + stand-alone. - Closes #5193 + Closes #5318 -- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - - test1294: "split request" being when the entire request isn't sent in - the first go, and the remainder is sent in the PERFORM state. A GET - request is otherwise not sending anything during PERFORM. - - test1295: same kind of split but with POST +- examples: remove asiohiper.cpp - Closes #5197 - -- http: don't consider upload done if the request isn't completely sent off + This example has repeatedly been reported to contain bugs, and as users + copy and paste code from this into production, I now deem it better to + not provide the example at all. - Fixes #4919 - Closes #5197 + Closes #5090 + Closes #5322 -- http: allow Curl_add_buffer_send() to do a short first send by force - - In a debug build, settting the environment variable "CURL_SMALLREQSEND" - will make the first HTTP request send not send more bytes than the set - amount, thus ending up verifying that the logic for handling a split - HTTP request send works correctly. +- [Emil Engler brought this change] -- connect: store connection info for QUIC connections - - Restores the --head functionality to the curl utility which extracts - 'protocol' that is stored that way. + doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 - Reported-by: James Fuller - Fixes #5196 - Closes #5198 + Closes #5320 -- tests/README: update the port numbers list - - Since the pipelining server is long gone. - Reported-by: James Fuller +- [Emil Engler brought this change] -- select: remove typecast from SOCKET_WRITABLE/READABLE macros + KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" - So that they don't hide conversions-by-mistake + It was fixed in 9a2cbf3 - Reviewed-by: Jay Satiro - Closes #5190 + Closes #5319 -- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also +- cirrus: disable SFTP and SCP tests - Closes #5192 - -- release-notes: output trailing references sorted numerically + ... as we can't seem to start the sshd server on it. Those problems + existed before d1239b50bececd (running the SSH server on a random port), + but they're more noticable now since there are more failed attempts in + the logs. + + Closes #5315 -- cleanup: correct copyright year range on a few files +- [Emil Engler brought this change] -- configure: remove use of -vec-report0 from CFLAGS with icc + runtests: fix typo in the existence of disabled tests checker - ... as it apparently isn't (always) supported. - Reported-by: Alain Miniussi - Fixes #5096 - Closes #5191 + Closes #5316 -- warnless: remove code block for icc that didn't work +Dan Fandrich (30 Apr 2020) +- test75: Remove precheck test - Reported-by: Alain Miniussi - Fixes #5096 + This has not been needed since commit 9fa42bed and often prevents it + from running at all with dynamic test ports. -Marc Hoersken (6 Apr 2020) -- dist: add missing setup-win32.h +- tests: Stop referring to server ports when they're not used - Follow up to d820224b8b + Several tests referred to specific server ports even when the test + didn't actually use that server or specify that it's needed. In such + cases, the test harness substitutes the text "[not running]" as the port + number which causes many such tests to fail due to the inability to + parse the URL. These tests are changed to use %NOLISTENPORT which will + always be substituted correctly. -Daniel Stenberg (6 Apr 2020) -- RELEASE-NOTES: synced +Daniel Stenberg (30 Apr 2020) +- [Emil Engler brought this change] -- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance + GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT - This script helps putting entries in the RELEASE-NOTES using a coherent - style and sorting with a minimal human editing effort - as long as the - first line in the commit message is good enough! There's a short howto - at the top of the file. - -- [Dennis Felsing brought this change] + Closes #5287 - configure: don't check for Security.framework when cross-compiling - - Since it checks for the local file, not the cross-compiled one. +- conncache: various concept cleanups - Closes #5189 - -- TODO: Option to make -Z merge lined based outputs on stdout + More connection cache accesses are protected by locks. - Closes #5175 - -- lib: never define CURL_CA_BUNDLE with a getenv + CONNCACHE_* is a beter prefix for the connection cache lock macros. - - it breaks the build (since 6de756c9b1de34b7a1) - - it's not documented and not consistent across platforms - - the curl tool does that getenv magic + Curl_attach_connnection: now called as soon as there's a connection + struct available and before the connection is added to the connection + cache. - Bug: https://github.com/curl/curl/commit/6de756c#r38127030 - Reported-by: Gisle Vanem + Curl_disconnect: now assumes that the connection is already removed from + the connection cache. - Closes #5187 - -Marc Hoersken (5 Apr 2020) -- lib670: use the same Win32 API check as all other lib tests + Ref: #4915 + Closes #5009 -- appveyor: use random test server ports based upon APPVEYOR_API_URL +- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports - Avoid conflicts of test server ports with AppVeyor API on localhost. + As stunnel is an external tool and it has no specific option to export + the actually used port number when asked to listen to 0, runtests + instead iterates over ten randomly picked high number ports and sticks + to the first one stunnel can listen to. - Closes #5034 + Closes #5267 -- appveyor: sort builds by type and add two new variants +- tests: pick a random port number for SSH - Related to #5034 and #5063 - -- appveyor: show failed tests in log even if test is ignored + Since sshd doesn't have such an option by itself, we iterate over a + series of random ports until one works. - And print API response with newline only if there is one + Closes #5273 -- appveyor: turn disabled tests into ignored result tests +- [Rikard Falkeborn brought this change] -Daniel Stenberg (5 Apr 2020) -- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" - - Fixed with #5170 (commit 23a870f2fd041278) - -- test1566: verify --etag-compare that gets a 304 back + libtest/cmake: Remove commented code - Verifies the fix in #5183 + These were commented out in e9dd0998706a when Makefile.inc was included + instead. 11 years have passed since then and the commented code is of + course very outdated. Remove it to avoid confusion. - Closes #5186 - -- [Kwon-Young Choi brought this change] + Closes #5311 - CURLINFO_CONDITION_UNMET: return true for 304 http status code +- schannel: source code reindent - In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the - output file if the server did not transfered a file based on time - condition. In the same manner, getting a 304 HTTP response back from the - server, for example after passing a custom If-Match-* header, also - fulfill this condition. + White space edits only. Conform better to standard curl source code + indenting style. - Fixes #5181 - Closes #5183 - -- [Kwon-Young Choi brought this change] + Closes #5305 - curl: allow both --etag-compare and --etag-save with same file name - - This change inverse the order of processing for the --etag-compare and - --etag-save option to process first --etag-compare. This in turn allows - to use the same file name to compare and save an etag. +Kamil Dudka (29 Apr 2020) +- test1177: look for curl.h in source directory - The original behavior of not failing if the etag file does not exists is - conserved. + If we use a separate build directory, there is no copy of the header. - Fixes #5179 - Closes #5180 + Closes #5310 -Viktor Szakats (4 Apr 2020) -- windows: enable UnixSockets with all build toolchains +- tests: look for preprocessed tests in build directory - Extend existing unix socket support in Windows builds to be - enabled for all toolchain vendors or versions. (Previously - it was only supported with certain MSVC versions + more recent - Windows 10 SDKs) + ... which is not always the same directory as source directory - Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ - Ref: https://github.com/curl/curl/issues/5162 - Closes: https://github.com/curl/curl/pull/5170 + Closes #5310 -Daniel Stenberg (4 Apr 2020) -- KNOWN_BUGS: Store TLS context per transfer instead of per connection +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: synced - Closes #5102 + ... and bumped curlver.h to 7.70.1 -Marc Hoersken (3 Apr 2020) -- sockfilt: remove redundancy in timeout handling - - And update other logmsg output in select_ws on Windows. +Version 7.70.0 (29 Apr 2020) -- sockfilt: fix handling of ready closed sockets on Windows - - Replace the incomplete workaround regarding FD_CLOSE - only signalling once by instead doing a pre-check with - standard select and storing the result for later use. - - select keeps triggering on closed sockets on Windows while - WSAEventSelect fires only once with data still available. - By doing the pre-check we do not run in a deadlock - due to waiting forever for another FD_CLOSE event. +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: 7.70.0 -- sockfilt: fix race-condition of waiting threads and event handling - - Fix race-condition of waiting threads finishing while events are - already being processed which lead to invalid or skipped events. - - Use mutex to check for one event at a time or do post-processing. - In addition to mutex-based locking use specific event as signal. - - Closes #5156 +- THANKS: synced with the 7.70.0 release -Daniel Stenberg (2 Apr 2020) -- [Leo Neat brought this change] +- headers: copyright range fix - CI-fuzz: increase fuzz time to 40 minutes - - Closes #5174 +- [Rikard Falkeborn brought this change] -Marc Hoersken (2 Apr 2020) -- CI: increase Azure Pipelines timeouts due to performance issues + doh: Constify some input pointers - The current demand on Azure negatively impacts the CI performance. - -- runtests.pl: log host OS as detected by Perl environment - -- ftpserver.pl: log before and after data connection is closed - -Daniel Stenberg (1 Apr 2020) -- RELEASE-NOTES: synced - -- RELEASE-PROCEDURE.md: run the copyright.pl script! + Closes #5306 -- vquic/ngtcp2.h: update copyright year range +- nss: check for PK11_CreateDigestContext() returning NULL - Follow-up to 0736ee73d346a52 + ... to avoid crashes! + + Reported-by: Hao Wu + Fixes #5302 + Closes #5303 -- [Daiki Ueno brought this change] +- travis: bump the wolfssl CI build to use 4.4.0 + + Closes #5301 - CI: add build with ngtcp2 + gnutls on Travis CI +- copyright updates: adjust year ranges -- [Daiki Ueno brought this change] +Marc Hoersken (26 Apr 2020) +- CI: do not include */ci branches in PR builds + + Align Azure Pipelines with GitHub Actions. - vquic: add support for GnuTLS backend of ngtcp2 +Daniel Stenberg (25 Apr 2020) +- runtests: check for the disabled tests relative srcdir - Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile - time. Therefore OpenSSL support needs to be explicitly disabled. + To make it work correctly for out-of-tree builds. - Signed-off-by: Daiki Ueno - Closes #5148 - -- [Gisle Vanem brought this change] - - examples/sessioninfo.c: add include to fix compiler warning + Follow-up to 75e8feb6fb08b - Fixes #5171 + Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 + Reported-by: Marcel Raad + Closes #5297 -- misc: copyright year updates +- runtests: revert commenting out a line I did for debugging - Follow-up to 7a71965e9 - -- [Harry Sintonen brought this change] + Follow-up to 11091cd4d. It was not meant to be pushed! - build: fixed build for systems with select() in unistd.h +- smtp: set auth correctly - Closes #5169 - -- memdebug: don't log free(NULL) + Regression since 7.69.0 and 68fb25fa3fcff. - ... it serves no purpose and fills up the log. - -- cleanup: insert newline after if() conditions + The code wrongly assigned 'from' instead of 'auth' which probably was a + copy and paste mistake from other code, leading to that auth could + remain NULL and later cause an error to be returned. - Our code style mandates we put the conditional block on a separate - line. These mistakes are now detected by the updated checksrc. + Assisted-by: Eric Sauvageau + Fixes #5294 + Closes #5295 -- checksrc: warn on obvious conditional blocks on the same line as if() +Marcel Raad (25 Apr 2020) +- lib: clean up whitespace - Closes #5164 + This fixes CodeFactor warnings. -- [Roger Orr brought this change] +Daniel Stenberg (25 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] - cmake: add CMAKE_MSVC_RUNTIME_LIBRARY + libssh: avoid options override by configuration files - Fixes #5165 - Closes #5167 - -- [Daiki Ueno brought this change] + Previously, options set explicitly through command line options could be + overridden by the configuration files parsed automatically when + ssh_connect() was called. + + By calling ssh_options_parse_config() explicitly, the configuration + files are parsed before setting the options, avoiding the options + override. Once the configuration files are parsed, the automatic + configuration parsing is not executed. + + Fixes #4972 + Closes #5283 + Signed-off-by: Anderson Toshiyuki Sasaki - ngtcp2: update to git master for the key installation API change +- runtests: when mentions http, kill http/2 too - This updates the ngtcp2 OpenSSL backend to follow the API change in - commit 32e703164 of ngtcp2. + Since the http2 test server is a mere proxy that needs to know about the + dynamic port the HTTP server is using, it too needs to get restarted + when the http server is killed. - Notable changes are: - - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace - ngtcp2_crypto_derive_and_install_key - - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key - has been removed + A regression caused by 80d6515. - Fixes #5166 - Closes #5168 + Fixes #5289 + Closes #5291 -- [Cyrus brought this change] +- [Yuri Slobodyanyuk brought this change] - SECURITY.md: minor rephrase + docs: fix two typos - Closes #5158 + Closes #5292 -- output.d: quote the URL when globbing - - Some shells do globbing of their own unless the URL is quoted, so maybe - encourage this. +- [Emil Engler brought this change] + + tests/git: ignore mqttd and port files - Co-authored-by: Jay Satiro - Closes #5160 + Closes #5290 -- dist: add tests/version-scan.pl to tarball +- tests: make runtests check that disabled tests exists - ... used in test 1177. + ... and error out if so. Removed '536' from DISABLED as there is no such + test file. - Follow-up to a97d826f6de3 + Closes #5288 -- test1177: verify that all the CURL_VERSION_ bits are documented +- test1154: set a proper name -- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented +- select: make Curl_socket_check take timediff_t timeout - Considered experimental and therefore we can do this. + Coverity found CID 1461718: - Closes #5157 - -- KNOWN_BUGS: DoH doesn't inherit all transfer options + Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > + 9223372036854775807L" is always false regardless of the values of its + operands. This occurs as the logical second operand of "||". - Closes #4578 - Closes #4579 + Closes #5240 -- KNOWN_BUGS: DoH leaks memory after followlocation - - Closes #4592 +- [i-ky brought this change] -- KNOWN_BUGS: "FTPS needs session reuse" + libcurl-multi.3: added missing full stop - Closes #4654 + Closes #5285 -- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed - -- TODO: Set custom client ip when using haproxy protocol - - Closes #5125 - -Michael Kaufmann (27 Mar 2020) -- writeout_json: Fix data type issues - - Load long values correctly (e.g. for http_code). +Jay Satiro (22 Apr 2020) +- transfer: Switch PUT to GET/HEAD on 303 redirect - Use curl_off_t (not long) for: - - size_download (CURLINFO_SIZE_DOWNLOAD_T) - - size_upload (CURLINFO_SIZE_UPLOAD_T) + Prior to this change if there was a 303 reply to a PUT request then + the subsequent request to respond to that redirect would also be a PUT. + It was determined that was most likely incorrect based on the language + of the RFCs. Basically 303 means "see other" resource, which implies it + is most likely not the same resource, therefore we should not try to PUT + to that different resource. - The unit for these values is bytes/second, not microseconds: - - speed_download (CURLINFO_SPEED_DOWNLOAD_T) - - speed_upload (CURLINFO_SPEED_UPLOAD_T) + Refer to the discussions in #5237 and #5248 for more information. - Fixes #5131 - Closes #5152 + Fixes https://github.com/curl/curl/issues/5237 + Closes https://github.com/curl/curl/pull/5248 -Daniel Stenberg (27 Mar 2020) -- mailmap: fixup a few author names/fields +Daniel Stenberg (22 Apr 2020) +- lib/mk-ca-bundle: skip empty certs - Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and - Svyatoslav Mishyn + Reviewed-by: Emil Engler + Reported-by: Ashwin Metpalli + Fixes #5278 + Closes #5280 -- version: add 'cainfo' and 'capath' to version info struct +- version: skip idn2_check_version() check and add precaution - Suggested-by: Timothe Litt - URL: https://curl.haxx.se/mail/lib-2020-03/0090.html - Reviewed-by: Jay Satiro + A gcc-10's -fanalyze complaint made me spot and do these improvements. - Closes #5150 + Closes #5281 - RELEASE-NOTES: synced -Jay Satiro (26 Mar 2020) -- SSLCERTS.md: Fix example code for setting CA cert file +- [Brian Bergeron brought this change] + + curl.h: update comment typo - Prior to this change the documentation erroneously said use - CURLOPT_CAPATH to set a CA cert file. + "routines with be invoked" -> "routines will be invoked" - Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html - Reported-by: Timothe Litt + Closes #5279 + +- [Emil Engler brought this change] + + GnuTLS: Don't skip really long certificate fields - Closes https://github.com/curl/curl/pull/5151 + Closes #5271 -Marc Hoersken (26 Mar 2020) -- sockfilt: add logmsg output to select_ws_wait_thread on Windows +- gnutls: bump lowest supported version to 3.1.10 - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + GnuTLS 3.1.10 added new functions we want to use. That version was + released on Mar 22, 2013. Removing support for older versions also + greatly simplifies the code. - Closes #5086 + Ref: #5271 + Closes #5276 -Daniel Stenberg (26 Mar 2020) -- docs/make: generate curl.1 from listed files only - - Previously it rendered the page from files matching "*.d" in the correct - directory, which worked fine in git builds when the files were added but - made it easy to forget adding the files to the dist. +- mqtt: make NOSTATE get within the debug name array + +- tests: run the RTSP test server on a dynamic port number - Now, only man page sections listed in DPAGES in Makefile.inc will be - used, thus "forcing" us to update this to get the man page right and get - it included in the dist at the same time. + To avoid port collisions. - Ref: #5146 - Closes #5149 + Closes #5272 -- openssl: adapt to functions marked as deprecated since version 3 +- tests: add %NOLISTENPORT and use it - OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES - functions we use. + The purpose with this variable is to provide a port number that is + reasonably likely to not have a listener on the local host so that tests + can try connect failures against it. It uses port 47 - "reserved" + according to IANA. - Fix the MD4 and SSL_CTX_load_verify_locations warnings. + Updated six tests to use it instead of the previous different ports. - In configure, detect OpenSSL v3 and if so, inhibit the deprecation - warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and - until we rewrite the code to use non-deprecated functions we better - ignore these warnings as they don't help us. + Assisted-by: Emil Engler + Closes #5270 + +- mqtt: remove code with no purpose - Closes #5139 + Detected by Coverity. CID 1462319. + + "The same code is executed when the condition result is true or false, + because the code in the if-then branch and after the if statement is + identical." + + Closes #5275 -- dist: add mail-rcpt-allowfails.d to the tarball +- mqtt: fix Curl_read() error handling while reading remaining length - Reported-by: Maksim Stsepanenka - Reviewed-by: Jat Satiro + Detected by Coverity. CID 1462320. - Closes #5146 + Closes #5274 -- travis: update the ngtcp2 build to use the latest OpenSSL patch +- server/tftpd: fix compiler warning - ... which also makes it OpenSSL 1.1.1d based and not v3. + Follow-up from 369ce38ac1d + Reported-by: Marc Hörsken -Marc Hoersken (24 Mar 2020) -- CI: remove default Ubuntu build from GitHub Actions +- http: free memory when Alt-Used header creation fails due to OOM - We are already running a very similar Ubuntu build on Travis CI. - The macOS variant of this default build is kept on Github Actions. + Reported-by: James Fuller + Fixes #5268 + Closes #5269 -- CI: bring GitHub Actions fuzzing job in line with macOS jobs +Daniel Gustafsson (20 Apr 2020) +- lib: fix typos in comments and errormessages - Update YAML formatting, job naming and triggers. + This fixes a few randomly spotted typos in recently merged code, most + notably one in a userfacing errormessage the schannel code. -- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions +Daniel Stenberg (20 Apr 2020) +- tests: run the SOCKS test server on a dynamic port number - Reduce workload on Azure Pipelines and Travis CI while - consolidating macOS jobs onto less utilized GitHub Actions. + Closes #5266 + +- [Johannes Schindelin brought this change] + + multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - Reviewed-by: Daniel Stenberg + When cURL is compiled with support for multiple SSL backends, it is + possible to configure an SSL backend via `curl_global_sslset()`, but + only *before* `curl_global_init()` was called. - Closes #5124 - -Daniel Stenberg (24 Mar 2020) -- config: remove all defines of HAVE_DES_H + If another SSL backend should be used after that, a user might be + tempted to call `curl_global_cleanup()` to start over. However, we did + not foresee that use case and forgot to reset the SSL backend in that + cleanup. - As there's no code using it. + Let's allow that use case. - Closes #5144 + Fixes #5255 + Closes #5257 + Reported-by: davidedec on github + Signed-off-by: Johannes Schindelin -- copyright: fix out-of-date copyright ranges and missing headers - - Reported by the new script 'scripts/copyright.pl'. The script has a - regex whitelist for the files that don't need copyright headers. +- tests: run the TFTP test server on a dynamic port number - Removed three (mostly usesless) README files from docs/ + Picking a dynamic unused port is better than a fixed to avoid the + collision risk. - Closes #5141 + Closes #5265 -- packages: add OS400/chkstrings.c to the dist +- mqtt: improve the state machine - Reported-by: Jon Rumsey - Fixes #5142 - Closes #5143 - -- [Clément Notin brought this change] - - nghttp2: 1.12.0 required + To handle PUBLISH before SUBACK and more. - since nghttp2_session_set_local_window_size is needed + Updated the existing tests and added three new ones. - Closes #5140 + Reported-by: Christoph Krey + Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html + Closes #5246 + +- runtests: always put test number in servercmd file - RELEASE-NOTES: synced -- [Calvin Buckley brought this change] +- release-notes.pl: fix parsing typo - OS400: Update strings for ccsid-ifier - - Fixes build. +James Fuller (20 Apr 2020) +- ensure all references to ports are replaced by vars + +- add more alt-svc test coverage + +Daniel Stenberg (20 Apr 2020) +- test1247: use http server to get the port number set - Closes #5132 + Follow-up to 0f5db7b263f -- cirrus: make freebsd ignore the tests instead of skipping +- runtests: use a unix domain socket path with the pid in the name - To allow us to see in the CI logs how they actually behave + To make it impossible for test cases to access the file name without + using the proper variable for the purpose. - Closes #5091 + Closes #5264 -- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 +Daniel Gustafsson (19 Apr 2020) +- [Tom brought this change] -- Revert "cirrus-ci: disable the FreeBSD 13 builds" + src: Remove C99 constructs to ensure C89 compliance - This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. - -- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override + This fixes the error: 'for' loop initial declaration used outside C99 + mode by declaring the loop increment variable in the beginning of the + block instead of inside the for loop. - To let debug-builds return fake values, like in test 970. + Fixes #5254 + Reviewed-by: Daniel Gustafsson + +Daniel Stenberg (19 Apr 2020) +- runtests: dummy init the ports variables to avoid warnings - Ref: #5131 - Closes #5136 + ... and generate something that can help debug test cases. -- test970: improve the test +- [Patrick Monnerat brought this change] + + mime: properly check Content-Type even if it has parameters - - send more data to make problems more obvious - - don't start the data with minus, it makes diffs harder to read - - skip the headers in the stdout comparison - - save to a file name to also verify 'filename_effective' + New test 669 checks this fix is effective. - Ref: #5131 + Fixes #5256 + Closes #5258 + Reported-by: thanhchungbtc on github -- CURLINFO_NUM_CONNECTS: improve accuracy +- tests/FILEFORMAT: converted to markdown and extended - The counter was not bumped in all cases correctly. - - Reported-by: Marcel Raad - Ref: #5131 - Closes #5135 + Closes #5261 -- TODO: Use "random" ports for the test servers +- test1245: make it work with dynamic FTP server port -- lib/curl_setup: adjust the copyright year range - - Follow-up from d820224b8 +- test1055: make it work with dynamic FTP port -Jay Satiro (21 Mar 2020) -- curl_setup: define _WIN32_WINNT_[OS] symbols +- test1028: make it run on dynamic FTP server port + +- tests: move pingpong server to dynamic listening port - .. because not all Windows build systems have those symbols, and even - those that do may be missing newer symbols (eg the Windows 7 SDK does - not define _WIN32_WINNT_WIN10). + FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic + ports - Those symbols are used in build-time logic to decide which API to use - and prior to this change if the symbols were missing it would have - resulted in deprecated API being used when more recent functions were - available (eg GetVersionEx used instead of VerifyVersionInfo). + Test 842-845 are unfortunately a bit hard to move over to this concept + right now and require "default port" still... + +- test1056: work with dynamic HTTP ipv6 port + +- test1448: work with dynamic HTTP server port + +- tests: introduce preprocessed test cases - Reported-by: FuccDucc@users.noreply.github.com + The runtests script now always performs variable replacement on the + entire test source file before the test gets executed, and saves the + updated version in a temporary file (log/test[num]) so that all test + case readers/servers can use that version (if present) and thus enjoy + the powers of test case variable substitution. - Probably fixes https://github.com/curl/curl/issues/4995 - Closes https://github.com/curl/curl/pull/5057 + This is necessary to allow complete port number freedom. + + Test 309 is updated to work with a non-fixed port number thanks to this. -- [Ross Burton brought this change] +- tests: make 2006-2010 handle different port number lengths - curl-functions.m4: remove inappropriate AC_REQUIRE - - AC_REQUIRE means "if this macro hasn't been executed already, execute - it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) - isn't correct at that will execute AC_RUN_IFELSE without any arguments. +- tests: run the sws server on "any port" - With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, - AC_RUN_IFELSE without a default value when cross-compiling is fatal. - The result is that curl with autoconf 2.70 cannot cross-compile. + Makes the test servers for HTTP and Gopher pop up on a currently unused + port and runtests adapts to that! - Fixes https://github.com/curl/curl/issues/5126 - Closes https://github.com/curl/curl/pull/5130 + Closes #5247 -Marc Hoersken (20 Mar 2020) -- ci/tests: fix Azure Pipelines not running Windows containers +Marc Hoersken (18 Apr 2020) +- sockfilt: tidy variable naming and data structure in select_ws - Workaround posted here: microsoft/azure-pipelines-agent#2864 + This commit does not introduce any logical changes to the code. - Assisted-by: Simon Chalifoux - Assisted-by: Tommy Petty + Reviewed-by: Jay Satiro and Marcel Raad + Closes #5238 + +Daniel Stenberg (17 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] + + libssh: Use new ECDSA key types to check known hosts - Fixes #5117 - Closes #5129 + From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA + keys depending on the curve. + + Signed-off-by: Anderson Toshiyuki Sasaki + Fixes #5252 + Closes #5253 -Daniel Stenberg (20 Mar 2020) -- tests: add test 430, 431 and 432 to verify the --config fix +Marcel Raad (17 Apr 2020) +- appveyor: add Unicode winbuild jobs - Verify the fixes in 4e0b4fee4 + These are cheap as they don't build tests. + + Closes https://github.com/curl/curl/pull/5063 -- [Rici Lake brought this change] +Daniel Stenberg (16 Apr 2020) +- mqttd: s/errno/SOCKERRNO + + To behave proper on Windows + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 + Closes #5241 - cmdline: fix handling of OperationConfig linked list (--next) +- buildconf: use find -execdir instead, remove -print and the ares files - Ensures that -K/--config inserts new items at the end of the list - instead of overwriting the second item, and that after a -K/--config - option has been parsed, the option parser's view of the current config - is update. + Follow-up to 1e41bec96a6e - Fixes #5120 - Closes #5123 - -Marc Hoersken (20 Mar 2020) -- test2100: fix static port instead of dynamic value being used + Suggested-by: Marc Hörsken -- test970: fix static ip:port instead of dynamic values being used +- [Alexander V. Tikhonov brought this change] -Daniel Stenberg (19 Mar 2020) -- secure transport: remove the BACKEND define kludge + buildconf: avoid using tempfile when removing files - Closes #5122 + Closes #5213 -- mbedtls: remove the BACKEND define kludge +- copyright: bump the copyright year range -- bearssl: remove the BACKEND define kludge +- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords -- wolfssl: remove the BACKEND define kludge +- [JP Mens brought this change] -- nss: remove the BACKEND define kludge + docs/MQTT: replace confusing 80 by 75 + + I was a bit surprised by the `80`: first thought: what's HTTP doing + here? ;) + + Closes #5236 -- gnutls: remove the BACKEND define kludge +- [Brad King brought this change] -- openssl: remove the BACKEND define kludge + cmake: Avoid MSVC C4273 warnings in send/recv checks - Use a proper variable instead to make it easier to use a debugger and - read the code. + We use `check_c_source_compiles` to check possible send/recv signatures + by reproducing the forward declarations from system headers. On Windows + the `winsock2.h` header adds dll linkage settings to its forward + declaration. If ours does not match the compiler warns: + + warning C4273: 'recv': inconsistent dll linkage + + Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so + that our linkage is consistent with that from `winsock2.h`. + + Fixes #4764 + Closes #5232 -Marc Hoersken (19 Mar 2020) -- tests: make Python-based servers compatible with Python 2 and 3 +Jay Satiro (14 Apr 2020) +- KNOWN_BUGS: Add entry 'Blocking socket operations' - Update smbserver.py and negtelnetserver.py to be compatible with - Python 3 while staying backwards-compatible to support Python 2. + - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of + known blocking operations. - Fix string encoding and handling of echoed and transferred data. + - New known bugs entry 'Blocking socket operations in non-blocking API' + that directs to the TODO's list of known blocking operations. - Tested with both Python 2.7.17 and Python 3.7.7 + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 - Reported-by: Daniel Stenberg - Assisted-by: Kamil Dudka - Reviewed-by: Marcel Raad + Reported-by: Marc Hoersken - Fixes #5104 - Closes #5110 + Closes https://github.com/curl/curl/pull/5216 -Daniel Stenberg (18 Mar 2020) -- writeout_json: use curl_off_t printf() option for the time output +Marc Hoersken (14 Apr 2020) +- test2043: use revoked.badssl.com instead of revoked.grc.com - Follow-up to: 04c03416e68fd635a15 + The certificate of revoked.grc.com has expired on 2020-04-13. - Closes #5115 + Reviewed-by: Jay Satiro + + Closes #5233 -- RELEASE-NOTES: synced +- sockfilt: fix broken pipe on Windows to be ready in select_ws - Uh, I missed this in 1a46b218db + Closes #5228 +Daniel Stenberg (14 Apr 2020) - RELEASE-NOTES: synced + +- scripts/release-notes: fix duplicate output header + +- github/workflow: enable MQTT in the macOS debug build + +- azure: add mqtt support to one of the Windows builds + +- travis: add mqtt job on Linux + +- tests: add four MQTT tests 1190 - 1193 + +- tests: add the mqtt test server mqttd + +- tests: support hex encoded data and mqtt server - ... and bumped curlver.h to 7.70.0 + The mqtt server is started using a "random" port. -Jay Satiro (18 Mar 2020) -- http2: Fix erroneous debug message that h2 connection closed +- [Björn Stenberg brought this change] + + mqtt: add new experimental protocol - Prior to this change in libcurl debug builds http2 stream closure was - erroneously referred to as connection closure. + Closes #5173 + +- TODO: Consider convenience options for JSON and XML? - Before: - * nread <= 0, server closed connection, bailing + Closes #5203 + +- tool: do not declare functions with Curl_ prefix - After: - * nread == 0, stream closed, bailing + To avoid collision risks with private libcurl symbols when linked with + static versions (or just versions not hiding internal symbols). - Closes https://github.com/curl/curl/pull/5118 + Reported-by: hydra3333 on github + Fixes #5219 + Closes #5234 -Daniel Stenberg (18 Mar 2020) -- tool_setopt: correct the copyright year range - - Follow-up to 5450428491 +- [Nathaniel R. Lewis brought this change] -Jay Satiro (18 Mar 2020) -- [Johannes Schindelin brought this change] + cmake: add aliases so exported target names are available in tree + + Reviewed-by: Brad King + Closes #5206 - schannel: add "best effort" revocation check option +- version: increase buffer space for ssl version output - - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and - --ssl-revoke-best-effort to allow a "best effort" revocation check. + To avoid it getting truncated, especially when several SSL backends are + built-in. - A best effort revocation check ignores errors that the revocation check - was unable to take place. The reasoning is described in detail below and - discussed further in the PR. + Reported-by: Gisle Vanem + Fixes #5222 + Closes #5226 + +Marc Hoersken (13 Apr 2020) +- cirrus: no longer ignore test 504 which is working again - --- + The test is working again, because TCP blackholing is disabled. + +- appveyor: completely disable tests that fail to timeout early - When running e.g. with Fiddler, the schannel backend fails with an - unhelpful error message: + The tests changed from ignored to disabled are tests that are + about connecting to non-listening socket. On AppVeyor these + tests are not reliable, because for some unknown reason the + connect is not timing out before the test time limit is reached. + +Daniel Stenberg (13 Apr 2020) +- test1908: avoid using fixed port number in test data - Unknown error (0x80092012) - The revocation function was unable - to check revocation for the certificate. + Closes #5225 + +Jay Satiro (12 Apr 2020) +- [Andrew Kurushin brought this change] + + schannel: Fix blocking timeout logic - Sadly, many enterprise users who are stuck behind MITM proxies suffer - the very same problem. + - Fix schannel_send for the case when no timeout was set. - This has been discussed in plenty of issues: - https://github.com/curl/curl/issues/3727, - https://github.com/curl/curl/issues/264, for example. + Prior to this change schannel would error if the socket was not ready + to send data and no timeout was set. - In the latter, a Microsoft Edge developer even made the case that the - common behavior is to ignore issues when a certificate has no recorded - distribution point for revocation lists, or when the server is offline. - This is also known as "best effort" strategy and addresses the Fiddler - issue. + This commit is similar to parent commit 89dc6e0 which recently made the + same change for SOCKS, for the same reason. Basically it was not well + understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms + but actually means no timeout. - Unfortunately, this strategy was not chosen as the default for schannel - (and is therefore a backend-specific behavior: OpenSSL seems to happily - ignore the offline servers and missing distribution points). + Fixes https://github.com/curl/curl/issues/5177 + Closes https://github.com/curl/curl/pull/5221 + +- socks: Fix blocking timeout logic - To maintain backward-compatibility, we therefore add a new flag - (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option - (`--ssl-revoke-best-effort`) to select the new behavior. + - Document in Curl_timeleft's comment block that returning 0 signals no + timeout (ie there's infinite time left). - Due to the many related issues Git for Windows and GitHub Desktop, the - plan is to make this behavior the default in these software packages. + - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. - The test 2070 was added to verify this behavior, adapted from 310. + Prior to this change if the timeout had a value of 0 and that was passed + to SOCKET_READABLE it would return right away instead of blocking. That + was likely because it was not well understood that when Curl_timeleft + returns 0 it is not a timeout of 0 ms but actually means no timeout. - Based-on-work-by: georgeok - Co-authored-by: Markus Olsson - Signed-off-by: Johannes Schindelin + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 - Closes https://github.com/curl/curl/pull/4981 + Closes https://github.com/curl/curl/pull/5220 -- multi: Improve parameter check for curl_multi_remove_handle +- [Marc Hoersken brought this change] + + gopher: check remaining time left during write busy loop - - If an easy handle is owned by a multi different from the one specified - then return CURLM_BAD_EASY_HANDLE. + Prior to this change gopher's blocking code would block forever, + ignoring any set timeout value. - Prior to this change I assume user error could cause corruption. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5116 + Similar to #5220 and #5221 + Closes #5214 -Viktor Szakats (17 Mar 2020) -- windows: suppress UI in all CryptAcquireContext() calls - - Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters - Reviewed-by: Marc Hörsken - Closes https://github.com/curl/curl/pull/5088 +Daniel Stenberg (13 Apr 2020) +- [Dirkjan Bussink brought this change] -Daniel Stenberg (17 Mar 2020) -- writeout_json: add missing comma to fix the HTTP version + gnutls: ensure TLS 1.3 when SRP isn't requested - Follow-up to 04c03416e68fd635a15 - -- test 970: verify --write-out '%{json}' + When SRP is requested in the priority string, GnuTLS will disable + support for TLS 1.3. Before this change, curl would always add +SRP to + the priority list, effectively always disabling TLS 1.3 support. - Makes curl_easy_getinfo() of "variable" numerical content instead return - the number set in the env variable `CURL_TIME`. + With this change, +SRP is only added to the priority list when SRP + authentication is also requested. This also allows updating the error + handling here to not have to retry without SRP. This is because SRP is + only added when requested and in that case a retry is not needed. - Makes curl_version() of "variable" textual content. This guarantees a - stable version string which can be tested against. Environment variable - `CURL_VERSION` defines the content. + Closes #5223 + +Marc Hoersken (12 Apr 2020) +- tests/server: add hidden window to gracefully handle WM_CLOSE - Assisted-by: Mathias Gumz + Forward Window events as signals to existing signal event handler. -- [Mathias Gumz brought this change] +- tests/server: add CTRL event handler for Win32 consoles + + Forward CTRL events as signals to existing signal event handler. - writeout: support to generate JSON output +- tests/server: move all signal handling routines to util.[ch] - This commit adds support to generate JSON via the writeout feature: + Avoid code duplication to prepare for portability enhancements. + +Daniel Stenberg (12 Apr 2020) +- compressed.d: stress that the headers are not modified - -w "%{json}" + Suggested-by: Michael Osipov + Assisted-by: Jay Satiro + Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 + Closes #5217 + +Marc Hoersken (11 Apr 2020) +- tests/server/util.c: use curl_off_t instead of long for pid - It leverages the existing infrastructure as much as possible. Thus, - generating the JSON on STDERR is possible by: + Avoid potential overflow of huge PIDs on Windows. - -w "%{stderr}%{json}" + Related to #5188 + Assisted-by: Marcel Raad + +- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows - This implements a variant of - https://github.com/curl/curl/wiki/JSON#--write-out-json. + Since the Windows versions of both programs would write Windows + PIDs to their pidfiles which we cannot handle, we need to use + our known perl.exe Cygwin/msys PID together with exec() in order + to tie the spawned processes to the existance of our perl.exe - Closes #4870 + The perl.exe that is executing secureserver.pl and sshserver.pl + has a Cygwin/msys PID, because it is started inside Cygwin/msys. + + Related to #5188 -- CI: stop ignoring 323, it is disabled +- tests: add Windows compatible pidwait like pidkill and pidterm + + Related to #5188 -- DISABLED: disable test 323 +- tests: fix conflict between Cygwin/msys and Windows PIDs - The test uses SRP to "a server not supporting it" but modern stunnel - versions will silently accept it and remain happy. The test is therefore - faulty. + Add 65536 to Windows PIDs to allow Windows specific treatment + by having disjunct ranges for Cygwin/msys and Windows PIDs. - I haven't figured out how to make stunnel explicitly reject SRP-using - connects. + See also: + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=b5e1003722cb14235c4f166be72c09acdffc62ea + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe - Reported-by: Marc Hörsken - Fixes #5105 - Closes #5113 + Replaces #5178 + Closes #5188 -Marc Hoersken (17 Mar 2020) -- ci/tests: increase timeouts for torture builds on Azure Pipelines - - For some reason the torture builds have slowed down recently. - - Reported-by: Daniel Stenberg +Daniel Stenberg (11 Apr 2020) +- RELEASE-NOTES: synced -Daniel Stenberg (16 Mar 2020) -- cmake: add support for building with wolfSSL +- release-notes.pl: detect the start of the references in cleanup mode + +- Revert "file: on Windows, refuse paths that start with \\" - My working build cmdline: + This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. - $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . + Reminded-by: Chris Roberts + Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html - Assisted-by: Brad King - Closes #5095 + Closes #5215 -- tool_operate: fix add_parallel_transfers when more are in queue +Jay Satiro (11 Apr 2020) +- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE - Trying to return early from the function if no new transfers were added - would break the "morep" argument and cause issues. This could lead to - zero content "transfers" (within quotes since they would never be - started) when parallel-max was reduced. + - If loss of data may occur converting a timediff_t to time_t and + the time value is > TIME_T_MAX then treat it as TIME_T_MAX. - Reported-by: Gavin Wong - Analyzed-by: Jay Satiro - Fixes #4937 - Closes #5112 + This is a follow-up to 8843678 which removed the (time_t) typecast + from the macros so that conversion warnings could be identified. + + Closes https://github.com/curl/curl/pull/5199 -- vtls: free ssl_config leftovers on out-of-memory +- test1148: tolerate progress updates better (again) - Torture testing 2034 and 2037 found this. + - Ignore intermediate progress updates. - Reported-by: Marc Hörsken - Fixes #5108 - Closes #5109 - -Marc Hoersken (16 Mar 2020) -- ci/tests: fix Azure Pipelines not running for pull requests + - Support locales that use a character other than period as decimal + separator (eg 100,0%). - Closes #5111 - -Daniel Stenberg (15 Mar 2020) -- gskit: update the copyright year range + test1148 checks that the progress finishes at 100% and has the right + bar width. Prior to this change the test assumed that the only progress + reported for such a quick transfer was 100%, however in rare instances + (like in the CI where transfer time can slow considerably) there may be + intermediate updates. For example, below is stderrlog1148 from a failed + CI run with explicit \r and \n added (it is one line; broken up so that + it's easier to understand). + + \r + \r################################## 48.3% + \r######################################################################## 100.0% + \n - Follow-up from 083603c63a3 + Closes https://github.com/curl/curl/pull/5194 -Marc Hoersken (15 Mar 2020) -- gskit: use our internal select wrapper for portability - - Follow up to c52b342 - Closes #5106 +Marc Hoersken (10 Apr 2020) +- sshserver.pl: use cached Win32 environment check variable -- tests: fix verification of stdout in test 1452 due to newline +- appveyor: partially revert 3413a110 to keep build without proxy - Fixes test1452:41:1: error: missing tag before + Ref: #5211 and #4526 + Reported-by: Marcel Raad -- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI - - Also force the package index/cache to be updated before installing. +- appveyor: ignore failing 'connect to non-listening proxy' tests - Closes #5103 + Closes #5211 -- tests/README: add note about manually installing python-impacket - - Follow up to 4be2560 +- CI/macos: convert CRLF to LF and align indentation -Daniel Stenberg (15 Mar 2020) -- transfer: cap retries of "dead connections" to 5 +Daniel Stenberg (9 Apr 2020) +- url: allow non-HTTPS altsvc-matching for debug builds - When libcurl retries a connection due to it being "seemingly dead" or by - REFUSED_STREAM, it will now only do it up five times before giving up, - to avoid never-ending loops. + This is already partly supported but this part was missing. + Reported-by: James Fuller - Reported-by: Dima Tisnek - Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html - Closes #5074 + Closes #5205 -- TODO: TLS-PSK with OpenSSL +- server/resolve: remove AI_CANONNAME to make macos tell the truth - Closes #5081 - -Marc Hoersken (15 Mar 2020) -- select: add 'timeout_ms' wrap-around precaution to Curl_select - -- select: fix 'pending_ms' is assigned a value that is never used + With this bit set, my mac successfully resolves "ip6-localhost" when in + fact there is no such host known to my machine! That in turn made test + 241 wrongly execute and fail. - Detected by Codacy + Closes #5202 -- select: move duplicate select preparation code into Curl_select +- runtests: fix warning about using an undefined variable - Reviewed by Daniel Stenberg - Reviewed by Marcel Raad - Closes #5078 + Follow-up from 4d939ef6ceb2db1 -Daniel Stenberg (15 Mar 2020) -- connect: happy eyeballs cleanup - - Make sure each separate index in connn->tempaddr[] is used for a fixed - family (and only that family) during the connection process. +- release-notes: fix the initial reference list output + +- github actions: run when pushed to master or */ci + PRs - If family one takes a long time and family two fails immediately, the - previous logic could misbehave and retry the same family two address - repeatedly. + Avoid double-builds when using "local" branches for PRs. For both macos + and fuzz jobs. - Reported-by: Paul Vixie - Reported-by: Jay Satiro - Fixes #5083 - Fixes #4954 - Closes #5089 + Closes #5201 -Marc Hoersken (15 Mar 2020) -- ci/tests: fix and align setting TFLAGS for make test-nonflaky +- runtests: provide nicer errormsg when protocol "dump" file is empty -- ci/tests: install test suite dependencies stunnel and impacket +- [Gilles Vollant brought this change] -- tests: remove python_dependencies for smbserver from our tree - - Users of the SMB tests will have to install impacket manually. + schannel: support .P12 or .PFX client certificates - Reasoning: our in-tree version of impacket was quite outdated - and only compatible with Python 2 which is already end-of-life. - Upgrading to Python 3 and a compatible impacket version would - require to import additional Python-only and CPython-extension - dependencies. This would have hindered portability enormously. + Used with curl command line option like this: --cert + : --cert-type p12 - Closes #5094 + Closes #5193 -Jay Satiro (14 Mar 2020) -- Makefile.m32: Improve windres parameter compatibility - - - s/COFF/coff/ +- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - Some versions of windres do not recognize uppercase COFF as a valid - way to specify the COFF output format. + test1294: "split request" being when the entire request isn't sent in + the first go, and the remainder is sent in the PERFORM state. A GET + request is otherwise not sending anything during PERFORM. - Reported-by: Steven Penny + test1295: same kind of split but with POST - Fixes https://github.com/curl/curl/issues/5099 - Closes https://github.com/curl/curl/pull/5101 + Closes #5197 -- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares - - - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in - curl_easy_duphandle. - - Prior to this change if c-ares was used as the resolver backend and - either it was too old or libcurl was built without IPv6 support then - some of our resolver functions could return CURLE_NOT_BUILT_IN to - curl_easy_duphandle causing it to fail. - - Caused by c8f086b which shipped in 7.69.1. - - Reported-by: Karl Chen +- http: don't consider upload done if the request isn't completely sent off - Fixes https://github.com/curl/curl/issues/5097 - Closes https://github.com/curl/curl/pull/5100 + Fixes #4919 + Closes #5197 -Daniel Stenberg (13 Mar 2020) -- docs: add warnings about FILE: URLs on Windows - - - --url man page section - - libcurl-security.3 gets the full text - - CURLOPT_URL.3 +- http: allow Curl_add_buffer_send() to do a short first send by force - Reported-by: Tim Sedlmeyer + In a debug build, settting the environment variable "CURL_SMALLREQSEND" + will make the first HTTP request send not send more bytes than the set + amount, thus ending up verifying that the logic for handling a split + HTTP request send works correctly. -- server/getpart: make the "XML-parser" stricter +- connect: store connection info for QUIC connections - When extracting a
and there's no before -
, this now outputs an error and returns a wrong string to - make users spot the mistake. + Restores the --head functionality to the curl utility which extracts + 'protocol' that is stored that way. - Ref: #5070 - Closes #5071 + Reported-by: James Fuller + Fixes #5196 + Closes #5198 -Marc Hoersken (13 Mar 2020) -- impacket: some more Python 3 code compatibility updates +- tests/README: update the port numbers list - This makes smbserver load on Python 3, but still not work completely. + Since the pipelining server is long gone. + Reported-by: James Fuller -- smbserver: pin Python version to 2 since we are not yet 3 compatible +- select: remove typecast from SOCKET_WRITABLE/READABLE macros - Even though the existing code can be fixed to run on Python 3, the - tests will fail due to the Unicode transition the protocol is invalid. + So that they don't hide conversions-by-mistake - Follow up to ee63837 - Closes #5085 - -Daniel Stenberg (12 Mar 2020) -- [Viktor Szakats brought this change] + Reviewed-by: Jay Satiro + Closes #5190 - cleanup: fix some text/comment typos +- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also - Closes #5087 + Closes #5192 -Marc Hoersken (12 Mar 2020) -- smbserver: fix Python version specific ConfigParser import +- release-notes: output trailing references sorted numerically + +- cleanup: correct copyright year range on a few files + +- configure: remove use of -vec-report0 from CFLAGS with icc - Follow up to ee63837 and 8c7c4a6 - Fixes #5077 + ... as it apparently isn't (always) supported. + Reported-by: Alain Miniussi + Fixes #5096 + Closes #5191 -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: synced +- warnless: remove code block for icc that didn't work - bumped to 7.69.2 + Reported-by: Alain Miniussi + Fixes #5096 -Dan Fandrich (11 Mar 2020) -- tests/data: Fix some XML formatting issues in test cases +Marc Hoersken (6 Apr 2020) +- dist: add missing setup-win32.h - This allows these test files to pass xmllint. + Follow up to d820224b8b -Daniel Stenberg (11 Mar 2020) -- [Muhammad Herdiansyah brought this change] +Daniel Stenberg (6 Apr 2020) +- RELEASE-NOTES: synced - Makefile: run the cd commands in a subshell - - In bmake, if the directory is changed (with cd or anything else), bmake - won't return to the "root directory" on the next command (in the same - Makefile rule). This commit runs the cd command in a subshell so it - would work in bmake. +- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance - Closes #5073 + This script helps putting entries in the RELEASE-NOTES using a coherent + style and sorting with a minimal human editing effort - as long as the + first line in the commit message is good enough! There's a short howto + at the top of the file. -- configure: convert -I to -isystem as a last step - - As all the -I uses in CFLAGS at that point are for system headers and - third party libraries this helps us remove/ignore warnings on those! - - Closes #5060 +- [Dennis Felsing brought this change] -- configure: fix -pedantic-errors for GCC 5 and later + configure: don't check for Security.framework when cross-compiling - If --enable-werror is used. + Since it checks for the local file, not the cross-compiled one. - Follow-up to d5c0351055d5709da which added it too early in the configure - script before $compiler_num was set correctly and thus this option was - never used. + Closes #5189 + +- TODO: Option to make -Z merge lined based outputs on stdout - Reported-by: Stepan Efremov - Fixes #5067 - Closes #5068 + Closes #5175 -- configure: document 'compiler_num' for gcc +- lib: never define CURL_CA_BUNDLE with a getenv - The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + - MINOR and ignores the patch version, and since gcc version 7 it only - sets it to MAJOR*100. + - it breaks the build (since 6de756c9b1de34b7a1) + - it's not documented and not consistent across platforms + - the curl tool does that getenv magic - Reported-by: Stepan Efremov - Ref: #5067 - Closes #5069 - -Version 7.69.1 (11 Mar 2020) - -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: 7.69.1 - -- THANKS: from the 7.69.1 release + Bug: https://github.com/curl/curl/commit/6de756c#r38127030 + Reported-by: Gisle Vanem + + Closes #5187 -- [Marc Hoersken brought this change] +Marc Hoersken (5 Apr 2020) +- lib670: use the same Win32 API check as all other lib tests - test1129: fix invalid case of closing XML-tag and Content-Length +- appveyor: use random test server ports based upon APPVEYOR_API_URL - Fixes #5070 - Closes #5072 - -Marc Hoersken (10 Mar 2020) -- tests/data: fix static ip instead of dynamic value being used + Avoid conflicts of test server ports with AppVeyor API on localhost. - Follow up to 94ced8e + Closes #5034 -- tests/data: fix static ip:port instead of dynamic values being used +- appveyor: sort builds by type and add two new variants - Closes #5065 + Related to #5034 and #5063 -- tests/server: fix missing use of exe_ext helper function +- appveyor: show failed tests in log even if test is ignored - Follow up to 9819984 and 3dce984 - Reviewed-By: Daniel Stenberg - Closes #5064 + And print API response with newline only if there is one -- runtests: log minimal and maximal used port numbers +- appveyor: turn disabled tests into ignored result tests -Daniel Stenberg (9 Mar 2020) -- [James Fuller brought this change] +Daniel Stenberg (5 Apr 2020) +- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" + + Fixed with #5170 (commit 23a870f2fd041278) - sftp: fix segfault regression introduced by #4747 +- test1566: verify --etag-compare that gets a 304 back - This fix adds a defensive check for the case where the char *name in - struct libssh2_knownhost is NULL + Verifies the fix in #5183 - Fixes #5041 - Closes #5062 + Closes #5186 -- RELEASE-NOTES: synced +- [Kwon-Young Choi brought this change] -- socks4: fix host resolve regression - - 1. The socks4 state machine was broken in the host resolving phase - - 2. The code now insists on IPv4-only when using SOCKS4 as the protocol - only supports that. + CURLINFO_CONDITION_UNMET: return true for 304 http status code - Regression from #4907 and 4a4b63d, shipped in 7.69.0 + In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the + output file if the server did not transfered a file based on time + condition. In the same manner, getting a 304 HTTP response back from the + server, for example after passing a custom If-Match-* header, also + fulfill this condition. - Reported-by: amishmm on github - Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 - Closes #5061 + Fixes #5181 + Closes #5183 -- [Patrick Monnerat brought this change] +- [Kwon-Young Choi brought this change] - silly web server: silent a compilation warning + curl: allow both --etag-compare and --etag-save with same file name - Recent gcc warns when byte count of strncpy() equals the destination - buffer size. Since the destination buffer is previously cleared and - the source string is always shorter, reducing the byte count by one - silents the warning without affecting the result. + This change inverse the order of processing for the --etag-compare and + --etag-save option to process first --etag-compare. This in turn allows + to use the same file name to compare and save an etag. - Closes #5059 - -- [Patrick Monnerat brought this change] - - cookie: get_top_domain() sets zero length for null domains + The original behavior of not failing if the etag file does not exists is + conserved. - This silents a compilation warning with gcc -O3. - -- [Patrick Monnerat brought this change] + Fixes #5179 + Closes #5180 - test 1560: avoid valgrind false positives +Viktor Szakats (4 Apr 2020) +- windows: enable UnixSockets with all build toolchains - When using maximum code optimization level (-O3), valgrind wrongly - detects uses of uninitialized values in strcmp(). + Extend existing unix socket support in Windows builds to be + enabled for all toolchain vendors or versions. (Previously + it was only supported with certain MSVC versions + more recent + Windows 10 SDKs) - Preset buffers with all zeroes to avoid that. + Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ + Ref: https://github.com/curl/curl/issues/5162 + Closes: https://github.com/curl/curl/pull/5170 -Steve Holme (8 Mar 2020) -- sha256: Added WinCrypt implementation +Daniel Stenberg (4 Apr 2020) +- KNOWN_BUGS: Store TLS context per transfer instead of per connection - Closed #5030 + Closes #5102 -- sha256: Added SecureTransport implementation +Marc Hoersken (3 Apr 2020) +- sockfilt: remove redundancy in timeout handling + + And update other logmsg output in select_ws on Windows. -Daniel Stenberg (7 Mar 2020) -- lib1564: reduce number of mid-wait wakeup calls +- sockfilt: fix handling of ready closed sockets on Windows - This test does A LOT of *wakeup() calls and then calls curl_multi_poll() - twice. The first *poll() is then expected to return early and the second - not - as the first is supposed to drain the socketpair pipe. + Replace the incomplete workaround regarding FD_CLOSE + only signalling once by instead doing a pre-check with + standard select and storing the result for later use. - It turns out however that when given "excessive" amounts of writes to - the pipe, some operating systems (the Solaris based are known) will - return EAGAIN before the pipe is drained, which in our test case causes - the second *poll() call to also abort early. + select keeps triggering on closed sockets on Windows while + WSAEventSelect fires only once with data still available. + By doing the pre-check we do not run in a deadlock + due to waiting forever for another FD_CLOSE event. + +- sockfilt: fix race-condition of waiting threads and event handling - This change attempts to avoid the OS-specific behaviors in the test by - reducing the amount of wakeup calls from 1234567 to 10. + Fix race-condition of waiting threads finishing while events are + already being processed which lead to invalid or skipped events. - Reported-by: Andy Fiddaman - Fixes #5037 - Closes #5058 + Use mutex to check for one event at a time or do post-processing. + In addition to mutex-based locking use specific event as signal. + + Closes #5156 -- [Patrick Monnerat brought this change] +Daniel Stenberg (2 Apr 2020) +- [Leo Neat brought this change] - mime: fix the binary encoder to handle large data properly + CI-fuzz: increase fuzz time to 40 minutes - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. + Closes #5174 + +Marc Hoersken (2 Apr 2020) +- CI: increase Azure Pipelines timeouts due to performance issues - Fixes #4860 - Closes #4833 - Reported-by: RuurdBeerstra on github + The current demand on Azure negatively impacts the CI performance. -- [Patrick Monnerat brought this change] +- runtests.pl: log host OS as detected by Perl environment - mime: do not perform more than one read in a row - - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. - - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 667 checks mimepost using single-byte read callback with encoder. - New test 668 checks the end of part data early detection. - - Fixes #4826 - Reported-by: MrdUkk on github +- ftpserver.pl: log before and after data connection is closed -- [Patrick Monnerat brought this change] +Daniel Stenberg (1 Apr 2020) +- RELEASE-NOTES: synced - mime: latch last read callback status. - - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. +- RELEASE-PROCEDURE.md: run the copyright.pl script! + +- vquic/ngtcp2.h: update copyright year range - Fixes #4813 - Reported-by: MrdUkk on github + Follow-up to 0736ee73d346a52 -Marc Hoersken (7 Mar 2020) -- runtests: fix missing use of exe_ext helper function +- [Daiki Ueno brought this change] -Daniel Stenberg (7 Mar 2020) -- [Ernst Sjöstrand brought this change] + CI: add build with ngtcp2 + gnutls on Travis CI - ares: store dns parameters for duphandle +- [Daiki Ueno brought this change] + + vquic: add support for GnuTLS backend of ngtcp2 - With c-ares the dns parameters lives in ares_channel. Store them in the - curl handle and set them again in easy_duphandle. + Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile + time. Therefore OpenSSL support needs to be explicitly disabled. - Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. + Signed-off-by: Daiki Ueno + Closes #5148 + +- [Gisle Vanem brought this change] + + examples/sessioninfo.c: add include to fix compiler warning - Fixes #4893 - Closes #5020 - Signed-off-by: Ernst Sjöstrand + Fixes #5171 -- version: make curl_version* thread-safe without using global context +- misc: copyright year updates - Closes #5010 + Follow-up to 7a71965e9 -- RELEASE-NOTES: synced +- [Harry Sintonen brought this change] -Marc Hoersken (7 Mar 2020) -- tests: use native Sleep function as fallback on Windows + build: fixed build for systems with select() in unistd.h - Reviewed-By: Daniel Stenberg - Closes #5054 + Closes #5169 -- perl: align order and completeness of Windows OS checks +- memdebug: don't log free(NULL) + + ... it serves no purpose and fills up the log. -Daniel Stenberg (7 Mar 2020) -- tool_cb_see: set correct copyright year range +- cleanup: insert newline after if() conditions - Follow-up to a39e5bfb9 + Our code style mandates we put the conditional block on a separate + line. These mistakes are now detected by the updated checksrc. -Marc Hoersken (7 Mar 2020) -- seek: fix fallback for missing ftruncate on Windows +- checksrc: warn on obvious conditional blocks on the same line as if() - This fixes test 198 on versions of MinGW-w64 without ftruncate + Closes #5164 + +- [Roger Orr brought this change] + + cmake: add CMAKE_MSVC_RUNTIME_LIBRARY - Reviewed-By: Daniel Stenberg - Reviewed-By: Marcel Raad - Closes #5055 + Fixes #5165 + Closes #5167 -- config-win32: Windows does not have ftruncate +- [Daiki Ueno brought this change] -Daniel Stenberg (7 Mar 2020) -- pause: force a connection (re-)check after unpausing + ngtcp2: update to git master for the key installation API change - There might be data available that was already read off the socket, for - example in the TLS layer. + This updates the ngtcp2 OpenSSL backend to follow the API change in + commit 32e703164 of ngtcp2. - Reported-by: Anders Berg - Fixes #4966 - Closes #5049 - -- socks5: switch state properly when the resolve is done + Notable changes are: + - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace + ngtcp2_crypto_derive_and_install_key + - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key + has been removed - Regression from 4a4b63d (and #4907) - Reported-by: vitaha85 on github - Fixes #5053 - Closes #5056 + Fixes #5166 + Closes #5168 -Jay Satiro (7 Mar 2020) -- libssh: Fix matching user-specified MD5 hex key +- [Cyrus brought this change] + + SECURITY.md: minor rephrase - Prior to this change a match would never be successful because it - was mistakenly coded to compare binary data from libssh to a - user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). + Closes #5158 + +- output.d: quote the URL when globbing - Reported-by: fds242@users.noreply.github.com + Some shells do globbing of their own unless the URL is quoted, so maybe + encourage this. - Fixes https://github.com/curl/curl/issues/4971 - Closes https://github.com/curl/curl/pull/4974 + Co-authored-by: Jay Satiro + Closes #5160 -Daniel Stenberg (6 Mar 2020) -- pause: bail out on bad input +- dist: add tests/version-scan.pl to tarball - A NULL easy handle or an easy handle without an associated connection - cannot be paused or unpaused. + ... used in test 1177. - Closes #5050 + Follow-up to a97d826f6de3 -Steve Holme (6 Mar 2020) -- unit1612: fixed the inclusion and compilation of the HMAC unit test - - Follow up to 3f74e5e6 to fix: +- test1177: verify that all the CURL_VERSION_ bits are documented + +- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented - - A typo in Makefile.inc where unit1611 was used instead - - Some compilation issues in unit1612.c + Considered experimental and therefore we can do this. - Closes #5024 + Closes #5157 -Daniel Stenberg (6 Mar 2020) -- pause: return early for calls that don't change pause state +- KNOWN_BUGS: DoH doesn't inherit all transfer options - Reviewed-by: Patrick Monnerat - Ref: #4833 - Closes #5026 + Closes #4578 + Closes #4579 -Jay Satiro (6 Mar 2020) -- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - - Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused - CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. +- KNOWN_BUGS: DoH leaks memory after followlocation - Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html - Reported-by: Felipe Gasper + Closes #4592 + +- KNOWN_BUGS: "FTPS needs session reuse" - Closes https://github.com/curl/curl/pull/5048 + Closes #4654 -- multi: skip EINTR check on wakeup socket if it was closed +- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed + +- TODO: Set custom client ip when using haproxy protocol - - Don't check errno on wakeup socket if sread returned 0 since sread - doesn't set errno in that case. + Closes #5125 + +Michael Kaufmann (27 Mar 2020) +- writeout_json: Fix data type issues - This is a follow-up to cf7760a from several days ago which fixed - Curl_multi_wait to stop busy looping sread on the non-blocking wakeup - socket if it was closed (ie sread returns 0). Due to a logic error it - was still possible to busy loop in that case if errno == EINTR. + Load long values correctly (e.g. for http_code). - Closes https://github.com/curl/curl/pull/5047 - -Daniel Stenberg (6 Mar 2020) -- transfer: set correct copyright year range - -- urldata: remove the 'stream_was_rewound' connectdata struct member + Use curl_off_t (not long) for: + - size_download (CURLINFO_SIZE_DOWNLOAD_T) + - size_upload (CURLINFO_SIZE_UPLOAD_T) - ... as it is never set anywhere. + The unit for these values is bytes/second, not microseconds: + - speed_download (CURLINFO_SPEED_DOWNLOAD_T) + - speed_upload (CURLINFO_SPEED_UPLOAD_T) - Follow-up to 2f44e94ef - Closes #5046 + Fixes #5131 + Closes #5152 -- Revert "pause: force-drain the transfer on unpause" +Daniel Stenberg (27 Mar 2020) +- mailmap: fixup a few author names/fields - This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) + Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and + Svyatoslav Mishyn + +- version: add 'cainfo' and 'capath' to version info struct - Clearly that didn't solve the problem correctly. + Suggested-by: Timothe Litt + URL: https://curl.haxx.se/mail/lib-2020-03/0090.html + Reviewed-by: Jay Satiro - Reported-by: Christopher Reid - Reopens #4966 - Fixes #5044 + Closes #5150 - RELEASE-NOTES: synced - - and bumped curlver.h -- MANUAL: update a dict-using command line +Jay Satiro (26 Mar 2020) +- SSLCERTS.md: Fix example code for setting CA cert file - The 'web1913' database is now invalid, use 'gcide' instead. - -- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS + Prior to this change the documentation erroneously said use + CURLOPT_CAPATH to set a CA cert file. - Closes #3841 + Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html + Reported-by: Timothe Litt + + Closes https://github.com/curl/curl/pull/5151 -- polarssl: remove more references and mentions +Marc Hoersken (26 Mar 2020) +- sockfilt: add logmsg output to select_ws_wait_thread on Windows Assisted-by: Jay Satiro - Follow-up to 6357a19ff29dac04 - Closes #5036 - -Marc Hoersken (4 Mar 2020) -- tests: wrap ignored test failures in braces - -- tests: align some Windows sleep defines with each other + Reviewed-by: Daniel Stenberg + + Closes #5086 -- tests: try to make sleeping portable by avoiding select +Daniel Stenberg (26 Mar 2020) +- docs/make: generate curl.1 from listed files only - select does not support just waiting on Windows: - https://perldoc.perl.org/perlport.html#select + Previously it rendered the page from files matching "*.d" in the correct + directory, which worked fine in git builds when the files were added but + made it easy to forget adding the files to the dist. - Reviewed-By: Daniel Stenberg - Closes #5035 + Now, only man page sections listed in DPAGES in Makefile.inc will be + used, thus "forcing" us to update this to get the man page right and get + it included in the dist at the same time. + + Ref: #5146 + Closes #5149 -Daniel Stenberg (4 Mar 2020) -- runtests.1: rephrase how to specify what tests to run +- openssl: adapt to functions marked as deprecated since version 3 - Also mention the new tilde-prefixed way to ignore test results. + OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES + functions we use. - Reviewed-By: Marc Hoersken - Closes #5033 - -- cirrus-ci: disable the FreeBSD 13 builds + Fix the MD4 and SSL_CTX_load_verify_locations warnings. - FreeBSD 13.0 is apparently close to a year away from a stable release - and has proven to cause intermittent builds failures recently. + In configure, detect OpenSSL v3 and if so, inhibit the deprecation + warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and + until we rewrite the code to use non-deprecated functions we better + ignore these warnings as they don't help us. - Assisted-by: Dan Fandrich - Assisted-by: Fedor Korotkov - Fixes #5028 - Closes #5029 + Closes #5139 -Version 7.69.0 (4 Mar 2020) +- dist: add mail-rcpt-allowfails.d to the tarball + + Reported-by: Maksim Stsepanenka + Reviewed-by: Jat Satiro + + Closes #5146 -Daniel Stenberg (4 Mar 2020) -- RELEASE-NOTES: 7.69.0 +- travis: update the ngtcp2 build to use the latest OpenSSL patch + + ... which also makes it OpenSSL 1.1.1d based and not v3. -- THANKS: from 7.69.0 +Marc Hoersken (24 Mar 2020) +- CI: remove default Ubuntu build from GitHub Actions - Now sorted case insensitive + We are already running a very similar Ubuntu build on Travis CI. + The macOS variant of this default build is kept on Github Actions. -Marc Hoersken (3 Mar 2020) -- ci/tests: fix escaping of testnames and disable proxy for CI APIs +- CI: bring GitHub Actions fuzzing job in line with macOS jobs - Follow up to ada581f and c0d8b96 - Closes #5031 + Update YAML formatting, job naming and triggers. -Jay Satiro (3 Mar 2020) -- cmake: Show HTTPS-proxy in the features output +- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions - - Show HTTPS-proxy in the features output for those backends that - support it: OpenSSL, GnuTLS and NSS. + Reduce workload on Azure Pipelines and Travis CI while + consolidating macOS jobs onto less utilized GitHub Actions. - Prior to this change HTTPS-proxy was missing from the cmake features - output even if curl was built with it. Only cmake output was affected. - Both the library and tool correctly reported the feature. + Reviewed-by: Daniel Stenberg - Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html - Reported-by: David Lopes + Closes #5124 + +Daniel Stenberg (24 Mar 2020) +- config: remove all defines of HAVE_DES_H - Closes https://github.com/curl/curl/pull/5025 + As there's no code using it. + + Closes #5144 -Marc Hoersken (3 Mar 2020) -- ci/tests: Make it possible to still run but ignore failing tests +- copyright: fix out-of-date copyright ranges and missing headers - This enables the development of a solution for the failing tests by - running them on CI while ignoring their result for the overall status. + Reported by the new script 'scripts/copyright.pl'. The script has a + regex whitelist for the files that don't need copyright headers. + + Removed three (mostly usesless) README files from docs/ + + Closes #5141 + +- packages: add OS400/chkstrings.c to the dist - Closes #4994 + Reported-by: Jon Rumsey + Fixes #5142 + Closes #5143 -- README.md: add Azure DevOps Pipelines build status badge +- [Clément Notin brought this change] -- ci/tests: Move CI test result creation above environment setup + nghttp2: 1.12.0 required - This avoids using our test servers as proxy to the AppVeyor API. + since nghttp2_session_set_local_window_size is needed - Closes #5022 + Closes #5140 -- ci/tests: Send test results to AppVeyor for status overview - - Closes #5021 +- RELEASE-NOTES: synced -Daniel Stenberg (3 Mar 2020) -- Revert "sha256: Added SecureTransport implementation" +- [Calvin Buckley brought this change] + + OS400: Update strings for ccsid-ifier - This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) + Fixes build. - That commit broke test 1610 on macos builds without TLS. + Closes #5132 + +- cirrus: make freebsd ignore the tests instead of skipping - Closes #5027 + To allow us to see in the CI logs how they actually behave + + Closes #5091 -- dist: include tests/azure.pm in the tarball +- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 + +- Revert "cirrus-ci: disable the FreeBSD 13 builds" - Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 - Reported-by: Marcel Raad + This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. -Steve Holme (3 Mar 2020) -- configure.ac: Disable metalink if mbedTLS is specified +- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override - Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as - being one of the backends that metalink needs to be disabled for, I - seem to have included it in the list of allowed SSL/TLS backends in - comnfigure.ac :( + To let debug-builds return fake values, like in test 970. - Closes #5013 + Ref: #5131 + Closes #5136 -- sha256: Tidy up following recent changes +- test970: improve the test - Reviewed-by: Daniel Stenberg - Closes #4956 - -- sha256: Added WinCrypt implementation - -- sha256: Added SecureTransport implementation - -- sha256: Added mbedtls implementation + - send more data to make problems more obvious + - don't start the data with minus, it makes diffs harder to read + - skip the headers in the stdout comparison + - save to a file name to also verify 'filename_effective' + + Ref: #5131 -- sha256: Added GNU TLS gcrypt implementation +- CURLINFO_NUM_CONNECTS: improve accuracy + + The counter was not bumped in all cases correctly. + + Reported-by: Marcel Raad + Ref: #5131 + Closes #5135 -- sha256: Added GNU TLS Nettle implementation +- TODO: Use "random" ports for the test servers -Jay Satiro (2 Mar 2020) -- curl_escape.3: Add a link to curl_free +- lib/curl_setup: adjust the copyright year range - Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 + Follow-up from d820224b8 -- curl_getenv.3: Fix the memory handling description - - - Tell the user to call curl_free() to free the pointer returned by - curl_getenv(). +Jay Satiro (21 Mar 2020) +- curl_setup: define _WIN32_WINNT_[OS] symbols - Prior to this change the user was directed to call free(), but that - would not work in cases where the library and application use separate C - runtimes and therefore have separate heap memory management. + .. because not all Windows build systems have those symbols, and even + those that do may be missing newer symbols (eg the Windows 7 SDK does + not define _WIN32_WINNT_WIN10). - Closes https://github.com/curl/curl/pull/5016 - -Daniel Stenberg (2 Mar 2020) -- [Nick Zitzmann brought this change] - - md4: use init/update/final functions in Secure Transport + Those symbols are used in build-time logic to decide which API to use + and prior to this change if the symbols were missing it would have + resulted in deprecated API being used when more recent functions were + available (eg GetVersionEx used instead of VerifyVersionInfo). - We can use CC_MD4_Init/Update/Final without having to allocate memory - directly. + Reported-by: FuccDucc@users.noreply.github.com - Closes #4979 + Probably fixes https://github.com/curl/curl/issues/4995 + Closes https://github.com/curl/curl/pull/5057 -Marc Hoersken (2 Mar 2020) -- ci/tests: some MacOS builds randomly take longer than 20min +- [Ross Burton brought this change] -Daniel Stenberg (2 Mar 2020) -- multi_wait: stop loop when sread() returns zero + curl-functions.m4: remove inappropriate AC_REQUIRE - It's unclear why it would ever return zero here, but this change fixes - Robert's problem and it shouldn't loop forever... + AC_REQUIRE means "if this macro hasn't been executed already, execute + it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) + isn't correct at that will execute AC_RUN_IFELSE without any arguments. - Reported-by: Robert Dunaj - Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html - Closes #5019 + With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, + AC_RUN_IFELSE without a default value when cross-compiling is fatal. + The result is that curl with autoconf 2.70 cannot cross-compile. + + Fixes https://github.com/curl/curl/issues/5126 + Closes https://github.com/curl/curl/pull/5130 -- http: mark POSTs with no body as "upload done" from the start +Marc Hoersken (20 Mar 2020) +- ci/tests: fix Azure Pipelines not running Windows containers - As we have logic that checks if we get a >= 400 reponse code back before - the upload is done, which then got confused since it wasn't "done" but - yet there was no data to send! + Workaround posted here: microsoft/azure-pipelines-agent#2864 - Reported-by: IvanoG on github - Fixes #4996 - Closes #5002 - -- tests: disable 962, 963 and 964 on Windows + Assisted-by: Simon Chalifoux + Assisted-by: Tommy Petty - These tests are also doing UTF-8 SMTP. + Fixes #5117 + Closes #5129 + +Daniel Stenberg (20 Mar 2020) +- tests: add test 430, 431 and 432 to verify the --config fix - Follow-up to df207d2dd93b9e73 + Verify the fixes in 4e0b4fee4 -Marc Hoersken (2 Mar 2020) -- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer +- [Rici Lake brought this change] -Daniel Stenberg (2 Mar 2020) -- configure: bump the AC_COPYRIGHT year range + cmdline: fix handling of OperationConfig linked list (--next) + + Ensures that -K/--config inserts new items at the end of the list + instead of overwriting the second item, and that after a -K/--config + option has been parsed, the option parser's view of the current config + is update. + + Fixes #5120 + Closes #5123 -- [Steve Holme brought this change] +Marc Hoersken (20 Mar 2020) +- test2100: fix static port instead of dynamic value being used - tests: disable SMTP UTF-8 tests on Windows - - Fixes #4988 - Closes #4992 +- test970: fix static ip:port instead of dynamic values being used -- formdata/mime: copyright year range update +Daniel Stenberg (19 Mar 2020) +- secure transport: remove the BACKEND define kludge - Due to the merge/revert cycle + Closes #5122 -- Revert "mime: latch last read callback status." - - This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. - - Fixes #5014 - Closes #5015 - Reopens #4833 +- mbedtls: remove the BACKEND define kludge -- Revert "mime: do not perform more than one read in a row" - - This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. +- bearssl: remove the BACKEND define kludge -- Revert "mime: fix the binary encoder to handle large data properly" - - This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. +- wolfssl: remove the BACKEND define kludge -- altsvc: both h3 backends now speak h3-27 - - ... also updated the HTTP3 build description for ngtcp2 accordingly. +- nss: remove the BACKEND define kludge -- [Patrick Monnerat brought this change] +- gnutls: remove the BACKEND define kludge - mime: fix the binary encoder to handle large data properly - - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. +- openssl: remove the BACKEND define kludge - Fixes #4860 - Reported-by: RuurdBeerstra on github - -- [Patrick Monnerat brought this change] + Use a proper variable instead to make it easier to use a debugger and + read the code. - mime: do not perform more than one read in a row +Marc Hoersken (19 Mar 2020) +- tests: make Python-based servers compatible with Python 2 and 3 - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. + Update smbserver.py and negtelnetserver.py to be compatible with + Python 3 while staying backwards-compatible to support Python 2. - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 664 checks mimepost using single-byte read callback with encoder. - New test 665 checks the end of part data early detection. + Fix string encoding and handling of echoed and transferred data. - Fixes #4826 - Reported-by: MrdUkk on github - -- [Patrick Monnerat brought this change] - - mime: latch last read callback status. + Tested with both Python 2.7.17 and Python 3.7.7 - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. + Reported-by: Daniel Stenberg + Assisted-by: Kamil Dudka + Reviewed-by: Marcel Raad - Fixes #4813 - Reported-by: MrdUkk on github - Closes #4833 + Fixes #5104 + Closes #5110 -Steve Holme (1 Mar 2020) -- unit1651: Fixed conversion compilation warning +Daniel Stenberg (18 Mar 2020) +- writeout_json: use curl_off_t printf() option for the time output - 371:17: warning: conversion to 'unsigned char' from 'int' may alter its - value [-Wconversion] + Follow-up to: 04c03416e68fd635a15 - Closes #5008 + Closes #5115 -- configure.ac: Disable metalink support if an incompatible SSL/TLS specified +- RELEASE-NOTES: synced - tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, - The Win32 Crypto library and Apple's Common Crypto library. + Uh, I missed this in 1a46b218db + +- RELEASE-NOTES: synced - If an TLS backend such as mbedTLS or WolfSSL is specified then the - following error is given during compilation along, with a load of - unresolved extern errors: + ... and bumped curlver.h to 7.70.0 + +Jay Satiro (18 Mar 2020) +- http2: Fix erroneous debug message that h2 connection closed - Can't compile METALINK support without a crypto library. + Prior to this change in libcurl debug builds http2 stream closure was + erroneously referred to as connection closure. - Reviewed-by: Daniel Stenberg - Closes #5006 - -Marc Hoersken (1 Mar 2020) -- ci/tests: Update Azure DevOps pipeline job display names + Before: + * nread <= 0, server closed connection, bailing - Make the configure step more descriptive and align others. - -- ci/tests: Fix typo in previous commit 597cf2 - -- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available + After: + * nread == 0, stream closed, bailing - For security reasons the access token is not available to PR builds. - Therefore we should not try to use the DevOps API with an empty token. + Closes https://github.com/curl/curl/pull/5118 -Daniel Stenberg (1 Mar 2020) -- build: remove all HAVE_OPENSSL_ENGINE_H defines - - ... as there's nothing in the code that actually uses the define! The - last reference was removed in 38203f158. +Daniel Stenberg (18 Mar 2020) +- tool_setopt: correct the copyright year range - Closes #5007 + Follow-up to 5450428491 -Jay Satiro (29 Feb 2020) -- [Rolf Eike Beer brought this change] +Jay Satiro (18 Mar 2020) +- [Johannes Schindelin brought this change] - CMake: clean up and improve build procedures - - - remove check for unsupported old CMake versions + schannel: add "best effort" revocation check option - - do not link to c-ares library twice + - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and + --ssl-revoke-best-effort to allow a "best effort" revocation check. - - modernize custom Find modules + A best effort revocation check ignores errors that the revocation check + was unable to take place. The reasoning is described in detail below and + discussed further in the PR. - - FindLibSSH2: - - pass version to FPHSA to show it in the output - - use LIBSSH2_VERSION define to extract the version number in - one shot. This variable exists in the header for 10 years. - - remove unneeded code + --- - - FindNGHTTP2.cmake: - - drop needless FPHSA argument - - mark found variables as advanced + When running e.g. with Fiddler, the schannel backend fails with an + unhelpful error message: - - FindNSS.cmake: - - show version number + Unknown error (0x80092012) - The revocation function was unable + to check revocation for the certificate. - - FindCARES.cmake: - - drop default paths - - use FPHSA instead of checking things by hand + Sadly, many enterprise users who are stuck behind MITM proxies suffer + the very same problem. - - remove needless explict variable dereference + This has been discussed in plenty of issues: + https://github.com/curl/curl/issues/3727, + https://github.com/curl/curl/issues/264, for example. - - simplify count_true() + In the latter, a Microsoft Edge developer even made the case that the + common behavior is to ignore issues when a certificate has no recorded + distribution point for revocation lists, or when the server is offline. + This is also known as "best effort" strategy and addresses the Fiddler + issue. - - allow all policies up to version 3.16 to be set to NEW + Unfortunately, this strategy was not chosen as the default for schannel + (and is therefore a backend-specific behavior: OpenSSL seems to happily + ignore the offline servers and missing distribution points). - - do not rerun check for -Wstrict-aliasing=3 every time + To maintain backward-compatibility, we therefore add a new flag + (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option + (`--ssl-revoke-best-effort`) to select the new behavior. - In contrast to every other compiler flag this has a = in it, which CMake - can't have in a variable name. + Due to the many related issues Git for Windows and GitHub Desktop, the + plan is to make this behavior the default in these software packages. - - only read the interesting strings from curlver.h + The test 2070 was added to verify this behavior, adapted from 310. - Reviewed-by: Peter Wu + Based-on-work-by: georgeok + Co-authored-by: Markus Olsson + Signed-off-by: Johannes Schindelin - Closes https://github.com/curl/curl/pull/4975 + Closes https://github.com/curl/curl/pull/4981 -- runtests: fix output to command log +- multi: Improve parameter check for curl_multi_remove_handle - - Record only the command of the most recently ran test in the command - log. + - If an easy handle is owned by a multi different from the one specified + then return CURLM_BAD_EASY_HANDLE. - This is a follow-up to 02988b7 from several weeks ago which fixed - writing to the command log, however it saved all commands for all tests - instead of just the most recently ran test as we would now expect. + Prior to this change I assume user error could cause corruption. - Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 - Closes https://github.com/curl/curl/pull/5001 + Closes https://github.com/curl/curl/pull/5116 -Steve Holme (1 Mar 2020) -- polarssl: Additional removal - - Follow up to 6357a19f. +Viktor Szakats (17 Mar 2020) +- windows: suppress UI in all CryptAcquireContext() calls - Reviewed-by: Daniel Stenberg - Closes #5004 - -- [Jonathan Cardoso Machado brought this change] + Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters + Reviewed-by: Marc Hörsken + Closes https://github.com/curl/curl/pull/5088 - docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always +Daniel Stenberg (17 Mar 2020) +- writeout_json: add missing comma to fix the HTTP version - Reviewed-by: Steve Holme - Closes #5005 + Follow-up to 04c03416e68fd635a15 -- md5: Added implementation for mbedTLS +- test 970: verify --write-out '%{json}' - Reviewed-by: Jay Satiro - Closes #4980 + Makes curl_easy_getinfo() of "variable" numerical content instead return + the number set in the env variable `CURL_TIME`. + + Makes curl_version() of "variable" textual content. This guarantees a + stable version string which can be tested against. Environment variable + `CURL_VERSION` defines the content. + + Assisted-by: Mathias Gumz -- md5: Use pointer notation for array parameters in GnuTLS implementation +- [Mathias Gumz brought this change] -- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 + writeout: support to generate JSON output - Closes #4983 + This commit adds support to generate JSON via the writeout feature: + + -w "%{json}" + + It leverages the existing infrastructure as much as possible. Thus, + generating the JSON on STDERR is possible by: + + -w "%{stderr}%{json}" + + This implements a variant of + https://github.com/curl/curl/wiki/JSON#--write-out-json. + + Closes #4870 -Marc Hoersken (29 Feb 2020) -- ci/tests: Send test results to Azure DevOps for reporting +- CI: stop ignoring 323, it is disabled -Daniel Stenberg (29 Feb 2020) -- pause: force-drain the transfer on unpause +- DISABLED: disable test 323 - ... since the socket might not actually be readable anymore when for - example the data is already buffered in the TLS layer. + The test uses SRP to "a server not supporting it" but modern stunnel + versions will silently accept it and remain happy. The test is therefore + faulty. - Fixes #4966 - Reported-by: Anders Berg - Closes #5000 + I haven't figured out how to make stunnel explicitly reject SRP-using + connects. + + Reported-by: Marc Hörsken + Fixes #5105 + Closes #5113 -- TODO: curl --proxycommand +Marc Hoersken (17 Mar 2020) +- ci/tests: increase timeouts for torture builds on Azure Pipelines - Suggested-by: Kristian Mide - Closes #4941 + For some reason the torture builds have slowed down recently. + + Reported-by: Daniel Stenberg -- smtp: overwriting 'from' leaks memory +Daniel Stenberg (16 Mar 2020) +- cmake: add support for building with wolfSSL - Detected by Coverity. CID 1418139. + My working build cmdline: - Also, make sure to return error if the new 'from' allocation fails. + $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . - Closes #4997 + Assisted-by: Brad King + Closes #5095 -- CIfuzz: switch off 'dry_run' mode +- tool_operate: fix add_parallel_transfers when more are in queue - Follow-up from #4960: now make it fail if it detects problems. + Trying to return early from the function if no new transfers were added + would break the "morep" argument and cause issues. This could lead to + zero content "transfers" (within quotes since they would never be + started) when parallel-max was reduced. - Closes #4998 + Reported-by: Gavin Wong + Analyzed-by: Jay Satiro + Fixes #4937 + Closes #5112 -Marc Hoersken (28 Feb 2020) -- ci/tests: Increase timeouts of Windows builds due to new tests +- vtls: free ssl_config leftovers on out-of-memory - Recently added tests increased their runtime above the limit of 60min. - -- ci/tests: align Azure Pipeline job names with each other + Torture testing 2034 and 2037 found this. + + Reported-by: Marc Hörsken + Fixes #5108 + Closes #5109 -- ci/tests: Add Windows builds via Azure Pipelines using Docker +Marc Hoersken (16 Mar 2020) +- ci/tests: fix Azure Pipelines not running for pull requests + + Closes #5111 -- tests: fix Python 3 compatibility of smbserver.py +Daniel Stenberg (15 Mar 2020) +- gskit: update the copyright year range + + Follow-up from 083603c63a3 -Daniel Stenberg (27 Feb 2020) -- runtests: restore the command log +Marc Hoersken (15 Mar 2020) +- gskit: use our internal select wrapper for portability - The log file with all command lines for the invoked command lines is now - called logs/commands.log + Follow up to c52b342 + Closes #5106 + +- tests: fix verification of stdout in test 1452 due to newline - Fixes #4911 - Closes #4989 + Fixes test1452:41:1: error: missing tag before -- smtp: fix memory leak on exit path +- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI - Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' - going out of scope leaks the storage it points to" + Also force the package index/cache to be updated before installing. - Closes #4990 + Closes #5103 -Steve Holme (27 Feb 2020) -- gtls: Fixed compilation when using GnuTLS < 3.5.0 - - Reverts the functionality from 41fcb4f when compiling with GnuTLS older - than 3.5.0. +- tests/README: add note about manually installing python-impacket - Reviewed-by: Daniel Stenberg - Closes #4984 + Follow up to 4be2560 -- RELEASE-NOTES: Corrected the link to issue #4892 +Daniel Stenberg (15 Mar 2020) +- transfer: cap retries of "dead connections" to 5 + + When libcurl retries a connection due to it being "seemingly dead" or by + REFUSED_STREAM, it will now only do it up five times before giving up, + to avoid never-ending loops. + + Reported-by: Dima Tisnek + Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html + Closes #5074 -Daniel Stenberg (27 Feb 2020) -- Curl_is_ASCII_name: handle a NULL argument +- TODO: TLS-PSK with OpenSSL - Make the function tolerate a NULL pointer input to avoid dereferencing - that pointer. + Closes #5081 + +Marc Hoersken (15 Mar 2020) +- select: add 'timeout_ms' wrap-around precaution to Curl_select + +- select: fix 'pending_ms' is assigned a value that is never used - Follow-up to efce3ea5a85126d - Detected by OSS-Fuzz - Reviewed-By: Steve Holme - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 - Fixes #4985 - Closes #4986 + Detected by Codacy -- RELEASE-NOTES: synced +- select: move duplicate select preparation code into Curl_select + + Reviewed by Daniel Stenberg + Reviewed by Marcel Raad + Closes #5078 -- http2: make pausing/unpausing set/clear local stream window +Daniel Stenberg (15 Mar 2020) +- connect: happy eyeballs cleanup - This reduces the HTTP/2 window size to 32 MB since libcurl might have to - buffer up to this amount of data in memory and yet we don't want it set - lower to potentially impact tranfer performance on high speed networks. + Make sure each separate index in connn->tempaddr[] is used for a fixed + family (and only that family) during the connection process. - Requires nghttp2 commit b3f85e2daa629 - (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end - up in the next release after 1.40.0. + If family one takes a long time and family two fails immediately, the + previous logic could misbehave and retry the same family two address + repeatedly. - Fixes #4939 - Closes #4940 + Reported-by: Paul Vixie + Reported-by: Jay Satiro + Fixes #5083 + Fixes #4954 + Closes #5089 -- [Anderson Toshiyuki Sasaki brought this change] +Marc Hoersken (15 Mar 2020) +- ci/tests: fix and align setting TFLAGS for make test-nonflaky - libssh: improve known hosts handling - - Previously, it was not possible to get a known hosts file entry due to - the lack of an API. ssh_session_get_known_hosts_entry(), introduced in - libssh-0.9.0, allows libcurl to obtain such information and behave the - same as when compiled with libssh2. +- ci/tests: install test suite dependencies stunnel and impacket + +- tests: remove python_dependencies for smbserver from our tree - This also tries to avoid the usage of deprecated functions when the - replacements are available. The behaviour will not change if versions - older than libssh-0.8.0 are used. + Users of the SMB tests will have to install impacket manually. - Signed-off-by: Anderson Toshiyuki Sasaki + Reasoning: our in-tree version of impacket was quite outdated + and only compatible with Python 2 which is already end-of-life. + Upgrading to Python 3 and a compatible impacket version would + require to import additional Python-only and CPython-extension + dependencies. This would have hindered portability enormously. - Fixes #4953 - Closes #4962 + Closes #5094 -Steve Holme (27 Feb 2020) -- tests: Automatically deduce the tool name from the test case for unit tests - - It is still possible to override the executable to run during the test, - using the tag, but this patch removes the requirement that the - tag must be present for unit tests. +Jay Satiro (14 Mar 2020) +- Makefile.m32: Improve windres parameter compatibility - It also removes the possibility of human error when existing test cases - are used as the basis for new tests, as recently witnessed in 81c37124. + - s/COFF/coff/ - Reviewed-by: Daniel Stenberg - Closes #4976 - -- test1323: Added the missing 'unit test' feature requirement in the test case - -Daniel Stenberg (26 Feb 2020) -- cookie: remove unnecessary check for 'out != 0' + Some versions of windres do not recognize uppercase COFF as a valid + way to specify the COFF output format. - ... as it will always be non-NULL at this point. + Reported-by: Steven Penny - Detected by Coverity: CID 1459009 + Fixes https://github.com/curl/curl/issues/5099 + Closes https://github.com/curl/curl/pull/5101 -- http: added 417 response treatment +- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares - When doing a request with a body + Expect: 100-continue and the server - responds with a 417, the same request will be retried immediately - without the Expect: header. + - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in + curl_easy_duphandle. - Added test 357 to verify. + Prior to this change if c-ares was used as the resolver backend and + either it was too old or libcurl was built without IPv6 support then + some of our resolver functions could return CURLE_NOT_BUILT_IN to + curl_easy_duphandle causing it to fail. - Also added a control instruction to tell the sws test server to not read - the request body if Expect: is present, which the new test 357 uses. + Caused by c8f086b which shipped in 7.69.1. - Reported-by: bramus on github - Fixes #4949 - Closes #4964 - -Steve Holme (26 Feb 2020) -- smtp: Tidy up, following recent changes, to maintain the coding style + Reported-by: Karl Chen - Closes #4892 + Fixes https://github.com/curl/curl/issues/5097 + Closes https://github.com/curl/curl/pull/5100 -- smtp: Support the SMTPUTF8 extension for the EXPN command +Daniel Stenberg (13 Mar 2020) +- docs: add warnings about FILE: URLs on Windows - Simply notify the server we support the SMTPUTF8 extension if it does. + - --url man page section + - libcurl-security.3 gets the full text + - CURLOPT_URL.3 + + Reported-by: Tim Sedlmeyer -- smtp: Support the SMTPUTF8 extension in the VRFY command +- server/getpart: make the "XML-parser" stricter + + When extracting a
and there's no before +
, this now outputs an error and returns a wrong string to + make users spot the mistake. + + Ref: #5070 + Closes #5071 -- smtp: Support the SMTPUTF8 extension in the RCPT TO command +Marc Hoersken (13 Mar 2020) +- impacket: some more Python 3 code compatibility updates - Note: The RCPT TO command isn't required to advertise to the server that - it contains UTF-8 characters, instead the server is told that a mail may - contain UTF-8 in any envelope command via the MAIL command. + This makes smbserver load on Python 3, but still not work completely. -- smtp: Support the SMTPUTF8 extension in the MAIL command +- smbserver: pin Python version to 2 since we are not yet 3 compatible - Support the SMTPUTF8 extension when sending mailbox information in the - MAIL command (FROM and AUTH parameters). Non-ASCII domain names will - be ACE encoded, if IDN is supported, whilst non-ASCII characters in - the local address part are passed to the server. + Even though the existing code can be fixed to run on Python 3, the + tests will fail due to the Unicode transition the protocol is invalid. - Reported-by: ygthien on github - Fixes #4828 - -- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 + Follow up to ee63837 + Closes #5085 -- smtp: Support UTF-8 based host names in the VRFY command +Daniel Stenberg (12 Mar 2020) +- [Viktor Szakats brought this change] -- smtp: Support UTF-8 based host names in the RCPT TO command + cleanup: fix some text/comment typos + + Closes #5087 -- smtp: Support UTF-8 based host names in the MAIL command +Marc Hoersken (12 Mar 2020) +- smbserver: fix Python version specific ConfigParser import - Non-ASCII host names will be ACE encoded if IDN is supported. + Follow up to ee63837 and 8c7c4a6 + Fixes #5077 -- url: Make the IDN conversion functions available to others +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: synced + + bumped to 7.69.2 -- smtp: Added UTF-8 mailbox tests to verify existing behaviour +Dan Fandrich (11 Mar 2020) +- tests/data: Fix some XML formatting issues in test cases + + This allows these test files to pass xmllint. -- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case +Daniel Stenberg (11 Mar 2020) +- [Muhammad Herdiansyah brought this change] -- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() + Makefile: run the cd commands in a subshell - The dot character between the host and the tld was not being escaped, - which meant it specified a match of 'any' character rather than an - explicit dot separator. + In bmake, if the directory is changed (with cd or anything else), bmake + won't return to the "root directory" on the next command (in the same + Makefile rule). This commit runs the cd command in a subshell so it + would work in bmake. - Additionally removed the dot character from the host name as it allowed - the following to be specified as a valid address in our test cases: + Closes #5073 + +- configure: convert -I to -isystem as a last step - + As all the -I uses in CFLAGS at that point are for system headers and + third party libraries this helps us remove/ignore warnings on those! - Both are typos from 98f7ca7 and 8880f84 :( + Closes #5060 + +- configure: fix -pedantic-errors for GCC 5 and later - I can't remember whether my intention was to allow sub-domains to be - specified in the host or not with these additional dots, but by placing - it outside of the host means it can only be specified once per domain - and by placing a + after the new grouping support for sub-domains is - kept. + If --enable-werror is used. - Closes #4912 + Follow-up to d5c0351055d5709da which added it too early in the configure + script before $compiler_num was set correctly and thus this option was + never used. + + Reported-by: Stepan Efremov + Fixes #5067 + Closes #5068 -- hmac: Added a unit test for the HMAC hash generation +- configure: document 'compiler_num' for gcc + + The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + + MINOR and ignores the patch version, and since gcc version 7 it only + sets it to MAJOR*100. - Closes #4973 + Reported-by: Stepan Efremov + Ref: #5067 + Closes #5069 -- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function +Version 7.69.1 (11 Mar 2020) -- tests: Added a unit test for MD4 digest generation - - Closes #4970 +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: 7.69.1 -- md4: Use const for the length input parameter - - This keeps the interface the same as md5 and sha256. +- THANKS: from the 7.69.1 release -- test1610: Fixed the link to the unit test - - Typo from 81c37124. +- [Marc Hoersken brought this change] -- ntlm: Removed the dependency on the TLS libaries when using MD5 - - As we have our own MD5 implementation use the MD5 wrapper to remove the - TLS dependency. + test1129: fix invalid case of closing XML-tag and Content-Length - Closes #4967 - -- md5/sha256: Updated the functions to allow non-string data to be hashed + Fixes #5070 + Closes #5072 -- digest: Corrected the name of the local HTTP digest function +Marc Hoersken (10 Mar 2020) +- tests/data: fix static ip instead of dynamic value being used - Follow up to 2b5b37cb. Local static functions do not require the Curl - prefix. + Follow up to 94ced8e -- tests: Added a unit test for SHA256 digest generation - - Follow up to 2b5b37c. +- tests/data: fix static ip:port instead of dynamic values being used - Closes #4968 + Closes #5065 -- md4: Fixed compilation issues when using GNU TLS gcrypt +- tests/server: fix missing use of exe_ext helper function - * Don't include 'struct' in the gcrypt MD4_CTX typedef - * The call to gcry_md_read() should use a dereferenced ctx - * The call to gcry_md_close() should use a dereferenced ctx + Follow up to 9819984 and 3dce984 + Reviewed-By: Daniel Stenberg + Closes #5064 + +- runtests: log minimal and maximal used port numbers + +Daniel Stenberg (9 Mar 2020) +- [James Fuller brought this change] + + sftp: fix segfault regression introduced by #4747 - Additional minor whitespace issue in the USE_WIN32_CRYPTO code. + This fix adds a defensive check for the case where the char *name in + struct libssh2_knownhost is NULL - Closes #4959 + Fixes #5041 + Closes #5062 -Daniel Stenberg (21 Feb 2020) - RELEASE-NOTES: synced -- http2: now require nghttp2 >= 1.12.0 +- socks4: fix host resolve regression - To simplify our code and since earlier versions lack important function - calls libcurl needs to function correctly. + 1. The socks4 state machine was broken in the host resolving phase - nghttp2 1.12.0 was relased on June 26, 2016. + 2. The code now insists on IPv4-only when using SOCKS4 as the protocol + only supports that. - Closes #4961 - -- gtls: fix the copyright year + Regression from #4907 and 4a4b63d, shipped in 7.69.0 - Follow-up from 41fcb4f609 + Reported-by: amishmm on github + Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 + Closes #5061 -- [jethrogb brought this change] +- [Patrick Monnerat brought this change] - GnuTLS: Always send client cert + silly web server: silent a compilation warning - TLS servers may request a certificate from the client. This request - includes a list of 0 or more acceptable issuer DNs. The client may use - this list to determine which certificate to send. GnuTLS's default - behavior is to not send a client certificate if there is no - match. However, OpenSSL's default behavior is to send the configured - certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL - behavior. + Recent gcc warns when byte count of strncpy() equals the destination + buffer size. Since the destination buffer is previously cleared and + the source string is always shorter, reducing the byte count by one + silents the warning without affecting the result. - Authored-by: jethrogb on github - Fixes #1411 - Closes #4958 + Closes #5059 -- [Leo Neat brought this change] +- [Patrick Monnerat brought this change] - github action: add CIFuzz + cookie: get_top_domain() sets zero length for null domains - Closes #4960 + This silents a compilation warning with gcc -O3. -- cleanup: comment typos - - Spotted by 'codespell' - - Closes #4957 +- [Patrick Monnerat brought this change] -Steve Holme (20 Feb 2020) -- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions + test 1560: avoid valgrind false positives - Whilst lib\md4.c used this pre-processor, lib\md5.c and - src\tool_metalink.c did not and simply relied on the WIN32 - pre-processor directive. + When using maximum code optimization level (-O3), valgrind wrongly + detects uses of uninitialized values in strcmp(). - Reviewed-by: Marcel Raad - Closes #4955 + Preset buffers with all zeroes to avoid that. -Daniel Stenberg (19 Feb 2020) -- connect: remove some spurious infof() calls - - As they were added primarily for debugging, they provide little use for - users. +Steve Holme (8 Mar 2020) +- sha256: Added WinCrypt implementation - Closes #4951 + Closed #5030 -- HTTP-COOKIES: mention that a trailing newline is required +- sha256: Added SecureTransport implementation + +Daniel Stenberg (7 Mar 2020) +- lib1564: reduce number of mid-wait wakeup calls - ... so that we know we got the whole and not a partial line. + This test does A LOT of *wakeup() calls and then calls curl_multi_poll() + twice. The first *poll() is then expected to return early and the second + not - as the first is supposed to drain the socketpair pipe. - Also, changed the formatting of the fields away from a table again since - the table format requires a github-markdown tool version that we don't - run on the web server atm. + It turns out however that when given "excessive" amounts of writes to + the pipe, some operating systems (the Solaris based are known) will + return EAGAIN before the pipe is drained, which in our test case causes + the second *poll() call to also abort early. - Reported-by: Sunny Bean - Fixes #4946 - Closes #4947 - -- nit: Copyright year out of date + This change attempts to avoid the OS-specific behaviors in the test by + reducing the amount of wakeup calls from 1234567 to 10. - Follow-up to 1fc0617dcc + Reported-by: Andy Fiddaman + Fixes #5037 + Closes #5058 -Jay Satiro (18 Feb 2020) -- tool_util: Improve Windows version of tvnow() - - - Change tool_util.c tvnow() for Windows to match more closely to - timeval.c Curl_now(). +- [Patrick Monnerat brought this change] + + mime: fix the binary encoder to handle large data properly - - Create a win32 init function for the tool, since some initialization - is required for the tvnow() changes. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Prior to this change the monotonic time function used by curl in Windows - was determined at build-time and not runtime. That was a problem because - when curl was built targeted for compatibility with old versions of - Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps - every 49.7 days that Windows has been running. + Fixes #4860 + Closes #4833 + Reported-by: RuurdBeerstra on github + +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - This change makes curl behave similar to libcurl's tvnow function, which - determines at runtime whether the OS is Vista+ and if so calls - QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used - because it has higher resolution than the more obvious candidate - GetTickCount64). The changes to tvnow are basically a copy and paste but - the types in some cases are different. + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Ref: https://github.com/curl/curl/issues/3309 + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 667 checks mimepost using single-byte read callback with encoder. + New test 668 checks the end of part data early detection. - Closes https://github.com/curl/curl/pull/4847 + Fixes #4826 + Reported-by: MrdUkk on github -Daniel Stenberg (18 Feb 2020) -- SOCKS: fix typo in printf formatting - - Follow-up to 4a4b63daa - - Reported-by: Peter Piekarski - Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 +- [Patrick Monnerat brought this change] -- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section + mime: latch last read callback status. - to be in sync with the description above + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Reported-by: Joonas Kuorilehto - Fixes #4943 - Closes #4945 + Fixes #4813 + Reported-by: MrdUkk on github -- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" +Marc Hoersken (7 Mar 2020) +- runtests: fix missing use of exe_ext helper function -- altsvc: make saving the cache an atomic operation - - ... by writing the file to temp name then rename to the final when done. - - Assisted-by: Jay Satiro - Fixes #4936 - Closes #4942 +Daniel Stenberg (7 Mar 2020) +- [Ernst Sjöstrand brought this change] -- rename: a new file for Curl_rename() + ares: store dns parameters for duphandle - And make the cookie save function use it. - -- cookies: make saving atomic with a rename + With c-ares the dns parameters lives in ares_channel. Store them in the + curl handle and set them again in easy_duphandle. - Saves the file as "[filename].[8 random hex digits].tmp" and renames - away the extension when done. + Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. + + Fixes #4893 + Closes #5020 + Signed-off-by: Ernst Sjöstrand + +- version: make curl_version* thread-safe without using global context - Co-authored-by: Jay Satiro - Reported-by: Mike Frysinger - Fixes #4914 - Closes #4926 + Closes #5010 - RELEASE-NOTES: synced -- socks: make the connect phase non-blocking +Marc Hoersken (7 Mar 2020) +- tests: use native Sleep function as fallback on Windows - Removes two entries from KNOWN_BUGS. + Reviewed-By: Daniel Stenberg + Closes #5054 + +- perl: align order and completeness of Windows OS checks + +Daniel Stenberg (7 Mar 2020) +- tool_cb_see: set correct copyright year range - Closes #4907 + Follow-up to a39e5bfb9 -- multi: if Curl_readwrite sets 'comeback' use expire, not loop +Marc Hoersken (7 Mar 2020) +- seek: fix fallback for missing ftruncate on Windows - Otherwise, a very fast single transfer ricks starving out other - concurrent transfers. + This fixes test 198 on versions of MinGW-w64 without ftruncate - Closes #4927 + Reviewed-By: Daniel Stenberg + Reviewed-By: Marcel Raad + Closes #5055 -- ftp: convert 'sock_accepted' to a plain boolean +- config-win32: Windows does not have ftruncate + +Daniel Stenberg (7 Mar 2020) +- pause: force a connection (re-)check after unpausing - This was an array indexed with sockindex but it was only ever used for - the secondary socket. + There might be data available that was already read off the socket, for + example in the TLS layer. - Closes #4929 + Reported-by: Anders Berg + Fixes #4966 + Closes #5049 -Jay Satiro (15 Feb 2020) -- CURLINFO_COOKIELIST.3: Fix example +- socks5: switch state properly when the resolve is done - Prior to this change the example would try to import cookies from stdin, - which wasn't what was intended. + Regression from 4a4b63d (and #4907) + Reported-by: vitaha85 on github + Fixes #5053 + Closes #5056 + +Jay Satiro (7 Mar 2020) +- libssh: Fix matching user-specified MD5 hex key - Reported-by: 3dyd@users.noreply.github.com + Prior to this change a match would never be successful because it + was mistakenly coded to compare binary data from libssh to a + user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). - Fixes https://github.com/curl/curl/issues/4930 - -Daniel Stenberg (14 Feb 2020) -- TODO: Paged searches on LDAP server + Reported-by: fds242@users.noreply.github.com - Closes #4452 + Fixes https://github.com/curl/curl/issues/4971 + Closes https://github.com/curl/curl/pull/4974 -- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS +Daniel Stenberg (6 Mar 2020) +- pause: bail out on bad input - Closes #4108 + A NULL easy handle or an easy handle without an associated connection + cannot be paused or unpaused. + + Closes #5050 -- azure: disable brotli on the macos debug-builds +Steve Holme (6 Mar 2020) +- unit1612: fixed the inclusion and compilation of the HMAC unit test - Because of: + Follow up to 3f74e5e6 to fix: - brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] - const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], + - A typo in Makefile.inc where unit1611 was used instead + - Some compilation issues in unit1612.c - Closes #4925 + Closes #5024 -Steve Holme (13 Feb 2020) -- tool_home: Fix the copyright year being out of date +Daniel Stenberg (6 Mar 2020) +- pause: return early for calls that don't change pause state - Follow up to 9dc350b6. + Reviewed-by: Patrick Monnerat + Ref: #4833 + Closes #5026 -Jay Satiro (12 Feb 2020) -- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() +Jay Satiro (6 Mar 2020) +- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - - Deduplicate GetEnv() code. + Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused + CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. - - On Windows change ultimate call to use Windows API - GetEnvironmentVariable() instead of C runtime getenv(). + Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html + Reported-by: Felipe Gasper - Prior to this change both libcurl and the tool had their own GetEnv - which over time diverged. Now the tool's GetEnv is a wrapper around - curl_getenv (libcurl API function which is itself a wrapper around - libcurl's GetEnv). + Closes https://github.com/curl/curl/pull/5048 + +- multi: skip EINTR check on wakeup socket if it was closed - Furthermore this change fixes a bug in that Windows API - GetEnvironmentVariable() is called instead of C runtime getenv() to get - the environment variable since some changes aren't always visible to the - latter. + - Don't check errno on wakeup socket if sread returned 0 since sread + doesn't set errno in that case. - Reported-by: Christoph M. Becker + This is a follow-up to cf7760a from several days ago which fixed + Curl_multi_wait to stop busy looping sread on the non-blocking wakeup + socket if it was closed (ie sread returns 0). Due to a logic error it + was still possible to busy loop in that case if errno == EINTR. - Fixes https://github.com/curl/curl/issues/4774 - Closes https://github.com/curl/curl/pull/4863 + Closes https://github.com/curl/curl/pull/5047 -Daniel Stenberg (12 Feb 2020) -- strerror.h: Copyright year out of date - - Follow-up to 1c4fa67e8a8fcf6 +Daniel Stenberg (6 Mar 2020) +- transfer: set correct copyright year range -Jay Satiro (12 Feb 2020) -- strerror: Increase STRERROR_LEN 128 -> 256 - - STRERROR_LEN is the constant used throughout the library to set the size - of the buffer on the stack that the curl strerror functions write to. +- urldata: remove the 'stream_was_rewound' connectdata struct member - Prior to this change some extended length Windows error messages could - be truncated. + ... as it is never set anywhere. - Closes https://github.com/curl/curl/pull/4920 + Follow-up to 2f44e94ef + Closes #5046 -- multi: fix outdated comment +- Revert "pause: force-drain the transfer on unpause" - - Do not say that conn->data is "cleared" by multi_done(). + This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) - If the connection is in use then multi_done assigns another easy handle - still using the connection to conn->data, therefore in that case it is - not cleared. + Clearly that didn't solve the problem correctly. - Closes https://github.com/curl/curl/pull/4901 + Reported-by: Christopher Reid + Reopens #4966 + Fixes #5044 -- easy: remove dead code +- RELEASE-NOTES: synced - multi is already assigned to data->multi by curl_multi_add_handle. + and bumped curlver.h + +- MANUAL: update a dict-using command line - Closes https://github.com/curl/curl/pull/4900 + The 'web1913' database is now invalid, use 'gcide' instead. -Daniel Stenberg (12 Feb 2020) -- create-dirs.d: mention the mode +- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - Reported-by: Dan Jacobson - Fixes #4766 - Closes #4916 + Closes #3841 -- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording +- polarssl: remove more references and mentions Assisted-by: Jay Satiro - Reported-by: Craig Andrews - Fixes #4909 - Closes #4910 + Follow-up to 6357a19ff29dac04 + Closes #5036 -- RELEASE-NOTES: synced +Marc Hoersken (4 Mar 2020) +- tests: wrap ignored test failures in braces -Steve Holme (9 Feb 2020) -- smtp: Simplify the MAIL command and avoid a duplication of send strings +- tests: align some Windows sleep defines with each other + +- tests: try to make sleeping portable by avoiding select - This avoids the duplication of strings when the optional AUTH and SIZE - parameters are required. It also assists with the modifications that - are part of #4892. + select does not support just waiting on Windows: + https://perldoc.perl.org/perlport.html#select - Closes #4903 + Reviewed-By: Daniel Stenberg + Closes #5035 -Daniel Stenberg (9 Feb 2020) -- altsvc: keep a copy of the file name to survive handle reset - - The alt-svc cache survives a call to curl_easy_reset fine, but the file - name to use for saving the cache was cleared. Now the alt-svc cache has - a copy of the file name to survive handle resets. +Daniel Stenberg (4 Mar 2020) +- runtests.1: rephrase how to specify what tests to run - Added test 1908 to verify. + Also mention the new tilde-prefixed way to ignore test results. - Reported-by: Craig Andrews - Fixes #4898 - Closes #4902 + Reviewed-By: Marc Hoersken + Closes #5033 -Steve Holme (9 Feb 2020) -- url: Include the failure reason when curl_win32_idn_to_ascii() fails +- cirrus-ci: disable the FreeBSD 13 builds - Provide the failure reason in the failf() info just as we do for the - libidn2 version of code. + FreeBSD 13.0 is apparently close to a year away from a stable release + and has proven to cause intermittent builds failures recently. - Closes #4899 + Assisted-by: Dan Fandrich + Assisted-by: Fedor Korotkov + Fixes #5028 + Closes #5029 -Jay Satiro (9 Feb 2020) -- asyn-thread: remove dead code +Version 7.69.0 (4 Mar 2020) -Daniel Stenberg (8 Feb 2020) -- [Emil Engler brought this change] +Daniel Stenberg (4 Mar 2020) +- RELEASE-NOTES: 7.69.0 - github: Instructions to post "uname -a" on Unix systems in issues +- THANKS: from 7.69.0 - Closes #4896 - -- [Cristian Greco brought this change] + Now sorted case insensitive - configure.ac: fix comments about --with-quiche - - A simple s/nghttp3/quiche in some comments of --with-quiche. - Looks like a copy-paste error from --with-nghttp3. +Marc Hoersken (3 Mar 2020) +- ci/tests: fix escaping of testnames and disable proxy for CI APIs - Closes #4897 + Follow up to ada581f and c0d8b96 + Closes #5031 -Steve Holme (7 Feb 2020) -- checksrc.bat: Fix not being able to run script from the main curl directory - - If the script was ran from the main curl directory rather then the - projects directory then the script would simply exit without error: +Jay Satiro (3 Mar 2020) +- cmake: Show HTTPS-proxy in the features output - C:\url> projects\checksrc.bat + - Show HTTPS-proxy in the features output for those backends that + support it: OpenSSL, GnuTLS and NSS. - The user would either need to change to the projects directory, - explicitly specify the current working directory, or perform a - oneline hacky workaround: + Prior to this change HTTPS-proxy was missing from the cmake features + output even if curl was built with it. Only cmake output was affected. + Both the library and tool correctly reported the feature. - C:\url> cd projects - C:\url\projects> checksrc.bat + Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html + Reported-by: David Lopes - C:\url> checksrc.bat %cd% + Closes https://github.com/curl/curl/pull/5025 + +Marc Hoersken (3 Mar 2020) +- ci/tests: Make it possible to still run but ignore failing tests - C:\url> pushd projects & checksrc.bat & popd + This enables the development of a solution for the failing tests by + running them on CI while ignoring their result for the overall status. - Closes #4894 + Closes #4994 -Daniel Stenberg (7 Feb 2020) -- [Pierre-Yves Bigourdan brought this change] +- README.md: add Azure DevOps Pipelines build status badge - digest: Do not quote algorithm in HTTP authorisation +- ci/tests: Move CI test result creation above environment setup - RFC 7616 section 3.4 (The Authorization Header Field) states that "For - historical reasons, a sender MUST NOT generate the quoted string syntax - for the following parameters: algorithm, qop, and nc". This removes the - quoting for the algorithm parameter. + This avoids using our test servers as proxy to the AppVeyor API. - Reviewed-by: Steve Holme - Closes #4890 + Closes #5022 -- ftp: remove the duplicated user/password struct fields +- ci/tests: Send test results to AppVeyor for status overview - Closes #4887 + Closes #5021 -- ftp: remove superfluous checking for crlf in user or pwd +Daniel Stenberg (3 Mar 2020) +- Revert "sha256: Added SecureTransport implementation" - ... as this is already done much earlier in the URL parser. + This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) - Also add test case 894 that verifies that pop3 with an encodedd CR in - the user name is rejected. + That commit broke test 1610 on macos builds without TLS. - Closes #4887 + Closes #5027 -Steve Holme (6 Feb 2020) -- ntlm_wb: Use Curl_socketpair() for greater portability +- dist: include tests/azure.pm in the tarball - Reported-by: Daniel Stenberg - Closes #4886 + Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 + Reported-by: Marcel Raad -Daniel Stenberg (5 Feb 2020) -- [Frank Gevaerts brought this change] +Steve Holme (3 Mar 2020) +- configure.ac: Disable metalink if mbedTLS is specified + + Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as + being one of the backends that metalink needs to be disabled for, I + seem to have included it in the list of allowed SSL/TLS backends in + comnfigure.ac :( + + Closes #5013 - contributors: Also include people who contributed to curl-www +- sha256: Tidy up following recent changes - Closes #4884 + Reviewed-by: Daniel Stenberg + Closes #4956 -- [Frank Gevaerts brought this change] +- sha256: Added WinCrypt implementation - contrithanks: Use the most recent tag by default - - (similar to 5296abe) +- sha256: Added SecureTransport implementation + +- sha256: Added mbedtls implementation + +- sha256: Added GNU TLS gcrypt implementation + +- sha256: Added GNU TLS Nettle implementation + +Jay Satiro (2 Mar 2020) +- curl_escape.3: Add a link to curl_free - Closes #4883 + Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 -- scripts: use last set tag if none given +- curl_getenv.3: Fix the memory handling description - Makes 'delta' and 'contributors.sh' easier to use. + - Tell the user to call curl_free() to free the pointer returned by + curl_getenv(). - Make the delta script invoke contrithanks to get current number of - contributors instead of counting THANKS, for accuracy. + Prior to this change the user was directed to call free(), but that + would not work in cases where the library and application use separate C + runtimes and therefore have separate heap memory management. - Closes #4881 + Closes https://github.com/curl/curl/pull/5016 -- ftp: shrink temp buffers used for PORT - - These two stack based buffers only need to be 46 + 66 bytes instead of - 256 + 1024. - - Closes #4880 +Daniel Stenberg (2 Mar 2020) +- [Nick Zitzmann brought this change] -- curl: error on --alt-svc use w/o support + md4: use init/update/final functions in Secure Transport - Make the tool check for alt-svc support at run-time and return error - accordingly if not present when the option is used. + We can use CC_MD4_Init/Update/Final without having to allocate memory + directly. - Reported-by: Harry Sintonen - Closes #4878 + Closes #4979 -- docs/HTTP3: add --enable-alt-svc to curl's configure +Marc Hoersken (2 Mar 2020) +- ci/tests: some MacOS builds randomly take longer than 20min -- RELEASE-PROCEDURE: feature win is closed post-release a few days +Daniel Stenberg (2 Mar 2020) +- multi_wait: stop loop when sread() returns zero - We've tried to uphold this already but let's make it official by - publicly stating this is the way we do it. + It's unclear why it would ever return zero here, but this change fixes + Robert's problem and it shouldn't loop forever... - Closes #4877 + Reported-by: Robert Dunaj + Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html + Closes #5019 -- altsvc: set h3 version at a common single spot +- http: mark POSTs with no body as "upload done" from the start - ... and move the #ifdefs out of the functions. Addresses the fact they - were different before this change. + As we have logic that checks if we get a >= 400 reponse code back before + the upload is done, which then got confused since it wasn't "done" but + yet there was no data to send! - Reported-by: Harry Sintonen - Closes #4876 - -- [Harry Sintonen brought this change] + Reported-by: IvanoG on github + Fixes #4996 + Closes #5002 - altsvc: improved header parser - - - Fixed the flag parsing to apply to specific alternative entry only, as - per RFC. The earlier code would also get totally confused by - multiprotocol header, parsing flags from the wrong part of the header. - - - Fixed the parser terminating on unknown protocols, instead of skipping - them. +- tests: disable 962, 963 and 964 on Windows - - Fixed a busyloop when protocol-id was present without an equal sign. + These tests are also doing UTF-8 SMTP. - Closes #4875 + Follow-up to df207d2dd93b9e73 -- [Harry Sintonen brought this change] +Marc Hoersken (2 Mar 2020) +- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer - ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 +Daniel Stenberg (2 Mar 2020) +- configure: bump the AC_COPYRIGHT year range -- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 +- [Steve Holme brought this change] + + tests: disable SMTP UTF-8 tests on Windows - Reported-by: James Fuller + Fixes #4988 + Closes #4992 -Steve Holme (4 Feb 2020) -- ntlm: Pass the Curl_easy structure to the private winbind functions +- formdata/mime: copyright year range update - ...rather than the full conndata structure. + Due to the merge/revert cycle -Daniel Stenberg (4 Feb 2020) -- RELEASE-NOTES: synced +- Revert "mime: latch last read callback status." + + This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. + + Fixes #5014 + Closes #5015 + Reopens #4833 -- tool_operhlp: Copyright year out of date, should be 2020 +- Revert "mime: do not perform more than one read in a row" - Follow-up from 2bc373740a3 + This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. -- [Orgad Shaneh brought this change] +- Revert "mime: fix the binary encoder to handle large data properly" + + This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. - curl: avoid using strlen for testing if a string is empty +- altsvc: both h3 backends now speak h3-27 - Closes #4873 + ... also updated the HTTP3 build description for ngtcp2 accordingly. -Steve Holme (3 Feb 2020) -- ntlm: Ensure the HTTP header data is not stored in the challenge/response +- [Patrick Monnerat brought this change] -Marcel Raad (3 Feb 2020) -- openssl: remove redundant assignment + mime: fix the binary encoder to handle large data properly - Fixes a scan-build failure on Bionic. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes https://github.com/curl/curl/pull/4872 + Fixes #4860 + Reported-by: RuurdBeerstra on github -- travis: update non-OpenSSL Linux jobs to Bionic +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is - failing with "curl returned 52, when expecting 35". + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Closes https://github.com/curl/curl/pull/4872 - -Dan Fandrich (3 Feb 2020) -- cirrus: Add some missing semicolons + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 664 checks mimepost using single-byte read callback with encoder. + New test 665 checks the end of part data early detection. - Newlines aren't preserved in this section so they're needed to separate - commands. The exports luckily worked anyway as a single long line, but - erroneously exported a variable called "export" - [skip ci] - -Daniel Gustafsson (2 Feb 2020) -- [Pedro Monreal brought this change] + Fixes #4826 + Reported-by: MrdUkk on github - cleanup: fix typos and wording in docs and comments - - Closes #4869 - Reviewed-by: Emil Engler and Daniel Gustafsson +- [Patrick Monnerat brought this change] -Steve Holme (2 Feb 2020) -- ntlm: Move the winbind data into the NTLM data structure + mime: latch last read callback status. - To assist with adding winbind support to the SASL NTLM authentication, - move the winbind specific data out of conndata into ntlmdata. - -Daniel Stenberg (30 Jan 2020) -- quiche: Copyright year out of date + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Follow-up to 7fc63d72333a + Fixes #4813 + Reported-by: MrdUkk on github + Closes #4833 -- altsvc: use h3-25 +Steve Holme (1 Mar 2020) +- unit1651: Fixed conversion compilation warning - Closes #4868 - -- [Alessandro Ghedini brought this change] - - quiche: update to draft-25 + 371:17: warning: conversion to 'unsigned char' from 'int' may alter its + value [-Wconversion] - Closes #4867 + Closes #5008 -- ngtcp2: update to git master and its draft-25 support +- configure.ac: Disable metalink support if an incompatible SSL/TLS specified - Closes #4865 - -- cookie: check __Secure- and __Host- case sensitively + tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, + The Win32 Crypto library and Apple's Common Crypto library. - While most keywords in cookies are case insensitive, these prefixes are - specified explicitly to get checked "with a case-sensitive match". + If an TLS backend such as mbedTLS or WolfSSL is specified then the + following error is given during compilation along, with a load of + unresolved extern errors: - (From the 6265bis document in progress) + Can't compile METALINK support without a crypto library. - Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 - Closes #4864 + Reviewed-by: Daniel Stenberg + Closes #5006 -- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header +Marc Hoersken (1 Mar 2020) +- ci/tests: Update Azure DevOps pipeline job display names + + Make the configure step more descriptive and align others. -- oauth2-bearer.d: works for HTTP too +- ci/tests: Fix typo in previous commit 597cf2 + +- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available - Reported-by: Mischa Salle - Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html - Closes #4862 + For security reasons the access token is not available to PR builds. + Therefore we should not try to use the DevOps API with an empty token. -- multi_done: if multiplexed, make conn->data point to another transfer +Daniel Stenberg (1 Mar 2020) +- build: remove all HAVE_OPENSSL_ENGINE_H defines - ... since the current transfer is being killed. Setting to NULL is - wrong, leaving it pointing to 'data' is wrong since that handle might be - about to get freed. + ... as there's nothing in the code that actually uses the define! The + last reference was removed in 38203f158. - Fixes #4845 - Closes #4858 - Reported-by: dmitrmax on github + Closes #5007 -- location.d: the method change is from POST to GET only +Jay Satiro (29 Feb 2020) +- [Rolf Eike Beer brought this change] + + CMake: clean up and improve build procedures - Not from generic non-GET to GET. + - remove check for unsupported old CMake versions - Reported-by: Andrius Merkys - Ref: #4859 - Closes #4861 - -- urlapi: guess scheme correct even with credentials given + - do not link to c-ares library twice - In the "scheme-less" parsing case, we need to strip off credentials - first before we guess scheme based on the host name! + - modernize custom Find modules - Assisted-by: Jay Satiro - Fixes #4856 - Closes #4857 - -- global_init: move the IPv6 works status bool to multi handle + - FindLibSSH2: + - pass version to FPHSA to show it in the output + - use LIBSSH2_VERSION define to extract the version number in + one shot. This variable exists in the header for 10 years. + - remove unneeded code - Previously it was stored in a global state which contributed to - curl_global_init's thread unsafety. This boolean is now instead figured - out in curl_multi_init() and stored in the multi handle. Less effective, - but thread safe. + - FindNGHTTP2.cmake: + - drop needless FPHSA argument + - mark found variables as advanced - Closes #4851 - -- [Jay Satiro brought this change] - - README: mention that the docs is in docs/ + - FindNSS.cmake: + - show version number - Reported-by: Austin Green - Fixes #4830 - Closes #4853 - -- curl.h: define CURL_WIN32 on windows + - FindCARES.cmake: + - drop default paths + - use FPHSA instead of checking things by hand - ... so that the subsequent logic below can use a single known define to know - when built on Windows (as we don't define WIN32 anymore). + - remove needless explict variable dereference - Follow-up to 1adebe7886ddf20b + - simplify count_true() - Reported-by: crazydef on github - Assisted-by: Marcel Raad - Fixes #4854 - Closes #4855 - -- RELEASE-NOTES: synced - -- [Jon Rumsey brought this change] - - urldata: do string enums without #ifdefs for build scripts + - allow all policies up to version 3.16 to be set to NEW - ... and check for inconsistencies for OS400 at build time with the new - chkstrings tool. + - do not rerun check for -Wstrict-aliasing=3 every time - Closes #4822 - -- curl: make the -# spaceship bar not wrap the line + In contrast to every other compiler flag this has a = in it, which CMake + can't have in a variable name. - The fixed-point math made us lose precision and thus a too high index - value could be used for outputting the hashtags which could overwrite - the newline. + - only read the interesting strings from curlver.h - The fix increases the precision in the sine table (*100) and the - associated position math. + Reviewed-by: Peter Wu - Reported-by: Andrew Potter - Fixes #4849 - Closes #4850 + Closes https://github.com/curl/curl/pull/4975 -- global_init: assume the EINTR bit by default - - - Removed from global_init since it isn't thread-safe. The symbol will - still remain to not break compiles, it just won't have any effect going - forward. +- runtests: fix output to command log - - make the internals NOT loop on EINTR (the opposite from previously). - It only risks returning from the select/poll/wait functions early, and that - should be risk-free. + - Record only the command of the most recently ran test in the command + log. - Closes #4840 - -- [Peter Piekarski brought this change] - - conn: do not reuse connection if SOCKS proxy credentials differ + This is a follow-up to 02988b7 from several weeks ago which fixed + writing to the command log, however it saved all commands for all tests + instead of just the most recently ran test as we would now expect. - Closes #4835 + Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 + Closes https://github.com/curl/curl/pull/5001 -- llist: removed unused Curl_llist_move() +Steve Holme (1 Mar 2020) +- polarssl: Additional removal - (and the corresponding unit test) + Follow up to 6357a19f. - Closes #4842 + Reviewed-by: Daniel Stenberg + Closes #5004 -- conncache: removed unused Curl_conncache_bundle_size() +- [Jonathan Cardoso Machado brought this change] -- strcase: turn Curl_raw_tolower into static + docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always - Only ever used from within this file. - -- singleuse.pl: support new API functions, fix curl_dbg_ handling + Reviewed-by: Steve Holme + Closes #5005 -- wolfssh: make it init properly via Curl_ssh_init() +- md5: Added implementation for mbedTLS - Closes #4846 + Reviewed-by: Jay Satiro + Closes #4980 -- [Aron Rotteveel brought this change] +- md5: Use pointer notation for array parameters in GnuTLS implementation - form.d: fix two minor typos +- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 - Closes #4843 + Closes #4983 -- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields - - Avoid "reparsing" the content and instead deliver more exactly what is - provided in the certificate and avoid truncating the data after 512 - bytes as done previously. This no longer removes embedded newlines. - - Fixes #4837 - Reported-by: bnfp on github - Closes #4841 +Marc Hoersken (29 Feb 2020) +- ci/tests: Send test results to Azure DevOps for reporting -Jay Satiro (23 Jan 2020) -- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 +Daniel Stenberg (29 Feb 2020) +- pause: force-drain the transfer on unpause - - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. + ... since the socket might not actually be readable anymore when for + example the data is already buffered in the TLS layer. - Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the - CURLOPT_PROXY_SSL_OPTIONS description. + Fixes #4966 + Reported-by: Anders Berg + Closes #5000 -Daniel Stenberg (22 Jan 2020) -- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER - - For now, no cert in the bundle actually sets a date there... +- TODO: curl --proxycommand - Co-Authored-by: Jay Satiro - Reported-by: Christian Heimes - Fixes #4834 - Closes #4836 - -- RELEASE-NOTES: synced - -- [Pavel Volgarev brought this change] + Suggested-by: Kristian Mide + Closes #4941 - smtp: Allow RCPT TO command to fail for some recipients +- smtp: overwriting 'from' leaks memory - Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. + Detected by Coverity. CID 1418139. - Verified with the new tests 3002-3007 + Also, make sure to return error if the new 'from' allocation fails. - Closes #4816 + Closes #4997 -- copyright: fix year ranges +- CIfuzz: switch off 'dry_run' mode - follow-up from dea17b519d (one of these days I'll learn to check before - I push) - -- [nao brought this change] - - http: move "oauth_bearer" from connectdata to Curl_easy + Follow-up from #4960: now make it fail if it detects problems. - Fixes the bug where oauth_bearer gets deallocated when we re-use a - connection. + Closes #4998 + +Marc Hoersken (28 Feb 2020) +- ci/tests: Increase timeouts of Windows builds due to new tests - Closes #4824 + Recently added tests increased their runtime above the limit of 60min. -- [Emil Engler brought this change] +- ci/tests: align Azure Pipeline job names with each other - curl: Let -D merge headers in one file again - - Closes #4762 - Fixes #4753 +- ci/tests: Add Windows builds via Azure Pipelines using Docker -- data.d: remove "Multiple files can also be specified" +- tests: fix Python 3 compatibility of smbserver.py + +Daniel Stenberg (27 Feb 2020) +- runtests: restore the command log - It is superfluous and could even be misleading. + The log file with all command lines for the invoked command lines is now + called logs/commands.log - Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html - Reported-by: Mike Norton - Closes #4832 + Fixes #4911 + Closes #4989 -Marcel Raad (20 Jan 2020) -- CMake: support specifying the target Windows version +- smtp: fix memory leak on exit path - Previously, it was only possible to set it to Windows Vista or XP by - setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. - Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to - explicitly set the target Windows version. `ENABLE_INET_PTON` is - ignored in this case. + Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' + going out of scope leaks the storage it points to" - Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 - Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 - Closes https://github.com/curl/curl/pull/4815 + Closes #4990 -Daniel Stenberg (20 Jan 2020) -- http.h: Copyright year out of date, should be 2020 +Steve Holme (27 Feb 2020) +- gtls: Fixed compilation when using GnuTLS < 3.5.0 - Follow-up to 7ff9222ced8c + Reverts the functionality from 41fcb4f when compiling with GnuTLS older + than 3.5.0. + + Reviewed-by: Daniel Stenberg + Closes #4984 -- [加藤郁之 brought this change] +- RELEASE-NOTES: Corrected the link to issue #4892 - HTTP: increase EXPECT_100_THRESHOLD to 1Mb - - Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html +Daniel Stenberg (27 Feb 2020) +- Curl_is_ASCII_name: handle a NULL argument - Closes #4814 - -- ROADMAP: thread-safe `curl_global_init()` + Make the function tolerate a NULL pointer input to avoid dereferencing + that pointer. - I'd like to see this happen. + Follow-up to efce3ea5a85126d + Detected by OSS-Fuzz + Reviewed-By: Steve Holme + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 + Fixes #4985 + Closes #4986 - RELEASE-NOTES: synced -- wolfssl: use the wc-prefixed symbol alternatives +- http2: make pausing/unpausing set/clear local stream window - The symbols without wc_ prefix are not always provided. + This reduces the HTTP/2 window size to 32 MB since libcurl might have to + buffer up to this amount of data in memory and yet we don't want it set + lower to potentially impact tranfer performance on high speed networks. - Ref: https://github.com/wolfSSL/wolfssl/issues/2744 + Requires nghttp2 commit b3f85e2daa629 + (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end + up in the next release after 1.40.0. - Closes #4827 + Fixes #4939 + Closes #4940 -- polarssl: removed - - As detailed in DEPRECATE.md, the polarssl support is now removed after - having been disabled for 6 months and nobody has missed it. +- [Anderson Toshiyuki Sasaki brought this change] + + libssh: improve known hosts handling - The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix - instead of the former 'polarssl' and the common functions that - previously were shared between mbedtls and polarssl and contained the - name 'polarssl' have now all been renamed to instead say 'mbedtls'. + Previously, it was not possible to get a known hosts file entry due to + the lack of an API. ssh_session_get_known_hosts_entry(), introduced in + libssh-0.9.0, allows libcurl to obtain such information and behave the + same as when compiled with libssh2. - Closes #4825 - -Marcel Raad (16 Jan 2020) -- libssh2: fix variable type + This also tries to avoid the usage of deprecated functions when the + replacements are available. The behaviour will not change if versions + older than libssh-0.8.0 are used. - This led to a conversion warning on 64-bit MinGW, which has 32-bit - `long` but 64-bit `size_t`. + Signed-off-by: Anderson Toshiyuki Sasaki - Closes https://github.com/curl/curl/pull/4823 + Fixes #4953 + Closes #4962 -Daniel Stenberg (16 Jan 2020) -- curl:progressbarinit: ignore column width from terminals < 20 +Steve Holme (27 Feb 2020) +- tests: Automatically deduce the tool name from the test case for unit tests - To avoid division by zero - or other issues. + It is still possible to override the executable to run during the test, + using the tag, but this patch removes the requirement that the + tag must be present for unit tests. - Reported-by: Daniel Marjamäki - Closes #4818 - -- wolfssh: set the password correctly for PASSWORD auth + It also removes the possibility of human error when existing test cases + are used as the basis for new tests, as recently witnessed in 81c37124. + + Reviewed-by: Daniel Stenberg + Closes #4976 -- wolfssh: remove fprintf() calls (and uses of __func__) +- test1323: Added the missing 'unit test' feature requirement in the test case -Marcel Raad (14 Jan 2020) -- CMake: use check_symbol_exists also for inet_pton +Daniel Stenberg (26 Feb 2020) +- cookie: remove unnecessary check for 'out != 0' - It doesn't make much sense to only check if the function can be linked - when it's not declared in any header and that is treated as an error. - With the correct target Windows version set, the function is declared - in ws2tcpip.h and the comment above the modified block is invalid. + ... as it will always be non-NULL at this point. - Also, move the definition of `_WIN32_WINNT` up to before all symbol - availability checks so that we don't have to care which ones must be - done after it. + Detected by Coverity: CID 1459009 + +- http: added 417 response treatment - Tested with Visual Studio 2019 and current MinGW-w64. + When doing a request with a body + Expect: 100-continue and the server + responds with a 417, the same request will be retried immediately + without the Expect: header. - Closes https://github.com/curl/curl/pull/4808 - -Jay Satiro (13 Jan 2020) -- schannel_verify: Fix alt names manual verify for UNICODE builds + Added test 357 to verify. - Follow-up to 29e40a6 from two days ago, which added that feature for - Windows 7 and earlier. The bug only occurred in same. + Also added a control instruction to tell the sws test server to not read + the request body if Expect: is present, which the new test 357 uses. - Ref: https://github.com/curl/curl/pull/4761 + Reported-by: bramus on github + Fixes #4949 + Closes #4964 -Daniel Stenberg (13 Jan 2020) -- HTTP-COOKIES.md: describe the cookie file format +Steve Holme (26 Feb 2020) +- smtp: Tidy up, following recent changes, to maintain the coding style - ... and refer to that file from from CURLOPT_COOKIEFILE.3 and - CURLOPT_COOKIELIST.3 + Closes #4892 + +- smtp: Support the SMTPUTF8 extension for the EXPN command - Assisted-by: Jay Satiro - Reported-by: bsammon on github - Fixes #4805 - Closes #4806 + Simply notify the server we support the SMTPUTF8 extension if it does. -- [Tobias Hieta brought this change] +- smtp: Support the SMTPUTF8 extension in the VRFY command - CMake: Add support for CMAKE_LTO option. +- smtp: Support the SMTPUTF8 extension in the RCPT TO command - This enables Link Time Optimization. LTO is a proven technique for - optimizing across compilation units. + Note: The RCPT TO command isn't required to advertise to the server that + it contains UTF-8 characters, instead the server is told that a mail may + contain UTF-8 in any envelope command via the MAIL command. + +- smtp: Support the SMTPUTF8 extension in the MAIL command - Closes #4799 + Support the SMTPUTF8 extension when sending mailbox information in the + MAIL command (FROM and AUTH parameters). Non-ASCII domain names will + be ACE encoded, if IDN is supported, whilst non-ASCII characters in + the local address part are passed to the server. + + Reported-by: ygthien on github + Fixes #4828 -- RELEASE-NOTES: synced +- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 -- ConnectionExists: respect the max_concurrent_streams limits - - A regression made the code use 'multiplexed' as a boolean instead of the - counter it is intended to be. This made curl try to "over-populate" - connections with new streams. +- smtp: Support UTF-8 based host names in the VRFY command + +- smtp: Support UTF-8 based host names in the RCPT TO command + +- smtp: Support UTF-8 based host names in the MAIL command - This regression came with 41fcdf71a1, shipped in curl 7.65.0. + Non-ASCII host names will be ACE encoded if IDN is supported. + +- url: Make the IDN conversion functions available to others + +- smtp: Added UTF-8 mailbox tests to verify existing behaviour + +- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case + +- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() - Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same - check. + The dot character between the host and the tld was not being escaped, + which meant it specified a match of 'any' character rather than an + explicit dot separator. - Reported-by: Kunal Ekawde - Fixes #4779 - Closes #4784 - -- curl: make #0 not output the full URL + Additionally removed the dot character from the host name as it allowed + the following to be specified as a valid address in our test cases: - It was not intended nor documented! + - Added test 1176 to verify. + Both are typos from 98f7ca7 and 8880f84 :( - Reported-by: vshmuk on hackerone + I can't remember whether my intention was to allow sub-domains to be + specified in the host or not with these additional dots, but by placing + it outside of the host means it can only be specified once per domain + and by placing a + after the new grouping support for sub-domains is + kept. - Closes #4812 + Closes #4912 -- wolfSSH: new SSH backend - - Adds support for SFTP (not SCP) using WolfSSH. +- hmac: Added a unit test for the HMAC hash generation - Closes #4231 + Closes #4973 -- curl: remove 'config' field from OutStruct +- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function + +- tests: Added a unit test for MD4 digest generation - As it was just unnecessary duplicated information already stored in the - 'per_transfer' struct and that's around mostly anyway. + Closes #4970 + +- md4: Use const for the length input parameter - The duplicated pointer caused problems when the code flow was aborted - before the dupe was filled in and could cause a NULL pointer access. + This keeps the interface the same as md5 and sha256. + +- test1610: Fixed the link to the unit test - Reported-by: Brian Carpenter - Fixes #4807 - Closes #4810 + Typo from 81c37124. -- misc: Copyright year out of date, should be 2020 +- ntlm: Removed the dependency on the TLS libaries when using MD5 - Follow-up to recent commits + As we have our own MD5 implementation use the MD5 wrapper to remove the + TLS dependency. - [skip ci] + Closes #4967 -Jay Satiro (11 Jan 2020) -- [Santino Keupp brought this change] +- md5/sha256: Updated the functions to allow non-string data to be hashed - libssh2: add support for forcing a hostkey type +- digest: Corrected the name of the local HTTP digest function - - Allow forcing the host's key type found in the known_hosts file. + Follow up to 2b5b37cb. Local static functions do not require the Curl + prefix. + +- tests: Added a unit test for SHA256 digest generation - Currently, curl (with libssh2) does not take keys from your known_hosts - file into account when talking to a server. With this patch the - known_hosts file will be searched for an entry matching the hostname - and, if found, libssh2 will be told to claim this key type from the - server. + Follow up to 2b5b37c. - Closes https://github.com/curl/curl/pull/4747 - -- [Nicolas Guillier brought this change] + Closes #4968 - cmake: Improve libssh2 check on Windows - - - Add "libssh2" name to FindLibSSH2 library search. +- md4: Fixed compilation issues when using GNU TLS gcrypt - On Windows systems, libSSH2 CMake installation may name the library - "LibSSH2". + * Don't include 'struct' in the gcrypt MD4_CTX typedef + * The call to gcry_md_read() should use a dereferenced ctx + * The call to gcry_md_close() should use a dereferenced ctx - Prior to this change cmake only checked for name "ssh2". On Linux that - works fine because it will prepend the "lib", but it doesn't do that on - Windows. + Additional minor whitespace issue in the USE_WIN32_CRYPTO code. - Closes https://github.com/curl/curl/pull/4804 + Closes #4959 -- [Faizur Rahman brought this change] +Daniel Stenberg (21 Feb 2020) +- RELEASE-NOTES: synced - schannel: Make CURLOPT_CAINFO work better on Windows 7 - - - Support hostname verification via alternative names (SAN) in the - peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. +- http2: now require nghttp2 >= 1.12.0 - CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a - result CertGetNameString doesn't quite work on those versions of - Windows. This change provides an alternative solution for - CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier - versions of Windows. + To simplify our code and since earlier versions lack important function + calls libcurl needs to function correctly. - Prior to this change many certificates failed the hostname validation - when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates - now represent multiple hostnames and rely on the alternative names field - exclusively to represent their hostnames. + nghttp2 1.12.0 was relased on June 26, 2016. - Reported-by: Jeroen Ooms + Closes #4961 + +- gtls: fix the copyright year - Fixes https://github.com/curl/curl/issues/3711 - Closes https://github.com/curl/curl/pull/4761 + Follow-up from 41fcb4f609 -- [Emil Engler brought this change] +- [jethrogb brought this change] - ngtcp2: Add an error code for QUIC connection errors - - - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection - errors. + GnuTLS: Always send client cert - Prior to this change CURLE_FAILED_INIT was used, but that was not - correct. + TLS servers may request a certificate from the client. This request + includes a list of 0 or more acceptable issuer DNs. The client may use + this list to determine which certificate to send. GnuTLS's default + behavior is to not send a client certificate if there is no + match. However, OpenSSL's default behavior is to send the configured + certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL + behavior. - Closes https://github.com/curl/curl/pull/4754 + Authored-by: jethrogb on github + Fixes #1411 + Closes #4958 -- multi: Change curl_multi_wait/poll to error on negative timeout - - - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when - curl_multi_wait/poll is passed timeout param < 0. - - Prior to this change passing a negative value to curl_multi_wait/poll - such as -1 could cause the function to wait forever. +- [Leo Neat brought this change] + + github action: add CIFuzz - Reported-by: hamstergene@users.noreply.github.com + Closes #4960 + +- cleanup: comment typos - Fixes https://github.com/curl/curl/issues/4763 + Spotted by 'codespell' - Closes https://github.com/curl/curl/pull/4765 - -- [Marc Aldorasi brought this change] + Closes #4957 - cmake: Enable SMB for Windows builds +Steve Holme (20 Feb 2020) +- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions - - Define USE_WIN32_CRYPTO by default. This enables SMB. + Whilst lib\md4.c used this pre-processor, lib\md5.c and + src\tool_metalink.c did not and simply relied on the WIN32 + pre-processor directive. - - Show whether SMB is enabled in the "Enabled features" output. + Reviewed-by: Marcel Raad + Closes #4955 + +Daniel Stenberg (19 Feb 2020) +- connect: remove some spurious infof() calls - - Fix mingw compiler warning for call to CryptHashData by casting away - const param. mingw CryptHashData prototype is wrong. + As they were added primarily for debugging, they provide little use for + users. - Closes https://github.com/curl/curl/pull/4717 + Closes #4951 -- vtls: Refactor Curl_multissl_version to make the code clearer +- HTTP-COOKIES: mention that a trailing newline is required - Reported-by: Johannes Schindelin + ... so that we know we got the whole and not a partial line. - Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 + Also, changed the formatting of the fields away from a table again since + the table format requires a github-markdown tool version that we don't + run on the web server atm. - Closes https://github.com/curl/curl/pull/4803 + Reported-by: Sunny Bean + Fixes #4946 + Closes #4947 -Daniel Stenberg (10 Jan 2020) -- fix: Copyright year out of date, should be 2020 +- nit: Copyright year out of date - Follow-up to 875314ed0bf3b + Follow-up to 1fc0617dcc -Marcel Raad (10 Jan 2020) -- hostip: move code to resolve IP address literals to `Curl_resolv` +Jay Satiro (18 Feb 2020) +- tool_util: Improve Windows version of tvnow() - The code was duplicated in the various resolver backends. + - Change tool_util.c tvnow() for Windows to match more closely to + timeval.c Curl_now(). - Also, it was called after the call to `Curl_ipvalid`, which matters in - case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This - caused test 1048 to fail on classic MinGW. + - Create a win32 init function for the tool, since some initialization + is required for the tvnow() changes. - The code ignores `conn->ip_version` as done previously in the - individual resolver backends. + Prior to this change the monotonic time function used by curl in Windows + was determined at build-time and not runtime. That was a problem because + when curl was built targeted for compatibility with old versions of + Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps + every 49.7 days that Windows has been running. - Move the call to the `resolver_start` callback up to appease test 655, - which wants it to be called also for literal addresses. + This change makes curl behave similar to libcurl's tvnow function, which + determines at runtime whether the OS is Vista+ and if so calls + QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used + because it has higher resolution than the more obvious candidate + GetTickCount64). The changes to tvnow are basically a copy and paste but + the types in some cases are different. - Closes https://github.com/curl/curl/pull/4798 + Ref: https://github.com/curl/curl/issues/3309 + + Closes https://github.com/curl/curl/pull/4847 -Daniel Stenberg (9 Jan 2020) -- scripts/delta: adapt to new public header layout +Daniel Stenberg (18 Feb 2020) +- SOCKS: fix typo in printf formatting + + Follow-up to 4a4b63daa + + Reported-by: Peter Piekarski + Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 -- test1167: verify global symbols in public headers are curl prefixed +- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section - ... using the new badsymbols.pl perl script + to be in sync with the description above - Fixes #4793 - Closes #4794 + Reported-by: Joonas Kuorilehto + Fixes #4943 + Closes #4945 -- libtest/mk-lib1521: adapt to new public header layout +- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" -- include: remove non-curl prefixed defines +- altsvc: make saving the cache an atomic operation - ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ - enums. + ... by writing the file to temp name then rename to the final when done. + + Assisted-by: Jay Satiro + Fixes #4936 + Closes #4942 -- curl.h: remove WIN32 define +- rename: a new file for Curl_rename() - It isn't our job to define this in a public header - and it defines a - name outside of our naming scope. + And make the cookie save function use it. -- tool_dirhie.c: fix the copyright year range +- cookies: make saving atomic with a rename - Follow-up to: 4027bd72d9 + Saves the file as "[filename].[8 random hex digits].tmp" and renames + away the extension when done. + + Co-authored-by: Jay Satiro + Reported-by: Mike Frysinger + Fixes #4914 + Closes #4926 -- bump: work towards 7.69.0 is started +- RELEASE-NOTES: synced -Jay Satiro (9 Jan 2020) -- tool_dirhie: Allow directory traversal during creation +- socks: make the connect phase non-blocking - - When creating a directory hierarchy do not error when mkdir fails due - to error EACCESS (13) "access denied". + Removes two entries from KNOWN_BUGS. - Some file systems allow for directory traversal; in this case that it - should be possible to create child directories when permission to the - parent directory is restricted. + Closes #4907 + +- multi: if Curl_readwrite sets 'comeback' use expire, not loop - This is a regression caused by me in f16bed0 (precedes curl-7_61_1). - Basically I had assumed that if a directory already existed it would - fail only with error EEXIST, and not error EACCES. The latter may - happen if the directory exists but has certain restricted permissions. + Otherwise, a very fast single transfer ricks starving out other + concurrent transfers. + + Closes #4927 + +- ftp: convert 'sock_accepted' to a plain boolean + + This was an array indexed with sockindex but it was only ever used for + the secondary socket. + + Closes #4929 + +Jay Satiro (15 Feb 2020) +- CURLINFO_COOKIELIST.3: Fix example - Reported-by: mbeifuss@users.noreply.github.com + Prior to this change the example would try to import cookies from stdin, + which wasn't what was intended. - Fixes https://github.com/curl/curl/issues/4796 - Closes https://github.com/curl/curl/pull/4797 - -Daniel Stenberg (9 Jan 2020) -- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers + Reported-by: 3dyd@users.noreply.github.com - Closes #4080 + Fixes https://github.com/curl/curl/issues/4930 -- docs/RELEASE-PROCEDURE.md: pushed some release dates +Daniel Stenberg (14 Feb 2020) +- TODO: Paged searches on LDAP server - Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html + Closes #4452 -- runtests: make random seed fixed for a month +- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS - When using randomized features of runtests (-R and --shallow) it is - useful to have a fixed random seed to make sure for example extra - commits in a branch or a rebase won't change the seed that would make - repeated runs work differently. + Closes #4108 + +- azure: disable brotli on the macos debug-builds - As it is also useful to change seed sometimes, the default seed is now - determined based on the current month (and first line curl -V - output). When the month changes, so will the random seed. + Because of: - The specific seed is also shown in the standard test suite top header - and it can be set explictly with the new --seed=[num] option so that the - exact order of a previous run can be achieved. + brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] + const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], - Closes #4734 + Closes #4925 -- RELEASE-PROCEDURE.md: fix next release date (Feb 26) +Steve Holme (13 Feb 2020) +- tool_home: Fix the copyright year being out of date - [skip ci] - -Version 7.68.0 (8 Jan 2020) - -Daniel Stenberg (8 Jan 2020) -- RELEASE-NOTES: 7.68.0 - -- THANKS: updated with names from the 7.68.0 release + Follow up to 9dc350b6. -- RELEASE-PROCEDURE: add four future release dates +Jay Satiro (12 Feb 2020) +- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() - and remove four past release dates + - Deduplicate GetEnv() code. - [skip ci] - -Marcel Raad (6 Jan 2020) -- TrackMemory tests: always remove CR before LF + - On Windows change ultimate call to use Windows API + GetEnvironmentVariable() instead of C runtime getenv(). - It was removed for output containing ' =' via `s/ =.*//`. With classic - MinGW, this made lines with `free()` end with CRLF, but lines with e.g. - `malloc()` end with only LF. The tests expect LF only. + Prior to this change both libcurl and the tool had their own GetEnv + which over time diverged. Now the tool's GetEnv is a wrapper around + curl_getenv (libcurl API function which is itself a wrapper around + libcurl's GetEnv). - Closes https://github.com/curl/curl/pull/4788 - -Daniel Stenberg (6 Jan 2020) -- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header + Furthermore this change fixes a bug in that Windows API + GetEnvironmentVariable() is called instead of C runtime getenv() to get + the environment variable since some changes aren't always visible to the + latter. - ... to the private multihhandle.h. It is not for public use and it - wasn't prefixed correctly anyway! + Reported-by: Christoph M. Becker - Closes #4790 + Fixes https://github.com/curl/curl/issues/4774 + Closes https://github.com/curl/curl/pull/4863 -- file: fix copyright year range +Daniel Stenberg (12 Feb 2020) +- strerror.h: Copyright year out of date - Follow-up to 1b71bc532bd + Follow-up to 1c4fa67e8a8fcf6 -- curl -w: handle a blank input file correctly +Jay Satiro (12 Feb 2020) +- strerror: Increase STRERROR_LEN 128 -> 256 - Previously it would end up with an uninitialized memory buffer that - would lead to a crash or junk getting output. + STRERROR_LEN is the constant used throughout the library to set the size + of the buffer on the stack that the curl strerror functions write to. - Added test 1271 to verify. + Prior to this change some extended length Windows error messages could + be truncated. - Reported-by: Brian Carpenter - Closes #4786 + Closes https://github.com/curl/curl/pull/4920 -- file: on Windows, refuse paths that start with \\ +- multi: fix outdated comment - ... as that might cause an unexpected SMB connection to a given host - name. + - Do not say that conn->data is "cleared" by multi_done(). - Reported-by: Fernando Muñoz - CVE-2019-15601 - Bug: https://curl.haxx.se/docs/CVE-2019-15601.html - -Jay Satiro (6 Jan 2020) -- CURLOPT_READFUNCTION.3: fix fopen params in example + If the connection is in use then multi_done assigns another easy handle + still using the connection to conn->data, therefore in that case it is + not cleared. + + Closes https://github.com/curl/curl/pull/4901 -- CURLOPT_READFUNCTION.3: fix variable name in example +- easy: remove dead code - Reported-by: Paul Joyce + multi is already assigned to data->multi by curl_multi_add_handle. - Fixes https://github.com/curl/curl/issues/4787 + Closes https://github.com/curl/curl/pull/4900 -Daniel Stenberg (5 Jan 2020) -- curl:getparameter return error for --http3 if libcurl doesn't support +Daniel Stenberg (12 Feb 2020) +- create-dirs.d: mention the mode - Closes #4785 + Reported-by: Dan Jacobson + Fixes #4766 + Closes #4916 -- docs: mention CURL_MAX_INPUT_LENGTH restrictions +- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording - ... for curl_easy_setopt() and curl_url_set(). + Assisted-by: Jay Satiro + Reported-by: Craig Andrews + Fixes #4909 + Closes #4910 + +- RELEASE-NOTES: synced + +Steve Holme (9 Feb 2020) +- smtp: Simplify the MAIL command and avoid a duplication of send strings - [skip ci] + This avoids the duplication of strings when the optional AUTH and SIZE + parameters are required. It also assists with the modifications that + are part of #4892. - Closes #4783 + Closes #4903 -- curl: properly free mimepost data +Daniel Stenberg (9 Feb 2020) +- altsvc: keep a copy of the file name to survive handle reset - ... as it could otherwise leak memory when a transfer failed. + The alt-svc cache survives a call to curl_easy_reset fine, but the file + name to use for saving the cache was cleared. Now the alt-svc cache has + a copy of the file name to survive handle resets. - Added test 1293 to verify. + Added test 1908 to verify. - Reported-by: Brian Carpenter - Fixes #4781 - Closes #4782 + Reported-by: Craig Andrews + Fixes #4898 + Closes #4902 -- curl: cleanup multi handle on failure +Steve Holme (9 Feb 2020) +- url: Include the failure reason when curl_win32_idn_to_ascii() fails - ... to fix memory leak in error path. + Provide the failure reason in the failf() info just as we do for the + libidn2 version of code. - Fixes #4772 - Closes #4780 - Reported-by: Brian Carpenter + Closes #4899 -Marcel Raad (3 Jan 2020) -- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` - - Closes https://github.com/curl/curl/pull/4775 +Jay Satiro (9 Feb 2020) +- asyn-thread: remove dead code -Daniel Stenberg (3 Jan 2020) -- COPYING: it's 2020! +Daniel Stenberg (8 Feb 2020) +- [Emil Engler brought this change] + + github: Instructions to post "uname -a" on Unix systems in issues - [skip ci] + Closes #4896 -Jay Satiro (3 Jan 2020) -- [Marc Aldorasi brought this change] +- [Cristian Greco brought this change] - tests: Fix bounce requests with truncated writes + configure.ac: fix comments about --with-quiche - Prior to this change the swsbounce check in service_connection could - fail because prevtestno and prevpartno were not set, which would cause - the wrong response data to be sent to some tests and cause them to fail. + A simple s/nghttp3/quiche in some comments of --with-quiche. + Looks like a copy-paste error from --with-nghttp3. - Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 + Closes #4897 -Marcel Raad (31 Dec 2019) -- tool: make a few char pointers point to const char instead - - These are read-only. +Steve Holme (7 Feb 2020) +- checksrc.bat: Fix not being able to run script from the main curl directory - Closes https://github.com/curl/curl/pull/4771 - -Jay Satiro (31 Dec 2019) -- tests: Change NTLM tests to require SSL + If the script was ran from the main curl directory rather then the + projects directory then the script would simply exit without error: - Prior to this change tests that required NTLM feature did not require - SSL feature. + C:\url> projects\checksrc.bat - There are pending changes to cmake builds that will allow enabling NTLM - in non-SSL builds in Windows. In that case the NTLM auth strings created - are different from what is expected by the NTLM tests and they fail: + The user would either need to change to the projects directory, + explicitly specify the current working directory, or perform a + oneline hacky workaround: - "The issue with NTLM is that previous non-SSL builds would not enable - NTLM and so the NTLM tests would be skipped." + C:\url> cd projects + C:\url\projects> checksrc.bat - Assisted-by: marc-groundctl@users.noreply.github.com + C:\url> checksrc.bat %cd% - Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 + C:\url> pushd projects & checksrc.bat & popd - Closes https://github.com/curl/curl/pull/4768 - -- [Michael Forney brought this change] + Closes #4894 - bearssl: Improve I/O handling - - Factor out common I/O loop as bearssl_run_until, which reads/writes TLS - records until the desired engine state is reached. This is now used for - the handshake, read, write, and close. - - Match OpenSSL SSL_write behavior, and don't return the number of bytes - written until the corresponding records have been completely flushed - across the socket. This involves keeping track of the length of data - buffered into the TLS engine, and assumes that when CURLE_AGAIN is - returned, the write function will be called again with the same data - and length arguments. This is the same requirement of SSL_write. - - Handle TLS close notify as EOF when reading by returning 0. - - Closes https://github.com/curl/curl/pull/4748 +Daniel Stenberg (7 Feb 2020) +- [Pierre-Yves Bigourdan brought this change] -- travis: Fix error detection - - - Stop using inline shell scripts for before_script and script sections. - - Prior to this change Travis could ignore errors from commands in inline - scripts. I don't understand how or why it happens. This is a workaround. + digest: Do not quote algorithm in HTTP authorisation - Assisted-by: Simon Warta + RFC 7616 section 3.4 (The Authorization Header Field) states that "For + historical reasons, a sender MUST NOT generate the quoted string syntax + for the following parameters: algorithm, qop, and nc". This removes the + quoting for the algorithm parameter. - Ref: https://github.com/travis-ci/travis-ci/issues/1066 + Reviewed-by: Steve Holme + Closes #4890 + +- ftp: remove the duplicated user/password struct fields - Fixes https://github.com/curl/curl/issues/3730 - Closes https://github.com/curl/curl/pull/3755 + Closes #4887 -- tool_operate: fix mem leak when failed config parse +- ftp: remove superfluous checking for crlf in user or pwd - Found by fuzzing the config file. + ... as this is already done much earlier in the URL parser. - Reported-by: Geeknik Labs + Also add test case 894 that verifies that pop3 with an encodedd CR in + the user name is rejected. - Fixes https://github.com/curl/curl/issues/4767 - -- [Xiang Xiao brought this change] + Closes #4887 - lib: remove erroneous +x file permission on some c files +Steve Holme (6 Feb 2020) +- ntlm_wb: Use Curl_socketpair() for greater portability - Modified by commit eb9a604 accidentally. + Reported-by: Daniel Stenberg + Closes #4886 + +Daniel Stenberg (5 Feb 2020) +- [Frank Gevaerts brought this change] + + contributors: Also include people who contributed to curl-www - Closes https://github.com/curl/curl/pull/4756 + Closes #4884 -- [Xiang Xiao brought this change] +- [Frank Gevaerts brought this change] - lib: fix warnings found when porting to NuttX + contrithanks: Use the most recent tag by default - - Undefine DEBUGASSERT in curl_setup_once.h in case it was already - defined as a system macro. + (similar to 5296abe) - - Don't compile write32_le in curl_endian unless - CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. + Closes #4883 + +- scripts: use last set tag if none given - - Include in socketpair.c. + Makes 'delta' and 'contributors.sh' easier to use. - Closes https://github.com/curl/curl/pull/4756 + Make the delta script invoke contrithanks to get current number of + contributors instead of counting THANKS, for accuracy. + + Closes #4881 -- os400: Add missing CURLE error constants +- ftp: shrink temp buffers used for PORT - Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 - Reported-by: Emil Engler + These two stack based buffers only need to be 46 + 66 bytes instead of + 256 + 1024. + + Closes #4880 -- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 +- curl: error on --alt-svc use w/o support - For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback - is passed two `size_t` parameters which, when multiplied, designate the - number of bytes of data passed in. In practice, CURL always sets the - first parameter (`size`) to 1. + Make the tool check for alt-svc support at run-time and return error + accordingly if not present when the option is used. - This practice is also enshrined in documentation and cannot be changed - in future. The documentation states that the default callback is - `fwrite`, which means `fwrite` must be a suitable function for this - purpose. However, the documentation also states that the callback must - return the number of *bytes* it successfully handled, whereas ISO C - `fwrite` returns the number of items (each of size `size`) which it - wrote. The only way these numbers can be equal is if `size` is 1. + Reported-by: Harry Sintonen + Closes #4878 + +- docs/HTTP3: add --enable-alt-svc to curl's configure + +- RELEASE-PROCEDURE: feature win is closed post-release a few days - Since `size` is 1 and can never be changed in future anyway, document - that fact explicitly and let users rely on it. + We've tried to uphold this already but let's make it official by + publicly stating this is the way we do it. - Reported-by: Frank Gevaerts - Commit-message-by: Christopher Head + Closes #4877 + +- altsvc: set h3 version at a common single spot - Ref: https://github.com/curl/curl/pull/2787 + ... and move the #ifdefs out of the functions. Addresses the fact they + were different before this change. - Fixes https://github.com/curl/curl/issues/4758 + Reported-by: Harry Sintonen + Closes #4876 -- examples/postinmemory.c: Call curl_global_cleanup always +- [Harry Sintonen brought this change] + + altsvc: improved header parser - Prior to this change curl_global_cleanup was not called if - curl_easy_init failed. + - Fixed the flag parsing to apply to specific alternative entry only, as + per RFC. The earlier code would also get totally confused by + multiprotocol header, parsing flags from the wrong part of the header. - Reported-by: kouzhudong@users.noreply.github.com + - Fixed the parser terminating on unknown protocols, instead of skipping + them. - Fixes https://github.com/curl/curl/issues/4751 + - Fixed a busyloop when protocol-id was present without an equal sign. + + Closes #4875 -Daniel Stenberg (21 Dec 2019) -- url2file.c: fix copyright year +- [Harry Sintonen brought this change] + + ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 + +- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 - Follow-up to 525787269599b5 + Reported-by: James Fuller -- [Rickard Hallerbäck brought this change] +Steve Holme (4 Feb 2020) +- ntlm: Pass the Curl_easy structure to the private winbind functions + + ...rather than the full conndata structure. - examples/url2file.c: corrected a comment +Daniel Stenberg (4 Feb 2020) +- RELEASE-NOTES: synced + +- tool_operhlp: Copyright year out of date, should be 2020 - The comment was confusing and suggested that setting CURLOPT_NOPROGRESS - to 0L would both enable and disable debug output at the same time, like - a Schrödinger's cat of CURLOPTs. + Follow-up from 2bc373740a3 + +- [Orgad Shaneh brought this change] + + curl: avoid using strlen for testing if a string is empty - Closes #4745 + Closes #4873 -- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 +Steve Holme (3 Feb 2020) +- ntlm: Ensure the HTTP header data is not stored in the challenge/response -- RELEASE-NOTES: synced +Marcel Raad (3 Feb 2020) +- openssl: remove redundant assignment + + Fixes a scan-build failure on Bionic. + + Closes https://github.com/curl/curl/pull/4872 -Jay Satiro (20 Dec 2019) -- ngtcp2: Support the latest update key callback type +- travis: update non-OpenSSL Linux jobs to Bionic - - Remove our cb_update_key in favor of ngtcp2's new - ngtcp2_crypto_update_key_cb which does the same thing. + For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is + failing with "curl returned 52, when expecting 35". - Several days ago the ngtcp2_update_key callback function prototype was - changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to - fix up our cb_update_key for that change they also added - ngtcp2_crypto_update_key_cb which does the same thing so we'll use that - instead. + Closes https://github.com/curl/curl/pull/4872 + +Dan Fandrich (3 Feb 2020) +- cirrus: Add some missing semicolons - Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c + Newlines aren't preserved in this section so they're needed to separate + commands. The exports luckily worked anyway as a single long line, but + erroneously exported a variable called "export" + [skip ci] + +Daniel Gustafsson (2 Feb 2020) +- [Pedro Monreal brought this change] + + cleanup: fix typos and wording in docs and comments - Closes https://github.com/curl/curl/pull/4735 + Closes #4869 + Reviewed-by: Emil Engler and Daniel Gustafsson -Daniel Stenberg (19 Dec 2019) -- sws: search for "Testno:" header uncondtionally if no testno +Steve Holme (2 Feb 2020) +- ntlm: Move the winbind data into the NTLM data structure - Even if the initial request line wasn't found. With the fix to 1455, the - test number is now detected correctly. + To assist with adding winbind support to the SASL NTLM authentication, + move the winbind specific data out of conndata into ntlmdata. + +Daniel Stenberg (30 Jan 2020) +- quiche: Copyright year out of date - (Problem found when running tests in random order.) + Follow-up to 7fc63d72333a + +- altsvc: use h3-25 - Closes #4744 + Closes #4868 -- tests: set LC_ALL in more tests +- [Alessandro Ghedini brought this change] + + quiche: update to draft-25 - Follow-up to 23208e330ac0c21 + Closes #4867 + +- ngtcp2: update to git master and its draft-25 support - Closes #4743 + Closes #4865 -- test165: set LC_ALL=en_US.UTF-8 too +- cookie: check __Secure- and __Host- case sensitively - On my current Debian Unstable with libidn2 2.2.0, I get an error if - LC_ALL is set to blank. Then curl errors out with: + While most keywords in cookies are case insensitive, these prefixes are + specified explicitly to get checked "with a case-sensitive match". - curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 + (From the 6265bis document in progress) - Closes #4738 + Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 + Closes #4864 -- curl.h: add two defines for the "pre ISO C" case +- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header + +- oauth2-bearer.d: works for HTTP too - Without this fix, this caused a compilation failure on AIX with IBM xlc - 13.1.3 compiler. + Reported-by: Mischa Salle + Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html + Closes #4862 + +- multi_done: if multiplexed, make conn->data point to another transfer - Reported-by: Ram Krushna Mishra - Fixes #4739 - Closes #4740 + ... since the current transfer is being killed. Setting to NULL is + wrong, leaving it pointing to 'data' is wrong since that handle might be + about to get freed. + + Fixes #4845 + Closes #4858 + Reported-by: dmitrmax on github -- create_conn: prefer multiplexing to using new connections +- location.d: the method change is from POST to GET only - ... as it would previously prefer new connections rather than - multiplexing in most conditions! The (now removed) code was a leftover - from the Pipelining code that was translated wrongly into a - multiplex-only world. + Not from generic non-GET to GET. - Reported-by: Kunal Ekawde - Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html - Closes #4732 + Reported-by: Andrius Merkys + Ref: #4859 + Closes #4861 -- test1456: remove the use of a fixed local port - - Fixup the test to instead not compare the port number. It sometimes - caused problems like this: +- urlapi: guess scheme correct even with credentials given - "curl: (45) bind failed with errno 98: Address already in use" + In the "scheme-less" parsing case, we need to strip off credentials + first before we guess scheme based on the host name! - Closes #4733 + Assisted-by: Jay Satiro + Fixes #4856 + Closes #4857 -Jay Satiro (18 Dec 2019) -- CURLOPT_QUOTE.3: fix typos - - Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man - pages would not compile because a variable name was incorrect. +- global_init: move the IPv6 works status bool to multi handle - Reported-by: Bylon2@users.noreply.github.com + Previously it was stored in a global state which contributed to + curl_global_init's thread unsafety. This boolean is now instead figured + out in curl_multi_init() and stored in the multi handle. Less effective, + but thread safe. - Fixes https://github.com/curl/curl/issues/4736 + Closes #4851 -- [Gisle Vanem brought this change] +- [Jay Satiro brought this change] - strerror: Fix compiler warning "empty expression" - - - Remove the final semi-colon in the SEC2TXT() macro definition. + README: mention that the docs is in docs/ - Before: #define SEC2TXT(sec) case sec: txt = #sec; break; + Reported-by: Austin Green + Fixes #4830 + Closes #4853 + +- curl.h: define CURL_WIN32 on windows - After: #define SEC2TXT(sec) case sec: txt = #sec; break + ... so that the subsequent logic below can use a single known define to know + when built on Windows (as we don't define WIN32 anymore). - Prior to this change SEC2TXT(foo); would generate break;; which caused - the empty expression warning. + Follow-up to 1adebe7886ddf20b - Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 + Reported-by: crazydef on github + Assisted-by: Marcel Raad + Fixes #4854 + Closes #4855 -Daniel Stenberg (18 Dec 2019) -- curl/parseconfig: use curl_free() to free memory allocated by libcurl - - Reported-by: bxac on github - Fixes #4730 - Closes #4731 +- RELEASE-NOTES: synced -- curl/parseconfig: fix mem-leak - - When looping, first trying '.curlrc' and then '_curlrc', the function - would not free the first string. - - Closes #4731 +- [Jon Rumsey brought this change] -- CURLOPT_URL.3: "curl supports SMB version 1 (only)" + urldata: do string enums without #ifdefs for build scripts - [skip ci] - -- test1270: a basic -w redirect_url test + ... and check for inconsistencies for OS400 at build time with the new + chkstrings tool. - Closes #4728 - -- HISTORY: the SMB(S) support landed in 2014 + Closes #4822 -- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore +- curl: make the -# spaceship bar not wrap the line - It is covered by USE_OPENSSL_ENGINE now. + The fixed-point math made us lose precision and thus a too high index + value could be used for outputting the hashtags which could overwrite + the newline. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 + The fix increases the precision in the sine table (*100) and the + associated position math. - Closes #4725 + Reported-by: Andrew Potter + Fixes #4849 + Closes #4850 -- lib: remove ASSIGNWITHINCONDITION exceptions, use our code style +- global_init: assume the EINTR bit by default - ... even for macros + - Removed from global_init since it isn't thread-safe. The symbol will + still remain to not break compiles, it just won't have any effect going + forward. - Reviewed-by: Daniel Gustafsson - Reviewed-by: Jay Satiro - Reported-by: Jay Satiro - Fixes #4683 - Closes #4722 + - make the internals NOT loop on EINTR (the opposite from previously). + It only risks returning from the select/poll/wait functions early, and that + should be risk-free. + + Closes #4840 -- tests: make sure checksrc runs on header files too +- [Peter Piekarski brought this change] -- Revert "checksrc: fix regexp for ASSIGNWITHINCONDITION" - - This reverts commit ba82673dac3e8d00a76aa5e3779a0cb80e7442af. + conn: do not reuse connection if SOCKS proxy credentials differ - Bug: #4683 + Closes #4835 -- KNOWN_BUGS: TLS session cache doesn't work with TFO +- llist: removed unused Curl_llist_move() - [skip ci] - Closes #4301 - -- KNOWN_BUGS: Connection information when using TCP Fast Open + (and the corresponding unit test) - Also point to #4296 for more details - Closes #4296 + Closes #4842 -- KNOWN_BUGS: LDAP on Windows doesn't work - - Closes #4261 +- conncache: removed unused Curl_conncache_bundle_size() -- docs: TLS SRP doesn't work with TLS 1.3 +- strcase: turn Curl_raw_tolower into static - Reported-by: sayrer on github - Closes #4262 - [skip ci] + Only ever used from within this file. -Dan Fandrich (16 Dec 2019) -- cirrus: Switch to the FreeBSD 12.1 point release & enable more tests. - - A few tests are now passing on FreeBSD, so no longer skip them. - [skip ci] +- singleuse.pl: support new API functions, fix curl_dbg_ handling -Daniel Stenberg (16 Dec 2019) -- azure: the macos cmake doesn't need to install cmake - - Error: cmake 3.15.5 is already installed - To upgrade to 3.16.1, run `brew upgrade cmake`. +- wolfssh: make it init properly via Curl_ssh_init() - Closes #4723 + Closes #4846 -Jay Satiro (15 Dec 2019) -- winbuild: Document CURL_STATICLIB requirement for static libcurl - - A static libcurl (ie winbuild mode=static) requires that the user define - CURL_STATICLIB when using it in their application. This is already - covered in the FAQ and INSTALL.md, but is a pretty important point so - now it's noted in the BUILD.WINDOWS.txt as well. - - Assisted-by: Michael Vittiglio - - Closes https://github.com/curl/curl/pull/4721 +- [Aron Rotteveel brought this change] -Daniel Stenberg (15 Dec 2019) -- [Santino Keupp brought this change] + form.d: fix two minor typos + + Closes #4843 - libssh2: add support for ECDSA and ed25519 knownhost keys +- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields - ... if a new enough libssh2 version is present. + Avoid "reparsing" the content and instead deliver more exactly what is + provided in the certificate and avoid truncating the data after 512 + bytes as done previously. This no longer removes embedded newlines. - Source: https://curl.haxx.se/mail/archive-2019-12/0023.html - Co-Authored-by: Daniel Stenberg - Closes #4714 + Fixes #4837 + Reported-by: bnfp on github + Closes #4841 -- lib1591: free memory properly on OOM, in the trailers callback +Jay Satiro (23 Jan 2020) +- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 - Detected by torture tests. + - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. - Closes #4720 + Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the + CURLOPT_PROXY_SSL_OPTIONS description. -- runtests: --repeat=[num] to repeat tests +Daniel Stenberg (22 Jan 2020) +- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER - Closes #4715 + For now, no cert in the bundle actually sets a date there... + + Co-Authored-by: Jay Satiro + Reported-by: Christian Heimes + Fixes #4834 + Closes #4836 - RELEASE-NOTES: synced -- azure: add a torture test on mac - - Uses --shallow=25 to keep it small enough to get through in time. - - Closes #4712 +- [Pavel Volgarev brought this change] -- multi: free sockhash on OOM + smtp: Allow RCPT TO command to fail for some recipients - This would otherwise leak memory in the error path. + Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. - Detected by torture test 1540. + Verified with the new tests 3002-3007 - Closes #4713 + Closes #4816 -Marcel Raad (13 Dec 2019) -- tests: use DoH feature for DoH tests - - Previously, http/2 was used instead. +- copyright: fix year ranges - Assisted-by: Jay Satiro - Closes https://github.com/curl/curl/pull/4692 + follow-up from dea17b519d (one of these days I'll learn to check before + I push) + +- [nao brought this change] -- hostip: suppress compiler warning + http: move "oauth_bearer" from connectdata to Curl_easy - With `--disable-doh --disable-threaded-resolver`, the `dns` parameter - is not used. + Fixes the bug where oauth_bearer gets deallocated when we re-use a + connection. - Closes https://github.com/curl/curl/pull/4692 + Closes #4824 -- tests: fix build with `CURL_DISABLE_DOH` +- [Emil Engler brought this change] + + curl: Let -D merge headers in one file again - Closes https://github.com/curl/curl/pull/4692 + Closes #4762 + Fixes #4753 -Daniel Stenberg (13 Dec 2019) -- azure: add a torture test +- data.d: remove "Multiple files can also be specified" - Skipping all FTP tests for speed reasons. + It is superfluous and could even be misleading. - Closes #4697 - -- azure: make the default build use --enable-debug --enable-werror + Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html + Reported-by: Mike Norton + Closes #4832 -- ntlm_wb: fix double-free in OOM +Marcel Raad (20 Jan 2020) +- CMake: support specifying the target Windows version - Detected by torture testing test 1310 + Previously, it was only possible to set it to Windows Vista or XP by + setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. + Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to + explicitly set the target Windows version. `ENABLE_INET_PTON` is + ignored in this case. - Closes #4710 + Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 + Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 + Closes https://github.com/curl/curl/pull/4815 -Dan Fandrich (13 Dec 2019) -- cirrus: Drop the FreeBSD 10.4 build +Daniel Stenberg (20 Jan 2020) +- http.h: Copyright year out of date, should be 2020 - Upstream support for 10.4 ended a year ago, and it looks like the image - is now gone, too. - [skip ci] + Follow-up to 7ff9222ced8c -Daniel Stenberg (13 Dec 2019) -- unit1620: fix bad free in OOM - - Closes #4709 +- [加藤郁之 brought this change] -- unit1609: fix mem-leak in OOM + HTTP: increase EXPECT_100_THRESHOLD to 1Mb - Closes #4709 - -- unit1607: fix mem-leak in OOM + Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html - Closes #4709 + Closes #4814 -- lib1559: fix mem-leak in OOM +- ROADMAP: thread-safe `curl_global_init()` - Closes #4709 + I'd like to see this happen. -- lib1557: fix mem-leak in OOM - - Closes #4709 +- RELEASE-NOTES: synced -- altsvc: make the save function ignore NULL filenames +- wolfssl: use the wc-prefixed symbol alternatives - It might happen in OOM situations. Detected bv torture tests. + The symbols without wc_ prefix are not always provided. + + Ref: https://github.com/wolfSSL/wolfssl/issues/2744 - Closes #4707 + Closes #4827 -- curl: fix memory leak in OOM in etags logic +- polarssl: removed + + As detailed in DEPRECATE.md, the polarssl support is now removed after + having been disabled for 6 months and nobody has missed it. - Detected by torture tests + The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix + instead of the former 'polarssl' and the common functions that + previously were shared between mbedtls and polarssl and contained the + name 'polarssl' have now all been renamed to instead say 'mbedtls'. - Closes #4706 + Closes #4825 -- doh: make it behave when built without proxy support +Marcel Raad (16 Jan 2020) +- libssh2: fix variable type - Reported-by: Marcel Raad - Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734 + This led to a conversion warning on 64-bit MinGW, which has 32-bit + `long` but 64-bit `size_t`. - Closes #4704 + Closes https://github.com/curl/curl/pull/4823 -- curl: improved cleanup in upload error path +Daniel Stenberg (16 Jan 2020) +- curl:progressbarinit: ignore column width from terminals < 20 - Memory leak found by torture test 58 + To avoid division by zero - or other issues. - Closes #4705 + Reported-by: Daniel Marjamäki + Closes #4818 -- mailmap: fix Andrew Ishchuk +- wolfssh: set the password correctly for PASSWORD auth -- travis: make torture use --shallow=40 - - As a first step to enable it to run over a more diverse set of tests in - a reasonable time. +- wolfssh: remove fprintf() calls (and uses of __func__) -- runtests: introduce --shallow to reduce huge torture tests +Marcel Raad (14 Jan 2020) +- CMake: use check_symbol_exists also for inet_pton + + It doesn't make much sense to only check if the function can be linked + when it's not declared in any header and that is treated as an error. + With the correct target Windows version set, the function is declared + in ws2tcpip.h and the comment above the modified block is invalid. + + Also, move the definition of `_WIN32_WINNT` up to before all symbol + availability checks so that we don't have to care which ones must be + done after it. + + Tested with Visual Studio 2019 and current MinGW-w64. - When set, shallow mode limits runtests -t to make no more than NUM fails - per test case. If more are found, it will randomly discard entries until - the number is right. The random seed can also be set. + Closes https://github.com/curl/curl/pull/4808 + +Jay Satiro (13 Jan 2020) +- schannel_verify: Fix alt names manual verify for UNICODE builds - This is particularly useful when running MANY tests as then most torture - failures will already fail the same functions over and over and make the - total operation painfully tedious. + Follow-up to 29e40a6 from two days ago, which added that feature for + Windows 7 and earlier. The bug only occurred in same. - Closes #4699 + Ref: https://github.com/curl/curl/pull/4761 -- conncache: CONNECT_ONLY connections assumed always in-use +Daniel Stenberg (13 Jan 2020) +- HTTP-COOKIES.md: describe the cookie file format - This makes them never to be considered "the oldest" to be discarded when - reaching the connection cache limit. The reasoning here is that - CONNECT_ONLY is primarily used in combination with using the - connection's socket post connect and since that is used outside of - curl's knowledge we must assume that it is in use until explicitly - closed. + ... and refer to that file from from CURLOPT_COOKIEFILE.3 and + CURLOPT_COOKIELIST.3 - Reported-by: Pavel Pavlov - Reported-by: Pavel Löbl - Fixes #4426 - Fixes #4369 - Closes #4696 + Assisted-by: Jay Satiro + Reported-by: bsammon on github + Fixes #4805 + Closes #4806 -- [Gisle Vanem brought this change] +- [Tobias Hieta brought this change] - vtls: make BearSSL possible to set with CURL_SSL_BACKEND + CMake: Add support for CMAKE_LTO option. - Ref: https://github.com/curl/curl/commit/9b879160df01e7ddbb4770904391d3b74114302b#commitcomment-36355622 + This enables Link Time Optimization. LTO is a proven technique for + optimizing across compilation units. - Closes #4698 + Closes #4799 - RELEASE-NOTES: synced -- travis: remove "coverage", make it "torture" +- ConnectionExists: respect the max_concurrent_streams limits - The coveralls service and test coverage numbers are just too unreliable. - Removed badge from README.md as well. + A regression made the code use 'multiplexed' as a boolean instead of the + counter it is intended to be. This made curl try to "over-populate" + connections with new streams. - Fixes #4694 - Closes #4695 - -- azure: add libssh2 and cmake macos builds + This regression came with 41fcdf71a1, shipped in curl 7.65.0. - Removed the macos libssh2 build from travis + Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same + check. - Closes #4686 + Reported-by: Kunal Ekawde + Fixes #4779 + Closes #4784 -- curl: use errorf() better +- curl: make #0 not output the full URL - Change series of error outputs to use errorf(). + It was not intended nor documented! - Only errors that are due to mistakes in command line option usage should - use helpf(), other types of errors in the tool should rather use - errorf(). + Added test 1176 to verify. - Closes #4691 - -Jay Satiro (9 Dec 2019) -- [Marc Hoersken brought this change] + Reported-by: vshmuk on hackerone + + Closes #4812 - tests: make it possible to set executable extensions +- wolfSSH: new SSH backend - This enables the use of Windows Subsystem for Linux (WSL) to run the - testsuite against Windows binaries while using Linux servers. + Adds support for SFTP (not SCP) using WolfSSH. - This commit introduces the following environment variables: - - CURL_TEST_EXE_EXT: set the executable extension for all components - - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only - - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only + Closes #4231 + +- curl: remove 'config' field from OutStruct - Later testcurl.pl could be adjusted to make use of those variables. - - CURL_TEST_EXE_EXT_SRV: set it for the test servers only + As it was just unnecessary duplicated information already stored in the + 'per_transfer' struct and that's around mostly anyway. - (This is one of several commits to support use of WSL for the tests.) + The duplicated pointer caused problems when the code flow was aborted + before the dupe was filled in and could cause a NULL pointer access. - Closes https://github.com/curl/curl/pull/3899 - -- [Marc Hoersken brought this change] + Reported-by: Brian Carpenter + Fixes #4807 + Closes #4810 - tests: fix permissions of ssh keys in WSL - - Keys created on Windows Subsystem for Linux (WSL) require it for some - reason. +- misc: Copyright year out of date, should be 2020 - (This is one of several commits to support use of WSL for the tests.) + Follow-up to recent commits - Ref: https://github.com/curl/curl/pull/3899 + [skip ci] -- [Marc Hoersken brought this change] +Jay Satiro (11 Jan 2020) +- [Santino Keupp brought this change] - tests: use \r\n for log messages in WSL + libssh2: add support for forcing a hostkey type - Bash in Windows Subsystem for Linux (WSL) requires it for some reason. + - Allow forcing the host's key type found in the known_hosts file. - (This is one of several commits to support use of WSL for the tests.) + Currently, curl (with libssh2) does not take keys from your known_hosts + file into account when talking to a server. With this patch the + known_hosts file will be searched for an entry matching the hostname + and, if found, libssh2 will be told to claim this key type from the + server. - Ref: https://github.com/curl/curl/pull/3899 + Closes https://github.com/curl/curl/pull/4747 -- [Andrew Ishchuk brought this change] +- [Nicolas Guillier brought this change] - winbuild: Define CARES_STATICLIB when WITH_CARES=static + cmake: Improve libssh2 check on Windows + + - Add "libssh2" name to FindLibSSH2 library search. - When libcurl is built with MODE=static, c-ares is forced into static - linkage too. That doesn't happen when MODE=dll so linker would break - over undefined symbols. + On Windows systems, libSSH2 CMake installation may name the library + "LibSSH2". - closes https://github.com/curl/curl/pull/4688 - -Daniel Stenberg (9 Dec 2019) -- conn: always set bits.close with connclose() + Prior to this change cmake only checked for name "ssh2". On Linux that + works fine because it will prepend the "lib", but it doesn't do that on + Windows. - Closes #4690 + Closes https://github.com/curl/curl/pull/4804 -- cirrus: enable clang sanitizers on freebsd 13 +- [Faizur Rahman brought this change] -- conncache: fix multi-thread use of shared connection cache - - It could accidentally let the connection get used by more than one - thread, leading to double-free and more. + schannel: Make CURLOPT_CAINFO work better on Windows 7 - Reported-by: Christopher Reid - Fixes #4544 - Closes #4557 - -- azure: add a vanilla macos build + - Support hostname verification via alternative names (SAN) in the + peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. - Closes #4685 - -- curl: make the etag load logic work without fseek + CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a + result CertGetNameString doesn't quite work on those versions of + Windows. This change provides an alternative solution for + CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier + versions of Windows. - The fseek()s were unnecessary and caused Coverity warning CID 1456554 + Prior to this change many certificates failed the hostname validation + when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates + now represent multiple hostnames and rely on the alternative names field + exclusively to represent their hostnames. - Closes #4681 - -- mailmap: Mohammad Hasbini - -- [Mohammad Hasbini brought this change] - - docs: fix some typos + Reported-by: Jeroen Ooms - Closes #4680 + Fixes https://github.com/curl/curl/issues/3711 + Closes https://github.com/curl/curl/pull/4761 -- RELEASE-NOTES: synced +- [Emil Engler brought this change] -Jay Satiro (5 Dec 2019) -- lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN - - Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS - and OS400 package spec. + ngtcp2: Add an error code for QUIC connection errors - Also I added the option to the NameValue list in the tool even though it - isn't exposed as a command-line option (...yet?). (NameValue stringizes - the option name for the curl cmd -> libcurl source generator) + - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection + errors. - Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN. + Prior to this change CURLE_FAILED_INIT was used, but that was not + correct. - Ref: https://github.com/curl/curl/pull/4655 + Closes https://github.com/curl/curl/pull/4754 -- setopt: Fix ALPN / NPN user option when built without HTTP2 - - - Stop treating lack of HTTP2 as an unknown option error result for - CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN. - - Prior to this change it was impossible to disable ALPN / NPN if libcurl - was built without HTTP2. Setting either option would result in - CURLE_UNKNOWN_OPTION and the respective internal option would not be - set. That was incorrect since ALPN and NPN are used independent of - HTTP2. +- multi: Change curl_multi_wait/poll to error on negative timeout - Reported-by: Shailesh Kapse + - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when + curl_multi_wait/poll is passed timeout param < 0. - Fixes https://github.com/curl/curl/issues/4668 - Closes https://github.com/curl/curl/pull/4672 - -Daniel Stenberg (5 Dec 2019) -- etag: allow both --etag-compare and --etag-save in same cmdline + Prior to this change passing a negative value to curl_multi_wait/poll + such as -1 could cause the function to wait forever. - Fixes #4669 - Closes #4678 - -Marcel Raad (5 Dec 2019) -- curl_setup: fix `CURLRES_IPV6` condition + Reported-by: hamstergene@users.noreply.github.com - Move the definition of `CURLRES_IPV6` to before undefining - `HAVE_GETADDRINFO`. Regression from commit 67a08dca27a which caused - some tests to fail and others to be skipped with c-ares. + Fixes https://github.com/curl/curl/issues/4763 - Fixes https://github.com/curl/curl/issues/4673 - Closes https://github.com/curl/curl/pull/4677 + Closes https://github.com/curl/curl/pull/4765 -Daniel Stenberg (5 Dec 2019) -- test342: make it return a 304 as the tag matches +- [Marc Aldorasi brought this change] -Peter Wu (4 Dec 2019) -- CMake: add support for building with the NSS vtls backend + cmake: Enable SMB for Windows builds - Options are cross-checked with configure.ac and acinclude.m4. - Tested on Arch Linux, untested on other platforms like Windows or macOS. + - Define USE_WIN32_CRYPTO by default. This enables SMB. - Closes #4663 - Reviewed-by: Kamil Dudka - -Daniel Stenberg (4 Dec 2019) -- azure: add more builds + - Show whether SMB is enabled in the "Enabled features" output. - ... removed two from travis (that now runs on azure instead) + - Fix mingw compiler warning for call to CryptHashData by casting away + const param. mingw CryptHashData prototype is wrong. - Closes #4671 - -- CURLOPT_VERBOSE.3: see also ERRORBUFFER - -- hostip4.c: bump copyright year range + Closes https://github.com/curl/curl/pull/4717 -Marcel Raad (3 Dec 2019) -- configure: enable IPv6 support without `getaddrinfo` +- vtls: Refactor Curl_multissl_version to make the code clearer - This makes it possible to recognize and connect to literal IPv6 - addresses when `getaddrinfo` is not available, which is already the - case for the CMake build. This affects e.g. classic MinGW because it - still targets Windows 2000 by default, where `getaddrinfo` is not - available, but general IPv6 support is. + Reported-by: Johannes Schindelin - Instead of checking for `getaddrinfo`, check for `sockaddr_in6` as the - CMake build does. + Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 - Closes https://github.com/curl/curl/pull/4662 + Closes https://github.com/curl/curl/pull/4803 -- curl_setup: disable IPv6 resolver without `getaddrinfo` - - Also, use `CURLRES_IPV6` only for actual DNS resolution, not for IPv6 - address support. This makes it possible to connect to IPv6 literals by - setting `ENABLE_IPV6` even without `getaddrinfo` support. It also fixes - the CMake build when using the synchronous resolver without - `getaddrinfo` support. +Daniel Stenberg (10 Jan 2020) +- fix: Copyright year out of date, should be 2020 - Closes https://github.com/curl/curl/pull/4662 + Follow-up to 875314ed0bf3b -Daniel Stenberg (3 Dec 2019) -- github action/azure pipeline: run 'make test-nonflaky' for tests +Marcel Raad (10 Jan 2020) +- hostip: move code to resolve IP address literals to `Curl_resolv` - To match travis and give more info on failures. - -- openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains + The code was duplicated in the various resolver backends. - Closes #4655 - -- openssl: set X509_V_FLAG_PARTIAL_CHAIN + Also, it was called after the call to `Curl_ipvalid`, which matters in + case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This + caused test 1048 to fail on classic MinGW. - Have intermediate certificates in the trust store be treated as - trust-anchors, in the same way as self-signed root CA certificates - are. This allows users to verify servers using the intermediate cert - only, instead of needing the whole chain. + The code ignores `conn->ip_version` as done previously in the + individual resolver backends. - Other TLS backends already accept partial chains. + Move the call to the `resolver_start` callback up to appease test 655, + which wants it to be called also for literal addresses. - Reported-by: Jeffrey Walton - Bug: https://curl.haxx.se/mail/lib-2019-11/0094.html + Closes https://github.com/curl/curl/pull/4798 -- curl: show better error message when no homedir is found - - Reported-by: Vlastimil Ovčáčík - Fixes #4644 - Closes #4665 +Daniel Stenberg (9 Jan 2020) +- scripts/delta: adapt to new public header layout -- OPENSOCKETFUNCTION.3: correct the purpose description +- test1167: verify global symbols in public headers are curl prefixed - Reported-by: Jeff Mears - Bug: https://curl.haxx.se/mail/lib-2019-12/0007.html + ... using the new badsymbols.pl perl script - Closes #4667 + Fixes #4793 + Closes #4794 -- [Peter Wu brought this change] +- libtest/mk-lib1521: adapt to new public header layout - travis: do not use OVERRIDE_CC or OVERRIDE_CXX if empty - - Fixes the macOS builds where OVERRIDE_CC and OVERRIDE_CXX are not set. +- include: remove non-curl prefixed defines - Reported-by: Jay Satiro - Fixes #4659 - Closes #4661 - Closes #4664 - -- azure-pipelines: fix the test script + ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ + enums. -- Azure Pipelines: initial CI setup +- curl.h: remove WIN32 define - [skip ci] - -- docs: add "added: 7.68.0" to the --etag-* docs + It isn't our job to define this in a public header - and it defines a + name outside of our naming scope. -- copyright: fix the year ranges for two files +- tool_dirhie.c: fix the copyright year range - Follow-up to 9c1806ae + Follow-up to: 4027bd72d9 -Jay Satiro (1 Dec 2019) -- build: Disable Visual Studio warning "conditional expression is constant" - - - Disable warning C4127 "conditional expression is constant" globally - in curl_setup.h for when building with Microsoft's compiler. - - This mainly affects building with the Visual Studio project files found - in the projects dir. - - Prior to this change the cmake and winbuild build systems already - disabled 4127 globally for when building with Microsoft's compiler. - Also, 4127 was already disabled for all build systems in the limited - circumstance of the WHILE_FALSE macro which disabled the warning - specifically for while(0). This commit removes the WHILE_FALSE macro and - all other cruft in favor of disabling globally in curl_setup. - - Background: +- bump: work towards 7.69.0 is started + +Jay Satiro (9 Jan 2020) +- tool_dirhie: Allow directory traversal during creation - We have various macros that cause 0 or 1 to be evaluated, which would - cause warning C4127 in Visual Studio. For example this causes it: + - When creating a directory hierarchy do not error when mkdir fails due + to error EACCESS (13) "access denied". - #define Curl_resolver_asynch() 1 + Some file systems allow for directory traversal; in this case that it + should be possible to create child directories when permission to the + parent directory is restricted. - Full behavior is not clearly defined and inconsistent across versions. - However it is documented that since VS 2015 Update 3 Microsoft has - addressed this somewhat but not entirely, not warning on while(true) for - example. + This is a regression caused by me in f16bed0 (precedes curl-7_61_1). + Basically I had assumed that if a directory already existed it would + fail only with error EEXIST, and not error EACCES. The latter may + happen if the directory exists but has certain restricted permissions. - Prior to this change some C4127 warnings occurred when I built with - Visual Studio using the generated projects in the projects dir. + Reported-by: mbeifuss@users.noreply.github.com - Closes https://github.com/curl/curl/pull/4658 + Fixes https://github.com/curl/curl/issues/4796 + Closes https://github.com/curl/curl/pull/4797 -- openssl: retrieve reported LibreSSL version at runtime - - - Retrieve LibreSSL runtime version when supported (>= 2.7.1). - - For earlier versions we continue to use the compile-time version. - - Ref: https://man.openbsd.org/OPENSSL_VERSION_NUMBER.3 +Daniel Stenberg (9 Jan 2020) +- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers - Closes https://github.com/curl/curl/pull/2425 + Closes #4080 -- strerror: Add Curl_winapi_strerror for Win API specific errors - - - In all code call Curl_winapi_strerror instead of Curl_strerror when - the error code is known to be from Windows GetLastError. - - Curl_strerror prefers CRT error codes (errno) over Windows API error - codes (GetLastError) when the two overlap. When we know the error code - is from GetLastError it is more accurate to prefer the Windows API error - messages. - - Reported-by: Richard Alcock +- docs/RELEASE-PROCEDURE.md: pushed some release dates - Fixes https://github.com/curl/curl/issues/4550 - Closes https://github.com/curl/curl/pull/4581 + Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html -Daniel Stenberg (2 Dec 2019) -- global_init: undo the "intialized" bump in case of failure +- runtests: make random seed fixed for a month - ... so that failures in the global init function don't count as a - working init and it can then be called again. + When using randomized features of runtests (-R and --shallow) it is + useful to have a fixed random seed to make sure for example extra + commits in a branch or a rebase won't change the seed that would make + repeated runs work differently. - Reported-by: Paul Groke - Fixes #4636 - Closes #4653 - -- parsedate: offer a getdate_capped() alternative + As it is also useful to change seed sometimes, the default seed is now + determined based on the current month (and first line curl -V + output). When the month changes, so will the random seed. - ... and use internally. This function will return TIME_T_MAX instead of - failure if the parsed data is found to be larger than what can be - represented. TIME_T_MAX being the largest value curl can represent. + The specific seed is also shown in the standard test suite top header + and it can be set explictly with the new --seed=[num] option so that the + exact order of a previous run can be achieved. - Reviewed-by: Daniel Gustafsson - Reported-by: JanB on github - Fixes #4152 - Closes #4651 + Closes #4734 -- docs: add more references to curl_multi_poll +- RELEASE-PROCEDURE.md: fix next release date (Feb 26) - Fixes #4643 - Closes #4652 + [skip ci] -- sha256: bump the copyright year range - - Follow-up from 66e21520f +Version 7.68.0 (8 Jan 2020) -Daniel Gustafsson (28 Nov 2019) -- curl_setup_once: consistently use WHILE_FALSE in macros - - The WHILE_FALSE construction is used to avoid compiler warnings in - macro constructions. This fixes a few instances where it was not - used in order to keep the code consistent. - - Closes #4649 - Reviewed-by: Daniel Stenberg +Daniel Stenberg (8 Jan 2020) +- RELEASE-NOTES: 7.68.0 -Daniel Stenberg (28 Nov 2019) -- [Steve Holme brought this change] +- THANKS: updated with names from the 7.68.0 release - http_ntlm: Remove duplicate NSS initialisation +- RELEASE-PROCEDURE: add four future release dates - Given that this is performed by the NTLM code there is no need to - perform the initialisation in the HTTP layer. This also keeps the - initialisation the same as the SASL based protocols and also fixes a - possible compilation issue if both NSS and SSPI were to be used as - multiple SSL backends. + and remove four past release dates - Reviewed-by: Kamil Dudka - Closes #3935 + [skip ci] -Daniel Gustafsson (28 Nov 2019) -- checksrc: fix regexp for ASSIGNWITHINCONDITION - - The regexp looking for assignments within conditions was too greedy - and matched a too long string in the case of multiple conditionals - on the same line. This is basically only a problem in single line - macros, and the code which exemplified this was essentially: - - do { if((x) != NULL) { x = NULL; } } while(0) - - ..where the final parenthesis of while(0) matched the regexp, and - the legal assignment in the block triggered the warning. Fix by - making the regexp less greedy by matching for the tell-tale signs - of the if statement ending. +Marcel Raad (6 Jan 2020) +- TrackMemory tests: always remove CR before LF - Also remove the one occurrence where the warning was disabled due - to a construction like the above, where the warning didn't apply - when fixed. + It was removed for output containing ' =' via `s/ =.*//`. With classic + MinGW, this made lines with `free()` end with CRLF, but lines with e.g. + `malloc()` end with only LF. The tests expect LF only. - Closes #4647 - Reviewed-by: Daniel Stenberg - -Daniel Stenberg (28 Nov 2019) -- RELEASE-NOTES: synced - -- [Maros Priputen brought this change] + Closes https://github.com/curl/curl/pull/4788 - curl: two new command line options for etags +Daniel Stenberg (6 Jan 2020) +- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header - --etag-compare and --etag-save + ... to the private multihhandle.h. It is not for public use and it + wasn't prefixed correctly anyway! - Suggested-by: Paul Hoffman - Fixes #4277 - Closes #4543 - -Daniel Gustafsson (28 Nov 2019) -- docs: fix typos - -Daniel Stenberg (28 Nov 2019) -- mailmap: Niall O'Reilly's name + Closes #4790 -- [Niall O'Reilly brought this change] +- file: fix copyright year range + + Follow-up to 1b71bc532bd - doh: use dedicated probe slots +- curl -w: handle a blank input file correctly - ... to easier allow additional DNS transactions. + Previously it would end up with an uninitialized memory buffer that + would lead to a crash or junk getting output. + + Added test 1271 to verify. - Closes #4629 + Reported-by: Brian Carpenter + Closes #4786 -- travis: build ngtcp2 with --enable-lib-only +- file: on Windows, refuse paths that start with \\ - ... makes it skip the examples and other stuff we don't neeed. + ... as that might cause an unexpected SMB connection to a given host + name. - Closes #4646 + Reported-by: Fernando Muñoz + CVE-2019-15601 + Bug: https://curl.haxx.se/docs/CVE-2019-15601.html -- [David Benjamin brought this change] +Jay Satiro (6 Jan 2020) +- CURLOPT_READFUNCTION.3: fix fopen params in example - ngtcp2: fix thread-safety bug in error-handling +- CURLOPT_READFUNCTION.3: fix variable name in example - ERR_error_string(NULL) should never be called. It places the error in a - global buffer, which is not thread-safe. Use ERR_error_string_n with a - local buffer instead. + Reported-by: Paul Joyce - Closes #4645 + Fixes https://github.com/curl/curl/issues/4787 -- travis: export the CC/CXX variables when set +Daniel Stenberg (5 Jan 2020) +- curl:getparameter return error for --http3 if libcurl doesn't support - Suggested-by: Peter Wu - Fixes #4637 - Closes #4640 + Closes #4785 -Marcel Raad (26 Nov 2019) -- dist: add error-codes.pl +- docs: mention CURL_MAX_INPUT_LENGTH restrictions - Follow-up to commit 74f441c6d31. - This should fix test 1175 when run via the daily source tarballs. + ... for curl_easy_setopt() and curl_url_set(). - Closes https://github.com/curl/curl/pull/4638 - -Daniel Stenberg (26 Nov 2019) -- [John Schroeder brought this change] + [skip ci] + + Closes #4783 - curl: fix --upload-file . hangs if delay in STDIN +- curl: properly free mimepost data - Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION. + ... as it could otherwise leak memory when a transfer failed. - When uploading from stdin in non-blocking mode, a delay in reading - the stream (EAGAIN) causes curl to pause sending data - (CURL_READFUNC_PAUSE). Prior to this change, a busy read was - detected and unpaused only in the CURLOPT_WRITEFUNCTION handler. - This change performs the same busy read handling in a - CURLOPT_XFERINFOFUNCTION handler. + Added test 1293 to verify. - Fixes #2051 - Closes #4599 - Reported-by: bdry on github - -- [John Schroeder brought this change] + Reported-by: Brian Carpenter + Fixes #4781 + Closes #4782 - XFERINFOFUNCTION: support CURL_PROGRESSFUNC_CONTINUE - - (also for PROGRESSFUNCTION) +- curl: cleanup multi handle on failure - By returning this value from the callback, the internal progress - function call is still called afterward. + ... to fix memory leak in error path. - Closes #4599 - -- [Michael Forney brought this change] + Fixes #4772 + Closes #4780 + Reported-by: Brian Carpenter - TLS: add BearSSL vtls implementation +Marcel Raad (3 Jan 2020) +- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` - Closes #4597 + Closes https://github.com/curl/curl/pull/4775 -- curl_multi_wakeup.3: add example and AVAILABILITY +Daniel Stenberg (3 Jan 2020) +- COPYING: it's 2020! - Reviewed-by: Gergely Nagy - Closes #4635 + [skip ci] -- [Gergely Nagy brought this change] +Jay Satiro (3 Jan 2020) +- [Marc Aldorasi brought this change] - multi: add curl_multi_wakeup() - - This commit adds curl_multi_wakeup() which was previously in the TODO - list under the curl_multi_unblock name. + tests: Fix bounce requests with truncated writes - On some platforms and with some configurations this feature might not be - available or can fail, in these cases a new error code - (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). + Prior to this change the swsbounce check in service_connection could + fail because prevtestno and prevpartno were not set, which would cause + the wrong response data to be sent to some tests and cause them to fail. - Fixes #4418 - Closes #4608 - -Jay Satiro (24 Nov 2019) -- [Xiaoyin Liu brought this change] + Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 - schannel: fix --tls-max for when min is --tlsv1 or default +Marcel Raad (31 Dec 2019) +- tool: make a few char pointers point to const char instead - Prior to this change schannel ignored --tls-max (CURL_SSLVERSION_MAX_ - macros) when --tlsv1 (CURL_SSLVERSION_TLSv1) or default TLS - (CURL_SSLVERSION_DEFAULT), using a max of TLS 1.2 always. + These are read-only. - Closes https://github.com/curl/curl/pull/4633 + Closes https://github.com/curl/curl/pull/4771 -- checksrc.bat: Add a check for vquic and vssh directories +Jay Satiro (31 Dec 2019) +- tests: Change NTLM tests to require SSL - Ref: https://github.com/curl/curl/pull/4607 - -- projects: Fix Visual Studio projects SSH builds + Prior to this change tests that required NTLM feature did not require + SSL feature. + + There are pending changes to cmake builds that will allow enabling NTLM + in non-SSL builds in Windows. In that case the NTLM auth strings created + are different from what is expected by the NTLM tests and they fail: + + "The issue with NTLM is that previous non-SSL builds would not enable + NTLM and so the NTLM tests would be skipped." - - Generate VQUIC and VSSH filenames in Visual Studio project files. + Assisted-by: marc-groundctl@users.noreply.github.com - Prior to this change generated Visual Studio project configurations that - enabled SSH did not build properly. Broken since SSH files were moved to - lib/vssh 3 months ago in 5b2d703. + Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 - Fixes https://github.com/curl/curl/issues/4492 - Fixes https://github.com/curl/curl/issues/4630 - Closes https://github.com/curl/curl/pull/4607 + Closes https://github.com/curl/curl/pull/4768 -Daniel Stenberg (23 Nov 2019) -- RELEASE-NOTES: synced +- [Michael Forney brought this change] -Jay Satiro (22 Nov 2019) -- openssl: Revert to less sensitivity for SYSCALL errors + bearssl: Improve I/O handling + + Factor out common I/O loop as bearssl_run_until, which reads/writes TLS + records until the desired engine state is reached. This is now used for + the handshake, read, write, and close. - - Disable the extra sensitivity except in debug builds (--enable-debug). + Match OpenSSL SSL_write behavior, and don't return the number of bytes + written until the corresponding records have been completely flushed + across the socket. This involves keeping track of the length of data + buffered into the TLS engine, and assumes that when CURLE_AGAIN is + returned, the write function will be called again with the same data + and length arguments. This is the same requirement of SSL_write. - - Improve SYSCALL error message logic in ossl_send and ossl_recv so that - "No error" / "Success" socket error text isn't shown on SYSCALL error. + Handle TLS close notify as EOF when reading by returning 0. - Prior to this change 0ab38f5 (precedes 7.67.0) increased the sensitivity - of OpenSSL's SSL_ERROR_SYSCALL error so that abrupt server closures were - also considered errors. For example, a server that does not send a known - protocol termination point (eg HTTP content length or chunked encoding) - _and_ does not send a TLS termination point (close_notify alert) would - cause an error if it closed the connection. + Closes https://github.com/curl/curl/pull/4748 + +- travis: Fix error detection - To be clear that behavior made it into release build 7.67.0 - unintentionally. Several users have reported it as an issue. + - Stop using inline shell scripts for before_script and script sections. - Ultimately the idea is a good one, since it can help prevent against a - truncation attack. Other SSL backends may already behave similarly (such - as Windows native OS SSL Schannel). However much more of our user base - is using OpenSSL and there is a mass of legacy users in that space, so I - think that behavior should be partially reverted and then rolled out - slowly. + Prior to this change Travis could ignore errors from commands in inline + scripts. I don't understand how or why it happens. This is a workaround. - This commit changes the behavior so that the increased sensitivity is - disabled in all curl builds except curl debug builds (DEBUGBUILD). If - after a period of time there are no major issues then it can be enabled - in dev and release builds with the newest OpenSSL (1.1.1+), since users - using the newest OpenSSL are the least likely to have legacy problems. + Assisted-by: Simon Warta - Bug: https://github.com/curl/curl/issues/4409#issuecomment-555955794 - Reported-by: Bjoern Franke + Ref: https://github.com/travis-ci/travis-ci/issues/1066 - Fixes https://github.com/curl/curl/issues/4624 - Closes https://github.com/curl/curl/pull/4623 - -- [Daniel Stenberg brought this change] + Fixes https://github.com/curl/curl/issues/3730 + Closes https://github.com/curl/curl/pull/3755 - openssl: improve error message for SYSCALL during connect +- tool_operate: fix mem leak when failed config parse + + Found by fuzzing the config file. - Reported-by: Paulo Roberto Tomasi - Bug: https://curl.haxx.se/mail/archive-2019-11/0005.html + Reported-by: Geeknik Labs - Closes https://github.com/curl/curl/pull/4593 + Fixes https://github.com/curl/curl/issues/4767 -Daniel Stenberg (22 Nov 2019) -- test1175: verify symbols-in-versions and libcurl-errors.3 in sync - - Closes #4628 +- [Xiang Xiao brought this change] -- include: make CURLE_HTTP3 use a new error code + lib: remove erroneous +x file permission on some c files - To avoid potential issues with error code reuse. + Modified by commit eb9a604 accidentally. - Reported-by: Christoph M. Becker - Assisted-by: Dan Fandrich - Fixes #4601 - Closes #4627 + Closes https://github.com/curl/curl/pull/4756 -- bump: next release will be 7.68.0 +- [Xiang Xiao brought this change] -- curl: add --parallel-immediate + lib: fix warnings found when porting to NuttX - Starting with this change when doing parallel transfers, without this - option set, curl will prefer to create new transfers multiplexed on an - existing connection rather than creating a brand new one. + - Undefine DEBUGASSERT in curl_setup_once.h in case it was already + defined as a system macro. - --parallel-immediate can be set to tell curl to prefer to use new - connections rather than to wait and try to multiplex. + - Don't compile write32_le in curl_endian unless + CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. - libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default - on parallel transfers. + - Include in socketpair.c. - Suggested-by: Tom van der Woerdt - Closes #4500 + Closes https://github.com/curl/curl/pull/4756 -Daniel Gustafsson (20 Nov 2019) -- [Victor Magierski brought this change] +- os400: Add missing CURLE error constants + + Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 + Reported-by: Emil Engler - docs: fix typos +- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 - Change 'experiemental' to 'experimental'. + For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback + is passed two `size_t` parameters which, when multiplied, designate the + number of bytes of data passed in. In practice, CURL always sets the + first parameter (`size`) to 1. - Closes #4618 - Reviewed-by: Daniel Gustafsson - -Jay Satiro (18 Nov 2019) -- projects: Fix Visual Studio wolfSSL configurations + This practice is also enshrined in documentation and cannot be changed + in future. The documentation states that the default callback is + `fwrite`, which means `fwrite` must be a suitable function for this + purpose. However, the documentation also states that the callback must + return the number of *bytes* it successfully handled, whereas ISO C + `fwrite` returns the number of items (each of size `size`) which it + wrote. The only way these numbers can be equal is if `size` is 1. - - s/USE_CYASSL/USE_WOLFSSL/ + Since `size` is 1 and can never be changed in future anyway, document + that fact explicitly and let users rely on it. - - Remove old compatibility macros. + Reported-by: Frank Gevaerts + Commit-message-by: Christopher Head - Follow-up to 1c6c59a from several months ago when CyaSSL named symbols - were renamed to wolfSSL. The wolfSSL library was formerly named CyaSSL - and we kept using their old name for compatibility reasons, until - earlier this year. - -Daniel Stenberg (18 Nov 2019) -- RELEASE-NOTES: synced - -- [Javier Blazquez brought this change] - - ngtcp2: use overflow buffer for extra HTTP/3 data + Ref: https://github.com/curl/curl/pull/2787 - Fixes #4525 - Closes #4603 + Fixes https://github.com/curl/curl/issues/4758 -- altsvc: bump to h3-24 +- examples/postinmemory.c: Call curl_global_cleanup always - ... as both ngtcp2 and quiche now support that in their master branches + Prior to this change curl_global_cleanup was not called if + curl_easy_init failed. + + Reported-by: kouzhudong@users.noreply.github.com - Closes #4604 + Fixes https://github.com/curl/curl/issues/4751 -- ngtcp2: free used resources on disconnect +Daniel Stenberg (21 Dec 2019) +- url2file.c: fix copyright year - Fixes #4614 - Closes #4615 + Follow-up to 525787269599b5 -- ngtcp2: handle key updates as ngtcp2 master branch tells us +- [Rickard Hallerbäck brought this change] + + examples/url2file.c: corrected a comment - Reviewed-by: Tatsuhiro Tsujikawa + The comment was confusing and suggested that setting CURLOPT_NOPROGRESS + to 0L would both enable and disable debug output at the same time, like + a Schrödinger's cat of CURLOPTs. - Fixes #4612 - Closes #4613 + Closes #4745 + +- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 -Jay Satiro (17 Nov 2019) -- [Gergely Nagy brought this change] +- RELEASE-NOTES: synced - multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds +Jay Satiro (20 Dec 2019) +- ngtcp2: Support the latest update key callback type - Prior to this change: + - Remove our cb_update_key in favor of ngtcp2's new + ngtcp2_crypto_update_key_cb which does the same thing. - The check if an extra wait is necessary was based not on the - number of extra fds but on the pointer. + Several days ago the ngtcp2_update_key callback function prototype was + changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to + fix up our cb_update_key for that change they also added + ngtcp2_crypto_update_key_cb which does the same thing so we'll use that + instead. - If a non-null pointer was given in extra_fds, but extra_nfds - was zero, then the wait was skipped even though poll was not - called. + Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c - Closes https://github.com/curl/curl/pull/4610 + Closes https://github.com/curl/curl/pull/4735 -- lib: Move lib/ssh.h -> lib/vssh/ssh.h +Daniel Stenberg (19 Dec 2019) +- sws: search for "Testno:" header uncondtionally if no testno - Follow-up to 5b2d703 which moved ssh source files to vssh. + Even if the initial request line wasn't found. With the fix to 1455, the + test number is now detected correctly. - Closes https://github.com/curl/curl/pull/4609 - -Daniel Stenberg (16 Nov 2019) -- [Andreas Falkenhahn brought this change] - - INSTALL.md: provide Android build instructions + (Problem found when running tests in random order.) - Closes #4606 + Closes #4744 -- [Niall O'Reilly brought this change] +- tests: set LC_ALL in more tests + + Follow-up to 23208e330ac0c21 + + Closes #4743 - doh: improced both encoding and decoding +- test165: set LC_ALL=en_US.UTF-8 too - Improved estimation of expected_len and updated related comments; - increased strictness of QNAME-encoding, adding error detection for empty - labels and names longer than the overall limit; avoided treating DNAME - as unexpected; + On my current Debian Unstable with libidn2 2.2.0, I get an error if + LC_ALL is set to blank. Then curl errors out with: - updated unit test 1655 with more thorough set of proofs and tests + curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 - Closes #4598 + Closes #4738 -- ngtcp2: increase QUIC window size when data is consumed +- curl.h: add two defines for the "pre ISO C" case - Assisted-by: Javier Blazquez - Ref #4525 (partial fix) - Closes #4600 - -- [Melissa Mears brought this change] + Without this fix, this caused a compilation failure on AIX with IBM xlc + 13.1.3 compiler. + + Reported-by: Ram Krushna Mishra + Fixes #4739 + Closes #4740 - config-win32: cpu-machine-OS for Windows on ARM +- create_conn: prefer multiplexing to using new connections - Define the OS macro properly for Windows on ARM builds. Also, we might - as well add the GCC-style IA-64 macro. + ... as it would previously prefer new connections rather than + multiplexing in most conditions! The (now removed) code was a leftover + from the Pipelining code that was translated wrongly into a + multiplex-only world. - Closes #4590 + Reported-by: Kunal Ekawde + Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html + Closes #4732 -- examples: add multi-poll.c +- test1456: remove the use of a fixed local port - Show how curl_multi_poll() makes it even easier to use the multi - interface. + Fixup the test to instead not compare the port number. It sometimes + caused problems like this: - Closes #4596 - -- multi_poll: avoid busy-loop when called without easy handles attached + "curl: (45) bind failed with errno 98: Address already in use" - Fixes #4594 - Closes #4595 - Reported-by: 3dyd on github + Closes #4733 -- curl: fix -T globbing - - Regression from e59371a4936f8 (7.67.0) +Jay Satiro (18 Dec 2019) +- CURLOPT_QUOTE.3: fix typos - Added test 490, 491 and 492 to verify the functionality. + Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man + pages would not compile because a variable name was incorrect. - Reported-by: Kamil Dudka - Reported-by: Anderson Sasaki + Reported-by: Bylon2@users.noreply.github.com - Fixes #4588 - Closes #4591 + Fixes https://github.com/curl/curl/issues/4736 -- HISTORY: added cmake, HTTP/3 and parallel downloads with curl +- [Gisle Vanem brought this change] -- quiche: reject headers in the wrong order - - Pseudo header MUST come before regular headers or cause an error. + strerror: Fix compiler warning "empty expression" - Reported-by: Cynthia Coan - Fixes #4571 - Closes #4584 - -- openssl: prevent recursive function calls from ctx callbacks + - Remove the final semi-colon in the SEC2TXT() macro definition. - Follow the pattern of many other callbacks. + Before: #define SEC2TXT(sec) case sec: txt = #sec; break; - Ref: #4546 - Closes #4585 - -- CURL-DISABLE: initial docs for the CURL_DISABLE_* defines + After: #define SEC2TXT(sec) case sec: txt = #sec; break - The disable-scan script used in test 1165 is extended to also verify - that the docs cover all used defines and all defines offered by - configure. + Prior to this change SEC2TXT(foo); would generate break;; which caused + the empty expression warning. - Reported-by: SLDiggie on github - Fixes #4545 - Closes #4587 + Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 -- remove_handle: clear expire timers after multi_done() - - Since 59041f0, a new timer might be set in multi_done() so the clearing - of the timers need to happen afterwards! +Daniel Stenberg (18 Dec 2019) +- curl/parseconfig: use curl_free() to free memory allocated by libcurl - Reported-by: Max Kellermann - Fixes #4575 - Closes #4583 + Reported-by: bxac on github + Fixes #4730 + Closes #4731 -Marcel Raad (10 Nov 2019) -- test1558: use double slash after file: +- curl/parseconfig: fix mem-leak - Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this - test unnecessarily failed when using `file:/` instead of `file:///`. + When looping, first trying '.curlrc' and then '_curlrc', the function + would not free the first string. - Closes https://github.com/curl/curl/pull/4554 + Closes #4731 -Daniel Stenberg (10 Nov 2019) -- pause: avoid updating socket if done was already called - - ... avoids unnecesary recursive risk when the transfer is already done. +- CURLOPT_URL.3: "curl supports SMB version 1 (only)" - Reported-by: Richard Bowker - Fixes #4563 - Closes #4574 + [skip ci] -Jay Satiro (9 Nov 2019) -- strerror: Fix an error looking up some Windows error strings - - - Use FORMAT_MESSAGE_IGNORE_INSERTS to ignore format specifiers in - Windows error strings. - - Since we are not in control of the error code we don't know what - information may be needed by the error string's format specifiers. - - Prior to this change Windows API error strings which contain specifiers - (think specifiers like similar to printf specifiers) would not be shown. - The FormatMessage Windows API call which turns a Windows error code into - a string could fail and set error ERROR_INVALID_PARAMETER if that error - string contained a format specifier. FormatMessage expects a va_list for - the specifiers, unless inserts are ignored in which case no substitution - is attempted. +- test1270: a basic -w redirect_url test - Ref: https://devblogs.microsoft.com/oldnewthing/20071128-00/?p=24353 + Closes #4728 -- [r-a-sattarov brought this change] +- HISTORY: the SMB(S) support landed in 2014 - system.h: fix for MCST lcc compiler - - Fixed build by MCST lcc compiler on MCST Elbrus 2000 architecture and do - some code cleanup. - - e2k (Elbrus 2000) - this is VLIW/EPIC architecture, like Intel Itanium - architecture. - - Ref: https://en.wikipedia.org/wiki/Elbrus_2000 +- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore - Closes https://github.com/curl/curl/pull/4576 - -Daniel Stenberg (8 Nov 2019) -- TODO: curl_multi_unblock + It is covered by USE_OPENSSL_ENGINE now. - Closes #4418 - -- TODO: Run web-platform-tests url tests + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 - Closes #4477 + Closes #4725 -- cgit v1.2.3