From 43f100ad4b599dbc564f9920e63afdc242f0a27c Mon Sep 17 00:00:00 2001 From: dartraiden Date: Wed, 26 May 2021 22:25:25 +0300 Subject: libcurl: update to 7.77 --- libs/libcurl/docs/CHANGES | 10504 ++++++++++++++++++++++---------------------- libs/libcurl/docs/THANKS | 44 + 2 files changed, 5372 insertions(+), 5176 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 62dce2ee97..e7a462b50e 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,7673 +6,7825 @@ Changelog -Version 7.76.1 (14 Apr 2021) +Version 7.77.0 (26 May 2021) -Daniel Stenberg (14 Apr 2021) +Daniel Stenberg (26 May 2021) - RELEASE-NOTES: synced - - curl 7.76.1 release -- THANKS: add names from 7.76.1 +- THANKS: added contributors from 7.77.0 cycle -- misc: update copyright year ranges to match latest updates +- copyright: update copyright year ranges to 2021 -- [Tatsuhiro Tsujikawa brought this change] +- [Radek Zajic brought this change] - ngtcp2: Use ALPN h3-29 for now + hostip: fix broken macOS/CMake/GCC builds - Fixes #6864 - Cloes #6886 + Follow-up to 31f631a142d855f06 + + Fixes #7128 + Closes #7129 -Jay Satiro (11 Apr 2021) -- TODO: remove 18.22 --fail-with-body +- TODO: netrc caching and sharing - --fail-with-body was added in 8a964cb (precedes curl-7_76_0). + URL: https://curl.se/mail/archive-2021-05/0018.html -Daniel Stenberg (10 Apr 2021) -- [Jürgen Gmach brought this change] +- [Orgad Shaneh brought this change] - src/tool_vms.c: remove duplicated word in comment + setopt: streamline ssl option code - Closes #6881 + Make it use the same style as the code next to it + + Closes #7123 -- configure: fix CURL_DARWIN_CFLAGS use +- [Radek Zajic brought this change] + + lib/hostip6.c: make NAT64 address synthesis on macOS work - The macro name change was not completely done. + Closes #7121 + +- [ejanchivdorj brought this change] + + sectransp: fix EXC_BAD_ACCESS caused by uninitialized buffer - Follow-up to 5d2c384452543c - Bug: https://github.com/curl/curl/commit/5d2c384452543c7b6c9fb02eaa0afc84fd5ab941#commitcomment-49315187 - Reported-by: Marcel Raad - Closes #6878 + When the SecCertificateCopyCommonName function fails, it leaves + common_name in a invalid state so CFStringCompare uses the invalid + result, causing EXC_BAD_ACCESS. + + The fix is to check the return value of the function before using the + name. + + Closes #7126 -- [Anthony Shaw brought this change] +- [Paweł Wegner brought this change] - github/workflow: add "security-extended" to codeql-analysis.yml + CMake: add CURL_ENABLE_EXPORT_TARGET option - Extends the CodeQL code scan. + install(EXPORT ...) causes trouble when embedding curl dependencies + which don't provide install(EXPORT ...) targets (e.g libressl and + nghttp2) with cmake's add_subdirectory. - Closes #6815 + Reviewed-by: Jakub Zakrzewski + Closes #7060 -- [Jochem Broekhoff brought this change] +- [Alessandro Ghedini brought this change] - examples/hiperfifo.c: check event_initialized before delete + quiche: update for network path aware API - If event_del is called with the event struct (still) zeroed out, a - segmentation fault may occur. event_initialized checks whether the - event struct is nonzero. + Latest version of quiche requires the application to pass the peer + address of received packets, and it provides the address for outgoing + packets back. - Closes #6876 + Closes #7120 -- [Patrick Monnerat brought this change] +- [Jacob Hoffman-Andrews brought this change] - ntlm: fix negotiated flags usage + rustls: switch read_tls and write_tls to callbacks - According to Microsoft document MS-NLMP, current flags usage is not - accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of - extended security in an NTLM authentication message and NTLM version 2 - cannot be negotiated within the protocol. + And update to 0.6.0, including a rename from session to connection for + many fields. - The solution implemented here is: if the extended security flag is set, - prefer using NTLM version 2 (as a server featuring extended security - should also support version 2). If version 2 has been disabled at - compile time, use extended security. + Closes #7071 + +- [Koichi Shiraishi brought this change] + + sectransp: fix 7f4a9a9b2a49 commit about missing comma - Tests involving NTLM are adjusted to this new behavior. + Follow-up to 7f4a9a9b2a495 - Fixes #6813 - Closes #6849 + Closes #7119 -- [Patrick Monnerat brought this change] +- [Harry Sintonen brought this change] - ntlm: support version 2 on 32-bit platforms + openssl: associate/detach the transfer from connection - Closes #6849 + CVE-2021-22901 + + Bug: https://curl.se/docs/CVE-2021-22901.html -- [Patrick Monnerat brought this change] +- [Harry Sintonen brought this change] - curl_ntlm_core.h: simplify conditionals for USE_NTLM2SESSION + telnet: check sscanf() for correct number of matches - ... as !defined(CURL_DISABLE_CRYPTO_AUTH) is a prerequisite for the - whole NTLM. + CVE-2021-22898 - Closes #6849 + Bug: https://curl.se/docs/CVE-2021-22898.html -- lib: remove unused HAVE_INET_NTOA_R* defines +- schannel: don't use static to store selected ciphers - Closes #6867 + CVE-2021-22897 + + Bug: https://curl.se/docs/CVE-2021-22897.html -- [Michael Forney brought this change] +- docs/tests: remove freenode references - configure: include unconditionally +- RELEASE-NOTES: synced + +- [Sergey Markelov brought this change] + + NSS: make colons, commas and spaces valid separators in cipher list - In 2682e5f5, several instances of AC_HEADER_TIME were removed since - it is a deprecated autoconf macro. However, this was the macro that - defined TIME_WITH_SYS_TIME, which was used to indicate that - can be included alongside . TIME_WITH_SYS_TIME is still - used in the configure test body and since it is no longer defined, - is *not* included on systems that have . + Fixes #7110 + Closes #7115 + +- curl: include libmetalink version in --version output - In particular, at least on musl libc and glibc, does - not implicitly include and does not declare clock_gettime, - gmtime_r, or localtime_r. This causes configure to fail to detect - those functions. + Closes #7112 + +Jay Satiro (21 May 2021) +- [Matias N. Goldberg brought this change] + + cmake: Use multithreaded compilation on VS 2008+ - The AC_HEADER_TIME macro deprecation text says + Multithreaded compilation has been supported since at least VS 2005 and + been robustly stable since at least VS 2008 - > All current systems provide time.h; it need not be checked for. - > Not all systems provide sys/time.h, but those that do, all allow - > you to include it and time.h simultaneously. + Closes https://github.com/curl/curl/pull/7109 + +Daniel Stenberg (21 May 2021) +- [Matias N. Goldberg brought this change] + + cmake: fix two invokes result in different curl_config.h - So, to fix this issue, simply include unconditionally when - testing for time-related functions and in libcurl, and don't bother - checking for it. + Fixes #7100 + Closes #7101 - Closes #6859 + Reviewed-by: Jakub Zakrzewski + Signed-off-by: Matias N. Goldberg -- [Michael Forney brought this change] +- [Peng-Yu Chen brought this change] - configure: remove use of RETSIGTYPE + cmake: detect CURL_SA_FAMILY_T - This was previously defined by the obsolete AC_TYPE_SIGNAL macro, - which was removed in 2682e5f5. The deprecation text says + Fixes #7049 + Closes #7065 + +- [Lucas Clemente Vella brought this change] + + CURLOPT_IPRESOLVE: preventing wrong IP version from being used - > Your code may safely assume C89 semantics that RETSIGTYPE is void. + In some situations, it was possible that a transfer was setup to + use an specific IP version, but due do DNS caching or connection + reuse, it ended up using a different IP version from requested. - So, remove it and just use void instead. + This commit changes the effect of CURLOPT_IPRESOLVE from simply + restricting address resolution to preventing the wrong connection + type being used, when choosing a connection from the pool, and + to restricting what addresses could be used when establishing + a new connection. - Closes #6861 + It is important that all addresses versions are resolved, even if + not used in that transfer in particular, because the result is + cached, and could be useful for a different transfer with a + different CURLOPT_IPRESOLVE setting. + + Closes #6853 -- [Muhammed Yavuz Nuzumlalı brought this change] +- [Oliver Urbann brought this change] - install: add instructions for Apple Darwin platforms + AmigaOS: add functions definitions for SHA256 - Closes #6860 + AmiSSL replaces many functions with macros. Curl requires pointer + to some of these functions. Thus, we have to encapsulate these macros: + SHA256_Init, SHA256_Update, SHA256_Final, X509_INFO_free. + + Bug: https://github.com/jens-maus/amissl/issues/15 + Co-authored-by: Daniel Stenberg + + Closes #7099 -- [Muhammed Yavuz Nuzumlalı brought this change] +- test2100: make it run with and require IPv6 + + Closes #7083 - configure: disable min version set for Darwin +- tests/getpart: generate output URL encoded for better diffs - Fixes #6838 - Closes #6860 + Closes #7083 -- [David Hu brought this change] +- [Ryan Beck-Buysse brought this change] - docs/HTTP3.md: update the build instruction using gnutls + docs/TheArtOfHttpScripting: fix markdown links - In ngtcp2 the `with-gnutls` option is disabled by default, which will - cause `curl` unable to be `make` because of lacking the libraries - needed. + extra parens cause the links to be incorrectly formatted + and inconsistent with the rest of the document. - Closes #6857 + Signed-off-by: Ryan Beck-Buysse + Closes #7097 - RELEASE-NOTES: synced -- typecheck-gcc: make the ssl-ctx-cb check use SSL_CTX pointers - - ... and not values. - - Reported-by: locpyl-tidnyd on github - Fixes #6818 - Closes #6819 +- [Emil Engler brought this change] -- ngtcp2+gnutls: clear credentials when freed + docs: replace dots with dashes in markdown enums - ... to avoid double-free. + We use dashes instead of dots nearly everywhere except for those few + cases. This commit addresses this issues and brings more coherency into + it. - Reported-by: Kenneth Davidson - Fixes #6824 - Closes #6856 + Closes #7093 -Jay Satiro (5 Apr 2021) -- [Cherish98 brought this change] +- [Emil Engler brought this change] - tool_progress: Fix progress meter in parallel mode - - Make sure the total amount of DL/UL bytes are counted before the - transfer finalizes. Otherwise if a transfer finishes too quick, its - total numbers are not added, and results in a DL%/UL% that goes above - 100%. + docs: improve INTERNALS.md regarding getsock cb - Detail: + This adds the I/O prefix to indicate that those "actions" are kind-of + related to those found in select(2) or poll(2) (reading/writing). - progress_meter() is called periodically, and it may not catch a - transfer's total bytes if the value was unknown during the last call, - and the transfer is finished and deleted (i.e., lost) during the next - call. + It also adds a note where the prototypes of those functions can be found + in the source code. - Closes https://github.com/curl/curl/pull/6840 + Closes #7092 - [Emil Engler brought this change] - libssh: get rid of PATH_MAX + docs: document attach in INTERNALS.md - This removes the last occurrence of PATH_MAX inside our libssh - implementation by calculating the path length from the string length of - the two components. + The new field in the Curl_handler struct still lacks documentation. This + adds it it from the information extracted from lib/urldata.h:797 - Closes #6829 + Closes #7091 -Daniel Stenberg (5 Apr 2021) -- http_proxy: only loop on 407 + close if we have credentials +- [Marc Aldorasi brought this change] + + config: remove now-unused macros - ... to fix the retry-loop. + Closes #7094 + +- [Marc Aldorasi brought this change] + + hostip.h: remove declaration of unimplemented function - Add test 718 to verify. + Closes #7094 + +- h3: add 'attach' callback to protocol handlers - Reported-by: Daniel Kurečka - Fixes #6828 - Closes #6850 + Follow-up to 0c55fbab45be + + Reviewed-by: Emil Engler + Closes #7090 -- h2: allow 100 streams by default +- wolfssl: remove SSLv3 support leftovers - instead of 13, before the server has told how many streams it - accepts. The server can always reject new streams anyway if we go above - what it accepts. + Closes #7088 + +- curl-wolfssl.m4: without custom include path, assume /usr/include - Ref: #6826 - Closes #6852 + ... so that we can point out the root of the OpenSSL emulation headers. + Previously this used the '$includedir' variable which is wrong since + that defaults to the dir where the current configure invoke will install + the built libcurl headers: /usr/local by default. + + Fixes #7085 + Reported-by: Joel Jakobsson + Closes #7087 -- [Luke Granger-Brown brought this change] +- [Joel Depooter brought this change] - file: support GETing directories again + data_pending: check only SECONDARY socket for FTP(S) transfers - After 957bc1881e686f9714c4e6a01bf33535091f0e21, we no longer compute an - expected_size for directories. This has the upshot that when we compare - even an empty Range with the available size, we fail. + Check the FIRST for all other protocols. - This brings back the previous behaviour, which was to succeed, but with - empty content. This also removes the "Accept-ranges: bytes" header, - which is nonsensical on directories. + This fixes a timeout in an ftps download. The server sends a TLS + close_notify message in the same packet as the file data. The + close_notify seems to not be handled in the schannel_recv function, so + libcurl is not aware that the server has closed the connection. Thus + libcurl ends up waiting for action on the socket until a timeout is + reached. With the secondary socket check added to the data_pending + function, the close_notify is properly handled, and the ftps transfer + terminates as expected. - Adds test 3016 - Fixes #6845 - Closes #6846 + Fixes #7068 + Closes #7069 -- RELEASE-NOTES: synced +- github: inhibit deprecated declarations for clang on macOS - and bumped to 7.76.1 + ... as they otherwise cause ldap build errors in the CI. + + Fixes #7081 + Closes #7082 -- TLS: fix HTTP/2 selection +- conn: add 'attach' to protocol handler, make libssh2 use it - for GnuTLS, BearSSL, mbedTLS, NSS, SChannnel, Secure Transport and - wolfSSL... + The libssh2 backend has SSH session associated with the connection but + the callback context is the easy handle, so when a connection gets + attached to a transfer, the protocol handler now allows for a custom + function to get used to set things up correctly. - Regression since 88dd1a8a115b1f5ece (shipped in 7.76.0) - Reported-by: Kenneth Davidson - Reported-by: romamik om github - Fixes #6825 - Closes #6827 + Reported-by: Michael O'Farrell + Fixes #6898 + Closes #7078 -Jay Satiro (2 Apr 2021) -- hostip: Fix for builds that disable all asynchronous DNS +- http2: make sure pause is done on HTTP - - Define Curl_resolver_error function only when USE_CURL_ASYNC. + Since the function is called for any protocol, we can't assume that the + HTTP struct is there without first making sure it is HTTP. - Prior to this change building curl without an asynchronous resolver - backend (c-ares or threaded) and without DoH (DNS-over-HTTPS, which is - also asynchronous but independent of resolver backend) would cause a - build error since Curl_resolver_error is called by and evaluates - variables only available in asynchronous builds. + Reported-by: Denis Goleshchikhin + Fixes #7079 + Closes #7080 + +- docs: cookies from HTTP headers need domain set - Reported-by: Benbuck Nason + ... or the cookies won't get sent. Push users to using the "Netscape" + format instead, which curl uses when saving a cookie "jar". - Fixes https://github.com/curl/curl/issues/6831 - Closes https://github.com/curl/curl/pull/6832 + Reported-by: Martin Dorey + Reviewed-by: Daniel Gustafsson + Fixes #6723 + Closes #7077 -Daniel Stenberg (31 Mar 2021) -- [Gilles Vollant brought this change] +- RELEASE-NOTES: synced - openssl: Fix CURLOPT_SSLCERT_BLOB without CURLOPT_SSLCERT_KEY +- github: add a workflow with libssh2 on macOS using cmake - Reported-by: Christian Schmitz - Fixes #6816 - Closes #6820 - -Version 7.76.0 (31 Mar 2021) + Closes #7047 -Daniel Stenberg (31 Mar 2021) -- RELEASE-NOTES: synced +- sws: allow HTTP requests up to 2MB in size - curl 7.76.0 release - -- THANKS: added names from 7.76.0 + To allow tests with slightly larger payloads. Like #7071 ... + + Closes #7075 -- CURLOPT_AUTOREFERER.3: clarify that it sets the full URL +Marc Hoersken (16 May 2021) +- CI/azure: increase verbosity and fix outdated task names - ... some users may not want that! + Closes #7063 -- define: remove CURL_DISABLE_NTLM ifdefs +- CI/cirrus: add shared and static Windows release builds - It was never defined anywhere. Fixed disable-scan (test 1165) to also - scan headers, which found this issue. + Azure Pipelines is currently being used for debug builds, + let's also run some non-debug (release) Windows builds and + make use of previously underutilized Cirrus CI for that. - Closes #6809 + Reviewed-by: Marcel Raad + + Closes #6991 -- vtls: fix addsessionid for non-proxy builds +Daniel Stenberg (16 May 2021) +- CURLOPT_CAPATH.3: defaults to a path, not NULL - Follow-up to b09c8ee15771c61 - Fixes #6812 - Closes #6811 + Reported-by: Andrew Barnert + + Closes #7062 -- [Li Xinwei brought this change] +- [Jacob Hoffman-Andrews brought this change] - cmake: support WinIDN + c-hyper: handle body on HYPER_TASK_EMPTY - Closes #6807 - -- transfer: clear 'referer' in declaration + Some of the time, we get a HYPER_TASK_EMPTY response before the status + line, headers, and body have been read. Previously, that would cause us + to poll again, leading to a 1 second timeout. - To silence (false positive) compiler warnings about it. + The HYPER_TASK_EMPTY docs say: - Follow-up to 7214288898f5625 + The value of this task is null (does not imply an error). - Reviewed-by: Marcel Raad - Closes #6810 + So, if we receive a HYPER_TASK_EMPTY, continue on with processing the + response. + + Reported-by: Kevin Burke + Fixes #7064 + Closes #7070 -- [Marc Hoersken brought this change] +- [Ikko Ashimine brought this change] - config: fix SSPI enabling NTLM if crypto auth is disabled + tool_getparam: fix comment typo in tool_getparam.c - Avoid enabling NTLM feature based upon Windows SSPI - being enabled in case that crypto auth is disabled. + enfore -> enforce - Reported-by: Marcel Raad + Closes #7074 + +- mem-include-scan.pl: require a non-word letter before memory funcs - Follow-up to #6277 - Fixes #6803 - Closes #6808 + ... so that ldap_memfree() for example doesn't match the scan for free. + + Closes #7061 -- HISTORY: add two 2021 events +- version: free the openldap info correctly + + ... to avoid memory leaks. + + Follow-up to: bf0feae7768d9 + Closes #7061 -- vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid() +- dupset: remove totally off comment - To make sure we set and extract the correct session. + Closes #7067 + +- configure: if asked for, fail if ldap is not found - Reported-by: Mingtao Yang - Bug: https://curl.se/docs/CVE-2021-22890.html + Reported-by: Jakub Zakrzewski + Fixes #7053 + Closes #7055 + +- version: add OpenLDAP version in the output - CVE-2021-22890 + Assisted-by: Howard Chu + Closes #7054 -- [Viktor Szakats brought this change] +Jay Satiro (13 May 2021) +- [Joel Depooter brought this change] - transfer: strip credentials from the auto-referer header field + schannel: Ensure the security context request flags are always set - Added test 2081 to verify. + As of commit 54e7475, these flags would only be set when using a new + credential handle. When re-using an existing credential handle, the + flags would not be set. - CVE-2021-22876 + Closes https://github.com/curl/curl/pull/7051 + +Dan Fandrich (12 May 2021) +- tests: Fix some tag matching issues in a number of tests + +Daniel Stenberg (12 May 2021) +- sasl: use 'unsigned short' to store mechanism - Bug: https://curl.se/docs/CVE-2021-22876.html + ... saves a few bytes of struct size in memory and it only uses + 10 bits anyway. + + Closes #7045 -- curl_sasl: fix compiler error with --disable-crypto-auth +- hostip: remove the debug code for LocalHost - ... if libgsasl was found. + The Curl_resolv() had special code (when built in debug mode) for when + resolving the host name "LocalHost" (using that exact casing). It would + then get the host name from the --interface option instead. - Closes #6806 + This development-only feature was not used by anything (anymore) and we + have the --resolve feature if we want to play similar tricks properly + going forward. + + Closes #7044 -- [Patrick Monnerat brought this change] +- progress: reset limit_size variables at transfer start + + Otherwise the old value would linger from a previous use and would mess + up the network speed cap logic. + + Reported-by: Ymir1711 on github + + Fixes #7042 + Closes #7043 - ldap: only set the callback ptr for TLS context when TLS is used +- RELEASE-NOTES: synced + +- [Daniel Gustafsson brought this change] + + cookies: use CURLcode for cookie_output reporting - Follow-up to a5eee22e594c2460f - Fixes #6804 - Closes #6805 + Writing the cookie file has multiple error conditions, and was using an + int with magic numbers to report the different error (which in turn were + disregarded anyways). This moves reporting to use a CURLcode value. + + Lightly-touched-by: Daniel Stenberg + + Closes #7037 + Closes #6749 -- copyright: update copyright year ranges to 2021 +- [Daniel Gustafsson brought this change] + + cookies: make use of string duplication function - Reviewed-by: Emil Engler - Closes #6802 + strstore() is defined as a strdup which ensures to free the target + pointer before duping the source char * into it. Make use of it in + two more cases where it can simplify the code. -- send_speed: simplify the checks for if a speed limit is set +- [Daniel Gustafsson brought this change] + + cookies: refactor comments - ... as we know the value cannot be set to negative: enforced by - setopt() + Comments in the cookie code were a bit all over the place in terms of + style and wording. This takes a stab at cleaning them up by keeping to + a single style and overall shape. Some comments are moved a little and + some removed alltogether due to being redundant. No functional changes + have been made, -- http: cap body data amount during send speed limiting +- [Peng-Yu Chen brought this change] + + http2: skip immediate parsing of payload following protocol switch - By making sure never to send off more than the allowed number of bytes - per second the speed limit logic is given more room to actually work. + This is considered not harmful as a following http2_recv shall be + called very soon. - Reported-by: Fabian Keil - Bug: https://curl.se/mail/lib-2021-03/0042.html - Closes #6797 + This is considered helpful in the specific situation where some + servers (e.g. nghttpx v1.43.0) may fulfill stream 1 immediately + following the return of HTTP status 101, other than waiting for + the client-side connection preface to arrive. + + Fixes #7036 + Closes #7040 -- urldata: merge "struct DynamicStatic" into "struct UrlState" +- [Peng-Yu Chen brought this change] + + http2: use nghttp2_session_upgrade2 instead of nghttp2_session_upgrade - Both were used for the same purposes and there was no logical separation - between them. Combined, this also saves 16 bytes in less holes in my - test build. + Following the upstream deprecation of nghttp2_session_upgrade. - Closes #6798 - -- tests/README.md: mentioned that en_US.UTF-8 is required + Also provides further checks for requests with the HEAD method. - Reported-by: Oumph on github - Fixes #6768 + Closes #7041 -- HISTORY: fixed the Mac OS X 10.1 release date +- progress/trspeed: use a local convenient pointer to beautify code - Based on what Wikipedia says + The function becomes easier to read and understand with less repetition. -Jay Satiro (26 Mar 2021) -- examples: Remove threaded-shared-conn.c due to bug +- trspeed: use long double for transfer speed calculation + +- progress: move transfer speed calc into function - Known bug 11.11 is the shared object's connection cache is not thread - safe, so we should not have an example for it. + This silences two scan-build-11 warnings: "The result of the '/' + expression is undefined" - Ref: https://github.com/curl/curl/issues/4915 - Ref: https://curl.se/docs/knownbugs.html#A_shared_connection_cache_is_not + Bug: https://curl.se/mail/lib-2021-05/0022.html + Closes #7035 + +- [Cameron Cawley brought this change] + + openssl: remove unneeded cast for CertOpenSystemStore() - Closes https://github.com/curl/curl/pull/6795 + Closes #7025 -- KNOWN_BUGS: Update 11.9 - DoH option inheritance +- travis: disable the libssh build - - Add description: Explain that some options aren't inherited because - they are not relevant for the DoH SSL connections or may result in - unexpected behavior. + It can't run on focal and causes warnings on bionic. Since the focal + failure started rather suddenly a while ago, we can suspect it might be + temporary. - - Remove the reference to #4578 (SSL verify options not inherited) since - that was fixed by #6597 (separate DoH-specific options for verify). + Added "bring back the build" to the TODO document. - - Explain that DoH-specific options (those created by #6597) are - available: CURLOPT_DOH_SSL_VERIFYHOST, CURLOPT_DOH_SSL_VERIFYPEER and - CURLOPT_DOH_SSL_VERIFYSTATUS. + Fixes #7011 + Closes #7012 + +- [Peng-Yu Chen brought this change] + + http: use calculated offsets inst of integer literals for header parsing - - Add a reference to #6605 and explain that the user's debug function is - not inherited because it would be unexpected to pass internal handles - (ie DoH handles) to the user's callback. + Assumed to be a minor coding style improvement with no behavior change. - Closes https://github.com/curl/curl/issues/6605 + A modern compiler is expected to have the calculation optimized during + compilation. It may be deemed okay even if that's not the case, since + the added overhead is considered very low. + + Closes #7032 -Daniel Stenberg (26 Mar 2021) -- curl_easy_setopt.3: add curl_easy_option* functions to SEE ALSO +- [Peng-Yu Chen brought this change] -- [Jean-Philippe Menil brought this change] + GIT-INFO: suggest using autoreconf instead of buildconf + + Follow-up to 85868537d + + Closes #7033 - openssl: ensure to check SSL_CTX_set_alpn_protos return values +- http: deal with partial CONNECT sends - SSL_CTX_set_alpn_protos() return 0 on success, and non-0 on failure + Also added 'CURL_SMALLSENDS' to make Curl_write() send short packets, + which helped verifying this even more. - Signed-off-by: Jean-Philippe Menil + Add test 363 to verify. - Closes #6794 + Reported-by: ustcqidi on github + Fixes #6950 + Closes #7024 -- multi: close the connection when h2=>h1 downgrading +- HTTP3: make the ngtcp2 build use the quictls fork - Otherwise libcurl is likely to reuse the connection again in the next - attempt since the connection reuse logic doesn't take downgrades into - account. + ... as ngtcp2 itself documents the build this way. - Reported-by: Anthony Ramine - Fixes #6788 - Closes #6793 + Closes #7031 -- openssl: set the transfer pointer for logging early +- http: limit the initial send amount to used upload buffer size - Otherwise, the transfer will be NULL in the trace function when the - early handshake details arrive and then curl won't show them. + Previously this logic would cap the send to CURL_MAX_WRITE_SIZE bytes, + but for the situations where a larger upload buffer has been set, this + function can benefit from sending more bytes. With default size used, + this does the same as before. - Regresssion in 7.75.0 + Also changed the storage of the size to an 'unsigned int' as it is not + allowed to be set larger than 2M. - Reported-by: David Hu - Fixes #6783 - Closes #6792 + Also added cautions to the man pages about changing buffer sizes in + run-time. + + Closes #7022 - RELEASE-NOTES: synced -- TODO: Custom progress meter update interval +- ngtcp2: fix the cb_acked_stream_data_offset proto - Ref: https://stackoverflow.com/q/66789977/93747 + The 'datalen' value should be 64 bit, not size_t! + + Reported-by: Dmitry Karpov + Bug: https://curl.se/mail/lib-2021-05/0019.html + Closes #7027 -- docs/ABI: tighten up the language +- progress: when possible, calculate transfer speeds with microseconds - Make the promises more firm + ... this improves precision, especially for transfers in the few or even + sub millisecond range. - Closes #6786 + Reported-by: J. Bromley + Fixes #7017 + Closes #7020 -- openldap: disconnect better +- http: reset the header buffer when sending the request - Instead of clearing the callback argument in disconnect, set it to the - (new) transfer to make sure the correct data is passed to the callbacks. + A reused transfer handle could otherwise reuse the previous leftover + buffer and havoc would ensue. - Follow-up to e467ea3bd937f38 - Assisted-by: Patrick Monnerat - Closes #6787 + Reported-by: sergio-nsk on github + Fixes #7018 + Closes #7021 -- libssh2: kdb_callback: get the right struct pointer +- curl_mprintf.3: add description - After the recent conn/data refactor in this source file, this function - was mistakenly still getting the old struct pointer which would lead to - crash on servers with keyboard-interactive auth enabled. - - Follow-up to a304051620b92e12b (shipped in 7.75.0) + These functions have existed in the API since the dawn of time. It is + about time we describe how they work, even if we discourage users from + using them. - Reported-by: Christian Schmitz - Fixes #6691 - Closes #6782 + Closes #7010 -- tftp: remove unused struct fields +- [Timothy Gu brought this change] + + URL-SYNTAX: update IDNA section for WHATWG spec changes - Follow-up to d3d90ad9c00530d + WHATWG URL has dictated the use of Nontransitional Processing (IDNA + 2008) for several years now. Chrome (and derivatives) still use + Transitional Processing, but Firefox and Safari have both switched. - Closes #6781 + Also document the fact that winidn functions differently from libidn2 + here. + + Closes #7026 -- openldap: avoid NULL pointer dereferences +- [Calvin Buckley brought this change] + + INSTALL: add IBM i specific quirks - Follow-up to a59c33ceffb8f78 - Reported-by: Patrick Monnerat - Fixes #6676 - Closes #6780 + Fixes #6830 + Closes #7013 -- http: strip default port from URL sent to proxy +- libcurl.3: mention the URL API - To make sure the Host: header and the URL provide the same authority - portion when sent to the proxy, strip the default port number from the - URL if one was provided. + To make it easier to find. Also a minor polish of libcurl-url.3 - Reported-by: Michael Brown - Fixes #6769 - Closes #6778 + Closes #7009 -- azure: disable test 433 on azure-ubuntu +- GnuTLS: don't allow TLS 1.3 for versions that don't support it - Something in that environment sets XDG_CONFIG_HOME for us in a way that - breaks the test. + Follow-up to 781864bedbc5 - Reported-by: Marc Hörsken - Fixes #6739 - Closes #6777 + ... as they don't understand it and will return error at us! + + Closes #7014 -- tftp: remove the 3600 second default timeout +Kamil Dudka (6 May 2021) +- tool_getparam: handle failure of curlx_convert_tchar_to_UTF8() - ... it was never meant to be there. + Reported by GCC analyzer: - Reported-by: Tomas Berger - Fixes #6774 - Closes #6776 + Error: GCC_ANALYZER_WARNING (CWE-476): + src/tool_getparam.c: scope_hint: In function 'parse_args' + src/tool_getparam.c:2318:38: warning[-Wanalyzer-possible-null-dereference]: dereference of possibly-NULL 'orig_opt' + lib/curlx.h:56: included_from: Included from here. + src/tool_getparam.c:28: included_from: Included from here. + lib/curl_multibyte.h:70:51: note: in definition of macro 'curlx_convert_tchar_to_UTF8' + src/tool_getparam.c:2316:16: note: in expansion of macro 'curlx_convert_tchar_to_UTF8' + + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Closes #7023 -- docs: make gen.pl support *italic* and **bold** +Daniel Stenberg (6 May 2021) +- scripts/delta: also show total number of days + +Marc Hoersken (5 May 2021) +- sockfilt: fix invalid increment of handles index variable nfd - Remove some nroffisms from the cmdline doc files to simplify editing, - and instead support this markdown style. + Only increment the array index if we actually stored a handle. - Closes #6771 + Follow up to e917492048f4b85a0fd58a033d10072fc7666c3b + Closes #6992 -- ngtcp2: sync with recent API updates +- sockfilt: avoid getting stuck waiting for writable socket - Closes #6770 - -- RELEASE-NOTES: synced + Reset FD_WRITE event using the same approach as in multi.c + + Follow up to b36442b24305f3cda7c13cc64b46838995a4985b + Closes #6992 -- libssh2:ssh_connect: clear session pointer after free +Jay Satiro (5 May 2021) +- test678: Fix for Windows multibyte builds - If libssh2_knownhost_init() returns NULL, like in an OOM situation, the - ssh session was freed but the pointer wasn't cleared which made libcurl - later call libssh2 to cleanup using the stale pointer. + Follow-up to 77fc385 from yesterday. - Fixes #6764 - Closes #6766 + Bug: https://github.com/curl/curl/pull/6662#issuecomment-832966557 + Reported-by: Marc Hörsken -- [Jacob Hoffman-Andrews brought this change] +- [Dmitry Kostjuchenko brought this change] - docs: document version of crustls dependency + build: fix compilation for Windows UWP platform - This also pins a specific release in the Travis test so future - API-breaking changins in crustls won't break curl builds. + - Include afunix.h which is necessary for sockaddr_un when + USE_UNIX_SOCKETS is defined on Windows. - Add RUSTLS documentation to release tarball. + Closes https://github.com/curl/curl/pull/7006 + +Daniel Stenberg (5 May 2021) +- gnutls: make setting only the MAX TLS allowed version work - Enable running tests for rustls, minus FTP tests (require - connect_blocking, which rustls doesn't implement) and 313 (requires CRL - handling). + Previously, settting only the max allowed TLS version, leaving the + minimum one at default, didn't actually set it and left it to default + (TLS 1.3) too! - Closes #6763 - -- [Jacob Hoffman-Andrews brought this change] + As a bonus, this change also removes the dead code handling of SSLv3 + since that version can't be set anymore (since eff614fb0242cb). + + Reported-by: Daniel Carpenter + Fixes #6998 + Closes #7000 - rustls: Handle close_notify. +- openldap: replace ldap_ prefix on private functions - If we get a close_notify, treat that as EOF. If we get an EOF from the - TCP stream, treat that as an error (because we should have ended the - connection earlier, when we got a close_notify). + Since openldap itself uses that prefix and with OpenĹDAP 2.5.4 (at + least) there's a symbol collision because of that. - Closes #6763 + The private functions now use the 'oldap_' prefix where it previously + used 'ldap_'. + + Reported-by: 3eka on github + Fixes #7004 + Closes #7005 -- docs: clarify timeouts for queued transfers in multi API +Jay Satiro (5 May 2021) +- http2: fix potentially uninitialized variable - Closes #6758 + introduced several days ago in 3193170. caught by visual studio linker. -- ftpserver: only load the preprocessed test file +- [Gilles Vollant brought this change] + + SSL: support in-memory CA certs for some backends - We always preprocess and tests are no longer sensible to load "raw" + - New options CURLOPT_CAINFO_BLOB and CURLOPT_PROXY_CAINFO_BLOB to + specify in-memory PEM certificates for OpenSSL, Schannel (Windows) + and Secure Transport (Apple) SSL backends. - Closes #6738 - -- tests: use %TESTNUMBER instead of fixed number + Prior to this change PEM certificates could only be imported from a file + and not from memory. - This makes the tests easier to copy and relocate to other test numbers - without having to update content. + Co-authored-by: moparisthebest@users.noreply.github.com - Closes #6738 - -- KNOWN_BUGS: CURLOPT_OPENSOCKETPAIRFUNCTION is missing + Ref: https://github.com/curl/curl/pull/4679 + Ref: https://github.com/curl/curl/pull/5677 + Ref: https://github.com/curl/curl/pull/6109 - Closes #5747 + Closes https://github.com/curl/curl/pull/6662 -- TODO: provide timing info for each redirect +Daniel Stenberg (4 May 2021) +- [David Cook brought this change] + + tests: ignore case of chunked hex numbers in tests - Closes #6743 + When hyper is used, it emits uppercase hexadecimal numbers for chunked + encoding lengths. Without hyper, lowercase hexadecimal numbers are used. + This change adds preprocessor statements to tests where this is an + issue, and adapts the fixtures to match. + + Closes #6987 -Jay Satiro (17 Mar 2021) -- docs: Add SSL backend names to CURL_SSL_BACKEND +- cmake: check for getppid and utimes - - Document the names that can be used with CURL_SSL_BACKEND: - bearssl, gnutls, gskit, mbedtls, mesalink, nss, openssl, rustls, - schannel, secure-transport, wolfssl + ... as they're checked for in the configure script and are used by + source code. - Ref: https://github.com/curl/curl/issues/2209#issuecomment-360623286 - Ref: https://github.com/curl/curl/issues/6717#issuecomment-800745201 + Removed checks for perror, setvbuf and strlcat since those defines are + not checked for in source code. - Closes https://github.com/curl/curl/pull/6755 + Bonus: removed HAVE_STRLCPY from a few config-*.h files since that + symbol is not used in source code. + + Closes #6997 -- docs: Explain DOH transfers inherit some SSL settings +- libtest: remove lib530.c - - Document in DOH that some SSL settings are inherited but DOH hostname - and peer verification are not and are controlled separately. + Follow up from e50a877df when test 530 was removed. Since then this + source file has not been used/needed. - - Document that CURLOPT_SSL_CTX_FUNCTION is inherited by DOH handles but - we're considering changing behavior to no longer inherit it. Request - feedback. + Closes #6999 + +- FILEFORMAT: mention sectransp as a feature - Closes https://github.com/curl/curl/pull/6688 + Been supported since at least 40259ca65 + + Closes #7001 -Daniel Stenberg (17 Mar 2021) -- http: make 416 not fail with resume + CURLOPT_FAILONERRROR +- RELEASE-NOTES: synced + +- libssh2: ignore timeout during disconnect - When asked to resume a download, libcurl will convert that to HTTP logic - and if then the entire file is already transferred it will result in a - 416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that - scenario, it should *not* lead to an error return. + ... to avoid memory leaks! - Updated test 1156, added test 1273 + libssh2 is tricky as we have to deal with the non-blockiness even in + close and shutdown cases. In the cases when we shutdown after a timeout + already expired, it is crucial that curl doen't let the timeout abort + the shutdown process as that then leaks memory! - Reported-by: Jonathan Watt - Fixes #6740 - Closes #6753 + Reported-by: Benjamin Riefenstahl + Fixes #6990 -- Curl_timeleft: check both timeouts during connect +- KNOWN_BUGS: add two HTTP/2 bugs + +- KNOWN_BUGS: add three HTTP/3 issues - The duration of a connect and the total transfer are calculated from two - different time-stamps. It can end up with the total timeout triggering - before the connect timeout expires and we should make sure to - acknowledge whichever timeout that is reached first. + ... and moved the HTTP/2 issues to its own section - This is especially notable when a transfer first sits in PENDING, as - that time is counted in the total time but the connect timeout is based - on the time since the handle changed to the CONNECT state. + Closes #6606 + Closes #6510 + Closes #6494 + +- [ejanchivdorj brought this change] + + CURLcode: add CURLE_SSL_CLIENTCERT - The CONNECTTIMEOUT is per connect attempt. The TIMEOUT is for the entire - operation. + When a TLS server requests a client certificate during handshake and + none can be provided, libcurl now returns this new error code + CURLE_SSL_CLIENTCERT - Fixes #6744 - Closes #6745 - Reported-by: Andrei Bica - Assisted-by: Jay Satiro - -- configure: remove use of deprecated macros + Only supported by Secure Transport and OpenSSL for TLS 1.3 so far. - AC_HEADER_TIME, AC_HEADER_STDC and AC_TYPE_SIGNAL + Closes #6721 -- configure: make AC_TRY_* into AC_*_IFELSE - - ... as the former versions are deprecated. +- [Tobias Gabriel brought this change] -- configure: s/AC_HELP_STRING/AS_HELP_STRING - - AC_HELP_STRING is deprecated in 2.70+ and I believe AS_HELP_STRING works - already since 2.59 so bump the minimum required version to that. + .github/FUNDING: add link to GitHub sponsors - Reported-by: Emil Engler - Fixes #6647 - Closes #6748 + Closes #6985 -- RELEASE-NOTES: synced +- [Harry Sintonen brought this change] -- travis: use ubuntu nghttp2 package instead of build our own + krb5/name_to_level: replace checkprefix with curl_strequal - Closes #6751 + Closes #6993 -- travis: bump wolfssl to 4.7.0 +- [Harry Sintonen brought this change] -- travis: only build wolfssl when needed + Curl_input_digest: require space after Digest - Closes #6751 + Closes #6993 -- [Jacob Hoffman-Andrews brought this change] +- [Harry Sintonen brought this change] - rustls: allocate a buffer for TLS data. + Curl_http_header: check for colon when matching Persistent-Auth - Previously, rustls was using an on-stack array for TLS data. However, - crustls has an (unusual) requirement that buffers it deals with are - initialized before writing to them. By using calloc, we can ensure the - buffer is initialized once and then reuse it across calls. + Closes #6993 + +- [Harry Sintonen brought this change] + + Curl_http_input_auth: require valid separator after negotiation type - Closes #6742 + Closes #6993 -- travis: add a rustls build +- http: fix the check for 'Authorization' with Bearer - ... that doesn't run any tests (yet) + The code would wrongly check for it using an additional colon. - Closes #6750 - -- HTTP2: remove the outdated remark about multiplexing for the tool + Reported-by: Blake Burkhart + Closes #6988 -- [Robert Ronto brought this change] +- [Kamil Dudka brought this change] - http2: don't set KEEP_SEND when there's no more data to be sent + http2: fix a resource leak in push_promise() - this should fix an issue where curl sometimes doesn't send out a request - with authorization info after a 401 is received over http2 + ... detected by Coverity: - Closes #6747 + Error: RESOURCE_LEAK (CWE-772): + lib/http2.c:532: alloc_fn: Storage is returned from allocation function "duphandle". + lib/http2.c:532: var_assign: Assigning: "newhandle" = storage returned from "duphandle(data)". + lib/http2.c:552: noescape: Resource "newhandle" is not freed or pointed-to in "set_transfer_url". + lib/http2.c:555: leaked_storage: Variable "newhandle" going out of scope leaks the storage it points to. + + Closes #6986 -Marc Hoersken (15 Mar 2021) -- config: fix building SMB with configure using Win32 Crypto +- [Kamil Dudka brought this change] + + http2: fix resource leaks in set_transfer_url() - Align conditions for NTLM features between CMake and configure - builds by differentiating between USE_NTLM and USE_CURL_NTLM_CORE, - just like curl_setup.h does internally to detect support of: + ... detected by Coverity: - - USE_NTLM: required for NTLM crypto authentication feature - - USE_CURL_NTLM_CORE: required for SMB protocol + Error: RESOURCE_LEAK (CWE-772): + lib/http2.c:480: alloc_fn: Storage is returned from allocation function "curl_url". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:480: var_assign: Assigning: "u" = storage returned from "curl_url()". + lib/http2.c:486: noescape: Resource "u" is not freed or pointed-to in "curl_url_set". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:488: leaked_storage: Variable "u" going out of scope leaks the storage it points to. - Implement USE_WIN32_CRYPTO detection by checking for Crypt functions - in wincrypt.h which are not available in the Windows App environment. + Error: RESOURCE_LEAK (CWE-772): + lib/http2.c:480: alloc_fn: Storage is returned from allocation function "curl_url". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:480: var_assign: Assigning: "u" = storage returned from "curl_url()". + lib/http2.c:493: noescape: Resource "u" is not freed or pointed-to in "curl_url_set". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:495: leaked_storage: Variable "u" going out of scope leaks the storage it points to. - Link advapi32 and crypt32 for Crypto API and Schannel SSL backend. - Fix condition of Schannel SSL backend in CMake build accordingly. + Error: RESOURCE_LEAK (CWE-772): + lib/http2.c:480: alloc_fn: Storage is returned from allocation function "curl_url". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:480: var_assign: Assigning: "u" = storage returned from "curl_url()". + lib/http2.c:500: noescape: Resource "u" is not freed or pointed-to in "curl_url_set". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:502: leaked_storage: Variable "u" going out of scope leaks the storage it points to. - Reviewed-by: Marcel Raad + Error: RESOURCE_LEAK (CWE-772): + lib/http2.c:480: alloc_fn: Storage is returned from allocation function "curl_url". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:480: var_assign: Assigning: "u" = storage returned from "curl_url()". + lib/http2.c:505: noescape: Resource "u" is not freed or pointed-to in "curl_url_get". [Note: The source code implementation of the function has been overridden by a builtin model.] + lib/http2.c:507: leaked_storage: Variable "u" going out of scope leaks the storage it points to. - Closes #6277 + Closes #6986 -- config: fix detection of restricted Windows App environment +- [Jacob Hoffman-Andrews brought this change] + + rustls: use ALPN - Move the detection of the restricted Windows App environment - in curl_setup.h before the definition of USE_WIN32_CRYPTO - via included config-win32.h in case no build system is used. + Update required rustls to 0.5.0 - Reviewed-by: Marcel Raad + Closes #6960 + +- [MAntoniak brought this change] + + gskit: fix CURL_DISABLE_PROXY build - Part of #6277 + Removed localfd and remotefd from ssl_backend_data (ued only with proxy + connection). Function pipe_ssloverssl return always 0, when proxy is not + used. + + Closes #6981 -Daniel Stenberg (15 Mar 2021) -- HISTORY: curl 7.7.2 was the first version used in Mac OS X 10.1 +- [MAntoniak brought this change] -- gen.pl: quote "bare" minuses in the nroff curl.1 + gskit: fix undefined reference to 'conn' - Reported-by: Alejandro Colomar - Fixes #6698 - Closes #6722 + Closes #6980 -Daniel Gustafsson (14 Mar 2021) -- hsts: remove unused defines +- [Jacob Hoffman-Andrews brought this change] + + tls: add USE_HTTP2 define - MAX_HSTS_SUBLEN and MAX_HSTS_SUBLENSTR were unused from the initial commit, - and mostly likely leftovers from early development. Remove as they're not - used for anything. + This abstracts across the two HTTP/2 backends: nghttp2 and Hyper. - Closes #6741 - Reviewed-by: Daniel Stenberg + Add our own define for the "h2" ALPN protocol, so TLS backends can use + it without depending on a specific HTTP backend. + + Closes #6959 -Daniel Stenberg (12 Mar 2021) -- github: add torture-ftp for FTP-only torture testing +- [Jacob Hoffman-Andrews brought this change] + + lib: fix 0-length Curl_client_write calls - and at 20% to try to keep the run-time reasonable + Closes #6954 + +- [Jacob Hoffman-Andrews brought this change] + + lib: remove strlen call from Curl_client_write - Closes #6728 + At all call sites with an explicit 0 len, pass an appropriate nonzero + len. + + Closes #6954 -- travis: split "torture" into a separate "events" build as well +- [Ayushman Singh Chauhan brought this change] + + docs: camelcase it like GitHub everywhere - Run torture without FTP and reducing coverage to 20% + Closes #6979 + +Jay Satiro (27 Apr 2021) +- [Lucas Servén Marín brought this change] + + docs: fix typo in fail-with-body doc - For some reason the torture tests now run a lot slower on travis and run - into the 50 minute limit all the time. + This commit fixes a small typo in the documentation for the + --fail-with-body flag. - Closes #6728 + Closes https://github.com/curl/curl/pull/6977 -- ftp: fix memory leak in ftp_done +- lib: fix some misuse of curlx_convert_UTF8_to_tchar - If after a transfer is complete Curl_GetFTPResponse() returns an error, - curl would not free the ftp->pathalloc block. + curlx_convert_UTF8_to_tchar must be freed by curlx_unicodefree, but + prior to this change some uses mistakenly called free. - Found by torture-testing test 576 + I've reviewed all other uses of curlx_convert_UTF8_to_tchar and + curlx_convert_tchar_to_UTF8. - Closes #6737 - -- [oxalica brought this change] + Bug: https://github.com/curl/curl/pull/6602#issuecomment-825236763 + Reported-by: sergio-nsk@users.noreply.github.com + + Closes https://github.com/curl/curl/pull/6938 - http2: fail if connection terminated without END_STREAM +Daniel Stenberg (27 Apr 2021) +- ntlm: precaution against super huge type2 offsets - Closes #6736 + ... which otherwise caused an integer overflow and circumvented the if() + conditional size check. + + Detected by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=33720 + Assisted-by: Max Dymond + Closes #6975 -- RELEASE-NOTES: synced +- c-hyper: fix unused variable ‘wrote’ -- [Jacob Hoffman-Andrews brought this change] +- libcurl-security.3: be careful of setuid + + Reported-by: Harry Sintonen + Closes #6970 - rustls: support CURLOPT_SSL_VERIFYPEER +- [Kevin Burke brought this change] + + c-hyper: don't write to set.writeheader if null - This requires the latest main branch of crustls, which provides - rustls_client_config_builder_dangerous_set_certificate_verifier and - rustls_client_config_builder_set_enable_sni. + Previously if a caller set CURLOPT_WRITEFUNCTION but did not set a + CURLOPT_HEADERDATA buffer, Hyper would still attempt to write headers to + the data->set.writeheader header buffer, even though it is null. This + led to NPE segfaults attempting to use libcurl+Hyper with Git, for + example. - This refactors the session setup into its own function, and adds a new - function cr_hostname_is_ip. Because crustls doesn't support verification - of IP addresses, special handling is needed: We disable SNI and set a - placeholder hostname (which never actually gets sent on the wire). + Instead, process the client write for the status line using the same + logic we use to process the client write for the later HTTP headers, + which contains the appropriate guard logic. As a side benefit, + data->set.writeheader is now only read in one file instead of two. - Closes #6719 + Fixes #6619 + Fixes abetterinternet/crustls#49 + Fixes hyperium/hyper#2438 + Closes #6971 -Daniel Gustafsson (12 Mar 2021) -- cookies: Fix potential NULL pointer deref with PSL +- wolfssl: handle SSL_write() returns 0 for error - Curl_cookie_init can be called with data being NULL, and this can in turn - be passed to Curl_cookie_add, meaning that both functions must be careful - to only use data where it's checked for being a NULL pointer. The libpsl - support code does however dereference data without checking, so if we are - indeed having an unset data pointer we cannot PSL check the cookiedomain. + Reported-by: Timo Lange - This is currently not a reachable dereference, as the only caller with a - NULL data isn't passing a file to initialize cookies from, but since the - API has this contract let's ensure we hold it. + Closes #6967 + +- easy: ignore sigpipe in curl_easy_send - Closes #6731 - Reviewed-by: Daniel Stenberg + Closes #6965 -Daniel Stenberg (12 Mar 2021) -- [Michael Hordijk brought this change] +- sigpipe: ignore SIGPIPE when using wolfSSL as well + + Closes #6966 - configure: only add OpenSSL paths if they are defined +- libcurl-security.3: don't try to filter IPv4 hosts based on the URL - Add paths for OpenSSL compiling and linking only if they have been - defined. If they haven't been defined, we'll assume that the paths are - already available to the toolchain. + Closes #6942 + +- [Harry Sintonen brought this change] + + nss_set_blocking: avoid static for sock_opt - Closes #6730 + Reviewed-by: Kamil Dudka + Closes #6945 -Jay Satiro (12 Mar 2021) -- retry.d: Clarify transient 5xx HTTP response codes +- RELEASE-NOTES: synced + +- [Yusuke Nakamura brought this change] + + docs/HTTP3.md: fix nghttp2's HTTP/3 server port - - Clarify the only 5xx response codes that are treated as transient are - 500, 502, 503 and 504. + Port 8443 does not work now. + Correct origin is in the quicwg's wiki. + https://github.com/quicwg/base-drafts/wiki/Implementations#ngtcp2 - Prior to this change it said it treated all 5xx as transient, but the - code says otherwise. + Closes #6964 + +- krb5: don't use 'static' to store PBSZ size response - Ref: https://github.com/curl/curl/blob/curl-7_75_0/src/tool_operate.c#L462-L495 + ... because it makes the knowledge and usage cross-transfer in funny and + unexpected ways. - Closes https://github.com/curl/curl/pull/6724 + Reported-by: Harry Sintonen + Closes #6963 -- retry-all-errors.d: Explain curl errors versus HTTP response errors +- [Kevin Burke brought this change] + + m4: add security frameworks on Mac when compiling rustls - - Add a paragraph explaining that curl does not consider HTTP response - errors as curl errors, and how that behavior can be modified by using - --retry and --fail. + Previously compiling rustls on Mac would only complete if you also + compiled the SecureTransport TLS backend, which curl would prefer to + the Rust backend. - The --retry-all-errors doc says "Retry on any error" which some users - may find misleading without the added explanation. + Appending these flags to LDFLAGS makes it possible to compile the + Rustls backend on Mac without the SecureTransport backend, which means + this patch will make it possible for Mac users to use the Rustls + backend for TLS. - Ref: https://curl.se/docs/faq.html#Why_do_I_get_downloaded_data_eve - Ref: https://curl.se/docs/faq.html#curl_doesn_t_return_error_for_HT + Reviewed-by: Jacob Hoffman-Andrews - Reported-by: Lawrence Gripper + Fixes #6955 + Cloes #6956 + +- krb5: remove the unused 'overhead' function - Fixes https://github.com/curl/curl/issues/6712 - Closes https://github.com/curl/curl/pull/6720 + Closes #6947 -Daniel Stenberg (11 Mar 2021) -- travis: switch ngtcp2 build over to quictls +- [Johann150 brought this change] + + curl_url_set.3: add memory management information - The ngtcp2 project switched over to using the quictls OpenSSL fork - instead of their own patched OpenSSL. We follow suit. + wording taken from man page for CURLOPT_URL.3 - Closes #6729 + As far as I can see, the URL part is either malloc'ed before due to + encoding or it is strdup'ed. + + Closes #6953 -- test220/314: adjust to run with Hyper +- [Jacob Hoffman-Andrews brought this change] -- c-hyper: support automatic content-encoding + c-hpyer: fix handling of zero-byte chunk from hyper - Closes #6727 + Closes #6951 -- http: remove superfluous NULL assign +- CURLOPT_POSTFIELDS.3: clarify how it gets the size of the data - Closes #6727 + Ref: https://curl.se/mail/lib-2021-04/0085.html + Closes #6943 -- tool_operate: bail if set CURLOPT_HTTP09_ALLOWED returns error +- [Ralph Langendam brought this change] + + cmake: make libcurl output filename configurable - Closes #6727 + Reviewed-by: Jakub Zakrzewski + Closes #6933 -- setopt: error on CURLOPT_HTTP09_ALLOWED set true with Hyper +- [Patrick Monnerat brought this change] + + vtls: reset ssl use flag upon negotiation failure - Not supported. + Fixes the segfault in ldaps disconnect. - Closes #6727 + Reported-by: Illarion Taev + Fixes #6934 + Closes #6937 -- test306: make it not run with Hyper +- configure: fix typo in TLS error message - ... as it tests HTTP/0.9 which Hyper doesn't support. + Reported-by: Pontus Lundkvist -- test304: header CRLF cleanup to work with Hyper +- README: link to the commercial support option -- FTP: allow SIZE to fail when doing (resumed) upload +Jay Satiro (22 Apr 2021) +- [Martin Halle brought this change] + + version: add gsasl_version to curl_version_info_data - Added test 362 to verify. + - Add gsasl_version string and bump to CURLVERSION_TENTH. - Reported-by: Jordan Brown - Regression since 7ea2e1d0c5a7f (7.73.0) - Fixes #6715 - Closes #6725 + Ref: https://curl.se/mail/lib-2021-04/0003.html + + Closes https://github.com/curl/curl/pull/6843 -- configure: provide Largefile feature for curl-config +- [Morten Minde Neergaard brought this change] + + schannel: Support strong crypto option - ... as cmake now does it correctly, and make test1014 check for it + - Support enabling strong crypto via optional user cipher list when + USE_STRONG_CRYPTO or SCH_USE_STRONG_CRYPTO is in the list. - Closes #6702 - -- config: remove CURL_SIZEOF_CURL_OFF_T use only SIZEOF_CURL_OFF_T + MSDN says SCH_USE_STRONG_CRYPTO "Instructs Schannel to disable known + weak cryptographic algorithms, cipher suites, and SSL/TLS protocol + versions that may be otherwise enabled for better interoperability." - Make the code consistently use a single name for the size of the - "curl_off_t" type. + Ref: https://curl.se/mail/lib-2021-02/0066.html + Ref: https://curl.se/docs/manpage.html#--ciphers + Ref: https://curl.se/libcurl/c/CURLOPT_SSL_CIPHER_LIST.html + Ref: https://docs.microsoft.com/en-us/windows/win32/api/schannel/ns-schannel-schannel_cred - Closes #6702 + Closes https://github.com/curl/curl/pull/6734 -Jay Satiro (10 Mar 2021) -- [Jun-ya Kato brought this change] +Daniel Stenberg (22 Apr 2021) +- RELEASE-NOTES: synced - ngtcp2: Fix build error due to change in ngtcp2_addr_init - - ngtcp2/ngtcp2@b8d90a9 changed the function prototype. - - Closes https://github.com/curl/curl/pull/6716 +- ci: adapt to configure requiring an explicit TLS choice -Daniel Stenberg (10 Mar 2021) -- [ejanchivdorj brought this change] +- configure: split out each TLS library detector into its own function + + ... and put those functions in separate m4 files per TLS library. - multi: update pending list when removing handle +- configure: make the TLS library choice(s) explicit - when removing a handle, most of the lists are updated but pending list - is not updated. Updating now. + configure no longer tries to find a TLS library by default, but all + libraries are now equal: the user needs to explicitly ask what TLS + library or libraries to use. - Closes #6713 + If no TLS library is selected, configure will error out unless + --without-ssl is explicitly used to request a built without TLS (as that + is very rare these days). + + Removes: --with-winssl, --with-darwinssl and all --without-* options for + TLS libraries. + + Closes #6897 -- [kokke brought this change] +- tests/disable-scan.pl: also scan all m4 files + + Fixes test 1165 when functions are moved from configure.ac to files in + m4/ - lib1536: check ptr against NULL before dereferencing it +Jay Satiro (22 Apr 2021) +- schannel: Disable auto credentials; add an option to enable it - Closes #6710 + - Disable auto credentials by default. This is a breaking change + for clients that are using it, wittingly or not. + + - New libcurl ssl option value CURLSSLOPT_AUTO_CLIENT_CERT tells libcurl + to automatically locate and use a client certificate for + authentication, when requested by the server. + + - New curl tool options --ssl-auto-client-cert and + --proxy-ssl-auto-client-cert map to CURLSSLOPT_AUTO_CLIENT_CERT. + + This option is only supported for Schannel (the native Windows SSL + library). Prior to this change Schannel would, with no notification to + the client, attempt to locate a client certificate and send it to the + server, when requested by the server. Since the server can request any + certificate that supports client authentication in the OS certificate + store it could be a privacy violation and unexpected. + + Fixes https://github.com/curl/curl/issues/2262 + Reported-by: Jeroen Ooms + Assisted-by: Wes Hinsley + Assisted-by: Rich FitzJohn + + Ref: https://curl.se/mail/lib-2021-02/0066.html + Reported-by: Morten Minde Neergaard + + Closes https://github.com/curl/curl/pull/6673 -- [kokke brought this change] +Daniel Stenberg (22 Apr 2021) +- [Michał Antoniak brought this change] - lib1537: check ptr against NULL before dereferencing it + vtls: deduplicate some DISABLE_PROXY ifdefs - Fixes #6707 - Closes #6708 - -- travis: make torture tests skip TLS-SRP tests + continue from #5735 - ... as it seems to often hang. + - using SSL_HOST_NAME, SSL_HOST_DISPNAME, SSL_PINNED_PUB_KEY for other + tls backend - Also: skip the "normal" tests as they're already run by many other - builds. + - create SSL_HOST_PORT - Closes #6705 + Closes #6660 -- openssl: adapt to v3's new const for a few API calls +Jay Satiro (22 Apr 2021) +- OS400: fix typo - Closes #6703 + CURLVERSION_HEIGHTH -> CURLVERSION_EIGHTH -- quiche: fix crash when failing to connect +Daniel Stenberg (22 Apr 2021) +- checksrc: complain on == NULL or != 0 checks in conditions - Reported-by: ウさん - Fixes #6664 - Closes #6701 + ... to make them all consistenly use if(!var) and if(var) + + Also added a few missing warnings to the documentation. + + Closes #6912 -- RELEASE-NOTES: synced +- tidy-up: make conditional checks more consistent - Fixed the release counter and added a missing contributor + ... remove '== NULL' and '!= 0' + + Closes #6912 -- RELEASE-NOTES: synced +- [Patrick Monnerat brought this change] -- dynbuf: bump the max HTTP request to 1MB + vauth: factor base64 conversions out of authentication procedures - Raised from 128KB to allow longer request headers. + Input challenges and returned messages are now in binary. + Conversions from/to base64 are performed by callers (currently curl_sasl.c + and http_ntlm.c). - Reported-by: Carl Zogheib - Fixes #6681 - Closes #6685 + Closes #6654 -Jay Satiro (6 Mar 2021) -- schannel: Evaluate CURLOPT_SSL_OPTIONS via SSL_SET_OPTION macro +- [Patrick Monnerat brought this change] + + bufref: buffer reference support - - Change use of those options from CURLOPT_SSL_OPTIONS that are not - already evaluated via SSL_SET_OPTION in schannel and secure transport - to use that instead of data->set.ssl.optname. + A struct bufref holds a buffer pointer, a data size and a destructor. + When freed or its contents are changed, the previous buffer is implicitly + released by the associated destructor. The data size, although not used + internally, allows binary data support. - Example: + A unit test checks its handling methods: test 1661 - Evaluate SSL_SET_OPTION(no_revoke) instead of data->set.ssl.no_revoke. + Closes #6654 + +- [Patrick Monnerat brought this change] + + os400: additional support for options metadata - This change is because options set via CURLOPT_SSL_OPTIONS - (data->set.ssl.optname) are separate from those set for HTTPS proxy via - CURLOPT_PROXY_SSL_OPTIONS (data->set.proxy_ssl.optname). The - SSL_SET_OPTION macro determines whether the connection is for HTTPS - proxy and based on that which option to evaluate. + New functions curl_easy_option_by_name_ccsid() and + curl_easy_option_get_name_ccsid() allows accessing metadata in alternate + character encoding. - Since neither Schannel nor Secure Transport backends currently support - HTTPS proxy in libcurl, this change is for posterity and has no other - effect. + This commit also updates curl_version_info_ccsid() to handle info version 9 + and adds recent definitions to the ILE/RPG include file. - Closes https://github.com/curl/curl/pull/6690 + Documentation updated accordingly. + + Reviewed-by: Jon Rumsey + Closes #6574 -- [kokke brought this change] +- [Patrick Monnerat brought this change] - c-hyper: Remove superfluous pointer check + test server: take care of siginterrupt() deprecation - `n` pointer is never NULL once set. Found by static analysis. + Closes #6529 + +Marc Hoersken (21 Apr 2021) +- lib1564.c: enable last wakeup test part on Windows - Ref: https://github.com/curl/curl/issues/6696 + Suggested-by: Gergely Nagy + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad - Closes https://github.com/curl/curl/pull/6697 + Closes #6245 -- version.d: Add missing features to the features list +- multi: fix slow write/upload performance on Windows - - Add missing entries for gsasl, Kerberos, NTLM_WB, TrackMemory, - Unicode and zstd. + Reset FD_WRITE by sending zero bytes which is permissible + and will be treated by implementations as successful send. - - Remove krb4 since it's no longer a feature. + Without this we won't be notified in case a socket is still + writable if we already received such a notification and did + not send any data afterwards on the socket. This would lead + to waiting forever on a writable socket being writable again. - Reported-by: Ádler Jonas Gross + Assisted-by: Tommy Odom + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + Tested-by: tmkk on github - Fixes https://github.com/curl/curl/issues/6677 - Closes https://github.com/curl/curl/pull/6687 - -- [Vladimir Varlamov brought this change] + Bug: #6146 + Closes #6245 - docs: add missing Arg tag to --stderr +- multi: reduce Win32 API calls to improve performance - Prior to this change the required argument was not shown. + 1. Consolidate pre-checks into a single Curl_poll call: - curl.1 before: --stderr - curl.1 after: --stderr + This is an attempt to restructure the code in Curl_multi_wait + in such a way that less syscalls are made by removing individual + calls to Curl_socket_check via SOCKET_READABLE/SOCKET_WRITABLE. - curl --help before: - --stderr Where to redirect stderr + 2. Avoid resetting the WinSock event multiple times: - curl --help after: - --stderr Where to redirect stderr + We finally call WSAResetEvent anyway, so specifying it as + an optional parameter to WSAEnumNetworkEvents is redundant. - Closes https://github.com/curl/curl/pull/6692 + 3. Wakeup directly in case no sockets are being monitoring: + + Fix the WinSock based implementation to skip extra waiting by + not sleeping in case no sockets are to be waited on and just + the WinSock event is being monitored for wakeup functionality. + + Assisted-by: Tommy Odom + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + + Bug: #6146 + Closes #6245 -- projects: Update VS projects for OpenSSL 1.1.x +- Revert "Revert 'multi: implement wait using winsock events'" - - Update VS project templates to use the OpenSSL lib names and include - directories for OpenSSL 1.1.x. + This reverts commit 2260e0ebe6d45529495231b3e37a0c58fb92a6a2, + also restoring previous follow up changes which were reverted. - This change means the VS project files will now build only with OpenSSL - 1.1.x when an OpenSSL configuration is chosen. Prior to this change the - project files built only with OpenSSL 1.0.x (end-of-life) when an - OpenSSL configuration was chosen. + Authored-by: rcombs on github + Authored-by: Marc Hörsken + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad - The template changes in this commit were made by script: + Restores #5634 + Reverts #6281 + Part of #6245 + +Daniel Stenberg (21 Apr 2021) +- Revert "cmake: make libcurl library output name configurable" - libeay32.lib => libcrypto.lib - ssleay32.lib => libssl.lib - ..\..\..\..\..\openssl\inc32 => ..\..\..\..\..\openssl\include + This reverts commit 1cba36d2166c396f987eea587cf92671b27acb92. - And since the output directory now contains the includes it's prepended: - ..\..\..\..\..\openssl\build\Win{32,64}\VC{6..15}\{DLL,LIB} - {Debug,Release}\include + CMake provides properties that can be set on a target to rename the + output artifact without changing the name of a target. - - Change build-openssl.bat to copy the build's include directory to the - output directory (as seen above). + Ref: #6899 + +- [Michael Kolechkin brought this change] + + sectransp: allow cipher name to be specified - Each build has its own opensslconf.h which is different so we can't just - include the source include directory any longer. + Add parser for CURLOPT_SSL_CIPHER_LIST option for Secure Transport (ST) + back-end. Similar to NSS and GSKit back-ends, new code parses string + value and configures ST library to use those ciphers for communication. + Create cipher spec data structure and initialize the array of specs with + cipher number, name, alias, and 'weak' flag. - Note the include directory in the output directory is a full copy from - the build so technically we don't need to include the OpenSSL source - include directory in the template. However, I left it last in case the - user made a custom OpenSSL build using the old method which would put - opensslconf in the OpenSSL source include directory. + Mark triple-DES ciphers as 'weak', and exclude them from the default + ciphers list. - - Change build-openssl.bat to use a temporary install directory that is - different from the temporary build directory. + Closes #6464 + +- [Michael Kolechkin brought this change] + + NSS: add ciphers to map - For OpenSSL 1.1.x the temporary paths must be separate not a descendant - of the other, otherwise pdb files will be lost between builds. + Add cipher names to the `cipherlist` map, based on the list of ciphers + implemented by the NSS in the source code file + https://github.com/nss-dev/nss/blob/master/lib/ssl/sslenum.c - Ref: https://curl.se/mail/lib-2018-10/0049.html - Ref: https://gist.github.com/jay/125191c35bbeb894444eff827651f755 - Ref; https://github.com/openssl/openssl/issues/10005 + Closes #6670 + +- http2: remove DEBUG_HTTP2 - Fixes https://github.com/curl/curl/issues/984 - Closes https://github.com/curl/curl/pull/6675 + Accidentally committed in 605e84235 -- doh: Inherit CURLOPT_STDERR from user's easy handle +- [Ralph Langendam brought this change] + + cmake: make libcurl library output name configurable - Prior to this change if the user set their easy handle's error stream - to something other than stderr it was not inherited by the doh handles, - which meant that they would still write to the default standard error - stream (stderr) for verbose output. + Closes #6899 + +- sws: #ifdef S_IFSOCK use - Bug: https://github.com/curl/curl/issues/6605 - Reported-by: arvids-kokins-bidstack@users.noreply.github.com + SCO OpenServer 5.0.7 does not define S_IFSOCK. - Closes https://github.com/curl/curl/pull/6661 + Reported-by: Kevin R. Bulgrien + Bug: https://curl.se/mail/lib-2021-04/0074.html + Closes #6926 -Marc Hoersken (1 Mar 2021) -- CI/azure: replace python-impacket with python3-impacket +- curl_setup: provide the shutdown flags wider - As of this month Azure DevOps uses Ubuntu 20.04 LTS which - no longer supports Python 2 and instead ships Python 3. + By using #ifdef on the symbol names to work on anything that don't + provide them. SCO OpenServer 5.0.7, sys/socket.h does not define either + SHUT_RDWR, SHUT_RD, and SHUT_WR. - Closes #6678 + Reported-by: Kevin R. Bulgrien + Bug: https://curl.se/mail/lib-2021-04/0073.html + Closes #6925 -- runtests.pl: kill processes locking test log files +- connect: use CURL_SA_FAMILY_T for portability - Introduce a new runtests.pl command option: -rm + Reported-by: Kevin R. Bulgrien + Bug: https://curl.se/mail/lib-2021-04/0071.html - For now only required and implemented for Windows. - Ignore stunnel logs due to long running processes. + Closes #6918 + +- urlapi: make sure no +/- signs are accepted in IPv4 numericals - Requires Sysinternals handle[64].exe to be on PATH. + Follow-up to 56a037cc0ad1b2. Extends test 1560 to verify. - Reviewed-by: Jay Satiro + Reported-by: Tuomas Siipola + Fixes #6916 + Closes #6917 + +- ConnectionExists: respect requests for h1 connections better - Ref: #6058 - Closes #6179 + ... for situations when multiplexing isn't enabled on the h2 connection + and h1 is explicitly requested for the transfer. + + Assisted-by: Gergely Nagy -- pathhelp.pm: fix use of pwd -L in Msys environment +- multi: don't close connection HTTP_1_1_REQUIRED - While Msys2 has a pwd binary which supports -L, - Msys1 only has a shell built-in with that feature. + The ConnectionExists() function will note that the new transfer wants + less then h2 and that it can't multiplex it and therefor opt to open a + new connection instead. + +- http2: move the stream error field to the per-transfer storage - Reviewed-by: Jay Satiro + Storing a stream error in the per-connection struct was an error that lead to + race conditions as subsequent stream handling could overwrite the error code + before it was used for the stream with the actual problem. - Part of #6179 + Closes #6910 -Daniel Gustafsson (1 Mar 2021) -- ldap: use correct memory free function +- http2: call the handle-closed function correctly on closed stream - unescaped is coming from Curl_urldecode and not a unicode conversion - function, so reclaiming its memory should be performed with a normal - call to free rather than curlx_unicodefree. In reality, this is the - same thing as curlx_unicodefree is implemented as a call to free but - that's not guaranteed to always hold. Using the curlx macro present - issues with memory debugging as well. + This was this one condition where the stream could be closed due to an + error and the function would still wrongly just return 0 for it. - Closes #6671 - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Reported-by: Gergely Nagy + Fixes #6862 + Closes #6910 -- url: fix typo in comment +- test1660: check the created HSTS file as text mode - Correct a small typo which snuck in with a304051620. + Closes #6922 -Jay Satiro (28 Feb 2021) -- tool_help: Increase space between option and description +- RELEASE-NOTES: synced + +- test 493: require https in curl to run - - Increase the minimum number of spaces between the option and the - description from 1 to 2. + Closes #6927 + +Jay Satiro (20 Apr 2021) +- tool_operate: don't discard failed parallel transfer result + + - Save a parallel transfer's result code only when it fails and the + transfer is not being retried. + + Prior to this change the result code was always set which meant that a + failed result could be erroneously discarded if a different transfer + later had a successful result (CURLE_OK). Before: - ~~~ - -u, --user Server user and password - -A, --user-agent Send User-Agent to server - -v, --verbose Make the operation more talkative - -V, --version Show version number and quit - -w, --write-out Use output FORMAT after completion - --xattr Store metadata in extended file attributes - ~~~ - After: - ~~~ - -u, --user Server user and password - -A, --user-agent Send User-Agent to server - -v, --verbose Make the operation more talkative - -V, --version Show version number and quit - -w, --write-out Use output FORMAT after completion - --xattr Store metadata in extended file attributes - ~~~ + > curl --fail -Z https://httpbin.org/status/404 https://httpbin.org/delay/10 + > echo %ERRORLEVEL% + 0 - Closes https://github.com/curl/curl/pull/6674 - -Daniel Stenberg (27 Feb 2021) -- curl: set CURLOPT_NEW_FILE_PERMS if requested + After: - The --create-file-mode code logic accepted the value but never actually - passed it on to libcurl! + > curl --fail -Z https://httpbin.org/status/404 https://httpbin.org/delay/10 + > echo %ERRORLEVEL% + 22 - Follow-up to a7696c73436f (shipped in 7.75.0) - Reported-by: Johannes Lesr - Fixes #6657 - Closes #6666 + Closes #xxxx -- tool_operate: check argc before accessing argv[1] +- [Georeth Zhou brought this change] + + openssl: fix build error with OpenSSL < 1.0.2 - Follow-up to 09363500b - Reported-by: Emil Engler - Reviewed-by: Daniel Gustafsson - Closes #6668 + Closes https://github.com/curl/curl/pull/6920 -Daniel Gustafsson (26 Feb 2021) -- [Jean-Philippe Menil brought this change] +Viktor Szakats (19 Apr 2021) +- README.md: delete Codacy UTM parameters & follow permanent redirect [ci skip] + + UTM parameters leak referrer and various marketing/tracking information + even if these would normally be stripped by website or client policy. + This link also works fine without them. Also took the opportunity to + update the URL to the one pointed to by the previous one via permanent + redirect. + + Reviewed-by: Daniel Stenberg + Closes #6919 - openssl: remove get_ssl_version_txt in favor of SSL_get_version +Daniel Stenberg (19 Apr 2021) +- urlapi: "normalize" numerical IPv4 host names - openssl: use SSL_get_version to get connection protocol + When the host name in a URL is given as an IPv4 numerical address, the + address can be specified with dotted numericals in four different ways: + a32, a.b24, a.b.c16 or a.b.c.d and each part can be specified in + decimal, octal (0-prefixed) or hexadecimal (0x-prefixed). - Replace our bespoke get_ssl_version_txt in favor of SSL_get_version. - We can get rid of few lines of code, since SSL_get_version achieve - the exact same thing + Instead of passing on the name as-is and leaving the handling to the + underlying name functions, which made them not work with c-ares but work + with getaddrinfo, this change now makes the curl URL API itself detect + and "normalize" host names specified as IPv4 numericals. - Closes #6665 - Reviewed-by: Daniel Gustafsson - Signed-off-by: Jean-Philippe Menil - -- gnutls: Fix nettle discovery + The WHATWG URL Spec says this is an okay way to specify a host name in a + URL. RFC 3896 does not allow them, but curl didn't prevent them before + and it seems other RFC 3896-using tools have not either. Host names used + like this are widely supported by other tools as well due to the + handling being done by getaddrinfo and friends. - Commit e06fa7462ac258c removed support for libgcrypt leaving only - support for nettle which has been the default crypto library in - GnuTLS for a long time. There were however a few conditionals on - USE_GNUTLS_NETTLE which cause compilation errors in the metalink - code (as it used the gcrypt fallback instead as a result). See the - below autobuild for an example of the error: + I decided to add the functionality into the URL API itself so that all + users of these functions get the benefits, when for example wanting to + compare two URLs. Also, it makes curl built to use c-ares now support + them as well and make curl builds more consistent. - https://curl.se/dev/log.cgi?id=20210225123226-30704#prob1 + The normalization makes HTTPS and virtual hosted HTTP work fine even + when curl gets the address specified using one of the "obscure" formats. - This removes all uses of USE_GNUTLS_NETTLE and also removes the - gcrypt support from the metalink code while at it. + Test 1560 is extended to verify. - Closes #6656 - Reviewed-by: Daniel Stenberg + Fixes #6863 + Closes #6871 -- cookies: Support multiple -b parameters +- libssh: fix "empty expression statement has no effect" warnings - Previously only a single -b cookie parameter was supported with the last - one winning. This adds support for supplying multiple -b params to have - them serialized semicolon separated. Both cookiefiles and cookies can be - entered multiple times. + ... by fixing macros to do-while constructs and moving out the calls to + "break" outside of the actual macro. It also fixes the problem where the + macro was used witin a loop and the break didn't do right. - Closes #6649 - Reviewed-by: Daniel Stenberg + Reported-by: Emil Engler + Fixes #6847 + Closes #6909 -Daniel Stenberg (25 Feb 2021) -- build: remove all traces of USE_BLOCKING_SOCKETS - - libcurl doesn't behave properly with the define set +- hsts: enable by default - Closes #6655 - -- RELEASE-NOTES: synced - -Daniel Gustafsson (25 Feb 2021) -- docs: Fix typos + No longer considered experimental. - Random typos spotted when skimming docs. + Closes #6700 -- cookies: Use named parameters in header prototypes +- vtls: refuse setting any SSL version - Align header with project style of using named parameters in the - function prototypes to aid readability and self-documentation. + ... previously they were supported if a TLS library would (unexpectedly) + still support them, but from this change they will be refused already in + curl_easy_setopt(). SSLv2 and SSLv3 have been known to be insecure for + many years now. - Closes #6653 - Reviewed-by: Daniel Stenberg + Closes #6773 -Daniel Stenberg (24 Feb 2021) -- urldata: make 'actions[]' use unsigned char instead of int +- curl: ignore options asking for SSLv2 or SSLv3 - ... as it only needs a few bits per index anyway. + Instead output a warning about it and continue with the defaults. - Reviewed-by: Daniel Gustafsson - Closes #6648 - -- configure: fail if --with-quiche is used and quiche isn't found + These SSL versions are typically not supported by the TLS libraries since a + long time back already since they are inherently insecure and broken. Asking + for them to be used will just cause an error to be returned slightly later. - Closes #6652 - -- [Gregor Jasny brought this change] - - cmake: use CMAKE_INSTALL_INCLUDEDIR indirection + In the unlikely event that a user's TLS library actually still supports these + protocol versions, this change might make the request a little less insecure. - Reviewed-by: Sergei Nikulov - Closes #6440 + Closes #6772 -Viktor Szakats (23 Feb 2021) -- mingw: enable using strcasecmp() +- test972: verify the json output with jsonlint - This makes the 'Features:' list sorted case-insensitively, - bringing output in-line with *nix builds. + Make sure one of the azure jobs has jsonlint installed so that the test + runs there. - Reviewed-by: Jay Satiro - Closes #6644 + Ref: #6905 -- build: delete unused feature guards +- [Jay Satiro brought this change] + + tool_writeout: fix the HTTP_CODE json output - - `HAVE_STRNCASECMP` - - `HAVE_TCGETATTR` - - `HAVE_TCSETATTR` + Update test 970 accordingly. - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes #6645 + Reported-by: Michal Rus + Fixes #6905 + Closes #6906 -Jay Satiro (23 Feb 2021) -- docs: add CURLOPT_CURLU to 'See also' in curl_url_ functions +- openldap: protect SSL-specific code with proper #ifdef - Closes https://github.com/curl/curl/pull/6639 + Closes #6901 -Daniel Stenberg (23 Feb 2021) -- [Jacob Hoffman-Andrews brought this change] - - configure: make hyper opt-in, and fail if missing +- libssh2: fix Value stored to 'sshp' is never read - Previously, configure would look for hyper by default, and use it if - found; otherwise it would not use hyper, and not error. + Pointed out by scan-build - Now, configure will not look for hyper unless --with-hyper is passed. If - configure looks for hyper and fails, it will error. + Closes #6900 + +- [Victor Vieux brought this change] + + tool_getparam: replace (in-place) '%20' by '+' according to RFC1866 - Also, add -ld -lpthread -lm to Hyper's libs. I think they are required. + Signed-off-by: Victor Vieux - Closes #6598 + Closes #6895 -- multi: do once-per-transfer inits in before_perform in DID state - - ... since the state machine might go to RATELIMITING and then back to - PERFORMING doing once-per-transfer inits in that function is wrong and - it caused problems with receiving chunked HTTP and it set the - PRETRANSFER time much too often... +- configure: provide --with-openssl, deprecate --with-ssl - Regression from b68dc34af341805aeb7b3715 (shipped in 7.75.0) + Makes the option more explicit. - Reported-by: Amaury Denoyelle - Fixes #6640 - Closes #6641 + Closes #6887 - RELEASE-NOTES: synced - -- CODE_STYLE.md: fix broken link to INTERNALS - ... the link would only work if browsed on GitHub, while this link now - takes the user to the website instead and thus should work on either. - - Reported-by: David Demelier + and bumped curlver to 7.77.0 -- curl_url_set.3: mention CURLU_PATH_AS_IS +- [Javier Blazquez brought this change] + + rustls: only return CURLE_AGAIN when TLS session is fully drained - ... it has been supported since the URL API was added. + The code in cr_recv was returning prematurely as soon as the socket + reported no more data to read. However, this could be leaving some + unread plaintext data in the rustls session from a previous call, + causing causing the transfer to hang if the socket never receives + further data. - Bug: https://curl.se/mail/lib-2021-02/0046.html + We need to ensure that the session is fully drained of plaintext data + before returning CURLE_AGAIN to the caller. - Closes #6638 + Reviewed-by: Jacob Hoffman-Andrews + Closes #6894 -Viktor Szakats (21 Feb 2021) -- time: enable 64-bit time_t in supported mingw environments +- cookie: CURLOPT_COOKIEFILE set to NULL switches off cookies - (Unless 32-bit `time_t` is selected manually via the `_USE_32BIT_TIME_T` - mingw macro.) - - Previously, 64-bit `time_t` was enabled on VS2005 and newer only, and - 32-bit `time_t` was used on all other Windows builds. + Add test 676 to verify that setting CURLOPT_COOKIEFILE to NULL again clears + the cookiejar from memory. - Assisted-by: Jay Satiro - Closes #6636 + Reported-by: Stefan Karpinski + Fixes #6889 + Closes #6891 -Jay Satiro (20 Feb 2021) -- test1188: Check for --fail HTTP status +Version 7.76.1 (14 Apr 2021) + +Daniel Stenberg (14 Apr 2021) +- RELEASE-NOTES: synced - - Change the test to check for curl error on HTTP 404 Not Found. + curl 7.76.1 release + +- THANKS: add names from 7.76.1 + +- misc: update copyright year ranges to match latest updates + +- [Tatsuhiro Tsujikawa brought this change] + + ngtcp2: Use ALPN h3-29 for now - test1188 tests "--write-out with %{onerror} and %{urlnum} to stderr". - Prior to this change it did that by specifying a non-existent host which - would cause an error. ISPs may hijack DNS and resolve non-existent hosts - so the test would not work if that was the case. + Fixes #6864 + Cloes #6886 + +Jay Satiro (11 Apr 2021) +- TODO: remove 18.22 --fail-with-body - Ref: https://en.wikipedia.org/wiki/DNS_hijacking#Manipulation_by_ISPs - Ref: https://github.com/curl/curl/issues/6621 - Ref: https://github.com/curl/curl/pull/6623 + --fail-with-body was added in 8a964cb (precedes curl-7_76_0). + +Daniel Stenberg (10 Apr 2021) +- [Jürgen Gmach brought this change] + + src/tool_vms.c: remove duplicated word in comment - Closes https://github.com/curl/curl/pull/6637 + Closes #6881 -- memdebug: close debug logfile explicitly on exit +- configure: fix CURL_DARWIN_CFLAGS use - - Use atexit to register a dbg cleanup function that closes the logfile. + The macro name change was not completely done. - LeakSantizier (LSAN) calls _exit() instead of exit() when a leak is - detected on exit so the logfile must be closed explicitly or data could - be lost. Though _exit() does not call atexit handlers such as this, - LSAN's call to _exit() comes after the atexit handlers are called. + Follow-up to 5d2c384452543c + Bug: https://github.com/curl/curl/commit/5d2c384452543c7b6c9fb02eaa0afc84fd5ab941#commitcomment-49315187 + Reported-by: Marcel Raad + Closes #6878 + +- [Anthony Shaw brought this change] + + github/workflow: add "security-extended" to codeql-analysis.yml - Prior to this change the logfile was not explicitly closed so it was - possible that if LSAN detected a leak and called _exit (which does - not flush or close files like exit) then the logfile could be missing - data. That could then cause curl's memanalyze to report false leaks - (eg a malloc was recorded to the logfile but the corresponding free was - discarded from the buffer instead of written to the logfile, then - memanalyze reports that as a leak). + Extends the CodeQL code scan. - Ref: https://github.com/google/sanitizers/issues/1374 + Closes #6815 + +- [Jochem Broekhoff brought this change] + + examples/hiperfifo.c: check event_initialized before delete - Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541 + If event_del is called with the event struct (still) zeroed out, a + segmentation fault may occur. event_initialized checks whether the + event struct is nonzero. - Closes https://github.com/curl/curl/pull/6620 + Closes #6876 -- curl_multibyte: always return a heap-allocated copy of string +- [Patrick Monnerat brought this change] + + ntlm: fix negotiated flags usage - - Change the Windows char <-> UTF-8 conversion functions to return an - allocated copy of the passed in string instead of the original. + According to Microsoft document MS-NLMP, current flags usage is not + accurate: flag NTLMFLAG_NEGOTIATE_NTLM2_KEY controls the use of + extended security in an NTLM authentication message and NTLM version 2 + cannot be negotiated within the protocol. - Prior to this change the curlx_convert_ functions would, as what I - assume was an optimization, not make a copy of the passed in string if - no conversion was required. No conversion is required in non-UNICODE - Windows builds since our tchar strings are type char and remain in - whatever the passed in encoding is, which is assumed to be UTF-8 but may - be other encoding. + The solution implemented here is: if the extended security flag is set, + prefer using NTLM version 2 (as a server featuring extended security + should also support version 2). If version 2 has been disabled at + compile time, use extended security. - In contrast the UNICODE Windows builds require conversion - (wchar <-> char) and do return a copy. That inconsistency could lead to - programming errors where the developer expects a copy, and does not - realize that won't happen in all cases. + Tests involving NTLM are adjusted to this new behavior. - Closes https://github.com/curl/curl/pull/6602 + Fixes #6813 + Closes #6849 -Viktor Szakats (19 Feb 2021) -- http: add new files missed from referrer commit - - Ref: 44872aefc2d54f297caf2b0cc887df321bc9d791 - Ref: #6591 +- [Patrick Monnerat brought this change] -- http: add support to read and store the referrer header - - - add CURLINFO_REFERER libcurl option - - add --write-out '%{referer}' command-line option - - extend --xattr command-line option to fill user.xdg.referrer.url extended - attribute with the referrer (if there was any) + ntlm: support version 2 on 32-bit platforms - Closes #6591 + Closes #6849 -Daniel Stenberg (19 Feb 2021) -- urldata: remove the _ORIG suffix from string names - - It doesn't provide any useful info but only makes the names longer. - - Closes #6624 +- [Patrick Monnerat brought this change] -- url: fix memory leak if OOM in the HSTS handling + curl_ntlm_core.h: simplify conditionals for USE_NTLM2SESSION - Reported-by: Viktor Szakats - Bug: https://github.com/curl/curl/pull/6627#issuecomment-781626205 + ... as !defined(CURL_DISABLE_CRYPTO_AUTH) is a prerequisite for the + whole NTLM. - Closes #6628 + Closes #6849 -- gnutls: assume nettle crypto support - - nettle has been the default crypto library with GnuTLS since 2010. By - dropping support for the previous libcrypto, we simplify code. +- lib: remove unused HAVE_INET_NTOA_R* defines - Closes #6625 + Closes #6867 -- asyn-ares: use consistent resolve error message - - ... with the help of Curl_resolver_error() which now is moved from - asyn-thead.c and is provided globally for this purpose. +- [Michael Forney brought this change] + + configure: include unconditionally - Follow-up to 35ca04ce1b77636 + In 2682e5f5, several instances of AC_HEADER_TIME were removed since + it is a deprecated autoconf macro. However, this was the macro that + defined TIME_WITH_SYS_TIME, which was used to indicate that + can be included alongside . TIME_WITH_SYS_TIME is still + used in the configure test body and since it is no longer defined, + is *not* included on systems that have . - Makes test 1188 work for c-ares builds + In particular, at least on musl libc and glibc, does + not implicitly include and does not declare clock_gettime, + gmtime_r, or localtime_r. This causes configure to fail to detect + those functions. - Closes #6626 + The AC_HEADER_TIME macro deprecation text says + + > All current systems provide time.h; it need not be checked for. + > Not all systems provide sys/time.h, but those that do, all allow + > you to include it and time.h simultaneously. + + So, to fix this issue, simply include unconditionally when + testing for time-related functions and in libcurl, and don't bother + checking for it. + + Closes #6859 -Viktor Szakats (18 Feb 2021) -- ci: stop building on freebsd-12-1 +- [Michael Forney brought this change] + + configure: remove use of RETSIGTYPE - An updated freebsd-12-2 image was added a few months ago, and this - older one is consistently failing to go past `pkginstall`: - ``` - Newer FreeBSD version for package py37-mlt: - To ignore this error set IGNORE_OSVERSION=yes - - package: 1202000 - - running kernel: 1201000 - Ignore the mismatch and continue? [Y/n]: pkg: repository FreeBSD contains packages for wrong OS version: FreeBSD:12:amd64 - ``` + This was previously defined by the obsolete AC_TYPE_SIGNAL macro, + which was removed in 2682e5f5. The deprecation text says - FreeBSD thread suggests that 12.1 is EOL, and best to avoid. + > Your code may safely assume C89 semantics that RETSIGTYPE is void. - Ref: https://forums.freebsd.org/threads/78856/ + So, remove it and just use void instead. - Reviewed-by: Daniel Stenberg - Closes #6622 + Closes #6861 -Daniel Stenberg (18 Feb 2021) -- test1188: change error from connect to resolve error +- [Muhammed Yavuz Nuzumlalı brought this change] + + install: add instructions for Apple Darwin platforms - Using the %NOLISTENPORT to trigger a connection failure is somewhat - "risky" (since it isn't guaranteed to not be listened to) and caused - occasional CI problems. This fix changes the infused error to be a more - reliable one but still verifies the --write-out functionality properly - - which is the purpose of this test. + Closes #6860 + +- [Muhammed Yavuz Nuzumlalı brought this change] + + configure: disable min version set for Darwin - Reported-by: Jay Satiro - Fixes #6621 - Closes #6623 + Fixes #6838 + Closes #6860 -- url.c: use consistent error message for failed resolve +- [David Hu brought this change] -- BUGS: language polish + docs/HTTP3.md: update the build instruction using gnutls + + In ngtcp2 the `with-gnutls` option is disabled by default, which will + cause `curl` unable to be `make` because of lacking the libraries + needed. + + Closes #6857 -- wolfssl: don't store a NULL sessionid +- RELEASE-NOTES: synced + +- typecheck-gcc: make the ssl-ctx-cb check use SSL_CTX pointers - This caused a memory leak as the session id cache entry was still - erroneously stored with a NULL sessionid and that would later be treated - as not needed to get freed. + ... and not values. - Reported-by: Gisle Vanem - Fixes #6616 - Closes #6617 + Reported-by: locpyl-tidnyd on github + Fixes #6818 + Closes #6819 -- parse_proxy: fix a memory leak in the OOM path +- ngtcp2+gnutls: clear credentials when freed - Reported-by: Jay Satiro - Reviewed-by: Jay Satiro - Reviewed-by: Emil Engler + ... to avoid double-free. - Closes #6614 - Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541 + Reported-by: Kenneth Davidson + Fixes #6824 + Closes #6856 -Jay Satiro (17 Feb 2021) -- url: fix possible use-after-free in default protocol +Jay Satiro (5 Apr 2021) +- [Cherish98 brought this change] + + tool_progress: Fix progress meter in parallel mode - Prior to this change if the user specified a default protocol and a - separately allocated non-absolute URL was used then it was freed - prematurely, before it was then used to make the replacement URL. + Make sure the total amount of DL/UL bytes are counted before the + transfer finalizes. Otherwise if a transfer finishes too quick, its + total numbers are not added, and results in a DL%/UL% that goes above + 100%. - Bug: https://github.com/curl/curl/issues/6604#issuecomment-780138219 - Reported-by: arvids-kokins-bidstack@users.noreply.github.com + Detail: - Closes https://github.com/curl/curl/pull/6613 + progress_meter() is called periodically, and it may not catch a + transfer's total bytes if the value was unknown during the last call, + and the transfer is finished and deleted (i.e., lost) during the next + call. + + Closes https://github.com/curl/curl/pull/6840 -Daniel Stenberg (16 Feb 2021) -- multi: rename the multi transfer states +- [Emil Engler brought this change] + + libssh: get rid of PATH_MAX - While working on documenting the states it dawned on me that step one is - to use more descriptive names on the states. This also changes prefix on - the states to make them shorter in the source. + This removes the last occurrence of PATH_MAX inside our libssh + implementation by calculating the path length from the string length of + the two components. - State names NOT ending with *ing are transitional ones. + Closes #6829 + +Daniel Stenberg (5 Apr 2021) +- http_proxy: only loop on 407 + close if we have credentials - Closes #6612 + ... to fix the retry-loop. + + Add test 718 to verify. + + Reported-by: Daniel Kurečka + Fixes #6828 + Closes #6850 -Viktor Szakats (16 Feb 2021) -- http: do not add a referrer header with empty value +- h2: allow 100 streams by default - Previously an empty 'Referer:' header was added to the HTTP request when - passing `--referer ';auto'` or `--referer ''` on the command-line. This - patch makes `--referer` work like `--header 'Referer:'` and will only add - the header if it has a non-zero length value. + instead of 13, before the server has told how many streams it + accepts. The server can always reject new streams anyway if we go above + what it accepts. - Reviewed-by: Jay Satiro - Closes #6610 + Ref: #6826 + Closes #6852 -Daniel Stenberg (16 Feb 2021) -- lib: remove 'conn->data' completely +- [Luke Granger-Brown brought this change] + + file: support GETing directories again - The Curl_easy pointer struct entry in connectdata is now gone. Just - before commit 215db086e0 landed on January 8, 2021 there were 919 - references to conn->data. + After 957bc1881e686f9714c4e6a01bf33535091f0e21, we no longer compute an + expected_size for directories. This has the upshot that when we compare + even an empty Range with the available size, we fail. - Closes #6608 + This brings back the previous behaviour, which was to succeed, but with + empty content. This also removes the "Accept-ranges: bytes" header, + which is nonsensical on directories. + + Adds test 3016 + Fixes #6845 + Closes #6846 -- openldap: pass 'data' to the callbacks instead of 'conn' +- RELEASE-NOTES: synced + + and bumped to 7.76.1 -Jay Satiro (15 Feb 2021) -- doh: Fix sharing user's resolve list with DOH handles +- TLS: fix HTTP/2 selection - - Share the shared object from the user's easy handle with the DOH - handles. + for GnuTLS, BearSSL, mbedTLS, NSS, SChannnel, Secure Transport and + wolfSSL... - Prior to this change if the user had set a shared object with shared - cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't - used by any associated DOH handles, since they used the multi's default - hostcache. + Regression since 88dd1a8a115b1f5ece (shipped in 7.76.0) + Reported-by: Kenneth Davidson + Reported-by: romamik om github + Fixes #6825 + Closes #6827 + +Jay Satiro (2 Apr 2021) +- hostip: Fix for builds that disable all asynchronous DNS - This change means all the handles now use the same hostcache, which is - either the shared hostcache from the user created shared object if it - exists or if not then the multi's default hostcache. + - Define Curl_resolver_error function only when USE_CURL_ASYNC. - Reported-by: Manuj Bhatia + Prior to this change building curl without an asynchronous resolver + backend (c-ares or threaded) and without DoH (DNS-over-HTTPS, which is + also asynchronous but independent of resolver backend) would cause a + build error since Curl_resolver_error is called by and evaluates + variables only available in asynchronous builds. - Fixes https://github.com/curl/curl/issues/6589 - Closes https://github.com/curl/curl/pull/6607 + Reported-by: Benbuck Nason + + Fixes https://github.com/curl/curl/issues/6831 + Closes https://github.com/curl/curl/pull/6832 -Daniel Stenberg (15 Feb 2021) -- http2: remove conn->data use +Daniel Stenberg (31 Mar 2021) +- [Gilles Vollant brought this change] + + openssl: Fix CURLOPT_SSLCERT_BLOB without CURLOPT_SSLCERT_KEY - ... but instead use a private alternative that points to the "driving - transfer" from the connection. We set the "user data" associated with - the connection to be the connectdata struct, but when we drive transfers - the code still needs to know the pointer to the transfer. We can change - the user data to become the Curl_easy handle, but with older nghttp2 - version we cannot dynamically update that pointer properly when - different transfers are used over the same connection. + Reported-by: Christian Schmitz + Fixes #6816 + Closes #6820 + +Version 7.76.0 (31 Mar 2021) + +Daniel Stenberg (31 Mar 2021) +- RELEASE-NOTES: synced - Closes #6520 + curl 7.76.0 release -- openssl: remove conn->data use +- THANKS: added names from 7.76.0 + +- CURLOPT_AUTOREFERER.3: clarify that it sets the full URL - We still make the trace callback function get the connectdata struct - passed to it, since the callback is anchored on the connection. + ... some users may not want that! + +- define: remove CURL_DISABLE_NTLM ifdefs - Repeatedly updating the callback pointer to set 'data' with - SSL_CTX_set_msg_callback_arg() doesn't seem to work, probably because - there might already be messages in the queue with the old pointer. + It was never defined anywhere. Fixed disable-scan (test 1165) to also + scan headers, which found this issue. - This code therefore makes sure to set the "logger" handle before using - OpenSSL calls so that the right easy handle gets used for tracing. + Closes #6809 + +- vtls: fix addsessionid for non-proxy builds - Closes #6522 + Follow-up to b09c8ee15771c61 + Fixes #6812 + Closes #6811 + +- [Li Xinwei brought this change] + + cmake: support WinIDN + + Closes #6807 + +- transfer: clear 'referer' in declaration + + To silence (false positive) compiler warnings about it. + + Follow-up to 7214288898f5625 + + Reviewed-by: Marcel Raad + Closes #6810 + +- [Marc Hoersken brought this change] + + config: fix SSPI enabling NTLM if crypto auth is disabled + + Avoid enabling NTLM feature based upon Windows SSPI + being enabled in case that crypto auth is disabled. + + Reported-by: Marcel Raad + + Follow-up to #6277 + Fixes #6803 + Closes #6808 + +- HISTORY: add two 2021 events + +- vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid() + + To make sure we set and extract the correct session. + + Reported-by: Mingtao Yang + Bug: https://curl.se/docs/CVE-2021-22890.html + + CVE-2021-22890 + +- [Viktor Szakats brought this change] + + transfer: strip credentials from the auto-referer header field + + Added test 2081 to verify. + + CVE-2021-22876 + + Bug: https://curl.se/docs/CVE-2021-22876.html + +- curl_sasl: fix compiler error with --disable-crypto-auth + + ... if libgsasl was found. + + Closes #6806 + +- [Patrick Monnerat brought this change] + + ldap: only set the callback ptr for TLS context when TLS is used + + Follow-up to a5eee22e594c2460f + Fixes #6804 + Closes #6805 + +- copyright: update copyright year ranges to 2021 + + Reviewed-by: Emil Engler + Closes #6802 + +- send_speed: simplify the checks for if a speed limit is set + + ... as we know the value cannot be set to negative: enforced by + setopt() + +- http: cap body data amount during send speed limiting + + By making sure never to send off more than the allowed number of bytes + per second the speed limit logic is given more room to actually work. + + Reported-by: Fabian Keil + Bug: https://curl.se/mail/lib-2021-03/0042.html + Closes #6797 + +- urldata: merge "struct DynamicStatic" into "struct UrlState" + + Both were used for the same purposes and there was no logical separation + between them. Combined, this also saves 16 bytes in less holes in my + test build. + + Closes #6798 + +- tests/README.md: mentioned that en_US.UTF-8 is required + + Reported-by: Oumph on github + Fixes #6768 + +- HISTORY: fixed the Mac OS X 10.1 release date + + Based on what Wikipedia says + +Jay Satiro (26 Mar 2021) +- examples: Remove threaded-shared-conn.c due to bug + + Known bug 11.11 is the shared object's connection cache is not thread + safe, so we should not have an example for it. + + Ref: https://github.com/curl/curl/issues/4915 + Ref: https://curl.se/docs/knownbugs.html#A_shared_connection_cache_is_not + + Closes https://github.com/curl/curl/pull/6795 + +- KNOWN_BUGS: Update 11.9 - DoH option inheritance + + - Add description: Explain that some options aren't inherited because + they are not relevant for the DoH SSL connections or may result in + unexpected behavior. + + - Remove the reference to #4578 (SSL verify options not inherited) since + that was fixed by #6597 (separate DoH-specific options for verify). + + - Explain that DoH-specific options (those created by #6597) are + available: CURLOPT_DOH_SSL_VERIFYHOST, CURLOPT_DOH_SSL_VERIFYPEER and + CURLOPT_DOH_SSL_VERIFYSTATUS. + + - Add a reference to #6605 and explain that the user's debug function is + not inherited because it would be unexpected to pass internal handles + (ie DoH handles) to the user's callback. + + Closes https://github.com/curl/curl/issues/6605 + +Daniel Stenberg (26 Mar 2021) +- curl_easy_setopt.3: add curl_easy_option* functions to SEE ALSO + +- [Jean-Philippe Menil brought this change] + + openssl: ensure to check SSL_CTX_set_alpn_protos return values + + SSL_CTX_set_alpn_protos() return 0 on success, and non-0 on failure + + Signed-off-by: Jean-Philippe Menil + + Closes #6794 + +- multi: close the connection when h2=>h1 downgrading + + Otherwise libcurl is likely to reuse the connection again in the next + attempt since the connection reuse logic doesn't take downgrades into + account. + + Reported-by: Anthony Ramine + Fixes #6788 + Closes #6793 + +- openssl: set the transfer pointer for logging early + + Otherwise, the transfer will be NULL in the trace function when the + early handshake details arrive and then curl won't show them. + + Regresssion in 7.75.0 + + Reported-by: David Hu + Fixes #6783 + Closes #6792 - RELEASE-NOTES: synced -Jay Satiro (14 Feb 2021) -- doh: add options to disable ssl verification +- TODO: Custom progress meter update interval - - New libcurl options CURLOPT_DOH_SSL_VERIFYHOST, - CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the - same as their respective counterparts. + Ref: https://stackoverflow.com/q/66789977/93747 + +- docs/ABI: tighten up the language - - New curl tool options --doh-insecure and --doh-cert-status do the same - as their respective counterparts. + Make the promises more firm - Prior to this change DOH SSL certificate verification settings for - verifyhost and verifypeer were supposed to be inherited respectively - from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug - were not. As a result DOH verification remained at the default, ie - enabled, and it was not possible to disable. This commit changes - behavior so that the DOH verification settings are independent and not - inherited. + Closes #6786 + +- openldap: disconnect better - Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676 + Instead of clearing the callback argument in disconnect, set it to the + (new) transfer to make sure the correct data is passed to the callbacks. - Fixes https://github.com/curl/curl/issues/4578 - Closes https://github.com/curl/curl/pull/6597 + Follow-up to e467ea3bd937f38 + Assisted-by: Patrick Monnerat + Closes #6787 -- hostip: fix crash in sync resolver builds that use DOH +- libssh2: kdb_callback: get the right struct pointer - - Guard some Curl_async accesses with USE_CURL_ASYNC instead of - !CURLRES_SYNCH. + After the recent conn/data refactor in this source file, this function + was mistakenly still getting the old struct pointer which would lead to + crash on servers with keyboard-interactive auth enabled. - This is another follow-up to 8335c64 which moved the async struct from - the connectdata struct into the Curl_easy struct. A previous follow-up - 6cd167a fixed building for sync resolver by guarding some async struct - accesses with !CURLRES_SYNCH. The problem is since DOH (DNS-over-HTTPS) - is available as an asynchronous secondary resolver the async struct may - be used even when libcurl is built for the sync resolver. That means - that CURLRES_SYNCH and USE_CURL_ASYNC may be defined at the same time. + Follow-up to a304051620b92e12b (shipped in 7.75.0) - Closes https://github.com/curl/curl/pull/6603 + Reported-by: Christian Schmitz + Fixes #6691 + Closes #6782 -Daniel Stenberg (13 Feb 2021) -- KNOWN_BUGS: cannot enable LDAPS on Windows with cmake +- tftp: remove unused struct fields - Reported-by: Jack Boos Yu - Closes #6284 + Follow-up to d3d90ad9c00530d + + Closes #6781 -- KNOWN_BUGS: Excessive HTTP/2 packets with TCP_NODELAY +- openldap: avoid NULL pointer dereferences + + Follow-up to a59c33ceffb8f78 + Reported-by: Patrick Monnerat + Fixes #6676 + Closes #6780 + +- http: strip default port from URL sent to proxy + + To make sure the Host: header and the URL provide the same authority + portion when sent to the proxy, strip the default port number from the + URL if one was provided. - Reported-by: Alex Xu - Closes #6363 + Reported-by: Michael Brown + Fixes #6769 + Closes #6778 -- http: use credentials from transfer, not connection +- azure: disable test 433 on azure-ubuntu - HTTP auth "accidentally" worked before this cleanup since the code would - always overwrite the connection credentials with the credentials from - the most recent transfer and since HTTP auth is typically done first - thing, this has not been an issue. It was still wrong and subject to - possible race conditions or future breakage if the sequence of functions - would change. + Something in that environment sets XDG_CONFIG_HOME for us in a way that + breaks the test. - The data.set.str[] strings MUST remain unmodified exactly as set by the - user, and the credentials to use internally are instead set/updated in - state.aptr.* + Reported-by: Marc Hörsken + Fixes #6739 + Closes #6777 + +- tftp: remove the 3600 second default timeout - Added test 675 to verify different credentials used in two requests done - over a reused HTTP connection, which previously behaved wrongly. + ... it was never meant to be there. - Fixes #6542 - Closes #6545 + Reported-by: Tomas Berger + Fixes #6774 + Closes #6776 -- test433: clear some home dir env variables +- docs: make gen.pl support *italic* and **bold** - Follow-up to bd6b54ba1f55b5 + Remove some nroffisms from the cmdline doc files to simplify editing, + and instead support this markdown style. - ... so that XDG_CONFIG_HOME is the only home dir variable set and thus - used correctly in the test! + Closes #6771 + +- ngtcp2: sync with recent API updates - Fixes #6599 - Closes #6600 + Closes #6770 - RELEASE-NOTES: synced - - bumped the version to 7.76.0 -- travis: install libgsasl-dev to add that to the builds +- libssh2:ssh_connect: clear session pointer after free - Closes #6588 + If libssh2_knownhost_init() returns NULL, like in an OOM situation, the + ssh session was freed but the pointer wasn't cleared which made libcurl + later call libssh2 to cleanup using the stale pointer. + + Fixes #6764 + Closes #6766 -- urldata: don't touch data->set.httpversion at run-time +- [Jacob Hoffman-Andrews brought this change] + + docs: document version of crustls dependency - Rename it to 'httpwant' and make a cloned field in the state struct as - well for run-time updates. + This also pins a specific release in the Travis test so future + API-breaking changins in crustls won't break curl builds. - Also: refuse non-supported HTTP versions. Verified with test 129. + Add RUSTLS documentation to release tarball. - Closes #6585 + Enable running tests for rustls, minus FTP tests (require + connect_blocking, which rustls doesn't implement) and 313 (requires CRL + handling). + + Closes #6763 -Viktor Szakats (11 Feb 2021) -- tests: disable .curlrc in more environments +- [Jacob Hoffman-Andrews brought this change] + + rustls: Handle close_notify. - by also setting CURL_HOME and XDG_CONFIG_HOME envvars to the local - directory. + If we get a close_notify, treat that as EOF. If we get an EOF from the + TCP stream, treat that as an error (because we should have ended the + connection earlier, when we got a close_notify). - Reviewed-by: Daniel Stenberg - Fixes #6595 - Closes #6596 + Closes #6763 -- docs/Makefile.inc: format to be update-friendly +- docs: clarify timeouts for queued transfers in multi API - - one source file per line - - convert tabs to spaces - - do not align line-continuation backslashes - - sort source files alphabetically + Closes #6758 + +- ftpserver: only load the preprocessed test file - Reviewed-by: Daniel Stenberg - Closes #6593 + We always preprocess and tests are no longer sensible to load "raw" + + Closes #6738 -Daniel Stenberg (11 Feb 2021) -- curl: provide libgsasl version and feature info in -V output +- tests: use %TESTNUMBER instead of fixed number - Closes #6592 + This makes the tests easier to copy and relocate to other test numbers + without having to update content. + + Closes #6738 -- gsasl: provide CURL_VERSION_GSASL if built-in +- KNOWN_BUGS: CURLOPT_OPENSOCKETPAIRFUNCTION is missing - To let applications know the feature is available. + Closes #5747 + +- TODO: provide timing info for each redirect - Closes #6592 + Closes #6743 -- curl: add --fail-with-body +Jay Satiro (17 Mar 2021) +- docs: Add SSL backend names to CURL_SSL_BACKEND - Prevent both --fail and --fail-with-body on the same command line. + - Document the names that can be used with CURL_SSL_BACKEND: + bearssl, gnutls, gskit, mbedtls, mesalink, nss, openssl, rustls, + schannel, secure-transport, wolfssl - Verify with test 349, 360 and 361. + Ref: https://github.com/curl/curl/issues/2209#issuecomment-360623286 + Ref: https://github.com/curl/curl/issues/6717#issuecomment-800745201 - Closes #6449 + Closes https://github.com/curl/curl/pull/6755 -- TODO: remove HSTS +- docs: Explain DOH transfers inherit some SSL settings - Provided now since commit 7385610d0c74 + - Document in DOH that some SSL settings are inherited but DOH hostname + and peer verification are not and are controlled separately. + + - Document that CURLOPT_SSL_CTX_FUNCTION is inherited by DOH handles but + we're considering changing behavior to no longer inherit it. Request + feedback. + + Closes https://github.com/curl/curl/pull/6688 -Jay Satiro (10 Feb 2021) -- tests: Fix tests failing due to change in curl --help +Daniel Stenberg (17 Mar 2021) +- http: make 416 not fail with resume + CURLOPT_FAILONERRROR - Follow-up to parent 3183217 which added add missing argument to - --create-file-mode . + When asked to resume a download, libcurl will convert that to HTTP logic + and if then the entire file is already transferred it will result in a + 416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that + scenario, it should *not* lead to an error return. - Ref: https://github.com/curl/curl/issues/6590 + Updated test 1156, added test 1273 + + Reported-by: Jonathan Watt + Fixes #6740 + Closes #6753 -- tool_help: add missing argument for --create-file-mode +- Curl_timeleft: check both timeouts during connect - Prior to this change the required argument was not shown in curl --help. + The duration of a connect and the total transfer are calculated from two + different time-stamps. It can end up with the total timeout triggering + before the connect timeout expires and we should make sure to + acknowledge whichever timeout that is reached first. - before: - --create-file-mode File mode for created files + This is especially notable when a transfer first sits in PENDING, as + that time is counted in the total time but the connect timeout is based + on the time since the handle changed to the CONNECT state. - after: - --create-file-mode File mode (octal) for created files + The CONNECTTIMEOUT is per connect attempt. The TIMEOUT is for the entire + operation. - Reported-by: ZimCodes@users.noreply.github.com + Fixes #6744 + Closes #6745 + Reported-by: Andrei Bica + Assisted-by: Jay Satiro + +- configure: remove use of deprecated macros - Fixes https://github.com/curl/curl/issues/6590 + AC_HEADER_TIME, AC_HEADER_STDC and AC_TYPE_SIGNAL -- create-file-mode.d: add missing Arg tag +- configure: make AC_TRY_* into AC_*_IFELSE - Prior to this change the required argument was not shown. + ... as the former versions are deprecated. + +- configure: s/AC_HELP_STRING/AS_HELP_STRING - curl.1 before: --create-file-mode - curl.1 after: --create-file-mode + AC_HELP_STRING is deprecated in 2.70+ and I believe AS_HELP_STRING works + already since 2.59 so bump the minimum required version to that. - Reported-by: ZimCodes@users.noreply.github.com + Reported-by: Emil Engler + Fixes #6647 + Closes #6748 + +- RELEASE-NOTES: synced + +- travis: use ubuntu nghttp2 package instead of build our own - Fixes https://github.com/curl/curl/issues/6590 + Closes #6751 -Viktor Szakats (10 Feb 2021) -- gsasl: fix errors/warnings building against libgsasl +- travis: bump wolfssl to 4.7.0 + +- travis: only build wolfssl when needed - - also fix an indentation - - make Curl_auth_gsasl_token() use CURLcode (by Daniel Stenberg) + Closes #6751 + +- [Jacob Hoffman-Andrews brought this change] + + rustls: allocate a buffer for TLS data. - Ref: https://github.com/curl/curl/pull/6372#issuecomment-776118711 - Ref: https://github.com/curl/curl/pull/6588 + Previously, rustls was using an on-stack array for TLS data. However, + crustls has an (unusual) requirement that buffers it deals with are + initialized before writing to them. By using calloc, we can ensure the + buffer is initialized once and then reuse it across calls. - Reviewed-by: Jay Satiro - Assisted-by: Daniel Stenberg - Reviewed-by: Simon Josefsson - Closes #6587 + Closes #6742 -- Makefile.m32: add support for libgsasl dependency +- travis: add a rustls build - Reviewed-by: Marcel Raad - Closes #6586 + ... that doesn't run any tests (yet) + + Closes #6750 -Marcel Raad (10 Feb 2021) -- ngtcp2: clarify calculation precedence +- HTTP2: remove the outdated remark about multiplexing for the tool + +- [Robert Ronto brought this change] + + http2: don't set KEEP_SEND when there's no more data to be sent - As suggested by Codacy/cppcheck. + this should fix an issue where curl sometimes doesn't send out a request + with authorization info after a 401 is received over http2 - Closes https://github.com/curl/curl/pull/6576 + Closes #6747 -- server: remove redundant condition +Marc Hoersken (15 Mar 2021) +- config: fix building SMB with configure using Win32 Crypto + + Align conditions for NTLM features between CMake and configure + builds by differentiating between USE_NTLM and USE_CURL_NTLM_CORE, + just like curl_setup.h does internally to detect support of: + + - USE_NTLM: required for NTLM crypto authentication feature + - USE_CURL_NTLM_CORE: required for SMB protocol + + Implement USE_WIN32_CRYPTO detection by checking for Crypt functions + in wincrypt.h which are not available in the Windows App environment. + + Link advapi32 and crypt32 for Crypto API and Schannel SSL backend. + Fix condition of Schannel SSL backend in CMake build accordingly. - `end` is always non-null here. + Reviewed-by: Marcel Raad - Closes https://github.com/curl/curl/pull/6576 + Closes #6277 -- lib: remove redundant code +- config: fix detection of restricted Windows App environment - Closes https://github.com/curl/curl/pull/6576 - -- mqttd: remove unused variable + Move the detection of the restricted Windows App environment + in curl_setup.h before the definition of USE_WIN32_CRYPTO + via included config-win32.h in case no build system is used. - Closes https://github.com/curl/curl/pull/6576 - -- tool_paramhlp: reduce variable scope + Reviewed-by: Marcel Raad - Closes https://github.com/curl/curl/pull/6576 + Part of #6277 -- tests: reduce variable scopes - - Closes https://github.com/curl/curl/pull/6576 +Daniel Stenberg (15 Mar 2021) +- HISTORY: curl 7.7.2 was the first version used in Mac OS X 10.1 -- lib: reduce variable scopes +- gen.pl: quote "bare" minuses in the nroff curl.1 - Closes https://github.com/curl/curl/pull/6576 + Reported-by: Alejandro Colomar + Fixes #6698 + Closes #6722 -- ftp: fix Codacy/cppcheck warning about null pointer arithmetic +Daniel Gustafsson (14 Mar 2021) +- hsts: remove unused defines - Increment `bytes` only if it is non-null. + MAX_HSTS_SUBLEN and MAX_HSTS_SUBLENSTR were unused from the initial commit, + and mostly likely leftovers from early development. Remove as they're not + used for anything. - Closes https://github.com/curl/curl/pull/6576 - -Daniel Stenberg (9 Feb 2021) -- ngtcp2: adapt to the new recv_datagram callback + Closes #6741 + Reviewed-by: Daniel Stenberg -- quiche: fix build error: use 'int' for port number +Daniel Stenberg (12 Mar 2021) +- github: add torture-ftp for FTP-only torture testing - Follow-up to cb2dc1ba8 + and at 20% to try to keep the run-time reasonable + + Closes #6728 -- ftp: add 'list_only' to the transfer state struct +- travis: split "torture" into a separate "events" build as well - and rename it from 'ftp_list_only' since it is also used for SSH and - POP3. The state is updated internally for 'type=D' FTP URLs. + Run torture without FTP and reducing coverage to 20% - Added test case 1570 to verify. + For some reason the torture tests now run a lot slower on travis and run + into the 50 minute limit all the time. - Closes #6578 + Closes #6728 -- ftp: add 'prefer_ascii' to the transfer state struct +- ftp: fix memory leak in ftp_done - ... and make sure the code never updates 'set.prefer_ascii' as it breaks - handle reuse which should use the setting as the user specified it. + If after a transfer is complete Curl_GetFTPResponse() returns an error, + curl would not free the ftp->pathalloc block. - Added test 1569 to verify: it first makes an FTP transfer with ';type=A' - and then another without type on the same handle and the second should - then use binary. Previously, curl failed this. + Found by torture-testing test 576 - Closes #6578 + Closes #6737 + +- [oxalica brought this change] + + http2: fail if connection terminated without END_STREAM + + Closes #6736 - RELEASE-NOTES: synced - [Jacob Hoffman-Andrews brought this change] - vtls: initial implementation of rustls backend + rustls: support CURLOPT_SSL_VERIFYPEER - This adds a new TLS backend, rustls. It uses the C-to-rustls bindings - from https://github.com/abetterinternet/crustls. + This requires the latest main branch of crustls, which provides + rustls_client_config_builder_dangerous_set_certificate_verifier and + rustls_client_config_builder_set_enable_sni. - Rustls is at https://github.com/ctz/rustls/. + This refactors the session setup into its own function, and adds a new + function cr_hostname_is_ip. Because crustls doesn't support verification + of IP addresses, special handling is needed: We disable SNI and set a + placeholder hostname (which never actually gets sent on the wire). - There is still a fair bit to be done, like sending CloseNotify on - connection shutdown, respecting CAPATH, and properly indicating features - like "supports TLS 1.3 ciphersuites." But it works well enough to make - requests and receive responses. + Closes #6719 + +Daniel Gustafsson (12 Mar 2021) +- cookies: Fix potential NULL pointer deref with PSL - Blog post for context: - https://www.abetterinternet.org/post/memory-safe-curl/ + Curl_cookie_init can be called with data being NULL, and this can in turn + be passed to Curl_cookie_add, meaning that both functions must be careful + to only use data where it's checked for being a NULL pointer. The libpsl + support code does however dereference data without checking, so if we are + indeed having an unset data pointer we cannot PSL check the cookiedomain. - Closes #6350 + This is currently not a reachable dereference, as the only caller with a + NULL data isn't passing a file to initialize cookies from, but since the + API has this contract let's ensure we hold it. + + Closes #6731 + Reviewed-by: Daniel Stenberg -- [Simon Josefsson brought this change] +Daniel Stenberg (12 Mar 2021) +- [Michael Hordijk brought this change] - sasl: support SCRAM-SHA-1 and SCRAM-SHA-256 via libgsasl + configure: only add OpenSSL paths if they are defined - Closes #6372 + Add paths for OpenSSL compiling and linking only if they have been + defined. If they haven't been defined, we'll assume that the paths are + already available to the toolchain. + + Closes #6730 -Jay Satiro (9 Feb 2021) -- lib: use int type for more port variables +Jay Satiro (12 Mar 2021) +- retry.d: Clarify transient 5xx HTTP response codes - This is a follow-up to 764c6bd. Prior to that change port variables - were usually type long. + - Clarify the only 5xx response codes that are treated as transient are + 500, 502, 503 and 504. - Closes https://github.com/curl/curl/pull/6553 - -- tool_writeout: refactor write-out and write-out json + Prior to this change it said it treated all 5xx as transient, but the + code says otherwise. - - Deduplicate the logic used by write-out and write-out json. + Ref: https://github.com/curl/curl/blob/curl-7_75_0/src/tool_operate.c#L462-L495 - Rather than have separate writeLong, writeString, etc, logic for - each of write-out and write-out json instead have respective shared - functions that can output either format and a 'use_json' parameter to - indicate whether it is json that is output. + Closes https://github.com/curl/curl/pull/6724 + +- retry-all-errors.d: Explain curl errors versus HTTP response errors - This will make it easier to maintain. Rather than have to go through - two sets of logic now we only have to go through one. + - Add a paragraph explaining that curl does not consider HTTP response + errors as curl errors, and how that behavior can be modified by using + --retry and --fail. - - Support write-out %{errormsg} and %{exitcode} in json. + The --retry-all-errors doc says "Retry on any error" which some users + may find misleading without the added explanation. - - Clarify in the doc that %{exitcode} is the exit code of the transfer. + Ref: https://curl.se/docs/faq.html#Why_do_I_get_downloaded_data_eve + Ref: https://curl.se/docs/faq.html#curl_doesn_t_return_error_for_HT - Prior to this change it just said "The numerical exitcode" which - implies it's the exit code of the tool, and it's not necessarily that. + Reported-by: Lawrence Gripper - Closes https://github.com/curl/curl/pull/6544 + Fixes https://github.com/curl/curl/issues/6712 + Closes https://github.com/curl/curl/pull/6720 -- lib: drop USE_SOCKETPAIR in favor of CURL_DISABLE_SOCKETPAIR +Daniel Stenberg (11 Mar 2021) +- travis: switch ngtcp2 build over to quictls - .. since the former is undocumented and they both do the same thing. + The ngtcp2 project switched over to using the quictls OpenSSL fork + instead of their own patched OpenSSL. We follow suit. - Closes https://github.com/curl/curl/pull/6517 + Closes #6729 -- curl_multibyte: fall back to local code page stat/access on Windows +- test220/314: adjust to run with Hyper + +- c-hyper: support automatic content-encoding - If libcurl is built with Unicode support for Windows then it is assumed - the filename string is Unicode in UTF-8 encoding and it is converted to - UTF-16 to be passed to the wide character version of the respective - function (eg wstat). However the filename string may actually be in the - local encoding so, even if it successfully converted to UTF-16, if it - could not be stat/accessed then try again using the local code page - version of the function (eg wstat fails try stat). + Closes #6727 + +- http: remove superfluous NULL assign - We already do this with fopen (ie wfopen fails try fopen), so I think it - makes sense to extend it to stat and access functions. + Closes #6727 + +- tool_operate: bail if set CURLOPT_HTTP09_ALLOWED returns error - Closes https://github.com/curl/curl/pull/6514 + Closes #6727 -- [Stephan Szabo brought this change] +- setopt: error on CURLOPT_HTTP09_ALLOWED set true with Hyper + + Not supported. + + Closes #6727 - file: Support unicode urls on windows +- test306: make it not run with Hyper - Closes https://github.com/curl/curl/pull/6501 + ... as it tests HTTP/0.9 which Hyper doesn't support. -- [Vincent Torri brought this change] +- test304: header CRLF cleanup to work with Hyper - cmake: fix import library name for non-MS compiler on Windows +- FTP: allow SIZE to fail when doing (resumed) upload - - Use _imp.lib suffix only for Microsoft's compiler (MSVC). + Added test 362 to verify. - Prior to this change library suffix _imp.lib was used for the import - library on Windows regardless of compiler. + Reported-by: Jordan Brown + Regression since 7ea2e1d0c5a7f (7.73.0) + Fixes #6715 + Closes #6725 + +- configure: provide Largefile feature for curl-config - With this change the other compilers should now use their default - suffix which should be .dll.a. + ... as cmake now does it correctly, and make test1014 check for it - This change is motivated by the usage of pkg-config on MSYS2. - Indeed, when 'pkg-config --libs libcurl' is used, -lcurl is - passed to ld. The documentation of ld on Windows : + Closes #6702 + +- config: remove CURL_SIZEOF_CURL_OFF_T use only SIZEOF_CURL_OFF_T - https://sourceware.org/binutils/docs/ld/WIN32.html + Make the code consistently use a single name for the size of the + "curl_off_t" type. - lists, in the 'direct linking to a dll' section, the pattern - of the searched import library, and libcurl_imp.lib is not there. + Closes #6702 + +Jay Satiro (10 Mar 2021) +- [Jun-ya Kato brought this change] + + ngtcp2: Fix build error due to change in ngtcp2_addr_init - Closes https://github.com/curl/curl/pull/6225 + ngtcp2/ngtcp2@b8d90a9 changed the function prototype. + + Closes https://github.com/curl/curl/pull/6716 -Daniel Stenberg (9 Feb 2021) -- urldata: move 'followlocation' to UrlState +Daniel Stenberg (10 Mar 2021) +- [ejanchivdorj brought this change] + + multi: update pending list when removing handle - As this is a state variable it does not belong in UserDefined which is - used to store values set by the user. + when removing a handle, most of the lists are updated but pending list + is not updated. Updating now. - Closes #6582 + Closes #6713 + +- [kokke brought this change] + + lib1536: check ptr against NULL before dereferencing it + + Closes #6710 -- [Ikko Ashimine brought this change] +- [kokke brought this change] - http_proxy: fix typo in http_proxy.c - - settting -> setting + lib1537: check ptr against NULL before dereferencing it - Closes #6583 - -- [Fabian Keil brought this change] + Fixes #6707 + Closes #6708 - tests/server: Bump MAX_TAG_LEN to 200 +- travis: make torture tests skip TLS-SRP tests - This is useful for tests containing HTML inside of sections. - For tags it's not uncommon to be longer than the previous - limit of 79 bytes. + ... as it seems to often hang. - An example of a previously problem-causing tag is: - - which is needed for a Privoxy test for the banners-by-size filter. + Also: skip the "normal" tests as they're already run by many other + builds. - Previously it caused server failures like: - 12:29:05.786961 ====> Client connect - 12:29:05.787116 accept_connection 3 returned 4 - 12:29:05.787194 accept_connection 3 returned 0 - 12:29:05.787285 Read 119 bytes - 12:29:05.787345 Process 119 bytes request - 12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1 - 12:29:05.787464 Requested test number 9 part 0 - 12:29:05.787686 getpart() failed with error: -2 - 12:29:05.787744 - request found to be complete (9) - 12:29:05.787912 getpart() failed with error: -2 - 12:29:05.788048 Wrote request (119 bytes) input to log/server.input - 12:29:05.788157 Send response test9 section - 12:29:05.788443 getpart() failed with error: -2 - 12:29:05.788498 instructed to close connection after server-reply - 12:29:05.788550 ====> Client disconnect 0 - 12:29:05.871448 exit_signal_handler: 15 - 12:29:05.871714 signalled to die - 12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15) - -- [Fabian Keil brought this change] - - tests/badsymbols.pl: when opening '$incdir' fails include it in the error message - -- [Fabian Keil brought this change] + Closes #6705 - runtests.1: document -o, -P, -L, and -E +- openssl: adapt to v3's new const for a few API calls + + Closes #6703 -- [Fabian Keil brought this change] +- quiche: fix crash when failing to connect + + Reported-by: ウさん + Fixes #6664 + Closes #6701 - runtests.pl: add %TESTNUMBER variable to make copying tests more convenient +- RELEASE-NOTES: synced + + Fixed the release counter and added a missing contributor -- [Fabian Keil brought this change] +- RELEASE-NOTES: synced - runtests.pl: add an -o option to change internal variables +- dynbuf: bump the max HTTP request to 1MB - runtests.pl has lots of internal variables one might want to - change in certain situations, but adding a dedicated option - for every single one of them isn't practical. + Raised from 128KB to allow longer request headers. - Usage: - ./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ... - -- [Fabian Keil brought this change] + Reported-by: Carl Zogheib + Fixes #6681 + Closes #6685 - runtests.pl: cleanups +Jay Satiro (6 Mar 2021) +- schannel: Evaluate CURLOPT_SSL_OPTIONS via SSL_SET_OPTION macro - - show the summarized test result in the last line of the report - - do not use $_ after mapping it to a named variable - Doing that makes the code harder to follow. - - log the restraints sorted by the number of their occurrences - - fix language when logging restraints that only occured once - - let runhttpserver() use $TESTDIR instead of $srcdir - ... so it works if a non-default $TESTDIR is being used. - -- [Fabian Keil brought this change] - - runtests.pl: add an -E option to specify an exclude file + - Change use of those options from CURLOPT_SSL_OPTIONS that are not + already evaluated via SSL_SET_OPTION in schannel and secure transport + to use that instead of data->set.ssl.optname. - It can contain additional restraints for test numbers, - keywords and tools. + Example: - The idea is to let third parties like the Privoxy project - distribute an exclude file with their tarballs that specifies - which curl tests are not expected to work when using Privoxy - as a proxy, without having to fork the whole curl test suite. + Evaluate SSL_SET_OPTION(no_revoke) instead of data->set.ssl.no_revoke. - The syntax could be changed to be extendable and maybe - more closely reflect the "curl test" syntax. Currently - it's a bunch of lines like these: + This change is because options set via CURLOPT_SSL_OPTIONS + (data->set.ssl.optname) are separate from those set for HTTPS proxy via + CURLOPT_PROXY_SSL_OPTIONS (data->set.proxy_ssl.optname). The + SSL_SET_OPTION macro determines whether the connection is for HTTPS + proxy and based on that which option to evaluate. - test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped - keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped - tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped + Since neither Schannel nor Secure Transport backends currently support + HTTPS proxy in libcurl, this change is for posterity and has no other + effect. - To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs - on a single line, split them with commas. + Closes https://github.com/curl/curl/pull/6690 -- [Fabian Keil brought this change] +- [kokke brought this change] - runtests.pl: add -L parameter to require additional perl libraries + c-hyper: Remove superfluous pointer check - This is useful to change the behaviour of the script without - having to modify the file itself, for example to use a custom - compareparts() function that ignores header differences that - are expected to occur when an external proxy is being used. + `n` pointer is never NULL once set. Found by static analysis. - Such differences are proxy-specific and thus the modifications - should be maintained together with the proxy. - -- [Fabian Keil brought this change] + Ref: https://github.com/curl/curl/issues/6696 + + Closes https://github.com/curl/curl/pull/6697 - runtests.pl: add a -P option to specify an external proxy +- version.d: Add missing features to the features list - ... that should be used when executing the tests. + - Add missing entries for gsasl, Kerberos, NTLM_WB, TrackMemory, + Unicode and zstd. - The assumption is that the proxy is an HTTP proxy. + - Remove krb4 since it's no longer a feature. - This option should be used together with -L to provide - a customized compareparts() version that knows which - proxy-specific header differences should be ignored. + Reported-by: Ádler Jonas Gross - This option doesn't work for all test types yet. + Fixes https://github.com/curl/curl/issues/6677 + Closes https://github.com/curl/curl/pull/6687 -- [Fabian Keil brought this change] +- [Vladimir Varlamov brought this change] - tests: fixup several tests - - missing CRs and modified %hostip + docs: add missing Arg tag to --stderr - lib556/test556: use a real HTTP version to make test reuse more convenient + Prior to this change the required argument was not shown. - make sure the weekday in Date headers matches the date + curl.1 before: --stderr + curl.1 after: --stderr - test61: replace stray "^M" (5e 4d) at the end of a cookie with a '^M' (0d) + curl --help before: + --stderr Where to redirect stderr - Gets the test working with external proxies like Privoxy again. + curl --help after: + --stderr Where to redirect stderr - Closes #6463 + Closes https://github.com/curl/curl/pull/6692 -- ftp: never set data->set.ftp_append outside setopt +- projects: Update VS projects for OpenSSL 1.1.x - Since the set value then risks getting used like that when the easy - handle is reused by the application. + - Update VS project templates to use the OpenSSL lib names and include + directories for OpenSSL 1.1.x. - Also: renamed the struct field from 'ftp_append' to 'remote_append' - since it is also used for SSH protocols. + This change means the VS project files will now build only with OpenSSL + 1.1.x when an OpenSSL configuration is chosen. Prior to this change the + project files built only with OpenSSL 1.0.x (end-of-life) when an + OpenSSL configuration was chosen. - Closes #6579 - -- urldata: remove the 'rtspversion' field + The template changes in this commit were made by script: - from struct connectdata and the corresponding code in http.c that set - it. It was never used for anything! + libeay32.lib => libcrypto.lib + ssleay32.lib => libssl.lib + ..\..\..\..\..\openssl\inc32 => ..\..\..\..\..\openssl\include - Closes #6581 - -- CURLOPT_QUOTE.3: clarify that libcurl doesn't parse what's sent + And since the output directory now contains the includes it's prepended: + ..\..\..\..\..\openssl\build\Win{32,64}\VC{6..15}\{DLL,LIB} + {Debug,Release}\include - ... so passed in commands may confuse libcurl's knowledge of state. + - Change build-openssl.bat to copy the build's include directory to the + output directory (as seen above). - Reported-by: Bodo Bergmann - Fixes #6577 - Closes #6580 - -- [Jacob Hoffman-Andrews brought this change] - - vtls: factor out Curl_ssl_getsock to field of Curl_ssl + Each build has its own opensslconf.h which is different so we can't just + include the source include directory any longer. - Closes #6558 - -- RELEASE-PROCEDURE: remove old release dates, add new - -- docs/SSL-PROBLEMS: enhanced + Note the include directory in the output directory is a full copy from + the build so technically we don't need to include the OpenSSL source + include directory in the template. However, I left it last in case the + user made a custom OpenSSL build using the old method which would put + opensslconf in the OpenSSL source include directory. - Elaborate on the intermediate cert issue, and mention that anything - below TLS 1.2 is generally considered insecure these days. + - Change build-openssl.bat to use a temporary install directory that is + different from the temporary build directory. - Closes #6572 - -- THANKS: remove a Jon Rumsey dupe - -Daniel Gustafsson (5 Feb 2021) -- [nimaje brought this change] - - docs: fix FILE example url in --metalink documentation + For OpenSSL 1.1.x the temporary paths must be separate not a descendant + of the other, otherwise pdb files will be lost between builds. - In a url after :// follows the possibly empty authority part - till the next /, so that url missed a /. + Ref: https://curl.se/mail/lib-2018-10/0049.html + Ref: https://gist.github.com/jay/125191c35bbeb894444eff827651f755 + Ref; https://github.com/openssl/openssl/issues/10005 - Closes #6573 - Reviewed-by: Daniel Stenberg - Reviewed-by: Daniel Gustafsson + Fixes https://github.com/curl/curl/issues/984 + Closes https://github.com/curl/curl/pull/6675 -Daniel Stenberg (5 Feb 2021) -- hostip: fix build with sync resolver +- doh: Inherit CURLOPT_STDERR from user's easy handle - Reported-by: David Goerger - Follow-up from 8335c6417 - Fixes #6566 - Closes #6568 - -- mailmap: Jon Rumsey - -- [Jon Rumsey brought this change] - - gskit: correct the gskit_send() prototype + Prior to this change if the user set their easy handle's error stream + to something other than stderr it was not inherited by the doh handles, + which meant that they would still write to the default standard error + stream (stderr) for verbose output. - gskit_send() first paramater is a pointer to Curl_easy not connectdata - struct. + Bug: https://github.com/curl/curl/issues/6605 + Reported-by: arvids-kokins-bidstack@users.noreply.github.com - Closes #6570 - Fixes #6569 + Closes https://github.com/curl/curl/pull/6661 -- urldata: fix build without HTTP and MQTT +Marc Hoersken (1 Mar 2021) +- CI/azure: replace python-impacket with python3-impacket - Reported-by: Joseph Chen - Fixes #6562 - Closes #6563 + As of this month Azure DevOps uses Ubuntu 20.04 LTS which + no longer supports Python 2 and instead ships Python 3. + + Closes #6678 -- ftp: avoid SIZE when asking for a TYPE A file +- runtests.pl: kill processes locking test log files - ... as we ignore it anyway because servers don't report the correct size - and proftpd even blatantly returns a 550. + Introduce a new runtests.pl command option: -rm - Updates a set of tests accordingly. + For now only required and implemented for Windows. + Ignore stunnel logs due to long running processes. - Reported-by: awesomenode on github - Fixes #6564 - Closes #6565 - -- pingpong: rename the curl_pp_transfer enum to use PP prefix + Requires Sysinternals handle[64].exe to be on PATH. - Using an FTP prefix for PP provided functionality was misleading. - -- RELEASE-NOTES: synced + Reviewed-by: Jay Satiro - ... and bump pending version to 7.75.1 (for now) + Ref: #6058 + Closes #6179 -Jay Satiro (4 Feb 2021) -- build: fix --disable-http-auth +- pathhelp.pm: fix use of pwd -L in Msys environment - Broken since 215db08 (precedes 7.75.0). + While Msys2 has a pwd binary which supports -L, + Msys1 only has a shell built-in with that feature. - Reported-by: Benbuck Nason + Reviewed-by: Jay Satiro - Fixes https://github.com/curl/curl/issues/6567 + Part of #6179 -- build: fix --disable-dateparse +Daniel Gustafsson (1 Mar 2021) +- ldap: use correct memory free function - Broken since 215db08 (precedes 7.75.0). + unescaped is coming from Curl_urldecode and not a unicode conversion + function, so reclaiming its memory should be performed with a normal + call to free rather than curlx_unicodefree. In reality, this is the + same thing as curlx_unicodefree is implemented as a call to free but + that's not guaranteed to always hold. Using the curlx macro present + issues with memory debugging as well. - Bug: https://curl.se/mail/lib-2021-02/0008.html - Reported-by: Firefox OS + Closes #6671 + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg -Daniel Stenberg (4 Feb 2021) -- [Jon Rumsey brought this change] +- url: fix typo in comment + + Correct a small typo which snuck in with a304051620. - OS400: update for CURLOPT_AWS_SIGV4 +Jay Satiro (28 Feb 2021) +- tool_help: Increase space between option and description - chkstrings fails because a new string option that could require codepage - conversion has been added. + - Increase the minimum number of spaces between the option and the + description from 1 to 2. - Closes #6561 - Fixes #6560 - -- BUG-BOUNTY: removed the cooperation mention - -Version 7.75.0 (3 Feb 2021) + Before: + ~~~ + -u, --user Server user and password + -A, --user-agent Send User-Agent to server + -v, --verbose Make the operation more talkative + -V, --version Show version number and quit + -w, --write-out Use output FORMAT after completion + --xattr Store metadata in extended file attributes + ~~~ + + After: + ~~~ + -u, --user Server user and password + -A, --user-agent Send User-Agent to server + -v, --verbose Make the operation more talkative + -V, --version Show version number and quit + -w, --write-out Use output FORMAT after completion + --xattr Store metadata in extended file attributes + ~~~ + + Closes https://github.com/curl/curl/pull/6674 -Daniel Stenberg (3 Feb 2021) -- RELEASE-NOTES: synced +Daniel Stenberg (27 Feb 2021) +- curl: set CURLOPT_NEW_FILE_PERMS if requested + + The --create-file-mode code logic accepted the value but never actually + passed it on to libcurl! + + Follow-up to a7696c73436f (shipped in 7.75.0) + Reported-by: Johannes Lesr + Fixes #6657 + Closes #6666 -- THANKS: added contributors from 7.75.0 +- tool_operate: check argc before accessing argv[1] + + Follow-up to 09363500b + Reported-by: Emil Engler + Reviewed-by: Daniel Gustafsson + Closes #6668 -- copyright: fix year ranges in need of updates +Daniel Gustafsson (26 Feb 2021) +- [Jean-Philippe Menil brought this change] -- TODO: remove items for next SONAME bump etc + openssl: remove get_ssl_version_txt in favor of SSL_get_version - We want to avoid that completely, so we don't plan for things after such - an event. - -- [Jay Satiro brought this change] + openssl: use SSL_get_version to get connection protocol + + Replace our bespoke get_ssl_version_txt in favor of SSL_get_version. + We can get rid of few lines of code, since SSL_get_version achieve + the exact same thing + + Closes #6665 + Reviewed-by: Daniel Gustafsson + Signed-off-by: Jean-Philippe Menil - ngtcp2: Fix build error due to change in ngtcp2_settings +- gnutls: Fix nettle discovery - - Separate ngtcp2_transport_params. + Commit e06fa7462ac258c removed support for libgcrypt leaving only + support for nettle which has been the default crypto library in + GnuTLS for a long time. There were however a few conditionals on + USE_GNUTLS_NETTLE which cause compilation errors in the metalink + code (as it used the gcrypt fallback instead as a result). See the + below autobuild for an example of the error: - ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from - ngtcp2_settings. + https://curl.se/dev/log.cgi?id=20210225123226-30704#prob1 - ngtcp2 master is required to build curl with http3 support. + This removes all uses of USE_GNUTLS_NETTLE and also removes the + gcrypt support from the metalink code while at it. - Closes #6554 + Closes #6656 + Reviewed-by: Daniel Stenberg -- vtls: remove md5sum +- cookies: Support multiple -b parameters - As it is not used anymore. + Previously only a single -b cookie parameter was supported with the last + one winning. This adds support for supplying multiple -b params to have + them serialized semicolon separated. Both cookiefiles and cookies can be + entered multiple times. - Reported-by: Jacob Hoffman-Andrews - Bug: https://curl.se/mail/lib-2021-02/0000.html + Closes #6649 + Reviewed-by: Daniel Stenberg + +Daniel Stenberg (25 Feb 2021) +- build: remove all traces of USE_BLOCKING_SOCKETS - Closes #6557 + libcurl doesn't behave properly with the define set + + Closes #6655 -- [Alessandro Ghedini brought this change] +- RELEASE-NOTES: synced - quiche: don't use primary_ip / primary_port +Daniel Gustafsson (25 Feb 2021) +- docs: Fix typos - Closes #6555 - -Alessandro Ghedini (1 Feb 2021) -- travis: enable quiche's FFI feature + Random typos spotted when skimming docs. -Daniel Stenberg (30 Jan 2021) -- [Dmitry Wagin brought this change] +- cookies: Use named parameters in header prototypes + + Align header with project style of using named parameters in the + function prototypes to aid readability and self-documentation. + + Closes #6653 + Reviewed-by: Daniel Stenberg - http: improve AWS HTTP v4 Signature auth +Daniel Stenberg (24 Feb 2021) +- urldata: make 'actions[]' use unsigned char instead of int - - Add support services without region and service prefixes in - the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc) - by providing region and service parameters via aws-sigv4 option. - - Add [:region[:service]] suffix to aws-sigv4 option; - - Fix memory allocation errors. - - Refactor memory management. - - Use Curl_http_method instead() STRING_CUSTOMREQUEST. - - Refactor canonical headers generating. - - Remove repeated sha256_to_hex() usage. - - Add some docs fixes. - - Add some codestyle fixes. - - Add overloaded strndup() for debug - curl_dbg_strndup(). - - Update tests. + ... as it only needs a few bits per index anyway. - Closes #6524 + Reviewed-by: Daniel Gustafsson + Closes #6648 -- hyper: fix CONNECT to set 'data' as userdata +- configure: fail if --with-quiche is used and quiche isn't found - Follow-up to 14e075d1a7fd + Closes #6652 -- [Layla brought this change] +- [Gregor Jasny brought this change] - connect: fix compile errors in `Curl_conninfo_local` + cmake: use CMAKE_INSTALL_INCLUDEDIR indirection - .. for the `#else` (`!HAVE_GETSOCKNAME`) case + Reviewed-by: Sergei Nikulov + Closes #6440 + +Viktor Szakats (23 Feb 2021) +- mingw: enable using strcasecmp() - Fixes https://github.com/curl/curl/issues/6548 - Closes #6549 + This makes the 'Features:' list sorted case-insensitively, + bringing output in-line with *nix builds. - Signed-off-by: Layla - -- [Michał Antoniak brought this change] + Reviewed-by: Jay Satiro + Closes #6644 - transfer: fix GCC 10 warning with flag '-Wint-in-bool-context' +- build: delete unused feature guards - ... and return the error code from the Curl_mime_rewind call. + - `HAVE_STRNCASECMP` + - `HAVE_TCGETATTR` + - `HAVE_TCSETATTR` - Closes #6537 - -- [Michał Antoniak brought this change] - - avoid warning: enum constant in boolean context + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes #6645 -- copyright: fix missing year (range) updates +Jay Satiro (23 Feb 2021) +- docs: add CURLOPT_CURLU to 'See also' in curl_url_ functions + + Closes https://github.com/curl/curl/pull/6639 -- RELEASE-NOTES: synced +Daniel Stenberg (23 Feb 2021) +- [Jacob Hoffman-Andrews brought this change] -- openssl: lowercase the hostname before using it for SNI + configure: make hyper opt-in, and fail if missing - ... because it turns out several servers out there don't actually behave - correctly otherwise in spite of the fact that the SNI field is - specifically said to be case insensitive in RFC 6066 section 3. + Previously, configure would look for hyper by default, and use it if + found; otherwise it would not use hyper, and not error. - Reported-by: David Earl - Fixes #6540 - Closes #6543 - -- KNOWN_BUGS: cmake: ExternalProject_Add does not set CURL_CA_PATH + Now, configure will not look for hyper unless --with-hyper is passed. If + configure looks for hyper and fails, it will error. - Closes #6313 - -- KNOWN_BUGS: Multi perform hangs waiting for threaded resolver + Also, add -ld -lpthread -lm to Hyper's libs. I think they are required. - Closes #4852 + Closes #6598 -- KNOWN_BUGS: "pulseUI VPN client" is known to be buggy +- multi: do once-per-transfer inits in before_perform in DID state - First entry in the new section "applications" for known problems in - libcurl using applications. + ... since the state machine might go to RATELIMITING and then back to + PERFORMING doing once-per-transfer inits in that function is wrong and + it caused problems with receiving chunked HTTP and it set the + PRETRANSFER time much too often... - Closes #6306 - -- tool_writeout: make %{errormsg} blank for no errors + Regression from b68dc34af341805aeb7b3715 (shipped in 7.75.0) - Closes #6539 + Reported-by: Amaury Denoyelle + Fixes #6640 + Closes #6641 -Jay Satiro (27 Jan 2021) -- [Gisle Vanem brought this change] +- RELEASE-NOTES: synced - build: fix djgpp builds +- CODE_STYLE.md: fix broken link to INTERNALS - - Update build instructions in packages/DOS/README + ... the link would only work if browsed on GitHub, while this link now + takes the user to the website instead and thus should work on either. - - Extend 'VPATH' with 'vquic' and 'vssh'. + Reported-by: David Demelier + +- curl_url_set.3: mention CURLU_PATH_AS_IS - - Allow 'Makefile.dist' to build both 'lib' and 'src'. + ... it has been supported since the URL API was added. - - Allow using the Windows hosted djgpp cross compiler to build for MSDOS - under Windows. + Bug: https://curl.se/mail/lib-2021-02/0046.html - - 'USE_SSL' -> 'USE_OPENSSL' + Closes #6638 + +Viktor Szakats (21 Feb 2021) +- time: enable 64-bit time_t in supported mingw environments - - Added a 'link_EXE' macro. Etc, etc. + (Unless 32-bit `time_t` is selected manually via the `_USE_32BIT_TIME_T` + mingw macro.) - - Linking 'curl.exe' needs '$(CURLX_CFILES)' too. + Previously, 64-bit `time_t` was enabled on VS2005 and newer only, and + 32-bit `time_t` was used on all other Windows builds. - - Do not pick-up '../lib/djgpp/*.o' files. Recompile locally. + Assisted-by: Jay Satiro + Closes #6636 + +Jay Satiro (20 Feb 2021) +- test1188: Check for --fail HTTP status - - Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'. + - Change the test to check for curl error on HTTP 404 Not Found. - - Remove 'djgpp-clean' + test1188 tests "--write-out with %{onerror} and %{urlnum} to stderr". + Prior to this change it did that by specifying a non-existent host which + would cause an error. ISPs may hijack DNS and resolve non-existent hosts + so the test would not work if that was the case. - - Adapt to new C-ares directory structure + Ref: https://en.wikipedia.org/wiki/DNS_hijacking#Manipulation_by_ISPs + Ref: https://github.com/curl/curl/issues/6621 + Ref: https://github.com/curl/curl/pull/6623 - - Use conditional variable assignments + Closes https://github.com/curl/curl/pull/6637 + +- memdebug: close debug logfile explicitly on exit - Clarify the 'conditional variable assignment' in 'common.dj'. + - Use atexit to register a dbg cleanup function that closes the logfile. - Closes https://github.com/curl/curl/pull/6382 - -Daniel Stenberg (27 Jan 2021) -- [Ikko Ashimine brought this change] - - hyper: fix typo in c-hyper.c + LeakSantizier (LSAN) calls _exit() instead of exit() when a leak is + detected on exit so the logfile must be closed explicitly or data could + be lost. Though _exit() does not call atexit handlers such as this, + LSAN's call to _exit() comes after the atexit handlers are called. - settting -> setting + Prior to this change the logfile was not explicitly closed so it was + possible that if LSAN detected a leak and called _exit (which does + not flush or close files like exit) then the logfile could be missing + data. That could then cause curl's memanalyze to report false leaks + (eg a malloc was recorded to the logfile but the corresponding free was + discarded from the buffer instead of written to the logfile, then + memanalyze reports that as a leak). - Closes #6538 - -- libssh2: fix CURL_LIBSSH2_DEBUG-enabled build + Ref: https://github.com/google/sanitizers/issues/1374 - Follow-up to 2dcc940959772a + Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541 - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/2dcc940959772a652f6813fb6bd3092095a4877b#commitcomment-46420088 + Closes https://github.com/curl/curl/pull/6620 -Jay Satiro (27 Jan 2021) -- asyn-thread: fix build for when getaddrinfo missing +- curl_multibyte: always return a heap-allocated copy of string - This is a follow-up to 8315343 which several days ago moved the resolver - pointer into the async struct but did not update the code that uses it - when getaddrinfo is not present. + - Change the Windows char <-> UTF-8 conversion functions to return an + allocated copy of the passed in string instead of the original. - Closes https://github.com/curl/curl/pull/6536 - -Daniel Stenberg (27 Jan 2021) -- urldata: move 'ints' to the end of 'connectdata' + Prior to this change the curlx_convert_ functions would, as what I + assume was an optimization, not make a copy of the passed in string if + no conversion was required. No conversion is required in non-UNICODE + Windows builds since our tchar strings are type char and remain in + whatever the passed in encoding is, which is assumed to be UTF-8 but may + be other encoding. - To optimize storage slightly. + In contrast the UNICODE Windows builds require conversion + (wchar <-> char) and do return a copy. That inconsistency could lead to + programming errors where the developer expects a copy, and does not + realize that won't happen in all cases. - Closes #6534 + Closes https://github.com/curl/curl/pull/6602 -- urldata: store ip version in a single byte +Viktor Szakats (19 Feb 2021) +- http: add new files missed from referrer commit - Closes #6534 + Ref: 44872aefc2d54f297caf2b0cc887df321bc9d791 + Ref: #6591 -- urldata: remove duplicate 'upkeep_interval_ms' from connectdata +- http: add support to read and store the referrer header - ... and rely only on the value already set in Curl_easy. + - add CURLINFO_REFERER libcurl option + - add --write-out '%{referer}' command-line option + - extend --xattr command-line option to fill user.xdg.referrer.url extended + attribute with the referrer (if there was any) - Closes #6534 + Closes #6591 -- urldata: remove 'local_ip' from the connectdata struct +Daniel Stenberg (19 Feb 2021) +- urldata: remove the _ORIG suffix from string names - As the info is already stored in the transfer handle anyway, there's no - need to carry around a duplicate buffer for the life-time of the handle. + It doesn't provide any useful info but only makes the names longer. - Closes #6534 + Closes #6624 -- urldata: remove duplicate port number storage +- url: fix memory leak if OOM in the HSTS handling - ... and use 'int' for ports. We don't use 'unsigned short' since -1 is - still often used internally to signify "unknown value" and 0 - 65535 are - all valid port numbers. + Reported-by: Viktor Szakats + Bug: https://github.com/curl/curl/pull/6627#issuecomment-781626205 - Closes #6534 + Closes #6628 -- urldata: remove the duplicate 'ip_addr_str' field +- gnutls: assume nettle crypto support - ... as the numerical IP address is already stored and kept in 'primary_ip'. + nettle has been the default crypto library with GnuTLS since 2010. By + dropping support for the previous libcrypto, we simplify code. - Closes #6534 + Closes #6625 -- select: convert Curl_select() to private static function +- asyn-ares: use consistent resolve error message - The old function should not be used anywhere anymore (the only remaining - gskit use has to be fixed to instead use Curl_poll or none at all). + ... with the help of Curl_resolver_error() which now is moved from + asyn-thead.c and is provided globally for this purpose. - The static function version is now called our_select() and is only built - if necessary. + Follow-up to 35ca04ce1b77636 - Closes #6531 + Makes test 1188 work for c-ares builds + + Closes #6626 -- Curl_chunker: shrink the struct +Viktor Szakats (18 Feb 2021) +- ci: stop building on freebsd-12-1 - ... by removing a field, converting the hex index into a byte and - rearranging the order. Cuts it down from 48 bytes to 32 on x86_64. + An updated freebsd-12-2 image was added a few months ago, and this + older one is consistently failing to go past `pkginstall`: + ``` + Newer FreeBSD version for package py37-mlt: + To ignore this error set IGNORE_OSVERSION=yes + - package: 1202000 + - running kernel: 1201000 + Ignore the mismatch and continue? [Y/n]: pkg: repository FreeBSD contains packages for wrong OS version: FreeBSD:12:amd64 + ``` - Closes #6527 - -- curl: include the file name in --xattr/--remote-time error msgs - -- curl: s/config->global/global/ in single_transfer() - -- curl: move fprintf outputs to warnf + FreeBSD thread suggests that 12.1 is EOL, and best to avoid. - For setting and getting time of the download. To make the outputs - respect --silent etc. + Ref: https://forums.freebsd.org/threads/78856/ - Reported-by: Viktor Szakats - Fixes #6533 - Closes #6535 - -- [Tatsuhiro Tsujikawa brought this change] + Reviewed-by: Daniel Stenberg + Closes #6622 - ngtcp2: Fix http3 upload stall +Daniel Stenberg (18 Feb 2021) +- test1188: change error from connect to resolve error - Closes #6521 + Using the %NOLISTENPORT to trigger a connection failure is somewhat + "risky" (since it isn't guaranteed to not be listened to) and caused + occasional CI problems. This fix changes the infused error to be a more + reliable one but still verifies the --write-out functionality properly - + which is the purpose of this test. + + Reported-by: Jay Satiro + Fixes #6621 + Closes #6623 -- [Tatsuhiro Tsujikawa brought this change] +- url.c: use consistent error message for failed resolve - ngtcp2: Fix stack buffer overflow +- BUGS: language polish + +- wolfssl: don't store a NULL sessionid - Closes #6521 + This caused a memory leak as the session id cache entry was still + erroneously stored with a NULL sessionid and that would later be treated + as not needed to get freed. + + Reported-by: Gisle Vanem + Fixes #6616 + Closes #6617 -- warnless.h: remove the prototype for curlx_ultosi +- parse_proxy: fix a memory leak in the OOM path - Follow-up to 217552503ff3 + Reported-by: Jay Satiro + Reviewed-by: Jay Satiro + Reviewed-by: Emil Engler + + Closes #6614 + Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541 -- warnless: remove curlx_ultosi +Jay Satiro (17 Feb 2021) +- url: fix possible use-after-free in default protocol - ... not used anywhere + Prior to this change if the user specified a default protocol and a + separately allocated non-absolute URL was used then it was freed + prematurely, before it was then used to make the replacement URL. - Closes #6530 - -- [Patrick Monnerat brought this change] - - lib: remove conn->data uses + Bug: https://github.com/curl/curl/issues/6604#issuecomment-780138219 + Reported-by: arvids-kokins-bidstack@users.noreply.github.com - Closes #6515 + Closes https://github.com/curl/curl/pull/6613 -- pingpong: remove the 'conn' struct member +Daniel Stenberg (16 Feb 2021) +- multi: rename the multi transfer states - ... as it's superfluous now when Curl_easy is passed in and we can - derive the connection from that instead and avoid the duplicate copy. + While working on documenting the states it dawned on me that step one is + to use more descriptive names on the states. This also changes prefix on + the states to make them shorter in the source. - Closes #6525 + State names NOT ending with *ing are transitional ones. + + Closes #6612 -- hostip/proxy: remove conn->data use +Viktor Szakats (16 Feb 2021) +- http: do not add a referrer header with empty value - Closes #6513 + Previously an empty 'Referer:' header was added to the HTTP request when + passing `--referer ';auto'` or `--referer ''` on the command-line. This + patch makes `--referer` work like `--header 'Referer:'` and will only add + the header if it has a non-zero length value. + + Reviewed-by: Jay Satiro + Closes #6610 -- url: reduce conn->data references +Daniel Stenberg (16 Feb 2021) +- lib: remove 'conn->data' completely - ... there are a few left but let's keep them to last + The Curl_easy pointer struct entry in connectdata is now gone. Just + before commit 215db086e0 landed on January 8, 2021 there were 919 + references to conn->data. - Closes #6512 + Closes #6608 -- scripts/singleuse: add curl_easy_option* +- openldap: pass 'data' to the callbacks instead of 'conn' -Jay Satiro (25 Jan 2021) -- test410: fix for windows +Jay Satiro (15 Feb 2021) +- doh: Fix sharing user's resolve list with DOH handles - - Pass the very long request header via file instead of command line. + - Share the shared object from the user's easy handle with the DOH + handles. - Prior to this change the 49k very long request header string was passed - via command line and on Windows that is too long so it was truncated and - the test would fail (specifically msys CI). + Prior to this change if the user had set a shared object with shared + cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't + used by any associated DOH handles, since they used the multi's default + hostcache. - Closes https://github.com/curl/curl/pull/6516 - -Daniel Stenberg (25 Jan 2021) -- libssh2: move data from connection object to transfer object + This change means all the handles now use the same hostcache, which is + either the shared hostcache from the user created shared object if it + exists or if not then the multi's default hostcache. - Readdir data, filenames and attributes are strictly related to the - transfer and not the connection. This also reduces the total size of the - fixed connectdata struct. + Reported-by: Manuj Bhatia - Closes #6519 - -- RELEASE-NOTES: synced - -- [Patrick Monnerat brought this change] + Fixes https://github.com/curl/curl/issues/6589 + Closes https://github.com/curl/curl/pull/6607 - lib: remove conn->data uses +Daniel Stenberg (15 Feb 2021) +- http2: remove conn->data use - Closes #6499 - -- hyper: remove the conn->data references + ... but instead use a private alternative that points to the "driving + transfer" from the connection. We set the "user data" associated with + the connection to be the connectdata struct, but when we drive transfers + the code still needs to know the pointer to the transfer. We can change + the user data to become the Curl_easy handle, but with older nghttp2 + version we cannot dynamically update that pointer properly when + different transfers are used over the same connection. - Closes #6508 + Closes #6520 -- travis: build ngtcp2 --with-gnutls +- openssl: remove conn->data use - ... since they disable it by default since a few days back. + We still make the trace callback function get the connectdata struct + passed to it, since the callback is anchored on the connection. - Closes #6506 - Fixes #6493 - -- hostip: remove conn->data from resolver functions + Repeatedly updating the callback pointer to set 'data' with + SSL_CTX_set_msg_callback_arg() doesn't seem to work, probably because + there might already be messages in the queue with the old pointer. - This also moves the 'async' struct from the connectdata struct into the - Curl_easy struct, which seems like a better home for it. + This code therefore makes sure to set the "logger" handle before using + OpenSSL calls so that the right easy handle gets used for tracing. - Closes #6497 + Closes #6522 -Jay Satiro (22 Jan 2021) -- strerror: skip errnum >= 0 assertion on windows +- RELEASE-NOTES: synced + +Jay Satiro (14 Feb 2021) +- doh: add options to disable ssl verification - On Windows an error number may be greater than INT_MAX and negative once - cast to int. + - New libcurl options CURLOPT_DOH_SSL_VERIFYHOST, + CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the + same as their respective counterparts. - The assertion is checked only in debug builds. + - New curl tool options --doh-insecure and --doh-cert-status do the same + as their respective counterparts. - Closes https://github.com/curl/curl/pull/6504 - -Daniel Stenberg (21 Jan 2021) -- doh: make Curl_doh_is_resolved survive a NULL pointer + Prior to this change DOH SSL certificate verification settings for + verifyhost and verifypeer were supposed to be inherited respectively + from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug + were not. As a result DOH verification remained at the default, ie + enabled, and it was not possible to disable. This commit changes + behavior so that the DOH verification settings are independent and not + inherited. - ... if Curl_doh() returned a NULL, this function gets called anyway as - in a asynch procedure. Then the doh struct pointer is NULL and signifies - an OOM situation. + Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676 - Follow-up to 6246a1d8c6776 + Fixes https://github.com/curl/curl/issues/4578 + Closes https://github.com/curl/curl/pull/6597 -- wolfssh: remove conn->data references - - ... and repair recent build breakage +- hostip: fix crash in sync resolver builds that use DOH - Closes #6507 - -- http: empty reply connection are not left intact + - Guard some Curl_async accesses with USE_CURL_ASYNC instead of + !CURLRES_SYNCH. - ... so mark the connection as closed in this condition to prevent that - verbose message to wrongly appear. + This is another follow-up to 8335c64 which moved the async struct from + the connectdata struct into the Curl_easy struct. A previous follow-up + 6cd167a fixed building for sync resolver by guarding some async struct + accesses with !CURLRES_SYNCH. The problem is since DOH (DNS-over-HTTPS) + is available as an asynchronous secondary resolver the async struct may + be used even when libcurl is built for the sync resolver. That means + that CURLRES_SYNCH and USE_CURL_ASYNC may be defined at the same time. - Reported-by: Matt Holt - Bug: https://twitter.com/mholt6/status/1352130240265375744 - Closes #6503 + Closes https://github.com/curl/curl/pull/6603 -- chunk/encoding: remove conn->data references - - ... by anchoring more functions on Curl_easy instead of connectdata +Daniel Stenberg (13 Feb 2021) +- KNOWN_BUGS: cannot enable LDAPS on Windows with cmake - Closes #6498 + Reported-by: Jack Boos Yu + Closes #6284 -Jay Satiro (20 Jan 2021) -- [Erik Olsson brought this change] +- KNOWN_BUGS: Excessive HTTP/2 packets with TCP_NODELAY + + Reported-by: Alex Xu + Closes #6363 - lib: save a bit of space with some structure packing +- http: use credentials from transfer, not connection - - Reorder some internal struct members so that less padding is used. + HTTP auth "accidentally" worked before this cleanup since the code would + always overwrite the connection credentials with the credentials from + the most recent transfer and since HTTP auth is typically done first + thing, this has not been an issue. It was still wrong and subject to + possible race conditions or future breakage if the sequence of functions + would change. - This is an attempt at saving a bit of space by packing some structs - (using pahole to find the holes) where it might make sense to do - so without losing readability. + The data.set.str[] strings MUST remain unmodified exactly as set by the + user, and the credentials to use internally are instead set/updated in + state.aptr.* - I.e., I tried to avoid separating fields that seem grouped - together (like the cwd... fields in struct ftp_conn for instance). - Also abstained from touching fields behind conditional macros as - that quickly can get complicated. + Added test 675 to verify different credentials used in two requests done + over a reused HTTP connection, which previously behaved wrongly. - Closes https://github.com/curl/curl/pull/6483 + Fixes #6542 + Closes #6545 -Daniel Stenberg (20 Jan 2021) -- INSTALL.md: fix typo +- test433: clear some home dir env variables - Found-by: Marcel Raad - -- [Fabian Keil brought this change] - - http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy + Follow-up to bd6b54ba1f55b5 - Added test 1613 to verify. + ... so that XDG_CONFIG_HOME is the only home dir variable set and thus + used correctly in the test! - Closes #6490 - -- Merge branch 'bagder/curl_range-data-conn' - -- ftp: remove conn->data leftover + Fixes #6599 + Closes #6600 -- curl_range: remove conn->data +- RELEASE-NOTES: synced - Closes #6496 - -- INSTALL: now at 85 operating systems + bumped the version to 7.76.0 -- quiche: fix unused parameter ‘conn’ +- travis: install libgsasl-dev to add that to the builds - Follow-up to 2bdec0b3 + Closes #6588 -- transfer: fix ‘conn’ undeclared mistake for iconv build +- urldata: don't touch data->set.httpversion at run-time - Follow-up to 219d9f8620d - -- doh: allocate state struct on demand + Rename it to 'httpwant' and make a cloned field in the state struct as + well for run-time updates. - ... instead of having it static within the Curl_easy struct. This takes - away 1176 bytes (18%) from the Curl_easy struct that aren't used very - often and instead makes the code allocate it when needed. + Also: refuse non-supported HTTP versions. Verified with test 129. - Closes #6492 + Closes #6585 -- socks: use the download buffer instead +Viktor Szakats (11 Feb 2021) +- tests: disable .curlrc in more environments - The SOCKS code now uses the generic download buffer for temporary - storage during the connection procedure, instead of having its own - private 600 byte buffer that adds to the connectdata struct size. This - works fine because this point the buffer is allocated but is not use for - download yet since the connection hasn't completed. + by also setting CURL_HOME and XDG_CONFIG_HOME envvars to the local + directory. - This reduces the connection struct size by 22% on a 64bit arch! + Reviewed-by: Daniel Stenberg + Fixes #6595 + Closes #6596 + +- docs/Makefile.inc: format to be update-friendly - The SOCKS buffer needs to be at least 600 bytes, and the download buffer - is guaranteed to never be smaller than 1000 bytes. + - one source file per line + - convert tabs to spaces + - do not align line-continuation backslashes + - sort source files alphabetically - Closes #6491 + Reviewed-by: Daniel Stenberg + Closes #6593 -- urldata: make magic be the first struct field +Daniel Stenberg (11 Feb 2021) +- curl: provide libgsasl version and feature info in -V output - By making the `magic` identifier the same size and at the same place - within the structs (easy, multi, share), libcurl will be able to more - reliably detect and safely error out if an application passes in the - wrong handle to APIs. Easier to detect and less likely to cause crashes - if done. + Closes #6592 + +- gsasl: provide CURL_VERSION_GSASL if built-in - Such mixups can't be detected at compile-time due to them being - typedefed void pointers - unless `CURL_STRICTER` is defined. + To let applications know the feature is available. - Closes #6484 + Closes #6592 -- http_chunks: correct and clarify a comment on hexnumber length +- curl: add --fail-with-body - ... and also rename the define for max length. + Prevent both --fail and --fail-with-body on the same command line. - Closes #6489 - -- curl_path: remove conn->data use + Verify with test 349, 360 and 361. - Closes #6487 + Closes #6449 -- transfer: remove conn->data use +- TODO: remove HSTS - Closes #6486 + Provided now since commit 7385610d0c74 -- quic: remove conn->data use +Jay Satiro (10 Feb 2021) +- tests: Fix tests failing due to change in curl --help - Closes #6485 - -- [Fabian Keil brought this change] - - Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" - -- [Fabian Keil brought this change] - - Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive" + Follow-up to parent 3183217 which added add missing argument to + --create-file-mode . - At the moment the test fails as curl sends two Proxy-Connection - headers. - -- c-hyper: avoid duplicated Proxy-Connection headers + Ref: https://github.com/curl/curl/issues/6590 -- http: make providing Proxy-Connection header not cause duplicated headers +- tool_help: add missing argument for --create-file-mode - Fixes test 1180 + Prior to this change the required argument was not shown in curl --help. - Bug: https://curl.se/mail/lib-2021-01/0095.html - Reported-by: Fabian Keil - Closes #6472 - -- runtests: preprocess DISABLED to allow conditionals + before: + --create-file-mode File mode for created files - ... with this function provided, we can disable tests for specific - environments and setups directly within this file. + after: + --create-file-mode File mode (octal) for created files - Closes #6477 - -- runtests: turn preprocessing into a separate function + Reported-by: ZimCodes@users.noreply.github.com - ... and remove all other variable substitutions as they're now done once - and for all in the preprocessor. + Fixes https://github.com/curl/curl/issues/6590 -- lib/Makefile.inc: convert to listing each file on its own line +- create-file-mode.d: add missing Arg tag - ... to make it diff friendlier and easier to read. + Prior to this change the required argument was not shown. - Closes #6448 - -- ftplistparser: remove use of conn->data + curl.1 before: --create-file-mode + curl.1 after: --create-file-mode - Closes #6482 - -- lib: more conn->data cleanups + Reported-by: ZimCodes@users.noreply.github.com - Closes #6479 + Fixes https://github.com/curl/curl/issues/6590 -- [Patrick Monnerat brought this change] +Viktor Szakats (10 Feb 2021) +- gsasl: fix errors/warnings building against libgsasl + + - also fix an indentation + - make Curl_auth_gsasl_token() use CURLcode (by Daniel Stenberg) + + Ref: https://github.com/curl/curl/pull/6372#issuecomment-776118711 + Ref: https://github.com/curl/curl/pull/6588 + + Reviewed-by: Jay Satiro + Assisted-by: Daniel Stenberg + Reviewed-by: Simon Josefsson + Closes #6587 - vtls: reduce conn->data use +- Makefile.m32: add support for libgsasl dependency - Closes #6474 + Reviewed-by: Marcel Raad + Closes #6586 -- hyper: deliver data to application with Curl_client_write +Marcel Raad (10 Feb 2021) +- ngtcp2: clarify calculation precedence - ... just as the native code path does. Avoids sending too large data - chunks in the callback and more. + As suggested by Codacy/cppcheck. - Reported-by: Gisle Vanem - Fixes #6462 - Closes #6473 - -- gopher: remove accidental conn->data leftover + Closes https://github.com/curl/curl/pull/6576 -- libssh: avoid plain free() of libssh-memory +- server: remove redundant condition - Since curl's own memory debugging system redefines free() calls to track - and fiddle with memory, it cannot be used on memory allocated by 3rd - party libraries. + `end` is always non-null here. - Third party libraries SHOULD NOT require free() to release allocated - resources for this reason - and libs can use separate healp allocators - on some systems (like Windows) so free() doesn't necessarily work - anyway. + Closes https://github.com/curl/curl/pull/6576 + +- lib: remove redundant code - Filed as an issue with libssh: https://bugs.libssh.org/T268 + Closes https://github.com/curl/curl/pull/6576 + +- mqttd: remove unused variable - Closes #6481 + Closes https://github.com/curl/curl/pull/6576 -- send: assert that Curl_write_plain() has a ->conn when called +- tool_paramhlp: reduce variable scope - To help catch bad invokes. + Closes https://github.com/curl/curl/pull/6576 + +- tests: reduce variable scopes - Closes #6476 + Closes https://github.com/curl/curl/pull/6576 -- test410: verify HTTPS GET with a 49K request header +- lib: reduce variable scopes - skip test 410 for mesalink in the CI as it otherwise hangs "forever" + Closes https://github.com/curl/curl/pull/6576 -- lib: pass in 'struct Curl_easy *' to most functions +- ftp: fix Codacy/cppcheck warning about null pointer arithmetic - ... in most cases instead of 'struct connectdata *' but in some cases in - addition to. + Increment `bytes` only if it is non-null. - - We mostly operate on transfers and not connections. + Closes https://github.com/curl/curl/pull/6576 + +Daniel Stenberg (9 Feb 2021) +- ngtcp2: adapt to the new recv_datagram callback + +- quiche: fix build error: use 'int' for port number - - We need the transfer handle to log, store data and more. Everything in - libcurl is driven by a transfer (the CURL * in the public API). + Follow-up to cb2dc1ba8 + +- ftp: add 'list_only' to the transfer state struct - - This work clarifies and separates the transfers from the connections - better. + and rename it from 'ftp_list_only' since it is also used for SSH and + POP3. The state is updated internally for 'type=D' FTP URLs. - - We should avoid "conn->data". Since individual connections can be used - by many transfers when multiplexing, making sure that conn->data - points to the current and correct transfer at all times is difficult - and has been notoriously error-prone over the years. The goal is to - ultimately remove the conn->data pointer for this reason. + Added test case 1570 to verify. - Closes #6425 + Closes #6578 -Emil Engler (17 Jan 2021) -- docs: fix typos in NEW-PROTOCOL.md +- ftp: add 'prefer_ascii' to the transfer state struct - This fixes a misspelled "it" and a grammatically wrong "-ing" suffix. + ... and make sure the code never updates 'set.prefer_ascii' as it breaks + handle reuse which should use the setting as the user specified it. - Closes #6471 + Added test 1569 to verify: it first makes an FTP transfer with ';type=A' + and then another without type on the same handle and the second should + then use binary. Previously, curl failed this. + + Closes #6578 -Daniel Stenberg (16 Jan 2021) - RELEASE-NOTES: synced -Jay Satiro (16 Jan 2021) -- [Razvan Cojocaru brought this change] +- [Jacob Hoffman-Andrews brought this change] - cmake: expose CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG - - This does for cmake builds what --disable-openssl-auto-load-config - does for autoconf builds. + vtls: initial implementation of rustls backend - Closes https://github.com/curl/curl/pull/6435 - -Daniel Stenberg (15 Jan 2021) -- test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id() + This adds a new TLS backend, rustls. It uses the C-to-rustls bindings + from https://github.com/abetterinternet/crustls. - ... and as a practical side-effect, make sure that the - Curl_easyopts_check() function is asserted in debug builds, which we - want to detect mismatches between the options list in easyoptions.c and - the options in curl.h + Rustls is at https://github.com/ctz/rustls/. - Found-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45991815 + There is still a fair bit to be done, like sending CloseNotify on + connection shutdown, respecting CAPATH, and properly indicating features + like "supports TLS 1.3 ciphersuites." But it works well enough to make + requests and receive responses. - Closes #6461 - -- [Gisle Vanem brought this change] - - easyoptions: add the missing AWS_SIGV4 + Blog post for context: + https://www.abetterinternet.org/post/memory-safe-curl/ - Follow-up from AWS_SIGV4 + Closes #6350 -- schannel_verify: fix safefree call typo - - Follow-up from e87ad71d1ba00519 - - Closes #6459 +- [Simon Josefsson brought this change] -- mime: make sure setting MIMEPOST to NULL resets properly - - ... so that a function can first use MIMEPOST and then set it to NULL to - reset it back to a blank POST. + sasl: support SCRAM-SHA-1 and SCRAM-SHA-256 via libgsasl - Added test 584 to verify the fix. + Closes #6372 + +Jay Satiro (9 Feb 2021) +- lib: use int type for more port variables - Reported-by: Christoph M. Becker + This is a follow-up to 764c6bd. Prior to that change port variables + were usually type long. - Fixes #6455 - Closes #6456 + Closes https://github.com/curl/curl/pull/6553 -- multi: set the PRETRANSFER time-stamp when we switch to PERFORM +- tool_writeout: refactor write-out and write-out json - ... instead of at end of the DO state. This makes the timer more - accurate for the protocols that use the DOING state (such as FTP), and - simplifies how the function (now called init_perform) is called. + - Deduplicate the logic used by write-out and write-out json. - The timer will then include the entire procedure up to PERFORM - - including all instructions for getting the transfer started. + Rather than have separate writeLong, writeString, etc, logic for + each of write-out and write-out json instead have respective shared + functions that can output either format and a 'use_json' parameter to + indicate whether it is json that is output. - Closes #6454 - -- CURLINFO_PRETRANSFER_TIME.3: clarify + This will make it easier to maintain. Rather than have to go through + two sets of logic now we only have to go through one. - ... the timer *does* include the instructions for getting the remote - file. + - Support write-out %{errormsg} and %{exitcode} in json. - Ref: #6452 - Closes #6453 - -- [Gisle Vanem brought this change] - - schannel: plug a memory-leak + - Clarify in the doc that %{exitcode} is the exit code of the transfer. - ... when built without -DUNICODE. + Prior to this change it just said "The numerical exitcode" which + implies it's the exit code of the tool, and it's not necessarily that. - Closes #6457 + Closes https://github.com/curl/curl/pull/6544 -Jay Satiro (14 Jan 2021) -- gitattributes: Set batch files to CRLF line endings on checkout - - If a batch file is run without CRLF line endings (ie LF-only) then - arbitrary behavior may occur. I consider that a bug in Windows, however - the effects can be serious enough (eg unintended code executed) that - we're fixing it in the repo by requiring CRLF line endings for batch - files on checkout. +- lib: drop USE_SOCKETPAIR in favor of CURL_DISABLE_SOCKETPAIR - Prior to this change the checked-out line endings of batch files were - dependent on a user's git preferences. On Windows it is common for git - users to have automatic CRLF conversion enabled (core.autocrlf true), - but those users that don't would run into this behavior. + .. since the former is undocumented and they both do the same thing. - For example a user has reported running the Visual Studio project - generator batch file (projects/generate.bat) and it looped forever. - Output showed that the Windows OS interpreter was occasionally jumping - to arbitrary points in the batch file and executing commands. This - resulted in unintended files being removed (a removal sequence called) - and looping forever. + Closes https://github.com/curl/curl/pull/6517 + +- curl_multibyte: fall back to local code page stat/access on Windows - Ref: https://serverfault.com/q/429594 - Ref: https://stackoverflow.com/q/232651 - Ref: https://www.dostips.com/forum/viewtopic.php?t=8988 - Ref: https://git-scm.com/docs/gitattributes#_checking_out_and_checking_in - Ref: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_core_autocrlf + If libcurl is built with Unicode support for Windows then it is assumed + the filename string is Unicode in UTF-8 encoding and it is converted to + UTF-16 to be passed to the wide character version of the respective + function (eg wstat). However the filename string may actually be in the + local encoding so, even if it successfully converted to UTF-16, if it + could not be stat/accessed then try again using the local code page + version of the function (eg wstat fails try stat). - Bug: https://github.com/curl/curl/discussions/6427 - Reported-by: Ganesh Kamath + We already do this with fopen (ie wfopen fails try fopen), so I think it + makes sense to extend it to stat and access functions. - Closes https://github.com/curl/curl/pull/6442 + Closes https://github.com/curl/curl/pull/6514 -Daniel Stenberg (14 Jan 2021) -- tool_operate: spellfix a comment +- [Stephan Szabo brought this change] -- ROADMAP: refreshed + file: Support unicode urls on windows - o removed HSTS - already implemented - o added HTTPS RR records - o mention HTTP/3 completion - -- http_chunks: remove Curl_ prefix from static functions - -- transfer: remove Curl_ prefix from static functions - -- tftp: remove Curl_ prefix from static functions - -- multi: remove Curl_ prefix from static functions - -- ldap: remove Curl_ prefix from static functions - -- doh: remove Curl_ prefix from static functions - -- asyn-ares: remove Curl_ prefix from static functions - -- vtls: remove Curl_ prefix from static functions - -- bearssl: remove Curl_ prefix from static functions - -- mbedtls: remove Curl_ prefix from static functions - -- wolfssl: remove Curl_ prefix from static functions - -- nss: remove Curl_ prefix from static functions - -- gnutls: remove Curl_ prefix from static functions + Closes https://github.com/curl/curl/pull/6501 -- openssl: remove Curl_ prefix from static functions - - ... as we reserve this prefix to library-wide functions. - - Closes #6443 +- [Vincent Torri brought this change] -- nss: get the run-time version instead of build-time + cmake: fix import library name for non-MS compiler on Windows - Closes #6445 - -Jay Satiro (12 Jan 2021) -- tool_doswin: Restore original console settings on CTRL signal + - Use _imp.lib suffix only for Microsoft's compiler (MSVC). - - Move Windows terminal init code from tool_main to tool_doswin. + Prior to this change library suffix _imp.lib was used for the import + library on Windows regardless of compiler. - - Restore the original console settings on CTRL+C and CTRL+BREAK. + With this change the other compilers should now use their default + suffix which should be .dll.a. - Background: On Windows the curl tool changes the console settings to - enable virtual terminal processing (eg color output) if supported - (ie Win 10). The original settings are restored on exit but prior to - this change were not restored in the case of the CTRL signals. + This change is motivated by the usage of pkg-config on MSYS2. + Indeed, when 'pkg-config --libs libcurl' is used, -lcurl is + passed to ld. The documentation of ld on Windows : - Windows VT behavior varies depending on console/powershell/terminal; - refer to the discussion in #6226. + https://sourceware.org/binutils/docs/ld/WIN32.html - Assisted-by: Rich Turner + lists, in the 'direct linking to a dll' section, the pattern + of the searched import library, and libcurl_imp.lib is not there. - Closes https://github.com/curl/curl/pull/6226 + Closes https://github.com/curl/curl/pull/6225 -Daniel Stenberg (12 Jan 2021) -- gen.pl: fix perl syntax +Daniel Stenberg (9 Feb 2021) +- urldata: move 'followlocation' to UrlState - Follow-up to 324cf1d2e + As this is a state variable it does not belong in UserDefined which is + used to store values set by the user. + + Closes #6582 -- [Emil Engler brought this change] +- [Ikko Ashimine brought this change] - help: update to current codebase + http_proxy: fix typo in http_proxy.c - This commit bumps the help to the current state of the project. + settting -> setting - Closes #6437 + Closes #6583 -- [Emil Engler brought this change] +- [Fabian Keil brought this change] - docs: fix line length bug in gen.pl + tests/server: Bump MAX_TAG_LEN to 200 - The script warns if the length of $opt and $desc is > 78. However, these - two variables are on totally separate lines so the check makes no sense. - Also the $bitmask field is totally forgotten. Currently this leads to - two warnings within `--resolve` and `--aws-sigv4`. + This is useful for tests containing HTML inside of sections. + For tags it's not uncommon to be longer than the previous + limit of 79 bytes. - Closes #6438 + An example of a previously problem-causing tag is: + + which is needed for a Privoxy test for the banners-by-size filter. + + Previously it caused server failures like: + 12:29:05.786961 ====> Client connect + 12:29:05.787116 accept_connection 3 returned 4 + 12:29:05.787194 accept_connection 3 returned 0 + 12:29:05.787285 Read 119 bytes + 12:29:05.787345 Process 119 bytes request + 12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1 + 12:29:05.787464 Requested test number 9 part 0 + 12:29:05.787686 getpart() failed with error: -2 + 12:29:05.787744 - request found to be complete (9) + 12:29:05.787912 getpart() failed with error: -2 + 12:29:05.788048 Wrote request (119 bytes) input to log/server.input + 12:29:05.788157 Send response test9 section + 12:29:05.788443 getpart() failed with error: -2 + 12:29:05.788498 instructed to close connection after server-reply + 12:29:05.788550 ====> Client disconnect 0 + 12:29:05.871448 exit_signal_handler: 15 + 12:29:05.871714 signalled to die + 12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15) -- [Emil Engler brought this change] +- [Fabian Keil brought this change] - docs: fix wrong documentation in help.d - - curl does not list all categories when you invoke "--help" without any - parameters. - - Closes #6436 + tests/badsymbols.pl: when opening '$incdir' fails include it in the error message -- aws-sigv4.d: polish the wording - - Make it shorter and imperative form - - Closes #6439 +- [Fabian Keil brought this change] + + runtests.1: document -o, -P, -L, and -E - [Fabian Keil brought this change] - misc: fix typos - - Bug: https://curl.se/mail/lib-2021-01/0063.html - Closes #6434 + runtests.pl: add %TESTNUMBER variable to make copying tests more convenient -- multi_runsingle: bail out early on data->conn == NULL +- [Fabian Keil brought this change] + + runtests.pl: add an -o option to change internal variables - As that's a significant error condition and scan-build warns for NULL - pointer dereferences if we don't. + runtests.pl has lots of internal variables one might want to + change in certain situations, but adding a dedicated option + for every single one of them isn't practical. - Closes #6433 + Usage: + ./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ... -- multi: skip DONE state if there's no connection left for ftp wildcard +- [Fabian Keil brought this change] + + runtests.pl: cleanups - ... to avoid running in that state with data->conn being NULL. + - show the summarized test result in the last line of the report + - do not use $_ after mapping it to a named variable + Doing that makes the code harder to follow. + - log the restraints sorted by the number of their occurrences + - fix language when logging restraints that only occured once + - let runhttpserver() use $TESTDIR instead of $srcdir + ... so it works if a non-default $TESTDIR is being used. -- libssh2: fix "Value stored to 'readdir_len' is never read" - - Detected by scan-build +- [Fabian Keil brought this change] -- connect: mark intentional ignores of setsockopt return values - - Pointed out by Coverity + runtests.pl: add an -E option to specify an exclude file - Closes #6431 - -Jay Satiro (11 Jan 2021) -- http_proxy: Fix CONNECT chunked encoding race condition + It can contain additional restraints for test numbers, + keywords and tools. - - During the end-of-headers response phase do not mark the tunnel - complete unless the response body was completely parsed/ignored. + The idea is to let third parties like the Privoxy project + distribute an exclude file with their tarballs that specifies + which curl tests are not expected to work when using Privoxy + as a proxy, without having to fork the whole curl test suite. - Prior to this change if the entirety of a CONNECT response with chunked - encoding was not received by the time the final header was parsed then - the connection would be marked done prematurely, before all the chunked - data could be read in and ignored (since this is what we do with any - CONNECT response body) and the connection could not be used. + The syntax could be changed to be extendable and maybe + more closely reflect the "curl test" syntax. Currently + it's a bunch of lines like these: - Bug: https://curl.se/mail/lib-2021-01/0033.html - Reported-by: Fabian Keil + test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped + keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped + tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped - Closes https://github.com/curl/curl/pull/6432 + To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs + on a single line, split them with commas. -Daniel Stenberg (11 Jan 2021) -- RELEASE-NOTES: synced +- [Fabian Keil brought this change] -- url: if IDNA conversion fails, fallback to Transitional + runtests.pl: add -L parameter to require additional perl libraries - This improves IDNA2003 compatiblity. + This is useful to change the behaviour of the script without + having to modify the file itself, for example to use a custom + compareparts() function that ignores header differences that + are expected to occur when an external proxy is being used. - Reported-by: Bubu on github - Fixes #6423 - Closes #6428 + Such differences are proxy-specific and thus the modifications + should be maintained together with the proxy. -- travis: make the Hyper build from its master branch - - Closes #6430 +- [Fabian Keil brought this change] -- http: make 'authneg' also work for Hyper - - When doing a request with a request body expecting a 401/407 back, that - initial request is sent with a zero content-length. Test 177 and more. + runtests.pl: add a -P option to specify an external proxy - Closes #6424 - -Jay Satiro (8 Jan 2021) -- cmake: Add an option to disable libidn2 + ... that should be used when executing the tests. - New option USE_LIBIDN2 defaults to ON for libidn2 detection. Prior to - this change libidn2 detection could not be turned off in cmake builds. + The assumption is that the proxy is an HTTP proxy. - Reported-by: William A Rowe Jr + This option should be used together with -L to provide + a customized compareparts() version that knows which + proxy-specific header differences should be ignored. - Fixes https://github.com/curl/curl/issues/6361 - Closes https://github.com/curl/curl/pull/6362 + This option doesn't work for all test types yet. -Daniel Stenberg (8 Jan 2021) -- HYPER: no longer needs the special branch +- [Fabian Keil brought this change] -- test179: use consistent header line endings + tests: fixup several tests - ... to make "Hyper mode" work better. - -- file: don't provide content-length for directories + missing CRs and modified %hostip - ... as it is misleading. + lib556/test556: use a real HTTP version to make test reuse more convenient - Ref #6379 - Closes #6421 - -- TODO: Directory listing for FILE: + make sure the weekday in Date headers matches the date - Ref #6379 - -- curl.h: add CURLPROTO_GOPHERS as own protocol identifier + test61: replace stray "^M" (5e 4d) at the end of a cookie with a '^M' (0d) - Follow-up to a1f06f32b860, to make sure it can be handled separately - from plain gopher. + Gets the test working with external proxies like Privoxy again. - Closes #6418 + Closes #6463 -- http: have CURLOPT_FAILONERROR fail after all headers +- ftp: never set data->set.ftp_append outside setopt - ... so that Retry-After and other meta-content can still be used. + Since the set value then risks getting used like that when the easy + handle is reused by the application. - Added 1634 to verify. Adjusted test 194 and 281 since --fail now also - includes the header-terminating CRLF in the output before it exits. + Also: renamed the struct field from 'ftp_append' to 'remote_append' + since it is also used for SSH protocols. - Fixes #6408 - Closes #6409 + Closes #6579 -- global_init: debug builds allocates a byte in init - - ... to make build tools/valgrind warn if no curl_global_cleanup is - called. - - This is conditionally only done for debug builds with the env variable - CURL_GLOBAL_INIT set. +- urldata: remove the 'rtspversion' field - Closes #6410 - -- lib/unit tests: add missing curl_global_cleanup() calls - -- travis: adapt to Hyper build change + from struct connectdata and the corresponding code in http.c that set + it. It was never used for anything! - Closes #6419 + Closes #6581 -- pretransfer: setup the User-Agent header here +- CURLOPT_QUOTE.3: clarify that libcurl doesn't parse what's sent - ... and not in the connection setup, as for multiplexed transfers the - connection setup might be skipped and then the transfer would end up - without the set user-agent! + ... so passed in commands may confuse libcurl's knowledge of state. - Reported-by: Flameborn on github - Assisted-by: Andrey Gursky - Assisted-by: Jay Satiro - Assisted-by: Mike Gelfand - Fixes #6312 - Closes #6417 + Reported-by: Bodo Bergmann + Fixes #6577 + Closes #6580 -- test66: disable with Hyper - - ...as Hyper doesn't support HTTP/0.9 +- [Jacob Hoffman-Andrews brought this change] -- c-hyper: poll the tasks until end correctly - - ... makes test 36 work. + vtls: factor out Curl_ssl_getsock to field of Curl_ssl - Closes #6412 + Closes #6558 -- [Gergely Nagy brought this change] +- RELEASE-PROCEDURE: remove old release dates, add new - mk-ca-bundle.pl: deterministic output when using -t +- docs/SSL-PROBLEMS: enhanced - Printing trust purposes are now sorted, making the output deterministic - when running on the same input certdata.txt. + Elaborate on the intermediate cert issue, and mention that anything + below TLS 1.2 is generally considered insecure these days. - Closes #6413 + Closes #6572 -- KNOWN_BUGS: fixed "wolfSSL lacks support for renegotiation" - - Fixed by #6411 +- THANKS: remove a Jon Rumsey dupe -- [Himanshu Gupta brought this change] +Daniel Gustafsson (5 Feb 2021) +- [nimaje brought this change] - wolfssl: add SECURE_RENEGOTIATION support + docs: fix FILE example url in --metalink documentation - Closes #6411 - -- RELEASE-NOTES: synced - -- wolfssl: update copyright year range + In a url after :// follows the possibly empty authority part + till the next /, so that url missed a /. - Follow-up to 7de2e96535e9 + Closes #6573 + Reviewed-by: Daniel Stenberg + Reviewed-by: Daniel Gustafsson -- c-hyper: make CURLE_GOT_NOTHING work - - Test 30 +Daniel Stenberg (5 Feb 2021) +- hostip: fix build with sync resolver - Closes #6407 + Reported-by: David Goerger + Follow-up from 8335c6417 + Fixes #6566 + Closes #6568 -- http_proxy: make CONNECT work with the Hyper backend - - Makes test 80 run - - Closes #6406 +- mailmap: Jon Rumsey -- TODO: --fail-with-body perchance? +- [Jon Rumsey brought this change] -Jay Satiro (4 Jan 2021) -- tool_operate: fix the suppression logic of some error messages - - - Fix the failed truncation and failed writing body error messages to - not be shown unless error messages are shown. (ie the user has - specified -sS, or has not specified -s). - - - Also prefix same error messages with "curl: ", for example: - curl: (23) Failed to truncate, exiting - - Prior to this change the failed truncation error messages would be shown - if not -s, but did not account for -sS which should show. - - Prior to this change the failed writing body error messages would be - shown always. - - Ref: https://curl.se/docs/manpage.html#-S + gskit: correct the gskit_send() prototype - Bug: https://curl.se/mail/archive-2020-12/0017.html - Reported-by: Hongyi Zhao + gskit_send() first paramater is a pointer to Curl_easy not connectdata + struct. - Closes https://github.com/curl/curl/pull/6402 + Closes #6570 + Fixes #6569 -- wolfssl: Support wolfSSL builds missing TLS 1.1 - - The wolfSSL TLS library defines NO_OLD_TLS in some of their build - configurations and that causes the library to be built without TLS 1.1. - For example if MD5 is explicitly disabled when building wolfSSL then - that defines NO_OLD_TLS and the library is built without TLS 1.1 [1]. - - Prior to this change attempting to build curl with a wolfSSL that was - built with NO_OLD_TLS would cause a build link error undefined reference - to wolfTLSv1_client_method. +- urldata: fix build without HTTP and MQTT - [1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366 + Reported-by: Joseph Chen + Fixes #6562 + Closes #6563 + +- ftp: avoid SIZE when asking for a TYPE A file - Bug: https://curl.se/mail/lib-2020-12/0121.html - Reported-by: Julian Montes + ... as we ignore it anyway because servers don't report the correct size + and proftpd even blatantly returns a 550. - Closes https://github.com/curl/curl/pull/6388 - -Daniel Stenberg (4 Jan 2021) -- test1633: set appropriate name + Updates a set of tests accordingly. - "--retry with a 429 response and Retry-After:" + Reported-by: awesomenode on github + Fixes #6564 + Closes #6565 -- travis: limit the tests with quiche builds to HTTPS and FTPS only +- pingpong: rename the curl_pp_transfer enum to use PP prefix - ... since it runs into the 50 minute time limit too often otherwise. + Using an FTP prefix for PP provided functionality was misleading. + +- RELEASE-NOTES: synced - Closes #6403 + ... and bump pending version to 7.75.1 (for now) -- HISTORY: added dates to early history +Jay Satiro (4 Feb 2021) +- build: fix --disable-http-auth - Mostly thanks to this archived web page for urlget: + Broken since 215db08 (precedes 7.75.0). - https://web.archive.org/web/19980216125115/http://www.inf.ufrgs.br/~sagula/urlget.html - -- httpauth: make multi-request auth work with custom port + Reported-by: Benbuck Nason - When doing HTTP authentication and a port number set with CURLOPT_PORT, - the code would previously have the URL's port number override as if it - had been a redirect to an absolute URL. + Fixes https://github.com/curl/curl/issues/6567 + +- build: fix --disable-dateparse - Added test 1568 to verify. + Broken since 215db08 (precedes 7.75.0). - Reported-by: UrsusArctos on github - Fixes #6397 - Closes #6400 + Bug: https://curl.se/mail/lib-2021-02/0008.html + Reported-by: Firefox OS -- [Emil Engler brought this change] +Daniel Stenberg (4 Feb 2021) +- [Jon Rumsey brought this change] - language: s/behaviour/behavior/g + OS400: update for CURLOPT_AWS_SIGV4 - We currently use both spellings the british "behaviour" and the american - "behavior". However "behavior" is more used in the project so I think - it's worth dropping the british name. + chkstrings fails because a new string option that could require codepage + conversion has been added. - Closes #6395 + Closes #6561 + Fixes #6560 -- cmdline-opts/retry.d: mention response code 429 as well - - Reported-by: Cherish98 - Bug: https://curl.se/mail/archive-2020-12/0018.html +- BUG-BOUNTY: removed the cooperation mention -- docs/HYPER.md: mention outstanding issues - - To make it more obvious to users what doesn't work (yet) +Version 7.75.0 (3 Feb 2021) + +Daniel Stenberg (3 Feb 2021) +- RELEASE-NOTES: synced + +- THANKS: added contributors from 7.75.0 + +- copyright: fix year ranges in need of updates + +- TODO: remove items for next SONAME bump etc - Closes #6389 + We want to avoid that completely, so we don't plan for things after such + an event. -- COPYING/configure: bump copyright year range +- [Jay Satiro brought this change] -- c-hyper: add timecondition to the request + ngtcp2: Fix build error due to change in ngtcp2_settings - Test 77-78 + - Separate ngtcp2_transport_params. - Closes #6391 - -- c-hyper: make Digest and NTLM work + ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from + ngtcp2_settings. - Test 64, 65, 67, 68, 69, 70, 72 + ngtcp2 master is required to build curl with http3 support. - Closes #6390 + Closes #6554 -- examples/curlgtk.c: fix the copyright year range +- vtls: remove md5sum - ... and make private functions static. - -- [Olaf Hering brought this change] - - docs/examples: adjust prototypes for CURLOPT_READFUNCTION + As it is not used anymore. - The type of the buffer in curl_read_callback is 'char *', not 'void *'. + Reported-by: Jacob Hoffman-Andrews + Bug: https://curl.se/mail/lib-2021-02/0000.html - Signed-off-by: Olaf Hering - Closes #6392 + Closes #6557 -- examples: fix more empty expression statement has no effect - - Follow-up to 26e46617b9 +- [Alessandro Ghedini brought this change] -- cleanup: fix two empty expression statement has no effect + quiche: don't use primary_ip / primary_port - Follow-up to 26e46617b9 + Closes #6555 -- configure: set -Wextra-semi-stmt for clang with --enable-debug +Alessandro Ghedini (1 Feb 2021) +- travis: enable quiche's FFI feature + +Daniel Stenberg (30 Jan 2021) +- [Dmitry Wagin brought this change] + + http: improve AWS HTTP v4 Signature auth - To have it properly complain on empty statements with no effect. + - Add support services without region and service prefixes in + the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc) + by providing region and service parameters via aws-sigv4 option. + - Add [:region[:service]] suffix to aws-sigv4 option; + - Fix memory allocation errors. + - Refactor memory management. + - Use Curl_http_method instead() STRING_CUSTOMREQUEST. + - Refactor canonical headers generating. + - Remove repeated sha256_to_hex() usage. + - Add some docs fixes. + - Add some codestyle fixes. + - Add overloaded strndup() for debug - curl_dbg_strndup(). + - Update tests. - Ref: #6376 - Closes #6378 + Closes #6524 -- tests/unit: fix empty statements with no effect +- hyper: fix CONNECT to set 'data' as userdata - ... by making macros use "do {} while(0)" + Follow-up to 14e075d1a7fd -- [Paul Groke brought this change] +- [Layla brought this change] - dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries - - Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix - (similar to the existing '-' prefix for removing entries) to add - DNS cache entries that will time out just like entries that are added - by libcurl itself. + connect: fix compile errors in `Curl_conninfo_local` - Append " (non-permanent)" to info log message in case a non-permanent - entry is added. + .. for the `#else` (`!HAVE_GETSOCKNAME`) case - Adjust relevant comments to reflect the new behavior. + Fixes https://github.com/curl/curl/issues/6548 + Closes #6549 - Adjust documentation. + Signed-off-by: Layla + +- [Michał Antoniak brought this change] + + transfer: fix GCC 10 warning with flag '-Wint-in-bool-context' - Extend unit1607 to test the new functionality. + ... and return the error code from the Curl_mime_rewind call. - Closes #6294 + Closes #6537 -- schannel: fix "empty expression statement has no effect" - - Bug: https://github.com/curl/curl/commit/8ab78f720ae478d533e30b202baec4b451741579#commitcomment-45445950 - Reported-by: Gisle Vanem - Closes #6381 +- [Michał Antoniak brought this change] -- [Denis Laxalde brought this change] + avoid warning: enum constant in boolean context - docs: remove redundant "better" in --fail help - - Closes #6385 +- copyright: fix missing year (range) updates -- [Kevin Ushey brought this change] +- RELEASE-NOTES: synced - curl.1: fix typo microsft -> microsoft +- openssl: lowercase the hostname before using it for SNI - Closes #6380 - -- [XhmikosR brought this change] - - misc: assorted typo fixes + ... because it turns out several servers out there don't actually behave + correctly otherwise in spite of the fact that the SNI field is + specifically said to be case insensitive in RFC 6066 section 3. - Closes #6375 + Reported-by: David Earl + Fixes #6540 + Closes #6543 -- RELEASE-NOTES: synced +- KNOWN_BUGS: cmake: ExternalProject_Add does not set CURL_CA_PATH + + Closes #6313 -- tool_operate: avoid NULL dereference of first_arg +- KNOWN_BUGS: Multi perform hangs waiting for threaded resolver - Follow-up to 6a5e020d4d2b04a - Identified by OSS-Fuzz - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28999 - Closes #6377 + Closes #4852 -- misc: fix "warning: empty expression statement has no effect" +- KNOWN_BUGS: "pulseUI VPN client" is known to be buggy - Turned several macros into do-while(0) style to allow their use to work - find with semicolon. + First entry in the new section "applications" for known problems in + libcurl using applications. - Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45433279 - Follow-up to 08e8455dddc5e4 - Reported-by: Gisle Vanem - Closes #6376 + Closes #6306 -- KNOWN_BUGS: 6.10 curl never completes Negotiate over HTTP +- tool_writeout: make %{errormsg} blank for no errors - Closes #5235 - Closes #6370 + Closes #6539 -- writeout: fix NULL dereference for "this url" - - Detected by torture test 1029 +Jay Satiro (27 Jan 2021) +- [Gisle Vanem brought this change] + + build: fix djgpp builds - Follow-up to 7a90ddf88f5a + - Update build instructions in packages/DOS/README - Closes #6374 - -- failf: remove newline from formatting strings + - Extend 'VPATH' with 'vquic' and 'vssh'. - ... as failf adds one itself. + - Allow 'Makefile.dist' to build both 'lib' and 'src'. - Also: add an assert() to failf() that triggers on a newline in the - format string! + - Allow using the Windows hosted djgpp cross compiler to build for MSDOS + under Windows. - Closes #6365 - -- [XhmikosR brought this change] - - CI: fix warning with the latest versions + - 'USE_SSL' -> 'USE_OPENSSL' - `git checkout HEAD^2` is no longer needed + - Added a 'link_EXE' macro. Etc, etc. - Closes #6369 - -- INSTALL: update the list known OSes and CPU archs curl has run on + - Linking 'curl.exe' needs '$(CURLX_CFILES)' too. - Closes #6366 - -- [Cherish98 brought this change] - - curl: fix handling of -q option + - Do not pick-up '../lib/djgpp/*.o' files. Recompile locally. - The match of the "-q" option (short for "--disable") should: - a) allow concatenation with other single-letters; and - b) be case-sensitive, lest confusing with "-Q" ("--quote") + - Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'. - Closes #6364 - -- tests/badsymbols.pl: ignore stand-alone single hash lines + - Remove 'djgpp-clean' - Bug: https://curl.se/mail/lib-2020-12/0084.html - Reported-by: Dennis Clarke - Assisted-by: Jay Satiro + - Adapt to new C-ares directory structure - Closes #6355 - -- curl_easy_pause.3: add multiplexed pause effects + - Use conditional variable assignments - and generally refresh and update. Remove details for ancient versions. + Clarify the 'conditional variable assignment' in 'common.dj'. - Reviewed-by: Jay Satiro - Closes #6360 + Closes https://github.com/curl/curl/pull/6382 -Jay Satiro (22 Dec 2020) -- curl_easy_pause.3: fix man page reference +Daniel Stenberg (27 Jan 2021) +- [Ikko Ashimine brought this change] + + hyper: fix typo in c-hyper.c - Follow-up to ac9a724 from earlier today. + settting -> setting - Ref: https://github.com/curl/curl/pull/6359 + Closes #6538 -Daniel Stenberg (22 Dec 2020) -- EXPERIMENTAL: add the Hyper backend to the list +- libssh2: fix CURL_LIBSSH2_DEBUG-enabled build - ... of current experimental features in curl. + Follow-up to 2dcc940959772a + + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/2dcc940959772a652f6813fb6bd3092095a4877b#commitcomment-46420088 -- speedcheck: exclude paused transfers +Jay Satiro (27 Jan 2021) +- asyn-thread: fix build for when getaddrinfo missing - Paused transfers should not be stopped due to slow speed even when - CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is - now reset when the transfer is unpaused - as otherwise it would easily - just trigger immediately after unpausing. + This is a follow-up to 8315343 which several days ago moved the resolver + pointer into the async struct but did not update the code that uses it + when getaddrinfo is not present. - Reported-by: Harry Sintonen - Fixes #6358 - Closes #6359 + Closes https://github.com/curl/curl/pull/6536 -- h2: do not wait for RECV on paused transfers +Daniel Stenberg (27 Jan 2021) +- urldata: move 'ints' to the end of 'connectdata' - ... as the socket might be readable all the time when paused and thus - causing a busy-loop. + To optimize storage slightly. - Reported-by: Harry Sintonen - Reviewed-by: Jay Satiro - Fixes #6356 - Closes #6357 + Closes #6534 -- RELEASE-NOTES: synced +- urldata: store ip version in a single byte + + Closes #6534 -- cmdline-opts/gen.pl: return hard on errors +- urldata: remove duplicate 'upkeep_interval_ms' from connectdata - ... as the warnings tend to go unnoticed otherwise! + ... and rely only on the value already set in Curl_easy. - Closes #6354 + Closes #6534 -- examples/libtest: add .checksrc to dist +- urldata: remove 'local_ip' from the connectdata struct - ... so that (auto)builds from tarballs also get the correct instructions. + As the info is already stored in the transfer handle anyway, there's no + need to carry around a duplicate buffer for the life-time of the handle. - Fixes #6176 - Closes #6353 + Closes #6534 -- test: verify new --write-out variables +- urldata: remove duplicate port number storage - Extended test 1029 and added 1188 - -- test970: adapted to the new internal order of variables - -- curl: add variables to --write-out + ... and use 'int' for ports. We don't use 'unsigned short' since -1 is + still often used internally to signify "unknown value" and 0 - 65535 are + all valid port numbers. - In particular, these ones can help a user to create its own error - message when one or transfers fail. + Closes #6534 + +- urldata: remove the duplicate 'ip_addr_str' field - writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg' + ... as the numerical IP address is already stored and kept in 'primary_ip'. - onerror - lets a user only show the rest on non-zero exit codes + Closes #6534 + +- select: convert Curl_select() to private static function - url - the input URL used for this transfer + The old function should not be used anywhere anymore (the only remaining + gskit use has to be fixed to instead use Curl_poll or none at all). - urlnum - the numerical URL counter (0 indexed) for this transfer + The static function version is now called our_select() and is only built + if necessary. - exitcode - the numerical exit code for the transfer + Closes #6531 + +- Curl_chunker: shrink the struct - errormsg - obvious + ... by removing a field, converting the hex index into a byte and + rearranging the order. Cuts it down from 48 bytes to 32 on x86_64. - Reported-by: Earnestly on github - Fixes #6199 - Closes #6207 + Closes #6527 -- [Matthias Gatto brought this change] +- curl: include the file name in --xattr/--remote-time error msgs - tests: add very simple AWS HTTP v4 Signature test +- curl: s/config->global/global/ in single_transfer() + +- curl: move fprintf outputs to warnf - Signed-off-by: Matthias Gatto + For setting and getting time of the download. To make the outputs + respect --silent etc. + + Reported-by: Viktor Szakats + Fixes #6533 + Closes #6535 -- [Matthias Gatto brought this change] +- [Tatsuhiro Tsujikawa brought this change] - docs: add AWS HTTP v4 Signature + ngtcp2: Fix http3 upload stall + + Closes #6521 -- [Matthias Gatto brought this change] +- [Tatsuhiro Tsujikawa brought this change] - tool: add AWS HTTP v4 Signature support + ngtcp2: Fix stack buffer overflow - Signed-off-by: Matthias Gatto + Closes #6521 -- [Matthias Gatto brought this change] +- warnless.h: remove the prototype for curlx_ultosi + + Follow-up to 217552503ff3 - http: Make the call to v4 signature +- warnless: remove curlx_ultosi - This patch allow to call the v4 signature introduce in previous commit + ... not used anywhere - Signed-off-by: Matthias Gatto + Closes #6530 -- [Matthias Gatto brought this change] +- [Patrick Monnerat brought this change] - http: introduce AWS HTTP v4 Signature - - It is a security process for HTTP. - - It doesn't seems to be standard, but it is used by some cloud providers. - - Aws: - https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html - Outscale: - https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request - GCP (I didn't test that this code work with GCP though): - https://cloud.google.com/storage/docs/access-control/signing-urls-manually - - most of the code is in lib/http_v4_signature.c - - Information require by the algorithm: - - The URL - - Current time - - some prefix that are append to some of the signature parameters. - - The data extracted from the URL are: the URI, the region, - the host and the API type - - example: - https://api.eu-west-2.outscale.com/api/latest/ReadNets - ~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ - ^ ^ ^ - / \ URI - API type region + lib: remove conn->data uses - Small description of the algorithm: - - make canonical header using content type, the host, and the date - - hash the post data - - make canonical_request using custom request, the URI, - the get data, the canonical header, the signed header - and post data hash - - hash canonical_request - - make str_to_sign using one of the prefix pass in parameter, - the date, the credential scope and the canonical_request hash - - compute hmac from date, using secret key as key. - - compute hmac from region, using above hmac as key - - compute hmac from api_type, using above hmac as key - - compute hmac from request_type, using above hmac as key - - compute hmac from str_to_sign using above hmac as key - - create Authorization header using above hmac, prefix pass in parameter, - the date, and above hash + Closes #6515 + +- pingpong: remove the 'conn' struct member - Signed-off-by: Matthias Gatto + ... as it's superfluous now when Curl_easy is passed in and we can + derive the connection from that instead and avoid the duplicate copy. - Closes #5703 - -- [Matthias Gatto brought this change] + Closes #6525 - http: add hmac support for sha256 +- hostip/proxy: remove conn->data use - It seems current hmac implementation use md5 for the hash, - V4 signature require sha256, so I've added the needed struct in - this commit. + Closes #6513 + +- url: reduce conn->data references - I've added the functions that do the hmac in v4 signature file - as a static function ,in the next patch of the serie, - because it's used only by this file. + ... there are a few left but let's keep them to last - Signed-off-by: Matthias Gatto + Closes #6512 -- [Cristian Rodríguez brought this change] +- scripts/singleuse: add curl_easy_option* - connect: on linux, enable reporting of all ICMP errors on UDP sockets - - The linux kernel does not report all ICMP errors back to userspace due - to historical reasons. +Jay Satiro (25 Jan 2021) +- test410: fix for windows - IP*_RECVERR sockopt must be turned on to have the correct behaviour - which is to pass all ICMP errors to userspace. + - Pass the very long request header via file instead of command line. - See https://bugzilla.kernel.org/show_bug.cgi?id=202355 + Prior to this change the 49k very long request header string was passed + via command line and on Windows that is too long so it was truncated and + the test would fail (specifically msys CI). - Closes #6341 + Closes https://github.com/curl/curl/pull/6516 -- curl: add --create-file-mode [mode] +Daniel Stenberg (25 Jan 2021) +- libssh2: move data from connection object to transfer object - This option sets the (octal) mode to use for the remote file when one is - created, using the SFTP, SCP or FILE protocols. When not set, the - default is 0644. + Readdir data, filenames and attributes are strictly related to the + transfer and not the connection. This also reduces the total size of the + fixed connectdata struct. - Closes #6244 + Closes #6519 -- c-hyper: fix compiler warnings - - Identified by clang on windows. - - Reported-by: Gisle Vanem - Bug: 58974d25d8173aec154e593ed9d866da566c9811 +- RELEASE-NOTES: synced + +- [Patrick Monnerat brought this change] + + lib: remove conn->data uses - Closes #6351 + Closes #6499 -- KNOWN_BUGS: Remote recursive folder creation with SFTP +- hyper: remove the conn->data references - Closes #5204 + Closes #6508 -Jay Satiro (20 Dec 2020) -- badsymbols.pl: Add verbose mode -v +- travis: build ngtcp2 --with-gnutls - Use -v as the first option to enable verbose mode which will show source - input, extracted symbol and line info. For example: + ... since they disable it by default since a few days back. - Source: ./../include/curl/typecheck-gcc.h - Symbol: curlcheck_socket_info(info) - Line #423: #define curlcheck_socket_info(info) \ + Closes #6506 + Fixes #6493 + +- hostip: remove conn->data from resolver functions - Ref: https://curl.se/mail/lib-2020-12/0084.html + This also moves the 'async' struct from the connectdata struct into the + Curl_easy struct, which seems like a better home for it. - Closes https://github.com/curl/curl/pull/6349 + Closes #6497 -- KNOWN_BUGS: Secure Transport disabling hostname validation also disables SNI - - That behavior is a limitation of Apple's Secure Transport. +Jay Satiro (22 Jan 2021) +- strerror: skip errnum >= 0 assertion on windows - Reported-by: Cory Benfield - Reported-by: Ian Spence - Confirmed-by: Nick Zitzmann + On Windows an error number may be greater than INT_MAX and negative once + cast to int. - Ref: https://github.com/curl/curl/issues/998 + The assertion is checked only in debug builds. - Closes https://github.com/curl/curl/issues/6347 - Closes https://github.com/curl/curl/pull/6348 + Closes https://github.com/curl/curl/pull/6504 -Daniel Stenberg (18 Dec 2020) -- TODO: alt-svc should fallback if alt-svc doesn't work +Daniel Stenberg (21 Jan 2021) +- doh: make Curl_doh_is_resolved survive a NULL pointer - Closes #4908 + ... if Curl_doh() returned a NULL, this function gets called anyway as + in a asynch procedure. Then the doh struct pointer is NULL and signifies + an OOM situation. + + Follow-up to 6246a1d8c6776 -- travis: restrict the openssl3 job to only run https and ftps tests +- wolfssh: remove conn->data references - ... as it runs too long otherwise and the other tests are verified in - other builds anyway. + ... and repair recent build breakage - Closes #6345 + Closes #6507 -- build: repair http disabled but mqtt enabled build +- http: empty reply connection are not left intact - ... as the mqtt code reuses the "method" originally used for HTTP. + ... so mark the connection as closed in this condition to prevent that + verbose message to wrongly appear. - Closes #6344 - -- [Jon Wilkes brought this change] + Reported-by: Matt Holt + Bug: https://twitter.com/mholt6/status/1352130240265375744 + Closes #6503 - cookie: avoid the C1001 internal compiler error with MSVC 14 +- chunk/encoding: remove conn->data references - Fixes #6112 - Closes #6135 + ... by anchoring more functions on Curl_easy instead of connectdata + + Closes #6498 -- RELEASE-NOTES: synced +Jay Satiro (20 Jan 2021) +- [Erik Olsson brought this change] -- mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE + lib: save a bit of space with some structure packing - Detected by OSS-Fuzz - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735 + - Reorder some internal struct members so that less padding is used. - Added test 1916 and 1917 to verify. + This is an attempt at saving a bit of space by packing some structs + (using pahole to find the holes) where it might make sense to do + so without losing readability. - Closes #6338 + I.e., I tried to avoid separating fields that seem grouped + together (like the cwd... fields in struct ftp_conn for instance). + Also abstained from touching fields behind conditional macros as + that quickly can get complicated. + + Closes https://github.com/curl/curl/pull/6483 -- travis: add CI job for Hyper build +Daniel Stenberg (20 Jan 2021) +- INSTALL.md: fix typo + + Found-by: Marcel Raad -- tests: updated tests for Hyper +- [Fabian Keil brought this change] -- lib: introduce c-hyper for using Hyper + http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy - ... as an alternative HTTP backend within libcurl. - -- tool_setopt: provide helper output in debug builds + Added test 1613 to verify. - ... for when setopt() returns error. - -- setopt: adjust to Hyper and disabled HTTP builds + Closes #6490 -- rtsp: disable if Hyper is used +- Merge branch 'bagder/curl_range-data-conn' -- getinfo: build with disabled HTTP support +- ftp: remove conn->data leftover -- version: include hyper version +- curl_range: remove conn->data + + Closes #6496 -- docs: add HYPER.md +- INSTALL: now at 85 operating systems -- configure: add --with-hyper +- quiche: fix unused parameter ‘conn’ - As the first (optional) HTTP backend alternative instead of native + Follow-up to 2bdec0b3 + +- transfer: fix ‘conn’ undeclared mistake for iconv build - Close #6110 + Follow-up to 219d9f8620d -- test1522: add debug tracing +- doh: allocate state struct on demand - I used this to track down some issues and I figured I could just as well - keep this extra logging in here for future needs. + ... instead of having it static within the Curl_easy struct. This takes + away 1176 bytes (18%) from the Curl_easy struct that aren't used very + often and instead makes the code allocate it when needed. - Closes #6331 + Closes #6492 -- http: show the request as headers even when split-sending +- socks: use the download buffer instead - When the initial request isn't possible to send in its entirety, the - remainder of request would be delivered to the debug callback as data - and would wrongly be counted internally as body-bytes sent. + The SOCKS code now uses the generic download buffer for temporary + storage during the connection procedure, instead of having its own + private 600 byte buffer that adds to the connectdata struct size. This + works fine because this point the buffer is allocated but is not use for + download yet since the connection hasn't completed. - Extended test 1295 to verify. + This reduces the connection struct size by 22% on a 64bit arch! - Closes #6328 + The SOCKS buffer needs to be at least 600 bytes, and the download buffer + is guaranteed to never be smaller than 1000 bytes. + + Closes #6491 -- multi: when erroring in TOOFAST state, act as for PERFORM +- urldata: make magic be the first struct field - When failing in TOOFAST, the multi_done() wasn't called so the same - cleanup and handling wasn't done like when it fails in PERFORM, which in - the case of FTP could mean that the control connection wouldn't be - marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused - ftp_disconnect() to use it to send "QUIT", which could end up waiting - for a response a long time before giving up! + By making the `magic` identifier the same size and at the same place + within the structs (easy, multi, share), libcurl will be able to more + reliably detect and safely error out if an application passes in the + wrong handle to APIs. Easier to detect and less likely to cause crashes + if done. - Reported-by: Tomas Berger - Fixes #6333 - Closes #6337 - -- cmake: enable gophers correctly in curl-config + Such mixups can't be detected at compile-time due to them being + typedefed void pointers - unless `CURL_STRICTER` is defined. - Closes #6336 + Closes #6484 -- test1198/9: add two mqtt publish tests without payload lengths +- http_chunks: correct and clarify a comment on hexnumber length - Closes #6335 - -- tests/mqttd: extract the client id from the correct offset + ... and also rename the define for max length. - Closes #6334 + Closes #6489 -- TODO: Prevent terminal injection when writing to terminal +- curl_path: remove conn->data use - Closes #6150 + Closes #6487 -- Revert "CI/github: work-around for brew breakage on macOS" - - This reverts commit 4cbb17a2cbbbe6337142d39479e21c3990b9c22f. - - ... as the work-around now causes failures. +- transfer: remove conn->data use - Closes #6332 + Closes #6486 -- examples: remove superfluous asterisk uses +- quic: remove conn->data use - ... for function pointers. Breaks in ancient compilers. - -- RELEASE-NOTES: synced + Closes #6485 -- test1272: fix line ending - - Follow-up to f24784f9143 +- [Fabian Keil brought this change] -- URL-SYNTAX: add gophers details + Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" -- test1272: test gophers +- [Fabian Keil brought this change] -- runtests: add support for gophers, gopher over TLS + Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive" + + At the moment the test fails as curl sends two Proxy-Connection + headers. -- [parazyd brought this change] +- c-hyper: avoid duplicated Proxy-Connection headers - gopher: Implement secure gopher protocol. - - This commit introduces a "gophers" handler inside the gopher protocol if - USE_SSL is defined. This protocol is no different than the usual gopher - prococol, with the added TLS encapsulation upon connecting. The protocol - has been adopted in the gopher community, and many people have enabled - TLS in their gopher daemons like geomyidae(8), and clients, like clic(1) - and hurl(1). - - I have not implemented test units for this protocol because my knowledge - of Perl is sub-par. However, for someone more knowledgeable it might be - fairly trivial, because the same test that tests the plain gopher - protocol can be used for "gophers" just by adding a TLS listener. +- http: make providing Proxy-Connection header not cause duplicated headers - Signed-off-by: parazyd + Fixes test 1180 - Closes #6208 + Bug: https://curl.se/mail/lib-2021-01/0095.html + Reported-by: Fabian Keil + Closes #6472 -- TODO: Package curl for Windows in a signed installer +- runtests: preprocess DISABLED to allow conditionals - Closes #5424 - -- mqtt: deal with 0 byte reads correctly + ... with this function provided, we can disable tests for specific + environments and setups directly within this file. - OSS-Fuzz found it - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676 + Closes #6477 + +- runtests: turn preprocessing into a separate function - Closes #6327 + ... and remove all other variable substitutions as they're now done once + and for all in the preprocessor. -- BUG-BOUNTY: minor language update +- lib/Makefile.inc: convert to listing each file on its own line - ... and remove the wording about entries from before 2019 as the "within - 12 months" is still there and covers that. + ... to make it diff friendlier and easier to read. - Closes #6318 + Closes #6448 -- tooĺ_writeout: fix the -w time output units +- ftplistparser: remove use of conn->data - Fix regression from commit fc813f80e1bcac (#6248) that changed the unit - to microseconds instead of seconds with fractions + Closes #6482 + +- lib: more conn->data cleanups - Reported-by: 不确定 - Fixes #6321 - Closes #6322 + Closes #6479 -- quiche: remove fprintf() leftover +- [Patrick Monnerat brought this change] -Jay Satiro (14 Dec 2020) -- KNOWN_BUGS: SHA-256 digest not supported in Windows SSPI builds + vtls: reduce conn->data use - Closes https://github.com/curl/curl/issues/6302 + Closes #6474 -- digest_sspi: Show InitializeSecurityContext errors in verbose mode +- hyper: deliver data to application with Curl_client_write - The error is shown with infof rather than failf so that the user will - see the extended error message information only in verbose mode, and - will still see the standard CURLE_AUTH_ERROR message. For example: + ... just as the native code path does. Avoids sending too large data + chunks in the callback and more. - --- + Reported-by: Gisle Vanem + Fixes #6462 + Closes #6473 + +- gopher: remove accidental conn->data leftover + +- libssh: avoid plain free() of libssh-memory - * schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED - (0x8009030A) - The per-message Quality of Protection is not supported by - the security package - * multi_done - * Connection #1 to host 127.0.0.1 left intact - curl: (94) An authentication function returned an error + Since curl's own memory debugging system redefines free() calls to track + and fiddle with memory, it cannot be used on memory allocated by 3rd + party libraries. - --- + Third party libraries SHOULD NOT require free() to release allocated + resources for this reason - and libs can use separate healp allocators + on some systems (like Windows) so free() doesn't necessarily work + anyway. - Ref: https://github.com/curl/curl/issues/6302 + Filed as an issue with libssh: https://bugs.libssh.org/T268 - Closes https://github.com/curl/curl/pull/6315 + Closes #6481 -Daniel Stenberg (13 Dec 2020) -- URL-SYNTAX: add default port numbers and IDNA details +- send: assert that Curl_write_plain() has a ->conn when called - Closes #6316 + To help catch bad invokes. + + Closes #6476 -- URL-SYNTAX: mention how FILE:// access can access network on windows +- test410: verify HTTPS GET with a 49K request header - Closes #6314 + skip test 410 for mesalink in the CI as it otherwise hangs "forever" -Jay Satiro (12 Dec 2020) -- URL-SYNTAX: Document default SMTP port 25 +- lib: pass in 'struct Curl_easy *' to most functions - Note that ports 25 and 587 are common ports for smtp, the former being - the default. + ... in most cases instead of 'struct connectdata *' but in some cases in + addition to. - Closes https://github.com/curl/curl/pull/6310 - -Daniel Stenberg (12 Dec 2020) -- CURLOPT_URL.3: remove scheme specific details + - We mostly operate on transfers and not connections. - ... that are now found in URL-SYNTAX.md + - We need the transfer handle to log, store data and more. Everything in + libcurl is driven by a transfer (the CURL * in the public API). - Closes #6307 - -Dan Fandrich (12 Dec 2020) -- docs: Fix some typos + - This work clarifies and separates the transfers from the connections + better. - [skip ci] - -Daniel Stenberg (12 Dec 2020) -- URL-SYNTAX: mention all supported schemes + - We should avoid "conn->data". Since individual connections can be used + by many transfers when multiplexing, making sure that conn->data + points to the current and correct transfer at all times is difficult + and has been notoriously error-prone over the years. The goal is to + ultimately remove the conn->data pointer for this reason. - Closes #6311 - -- [Douglas R. Reno brought this change] + Closes #6425 - URL-SYNTAX.md: minor language improvements +Emil Engler (17 Jan 2021) +- docs: fix typos in NEW-PROTOCOL.md - Closes #6308 - -- docs/URL-SYNTAX: the URL syntax curl accepts and works with + This fixes a misspelled "it" and a grammatically wrong "-ing" suffix. - Closes #6285 + Closes #6471 -- [0xflotus brought this change] +Daniel Stenberg (16 Jan 2021) +- RELEASE-NOTES: synced - docs: enable syntax highlighting in several docs files +Jay Satiro (16 Jan 2021) +- [Razvan Cojocaru brought this change] + + cmake: expose CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG - ... for better readability + This does for cmake builds what --disable-openssl-auto-load-config + does for autoconf builds. - Closes #6286 + Closes https://github.com/curl/curl/pull/6435 -- test1564/1565: require the 'wakeup' feature to run +Daniel Stenberg (15 Jan 2021) +- test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id() - Fixes #6299 - Fixes #6300 - Closes #6301 + ... and as a practical side-effect, make sure that the + Curl_easyopts_check() function is asserted in debug builds, which we + want to detect mismatches between the options list in easyoptions.c and + the options in curl.h + + Found-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45991815 + + Closes #6461 -- runtests: add 'wakeup' as a feature +- [Gisle Vanem brought this change] -- tests/server/disabled: add "wakeup" + easyoptions: add the missing AWS_SIGV4 - To allow the test suite to know if wakeup support is disabled in the - build. - -- lib1564/5: verify that curl_multi_wakeup returns OK + Follow-up from AWS_SIGV4 -- tests: make --libcurl tests only test FTP options if ftp enabled +- schannel_verify: fix safefree call typo - Adjust six --libcurl tests to only check the FTP option if FTP is - actually present in the build. + Follow-up from e87ad71d1ba00519 - Fixes #6303 - Closes #6305 + Closes #6459 -- runtests.pl: fix "uninitialized value" warning +- mime: make sure setting MIMEPOST to NULL resets properly - follow-up to e12825c642a88774 - -- runtests: add support for %if [feature] conditions + ... so that a function can first use MIMEPOST and then set it to NULL to + reset it back to a blank POST. - ... to make tests run differently or expect different results depending - on what features that are present or not in curl. + Added test 584 to verify the fix. - Bonus: initial minor 'Hyper' awareness but nothing is using that yet + Reported-by: Christoph M. Becker - Closes #6304 - -- [Jon Rumsey brought this change] + Fixes #6455 + Closes #6456 - OS400: update ccsidcurl.c +- multi: set the PRETRANSFER time-stamp when we switch to PERFORM - Add 'struct' to cast and declaration of cfcdata to fix compilation - error. + ... instead of at end of the DO state. This makes the timer more + accurate for the protocols that use the DOING state (such as FTP), and + simplifies how the function (now called init_perform) is called. - Fixes #6292 - Closes #6297 + The timer will then include the entire procedure up to PERFORM - + including all instructions for getting the transfer started. + + Closes #6454 -- ngtcp2: make it build it current master again +- CURLINFO_PRETRANSFER_TIME.3: clarify - Closes #6296 + ... the timer *does* include the instructions for getting the remote + file. + + Ref: #6452 + Closes #6453 -- [Cristian Rodríguez brought this change] +- [Gisle Vanem brought this change] - connect: defer port selection until connect() time + schannel: plug a memory-leak - If supported, defer port selection until connect() time - if --interface is given and source port is 0. + ... when built without -DUNICODE. - Reproducer: + Closes #6457 + +Jay Satiro (14 Jan 2021) +- gitattributes: Set batch files to CRLF line endings on checkout - * start fast webserver on port 80 - * starve system of ephemeral ports - $ sysctl net.ipv4.ip_local_port_range="60990 60999" + If a batch file is run without CRLF line endings (ie LF-only) then + arbitrary behavior may occur. I consider that a bug in Windows, however + the effects can be serious enough (eg unintended code executed) that + we're fixing it in the repo by requiring CRLF line endings for batch + files on checkout. - * start a curl/libcurl "crawler" - $curl --keepalive --parallel --parallel-immediate --head --interface - 127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt" + Prior to this change the checked-out line endings of batch files were + dependent on a user's git preferences. On Windows it is common for git + users to have automatic CRLF conversion enabled (core.autocrlf true), + but those users that don't would run into this behavior. - current result: - (possible some successful data) - curl: (45) bind failed with errno 98: Address already in use + For example a user has reported running the Visual Studio project + generator batch file (projects/generate.bat) and it looped forever. + Output showed that the Windows OS interpreter was occasionally jumping + to arbitrary points in the batch file and executing commands. This + resulted in unintended files being removed (a removal sequence called) + and looping forever. - result after patch: - (complete success or few connections failing, higlhy depending on load) + Ref: https://serverfault.com/q/429594 + Ref: https://stackoverflow.com/q/232651 + Ref: https://www.dostips.com/forum/viewtopic.php?t=8988 + Ref: https://git-scm.com/docs/gitattributes#_checking_out_and_checking_in + Ref: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_core_autocrlf - Fail only when all the possible 4-tuple combinations are exhausted, - which is impossible to do when port is selected at bind() time becuse - the kernel does not know if socket will be listen()'ed on or connect'ed - yet. + Bug: https://github.com/curl/curl/discussions/6427 + Reported-by: Ganesh Kamath - Closes #6295 + Closes https://github.com/curl/curl/pull/6442 + +Daniel Stenberg (14 Jan 2021) +- tool_operate: spellfix a comment + +- ROADMAP: refreshed + + o removed HSTS - already implemented + o added HTTPS RR records + o mention HTTP/3 completion + +- http_chunks: remove Curl_ prefix from static functions -- [Hans-Christian Noren Egtvedt brought this change] +- transfer: remove Curl_ prefix from static functions - connect: zero variable on stack to silence valgrind complaint - - Valgrind will complain that ssrem buffer usage if not explicit - initialized, hence initialize it to zero. - - This completes the change intially started in commit 2c0d7212151 ('ftp: - retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has - a similar memset to zero. - - Signed-off-by: Hans-Christian Noren Egtvedt - Closes #6289 +- tftp: remove Curl_ prefix from static functions -- RELEASE-NOTES: synced - - start over on the next release cycle +- multi: remove Curl_ prefix from static functions -Version 7.74.0 (9 Dec 2020) +- ldap: remove Curl_ prefix from static functions -Daniel Stenberg (9 Dec 2020) -- RELEASE-NOTES: synced - - for 7.74.0 +- doh: remove Curl_ prefix from static functions -Jay Satiro (7 Dec 2020) -- [Jacob Hoffman-Andrews brought this change] +- asyn-ares: remove Curl_ prefix from static functions - urldata: restore comment on ssl_connect_data.use - - This comment was originally on the `use` field, but was separated from - its field in 62a2534. - - Closes https://github.com/curl/curl/pull/6287 +- vtls: remove Curl_ prefix from static functions -Daniel Stenberg (7 Dec 2020) -- VERSIONS: refreshed - - We always use the patch number these days: all releases are - "major.minor.patch" +- bearssl: remove Curl_ prefix from static functions -- [Jakub Zakrzewski brought this change] +- mbedtls: remove Curl_ prefix from static functions - cmake: don't use reserved target name 'test' - - CMake up to 3.10 always reserves this name - - Fixes #6257 - Closes #6258 +- wolfssl: remove Curl_ prefix from static functions -- openssl: make the OCSP verification verify the certificate id +- nss: remove Curl_ prefix from static functions + +- gnutls: remove Curl_ prefix from static functions + +- openssl: remove Curl_ prefix from static functions - CVE-2020-8286 + ... as we reserve this prefix to library-wide functions. - Reported by anonymous + Closes #6443 + +- nss: get the run-time version instead of build-time - Bug: https://curl.se/docs/CVE-2020-8286.html + Closes #6445 -- ftp: make wc_statemach loop instead of recurse +Jay Satiro (12 Jan 2021) +- tool_doswin: Restore original console settings on CTRL signal - CVE-2020-8285 + - Move Windows terminal init code from tool_main to tool_doswin. - Fixes #6255 - Bug: https://curl.se/docs/CVE-2020-8285.html - Reported-by: xnynx on github - -- ftp: CURLOPT_FTP_SKIP_PASV_IP by default + - Restore the original console settings on CTRL+C and CTRL+BREAK. - The command line tool also independently sets --ftp-skip-pasv-ip by - default. + Background: On Windows the curl tool changes the console settings to + enable virtual terminal processing (eg color output) if supported + (ie Win 10). The original settings are restored on exit but prior to + this change were not restored in the case of the CTRL signals. - Ten test cases updated to adapt the modified --libcurl output. + Windows VT behavior varies depending on console/powershell/terminal; + refer to the discussion in #6226. - Bug: https://curl.se/docs/CVE-2020-8284.html - CVE-2020-8284 + Assisted-by: Rich Turner - Reported-by: Varnavas Papaioannou + Closes https://github.com/curl/curl/pull/6226 -- urlapi: don't accept blank port number field without scheme +Daniel Stenberg (12 Jan 2021) +- gen.pl: fix perl syntax - ... as it makes the URL parser accept "very-long-hostname://" as a valid - host name and we don't want that. The parser now only accepts a blank - (no digits) after the colon if the URL starts with a scheme. + Follow-up to 324cf1d2e + +- [Emil Engler brought this change] + + help: update to current codebase - Reported-by: d4d on hackerone + This commit bumps the help to the current state of the project. - Closes #6283 + Closes #6437 -- Revert "multi: implement wait using winsock events" - - This reverts commit d2a7d7c185f98df8f3e585e5620cbc0482e45fac. +- [Emil Engler brought this change] + + docs: fix line length bug in gen.pl - This commit also reverts the subsequent follow-ups to that commit, which - were all done within windows #ifdefs that are removed in this - change. Marc helped me verify this. + The script warns if the length of $opt and $desc is > 78. However, these + two variables are on totally separate lines so the check makes no sense. + Also the $bitmask field is totally forgotten. Currently this leads to + two warnings within `--resolve` and `--aws-sigv4`. - Fixes #6146 - Closes #6281 + Closes #6438 -- [Klaus Crusius brought this change] +- [Emil Engler brought this change] - ftp: retry getpeername for FTP with TCP_FASTOPEN - - In the case of TFO, the remote host name is not resolved at the - connetion time. + docs: fix wrong documentation in help.d - For FTP that has lead to missing hostname for the secondary connection. - Therefore the name resolution is done at the time, when FTP requires it. + curl does not list all categories when you invoke "--help" without any + parameters. - Fixes #6252 - Closes #6265 - Closes #6282 - -- [Thomas Danielsson brought this change] + Closes #6436 - scripts/completion.pl: parse all opts +- aws-sigv4.d: polish the wording - For tab-completion it may be preferable to include all the - available options. + Make it shorter and imperative form - Closes #6280 + Closes #6439 -- RELEASE-NOTES: synced +- [Fabian Keil brought this change] -- openssl: use OPENSSL_init_ssl() with >= 1.1.0 + misc: fix typos - Reported-by: Kovalkov Dmitrii and Per Nilsson - Fixes #6254 - Fixes #6256 - Closes #6260 + Bug: https://curl.se/mail/lib-2021-01/0063.html + Closes #6434 -- SECURITY-PROCESS: disclose on hackerone +- multi_runsingle: bail out early on data->conn == NULL - Once a vulnerability has been published, the hackerone issue should be - disclosed. For tranparency. + As that's a significant error condition and scan-build warns for NULL + pointer dereferences if we don't. - Closes #6275 + Closes #6433 -Marc Hoersken (3 Dec 2020) -- tests/util.py: fix compatibility with Python 2 - - Backporting the Python 3 implementation of setStream - to ClosingFileHandler as a fallback within Python 2. - - Reported-by: Jay Satiro +- multi: skip DONE state if there's no connection left for ftp wildcard - Fixes #6259 - Closes #6270 + ... to avoid running in that state with data->conn being NULL. -Daniel Gustafsson (3 Dec 2020) -- docs: fix typos and markup in ETag manpage sections +- libssh2: fix "Value stored to 'readdir_len' is never read" - Reported-by: emanruse on github - Fixes #6273 + Detected by scan-build -Daniel Stenberg (2 Dec 2020) -- quiche: close the connection +- connect: mark intentional ignores of setsockopt return values - Reported-by: Junho Choi - Fixes #6213 - Closes #6217 + Pointed out by Coverity + + Closes #6431 -Jay Satiro (2 Dec 2020) -- ngtcp2: Fix build error due to symbol name change +Jay Satiro (11 Jan 2021) +- http_proxy: Fix CONNECT chunked encoding race condition - - NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION + - During the end-of-headers response phase do not mark the tunnel + complete unless the response body was completely parsed/ignored. - ngtcp2/ngtcp2@76232e9 changed the name. + Prior to this change if the entirety of a CONNECT response with chunked + encoding was not received by the time the final header was parsed then + the connection would be marked done prematurely, before all the chunked + data could be read in and ignored (since this is what we do with any + CONNECT response body) and the connection could not be used. - ngtcp2 master is required to build curl with http3 support. + Bug: https://curl.se/mail/lib-2021-01/0033.html + Reported-by: Fabian Keil - Closes https://github.com/curl/curl/pull/6271 + Closes https://github.com/curl/curl/pull/6432 -Daniel Stenberg (1 Dec 2020) -- [Klaus Crusius brought this change] +Daniel Stenberg (11 Jan 2021) +- RELEASE-NOTES: synced - cmake: check for linux/tcp.h +- url: if IDNA conversion fails, fallback to Transitional - The HAVE_LINUX_TCP_H define was not set by cmake. + This improves IDNA2003 compatiblity. - Closes #6252 + Reported-by: Bubu on github + Fixes #6423 + Closes #6428 -- NEW-PROTOCOL: document what needs to be done to add one +- travis: make the Hyper build from its master branch - Closes #6263 + Closes #6430 -- splay: rename Curl_splayremovebyaddr to Curl_splayremove +- http: make 'authneg' also work for Hyper - ... and remove the old unused proto for the old Curl_splayremove - version. + When doing a request with a request body expecting a 401/407 back, that + initial request is sent with a zero content-length. Test 177 and more. - Closes #6269 + Closes #6424 -- openssl: free mem_buf in error path +Jay Satiro (8 Jan 2021) +- cmake: Add an option to disable libidn2 - To fix a memory-leak. + New option USE_LIBIDN2 defaults to ON for libidn2 detection. Prior to + this change libidn2 detection could not be turned off in cmake builds. + + Reported-by: William A Rowe Jr + + Fixes https://github.com/curl/curl/issues/6361 + Closes https://github.com/curl/curl/pull/6362 + +Daniel Stenberg (8 Jan 2021) +- HYPER: no longer needs the special branch + +- test179: use consistent header line endings + + ... to make "Hyper mode" work better. + +- file: don't provide content-length for directories + + ... as it is misleading. + + Ref #6379 + Closes #6421 + +- TODO: Directory listing for FILE: - Closes #6267 + Ref #6379 -- openssl: remove #if 0 leftover +- curl.h: add CURLPROTO_GOPHERS as own protocol identifier - Follow-up to 4c9768565ec3a9 (from Sep 2008) + Follow-up to a1f06f32b860, to make sure it can be handled separately + from plain gopher. - Closes #6268 + Closes #6418 -- ntlm: avoid malloc(0) on zero length user and domain +- http: have CURLOPT_FAILONERROR fail after all headers - ... and simplify the too-long checks somewhat. + ... so that Retry-After and other meta-content can still be used. - Detected by OSS-Fuzz + Added 1634 to verify. Adjusted test 194 and 281 since --fail now also + includes the header-terminating CRLF in the output before it exits. - Closes #6264 - -- RELEASE-NOTES: synced + Fixes #6408 + Closes #6409 -Marc Hoersken (28 Nov 2020) -- tests/server/tftpd.c: close upload file in case of abort - - Commit c353207 removed the closing right after do_tftp - which covered the case of abort. This handles that case. +- global_init: debug builds allocates a byte in init - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ... to make build tools/valgrind warn if no curl_global_cleanup is + called. - Follow up to #6209 - Closes #6234 - -Daniel Stenberg (26 Nov 2020) -- [Daiki Ueno brought this change] - - ngtcp2: use the minimal version of QUIC supported by ngtcp2 + This is conditionally only done for debug builds with the env variable + CURL_GLOBAL_INIT set. - Closes #6250 + Closes #6410 -- [Daiki Ueno brought this change] +- lib/unit tests: add missing curl_global_cleanup() calls - ngtcp2: advertise h3 ALPN unconditionally +- travis: adapt to Hyper build change - Closes #6250 - -- [Daiki Ueno brought this change] + Closes #6419 - vquic/ngtcp2.h: define local_addr as sockaddr_storage +- pretransfer: setup the User-Agent header here - This field needs to be wide enough to hold sockaddr_in6 when - connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the - packets because of the address mismatch: - I00000022 [...] con ignore packet from unknown path + ... and not in the connection setup, as for multiplexed transfers the + connection setup might be skipped and then the transfer would end up + without the set user-agent! - We can safely assume that struct sockaddr_storage is available, as it - is used in the public interface of ngtcp2. + Reported-by: Flameborn on github + Assisted-by: Andrey Gursky + Assisted-by: Jay Satiro + Assisted-by: Mike Gelfand + Fixes #6312 + Closes #6417 + +- test66: disable with Hyper - Closes #6250 + ...as Hyper doesn't support HTTP/0.9 -- socks: check for DNS entries with the right port number +- c-hyper: poll the tasks until end correctly - The resolve call is done with the right port number, but the subsequent - check used the wrong one, which then could find a previous resolve which - would return and leave the fresh resolve "incomplete" and leaking - memory. + ... makes test 36 work. - Fixes #6247 - Closes #6253 + Closes #6412 -- curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use - - ... so don't define it when instructed to use c-ares! +- [Gergely Nagy brought this change] -- test506: make it not run in c-ares builds + mk-ca-bundle.pl: deterministic output when using -t - As the asynch nature of it may trigger events in another order. A c-ares - upgrade made it break. + Printing trust purposes are now sorted, making the output deterministic + when running on the same input certdata.txt. - Reported-by: Marc Hörsken - Fixes #6247 + Closes #6413 -- runtests: make 'c-ares' a "feature" to depend on +- KNOWN_BUGS: fixed "wolfSSL lacks support for renegotiation" - ... also added to the docs. + Fixed by #6411 -- tool_writeout: use off_t getinfo-types instead of doubles - - Commit 3b80d3ca46b12e52342 (June 2017) introduced getinfo replacement - variables that use curl_off_t instead of doubles. Switch the --write-out - function over to use them. +- [Himanshu Gupta brought this change] + + wolfssl: add SECURE_RENEGOTIATION support - Closes #6248 + Closes #6411 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - file: avoid duplicated code sequence - - file_disconnect() is identical with file_do() except the function header - but as the arguments are unused anyway so why not just return file_do() - directly! +- wolfssl: update copyright year range - Reviewed-by: Daniel Stenberg - Closes #6249 - -- [Rikard Falkeborn brought this change] + Follow-up to 7de2e96535e9 - infof/failf calls: fix format specifiers +- c-hyper: make CURLE_GOT_NOTHING work - Update a few format specifiers to match what is being printed. + Test 30 - Closes #6241 + Closes #6407 -- docs/INTERNALS: remove reference to Curl_sendf() +- http_proxy: make CONNECT work with the Hyper backend - The function has been removed from common usage. Also removed comment in - gopher.c that still referenced it. + Makes test 80 run - Reported-by: Rikard Falkeborn - Fixes #6242 - Closes #6243 + Closes #6406 -- [Rikard Falkeborn brought this change] +- TODO: --fail-with-body perchance? - examples: update .gitignore +Jay Satiro (4 Jan 2021) +- tool_operate: fix the suppression logic of some error messages - Add files that are generated by 'make examples' and remove some that - have been renamed. + - Fix the failed truncation and failed writing body error messages to + not be shown unless error messages are shown. (ie the user has + specified -sS, or has not specified -s). - The commits that renamed the programs are e9625c5bc6c046a (imap.c and - simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and - ad39e7ec01e7 (pop3slist.c and pop3s.c were renamed to pop3-list.c and - pop3-ssl.c). + - Also prefix same error messages with "curl: ", for example: + curl: (23) Failed to truncate, exiting - Closes #6240 - -- asyn: use 'struct thread_data *' instead of 'void *' + Prior to this change the failed truncation error messages would be shown + if not -s, but did not account for -sS which should show. - To reduce use of types that can't be checked at compile time. Also - removes several typecasts. + Prior to this change the failed writing body error messages would be + shown always. - ... and rename the struct field from 'os_specific' to 'tdata'. + Ref: https://curl.se/docs/manpage.html#-S - Closes #6239 - Reviewed-by: Jay Satiro + Bug: https://curl.se/mail/archive-2020-12/0017.html + Reported-by: Hongyi Zhao + + Closes https://github.com/curl/curl/pull/6402 -Viktor Szakats (23 Nov 2020) -- Makefile.m32: add support for UNICODE builds +- wolfssl: Support wolfSSL builds missing TLS 1.1 - It requires the linker to support the `-municode` option. - This is available in more recent mingw-w64 releases. + The wolfSSL TLS library defines NO_OLD_TLS in some of their build + configurations and that causes the library to be built without TLS 1.1. + For example if MD5 is explicitly disabled when building wolfSSL then + that defines NO_OLD_TLS and the library is built without TLS 1.1 [1]. - Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html - Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847 + Prior to this change attempting to build curl with a wolfSSL that was + built with NO_OLD_TLS would cause a build link error undefined reference + to wolfTLSv1_client_method. - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad + [1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366 - Closes #6228 + Bug: https://curl.se/mail/lib-2020-12/0121.html + Reported-by: Julian Montes + + Closes https://github.com/curl/curl/pull/6388 -Daniel Stenberg (23 Nov 2020) -- urldata: remove 'void *protop' and create the union 'p' +Daniel Stenberg (4 Jan 2021) +- test1633: set appropriate name - ... to avoid the use of 'void *' for the protocol specific structs done - per transfer. + "--retry with a 429 response and Retry-After:" + +- travis: limit the tests with quiche builds to HTTPS and FTPS only - Closes #6238 + ... since it runs into the 50 minute time limit too often otherwise. + + Closes #6403 -- winbuild: remove docs from Makefiles and refer to README.md +- HISTORY: added dates to early history - Reduce risk for conflicting docs and makes it to a single place to fix - and polish. + Mostly thanks to this archived web page for urlget: - add these missing options to the readme: + https://web.archive.org/web/19980216125115/http://www.inf.ufrgs.br/~sagula/urlget.html + +- httpauth: make multi-request auth work with custom port - ENABLE_OPENSSL_AUTO_LOAD_CONFIG and ENABLE_UNICODE + When doing HTTP authentication and a port number set with CURLOPT_PORT, + the code would previously have the URL's port number override as if it + had been a redirect to an absolute URL. - clarify ENABLE_SCHANNEL default varies + Added test 1568 to verify. - Fixes #6216 - Closes #6227 - Co-Authored-by: Jay Satiro + Reported-by: UrsusArctos on github + Fixes #6397 + Closes #6400 -- [Daiki Ueno brought this change] +- [Emil Engler brought this change] - http3: use the master branch of GnuTLS for testing + language: s/behaviour/behavior/g - Closes #6235 - -- KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation + We currently use both spellings the british "behaviour" and the american + "behavior". However "behavior" is more used in the project so I think + it's worth dropping the british name. - Closes #5839 + Closes #6395 -- KNOWN_BUGS: wakeup socket disconnect causes havoc +- cmdline-opts/retry.d: mention response code 429 as well - Closes #6132 - Closes #6133 + Reported-by: Cherish98 + Bug: https://curl.se/mail/archive-2020-12/0018.html -- RELEASE-NOTES: synced +- docs/HYPER.md: mention outstanding issues + + To make it more obvious to users what doesn't work (yet) + + Closes #6389 -- [Oliver Urbann brought this change] +- COPYING/configure: bump copyright year range - curl: add compatibility for Amiga and GCC 6.5 +- c-hyper: add timecondition to the request - Changes are mainly reordering and adding of includes required - to compile with a more recent version of GCC. + Test 77-78 - Closes #6220 + Closes #6391 -Marc Hoersken (20 Nov 2020) -- tests/server/tftpd.c: close upload file right after transfer - - Make sure uploaded file is no longer locked after the - transfer while waiting for the final ACK to be handled. +- c-hyper: make Digest and NTLM work - Assisted-by: Daniel Stenberg + Test 64, 65, 67, 68, 69, 70, 72 - Bug: #6058 - Closes #6209 + Closes #6390 -- CI/cirrus: simplify logic for disabled tests - - The OpenSSH server instance for the testsuite cannot - be started on FreeBSD, therefore the SFTP and SCP - tests are disabled right away from the beginning. +- examples/curlgtk.c: fix the copyright year range - The previous OS version specific logic for SKIP_TESTS - is no longer needed/used and can therefore be removed. + ... and make private functions static. + +- [Olaf Hering brought this change] + + docs/examples: adjust prototypes for CURLOPT_READFUNCTION - Reviewed-by: Daniel Stenberg + The type of the buffer in curl_read_callback is 'char *', not 'void *'. - Follow up to #6211 - Closes #6229 + Signed-off-by: Olaf Hering + Closes #6392 -Daniel Gustafsson (20 Nov 2020) -- mailmap: Daniel Hwang - - Add Daniel Hwang to the mailmap to cover the alternative spelling - Daniel Lee Hwang which was used in one commit. +- examples: fix more empty expression statement has no effect - Closes #6230 - Reviewed-by: Daniel Stenberg + Follow-up to 26e46617b9 -- openssl: guard against OOM on context creation +- cleanup: fix two empty expression statement has no effect - EVP_MD_CTX_create will allocate memory for the context and returns - NULL in case the allocation fails. Make sure to catch any allocation - failures and exit early if so. + Follow-up to 26e46617b9 + +- configure: set -Wextra-semi-stmt for clang with --enable-debug - In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex - as the latter is intended for ENGINE selection which we don't do. + To have it properly complain on empty statements with no effect. - Closes #6224 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler - -Daniel Stenberg (19 Nov 2020) -- [Vincent Torri brought this change] + Ref: #6376 + Closes #6378 - cmake: use libcurl.rc in all Windows builds +- tests/unit: fix empty statements with no effect - Reviewed-by: Marcel Raad - Closes #6215 + ... by making macros use "do {} while(0)" -- [Cristian Morales Vega brought this change] +- [Paul Groke brought this change] - cmake: make CURL_ZLIB a tri-state variable + dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries - By differentiating between ON and AUTO it can make a missing zlib - library a hard error when CURL_ZLIB=ON is used. + Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix + (similar to the existing '-' prefix for removing entries) to add + DNS cache entries that will time out just like entries that are added + by libcurl itself. - Reviewed-by: Jakub Zakrzewski - Closes #6221 - Fixes #6173 - -- quiche: remove 'static' from local buffer + Append " (non-permanent)" to info log message in case a non-permanent + entry is added. - For thread-safety + Adjust relevant comments to reflect the new behavior. - Closes #6223 - -- KNOWN_BUGS: cmake: libspsl is not supported + Adjust documentation. - Closes #6214 - -- KNOWN_BUGS: cmake autodetects cert paths when cross-compiling + Extend unit1607 to test the new functionality. - Closes #6178 + Closes #6294 -- KNOWN_BUGS: cmake build doesn't fail if zlib not found +- schannel: fix "empty expression statement has no effect" - Closes #6173 + Bug: https://github.com/curl/curl/commit/8ab78f720ae478d533e30b202baec4b451741579#commitcomment-45445950 + Reported-by: Gisle Vanem + Closes #6381 -- KNOWN_BUGS: cmake libcurl.pc uses absolute library paths - - Closes #6169 +- [Denis Laxalde brought this change] -- KNOWN_BUGS: cmake: generated .pc file contains strange entries + docs: remove redundant "better" in --fail help - Closes #6167 + Closes #6385 -- KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads - - Closes #6166 +- [Kevin Ushey brought this change] -- KNOWN_BUGS: cmake build in Linux links libcurl to libdl + curl.1: fix typo microsft -> microsoft - Closes #6165 + Closes #6380 -- KNOWN_BUGS: make a new section for cmake topics +- [XhmikosR brought this change] + + misc: assorted typo fixes - Closes #6219 + Closes #6375 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - cirrus: build with FreeBSD 12.2 in CirrusCI +- tool_operate: avoid NULL dereference of first_arg - Closes #6211 + Follow-up to 6a5e020d4d2b04a + Identified by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28999 + Closes #6377 -Marc Hoersken (14 Nov 2020) -- tests/*server.py: close log file after each log line - - Make sure the log file is not locked once a test has - finished and align with the behavior of our logmsg. +- misc: fix "warning: empty expression statement has no effect" - Rename curl_test_data.py to be a general util.py. - Format and sort Python imports with isort/VSCode. + Turned several macros into do-while(0) style to allow their use to work + find with semicolon. - Bug: #6058 - Closes #6206 + Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45433279 + Follow-up to 08e8455dddc5e4 + Reported-by: Gisle Vanem + Closes #6376 -Daniel Stenberg (13 Nov 2020) -- CURLOPT_HSTS.3: document the file format +- KNOWN_BUGS: 6.10 curl never completes Negotiate over HTTP - Closes #6205 - -- RELEASE-NOTES: synced - -- release-notes.pl: detect #[number] better for Ref: etc + Closes #5235 + Closes #6370 -- curl: only warn not fail, if not finding the home dir +- writeout: fix NULL dereference for "this url" - ... as there's no good reason to error out completely. + Detected by torture test 1029 - Reported-by: Andreas Fischer - Fixes #6200 - Closes #6201 + Follow-up to 7a90ddf88f5a + + Closes #6374 -- httpput-postfields.c: new example doing PUT with POSTFIELDS +- failf: remove newline from formatting strings - Proposed-by: Jeroen Ooms - Ref: #6186 - Closes #6188 + ... as failf adds one itself. + + Also: add an assert() to failf() that triggers on a newline in the + format string! + + Closes #6365 -- [Tobias Hieta brought this change] +- [XhmikosR brought this change] - cmake: correctly handle linker flags for static libs + CI: fix warning with the latest versions - curl CMake was setting the the EXE flags for static libraries which made - the /manifest:no flag ended up when linking the static library, which is - not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit - with an error. + `git checkout HEAD^2` is no longer needed - The better way to handle this is to make sure that we pass the correct - linker flags to CMAKE_STATIC_LINKER_FLAGS instead. + Closes #6369 + +- INSTALL: update the list known OSes and CPU archs curl has run on - Reviewed-by: Jakub Zakrzewski - Closes #6195 + Closes #6366 -- [Tobias Hieta brought this change] +- [Cherish98 brought this change] - cmake: don't pass -fvisibility=hidden to clang-cl on Windows + curl: fix handling of -q option - When using clang-cl on windows -fvisibility=hidden is not an known - argument. Instead it behaves exactly like MSVC in this case. So let's - make sure we take that path. + The match of the "-q" option (short for "--disable") should: + a) allow concatenation with other single-letters; and + b) be case-sensitive, lest confusing with "-Q" ("--quote") - In CMake clang-cl sets both CMAKE_C_COMPILER_ID=clang and MSVC get's - defined since clang-cl is basically a MSVC emulator. So guarding like we - do in this patch seems logical. + Closes #6364 + +- tests/badsymbols.pl: ignore stand-alone single hash lines - Reviewed-by: Jakub Zakrzewski - Closes #6194 + Bug: https://curl.se/mail/lib-2020-12/0084.html + Reported-by: Dennis Clarke + Assisted-by: Jay Satiro + + Closes #6355 -- http_proxy: use enum with state names for 'keepon' +- curl_easy_pause.3: add multiplexed pause effects - To make the code clearer, change the 'keepon' from an int to an enum - with better state names. + and generally refresh and update. Remove details for ancient versions. - Reported-by: Niranjan Hasabnis - Bug: https://curl.se/mail/lib-2020-11/0026.html - Closes #6193 + Reviewed-by: Jay Satiro + Closes #6360 -- curl_easy_escape: limit output string length to 3 * max input +Jay Satiro (22 Dec 2020) +- curl_easy_pause.3: fix man page reference - ... instead of the limiting it to just the max input size. As every - input byte can be expanded to 3 output bytes, this could limit the input - string to 2.66 MB instead of the intended 8 MB. + Follow-up to ac9a724 from earlier today. - Reported-by: Marc Schlatter - Closes #6192 + Ref: https://github.com/curl/curl/pull/6359 -- docs: document the 8MB input string limit +Daniel Stenberg (22 Dec 2020) +- EXPERIMENTAL: add the Hyper backend to the list - for curl_easy_escape and curl_easy_setopt() + ... of current experimental features in curl. + +- speedcheck: exclude paused transfers - The limit is there to catch mistakes and abuse. It is meant to be large - enough to allow virtually all "fine" use cases. + Paused transfers should not be stopped due to slow speed even when + CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is + now reset when the transfer is unpaused - as otherwise it would easily + just trigger immediately after unpausing. - Reported-by: Marc Schlatter - Fixes #6190 - Closes #6191 + Reported-by: Harry Sintonen + Fixes #6358 + Closes #6359 -- mqttd: fclose test file when done +- h2: do not wait for RECV on paused transfers - Reported-by: Marc Hörsken + ... as the socket might be readable all the time when paused and thus + causing a busy-loop. + + Reported-by: Harry Sintonen Reviewed-by: Jay Satiro - Bug: #6058 - Closes #6189 + Fixes #6356 + Closes #6357 - RELEASE-NOTES: synced -- THANKS-filter: ignore autobuild links +- cmdline-opts/gen.pl: return hard on errors + + ... as the warnings tend to go unnoticed otherwise! + + Closes #6354 -- Revert "libcurl.pc: make it relocatable" +- examples/libtest: add .checksrc to dist - This reverts commit 3862c37b6373a55ca704171d45ba5ee91dec2c9f. + ... so that (auto)builds from tarballs also get the correct instructions. - That fix should either be done differently or with an option. + Fixes #6176 + Closes #6353 + +- test: verify new --write-out variables - Reported-by: asavah on github - Fixes #6157 - Closes #6183 + Extended test 1029 and added 1188 -- examples/httpput: remove use of CURLOPT_PUT +- test970: adapted to the new internal order of variables + +- curl: add variables to --write-out - It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD. + In particular, these ones can help a user to create its own error + message when one or transfers fail. - Reported-by: Jeroen Ooms - Fixes #6186 - Closes #6187 - -- Curl_pgrsStartNow: init speed limit time stamps at start + writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg' - By setting the speed limit time stamps unconditionally at transfer - start, we can start off a transfer without speed limits and yet allow - them to get set during transfer and have an effect. + onerror - lets a user only show the rest on non-zero exit codes - Reported-by: Kael1117 on github - Fixes #6162 - Closes #6184 - -- ngtcp2: adapt to recent nghttp3 updates + url - the input URL used for this transfer - 'reset_stream' was added to the nghttp3_conn_callbacks struct + urlnum - the numerical URL counter (0 indexed) for this transfer - Closes #6185 - -- configure: pass -pthread to Libs.private for pkg-config + exitcode - the numerical exit code for the transfer - Reported-by: Cristian Morales Vega - Fixes #6168 - Closes #6181 - -- altsvc: minimize variable scope and avoid "DEAD_STORE" + errormsg - obvious - Closes #6182 + Reported-by: Earnestly on github + Fixes #6199 + Closes #6207 -- FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?" - - This hasn't been the case for a while now, remove. +- [Matthias Gatto brought this change] -- FAQ: refresh "Why do I get "certificate verify failed" + tests: add very simple AWS HTTP v4 Signature test - Add more details, remove references to ancient curl version. + Signed-off-by: Matthias Gatto -- test493: verify --hsts upgrade and that %{url_effective} reflects that - - Closes #6175 +- [Matthias Gatto brought this change] -- url: make sure an HSTS upgrade updates URL and scheme correctly - - Closes #6175 + docs: add AWS HTTP v4 Signature -- tool_operate: set HSTS with CURLOPT_HSTS to pass on filename - - Closes #6175 +- [Matthias Gatto brought this change] -- hsts: remove debug code leftovers + tool: add AWS HTTP v4 Signature support - Closes #6175 + Signed-off-by: Matthias Gatto -- FAQ: refreshed - - - remove a few ancient questions - - add configure with static libs question - - updated wording in several places - - lowercased curl - - Closes #6177 +- [Matthias Gatto brought this change] -Daniel Gustafsson (5 Nov 2020) -- examples: fix comment syntax + http: Make the call to v4 signature - Commit ac0a88fd2 accidentally added a stray character outside of the - comment which broke compilation. Fix by removing. + This patch allow to call the v4 signature introduce in previous commit - Reported-by: autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742 + Signed-off-by: Matthias Gatto -- hsts: Remove pointless call to free in errorpath +- [Matthias Gatto brought this change] + + http: introduce AWS HTTP v4 Signature - The line variable will always be NULL in the error path, so remove - the free call since it's pointless. + It is a security process for HTTP. - Closes #6170 - Reviewed-by: Daniel Stenberg - -- docs: Fix various typos in documentation + It doesn't seems to be standard, but it is used by some cloud providers. - Closes #6171 - Reviewed-by: Daniel Stenberg - -Daniel Stenberg (5 Nov 2020) -- copyright: fix year ranges + Aws: + https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html + Outscale: + https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request + GCP (I didn't test that this code work with GCP though): + https://cloud.google.com/storage/docs/access-control/signing-urls-manually - Follow-up from 4d2f8006777 - -- HISTORY: the new domain - -- curl.se: new home + most of the code is in lib/http_v4_signature.c - Closes #6172 - -- KNOWN_BUGS: FTPS with Schannel times out file list operation + Information require by the algorithm: + - The URL + - Current time + - some prefix that are append to some of the signature parameters. - Reported-by: bobmitchell1956 on github - Closes #5284 - -- KNOWN_BUGS: SMB tests fail with Python 2 + The data extracted from the URL are: the URI, the region, + the host and the API type - Reported-by: Jay Satiro - Closes #5983 - -- KNOWN_BUGS: LDAPS with NSS is slow + example: + https://api.eu-west-2.outscale.com/api/latest/ReadNets + ~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ + ^ ^ ^ + / \ URI + API type region - Reported-by: nosajsnikta on github - Closes #5874 - -Sergei Nikulov (4 Nov 2020) -- travis: use ninja-build for CMake builds + Small description of the algorithm: + - make canonical header using content type, the host, and the date + - hash the post data + - make canonical_request using custom request, the URI, + the get data, the canonical header, the signed header + and post data hash + - hash canonical_request + - make str_to_sign using one of the prefix pass in parameter, + the date, the credential scope and the canonical_request hash + - compute hmac from date, using secret key as key. + - compute hmac from region, using above hmac as key + - compute hmac from api_type, using above hmac as key + - compute hmac from request_type, using above hmac as key + - compute hmac from str_to_sign using above hmac as key + - create Authorization header using above hmac, prefix pass in parameter, + the date, and above hash - Added package ninja-build to environment - Use ninja to speed up CMake builds + Signed-off-by: Matthias Gatto - Closes #6077 - -Daniel Stenberg (4 Nov 2020) -- [Harry Sintonen brought this change] - - rtsp: error out on empty Session ID, unified the code + Closes #5703 -- [Harry Sintonen brought this change] +- [Matthias Gatto brought this change] - rtsp: fixed the RTST Session ID mismatch in test 570 + http: add hmac support for sha256 - Closes #6161 - -- [Harry Sintonen brought this change] - - rtsp: fixed Session ID comparison to refuse prefix + It seems current hmac implementation use md5 for the hash, + V4 signature require sha256, so I've added the needed struct in + this commit. - Closes #6161 - -- RELEASE-NOTES: synced + I've added the functions that do the hmac in v4 signature file + as a static function ,in the next patch of the serie, + because it's used only by this file. - (forgot to update the list of contributors) - -- RELEASE-NOTES: synced + Signed-off-by: Matthias Gatto -- curlver: bumped to 7.74.0 +- [Cristian Rodríguez brought this change] -- hsts: add read/write callbacks + connect: on linux, enable reporting of all ICMP errors on UDP sockets - - read/write callback options - - man pages for the 4 new setopts - - test 1915 verifies the callbacks + The linux kernel does not report all ICMP errors back to userspace due + to historical reasons. - Closes #5896 + IP*_RECVERR sockopt must be turned on to have the correct behaviour + which is to pass all ICMP errors to userspace. + + See https://bugzilla.kernel.org/show_bug.cgi?id=202355 + + Closes #6341 -- hsts: add support for Strict-Transport-Security +- curl: add --create-file-mode [mode] - - enable in the build (configure) - - header parsing - - host name lookup - - unit tests for the above - - CI build - - CURL_VERSION_HSTS bit - - curl_version_info support - - curl -V output - - curl-config --features - - CURLOPT_HSTS_CTRL - - man page for CURLOPT_HSTS_CTRL - - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) - - man page for --hsts - - save cache to disk - - load cache from disk - - CURLOPT_HSTS - - man page for CURLOPT_HSTS - - added docs/HSTS.md - - fixed --version docs - - adjusted curl_easy_duphandle + This option sets the (octal) mode to use for the remote file when one is + created, using the SFTP, SCP or FILE protocols. When not set, the + default is 0644. - Closes #5896 - -- [Sergei Nikulov brought this change] + Closes #6244 - CI/tests: enable test target on TravisCI for CMake builds +- c-hyper: fix compiler warnings - Added test-nonflaky target to CMake builds + Identified by clang on windows. - Disabled test 1139 because the cmake build doesn't create docs/curl.1 + Reported-by: Gisle Vanem + Bug: 58974d25d8173aec154e593ed9d866da566c9811 - Closes #6074 + Closes #6351 -- tool_debug_cb: do not assume zero-terminated data +- KNOWN_BUGS: Remote recursive folder creation with SFTP - Follow-up to d70a5b5a0f5e3 + Closes #5204 -- sendf: move the verbose-check into Curl_debug +Jay Satiro (20 Dec 2020) +- badsymbols.pl: Add verbose mode -v - Saves us from having the same check done everywhere. + Use -v as the first option to enable verbose mode which will show source + input, extracted symbol and line info. For example: - Closes #6159 - -- travis: use valgrind when running tests for debug builds + Source: ./../include/curl/typecheck-gcc.h + Symbol: curlcheck_socket_info(info) + Line #423: #define curlcheck_socket_info(info) \ - Except the non-x86 and sanitizer builds + Ref: https://curl.se/mail/lib-2020-12/0084.html - Closes #6154 + Closes https://github.com/curl/curl/pull/6349 -- header.d: fix syntax mistake +- KNOWN_BUGS: Secure Transport disabling hostname validation also disables SNI - follow-up from 1144886f38fd0 - -- [Harry Sintonen brought this change] - - gnutls: fix memory leaks (certfields memory wasn't released) + That behavior is a limitation of Apple's Secure Transport. - Closes #6153 - -- tests: add missing global_init/cleanup calls + Reported-by: Cory Benfield + Reported-by: Ian Spence + Confirmed-by: Nick Zitzmann - Without the cleanup call in these test files, the mbedTLS backend leaks - memory. + Ref: https://github.com/curl/curl/issues/998 - Closes #6156 + Closes https://github.com/curl/curl/issues/6347 + Closes https://github.com/curl/curl/pull/6348 -- tool_operate: --retry for HTTP 408 responses too +Daniel Stenberg (18 Dec 2020) +- TODO: alt-svc should fallback if alt-svc doesn't work - This was inadvertently dropped from the code when the parallel support - was added. + Closes #4908 + +- travis: restrict the openssl3 job to only run https and ftps tests - Regression since b88940850 (7.66.0) + ... as it runs too long otherwise and the other tests are verified in + other builds anyway. - Reviewed-by: Jay Satiro - Closes #6155 + Closes #6345 -- http: pass correct header size to debug callback for chunked post +- build: repair http disabled but mqtt enabled build - ... when the chunked framing was added, the size of the "body part" of - the data was calculated wrongly so the debug callback would get told a - header chunk a few bytes too big that would also contain the first few - bytes of the request body. + ... as the mqtt code reuses the "method" originally used for HTTP. - Reported-by: Dirk Wetter - Ref: #6144 - Closes #6147 + Closes #6344 -- header.d: mention the "Transfer-Encoding: chunked" handling - - Ref: #6144 - Closes #6148 +- [Jon Wilkes brought this change] -- acinclude: detect manually set minimum macos/ipod version - - ... even if set in the CC or IPHONEOS/MACOSX_DEPLOYMENT_TARGET - variables. + cookie: avoid the C1001 internal compiler error with MSVC 14 - Reported-by: hamstergene on github - Fixes #6138 - Closes #6140 + Fixes #6112 + Closes #6135 -Jay Satiro (29 Oct 2020) -- tests: fix some http/2 tests for older versions of nghttpx - - - Add regex that strips http/2 server header name to those http/2 tests - that don't already have it. +- RELEASE-NOTES: synced + +- mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE - - Improve that regex in all http/2 tests. + Detected by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735 - Tests 358 and 359 were failing for me before this change on a system - that uses an older version of nghttpx which includes its version number - in the server header. + Added test 1916 and 1917 to verify. - Closes https://github.com/curl/curl/pull/6139 + Closes #6338 -Daniel Stenberg (30 Oct 2020) -- RELEASE-NOTES: synced +- travis: add CI job for Hyper build -- [Cristian Morales Vega brought this change] +- tests: updated tests for Hyper - configure: use pkgconfig to find openSSL when cross-compiling - - This reverts 736a40fec (November 2004), which doesn't explain why it was - done. +- lib: introduce c-hyper for using Hyper - Closes #6145 + ... as an alternative HTTP backend within libcurl. -- tool_operate: bail out proper on errors for parallel setup - - ... otherwise for example trying to upload a missing file just causes a - loop. +- tool_setopt: provide helper output in debug builds - Reported-by: BrumBrum on hackerone - Closes #6141 + ... for when setopt() returns error. -- [Sergei Nikulov brought this change] +- setopt: adjust to Hyper and disabled HTTP builds - CMake: make BUILD_TESTING dependent option +- rtsp: disable if Hyper is used + +- getinfo: build with disabled HTTP support + +- version: include hyper version + +- docs: add HYPER.md + +- configure: add --with-hyper - CMake will now handle BUILD_TESTING depending on PERL_FOUND and - CURL_DISABLE_TESTING + As the first (optional) HTTP backend alternative instead of native - Ref: #6036 - Closes #6072 + Close #6110 -- libssh2: fix transport over HTTPS proxy +- test1522: add debug tracing - The fix in #6021 was not enough. This fix makes sure SCP/SFTP content - can also be transfered over a HTTPS proxy. + I used this to track down some issues and I figured I could just as well + keep this extra logging in here for future needs. - Fixes #6113 - Closes #6128 + Closes #6331 -- curl.1: add an "OUTPUT" section at the top of the manpage +- http: show the request as headers even when split-sending - Explain the basic concepts behind curl output. + When the initial request isn't possible to send in its entirety, the + remainder of request would be delivered to the debug callback as data + and would wrongly be counted internally as body-bytes sent. - Inspired by #6124 + Extended test 1295 to verify. - Closes #6134 - -- mailmap: set Viktor Szakats's email + Closes #6328 -- runtests: show keywords when no tests ran +- multi: when erroring in TOOFAST state, act as for PERFORM - To help out future debugging, runtests now outputs the list of keywords - when it fails because no tests ran. + When failing in TOOFAST, the multi_done() wasn't called so the same + cleanup and handling wasn't done like when it fails in PERFORM, which in + the case of FTP could mean that the control connection wouldn't be + marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused + ftp_disconnect() to use it to send "QUIT", which could end up waiting + for a response a long time before giving up! - Ref: #6120 - Closes #6126 + Reported-by: Tomas Berger + Fixes #6333 + Closes #6337 -Jay Satiro (26 Oct 2020) -- CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo - - Reported-by: Rui LIU +- cmake: enable gophers correctly in curl-config - Closes https://github.com/curl/curl/issues/6131 + Closes #6336 -- range.d: fix typo +- test1198/9: add two mqtt publish tests without payload lengths - Follow-up to 15ae039 from earlier today. + Closes #6335 -Daniel Stenberg (26 Oct 2020) -- CI/github: work-around for brew breakage on macOS +- tests/mqttd: extract the client id from the correct offset - ... and make it use OpenSSL 1.1 properly + Closes #6334 + +- TODO: Prevent terminal injection when writing to terminal - Fixes #6130 - Closes #6129 + Closes #6150 -- [José Joaquín Atria brought this change] +- Revert "CI/github: work-around for brew breakage on macOS" + + This reverts commit 4cbb17a2cbbbe6337142d39479e21c3990b9c22f. + + ... as the work-around now causes failures. + + Closes #6332 - range.d: clarify that curl will not parse multipart responses +- examples: remove superfluous asterisk uses - Closes #6127 - Fixes #6124 + ... for function pointers. Breaks in ancient compilers. - RELEASE-NOTES: synced -- [Baruch Siach brought this change] - - libssh2: fix build with disabled proxy support - - Build breaks because the http_proxy field is missing: - - vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy' - - Regression from #6021, shipped in curl 7.73.0 +- test1272: fix line ending - Closes #6125 + Follow-up to f24784f9143 -- alt-svc: enable by default +- URL-SYNTAX: add gophers details + +- test1272: test gophers + +- runtests: add support for gophers, gopher over TLS + +- [parazyd brought this change] + + gopher: Implement secure gopher protocol. - Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported. + This commit introduces a "gophers" handler inside the gopher protocol if + USE_SSL is defined. This protocol is no different than the usual gopher + prococol, with the added TLS encapsulation upon connecting. The protocol + has been adopted in the gopher community, and many people have enabled + TLS in their gopher daemons like geomyidae(8), and clients, like clic(1) + and hurl(1). - alt-svc support in curl is no longer considered experimental + I have not implemented test units for this protocol because my knowledge + of Perl is sub-par. However, for someone more knowledgeable it might be + fairly trivial, because the same test that tests the plain gopher + protocol can be used for "gophers" just by adding a TLS listener. + + Signed-off-by: parazyd - Closes #5868 - -- CI/appveyor: remove (unused) runtests.pl -b option - -- [Emil Engler brought this change] + Closes #6208 - tool_help: make "output" description less confusing - - Currently the description of "output" is misleading when comparing it - "verbose". +- TODO: Package curl for Windows in a signed installer - Closes #6118 + Closes #5424 -- CI/appveyor: disable test 571 in two cmake builds +- mqtt: deal with 0 byte reads correctly - ... they're simply too flaky there. + OSS-Fuzz found it + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676 - Closes #6119 + Closes #6327 -- cmake: set the unicode feature in curl-config on Windows +- BUG-BOUNTY: minor language update - ... if built that way. To make it match curl -V output. + ... and remove the wording about entries from before 2019 as the "within + 12 months" is still there and covers that. - Reviewed-by: Marcel Raad - Closes #6117 + Closes #6318 -- libssh2: require version 1.0 or later +- tooĺ_writeout: fix the -w time output units - ... and simplify the code accordingly. libssh2 version 1.0 was released - in April 2009. + Fix regression from commit fc813f80e1bcac (#6248) that changed the unit + to microseconds instead of seconds with fractions - Closes #6116 + Reported-by: 不确定 + Fixes #6321 + Closes #6322 -- KNOWN_BUGS: mention the individual cmake issues +- quiche: remove fprintf() leftover + +Jay Satiro (14 Dec 2020) +- KNOWN_BUGS: SHA-256 digest not supported in Windows SSPI builds - ... to make them easier to refer to and address separately and - one-by-one. + Closes https://github.com/curl/curl/issues/6302 -- CMake: store IDN2 information in curl_config.h +- digest_sspi: Show InitializeSecurityContext errors in verbose mode - This allows the build to enable IDN properly and it makes test 1014 - happier. + The error is shown with infof rather than failf so that the user will + see the extended error message information only in verbose mode, and + will still see the standard CURLE_AUTH_ERROR message. For example: - Ref: #6074 - Closes #6108 - -- CMake: call the feature unixsockets without dash + --- - ... so that curl-config gets correct and makes test 1014 happy! + * schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED + (0x8009030A) - The per-message Quality of Protection is not supported by + the security package + * multi_done + * Connection #1 to host 127.0.0.1 left intact + curl: (94) An authentication function returned an error - Ref: #6074 - Closes #6108 - -- CI/travis: add brotli and zstd to the libssh2 build + --- - ... to make sure such tests are run with valgrind. Suppress the zstd - valgrind warnings we get with version 1.3.3 on Ubuntu 18.04 (for debug - and non-debug builds). + Ref: https://github.com/curl/curl/issues/6302 - Closes #6105 + Closes https://github.com/curl/curl/pull/6315 -- runtests: revert the mistaken edit of $CURL +Daniel Stenberg (13 Dec 2020) +- URL-SYNTAX: add default port numbers and IDNA details - Regression from c4693adc62 - -- RELEASE-NOTES: synced + Closes #6316 -- curl_url_set.3: fix typo in the RETURN VALUE section +- URL-SYNTAX: mention how FILE:// access can access network on windows - Reported-by: Basuke Suzuki - Fixes #6102 - -Jay Satiro (17 Oct 2020) -- [Daniel Stenberg brought this change] + Closes #6314 - packages/OS400: make the source code-style compliant +Jay Satiro (12 Dec 2020) +- URL-SYNTAX: Document default SMTP port 25 - ... and make sure 'make checksrc' in the root dir also verifies the - packages/OS400 sources. + Note that ports 25 and 587 are common ports for smtp, the former being + the default. - Closes https://github.com/curl/curl/pull/6085 + Closes https://github.com/curl/curl/pull/6310 -- os400: Sync libcurl API options - - This fixes the OS400 build and also an incorrect entry for - CURLINFO_APPCONNECT_TIME_T where it was treated as - CURLINFO_STARTTRANSFER_TIME_T. +Daniel Stenberg (12 Dec 2020) +- CURLOPT_URL.3: remove scheme specific details - Reported-by: Jon Rumsey + ... that are now found in URL-SYNTAX.md - Fixes https://github.com/curl/curl/issues/6083 - Closes https://github.com/curl/curl/pull/6084 + Closes #6307 -Daniel Stenberg (16 Oct 2020) -- CURLOPT_NOBODY.3: fix typo +Dan Fandrich (12 Dec 2020) +- docs: Fix some typos - Reported-by: Basuke Suzuki - Fixes #6097 + [skip ci] -Marc Hoersken (16 Oct 2020) -- CI/azure: improve on flakiness by avoiding libtool wrappers - - Install curl binaries into MinGW bin folder and use that - for the tests in order to avoid libtool wrapper binaries. - - The libtool wrapper binaries (not scripts) on Windows seem - to be one of the possible causes for the following issues: - - 1. Process output can be lost in the wrapper process chain. - 2. Killing the wrapper process does not kill the actual one. +Daniel Stenberg (12 Dec 2020) +- URL-SYNTAX: mention all supported schemes - Derived from #5904 - Closes #6049 + Closes #6311 -Daniel Stenberg (16 Oct 2020) -- CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well +- [Douglas R. Reno brought this change] -- [Zenju brought this change] + URL-SYNTAX.md: minor language improvements + + Closes #6308 - CURLOPT_TCP_NODELAY.3: fix comment in example code +- docs/URL-SYNTAX: the URL syntax curl accepts and works with - Closes #6096 + Closes #6285 -- openssl: acknowledge SRP disabling in configure properly +- [0xflotus brought this change] + + docs: enable syntax highlighting in several docs files - Follow-up to 68a513247409 + ... for better readability - Use a new separate define that is the combination of both - HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP + Closes #6286 + +- test1564/1565: require the 'wakeup' feature to run - Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html + Fixes #6299 + Fixes #6300 + Closes #6301 + +- runtests: add 'wakeup' as a feature + +- tests/server/disabled: add "wakeup" - Closes #6094 + To allow the test suite to know if wakeup support is disabled in the + build. -Viktor Szakats (16 Oct 2020) -- http3: fix two build errors, silence warnings +- lib1564/5: verify that curl_multi_wakeup returns OK + +- tests: make --libcurl tests only test FTP options if ftp enabled - * fix two build errors due to mismatch between function - declarations and their definitions - * silence two mismatched signs warnings via casts + Adjust six --libcurl tests to only check the FTP option if FTP is + actually present in the build. - Approved-by: Daniel Stenberg - Closes #6093 + Fixes #6303 + Closes #6305 -- Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 +- runtests.pl: fix "uninitialized value" warning - Approved-by: Daniel Stenberg - Closes #6092 + follow-up to e12825c642a88774 -Daniel Stenberg (16 Oct 2020) -- tool_operate: fix compiler warning when --libcurl is disabled +- runtests: add support for %if [feature] conditions - Closes #6095 - -- checksrc: warn on empty line before open brace + ... to make tests run differently or expect different results depending + on what features that are present or not in curl. - ... and fix a few occurances + Bonus: initial minor 'Hyper' awareness but nothing is using that yet - Closes #6088 + Closes #6304 -- urlapi: URL encode a '+' in the query part +- [Jon Rumsey brought this change] + + OS400: update ccsidcurl.c - ... when asked to with CURLU_URLENCODE. + Add 'struct' to cast and declaration of cfcdata to fix compilation + error. - Extended test 1560 to verify. - Reported-by: Dietmar Hauser - Fixes #6086 - Closes #6087 - -- [Cristian Morales Vega brought this change] + Fixes #6292 + Closes #6297 - libcurl.pc: make it relocatable - - It supposes when people specify the libdir/includedir they do it to - change where under prefix/exec_prefix it should be, not to make it - independent of prefix/exec_prefix. +- ngtcp2: make it build it current master again - Closes #6061 + Closes #6296 -- runtests: return error if no tests ran +- [Cristian Rodríguez brought this change] + + connect: defer port selection until connect() time - ... and make TESTFAIL stand out a little better by adding newlines - before and after. + If supported, defer port selection until connect() time + if --interface is given and source port is 0. - Reported-by: Marc Hörsken - Issue: #6052 - Closes #6053 - -- docs/FEATURE: convert to markdown + Reproducer: - ... and clean it up a bit. + * start fast webserver on port 80 + * starve system of ephemeral ports + $ sysctl net.ipv4.ip_local_port_range="60990 60999" - Closes #6067 - -- [Philipp Klaus Krause brought this change] - - strerror: use 'const' as the string should never be modified + * start a curl/libcurl "crawler" + $curl --keepalive --parallel --parallel-immediate --head --interface + 127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt" - Closes #6068 - -- [Jay Satiro brought this change] - - connect: repair build without ipv6 availability + current result: + (possible some successful data) + curl: (45) bind failed with errno 98: Address already in use - Assisted-by: Daniel Stenberg - Reported-by: Tom G. Christensen + result after patch: + (complete success or few connections failing, higlhy depending on load) - Fixes https://github.com/curl/curl/issues/6069 - Closes https://github.com/curl/curl/pull/6071 - -- RELEASE-NOTES: synced + Fail only when all the possible 4-tuple combinations are exhausted, + which is impossible to do when port is selected at bind() time becuse + the kernel does not know if socket will be listen()'ed on or connect'ed + yet. - Started over for the journey to next release. + Closes #6295 -- src/tool_filetime: disable -Wformat on mingw for this file +- [Hans-Christian Noren Egtvedt brought this change] + + connect: zero variable on stack to silence valgrind complaint - With gcc 10 on mingw we otherwise get this warning: + Valgrind will complain that ssrem buffer usage if not explicit + initialized, hence initialize it to zero. - error: ISO C does not support the 'I' printf flag [-Werror=format=] + This completes the change intially started in commit 2c0d7212151 ('ftp: + retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has + a similar memset to zero. - Fixes #6079 - Closes #6082 + Signed-off-by: Hans-Christian Noren Egtvedt + Closes #6289 -- test122[12]: remove these two tests - - ... and remove the objnames scripts they tested. They're not used for - anything anymore so testing them serves no purpose! +- RELEASE-NOTES: synced - Reported-by: Marc Hörsken - Fixes #6080 - Closes #6081 + start over on the next release cycle -Version 7.73.0 (14 Oct 2020) +Version 7.74.0 (9 Dec 2020) -Daniel Stenberg (14 Oct 2020) +Daniel Stenberg (9 Dec 2020) - RELEASE-NOTES: synced - for 7.73.0 + for 7.74.0 -- THANKS: from 7.73.0 and .mailmap fixes +Jay Satiro (7 Dec 2020) +- [Jacob Hoffman-Andrews brought this change] -- mailmap: fixups of some contributors + urldata: restore comment on ssl_connect_data.use + + This comment was originally on the `use` field, but was separated from + its field in 62a2534. + + Closes https://github.com/curl/curl/pull/6287 -- projects/build-wolfssl.bat: fix the copyright year range +Daniel Stenberg (7 Dec 2020) +- VERSIONS: refreshed + + We always use the patch number these days: all releases are + "major.minor.patch" -Marc Hoersken (14 Oct 2020) -- [Sergei Nikulov brought this change] +- [Jakub Zakrzewski brought this change] - CI/tests: fix invocation of tests for CMake builds - - Update appveyor.yml to set env variable TFLAGS and run tests - Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) - Move testdeps build to build step (per review comments) + cmake: don't use reserved target name 'test' - Reviewed-by: Marc Hörsken + CMake up to 3.10 always reserves this name - Closes #6066 - Fixes #6052 + Fixes #6257 + Closes #6258 -- tests/server/util.c: fix support for Windows Unicode builds +- openssl: make the OCSP verification verify the certificate id - Detected via #6066 - Closes #6070 - -Daniel Stenberg (13 Oct 2020) -- [Jay Satiro brought this change] + CVE-2020-8286 + + Reported by anonymous + + Bug: https://curl.se/docs/CVE-2020-8286.html - strerror: Revert to local codepage for Windows error string +- ftp: make wc_statemach loop instead of recurse - - Change get_winapi_error() to return the error string in the local - codepage instead of UTF-8 encoding. + CVE-2020-8285 - Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it - also changed the error string's encoding from local codepage to UTF-8. + Fixes #6255 + Bug: https://curl.se/docs/CVE-2020-8285.html + Reported-by: xnynx on github + +- ftp: CURLOPT_FTP_SKIP_PASV_IP by default - We return the local codepage version of the error string because if it - is output to the user's terminal it will likely be with functions which - expect the local codepage (eg fprintf, failf, infof). + The command line tool also independently sets --ftp-skip-pasv-ip by + default. - This is essentially a partial revert of bed5f84. The support for xbox - remains but the error string is reverted back to local codepage. + Ten test cases updated to adapt the modified --libcurl output. - Ref: https://github.com/curl/curl/pull/6005 + Bug: https://curl.se/docs/CVE-2020-8284.html + CVE-2020-8284 - Reviewed-by: Marcel Raad - Closes #6065 + Reported-by: Varnavas Papaioannou -Marc Hoersken (13 Oct 2020) -- CI/tests: use verification curl for test reporting APIs +- urlapi: don't accept blank port number field without scheme - Avoid using our own, potentially installed, curl for - the test reporting APIs in case it is broken. + ... as it makes the URL parser accept "very-long-hostname://" as a valid + host name and we don't want that. The parser now only accepts a blank + (no digits) after the colon if the URL starts with a scheme. - Reviewed-by: Daniel Stenberg + Reported-by: d4d on hackerone - Preparation for #6049 - Closes #6063 + Closes #6283 -Viktor Szakats (12 Oct 2020) -- windows: fix comparison of mismatched types warning +- Revert "multi: implement wait using winsock events" - clang 10, mingw-w64: - ``` - vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') - [-Wsign-compare] - if(GetLastError() != CRYPT_E_NOT_FOUND) - ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ - ``` + This reverts commit d2a7d7c185f98df8f3e585e5620cbc0482e45fac. - Approved-by: Daniel Stenberg - Closes #6062 + This commit also reverts the subsequent follow-ups to that commit, which + were all done within windows #ifdefs that are removed in this + change. Marc helped me verify this. + + Fixes #6146 + Closes #6281 -Daniel Stenberg (11 Oct 2020) -- [Viktor Szakats brought this change] +- [Klaus Crusius brought this change] - src/Makefile.m32: fix undefined curlx_dyn_* errors + ftp: retry getpeername for FTP with TCP_FASTOPEN - by linking `lib/dynbuf.c` when building a static curl binary. - Previously this source file was only included when building - a dynamic curl binary. This was likely possibly because no - functions from the `src/Makefile.inc` / `CURLX_CFILES` sources - were actually required for a curl tool build. This has - recently changed with the introduction of `curlx_dyn_*()` - memory functions and their use by the tool sources. + In the case of TFO, the remote host name is not resolved at the + connetion time. - Closes #6060 - -- HISTORY: curl verifies SSL certs by default since version 7.10 - -Marc Hoersken (8 Oct 2020) -- runtests.pl: use $LIBDIR variable instead of hardcoded path + For FTP that has lead to missing hostname for the secondary connection. + Therefore the name resolution is done at the time, when FTP requires it. - Reviewed-by: Daniel Stenberg - Closes #6051 + Fixes #6252 + Closes #6265 + Closes #6282 -Daniel Stenberg (7 Oct 2020) -- checksrc: detect // comments on column 0 +- [Thomas Danielsson brought this change] + + scripts/completion.pl: parse all opts - Spotted while working on #6045 + For tab-completion it may be preferable to include all the + available options. - Closes #6048 + Closes #6280 -- [Frederik Wedel-Heinen brought this change] +- RELEASE-NOTES: synced - mbedtls: add missing header when defining MBEDTLS_DEBUG +- openssl: use OPENSSL_init_ssl() with >= 1.1.0 - Closes #6045 + Reported-by: Kovalkov Dmitrii and Per Nilsson + Fixes #6254 + Fixes #6256 + Closes #6260 -- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long +- SECURITY-PROCESS: disclose on hackerone - Previously, it would pass on a define (int) which could make libcurl - read junk as a value - which prevented the CURLOPT_IPRESOLVE option to - "take". This could then make test 2100 do two DoH requests instead of - one! + Once a vulnerability has been published, the hackerone issue should be + disclosed. For tranparency. - Fixes #6042 - Closes #6043 - -- RELEASE-NOTES: synced + Closes #6275 -- scripts/release-notes.pl: don't "embed" $ in format string for printf() +Marc Hoersken (3 Dec 2020) +- tests/util.py: fix compatibility with Python 2 - ... since they might contain %-codes that mess up the output! + Backporting the Python 3 implementation of setStream + to ClosingFileHandler as a fallback within Python 2. + + Reported-by: Jay Satiro + + Fixes #6259 + Closes #6270 -Jay Satiro (5 Oct 2020) -- [M.R.T brought this change] +Daniel Gustafsson (3 Dec 2020) +- docs: fix typos and markup in ETag manpage sections + + Reported-by: emanruse on github + Fixes #6273 - build-wolfssl: fix build with Visual Studio 2019 +Daniel Stenberg (2 Dec 2020) +- quiche: close the connection - Closes https://github.com/curl/curl/pull/6033 + Reported-by: Junho Choi + Fixes #6213 + Closes #6217 -Daniel Stenberg (4 Oct 2020) -- runtests: add %repeat[]% for test files +Jay Satiro (2 Dec 2020) +- ngtcp2: Fix build error due to symbol name change - ... and use this new keywords in all the test files larger than 50K to reduce - their sizes and make them a lot easier to read and understand. + - NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION - Closes #6040 - -- [Emil Engler brought this change] - - --help: move two options from the misc category + ngtcp2/ngtcp2@76232e9 changed the name. - The cmdline opts delegation and suppress-connect-headers - fit better into auth and proxy rather than misc. + ngtcp2 master is required to build curl with http3 support. - Follow-up to aa8777f63febc - Closes #6038 + Closes https://github.com/curl/curl/pull/6271 -- [Samanta Navarro brought this change] +Daniel Stenberg (1 Dec 2020) +- [Klaus Crusius brought this change] - docs/opts: fix typos in two manual pages + cmake: check for linux/tcp.h - Closes #6039 - -- ldap: reduce the amount of #ifdefs needed + The HAVE_LINUX_TCP_H define was not set by cmake. - Closes #6035 + Closes #6252 -- runtests: provide curl's version string as %VERSION for tests +- NEW-PROTOCOL: document what needs to be done to add one - ... so that we can check HTTP requests for User-Agent: curl/%VERSION + Closes #6263 + +- splay: rename Curl_splayremovebyaddr to Curl_splayremove - Update 600+ test cases accordingly. + ... and remove the old unused proto for the old Curl_splayremove + version. - Closes #6037 + Closes #6269 -- checksrc: warn on space after exclamation mark +- openssl: free mem_buf in error path - Closes #6034 - -- test1465: verify --libcurl with binary POST data - -- runtests: allow generating a binary sequence from hex - -- tool_setopt: escape binary data to hex, not octal - -- curl: make --libcurl show binary posts correctly + To fix a memory-leak. - Reported-by: Stephan Mühlstrasser - Fixes #6031 - Closes #6032 + Closes #6267 -Jay Satiro (1 Oct 2020) -- strerror: fix null deref on winapi out-of-memory +- openssl: remove #if 0 leftover - Follow-up to bed5f84 from several days ago. + Follow-up to 4c9768565ec3a9 (from Sep 2008) - Ref: https://github.com/curl/curl/pull/6005 - -Daniel Stenberg (1 Oct 2020) -- [Kamil Dudka brought this change] + Closes #6268 - vtls: deduplicate some DISABLE_PROXY ifdefs +- ntlm: avoid malloc(0) on zero length user and domain - ... in the code of gtls, nss, and openssl + ... and simplify the too-long checks somewhat. - Closes #5735 + Detected by OSS-Fuzz + + Closes #6264 - RELEASE-NOTES: synced -- [Emil Engler brought this change] - - TODO: Add OpenBSD libtool notice +Marc Hoersken (28 Nov 2020) +- tests/server/tftpd.c: close upload file in case of abort - See #5862 - Closes #6030 - -- tests/unit/README: convert to markdown + Commit c353207 removed the closing right after do_tftp + which covered the case of abort. This handles that case. - ... and add to dist! + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #6028 + Follow up to #6209 + Closes #6234 -- tests/README: convert to markdown - - Closes #6028 +Daniel Stenberg (26 Nov 2020) +- [Daiki Ueno brought this change] -- include/README: convert to markdown + ngtcp2: use the minimal version of QUIC supported by ngtcp2 - Closes #6028 + Closes #6250 -- examples/README: convert to markdown - - Closes #6028 +- [Daiki Ueno brought this change] -- configure: don't say HTTPS-proxy is enabled when disabled! + ngtcp2: advertise h3 ALPN unconditionally - Reported-by: Kamil Dudka - Reviewed-by: Kamil Dudka - Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 - Closes #6029 + Closes #6250 -Daniel Gustafsson (30 Sep 2020) -- src: Consistently spell whitespace without whitespace +- [Daiki Ueno brought this change] + + vquic/ngtcp2.h: define local_addr as sockaddr_storage - Whitespace is spelled without a space between white and space, so - make sure to consistently spell it that way across the codebase. + This field needs to be wide enough to hold sockaddr_in6 when + connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the + packets because of the address mismatch: + I00000022 [...] con ignore packet from unknown path - Closes #6023 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler + We can safely assume that struct sockaddr_storage is available, as it + is used in the public interface of ngtcp2. + + Closes #6250 -- MANUAL: update examples to resolve without redirects +- socks: check for DNS entries with the right port number - www.netscape.com is redirecting to a cookie consent form on Aol, and - cool.haxx.se isn't responding to FTP anymore. Replace with examples - that resolves in case users try out the commands when reading the - manual. + The resolve call is done with the right port number, but the subsequent + check used the wrong one, which then could find a previous resolve which + would return and leave the fresh resolve "incomplete" and leaking + memory. - Closes #6024 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler - -Daniel Stenberg (30 Sep 2020) -- HISTORY: add some 2020 events + Fixes #6247 + Closes #6253 -- sectransp: make it build with --disable-proxy +- curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use - Follow-up from #5466 and f3d501dc678d80 - Reported-by: Javier Navarro - Fixes #6025 - Closes #6026 + ... so don't define it when instructed to use c-ares! -- ECH: renamed from ESNI in docs and configure +- test506: make it not run in c-ares builds - Encrypted Client Hello (ECH) is the current name. + As the asynch nature of it may trigger events in another order. A c-ares + upgrade made it break. - Closes #6022 + Reported-by: Marc Hörsken + Fixes #6247 -- configure: use "no" instead of "disabled" for the end summary +- runtests: make 'c-ares' a "feature" to depend on - ... for consistency but also to make them more distinctly stand out next - to the "enabled" lines. + ... also added to the docs. -- TODO: SSH over HTTPS proxy with more backends +- tool_writeout: use off_t getinfo-types instead of doubles - ... as right now only the libssh2 backend supports it. - -- libssh2: handle the SSH protocols done over HTTPS proxy + Commit 3b80d3ca46b12e52342 (June 2017) introduced getinfo replacement + variables that use curl_off_t instead of doubles. Switch the --write-out + function over to use them. - Reported-by: Robin Douine - Fixes #4295 - Closes #6021 + Closes #6248 - [Emil Engler brought this change] - memdebug: remove 9 year old unused debug function + file: avoid duplicated code sequence - There used to be a way to have memdebug fill allocated memory. 9 years - later this has no value there (valgrind and ASAN etc are way better). If - people need to know about it they can have a look at VCS logs. + file_disconnect() is identical with file_do() except the function header + but as the arguments are unused anyway so why not just return file_do() + directly! - Closes #5973 + Reviewed-by: Daniel Stenberg + Closes #6249 -- sendf: move Curl_sendf to dict.c and make it static - - ... as the only remaining user of that function. Also fix gopher.c to - instead use Curl_write() - - Closes #6020 +- [Rikard Falkeborn brought this change] -- ROADMAP: updates and cleanups + infof/failf calls: fix format specifiers - Fix the HSTS PR + Update a few format specifiers to match what is being printed. - Remove DoT, thread-safe init and hard-coded localhost. I feel very - little interest for these with users so I downgrade them to plain "TODO" - entries again. + Closes #6241 -- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root +- docs/INTERNALS: remove reference to Curl_sendf() - This matches what is returned in other TLS backends in the same - situation. + The function has been removed from common usage. Also removed comment in + gopher.c that still referenced it. - Reviewed-by: Jay Satiro - Reviewed-by: Emil Engler - Follow-up to 5a3efb1 - Reported-by: iammrtau on github - Fixes #6003 - Closes #6018 + Reported-by: Rikard Falkeborn + Fixes #6242 + Closes #6243 -- RELEASE-NOTES: synced +- [Rikard Falkeborn brought this change] -- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL + examples: update .gitignore - Added test 348 to verify. Added a 'STOR' command to the test FTP - server to enable test 348. Documented the command in FILEFORMAT.md + Add files that are generated by 'make examples' and remove some that + have been renamed. - Reported-by: Duncan Wilcox - Fixes #6016 - Closes #6017 + The commits that renamed the programs are e9625c5bc6c046a (imap.c and + simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and + ad39e7ec01e7 (pop3slist.c and pop3s.c were renamed to pop3-list.c and + pop3-ssl.c). + + Closes #6240 -- pause: only trigger a reread if the unpause sticks +- asyn: use 'struct thread_data *' instead of 'void *' - As an unpause might itself get paused again and then triggering another - reread doesn't help. + To reduce use of types that can't be checked at compile time. Also + removes several typecasts. - Follow-up from e040146f22608fd9 (shipped since 7.69.1) + ... and rename the struct field from 'os_specific' to 'tdata'. - Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html - Patch-by: Kunal Chandarana - Fixes #5988 - Closes #6013 + Closes #6239 + Reviewed-by: Jay Satiro -- test163[12]: require http to be built-in to run +Viktor Szakats (23 Nov 2020) +- Makefile.m32: add support for UNICODE builds - ... as speaking over an HTTPS proxy implies http! + It requires the linker to support the `-municode` option. + This is available in more recent mingw-w64 releases. - Closes #6014 - -- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define + Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html + Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847 - Closes #6012 - -- [Javier Blazquez brought this change] + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + + Closes #6228 - strerror: honor Unicode API choice on Windows +Daniel Stenberg (23 Nov 2020) +- urldata: remove 'void *protop' and create the union 'p' - Closes #6005 + ... to avoid the use of 'void *' for the protocol specific structs done + per transfer. + + Closes #6238 -- imap: make imap_send use dynbuf for the send buffer management +- winbuild: remove docs from Makefiles and refer to README.md + + Reduce risk for conflicting docs and makes it to a single place to fix + and polish. - Reuses the buffer and thereby reduces number of mallocs over a transfer. + add these missing options to the readme: - Closes #6010 - -- Curl_send: return error when pre_receive_plain can't malloc + ENABLE_OPENSSL_AUTO_LOAD_CONFIG and ENABLE_UNICODE - ... will probably trigger some false DEAD CODE positives on non-windows - code analyzers for the conditional code. + clarify ENABLE_SCHANNEL default varies - Closes #6011 + Fixes #6216 + Closes #6227 + Co-Authored-by: Jay Satiro -- ftp: separate FTPS from FTP over "HTTPS proxy" +- [Daiki Ueno brought this change] + + http3: use the master branch of GnuTLS for testing - When using HTTPS proxy, SSL is used but not in the view of the FTP - protocol handler itself so separate the connection's use of SSL from the - FTP control connection's sue. + Closes #6235 + +- KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation - Reported-by: Mingtao Yang - Fixes #5523 - Closes #6006 + Closes #5839 -Dan Fandrich (23 Sep 2020) -- tests/data: Fix some mismatched XML tags in test cases +- KNOWN_BUGS: wakeup socket disconnect causes havoc - This allows these test files to pass xmllint. + Closes #6132 + Closes #6133 -Daniel Stenberg (23 Sep 2020) -- pingpong: use a dynbuf for the *_pp_sendf() function +- RELEASE-NOTES: synced + +- [Oliver Urbann brought this change] + + curl: add compatibility for Amiga and GCC 6.5 - ... reuses the same dynamic buffer instead of doing repeated malloc/free - cycles. + Changes are mainly reordering and adding of includes required + to compile with a more recent version of GCC. - Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls - after this change in my test setup (132 => 125), curl 7.72.0 needed 140 - calls for this. + Closes #6220 + +Marc Hoersken (20 Nov 2020) +- tests/server/tftpd.c: close upload file right after transfer - Test case 103 makes 9 less allocations now (130). Down from 149 in - 7.72.0. + Make sure uploaded file is no longer locked after the + transfer while waiting for the final ACK to be handled. - Closes #6004 - -- dynbuf: add Curl_dyn_vaddf + Assisted-by: Daniel Stenberg - Closes #6004 + Bug: #6058 + Closes #6209 -- dynbuf: make *addf() not require extra mallocs +- CI/cirrus: simplify logic for disabled tests - ... by introducing a printf() function that appends directly into a - dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if - the buffer is already big enough it can just printf directly into it. + The OpenSSH server instance for the testsuite cannot + be started on FreeBSD, therefore the SFTP and SCP + tests are disabled right away from the beginning. - Since this less-malloc version requires tthe use of a library internal - printf function, we only provide this version when building libcurl and - not for the dynbuf code that is used when building the curl tool. + The previous OS version specific logic for SKIP_TESTS + is no longer needed/used and can therefore be removed. - Closes #5998 - -- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport + Reviewed-by: Daniel Stenberg - Closes #5403 + Follow up to #6211 + Closes #6229 -- pingpong: remove a malloc per Curl_pp_vsendf call +Daniel Gustafsson (20 Nov 2020) +- mailmap: Daniel Hwang - This typically makes 7-9 fewer mallocs per FTP transfer. + Add Daniel Hwang to the mailmap to cover the alternative spelling + Daniel Lee Hwang which was used in one commit. - Closes #5997 + Closes #6230 + Reviewed-by: Daniel Stenberg -- symbian: drop support +- openssl: guard against OOM on context creation - The OS is deprecated. I see no traces of anyone having actually built - curl for Symbian after 2012. + EVP_MD_CTX_create will allocate memory for the context and returns + NULL in case the allocation fails. Make sure to catch any allocation + failures and exit early if so. - The public headers are unmodified. + In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex + as the latter is intended for ENGINE selection which we don't do. - Closes #5989 + Closes #6224 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- RELEASE-NOTES: synced +Daniel Stenberg (19 Nov 2020) +- [Vincent Torri brought this change] -- curl_krb5.h: rename from krb5.h - - Follow-up from f4873ebd0be32cf + cmake: use libcurl.rc in all Windows builds - Turns out some older openssl installations go bananas otherwise. - Reported-by: Tom van der Woerdt - Fixes #5995 - Closes #5996 + Reviewed-by: Marcel Raad + Closes #6215 -- test1297: verify GOT_NOTHING with http proxy tunnel +- [Cristian Morales Vega brought this change] -- http_proxy: do not count proxy headers in the header bytecount + cmake: make CURL_ZLIB a tri-state variable - ... as that counter is subsequently used to detect if nothing was - returned from the peer. This made curl return CURLE_OK when it should - have returned CURLE_GOT_NOTHING. + By differentiating between ON and AUTO it can make a missing zlib + library a hard error when CURL_ZLIB=ON is used. - Fixes #5992 - Reported-by: Tom van der Woerdt - Closes #5994 + Reviewed-by: Jakub Zakrzewski + Closes #6221 + Fixes #6173 -- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument +- quiche: remove 'static' from local buffer - Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the - option is, yeah, not known. Clarified this in the setopt man page too. + For thread-safety - Closes #5993 + Closes #6223 -- krb5: merged security.c and krb specific FTP functions in here - - These two files were always tightly connected and it was hard to - understand what went into which. This also allows us to make the - ftpsend() function static (moved from ftp.c). - - Removed security.c - Renamed curl_sec.h to krb5.h +- KNOWN_BUGS: cmake: libspsl is not supported - Closes #5987 + Closes #6214 -- Curl_handler: add 'family' to each protocol - - Makes get_protocol_family() faster and it moves the knowledge about the - "families" to each protocol handler, where it belongs. +- KNOWN_BUGS: cmake autodetects cert paths when cross-compiling - Closes #5986 + Closes #6178 -- parsedate: tune the date to epoch conversion - - By avoiding an unnecessary error check and the temp use of the tm - struct, the time2epoch conversion function gets a little bit faster. - When repeating test 517, the updated version is perhaps 1% faster (on - one particular build on one particular architecture). +- KNOWN_BUGS: cmake build doesn't fail if zlib not found - Closes #5985 + Closes #6173 -- cmake: remove scary warning - - Remove the text saying - - "the curl cmake build system is poorly maintained. Be aware" +- KNOWN_BUGS: cmake libcurl.pc uses absolute library paths - ... not because anything changed just now, but to encourage users to use - it and subsequently improve it. + Closes #6169 + +- KNOWN_BUGS: cmake: generated .pc file contains strange entries - Closes #5984 + Closes #6167 -- docs/MQTT: remove outdated paaragraphs +- KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads + + Closes #6166 -- docs/MQTT: not experimental anymore +- KNOWN_BUGS: cmake build in Linux links libcurl to libdl - Follow-up to e37e4468688d8f + Closes #6165 -- docs/RESOURCES: remove +- KNOWN_BUGS: make a new section for cmake topics - This document is not maintained and rather than trying to refresh it, - let's kill it. A more up-to-date document with relevant RFCs is this - page on the curl website: https://curl.haxx.se/rfc/ + Closes #6219 + +- [Emil Engler brought this change] + + cirrus: build with FreeBSD 12.2 in CirrusCI - Closes #5980 + Closes #6211 -- docs/TheArtOfHttpScripting: convert to markdown +Marc Hoersken (14 Nov 2020) +- tests/*server.py: close log file after each log line - Makes it easier to browse on github etc. Offers (better) links. + Make sure the log file is not locked once a test has + finished and align with the behavior of our logmsg. - It should be noted that this document is already mostly outdated and - "Everything curl" at https://ec.haxx.se/ is a better resource and - tutorial. + Rename curl_test_data.py to be a general util.py. + Format and sort Python imports with isort/VSCode. - Closes #5981 + Bug: #6058 + Closes #6206 -- BUGS: convert document to markdown +Daniel Stenberg (13 Nov 2020) +- CURLOPT_HSTS.3: document the file format - Closes #5979 + Closes #6205 -- --help: strdup the category +- RELEASE-NOTES: synced + +- release-notes.pl: detect #[number] better for Ref: etc + +- curl: only warn not fail, if not finding the home dir - ... since it is converted and the original pointer is freed on Windows - unicode handling. + ... as there's no good reason to error out completely. - Follow-up to aa8777f63febc - Fixes #5977 - Closes #5978 - Reported-by: xwxbug on github + Reported-by: Andreas Fischer + Fixes #6200 + Closes #6201 -- CHECKSRC: document two missing warnings +- httpput-postfields.c: new example doing PUT with POSTFIELDS + + Proposed-by: Jeroen Ooms + Ref: #6186 + Closes #6188 -- RELEASE-NOTES: synced +- [Tobias Hieta brought this change] -- ftp: avoid risk of reading uninitialized integers + cmake: correctly handle linker flags for static libs - If the received PASV response doesn't match the expected pattern, we - could end up reading uninitialized integers for IP address and port - number. + curl CMake was setting the the EXE flags for static libraries which made + the /manifest:no flag ended up when linking the static library, which is + not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit + with an error. - Issue pointed out by muse.dev - Closes #5972 + The better way to handle this is to make sure that we pass the correct + linker flags to CMAKE_STATIC_LINKER_FLAGS instead. + + Reviewed-by: Jakub Zakrzewski + Closes #6195 -- [Quentin Balland brought this change] +- [Tobias Hieta brought this change] - easy_reset: clear retry counter + cmake: don't pass -fvisibility=hidden to clang-cl on Windows - Closes #5975 - Fixes #5974 - -- ftp: get rid of the PPSENDF macro + When using clang-cl on windows -fvisibility=hidden is not an known + argument. Instead it behaves exactly like MSVC in this case. So let's + make sure we take that path. - The use of such a macro hides some of what's actually going on to the - reader and is generally disapproved of in the project. + In CMake clang-cl sets both CMAKE_C_COMPILER_ID=clang and MSVC get's + defined since clang-cl is basically a MSVC emulator. So guarding like we + do in this patch seems logical. - Closes #5971 + Reviewed-by: Jakub Zakrzewski + Closes #6194 -- man pages: switch to https://example.com URLs +- http_proxy: use enum with state names for 'keepon' - Since HTTPS is "the new normal", this update changes a lot of man page - examples to use https://example.com instead of the previous "http://..." + To make the code clearer, change the 'keepon' from an int to an enum + with better state names. - Closes #5969 + Reported-by: Niranjan Hasabnis + Bug: https://curl.se/mail/lib-2020-11/0026.html + Closes #6193 -- github: remove the duplicate "Security vulnerability" entry +- curl_easy_escape: limit output string length to 3 * max input - ... since github adds an entry automatically by itself. + ... instead of the limiting it to just the max input size. As every + input byte can be expanded to 3 output bytes, this could limit the input + string to 2.66 MB instead of the intended 8 MB. - Closes #5970 - -- [Emil Engler brought this change] + Reported-by: Marc Schlatter + Closes #6192 - github: use new issue template feature +- docs: document the 8MB input string limit - This helps us to avoid getting feature requests as well as security - bugs reported into the issue tracker. + for curl_easy_escape and curl_easy_setopt() - Closes #5936 - -- [Emil Engler brought this change] + The limit is there to catch mistakes and abuse. It is meant to be large + enough to allow virtually all "fine" use cases. + + Reported-by: Marc Schlatter + Fixes #6190 + Closes #6191 - urlapi: use more Curl_safefree +- mqttd: fclose test file when done - Closes #5968 + Reported-by: Marc Hörsken + Reviewed-by: Jay Satiro + Bug: #6058 + Closes #6189 -Marc Hoersken (17 Sep 2020) -- multi: align WinSock mask variables in Curl_multi_wait +- RELEASE-NOTES: synced + +- THANKS-filter: ignore autobuild links + +- Revert "libcurl.pc: make it relocatable" - Also skip pre-checking sockets to set timeout_ms to 0 - after the first socket has been detected to be ready. + This reverts commit 3862c37b6373a55ca704171d45ba5ee91dec2c9f. - Reviewed-by: rcombs on github - Reviewed-by: Daniel Stenberg + That fix should either be done differently or with an option. - Follow up to #5886 + Reported-by: asavah on github + Fixes #6157 + Closes #6183 -- multi: reuse WinSock events variable in Curl_multi_wait - - Since the struct is quite large (1 long and 10 ints) we - declare it once at the beginning of the function instead - of multiple times inside loops to avoid stack movements. +- examples/httpput: remove use of CURLOPT_PUT - Reviewed-by: Viktor Szakats - Reviewed-by: Daniel Stenberg + It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD. - Closes #5886 + Reported-by: Jeroen Ooms + Fixes #6186 + Closes #6187 -Daniel Stenberg (16 Sep 2020) -- TODO: dynamically decide to use socketpair +- Curl_pgrsStartNow: init speed limit time stamps at start - Suggested-by: Anders Bakken + By setting the speed limit time stamps unconditionally at transfer + start, we can start off a transfer without speed limits and yet allow + them to get set during transfer and have an effect. - Closes #4829 + Reported-by: Kael1117 on github + Fixes #6162 + Closes #6184 -- TODO: add PR reference for native IDN support on macOS +- ngtcp2: adapt to recent nghttp3 updates - As there was work started on this that never got completed. + 'reset_stream' was added to the nghttp3_conn_callbacks struct - Closes #5371 + Closes #6185 -- tool_help.h: update copyright year range +- configure: pass -pthread to Libs.private for pkg-config - Follow-up from aa8777f63febca + Reported-by: Cristian Morales Vega + Fixes #6168 + Closes #6181 -- CI/azure: disable test 571 in the msys2 builds +- altsvc: minimize variable scope and avoid "DEAD_STORE" - It's just too flaky there + Closes #6182 + +- FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?" - Reviewed-by: Marc Hoersken - Closes #5954 + This hasn't been the case for a while now, remove. -- tool_writeout: protect fputs() from NULL +- FAQ: refresh "Why do I get "certificate verify failed" - When the code was changed to do fputs() instead of fprintf() it got - sensitive for NULL pointers; add checks for that. + Add more details, remove references to ancient curl version. + +- test493: verify --hsts upgrade and that %{url_effective} reflects that - Follow-up from 0c1e767e83ec66 + Closes #6175 + +- url: make sure an HSTS upgrade updates URL and scheme correctly - Closes #5963 + Closes #6175 -- test3015: verify stdout "as text" +- tool_operate: set HSTS with CURLOPT_HSTS to pass on filename - Follow-up from 0c1e767e83e to please win32 tests + Closes #6175 + +- hsts: remove debug code leftovers - Closes #5962 + Closes #6175 -- travis: use libressl v3.1.4 instead of master +- FAQ: refreshed - ... as their git master seems too fragile to use (and 3.2.1 which is the - latest has a build failure). + - remove a few ancient questions + - add configure with static libs question + - updated wording in several places + - lowercased curl - Closes #5964 - -- tests/FILEFORMAT: document type=shell for + Closes #6177 -- tests/FILEFORMAT: document nonewline support for +Daniel Gustafsson (5 Nov 2020) +- examples: fix comment syntax - The one in , that creates files. + Commit ac0a88fd2 accidentally added a stray character outside of the + comment which broke compilation. Fix by removing. - Follow-up from b83947c8df7 - -- [anio brought this change] + Reported-by: autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742 - tool_writeout: add new writeout variable, %{num_headers} +- hsts: Remove pointless call to free in errorpath - This variable gives the number of headers. + The line variable will always be NULL in the error path, so remove + the free call since it's pointless. - Closes #5947 + Closes #6170 + Reviewed-by: Daniel Stenberg -- tool_urlglob: fix compiler warning "unreachable code" +- docs: Fix various typos in documentation - (On Windows builds.) + Closes #6171 + Reviewed-by: Daniel Stenberg + +Daniel Stenberg (5 Nov 2020) +- copyright: fix year ranges - Follow-up to 70a3b003d9 + Follow-up from 4d2f8006777 -- [Gergely Nagy brought this change] +- HISTORY: the new domain - vtls: deduplicate client certificates in ssl_config_data +- curl.se: new home - Closes #5629 + Closes #6172 -- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND +- KNOWN_BUGS: FTPS with Schannel times out file list operation - This is primarily interesting for cases where CURLOPT_NOBODY is set as - previously curl would not return an error for this case. + Reported-by: bobmitchell1956 on github + Closes #5284 + +- KNOWN_BUGS: SMB tests fail with Python 2 - MDTM getting 550 now also returns this error (it returned - CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for - missing files across protocols and specific FTP commands. + Reported-by: Jay Satiro + Closes #5983 + +- KNOWN_BUGS: LDAPS with NSS is slow - libcurl already returns error on a 550 as a MDTM response (when - CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would - happen subsequently anyway since the RETR command would fail. + Reported-by: nosajsnikta on github + Closes #5874 + +Sergei Nikulov (4 Nov 2020) +- travis: use ninja-build for CMake builds - Add test 1913 and 1914 to verify. Updated several tests accordingly due - to the updated SIZE behavior. + Added package ninja-build to environment + Use ninja to speed up CMake builds - Reported-by: Tomas Berger - Fixes #5953 - Closes #5957 + Closes #6077 -- curl: make checkpasswd use dynbuf - - Closes #5952 +Daniel Stenberg (4 Nov 2020) +- [Harry Sintonen brought this change] -- curl: make glob_match_url use dynbuf - - Closes #5952 + rtsp: error out on empty Session ID, unified the code -- curl: make file2memory use dynbuf - - Closes #5952 +- [Harry Sintonen brought this change] -- curl: make file2string use dynbuf + rtsp: fixed the RTST Session ID mismatch in test 570 - Closes #5952 + Closes #6161 -- [Antarpreet Singh brought this change] +- [Harry Sintonen brought this change] - imap: set cselect_bits to CURL_CSELECT_IN initially - - ... when continuing a transfer from a FETCH response. + rtsp: fixed Session ID comparison to refuse prefix - When the size of the file was small enough that the entirety of the - transfer happens in a single go and schannel buffers holds the entire - data. However, it wasn't completely read in Curl_pp_readresp since a - line break was found before that could happen. So, by the time we are in - imap_state_fetch_resp - there's data in buffers that needs to be read - via Curl_read but nothing to read from the socket. After we setup a - transfer (Curl_setup_transfer), curl just waits on the socket state to - change - which doesn't happen since no new data ever comes. + Closes #6161 + +- RELEASE-NOTES: synced - Closes #5961 + (forgot to update the list of contributors) - RELEASE-NOTES: synced -- test434: test -K use in a single line without newline +- curlver: bumped to 7.74.0 + +- hsts: add read/write callbacks - Closes #5946 + - read/write callback options + - man pages for the 4 new setopts + - test 1915 verifies the callbacks + + Closes #5896 -- runtests: allow creating files without newlines +- hsts: add support for Strict-Transport-Security - Closes #5946 + - enable in the build (configure) + - header parsing + - host name lookup + - unit tests for the above + - CI build + - CURL_VERSION_HSTS bit + - curl_version_info support + - curl -V output + - curl-config --features + - CURLOPT_HSTS_CTRL + - man page for CURLOPT_HSTS_CTRL + - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) + - man page for --hsts + - save cache to disk + - load cache from disk + - CURLOPT_HSTS + - man page for CURLOPT_HSTS + - added docs/HSTS.md + - fixed --version docs + - adjusted curl_easy_duphandle + + Closes #5896 -- curl: use curlx_dynbuf for realloc when loading config files +- [Sergei Nikulov brought this change] + + CI/tests: enable test target on TravisCI for CMake builds - ... fixes an integer overflow at the same time. + Added test-nonflaky target to CMake builds - Reported-by: ihsinme on github - Assisted-by: Jay Satiro + Disabled test 1139 because the cmake build doesn't create docs/curl.1 - Closes #5946 + Closes #6074 -- dynbuf: provide curlx_ names for reuse by the curl tool +- tool_debug_cb: do not assume zero-terminated data - Closes #5946 + Follow-up to d70a5b5a0f5e3 -- dynbuf: make sure Curl_dyn_tail() zero terminates +- sendf: move the verbose-check into Curl_debug - Closes #5959 - -- tests: add test1912 to the dist + Saves us from having the same check done everywhere. - Follow-up to 70984ce1be4cab6c + Closes #6159 -- docs/LICENSE-MIXING: remove +- travis: use valgrind when running tests for debug builds - This document is not maintained and I feel that it doesn't provide much - value to users anymore (if it ever did). + Except the non-x86 and sanitizer builds - Closes #5955 - -- [Laramie Leavitt brought this change] + Closes #6154 - http: consolidate nghttp2_session_mem_recv() call paths - - Previously there were several locations that called - nghttp2_session_mem_recv and handled responses slightly differently. - Those have been converted to call the existing - h2_process_pending_input() function. +- header.d: fix syntax mistake - Moved the end-of-session check to h2_process_pending_input() since the - only place the end-of-session state can change is after nghttp2 - processes additional input frames. + follow-up from 1144886f38fd0 + +- [Harry Sintonen brought this change] + + gnutls: fix memory leaks (certfields memory wasn't released) - This will likely fix the fuzzing error. While I don't have a root cause - the out-of-bounds read seems like a use after free, so moving the - nghttp2_session_check_request_allowed() call to a location with a - guaranteed nghttp2 session seems reasonable. + Closes #6153 + +- tests: add missing global_init/cleanup calls - Also updated a few nghttp2 callsites to include error messages and added - a few additional error checks. + Without the cleanup call in these test files, the mbedTLS backend leaks + memory. - Closes #5648 + Closes #6156 -- HISTORY: mention alt-svc added in 2019 +- tool_operate: --retry for HTTP 408 responses too - ... and make 1996 the first year subtitle - -- base64: also build for pop3 and imap + This was inadvertently dropped from the code when the parallel support + was added. - Follow-up to the fix in 20417a13fb8f83 + Regression since b88940850 (7.66.0) - Reported-by: Michael Olbrich - Fixes #5937 - Closes #5948 + Reviewed-by: Jay Satiro + Closes #6155 -- base64: enable in build with SMTP +- http: pass correct header size to debug callback for chunked post - The oauth2 support is used with SMTP and it uses base64 functions. + ... when the chunked framing was added, the size of the "body part" of + the data was calculated wrongly so the debug callback would get told a + header chunk a few bytes too big that would also contain the first few + bytes of the request body. - Reported-by: Michael Olbrich - Fixes #5937 - Closes #5938 + Reported-by: Dirk Wetter + Ref: #6144 + Closes #6147 -- curl_mime_headers.3: fix the example's use of curl_slist_append +- header.d: mention the "Transfer-Encoding: chunked" handling - Reported-by: sofaboss on github - Fixes #5942 - Closes #5943 + Ref: #6144 + Closes #6148 -- lib583: fix enum mixup +- acinclude: detect manually set minimum macos/ipod version - grrr the previous follow-up to 17fcdf6a31 was wrong - -- libtest: fix build errors + ... even if set in the CC or IPHONEOS/MACOSX_DEPLOYMENT_TARGET + variables. - Follow-up from 17fcdf6a310d4c8076 + Reported-by: hamstergene on github + Fixes #6138 + Closes #6140 -- lib: fix -Wassign-enum warnings +Jay Satiro (29 Oct 2020) +- tests: fix some http/2 tests for older versions of nghttpx - configure --enable-debug now enables -Wassign-enum with clang, - identifying several enum "abuses" also fixed. + - Add regex that strips http/2 server header name to those http/2 tests + that don't already have it. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 + - Improve that regex in all http/2 tests. + + Tests 358 and 359 were failing for me before this change on a system + that uses an older version of nghttpx which includes its version number + in the server header. - Closes #5929 + Closes https://github.com/curl/curl/pull/6139 +Daniel Stenberg (30 Oct 2020) - RELEASE-NOTES: synced -- [Diven Qi brought this change] +- [Cristian Morales Vega brought this change] - url: use blank credentials when using proxy w/o username and password + configure: use pkgconfig to find openSSL when cross-compiling - Fixes proxy regression brought in commit ad829b21ae (7.71.0) + This reverts 736a40fec (November 2004), which doesn't explain why it was + done. - Fixed #5911 - Closes #5914 + Closes #6145 -- travis: add a build using libressl (from git master) +- tool_operate: bail out proper on errors for parallel setup - The v3.2.1 tag (latest release atm) results in a broken build. + ... otherwise for example trying to upload a missing file just causes a + loop. - Closes #5932 + Reported-by: BrumBrum on hackerone + Closes #6141 -- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 - - Unfortunately, this option is not detecting the same issues as clang's - -Wassign-enum flag, but should still be useful to detect future - mistakes. - - Closes #5930 +- [Sergei Nikulov brought this change] -- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - - If the error reason from the lib is - SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return - CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. + CMake: make BUILD_TESTING dependent option - This unifies the libcurl return code and makes libressl run test 313 - (CRL testing) fine. + CMake will now handle BUILD_TESTING depending on PERL_FOUND and + CURL_DISABLE_TESTING - Closes #5934 - -- FAQ: refreshed some very old language + Ref: #6036 + Closes #6072 -- cmake: make HTTP_ONLY also disable MQTT +- libssh2: fix transport over HTTPS proxy - ... and alphasort the order of disabling protocols to make it easier to - browse. + The fix in #6021 was not enough. This fix makes sure SCP/SFTP content + can also be transfered over a HTTPS proxy. - Closes #5931 + Fixes #6113 + Closes #6128 -- libtest: remove lib1541 leftovers +- curl.1: add an "OUTPUT" section at the top of the manpage - Caused automake errors. + Explain the basic concepts behind curl output. + + Inspired by #6124 - Follow-up to 8ca54a03ea08a + Closes #6134 -- tests/libtests: remove test 1900 and 2033 +- mailmap: set Viktor Szakats's email + +- runtests: show keywords when no tests ran - We already remove the test files, now remove the libtest codes as well. + To help out future debugging, runtests now outputs the list of keywords + when it fails because no tests ran. - Follow-up to e50a877df74 + Ref: #6120 + Closes #6126 -Marc Hoersken (7 Sep 2020) -- CI/azure: add test number to title for display in analytics +Jay Satiro (26 Oct 2020) +- CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo - To ease identification of tests the test number is added to - the test case title in order to have it on the Azure DevOps - Analytics pages and reports which currently do not show it. + Reported-by: Rui LIU - Bump test case revision to make Azure DevOps update titles. + Closes https://github.com/curl/curl/issues/6131 + +- range.d: fix typo - Closes #5927 + Follow-up to 15ae039 from earlier today. -Daniel Stenberg (6 Sep 2020) -- altsvc: clone setting in curl_easy_duphandle +Daniel Stenberg (26 Oct 2020) +- CI/github: work-around for brew breakage on macOS - The cache content is not duplicated, like other caches, but the setting - and specified file name are. + ... and make it use OpenSSL 1.1 properly - Test 1908 is extended to verify this somewhat. Since the duplicated - handle gets the same file name, the test unfortunately overwrites the - same file twice (with different contents) which makes it hard to check - automatically. + Fixes #6130 + Closes #6129 + +- [José Joaquín Atria brought this change] + + range.d: clarify that curl will not parse multipart responses - Closes #5923 + Closes #6127 + Fixes #6124 + +- RELEASE-NOTES: synced + +- [Baruch Siach brought this change] -- test1541: remove since it is a known bug + libssh2: fix build with disabled proxy support + + Build breaks because the http_proxy field is missing: - A shared connection cache is not thread-safe is a known issue. Stop - testing this until we believe this issue is addressed. Reduces - occasional test failures we don't care about. + vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy' - The test code in lib1541.c is left in git to allow us to restore it when - we get to fix this. + Regression from #6021, shipped in curl 7.73.0 - Closes #5922 + Closes #6125 -- tests: remove pipelining tests +- alt-svc: enable by default - Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were - previously disabled. + Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported. - The Pipelining code was removed from curl in commit 2f44e94efb3df8e, - April 2019. + alt-svc support in curl is no longer considered experimental - Closes #5921 + Closes #5868 -- curl: retry delays in parallel mode no longer sleeps blocking - - The previous sleep for retries would block all other concurrent - transfers. Starting now, the retry will instead be properly marked to - not get restarted until after the delay time but other transfers can - still continue in the mean time. - - Closes #5917 +- CI/appveyor: remove (unused) runtests.pl -b option -- curl:parallel_transfers: make sure retry readds the transfer - - Reported-by: htasta on github - Fixes #5905 - Closes #5917 +- [Emil Engler brought this change] -- build: drop support for building with Watcom + tool_help: make "output" description less confusing - These files are not maintained, they seem to have no users, Watcom - compilers look like not having users nor releases anymore. + Currently the description of "output" is misleading when comparing it + "verbose". - Closes #5918 + Closes #6118 -- winbuild/rundebug.cmd: remove +- CI/appveyor: disable test 571 in two cmake builds - Seems to have been added by mistake? Not included in dists. + ... they're simply too flaky there. - Closes #5919 + Closes #6119 -- curl: in retry output don't call all problems "transient" +- cmake: set the unicode feature in curl-config on Windows - ... because when --retry-all-errors is used, the error isn't necessarily - transient at all. + ... if built that way. To make it match curl -V output. - Closes #5916 + Reviewed-by: Marcel Raad + Closes #6117 -- easygetopt: pass a valid enum to avoid compiler warning - - "integer constant not in range of enumerated type 'CURLoption'" +- libssh2: require version 1.0 or later - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 + ... and simplify the code accordingly. libssh2 version 1.0 was released + in April 2009. - Closes #5915 + Closes #6116 -- [Emil Engler brought this change] +- KNOWN_BUGS: mention the individual cmake issues + + ... to make them easier to refer to and address separately and + one-by-one. - tests: Add tests for new --help +- CMake: store IDN2 information in curl_config.h - This commit is a part of "--help me if you can" + This allows the build to enable IDN properly and it makes test 1014 + happier. - Closes #5680 - -- [Emil Engler brought this change] + Ref: #6074 + Closes #6108 - tool: update --help with categories +- CMake: call the feature unixsockets without dash - This commit is a part of "--help me if you can" + ... so that curl-config gets correct and makes test 1014 happy! - Closes #5680 - -- [Emil Engler brought this change] + Ref: #6074 + Closes #6108 - docs: add categories to all cmdline opts +- CI/travis: add brotli and zstd to the libssh2 build - Adapted gen.pl with 'listcats' + ... to make sure such tests are run with valgrind. Suppress the zstd + valgrind warnings we get with version 1.3.3 on Ubuntu 18.04 (for debug + and non-debug builds). - This commit is a part of "--help me if you can" + Closes #6105 + +- runtests: revert the mistaken edit of $CURL - Closes #5680 + Regression from c4693adc62 - RELEASE-NOTES: synced -- [ihsinme brought this change] - - connect.c: remove superfluous 'else' in Curl_getconnectinfo +- curl_url_set.3: fix typo in the RETURN VALUE section - Closes #5912 + Reported-by: Basuke Suzuki + Fixes #6102 -- [Samuel Marks brought this change] +Jay Satiro (17 Oct 2020) +- [Daniel Stenberg brought this change] - CMake: remove explicit `CMAKE_ANSI_CFLAGS` - - This variable was removed from cmake in commit - https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later - CMake commit removes the variable from the tests, claiming that it was - removed in CMake 2.6 + packages/OS400: make the source code-style compliant - Reviewed-By: Peter Wu - Closes #5439 - -- [cbe brought this change] - - libssh2: pass on the error from ssh_force_knownhost_key_type + ... and make sure 'make checksrc' in the root dir also verifies the + packages/OS400 sources. - Closes #5909 + Closes https://github.com/curl/curl/pull/6085 -- scripts/delta: add diffstat summary +- os400: Sync libcurl API options - ... and make output more table-like - -- [Martin Bašti brought this change] - - http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set + This fixes the OS400 build and also an incorrect entry for + CURLINFO_APPCONNECT_TIME_T where it was treated as + CURLINFO_STARTTRANSFER_TIME_T. - ... in case NO_PROXY takes an effect + Reported-by: Jon Rumsey - Without this patch, the following command crashes: + Fixes https://github.com/curl/curl/issues/6083 + Closes https://github.com/curl/curl/pull/6084 + +Daniel Stenberg (16 Oct 2020) +- CURLOPT_NOBODY.3: fix typo - $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ - git clone https://github.com/curl/curl.git + Reported-by: Basuke Suzuki + Fixes #6097 + +Marc Hoersken (16 Oct 2020) +- CI/azure: improve on flakiness by avoiding libtool wrappers - Minimal libcurl-based reproducer: + Install curl binaries into MinGW bin folder and use that + for the tests in order to avoid libtool wrapper binaries. - #include + The libtool wrapper binaries (not scripts) on Windows seem + to be one of the possible causes for the following issues: - int main() { - CURL *curl = curl_easy_init(); - if(curl) { - CURLcode ret; - curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); - curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); - /* set the proxy type */ - curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); - curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); - curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); - ret = curl_easy_perform(curl); - curl_easy_cleanup(curl); - return ret; - } - return -1; - } + 1. Process output can be lost in the wrapper process chain. + 2. Killing the wrapper process does not kill the actual one. - Assisted-by: Kamil Dudka - Bug: https://bugzilla.redhat.com/1873327 - Closes #5902 + Derived from #5904 + Closes #6049 -- travis: add a CI job with openssl3 (from git master) - - Closes #5908 +Daniel Stenberg (16 Oct 2020) +- CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well -- openssl: avoid error conditions when importing native CA - - The code section that is OpenSSL 3+ specific now uses the same logic as - is used in the version < 3 section. It caused a compiler error without - it. - - Closes #5907 +- [Zenju brought this change] -- setopt: avoid curl_ on local variable + CURLOPT_TCP_NODELAY.3: fix comment in example code - Closes #5906 + Closes #6096 -- mqtt.c: avoid curl_ prefix on local variable +- openssl: acknowledge SRP disabling in configure properly - Closes #5906 - -- wildcard: strip "curl_" prefix from private symbols + Follow-up to 68a513247409 - Closes #5906 - -- vtls: make it 'struct Curl_ssl_session' + Use a new separate define that is the combination of both + HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP - Use uppercase C for internal symbols. + Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html - Closes #5906 + Closes #6094 -- curl_threads: make it 'struct Curl_actual_call' +Viktor Szakats (16 Oct 2020) +- http3: fix two build errors, silence warnings - Internal names should not be prefixed "curl_" + * fix two build errors due to mismatch between function + declarations and their definitions + * silence two mismatched signs warnings via casts - Closes #5906 + Approved-by: Daniel Stenberg + Closes #6093 -- schannel: make it 'struct Curl_schannel*' - - As internal global names should use captical C. +- Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 - Closes #5906 + Approved-by: Daniel Stenberg + Closes #6092 -- hash: make it 'struct Curl_hash' - - As internal global names should use captical C. +Daniel Stenberg (16 Oct 2020) +- tool_operate: fix compiler warning when --libcurl is disabled - Closes #5906 + Closes #6095 -- llist: make it "struct Curl_llist" +- checksrc: warn on empty line before open brace - As internal global names should use captical C. + ... and fix a few occurances - Closes #5906 + Closes #6088 -Marc Hoersken (2 Sep 2020) -- telnet.c: depend on static requirement of WinSock version 2 - - Drop dynamic loading of ws2_32.dll and instead rely on the - imported version which is now required to be at least 2.2. +- urlapi: URL encode a '+' in the query part - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Reviewed-by: Viktor Szakats + ... when asked to with CURLU_URLENCODE. - Closes #5854 + Extended test 1560 to verify. + Reported-by: Dietmar Hauser + Fixes #6086 + Closes #6087 -- win32: drop support for WinSock version 1, require version 2 - - IPv6, telnet and now also the multi API require WinSock - version 2 which is available starting with Windows 95. - - Therefore we think it is time to drop support for version 1. - - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Reviewed-by: Viktor Szakats - - Follow up to #5634 - Closes #5854 +- [Cristian Morales Vega brought this change] -- select: align poll emulation to return all relevant events + libcurl.pc: make it relocatable - The poll emulation via select already consumes POLLRDNORM, - POLLWRNORM and POLLRDBAND as input events. Therefore it - should also return them as output events if signaled. + It supposes when people specify the libdir/includedir they do it to + change where under prefix/exec_prefix it should be, not to make it + independent of prefix/exec_prefix. - Also fix indentation in input event handling block. + Closes #6061 + +- runtests: return error if no tests ran - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ... and make TESTFAIL stand out a little better by adding newlines + before and after. - Replaces #5852 - Closes #5883 + Reported-by: Marc Hörsken + Issue: #6052 + Closes #6053 -- CI/azure: MQTT is now enabled by default +- docs/FEATURE: convert to markdown - Reviewed-by: Daniel Stenberg + ... and clean it up a bit. - Follow up to #5858 - Closes #5903 + Closes #6067 + +- [Philipp Klaus Krause brought this change] -Daniel Stenberg (2 Sep 2020) -- copyright.pl: ignore buildconf + strerror: use 'const' as the string should never be modified + + Closes #6068 -- test971: show test mismatches "inline" +- [Jay Satiro brought this change] -- lib/Makefile.am: bump VERSIONINFO due to new functions + connect: repair build without ipv6 availability - ... we're generally bad at this, but we are adding new functions for - this release. + Assisted-by: Daniel Stenberg + Reported-by: Tom G. Christensen - Closes #5899 + Fixes https://github.com/curl/curl/issues/6069 + Closes https://github.com/curl/curl/pull/6071 -- optiontable: use DEBUGBUILD +- RELEASE-NOTES: synced - Follow-up to commit 6e18568ba38 (#5877) + Started over for the journey to next release. -- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 - - If there are more than two items in the list, use commas for all but the - last separator which is set to 'and'. Reads better. +- src/tool_filetime: disable -Wformat on mingw for this file - Closes #5898 - -- curl.1: add see also no-progress-meter on two spots + With gcc 10 on mingw we otherwise get this warning: - Ref: #5894 + error: ISO C does not support the 'I' printf flag [-Werror=format=] - Closes #5897 - -- RELEASE-NOTES: synced + Fixes #6079 + Closes #6082 -- mqtt: enable by default +- test122[12]: remove these two tests - No longer considered experimental. + ... and remove the objnames scripts they tested. They're not used for + anything anymore so testing them serves no purpose! - Closes #5858 + Reported-by: Marc Hörsken + Fixes #6080 + Closes #6081 -- [Michael Baentsch brought this change] +Version 7.73.0 (14 Oct 2020) - tls: add CURLOPT_SSL_EC_CURVES and --curves +Daniel Stenberg (14 Oct 2020) +- RELEASE-NOTES: synced - Closes #5892 + for 7.73.0 + +- THANKS: from 7.73.0 and .mailmap fixes -- url: remove funny embedded comments in Curl_disonnect calls +- mailmap: fixups of some contributors -- [Chris Paulson-Ellis brought this change] +- projects/build-wolfssl.bat: fix the copyright year range - conn: check for connection being dead before reuse - - Prevents incorrect reuse of an HTTP connection that has been prematurely - shutdown() by the server. - - Partial revert of 755083d00deb16 - - Fixes #5884 - Closes #5893 +Marc Hoersken (14 Oct 2020) +- [Sergei Nikulov brought this change] -Marc Hoersken (29 Aug 2020) -- buildconf: exec autoreconf to avoid additional process + CI/tests: fix invocation of tests for CMake builds - Also make buildconf exit with the return code of autoreconf. + Update appveyor.yml to set env variable TFLAGS and run tests + Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) + Move testdeps build to build step (per review comments) - Reviewed-by: Daniel Stenberg + Reviewed-by: Marc Hörsken - Follow up to #5853 - Closes #5890 + Closes #6066 + Fixes #6052 -- CI/azure: no longer ignore results of test 1013 +- tests/server/util.c: fix support for Windows Unicode builds - Follow up to #5771 - Closes #5889 + Detected via #6066 + Closes #6070 -- docs: add description about CI platforms to CONTRIBUTE.md - - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - - Closes #5882 +Daniel Stenberg (13 Oct 2020) +- [Jay Satiro brought this change] -Daniel Stenberg (29 Aug 2020) -- tests/getpart: use MIME::Base64 instead of home-cooked + strerror: Revert to local codepage for Windows error string - Since we already use the base64 package since a while back, we can just - as well switch to that here too. + - Change get_winapi_error() to return the error string in the local + codepage instead of UTF-8 encoding. - It also happens to use the exact same function name, which otherwise - causes a run-time warning. + Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it + also changed the error string's encoding from local codepage to UTF-8. - Reported-by: Marc Hörsken - Fixes #5885 - Closes #5887 - -Marcel Raad (29 Aug 2020) -- ntlm: fix condition for curl_ntlm_core usage + We return the local codepage version of the error string because if it + is output to the user's terminal it will likely be with functions which + expect the local codepage (eg fprintf, failf, infof). - `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES - backend is fine, but was excluded before. + This is essentially a partial revert of bed5f84. The support for xbox + remains but the error string is reverted back to local codepage. - This also fixes test 1013 as the condition for SMB support in - configure.ac didn't match the condition in the source code. Now it - does. + Ref: https://github.com/curl/curl/pull/6005 - Fixes https://github.com/curl/curl/issues/1262 - Closes https://github.com/curl/curl/pull/5771 + Reviewed-by: Marcel Raad + Closes #6065 -- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode - - The Schannel builds are the most useful to verify as they make the most - use of the Windows API. Classic MinGW doesn't support Unicode at all, - only MinGW-w64 and MSVC do. +Marc Hoersken (13 Oct 2020) +- CI/tests: use verification curl for test reporting APIs - Closes https://github.com/curl/curl/pull/5843 - -- CMake: add option to enable Unicode on Windows + Avoid using our own, potentially installed, curl for + the test reporting APIs in case it is broken. - As already existing for winbuild. + Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5843 + Preparation for #6049 + Closes #6063 -Marc Hoersken (29 Aug 2020) -- select: simplify return code handling for poll and select - - poll and select already return -1 on error according to POSIX, - so there is no need to perform a <0 to -1 conversion in code. - - Also we can just use one check with <= 0 on the return code. +Viktor Szakats (12 Oct 2020) +- windows: fix comparison of mismatched types warning - Assisted-by: Daniel Stenberg - Reviewed-by: Jay Satiro + clang 10, mingw-w64: + ``` + vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') + [-Wsign-compare] + if(GetLastError() != CRYPT_E_NOT_FOUND) + ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ + ``` - Replaces #5852 - Closes #5880 - -Daniel Stenberg (28 Aug 2020) -- RELEASE-NOTES: synced + Approved-by: Daniel Stenberg + Closes #6062 -- [Jeroen Ooms brought this change] +Daniel Stenberg (11 Oct 2020) +- [Viktor Szakats brought this change] - tests: add test1912 with typechecks + src/Makefile.m32: fix undefined curlx_dyn_* errors - Validates that gcc-typecheck macros match the new option type API. + by linking `lib/dynbuf.c` when building a static curl binary. + Previously this source file was only included when building + a dynamic curl binary. This was likely possibly because no + functions from the `src/Makefile.inc` / `CURLX_CFILES` sources + were actually required for a curl tool build. This has + recently changed with the introduction of `curlx_dyn_*()` + memory functions and their use by the tool sources. - Closes #5873 + Closes #6060 -- easyoptions: provide debug function when DEBUGBUILD - - ... not CURLDEBUG as they're not always set in conjunction. - - Follow-up to 6ebe63fac23f38df - - Fixes #5877 - Closes #5878 +- HISTORY: curl verifies SSL certs by default since version 7.10 -Marc Hoersken (28 Aug 2020) -- sockfilt: handle FD_CLOSE winsock event on write socket - - Learn from the way Cygwin handles and maps the WinSock events - to simulate correct and complete poll and select behaviour - according to Richard W. Stevens Network Programming book. +Marc Hoersken (8 Oct 2020) +- runtests.pl: use $LIBDIR variable instead of hardcoded path - Follow up to #5867 - Closes #5879 + Reviewed-by: Daniel Stenberg + Closes #6051 -- multi: handle connection state winsock events - - Learn from the way Cygwin handles and maps the WinSock events - to simulate correct and complete poll and select behaviour - according to Richard W. Stevens Network Programming book. +Daniel Stenberg (7 Oct 2020) +- checksrc: detect // comments on column 0 - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad + Spotted while working on #6045 - Follow up to #5634 - Closes #5867 + Closes #6048 -Daniel Stenberg (28 Aug 2020) -- Curl_pgrsTime - return new time to avoid timeout integer overflow - - Setting a timeout to INT_MAX could cause an immediate error to get - returned as timeout because of an overflow when different values of - 'now' were used. - - This is primarily fixed by having Curl_pgrsTime() return the "now" when - TIMER_STARTSINGLE is set so that the parent function will continue using - that time. - - Reported-by: Ionuț-Francisc Oancea - Fixes #5583 - Closes #5847 +- [Frederik Wedel-Heinen brought this change] -- TLS: fix SRP detection by using the proper #ifdefs - - USE_TLS_SRP will be true if *any* selected TLS backend can use SRP - - HAVE_OPENSSL_SRP is defined when OpenSSL can use it - - HAVE_GNUTLS_SRP is defined when GnuTLS can use it - - Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is - set if at least one of the supported backends offers SRP. + mbedtls: add missing header when defining MBEDTLS_DEBUG - Reported-by: Stefan Strogin - Fixes #5865 - Closes #5870 - -- [Dan Kenigsberg brought this change] + Closes #6045 - docs: SSLCERTS: fix English syntax +- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long - Signed-off-by: Dan Kenigsberg + Previously, it would pass on a define (int) which could make libcurl + read junk as a value - which prevented the CURLOPT_IPRESOLVE option to + "take". This could then make test 2100 do two DoH requests instead of + one! - Closes #5876 + Fixes #6042 + Closes #6043 -- [Alessandro Ghedini brought this change] +- RELEASE-NOTES: synced - docs: non-existing macros in man pages - - As reported by man(1) when invoked as: - - man --warnings -E UTF-8 -l -Tutf8 -Z >/dev/null +- scripts/release-notes.pl: don't "embed" $ in format string for printf() - Closes #5846 + ... since they might contain %-codes that mess up the output! -- [Alessandro Ghedini brought this change] +Jay Satiro (5 Oct 2020) +- [M.R.T brought this change] - curl.1: fix typo invokved -> invoked + build-wolfssl: fix build with Visual Studio 2019 - Closes #5846 + Closes https://github.com/curl/curl/pull/6033 -- buildconf: invoke 'autoreconf -fi' instead +Daniel Stenberg (4 Oct 2020) +- runtests: add %repeat[]% for test files - The custom script isn't necessary anymore - but remains for simplicity - and just invokes autoreconf. + ... and use this new keywords in all the test files larger than 50K to reduce + their sizes and make them a lot easier to read and understand. - Closes #5853 + Closes #6040 - [Emil Engler brought this change] - lib: make Curl_gethostname accept a const pointer + --help: move two options from the misc category - The address of that variable never gets changed, only the data in it so - why not make it a "char * const"? + The cmdline opts delegation and suppress-connect-headers + fit better into auth and proxy rather than misc. - Closes #5866 + Follow-up to aa8777f63febc + Closes #6038 -- docs/libcurl: update "Added in" version for curl_easy_option* - - Follow-up to 6ebe63fac23f38 +- [Samanta Navarro brought this change] -- scripts: improve the "get latest curl release tag" logic + docs/opts: fix typos in two manual pages - ... by insiting on it matching "^curl-". + Closes #6039 -- configure: added --disable-get-easy-options - - To allow disabling of the curl_easy_option APIs in a build. +- ldap: reduce the amount of #ifdefs needed - Closes #5365 + Closes #6035 -- options: API for meta-data about easy options - - const struct curl_easyoption *curl_easy_option_by_name(const char *name); - - const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); +- runtests: provide curl's version string as %VERSION for tests - const struct curl_easyoption * - curl_easy_option_next(const struct curl_easyoption *prev); + ... so that we can check HTTP requests for User-Agent: curl/%VERSION - The purpose is to provide detailed enough information to allow for - example libcurl bindings to get option information at run-time about - what easy options that exist and what arguments they expect. + Update 600+ test cases accordingly. - Assisted-by: Jeroen Ooms - Closes #5365 - -- [Eric Curtin brought this change] + Closes #6037 - HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 +- checksrc: warn on space after exclamation mark - Closes #5871 + Closes #6034 -- RELEASE-NOTES: synced +- test1465: verify --libcurl with binary POST data -Jay Satiro (26 Aug 2020) -- openssl: Fix wincrypt symbols conflict with BoringSSL - - OpenSSL undefines the conflicting symbols but BoringSSL does not so we - must do it ourselves. - - Reported-by: Samuel Tranchet - Assisted-by: Javier Blazquez - - Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 - Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 - - Fixes https://github.com/curl/curl/issues/5669 - Closes https://github.com/curl/curl/pull/5857 +- runtests: allow generating a binary sequence from hex -Daniel Stenberg (26 Aug 2020) -- socketpair: allow CURL_DISABLE_SOCKETPAIR - - ... to completely disable the use of socketpair - - Closes #5850 +- tool_setopt: escape binary data to hex, not octal -- curl_get_line: build only if cookies or alt-svc are enabled +- curl: make --libcurl show binary posts correctly - Closes #5851 - -- [fullincome brought this change] + Reported-by: Stephan Mühlstrasser + Fixes #6031 + Closes #6032 - schannel: fix memory leak when using get_cert_location +Jay Satiro (1 Oct 2020) +- strerror: fix null deref on winapi out-of-memory - The get_cert_location function allocates memory only on success. - Previously get_cert_location was able to allocate memory and return - error. It wasn't obvious and in this case the memory wasn't - released. + Follow-up to bed5f84 from several days ago. - Fixes #5855 - Closes #5860 + Ref: https://github.com/curl/curl/pull/6005 -- [Emil Engler brought this change] +Daniel Stenberg (1 Oct 2020) +- [Kamil Dudka brought this change] - git: ignore libtests in 3XXX area + vtls: deduplicate some DISABLE_PROXY ifdefs - Currently the file tests/libtest/lib3010 is not getting - ignored by git. This fixes it by adding the 3XXX area to - the according .gitignore file. + ... in the code of gtls, nss, and openssl - Closes #5859 + Closes #5735 + +- RELEASE-NOTES: synced - [Emil Engler brought this change] - doh: add error message for DOH_DNS_NAME_TOO_LONG - - When this error code was introduced in b6a53fff6c1d07e8a9, it was - forgotten to be added in the errors array and doh_strerror function. + TODO: Add OpenBSD libtool notice - Closes #5863 + See #5862 + Closes #6030 -- ngtcp2: adapt to the new pkt_info arguments +- tests/unit/README: convert to markdown - Guidance-by: Tatsuhiro Tsujikawa + ... and add to dist! - Closes #5864 + Closes #6028 -- winbuild/README.md: make visible +- tests/README: convert to markdown - Follow-up to be753add31c2d8c + Closes #6028 -- winbuild: convert the instruction text to README.md +- include/README: convert to markdown - Closes #5861 + Closes #6028 -- lib1560: verify "redirect" to double-slash leading URL +- examples/README: convert to markdown - Closes #5849 + Closes #6028 -Marc Hoersken (25 Aug 2020) -- multi: expand pre-check for socket readiness - - Check readiness of all sockets before waiting on them - to avoid locking in case the one-time event FD_WRITE - was already consumed by a previous wait operation. - - More information about WinSock network events: - https://docs.microsoft.com/en-us/windows/win32/api/ - winsock2/nf-winsock2-wsaeventselect#return-value +- configure: don't say HTTPS-proxy is enabled when disabled! - Closes #5634 - -- [rcombs brought this change] + Reported-by: Kamil Dudka + Reviewed-by: Kamil Dudka + Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 + Closes #6029 - multi: implement wait using winsock events - - This avoids using a pair of TCP ports to provide wakeup functionality - for every multi instance on Windows, where socketpair() is emulated - using a TCP socket on loopback which could in turn lead to socket - resource exhaustion. - - A previous version of this patch failed to account for how in WinSock, - FD_WRITE is set only once when writing becomes possible and not again - until after a send has failed due to the buffer filling. This contrasts - to how FD_READ and FD_OOB continue to be set until the conditions they - refer to no longer apply. This meant that if a user wrote some data to - a socket, but not enough data to completely fill its send buffer, then - waited on that socket to become writable, we'd erroneously stall until - their configured timeout rather than returning immediately. - - This version of the patch addresses that issue by checking each socket - we're waiting on to become writable with select() before the wait, and - zeroing the timeout if it's already writable. +Daniel Gustafsson (30 Sep 2020) +- src: Consistently spell whitespace without whitespace - Assisted-by: Marc Hörsken - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Tested-by: Gergely Nagy - Tested-by: Rasmus Melchior Jacobsen - Tested-by: Tomas Berger + Whitespace is spelled without a space between white and space, so + make sure to consistently spell it that way across the codebase. - Replaces #5397 - Reverts #5632 - Closes #5634 + Closes #6023 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- select: reduce duplication of Curl_poll in Curl_socket_check - - Change Curl_socket_check to use select-fallback in Curl_poll - instead of implementing it in Curl_socket_check and Curl_poll. +- MANUAL: update examples to resolve without redirects - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro + www.netscape.com is redirecting to a cookie consent form on Aol, and + cool.haxx.se isn't responding to FTP anymore. Replace with examples + that resolves in case users try out the commands when reading the + manual. - Replaces #5262 and #5492 - Closes #5707 + Closes #6024 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- select: fix poll-based check not detecting connect failure - - This commit changes Curl_socket_check to use POLLPRI to - check for connect failure on the write socket, because - POLLPRI maps to fds_err. This is in line with select(2). - - The select-based socket check correctly checks for connect - failures by adding the write socket also to fds_err. - - The poll-based implementation (which internally can itself - fallback to select again) did not previously check for - connect failure by using POLLPRI with the write socket. - - See the follow up commit to this for more information. - - This commit makes sure connect failures can be detected - and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. - - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - - Replaces #5509 - Prepares #5707 +Daniel Stenberg (30 Sep 2020) +- HISTORY: add some 2020 events -- select.h: make socket validation macros test for INVALID_SOCKET +- sectransp: make it build with --disable-proxy - With Winsock the valid range is [0..INVALID_SOCKET-1] according to - https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 + Follow-up from #5466 and f3d501dc678d80 + Reported-by: Javier Navarro + Fixes #6025 + Closes #6026 + +- ECH: renamed from ESNI in docs and configure - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg + Encrypted Client Hello (ECH) is the current name. - Closes #5760 + Closes #6022 -Daniel Stenberg (24 Aug 2020) -- docs: --output-dir is added in 7.73.0, nothing else +- configure: use "no" instead of "disabled" for the end summary - Follow-up to 5620d2cc78c0 + ... for consistency but also to make them more distinctly stand out next + to the "enabled" lines. -- curl: add --output-dir - - Works with --create-dirs and with -J - - Add test 3008, 3009, 3011, 3012 and 3013 to verify. +- TODO: SSH over HTTPS proxy with more backends - Closes #5637 + ... as right now only the libssh2 backend supports it. -- configure: fix pkg-config detecting wolfssl - - When amending the include path with "/wolfssl", this now properly strips - off all whitespace from the path variable! Previously this would lead to - pkg-config builds creating bad command lines. +- libssh2: handle the SSH protocols done over HTTPS proxy - Closes #5848 + Reported-by: Robin Douine + Fixes #4295 + Closes #6021 -- [Michael Musset brought this change] +- [Emil Engler brought this change] - sftp: add the option CURLKHSTAT_FINE_REPLACE + memdebug: remove 9 year old unused debug function - Replace the old fingerprint of the host with a new. + There used to be a way to have memdebug fill allocated memory. 9 years + later this has no value there (valgrind and ASAN etc are way better). If + people need to know about it they can have a look at VCS logs. - Closes #5685 + Closes #5973 -- RELEASE-NOTES: synced +- sendf: move Curl_sendf to dict.c and make it static + + ... as the only remaining user of that function. Also fix gopher.c to + instead use Curl_write() - The next release is now to become 7.73.0 + Closes #6020 -- checksrc: verify do-while and spaces between the braces +- ROADMAP: updates and cleanups - Updated mprintf.c to comply + Fix the HSTS PR - Closes #5845 + Remove DoT, thread-safe init and hard-coded localhost. I feel very + little interest for these with users so I downgrade them to plain "TODO" + entries again. -- curl: support XDG_CONFIG_HOME to find .curlrc +- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root - Added test433 to verify. Updated documentation. + This matches what is returned in other TLS backends in the same + situation. Reviewed-by: Jay Satiro - Suggested-by: Eli Schwartz - Fixes #5829 - Closes #5837 + Reviewed-by: Emil Engler + Follow-up to 5a3efb1 + Reported-by: iammrtau on github + Fixes #6003 + Closes #6018 -- etag: save and use the full received contents - - ... which makes it support weak tags and non-standard etags too! - - Added test case 347 to verify blank incoming ETag: - - Fixes #5610 - Closes #5833 +- RELEASE-NOTES: synced -- setopt: if the buffer exists, refuse the new BUFFERSIZE - - The buffer only exists during transfer and then we shouldn't change the - size (the setopt is not documented to work then). +- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL - Reported-by: Harry Sintonen - Closes #5842 - -- [COFFEETALES brought this change] - - sftp: add new quote commands 'atime' and 'mtime' + Added test 348 to verify. Added a 'STOR' command to the test FTP + server to enable test 348. Documented the command in FILEFORMAT.md - Closes #5810 + Reported-by: Duncan Wilcox + Fixes #6016 + Closes #6017 -- CURLE_PROXY: new error code +- pause: only trigger a reread if the unpause sticks - Failures clearly returned from a (SOCKS) proxy now causes this return - code. Previously the situation was not very clear as what would be - returned and when. + As an unpause might itself get paused again and then triggering another + reread doesn't help. - In addition: when this error code is returned, an application can use - CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then - returns a value from the new 'CURLproxycode' enum. + Follow-up from e040146f22608fd9 (shipped since 7.69.1) - Closes #5770 + Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html + Patch-by: Kunal Chandarana + Fixes #5988 + Closes #6013 -- runtests: make cleardir() erase dot files too +- test163[12]: require http to be built-in to run - Because test cases might use dot files. + ... as speaking over an HTTPS proxy implies http! - Closes #5838 + Closes #6014 -- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses - - Also: the current behavior is now documented in the curl.1 and - CURLOPT_NOPROXY.3 man pages. +- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define - Reported-by: Andrew Barnes - Closes #5745 - Closes #5841 + Closes #6012 -Viktor Szakats (22 Aug 2020) -- Makefile.m32: add ability to override zstd libs [ci skip] - - Similarly to brotli, where this was already possible. - E.g. it allows to link zstd statically to libcurl.dll. - - Ref: https://github.com/curl/curl-for-win/issues/12 - Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 - - Closes https://github.com/curl/curl/pull/5840 +- [Javier Blazquez brought this change] -Daniel Stenberg (21 Aug 2020) -- runtests: avoid 'fail to start' repeated messages in attempt loops + strerror: honor Unicode API choice on Windows - Closes #5834 + Closes #6005 -- runtests: clear pid variables when failing to start a server - - ... as otherwise the parent doesn't detect the failure and believe it - actually worked to start. +- imap: make imap_send use dynbuf for the send buffer management - Reported-by: Christian Weisgerber - Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html - Closes #5834 - -- TODO: Virtual external sockets + Reuses the buffer and thereby reduces number of mallocs over a transfer. - Closes #5835 - -- [Don J Olmstead brought this change] + Closes #6010 - dist: add missing CMake Find modules to the distribution +- Curl_send: return error when pre_receive_plain can't malloc - Closes #5836 - -- RELEASE-NOTES: synced + ... will probably trigger some false DEAD CODE positives on non-windows + code analyzers for the conditional code. - ... and version bumped to 7.72.1 + Closes #6011 -- tls: provide the CApath verbose log on its own line +- ftp: separate FTPS from FTP over "HTTPS proxy" - ... not newline separated from the previous line. This makes it output - asterisk prefixed properly like other verbose putput! + When using HTTPS proxy, SSL is used but not in the view of the FTP + protocol handler itself so separate the connection's use of SSL from the + FTP control connection's sue. - Reported-by: jmdavitt on github - Fixes #5826 - Closes #5827 - -Version 7.72.0 (19 Aug 2020) + Reported-by: Mingtao Yang + Fixes #5523 + Closes #6006 -Daniel Stenberg (19 Aug 2020) -- RELEASE-NOTES: synced +Dan Fandrich (23 Sep 2020) +- tests/data: Fix some mismatched XML tags in test cases - The curl 7.72.0 release - -- THANKS: add names from curl 7.72.0 release + This allows these test files to pass xmllint. -Jay Satiro (18 Aug 2020) -- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions - - Reported-by: plujon@users.noreply.github.com +Daniel Stenberg (23 Sep 2020) +- pingpong: use a dynbuf for the *_pp_sendf() function - Closes https://github.com/curl/curl/issues/5488 - -Daniel Stenberg (17 Aug 2020) -- Curl_easy: remember last connection by id, not by pointer + ... reuses the same dynamic buffer instead of doing repeated malloc/free + cycles. - CVE-2020-8231 + Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls + after this change in my test setup (132 => 125), curl 7.72.0 needed 140 + calls for this. - Bug: https://curl.haxx.se/docs/CVE-2020-8231.html + Test case 103 makes 9 less allocations now (130). Down from 149 in + 7.72.0. - Reported-by: Marc Aldorasi - Closes #5824 - -- examples/rtsp.c: correct the copyright year - -- RELEASE-PROCEDURE.md: add more future release dates - -- [H3RSKO brought this change] + Closes #6004 - docs: change "web site" to "website" - - According to wikipedia: - - While "web site" was the original spelling, this variant has become - rarely used, and "website" has become the standard spelling +- dynbuf: add Curl_dyn_vaddf - Closes #5822 - -- [Bevan Weiss brought this change] + Closes #6004 - CMake: don't complain about missing nroff +- dynbuf: make *addf() not require extra mallocs - The curl_nroff_check() was always being called, and complaining if - *NROFF wasn't found, even when not making the manual. + ... by introducing a printf() function that appends directly into a + dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if + the buffer is already big enough it can just printf directly into it. - Only check for nroff (and complain) if actually making the manual + Since this less-malloc version requires tthe use of a library internal + printf function, we only provide this version when building libcurl and + not for the dynbuf code that is used when building the curl tool. - Closes #5817 - -- [Brian Inglis brought this change] + Closes #5998 - libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin - - copy the LDFLAGS approach for adding same option with `libhostname` in - `libtest/Makefile.am`: - - - init `libstubgss_la_LDFLAGS_EXTRA` variable, - - add option to variable inside conditional, - - use variable in `libstubgss_la_LDFLAGS` +- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport - Fixes #5819 - Closes #5820 + Closes #5403 -- docs: clarify MAX_SEND/RECV_SPEED functionality +- pingpong: remove a malloc per Curl_pp_vsendf call - ... in particular what happens if the maximum speed limit is set to a - value that's smaller than the transfer buffer size in use. + This typically makes 7-9 fewer mallocs per FTP transfer. - Reported-by: Tomas Berger - Fixes #5788 - Closes #5813 + Closes #5997 -- test1140: compare stdout +- symbian: drop support - To make problems more immediately obvious when tests fail. + The OS is deprecated. I see no traces of anyone having actually built + curl for Symbian after 2012. - Closes #5814 - -- asyn-ares: correct some bad comments + The public headers are unmodified. - Closes #5812 + Closes #5989 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - docs: Add video link to docs/CONTRIBUTE.md +- curl_krb5.h: rename from krb5.h + + Follow-up from f4873ebd0be32cf - Closes #5811 + Turns out some older openssl installations go bananas otherwise. + Reported-by: Tom van der Woerdt + Fixes #5995 + Closes #5996 + +- test1297: verify GOT_NOTHING with http proxy tunnel -- curl-config: ignore REQUIRE_LIB_DEPS in --libs output +- http_proxy: do not count proxy headers in the header bytecount - Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is - not considered for the --libs output. + ... as that counter is subsequently used to detect if nothing was + returned from the peer. This made curl return CURLE_OK when it should + have returned CURLE_GOT_NOTHING. - Reported-by: ramsay-jones on github - Assisted-by: Brian Inglis and Ken Brown - Fixes #5793 - Closes #5808 - -- copyright: update/correct the year range on a few files - -- scripts/copyright.pl: ignore .muse files - -- [Emil Engler brought this change] + Fixes #5992 + Reported-by: Tom van der Woerdt + Closes #5994 - multi: Remove 10-year old out-commented code +- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument - The code hasn't been touched since 2010-08-18 + Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the + option is, yeah, not known. Clarified this in the setopt man page too. - Closes #5805 + Closes #5993 -- KNOWN_BUGS: A shared connection cache is not thread-safe +- krb5: merged security.c and krb specific FTP functions in here - Closes #4915 - Closes #5802 - -- CONTRIBUTE: extend git commit message description + These two files were always tightly connected and it was hard to + understand what went into which. This also allows us to make the + ftpsend() function static (moved from ftp.c). - In particular how the first line works. + Removed security.c + Renamed curl_sec.h to krb5.h - Closes #5803 - -- RELEASE-NOTES: synced - -- [Stefan Yohansson brought this change] + Closes #5987 - transfer: move retrycount from connect struct to easy handle +- Curl_handler: add 'family' to each protocol - This flag was applied to the connection struct that is released on - retry. These changes move the retry counter into Curl_easy struct that - lives across retries and retains the new connection. + Makes get_protocol_family() faster and it moves the knowledge about the + "families" to each protocol handler, where it belongs. - Reported-by: Cherish98 on github - Fixes #5794 - Closes #5800 + Closes #5986 -- libssh2: s/ssherr/sftperr/ - - The debug output used ssherr instead of sftperr which not only outputs - the wrong error code but also casues a warning on Windows. +- parsedate: tune the date to epoch conversion - Follow-up to 7370b4e39f1 + By avoiding an unnecessary error check and the temp use of the tm + struct, the time2epoch conversion function gets a little bit faster. + When repeating test 517, the updated version is perhaps 1% faster (on + one particular build on one particular architecture). - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 - Closes #5799 + Closes #5985 -- ftp: don't do ssl_shutdown instead of ssl_close +- cmake: remove scary warning - The shutdown function is for downgrading a connection from TLS to plain, - and this is not requested here. + Remove the text saying - Have ssl_close reset the TLS connection state. + "the curl cmake build system is poorly maintained. Be aware" - This partially reverts commit f002c850d98d + ... not because anything changed just now, but to encourage users to use + it and subsequently improve it. - Reported-by: Rasmus Melchior Jacobsen - Reported-by: Denis Goleshchikhin - Fixes #5797 + Closes #5984 -Marc Hoersken (9 Aug 2020) -- CI/azure: fix test outcome values and use latest API version - - This makes sure that tests ignored or skipped are not shown - just in the category "Other", but with their correct state. - - Closes #5796 +- docs/MQTT: remove outdated paaragraphs -- CI/azure: show runtime stats to investigate slowness - - Also avoid naming conflict of TFLAGS env and tflags variables. +- docs/MQTT: not experimental anymore - Closes #5776 + Follow-up to e37e4468688d8f -Daniel Stenberg (8 Aug 2020) -- TLS naming: fix more Winssl and Darwinssl leftovers +- docs/RESOURCES: remove - The CMake option is now called CMAKE_USE_SCHANNEL + This document is not maintained and rather than trying to refresh it, + let's kill it. A more up-to-date document with relevant RFCs is this + page on the curl website: https://curl.haxx.se/rfc/ - The winbuild flag is USE_SCHANNEL + Closes #5980 + +- docs/TheArtOfHttpScripting: convert to markdown - The CI jobs and build scripts only use the new names and the new name - options + Makes it easier to browse on github etc. Offers (better) links. - Tests now require 'Schannel' (when necessary) + It should be noted that this document is already mostly outdated and + "Everything curl" at https://ec.haxx.se/ is a better resource and + tutorial. - Closes #5795 + Closes #5981 -- smtp_parse_address: handle blank input string properly +- BUGS: convert document to markdown - Closes #5792 + Closes #5979 -- runtests: run the DICT server on a random port number +- --help: strdup the category - Removed support for -b (base port number) + ... since it is converted and the original pointer is freed on Windows + unicode handling. - Closes #5783 + Follow-up to aa8777f63febc + Fixes #5977 + Closes #5978 + Reported-by: xwxbug on github -- RELEASE-NOTES: synced +- CHECKSRC: document two missing warnings -- runtests: move the TELNET server to a dynamic port - - Rename the port variable to TELNETPORT to better match the existing - pattern. - - Closes #5785 +- RELEASE-NOTES: synced -- ngtcp2: adapt to error code rename +- ftp: avoid risk of reading uninitialized integers - Closes #5786 - -- runtests: move the smbserver to use a dynamic port number + If the received PASV response doesn't match the expected pattern, we + could end up reading uninitialized integers for IP address and port + number. - Closes #5782 + Issue pointed out by muse.dev + Closes #5972 -- runtests: run the http2 tests on a random port number - - Closes #5779 +- [Quentin Balland brought this change] -- gtls: survive not being able to get name/issuer + easy_reset: clear retry counter - Closes #5778 + Closes #5975 + Fixes #5974 -- runtests: move the gnutls-serv tests to a dynamic port +- ftp: get rid of the PPSENDF macro - Affects test 320, 321, 322 and 324. + The use of such a macro hides some of what's actually going on to the + reader and is generally disapproved of in the project. - Closes #5778 + Closes #5971 -- runtests: support dynamicly base64 encoded sections in tests - - This allows us to make test cases to use base64 at run-time and still - use and verify information determined at run-time, such as the IMAP test - server's port number in test 842. - - This change makes 12 tests run again that basically never ran since we - moved to dynamic port numbers. - - ftpserver.pl is adjusted to load test instructions and test number from - the preprocessed test file. +- man pages: switch to https://example.com URLs - FILEFORMAT.md now documents the new base64 encoding syntax. + Since HTTPS is "the new normal", this update changes a lot of man page + examples to use https://example.com instead of the previous "http://..." - Reported-by: Marcel Raad - Fixes #5761 - Closes #5775 + Closes #5969 -- curl.1: add a few missing valid exit codes +- github: remove the duplicate "Security vulnerability" entry - 93 - 96 can be returned as well. + ... since github adds an entry automatically by itself. - Closes #5777 + Closes #5970 -- TODO: Use multiple parallel transfers for a single download - - Closes #5774 +- [Emil Engler brought this change] -- TODO: Set the modification date on an uploaded file + github: use new issue template feature - Closes #5768 - -- [Thomas M. DuBuisson brought this change] - - CI: Add muse CI config + This helps us to avoid getting feature requests as well as security + bugs reported into the issue tracker. - Closes #5772 + Closes #5936 -- [Thomas M. DuBuisson brought this change] +- [Emil Engler brought this change] - travis/script.sh: fix use of `-n' with unquoted envvar - - Shellcheck tells us "-n doesn't work with unquoted arguments. quote or - use [[ ]]." - - And testing shows: - - ``` - docker run --rm -it ubuntu bash - root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" - I ran - root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" - root@fe85ce156856:/# - ``` + urlapi: use more Curl_safefree - Closes #5773 + Closes #5968 -- h2: repair trailer handling +Marc Hoersken (17 Sep 2020) +- multi: align WinSock mask variables in Curl_multi_wait - The previous h2 trailer fix in 54a2b63 was wrong and caused a - regression: it cannot deal with trailers immediately when read since - they may be read off the connection by the wrong 'data' owner. + Also skip pre-checking sockets to set timeout_ms to 0 + after the first socket has been detected to be ready. - This change reverts the logic back to gathering all trailers into a - single buffer, like before 54a2b63. + Reviewed-by: rcombs on github + Reviewed-by: Daniel Stenberg - Reported-by: Tadej Vengust - Fixes #5663 - Closes #5769 + Follow up to #5886 -Viktor Szakats (3 Aug 2020) -- windows: disable Unix Sockets for old mingw - - Classic mingw and 10y+ old versions of mingw-w64 don't ship with - Windows headers having the typedef necessary for Unix Sockets - support, so try detecting these environments to disable this - feature. +- multi: reuse WinSock events variable in Curl_multi_wait - Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ + Since the struct is quite large (1 long and 10 ints) we + declare it once at the beginning of the function instead + of multiple times inside loops to avoid stack movements. + Reviewed-by: Viktor Szakats Reviewed-by: Daniel Stenberg - Fixes #5674 - Closes #5758 + Closes #5886 -Marcel Raad (3 Aug 2020) -- test1908: treat file as text +Daniel Stenberg (16 Sep 2020) +- TODO: dynamically decide to use socketpair - Fixes the line endings on Windows. + Suggested-by: Anders Bakken - Closes https://github.com/curl/curl/pull/5767 + Closes #4829 -- TrackMemory tests: ignore realloc and free in getenv.c +- TODO: add PR reference for native IDN support on macOS - These are only called for WIN32. + As there was work started on this that never got completed. - Closes https://github.com/curl/curl/pull/5767 - -Daniel Stenberg (3 Aug 2020) -- tests/FILEFORMAT.md: mention %HTTP2PORT + Closes #5371 -- RELEASE-NOTES: synced +- tool_help.h: update copyright year range + + Follow-up from aa8777f63febca -- tlsv1.3.d. only for TLS-using connections +- CI/azure: disable test 571 in the msys2 builds - ... and rephrase that "not all" TLS backends support it. + It's just too flaky there - Closes #5764 + Reviewed-by: Marc Hoersken + Closes #5954 -- tls-max.d: this option is only for TLS-using connections +- tool_writeout: protect fputs() from NULL - Ref: #5763 - Closes #5764 - -Marcel Raad (2 Aug 2020) -- [Cameron Cawley brought this change] - - tool_doswin: Simplify Windows version detection + When the code was changed to do fputs() instead of fprintf() it got + sensitive for NULL pointers; add checks for that. - Closes https://github.com/curl/curl/pull/5754 - -- [Cameron Cawley brought this change] - - win32: Add Curl_verify_windows_version() to curlx + Follow-up from 0c1e767e83ec66 - Closes https://github.com/curl/curl/pull/5754 + Closes #5963 -- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL +- test3015: verify stdout "as text" - This makes the tests that require the OpenSSL feature also run for - those two compatible libraries. + Follow-up from 0c1e767e83e to please win32 tests - Closes https://github.com/curl/curl/pull/5762 + Closes #5962 -Daniel Stenberg (1 Aug 2020) -- multi: Condition 'extrawait' is always true +- travis: use libressl v3.1.4 instead of master - Reported by Codacy. + ... as their git master seems too fragile to use (and 3.2.1 which is the + latest has a build failure). - Reviewed-by: Marcel Raad - Closes #5759 + Closes #5964 -Marcel Raad (1 Aug 2020) -- openssl: fix build with LibreSSL < 2.9.1 - - `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were - introduced in LibreSSL 2.9.1 [0]. +- tests/FILEFORMAT: document type=shell for + +- tests/FILEFORMAT: document nonewline support for - [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef + The one in , that creates files. - Closes https://github.com/curl/curl/pull/5757 + Follow-up from b83947c8df7 -Daniel Stenberg (1 Aug 2020) -- [Marc Aldorasi brought this change] +- [anio brought this change] - multi_remove_handle: close unused connect-only connections + tool_writeout: add new writeout variable, %{num_headers} - Previously any connect-only connections in a multi handle would be kept - alive until the multi handle was closed. Since these connections cannot - be re-used, they can be marked for closure when the associated easy - handle is removed from the multi handle. + This variable gives the number of headers. - Closes #5749 + Closes #5947 -- checksrc: invoke script with -D to find .checksrc proper +- tool_urlglob: fix compiler warning "unreachable code" - Without the -D command line option, checksrc.pl won't know which - directory to load the ".checksrc" file from when building out of the - source tree. + (On Windows builds.) - Reported-by: Marcel Raad - Fixes #5715 - Closes #5755 + Follow-up to 70a3b003d9 -- [Carlo Marcelo Arenas Belón brought this change] +- [Gergely Nagy brought this change] - buildconf: retire ares buildconf invocation + vtls: deduplicate client certificates in ssl_config_data - no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 - -- [Carlo Marcelo Arenas Belón brought this change] + Closes #5629 - buildconf: excempt defunct reference to ACLOCAL_FLAGS +- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND - retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as - the name is generic enough that it might be in use and relied upon from - the environment. - -- [Carlo Marcelo Arenas Belón brought this change] - - buildconf: avoid array concatenation in die() + This is primarily interesting for cases where CURLOPT_NOBODY is set as + previously curl would not return an error for this case. - reported as error SC2145[1] by shellcheck, but not expected to cause - any behavioural differences otherwise. + MDTM getting 550 now also returns this error (it returned + CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for + missing files across protocols and specific FTP commands. - [1] https://github.com/koalaman/shellcheck/wiki/SC2145 + libcurl already returns error on a 550 as a MDTM response (when + CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would + happen subsequently anyway since the RETR command would fail. - Closes #5701 - -- travis: add ppc64le and s390x builds + Add test 1913 and 1914 to verify. Updated several tests accordingly due + to the updated SIZE behavior. - Closes #5752 + Reported-by: Tomas Berger + Fixes #5953 + Closes #5957 -Marc Hoersken (31 Jul 2020) -- connect: remove redundant message about connect failure - - Reviewed-by: Daniel Stenberg +- curl: make checkpasswd use dynbuf - Closes #5708 + Closes #5952 -- tests/sshserver.pl: fix compatibility with OpenSSH for Windows +- curl: make glob_match_url use dynbuf - Follow up to #5721 + Closes #5952 -- CI/azure: install libssh2 for use with msys2-based builds - - This enables building and running the SFTP tests. - Unfortunately OpenSSH for Windows does not support SCP (yet). - - Reviewed-by: Daniel Stenberg +- curl: make file2memory use dynbuf - Closes #5721 + Closes #5952 -- CI/azure: increase Windows job timeout once again - - Avoid aborted jobs due to performance issues on Azure DevOps. - - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro +- curl: make file2string use dynbuf - Closes #5738 + Closes #5952 -Jay Satiro (30 Jul 2020) -- TODO: Schannel: 'Add option to allow abrupt server closure' - - We should offer an option to allow abrupt server closures (server closes - SSL transfer without sending a known termination point such as length of - transfer or close_notify alert). Abrupt server closures are usually - because of misconfigured or very old servers. - - Closes https://github.com/curl/curl/issues/4427 +- [Antarpreet Singh brought this change] -- url: fix CURLU and location following + imap: set cselect_bits to CURL_CSELECT_IN initially - Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was - incorrectly used for the location follow, resulting in infinite requests - to the original location. + ... when continuing a transfer from a FETCH response. - Reported-by: sspiri@users.noreply.github.com + When the size of the file was small enough that the entirety of the + transfer happens in a single go and schannel buffers holds the entire + data. However, it wasn't completely read in Curl_pp_readresp since a + line break was found before that could happen. So, by the time we are in + imap_state_fetch_resp - there's data in buffers that needs to be read + via Curl_read but nothing to read from the socket. After we setup a + transfer (Curl_setup_transfer), curl just waits on the socket state to + change - which doesn't happen since no new data ever comes. - Fixes https://github.com/curl/curl/issues/5709 - Closes https://github.com/curl/curl/pull/5713 + Closes #5961 -Daniel Stenberg (30 Jul 2020) - RELEASE-NOTES: synced -- [divinity76 brought this change] - - docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions - - it helps make it obvious that most developers don't have to care about - the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 - years old, November 4 2009) +- test434: test -K use in a single line without newline - Closes #5744 + Closes #5946 -Jay Satiro (29 Jul 2020) -- tool_cb_wrt: fix outfile mode flags for Windows - - - Use S_IREAD and S_IWRITE mode permission flags to create the file - on Windows instead of S_IRUSR, S_IWUSR, etc. +- runtests: allow creating files without newlines - Windows only accepts a combination of S_IREAD and S_IWRITE. It does not - acknowledge other combinations, for which it may generate an assertion. + Closes #5946 + +- curl: use curlx_dynbuf for realloc when loading config files - This is a follow-up to 81b4e99 from yesterday, which improved the - existing file check with -J. + ... fixes an integer overflow at the same time. - Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks - Ref: https://github.com/curl/curl/pull/5731 + Reported-by: ihsinme on github + Assisted-by: Jay Satiro - Closes https://github.com/curl/curl/pull/5742 + Closes #5946 -Daniel Stenberg (28 Jul 2020) -- checksrc: ban gmtime/localtime - - They're not thread-safe so they should not be used in libcurl code. +- dynbuf: provide curlx_ names for reuse by the curl tool - Explictly enabled when deemed necessary and in examples and tests + Closes #5946 + +- dynbuf: make sure Curl_dyn_tail() zero terminates - Reviewed-by: Nicolas Sterchele - Closes #5732 + Closes #5959 diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index 6f164a9adf..2844ef1564 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -7,6 +7,7 @@ 0xflotus on github 1ocalhost on github 3dyd on github +3eka on github Aaro Koskinen Aaron Oneal Aaron Orenstein @@ -141,6 +142,7 @@ Andrei Sedoi Andrei Valeriu BICA Andrei Virtosu Andrej E Baranov +Andrew Barnert Andrew Barnes Andrew Benham Andrew Biggs @@ -212,6 +214,7 @@ Avery Fay awesomenode on github Axel Tillequin Ayoub Boudhar +Ayushman Singh Chauhan b9a1 on github Balaji Parasuram Balaji S Rao @@ -243,6 +246,7 @@ Benjamin Gerard Benjamin Gilbert Benjamin Johnson Benjamin Kircher +Benjamin Riefenstahl Benjamin Ritcey Benjamin Sergeant Benoit Neil @@ -272,6 +276,7 @@ Bjorn Augustsson Bjorn Reese Björn Stenberg Blaise Potard +Blake Burkhart bnfp on github Bob Relyea Bob Richmond @@ -460,6 +465,7 @@ Dan Zitter Daniel at touchtunes Daniel Bankhead Daniel Black +Daniel Carpenter Daniel Cater Daniel Egger Daniel Gustafsson @@ -503,6 +509,7 @@ David Binderman David Blaikie David Byron David Cohen +David Cook David Demelier David E. Narváez David Earl @@ -583,6 +590,7 @@ dmitrmax on github Dmitry Bartsevich Dmitry Eremin-Solenikov Dmitry Falko +Dmitry Karpov Dmitry Kostjuchenko Dmitry Kurochkin Dmitry Mikhirev @@ -769,6 +777,7 @@ Gaz Iqbal Gaël Portay Geeknik Labs Geoff Beier +Georeth Zhou Georg Horn Georg Huettenegger Georg Lippitsch @@ -899,6 +908,7 @@ Iida Yosiaki Ikko Ashimine Ilguiz Latypov Ilja van Sprundel +Illarion Taev Ilya Kosarev imilli on github Immanuel Gregoire @@ -919,6 +929,7 @@ Ivan Avdeev IvanoG on github Ivo Bellin Salarin iz8mbw on github +J. Bromley Jack Boos Yu Jack Zhang Jackarain on github @@ -1047,6 +1058,7 @@ Joe Malicki Joe Mason Joel Chen Joel Depooter +Joel Jakobsson Joel Teichroeb joey-l-us on github Jofell Gallardo @@ -1054,6 +1066,7 @@ Johan Anderson Johan Lantz Johan Nilsson Johan van Selst +Johann150 on github Johannes Bauer Johannes Ernst Johannes G. Kristinsson @@ -1202,6 +1215,7 @@ Kenny To Kent Boortz Keshav Krity Kevin Baughman +Kevin Burke Kevin Fisk Kevin Ji Kevin Lussier @@ -1222,6 +1236,7 @@ Klaus Stein Klevtsov Vadim Kobi Gurkan Koen Dergent +Koichi Shiraishi kokke on github Konstantin Isakov Konstantin Kushnir @@ -1311,7 +1326,9 @@ Luan Cestari Luca Altea Luca Boccassi Lucas Adamski +Lucas Clemente Vella Lucas Pardue +Lucas Servén Marín Lucas Severo Lucien Zürcher Ludek Finstrle @@ -1401,12 +1418,14 @@ Martijn Koster Martin Ankerl Martin Bašti Martin C. Martin +Martin Dorey Martin Drasar Martin Dreher Martin Frodl Martin Galvan Martin Gartner Martin Hager +Martin Halle Martin Hedenfalk Martin Jansen Martin Kammerhofer @@ -1428,6 +1447,7 @@ Mateusz Loskot Mathias Axelsson Mathias Gumz Mathieu Legare +Matias N. Goldberg Mats Lidell Matt Arsenault Matt Ford @@ -1490,6 +1510,7 @@ Michael Jerris Michael Kalinin Michael Kaufmann Michael Kilburn +Michael Kolechkin Michael Kujawa Michael König Michael Lee @@ -1497,6 +1518,7 @@ Michael Maltese Michael Mealling Michael Mueller Michael Musset +Michael O'Farrell Michael Olbrich Michael Osipov Michael Schmid @@ -1508,6 +1530,7 @@ Michael Vittiglio Michael Wallner Michal Bonino Michal Marek +Michal Rus Michal Trybus Michal Čaplygin Michał Antoniak @@ -1557,6 +1580,7 @@ Mohammad Hasbini Mohun Biswas momala454 on github moohoorama on github +Morten Minde Neergaard Mostyn Bramley-Moore Moti Avrahami MrdUkk on github @@ -1717,11 +1741,13 @@ Pavel Volgarev Pavol Markovic Pawel A. Gajda Pawel Kierski +Paweł Wegner Pedro Larroy Pedro Monreal Pedro Neves pendrek at hackerone Peng Li +Peng-Yu Chen Per Jensen Per Lundberg Per Malmberg @@ -1781,6 +1807,7 @@ Pierre-Yves Bigourdan Piotr Dobrogost Piotr Komborski Po-Chuan Hsieh +Pontus Lundkvist Pooyan McSporran Poul T Lomholt Pramod Sharma @@ -1797,6 +1824,7 @@ Quanah Gibson-Mount Quentin Balland Quinn Slack R. Dennis Steed +Radek Zajic Radoslav Georgiev Radu Simionescu Rafa Muyo @@ -1812,6 +1840,7 @@ Rajesh Naganathan Rajkumar Mandal Ralf S. Engelschall Ralph Beckmann +Ralph Langendam Ralph Mitchell Ram Krushna Mishra ramsay-jones on github @@ -1846,6 +1875,7 @@ Reza Arbab Ricardo Cadime Ricardo Gomes Rich Burridge +Rich FitzJohn Rich Gray Rich Mirch Rich Rauenzahn @@ -1942,6 +1972,7 @@ Ruslan Gazizov Rutger Hofman Ruurd Beerstra RuurdBeerstra on github +Ryan Beck-Buysse Ryan Braud Ryan Chan Ryan Nelson @@ -1998,6 +2029,7 @@ Sebastian Rasmussen Senthil Raja Velu Sergei Kuzmin Sergei Nikulov +Sergey Markelov Sergey Ogryzkov Sergey Tatarincev Sergii Kavunenko @@ -2005,6 +2037,7 @@ Sergii Pylypenko Sergio Ballestrero Sergio Barresi Sergio Borghese +sergio-nsk on github Serj Kalichev Seshubabu Pasam Seth Mos @@ -2060,6 +2093,7 @@ Stefan Eissing Stefan Esser Stefan Grether Stefan Kanthak +Stefan Karpinski Stefan Krause Stefan Neis Stefan Strogin @@ -2171,14 +2205,17 @@ Tim Stack Tim Starling Tim Tassonis Tim Verhoeven +Timo Lange Timo Sirainen Timotej Lazar Timothe Litt +Timothy Gu Timothy Polich Tinus van den Berg TJ Saunders tmkk on github Tobias Blomberg +Tobias Gabriel Tobias Hieta Tobias Hintze Tobias Lindgren @@ -2218,6 +2255,7 @@ Tomasz Kojm Tomasz Lacki Tommie Gannert tommink[at]post.pl +Tommy Odom Tommy Petty Tommy Tam Ton Voon @@ -2238,6 +2276,7 @@ Trivikram Kamat Troels Walsted Hansen Troy Engel Tseng Jun +Tuomas Siipola Tuomo Rinne Tupone Alfredo Tyler Hall @@ -2248,6 +2287,7 @@ Ulrich Doehner Ulrich Telle Ulrich Zadow UrsusArctos on github +ustcqidi on github Valentin David Valentyn Korniienko Valerii Zapodovnikov @@ -2261,6 +2301,7 @@ Venkataramana Mokkapati Vicente Garcia Victor Magierski Victor Snezhko +Victor Vieux Vijay Panghal Vikram Saxena Viktor Szakats @@ -2298,6 +2339,7 @@ Wenchao Li Wenxiang Qian Werner Koch Werner Stolz +Wes Hinsley wesinator on github Wesley Laxton Wesley Miaw @@ -2336,6 +2378,7 @@ ygthien on github Yi Huang Yiming Jing Yingwei Liu +Ymir1711 on github Yonggang Luo youngchopin on github Yousuke Kimoto @@ -2344,6 +2387,7 @@ Yukihiro Kawada Yun SangHo Yuri Slobodyanyuk Yuriy Sosov +Yusuke Nakamura Yves Arrouye Yves Lejeune Zachary Seguin -- cgit v1.2.3