From f223275b6ca65c29f8f098818241150338e00123 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Sat, 3 Apr 2021 16:14:19 +0300 Subject: libcurl: update to 7.76.0 --- libs/libcurl/docs/CHANGES | 10165 ++++++++++++++++++++++---------------------- libs/libcurl/docs/THANKS | 36 +- 2 files changed, 5225 insertions(+), 4976 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index f3439fd046..762a865fe1 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,7442 +6,7657 @@ Changelog -Version 7.75.0 (3 Feb 2021) +Version 7.76.0 (31 Mar 2021) -Daniel Stenberg (3 Feb 2021) +Daniel Stenberg (31 Mar 2021) - RELEASE-NOTES: synced + + curl 7.76.0 release -- THANKS: added contributors from 7.75.0 - -- copyright: fix year ranges in need of updates +- THANKS: added names from 7.76.0 -- TODO: remove items for next SONAME bump etc +- CURLOPT_AUTOREFERER.3: clarify that it sets the full URL - We want to avoid that completely, so we don't plan for things after such - an event. - -- [Jay Satiro brought this change] + ... some users may not want that! - ngtcp2: Fix build error due to change in ngtcp2_settings +- define: remove CURL_DISABLE_NTLM ifdefs - - Separate ngtcp2_transport_params. + It was never defined anywhere. Fixed disable-scan (test 1165) to also + scan headers, which found this issue. - ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from - ngtcp2_settings. + Closes #6809 + +- vtls: fix addsessionid for non-proxy builds - ngtcp2 master is required to build curl with http3 support. + Follow-up to b09c8ee15771c61 + Fixes #6812 + Closes #6811 + +- [Li Xinwei brought this change] + + cmake: support WinIDN - Closes #6554 + Closes #6807 -- vtls: remove md5sum +- transfer: clear 'referer' in declaration - As it is not used anymore. + To silence (false positive) compiler warnings about it. - Reported-by: Jacob Hoffman-Andrews - Bug: https://curl.se/mail/lib-2021-02/0000.html + Follow-up to 7214288898f5625 - Closes #6557 + Reviewed-by: Marcel Raad + Closes #6810 -- [Alessandro Ghedini brought this change] +- [Marc Hoersken brought this change] - quiche: don't use primary_ip / primary_port + config: fix SSPI enabling NTLM if crypto auth is disabled - Closes #6555 - -Alessandro Ghedini (1 Feb 2021) -- travis: enable quiche's FFI feature + Avoid enabling NTLM feature based upon Windows SSPI + being enabled in case that crypto auth is disabled. + + Reported-by: Marcel Raad + + Follow-up to #6277 + Fixes #6803 + Closes #6808 -Daniel Stenberg (30 Jan 2021) -- [Dmitry Wagin brought this change] +- HISTORY: add two 2021 events - http: improve AWS HTTP v4 Signature auth +- vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid() - - Add support services without region and service prefixes in - the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc) - by providing region and service parameters via aws-sigv4 option. - - Add [:region[:service]] suffix to aws-sigv4 option; - - Fix memory allocation errors. - - Refactor memory management. - - Use Curl_http_method instead() STRING_CUSTOMREQUEST. - - Refactor canonical headers generating. - - Remove repeated sha256_to_hex() usage. - - Add some docs fixes. - - Add some codestyle fixes. - - Add overloaded strndup() for debug - curl_dbg_strndup(). - - Update tests. + To make sure we set and extract the correct session. - Closes #6524 - -- hyper: fix CONNECT to set 'data' as userdata + Reported-by: Mingtao Yang + Bug: https://curl.se/docs/CVE-2021-22890.html - Follow-up to 14e075d1a7fd + CVE-2021-22890 -- [Layla brought this change] +- [Viktor Szakats brought this change] - connect: fix compile errors in `Curl_conninfo_local` + transfer: strip credentials from the auto-referer header field - .. for the `#else` (`!HAVE_GETSOCKNAME`) case + Added test 2081 to verify. - Fixes https://github.com/curl/curl/issues/6548 - Closes #6549 + CVE-2021-22876 - Signed-off-by: Layla - -- [Michał Antoniak brought this change] + Bug: https://curl.se/docs/CVE-2021-22876.html - transfer: fix GCC 10 warning with flag '-Wint-in-bool-context' +- curl_sasl: fix compiler error with --disable-crypto-auth - ... and return the error code from the Curl_mime_rewind call. + ... if libgsasl was found. - Closes #6537 - -- [Michał Antoniak brought this change] - - avoid warning: enum constant in boolean context + Closes #6806 -- copyright: fix missing year (range) updates - -- RELEASE-NOTES: synced +- [Patrick Monnerat brought this change] -- openssl: lowercase the hostname before using it for SNI + ldap: only set the callback ptr for TLS context when TLS is used - ... because it turns out several servers out there don't actually behave - correctly otherwise in spite of the fact that the SNI field is - specifically said to be case insensitive in RFC 6066 section 3. + Follow-up to a5eee22e594c2460f + Fixes #6804 + Closes #6805 + +- copyright: update copyright year ranges to 2021 - Reported-by: David Earl - Fixes #6540 - Closes #6543 + Reviewed-by: Emil Engler + Closes #6802 -- KNOWN_BUGS: cmake: ExternalProject_Add does not set CURL_CA_PATH +- send_speed: simplify the checks for if a speed limit is set - Closes #6313 + ... as we know the value cannot be set to negative: enforced by + setopt() -- KNOWN_BUGS: Multi perform hangs waiting for threaded resolver +- http: cap body data amount during send speed limiting - Closes #4852 + By making sure never to send off more than the allowed number of bytes + per second the speed limit logic is given more room to actually work. + + Reported-by: Fabian Keil + Bug: https://curl.se/mail/lib-2021-03/0042.html + Closes #6797 -- KNOWN_BUGS: "pulseUI VPN client" is known to be buggy +- urldata: merge "struct DynamicStatic" into "struct UrlState" - First entry in the new section "applications" for known problems in - libcurl using applications. + Both were used for the same purposes and there was no logical separation + between them. Combined, this also saves 16 bytes in less holes in my + test build. - Closes #6306 + Closes #6798 -- tool_writeout: make %{errormsg} blank for no errors +- tests/README.md: mentioned that en_US.UTF-8 is required - Closes #6539 + Reported-by: Oumph on github + Fixes #6768 -Jay Satiro (27 Jan 2021) -- [Gisle Vanem brought this change] +- HISTORY: fixed the Mac OS X 10.1 release date + + Based on what Wikipedia says - build: fix djgpp builds +Jay Satiro (26 Mar 2021) +- examples: Remove threaded-shared-conn.c due to bug - - Update build instructions in packages/DOS/README + Known bug 11.11 is the shared object's connection cache is not thread + safe, so we should not have an example for it. - - Extend 'VPATH' with 'vquic' and 'vssh'. + Ref: https://github.com/curl/curl/issues/4915 + Ref: https://curl.se/docs/knownbugs.html#A_shared_connection_cache_is_not - - Allow 'Makefile.dist' to build both 'lib' and 'src'. + Closes https://github.com/curl/curl/pull/6795 + +- KNOWN_BUGS: Update 11.9 - DoH option inheritance - - Allow using the Windows hosted djgpp cross compiler to build for MSDOS - under Windows. + - Add description: Explain that some options aren't inherited because + they are not relevant for the DoH SSL connections or may result in + unexpected behavior. - - 'USE_SSL' -> 'USE_OPENSSL' + - Remove the reference to #4578 (SSL verify options not inherited) since + that was fixed by #6597 (separate DoH-specific options for verify). - - Added a 'link_EXE' macro. Etc, etc. + - Explain that DoH-specific options (those created by #6597) are + available: CURLOPT_DOH_SSL_VERIFYHOST, CURLOPT_DOH_SSL_VERIFYPEER and + CURLOPT_DOH_SSL_VERIFYSTATUS. - - Linking 'curl.exe' needs '$(CURLX_CFILES)' too. + - Add a reference to #6605 and explain that the user's debug function is + not inherited because it would be unexpected to pass internal handles + (ie DoH handles) to the user's callback. - - Do not pick-up '../lib/djgpp/*.o' files. Recompile locally. + Closes https://github.com/curl/curl/issues/6605 + +Daniel Stenberg (26 Mar 2021) +- curl_easy_setopt.3: add curl_easy_option* functions to SEE ALSO + +- [Jean-Philippe Menil brought this change] + + openssl: ensure to check SSL_CTX_set_alpn_protos return values - - Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'. + SSL_CTX_set_alpn_protos() return 0 on success, and non-0 on failure - - Remove 'djgpp-clean' + Signed-off-by: Jean-Philippe Menil - - Adapt to new C-ares directory structure + Closes #6794 + +- multi: close the connection when h2=>h1 downgrading - - Use conditional variable assignments + Otherwise libcurl is likely to reuse the connection again in the next + attempt since the connection reuse logic doesn't take downgrades into + account. - Clarify the 'conditional variable assignment' in 'common.dj'. + Reported-by: Anthony Ramine + Fixes #6788 + Closes #6793 + +- openssl: set the transfer pointer for logging early - Closes https://github.com/curl/curl/pull/6382 + Otherwise, the transfer will be NULL in the trace function when the + early handshake details arrive and then curl won't show them. + + Regresssion in 7.75.0 + + Reported-by: David Hu + Fixes #6783 + Closes #6792 -Daniel Stenberg (27 Jan 2021) -- [Ikko Ashimine brought this change] +- RELEASE-NOTES: synced - hyper: fix typo in c-hyper.c +- TODO: Custom progress meter update interval - settting -> setting - - Closes #6538 + Ref: https://stackoverflow.com/q/66789977/93747 -- libssh2: fix CURL_LIBSSH2_DEBUG-enabled build +- docs/ABI: tighten up the language - Follow-up to 2dcc940959772a + Make the promises more firm - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/2dcc940959772a652f6813fb6bd3092095a4877b#commitcomment-46420088 + Closes #6786 -Jay Satiro (27 Jan 2021) -- asyn-thread: fix build for when getaddrinfo missing +- openldap: disconnect better - This is a follow-up to 8315343 which several days ago moved the resolver - pointer into the async struct but did not update the code that uses it - when getaddrinfo is not present. + Instead of clearing the callback argument in disconnect, set it to the + (new) transfer to make sure the correct data is passed to the callbacks. - Closes https://github.com/curl/curl/pull/6536 + Follow-up to e467ea3bd937f38 + Assisted-by: Patrick Monnerat + Closes #6787 -Daniel Stenberg (27 Jan 2021) -- urldata: move 'ints' to the end of 'connectdata' +- libssh2: kdb_callback: get the right struct pointer - To optimize storage slightly. + After the recent conn/data refactor in this source file, this function + was mistakenly still getting the old struct pointer which would lead to + crash on servers with keyboard-interactive auth enabled. - Closes #6534 - -- urldata: store ip version in a single byte + Follow-up to a304051620b92e12b (shipped in 7.75.0) - Closes #6534 + Reported-by: Christian Schmitz + Fixes #6691 + Closes #6782 -- urldata: remove duplicate 'upkeep_interval_ms' from connectdata +- tftp: remove unused struct fields - ... and rely only on the value already set in Curl_easy. + Follow-up to d3d90ad9c00530d - Closes #6534 + Closes #6781 -- urldata: remove 'local_ip' from the connectdata struct +- openldap: avoid NULL pointer dereferences - As the info is already stored in the transfer handle anyway, there's no - need to carry around a duplicate buffer for the life-time of the handle. - - Closes #6534 + Follow-up to a59c33ceffb8f78 + Reported-by: Patrick Monnerat + Fixes #6676 + Closes #6780 -- urldata: remove duplicate port number storage +- http: strip default port from URL sent to proxy - ... and use 'int' for ports. We don't use 'unsigned short' since -1 is - still often used internally to signify "unknown value" and 0 - 65535 are - all valid port numbers. + To make sure the Host: header and the URL provide the same authority + portion when sent to the proxy, strip the default port number from the + URL if one was provided. - Closes #6534 + Reported-by: Michael Brown + Fixes #6769 + Closes #6778 -- urldata: remove the duplicate 'ip_addr_str' field +- azure: disable test 433 on azure-ubuntu - ... as the numerical IP address is already stored and kept in 'primary_ip'. + Something in that environment sets XDG_CONFIG_HOME for us in a way that + breaks the test. - Closes #6534 + Reported-by: Marc Hörsken + Fixes #6739 + Closes #6777 -- select: convert Curl_select() to private static function - - The old function should not be used anywhere anymore (the only remaining - gskit use has to be fixed to instead use Curl_poll or none at all). +- tftp: remove the 3600 second default timeout - The static function version is now called our_select() and is only built - if necessary. + ... it was never meant to be there. - Closes #6531 + Reported-by: Tomas Berger + Fixes #6774 + Closes #6776 -- Curl_chunker: shrink the struct +- docs: make gen.pl support *italic* and **bold** - ... by removing a field, converting the hex index into a byte and - rearranging the order. Cuts it down from 48 bytes to 32 on x86_64. + Remove some nroffisms from the cmdline doc files to simplify editing, + and instead support this markdown style. - Closes #6527 + Closes #6771 -- curl: include the file name in --xattr/--remote-time error msgs +- ngtcp2: sync with recent API updates + + Closes #6770 -- curl: s/config->global/global/ in single_transfer() +- RELEASE-NOTES: synced -- curl: move fprintf outputs to warnf +- libssh2:ssh_connect: clear session pointer after free - For setting and getting time of the download. To make the outputs - respect --silent etc. + If libssh2_knownhost_init() returns NULL, like in an OOM situation, the + ssh session was freed but the pointer wasn't cleared which made libcurl + later call libssh2 to cleanup using the stale pointer. - Reported-by: Viktor Szakats - Fixes #6533 - Closes #6535 + Fixes #6764 + Closes #6766 -- [Tatsuhiro Tsujikawa brought this change] +- [Jacob Hoffman-Andrews brought this change] - ngtcp2: Fix http3 upload stall + docs: document version of crustls dependency - Closes #6521 + This also pins a specific release in the Travis test so future + API-breaking changins in crustls won't break curl builds. + + Add RUSTLS documentation to release tarball. + + Enable running tests for rustls, minus FTP tests (require + connect_blocking, which rustls doesn't implement) and 313 (requires CRL + handling). + + Closes #6763 -- [Tatsuhiro Tsujikawa brought this change] +- [Jacob Hoffman-Andrews brought this change] - ngtcp2: Fix stack buffer overflow + rustls: Handle close_notify. - Closes #6521 + If we get a close_notify, treat that as EOF. If we get an EOF from the + TCP stream, treat that as an error (because we should have ended the + connection earlier, when we got a close_notify). + + Closes #6763 -- warnless.h: remove the prototype for curlx_ultosi +- docs: clarify timeouts for queued transfers in multi API - Follow-up to 217552503ff3 + Closes #6758 -- warnless: remove curlx_ultosi +- ftpserver: only load the preprocessed test file - ... not used anywhere + We always preprocess and tests are no longer sensible to load "raw" - Closes #6530 + Closes #6738 -- [Patrick Monnerat brought this change] - - lib: remove conn->data uses +- tests: use %TESTNUMBER instead of fixed number - Closes #6515 + This makes the tests easier to copy and relocate to other test numbers + without having to update content. + + Closes #6738 -- pingpong: remove the 'conn' struct member +- KNOWN_BUGS: CURLOPT_OPENSOCKETPAIRFUNCTION is missing - ... as it's superfluous now when Curl_easy is passed in and we can - derive the connection from that instead and avoid the duplicate copy. + Closes #5747 + +- TODO: provide timing info for each redirect - Closes #6525 + Closes #6743 -- hostip/proxy: remove conn->data use +Jay Satiro (17 Mar 2021) +- docs: Add SSL backend names to CURL_SSL_BACKEND - Closes #6513 + - Document the names that can be used with CURL_SSL_BACKEND: + bearssl, gnutls, gskit, mbedtls, mesalink, nss, openssl, rustls, + schannel, secure-transport, wolfssl + + Ref: https://github.com/curl/curl/issues/2209#issuecomment-360623286 + Ref: https://github.com/curl/curl/issues/6717#issuecomment-800745201 + + Closes https://github.com/curl/curl/pull/6755 -- url: reduce conn->data references +- docs: Explain DOH transfers inherit some SSL settings - ... there are a few left but let's keep them to last + - Document in DOH that some SSL settings are inherited but DOH hostname + and peer verification are not and are controlled separately. - Closes #6512 + - Document that CURLOPT_SSL_CTX_FUNCTION is inherited by DOH handles but + we're considering changing behavior to no longer inherit it. Request + feedback. + + Closes https://github.com/curl/curl/pull/6688 -- scripts/singleuse: add curl_easy_option* +Daniel Stenberg (17 Mar 2021) +- http: make 416 not fail with resume + CURLOPT_FAILONERRROR + + When asked to resume a download, libcurl will convert that to HTTP logic + and if then the entire file is already transferred it will result in a + 416 response from the HTTP server. With CURLOPT_FAILONERRROR set in that + scenario, it should *not* lead to an error return. + + Updated test 1156, added test 1273 + + Reported-by: Jonathan Watt + Fixes #6740 + Closes #6753 -Jay Satiro (25 Jan 2021) -- test410: fix for windows +- Curl_timeleft: check both timeouts during connect - - Pass the very long request header via file instead of command line. + The duration of a connect and the total transfer are calculated from two + different time-stamps. It can end up with the total timeout triggering + before the connect timeout expires and we should make sure to + acknowledge whichever timeout that is reached first. - Prior to this change the 49k very long request header string was passed - via command line and on Windows that is too long so it was truncated and - the test would fail (specifically msys CI). + This is especially notable when a transfer first sits in PENDING, as + that time is counted in the total time but the connect timeout is based + on the time since the handle changed to the CONNECT state. - Closes https://github.com/curl/curl/pull/6516 + The CONNECTTIMEOUT is per connect attempt. The TIMEOUT is for the entire + operation. + + Fixes #6744 + Closes #6745 + Reported-by: Andrei Bica + Assisted-by: Jay Satiro -Daniel Stenberg (25 Jan 2021) -- libssh2: move data from connection object to transfer object +- configure: remove use of deprecated macros - Readdir data, filenames and attributes are strictly related to the - transfer and not the connection. This also reduces the total size of the - fixed connectdata struct. + AC_HEADER_TIME, AC_HEADER_STDC and AC_TYPE_SIGNAL + +- configure: make AC_TRY_* into AC_*_IFELSE - Closes #6519 + ... as the former versions are deprecated. -- RELEASE-NOTES: synced +- configure: s/AC_HELP_STRING/AS_HELP_STRING + + AC_HELP_STRING is deprecated in 2.70+ and I believe AS_HELP_STRING works + already since 2.59 so bump the minimum required version to that. + + Reported-by: Emil Engler + Fixes #6647 + Closes #6748 -- [Patrick Monnerat brought this change] +- RELEASE-NOTES: synced - lib: remove conn->data uses +- travis: use ubuntu nghttp2 package instead of build our own - Closes #6499 + Closes #6751 -- hyper: remove the conn->data references +- travis: bump wolfssl to 4.7.0 + +- travis: only build wolfssl when needed - Closes #6508 + Closes #6751 -- travis: build ngtcp2 --with-gnutls +- [Jacob Hoffman-Andrews brought this change] + + rustls: allocate a buffer for TLS data. - ... since they disable it by default since a few days back. + Previously, rustls was using an on-stack array for TLS data. However, + crustls has an (unusual) requirement that buffers it deals with are + initialized before writing to them. By using calloc, we can ensure the + buffer is initialized once and then reuse it across calls. - Closes #6506 - Fixes #6493 + Closes #6742 -- hostip: remove conn->data from resolver functions +- travis: add a rustls build - This also moves the 'async' struct from the connectdata struct into the - Curl_easy struct, which seems like a better home for it. + ... that doesn't run any tests (yet) - Closes #6497 + Closes #6750 -Jay Satiro (22 Jan 2021) -- strerror: skip errnum >= 0 assertion on windows - - On Windows an error number may be greater than INT_MAX and negative once - cast to int. +- HTTP2: remove the outdated remark about multiplexing for the tool + +- [Robert Ronto brought this change] + + http2: don't set KEEP_SEND when there's no more data to be sent - The assertion is checked only in debug builds. + this should fix an issue where curl sometimes doesn't send out a request + with authorization info after a 401 is received over http2 - Closes https://github.com/curl/curl/pull/6504 + Closes #6747 -Daniel Stenberg (21 Jan 2021) -- doh: make Curl_doh_is_resolved survive a NULL pointer +Marc Hoersken (15 Mar 2021) +- config: fix building SMB with configure using Win32 Crypto - ... if Curl_doh() returned a NULL, this function gets called anyway as - in a asynch procedure. Then the doh struct pointer is NULL and signifies - an OOM situation. + Align conditions for NTLM features between CMake and configure + builds by differentiating between USE_NTLM and USE_CURL_NTLM_CORE, + just like curl_setup.h does internally to detect support of: - Follow-up to 6246a1d8c6776 - -- wolfssh: remove conn->data references + - USE_NTLM: required for NTLM crypto authentication feature + - USE_CURL_NTLM_CORE: required for SMB protocol - ... and repair recent build breakage + Implement USE_WIN32_CRYPTO detection by checking for Crypt functions + in wincrypt.h which are not available in the Windows App environment. - Closes #6507 - -- http: empty reply connection are not left intact + Link advapi32 and crypt32 for Crypto API and Schannel SSL backend. + Fix condition of Schannel SSL backend in CMake build accordingly. - ... so mark the connection as closed in this condition to prevent that - verbose message to wrongly appear. + Reviewed-by: Marcel Raad - Reported-by: Matt Holt - Bug: https://twitter.com/mholt6/status/1352130240265375744 - Closes #6503 + Closes #6277 -- chunk/encoding: remove conn->data references +- config: fix detection of restricted Windows App environment - ... by anchoring more functions on Curl_easy instead of connectdata + Move the detection of the restricted Windows App environment + in curl_setup.h before the definition of USE_WIN32_CRYPTO + via included config-win32.h in case no build system is used. - Closes #6498 + Reviewed-by: Marcel Raad + + Part of #6277 -Jay Satiro (20 Jan 2021) -- [Erik Olsson brought this change] +Daniel Stenberg (15 Mar 2021) +- HISTORY: curl 7.7.2 was the first version used in Mac OS X 10.1 - lib: save a bit of space with some structure packing - - - Reorder some internal struct members so that less padding is used. +- gen.pl: quote "bare" minuses in the nroff curl.1 - This is an attempt at saving a bit of space by packing some structs - (using pahole to find the holes) where it might make sense to do - so without losing readability. + Reported-by: Alejandro Colomar + Fixes #6698 + Closes #6722 + +Daniel Gustafsson (14 Mar 2021) +- hsts: remove unused defines - I.e., I tried to avoid separating fields that seem grouped - together (like the cwd... fields in struct ftp_conn for instance). - Also abstained from touching fields behind conditional macros as - that quickly can get complicated. + MAX_HSTS_SUBLEN and MAX_HSTS_SUBLENSTR were unused from the initial commit, + and mostly likely leftovers from early development. Remove as they're not + used for anything. - Closes https://github.com/curl/curl/pull/6483 + Closes #6741 + Reviewed-by: Daniel Stenberg -Daniel Stenberg (20 Jan 2021) -- INSTALL.md: fix typo +Daniel Stenberg (12 Mar 2021) +- github: add torture-ftp for FTP-only torture testing - Found-by: Marcel Raad - -- [Fabian Keil brought this change] + and at 20% to try to keep the run-time reasonable + + Closes #6728 - http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy +- travis: split "torture" into a separate "events" build as well - Added test 1613 to verify. + Run torture without FTP and reducing coverage to 20% - Closes #6490 + For some reason the torture tests now run a lot slower on travis and run + into the 50 minute limit all the time. + + Closes #6728 -- Merge branch 'bagder/curl_range-data-conn' +- ftp: fix memory leak in ftp_done + + If after a transfer is complete Curl_GetFTPResponse() returns an error, + curl would not free the ftp->pathalloc block. + + Found by torture-testing test 576 + + Closes #6737 -- ftp: remove conn->data leftover +- [oxalica brought this change] -- curl_range: remove conn->data + http2: fail if connection terminated without END_STREAM - Closes #6496 + Closes #6736 -- INSTALL: now at 85 operating systems +- RELEASE-NOTES: synced -- quiche: fix unused parameter ‘conn’ +- [Jacob Hoffman-Andrews brought this change] + + rustls: support CURLOPT_SSL_VERIFYPEER - Follow-up to 2bdec0b3 + This requires the latest main branch of crustls, which provides + rustls_client_config_builder_dangerous_set_certificate_verifier and + rustls_client_config_builder_set_enable_sni. + + This refactors the session setup into its own function, and adds a new + function cr_hostname_is_ip. Because crustls doesn't support verification + of IP addresses, special handling is needed: We disable SNI and set a + placeholder hostname (which never actually gets sent on the wire). + + Closes #6719 -- transfer: fix ‘conn’ undeclared mistake for iconv build +Daniel Gustafsson (12 Mar 2021) +- cookies: Fix potential NULL pointer deref with PSL - Follow-up to 219d9f8620d + Curl_cookie_init can be called with data being NULL, and this can in turn + be passed to Curl_cookie_add, meaning that both functions must be careful + to only use data where it's checked for being a NULL pointer. The libpsl + support code does however dereference data without checking, so if we are + indeed having an unset data pointer we cannot PSL check the cookiedomain. + + This is currently not a reachable dereference, as the only caller with a + NULL data isn't passing a file to initialize cookies from, but since the + API has this contract let's ensure we hold it. + + Closes #6731 + Reviewed-by: Daniel Stenberg -- doh: allocate state struct on demand +Daniel Stenberg (12 Mar 2021) +- [Michael Hordijk brought this change] + + configure: only add OpenSSL paths if they are defined - ... instead of having it static within the Curl_easy struct. This takes - away 1176 bytes (18%) from the Curl_easy struct that aren't used very - often and instead makes the code allocate it when needed. + Add paths for OpenSSL compiling and linking only if they have been + defined. If they haven't been defined, we'll assume that the paths are + already available to the toolchain. - Closes #6492 + Closes #6730 -- socks: use the download buffer instead +Jay Satiro (12 Mar 2021) +- retry.d: Clarify transient 5xx HTTP response codes - The SOCKS code now uses the generic download buffer for temporary - storage during the connection procedure, instead of having its own - private 600 byte buffer that adds to the connectdata struct size. This - works fine because this point the buffer is allocated but is not use for - download yet since the connection hasn't completed. + - Clarify the only 5xx response codes that are treated as transient are + 500, 502, 503 and 504. - This reduces the connection struct size by 22% on a 64bit arch! + Prior to this change it said it treated all 5xx as transient, but the + code says otherwise. - The SOCKS buffer needs to be at least 600 bytes, and the download buffer - is guaranteed to never be smaller than 1000 bytes. + Ref: https://github.com/curl/curl/blob/curl-7_75_0/src/tool_operate.c#L462-L495 - Closes #6491 + Closes https://github.com/curl/curl/pull/6724 -- urldata: make magic be the first struct field +- retry-all-errors.d: Explain curl errors versus HTTP response errors - By making the `magic` identifier the same size and at the same place - within the structs (easy, multi, share), libcurl will be able to more - reliably detect and safely error out if an application passes in the - wrong handle to APIs. Easier to detect and less likely to cause crashes - if done. + - Add a paragraph explaining that curl does not consider HTTP response + errors as curl errors, and how that behavior can be modified by using + --retry and --fail. - Such mixups can't be detected at compile-time due to them being - typedefed void pointers - unless `CURL_STRICTER` is defined. + The --retry-all-errors doc says "Retry on any error" which some users + may find misleading without the added explanation. - Closes #6484 - -- http_chunks: correct and clarify a comment on hexnumber length + Ref: https://curl.se/docs/faq.html#Why_do_I_get_downloaded_data_eve + Ref: https://curl.se/docs/faq.html#curl_doesn_t_return_error_for_HT - ... and also rename the define for max length. + Reported-by: Lawrence Gripper - Closes #6489 + Fixes https://github.com/curl/curl/issues/6712 + Closes https://github.com/curl/curl/pull/6720 -- curl_path: remove conn->data use +Daniel Stenberg (11 Mar 2021) +- travis: switch ngtcp2 build over to quictls - Closes #6487 - -- transfer: remove conn->data use + The ngtcp2 project switched over to using the quictls OpenSSL fork + instead of their own patched OpenSSL. We follow suit. - Closes #6486 + Closes #6729 -- quic: remove conn->data use +- test220/314: adjust to run with Hyper + +- c-hyper: support automatic content-encoding - Closes #6485 + Closes #6727 -- [Fabian Keil brought this change] +- http: remove superfluous NULL assign + + Closes #6727 - Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" +- tool_operate: bail if set CURLOPT_HTTP09_ALLOWED returns error + + Closes #6727 -- [Fabian Keil brought this change] +- setopt: error on CURLOPT_HTTP09_ALLOWED set true with Hyper + + Not supported. + + Closes #6727 - Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive" +- test306: make it not run with Hyper - At the moment the test fails as curl sends two Proxy-Connection - headers. + ... as it tests HTTP/0.9 which Hyper doesn't support. -- c-hyper: avoid duplicated Proxy-Connection headers +- test304: header CRLF cleanup to work with Hyper -- http: make providing Proxy-Connection header not cause duplicated headers +- FTP: allow SIZE to fail when doing (resumed) upload - Fixes test 1180 + Added test 362 to verify. - Bug: https://curl.se/mail/lib-2021-01/0095.html - Reported-by: Fabian Keil - Closes #6472 + Reported-by: Jordan Brown + Regression since 7ea2e1d0c5a7f (7.73.0) + Fixes #6715 + Closes #6725 -- runtests: preprocess DISABLED to allow conditionals - - ... with this function provided, we can disable tests for specific - environments and setups directly within this file. +- configure: provide Largefile feature for curl-config - Closes #6477 - -- runtests: turn preprocessing into a separate function + ... as cmake now does it correctly, and make test1014 check for it - ... and remove all other variable substitutions as they're now done once - and for all in the preprocessor. + Closes #6702 -- lib/Makefile.inc: convert to listing each file on its own line +- config: remove CURL_SIZEOF_CURL_OFF_T use only SIZEOF_CURL_OFF_T - ... to make it diff friendlier and easier to read. + Make the code consistently use a single name for the size of the + "curl_off_t" type. - Closes #6448 + Closes #6702 -- ftplistparser: remove use of conn->data - - Closes #6482 +Jay Satiro (10 Mar 2021) +- [Jun-ya Kato brought this change] -- lib: more conn->data cleanups + ngtcp2: Fix build error due to change in ngtcp2_addr_init - Closes #6479 + ngtcp2/ngtcp2@b8d90a9 changed the function prototype. + + Closes https://github.com/curl/curl/pull/6716 -- [Patrick Monnerat brought this change] +Daniel Stenberg (10 Mar 2021) +- [ejanchivdorj brought this change] - vtls: reduce conn->data use + multi: update pending list when removing handle - Closes #6474 - -- hyper: deliver data to application with Curl_client_write + when removing a handle, most of the lists are updated but pending list + is not updated. Updating now. - ... just as the native code path does. Avoids sending too large data - chunks in the callback and more. + Closes #6713 + +- [kokke brought this change] + + lib1536: check ptr against NULL before dereferencing it - Reported-by: Gisle Vanem - Fixes #6462 - Closes #6473 + Closes #6710 -- gopher: remove accidental conn->data leftover +- [kokke brought this change] -- libssh: avoid plain free() of libssh-memory + lib1537: check ptr against NULL before dereferencing it - Since curl's own memory debugging system redefines free() calls to track - and fiddle with memory, it cannot be used on memory allocated by 3rd - party libraries. + Fixes #6707 + Closes #6708 + +- travis: make torture tests skip TLS-SRP tests - Third party libraries SHOULD NOT require free() to release allocated - resources for this reason - and libs can use separate healp allocators - on some systems (like Windows) so free() doesn't necessarily work - anyway. + ... as it seems to often hang. - Filed as an issue with libssh: https://bugs.libssh.org/T268 + Also: skip the "normal" tests as they're already run by many other + builds. - Closes #6481 + Closes #6705 -- send: assert that Curl_write_plain() has a ->conn when called +- openssl: adapt to v3's new const for a few API calls - To help catch bad invokes. + Closes #6703 + +- quiche: fix crash when failing to connect - Closes #6476 + Reported-by: ウさん + Fixes #6664 + Closes #6701 -- test410: verify HTTPS GET with a 49K request header +- RELEASE-NOTES: synced - skip test 410 for mesalink in the CI as it otherwise hangs "forever" + Fixed the release counter and added a missing contributor -- lib: pass in 'struct Curl_easy *' to most functions +- RELEASE-NOTES: synced + +- dynbuf: bump the max HTTP request to 1MB - ... in most cases instead of 'struct connectdata *' but in some cases in - addition to. + Raised from 128KB to allow longer request headers. - - We mostly operate on transfers and not connections. + Reported-by: Carl Zogheib + Fixes #6681 + Closes #6685 + +Jay Satiro (6 Mar 2021) +- schannel: Evaluate CURLOPT_SSL_OPTIONS via SSL_SET_OPTION macro - - We need the transfer handle to log, store data and more. Everything in - libcurl is driven by a transfer (the CURL * in the public API). + - Change use of those options from CURLOPT_SSL_OPTIONS that are not + already evaluated via SSL_SET_OPTION in schannel and secure transport + to use that instead of data->set.ssl.optname. - - This work clarifies and separates the transfers from the connections - better. + Example: - - We should avoid "conn->data". Since individual connections can be used - by many transfers when multiplexing, making sure that conn->data - points to the current and correct transfer at all times is difficult - and has been notoriously error-prone over the years. The goal is to - ultimately remove the conn->data pointer for this reason. + Evaluate SSL_SET_OPTION(no_revoke) instead of data->set.ssl.no_revoke. - Closes #6425 - -Emil Engler (17 Jan 2021) -- docs: fix typos in NEW-PROTOCOL.md + This change is because options set via CURLOPT_SSL_OPTIONS + (data->set.ssl.optname) are separate from those set for HTTPS proxy via + CURLOPT_PROXY_SSL_OPTIONS (data->set.proxy_ssl.optname). The + SSL_SET_OPTION macro determines whether the connection is for HTTPS + proxy and based on that which option to evaluate. - This fixes a misspelled "it" and a grammatically wrong "-ing" suffix. + Since neither Schannel nor Secure Transport backends currently support + HTTPS proxy in libcurl, this change is for posterity and has no other + effect. - Closes #6471 - -Daniel Stenberg (16 Jan 2021) -- RELEASE-NOTES: synced + Closes https://github.com/curl/curl/pull/6690 -Jay Satiro (16 Jan 2021) -- [Razvan Cojocaru brought this change] +- [kokke brought this change] - cmake: expose CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG + c-hyper: Remove superfluous pointer check - This does for cmake builds what --disable-openssl-auto-load-config - does for autoconf builds. + `n` pointer is never NULL once set. Found by static analysis. - Closes https://github.com/curl/curl/pull/6435 + Ref: https://github.com/curl/curl/issues/6696 + + Closes https://github.com/curl/curl/pull/6697 -Daniel Stenberg (15 Jan 2021) -- test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id() +- version.d: Add missing features to the features list - ... and as a practical side-effect, make sure that the - Curl_easyopts_check() function is asserted in debug builds, which we - want to detect mismatches between the options list in easyoptions.c and - the options in curl.h + - Add missing entries for gsasl, Kerberos, NTLM_WB, TrackMemory, + Unicode and zstd. - Found-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45991815 + - Remove krb4 since it's no longer a feature. - Closes #6461 + Reported-by: Ádler Jonas Gross + + Fixes https://github.com/curl/curl/issues/6677 + Closes https://github.com/curl/curl/pull/6687 -- [Gisle Vanem brought this change] +- [Vladimir Varlamov brought this change] - easyoptions: add the missing AWS_SIGV4 + docs: add missing Arg tag to --stderr - Follow-up from AWS_SIGV4 - -- schannel_verify: fix safefree call typo + Prior to this change the required argument was not shown. - Follow-up from e87ad71d1ba00519 + curl.1 before: --stderr + curl.1 after: --stderr - Closes #6459 + curl --help before: + --stderr Where to redirect stderr + + curl --help after: + --stderr Where to redirect stderr + + Closes https://github.com/curl/curl/pull/6692 -- mime: make sure setting MIMEPOST to NULL resets properly +- projects: Update VS projects for OpenSSL 1.1.x - ... so that a function can first use MIMEPOST and then set it to NULL to - reset it back to a blank POST. + - Update VS project templates to use the OpenSSL lib names and include + directories for OpenSSL 1.1.x. - Added test 584 to verify the fix. + This change means the VS project files will now build only with OpenSSL + 1.1.x when an OpenSSL configuration is chosen. Prior to this change the + project files built only with OpenSSL 1.0.x (end-of-life) when an + OpenSSL configuration was chosen. - Reported-by: Christoph M. Becker + The template changes in this commit were made by script: - Fixes #6455 - Closes #6456 - -- multi: set the PRETRANSFER time-stamp when we switch to PERFORM + libeay32.lib => libcrypto.lib + ssleay32.lib => libssl.lib + ..\..\..\..\..\openssl\inc32 => ..\..\..\..\..\openssl\include - ... instead of at end of the DO state. This makes the timer more - accurate for the protocols that use the DOING state (such as FTP), and - simplifies how the function (now called init_perform) is called. + And since the output directory now contains the includes it's prepended: + ..\..\..\..\..\openssl\build\Win{32,64}\VC{6..15}\{DLL,LIB} + {Debug,Release}\include - The timer will then include the entire procedure up to PERFORM - - including all instructions for getting the transfer started. + - Change build-openssl.bat to copy the build's include directory to the + output directory (as seen above). - Closes #6454 - -- CURLINFO_PRETRANSFER_TIME.3: clarify + Each build has its own opensslconf.h which is different so we can't just + include the source include directory any longer. - ... the timer *does* include the instructions for getting the remote - file. + Note the include directory in the output directory is a full copy from + the build so technically we don't need to include the OpenSSL source + include directory in the template. However, I left it last in case the + user made a custom OpenSSL build using the old method which would put + opensslconf in the OpenSSL source include directory. - Ref: #6452 - Closes #6453 - -- [Gisle Vanem brought this change] + - Change build-openssl.bat to use a temporary install directory that is + different from the temporary build directory. + + For OpenSSL 1.1.x the temporary paths must be separate not a descendant + of the other, otherwise pdb files will be lost between builds. + + Ref: https://curl.se/mail/lib-2018-10/0049.html + Ref: https://gist.github.com/jay/125191c35bbeb894444eff827651f755 + Ref; https://github.com/openssl/openssl/issues/10005 + + Fixes https://github.com/curl/curl/issues/984 + Closes https://github.com/curl/curl/pull/6675 - schannel: plug a memory-leak +- doh: Inherit CURLOPT_STDERR from user's easy handle - ... when built without -DUNICODE. + Prior to this change if the user set their easy handle's error stream + to something other than stderr it was not inherited by the doh handles, + which meant that they would still write to the default standard error + stream (stderr) for verbose output. - Closes #6457 + Bug: https://github.com/curl/curl/issues/6605 + Reported-by: arvids-kokins-bidstack@users.noreply.github.com + + Closes https://github.com/curl/curl/pull/6661 -Jay Satiro (14 Jan 2021) -- gitattributes: Set batch files to CRLF line endings on checkout +Marc Hoersken (1 Mar 2021) +- CI/azure: replace python-impacket with python3-impacket - If a batch file is run without CRLF line endings (ie LF-only) then - arbitrary behavior may occur. I consider that a bug in Windows, however - the effects can be serious enough (eg unintended code executed) that - we're fixing it in the repo by requiring CRLF line endings for batch - files on checkout. + As of this month Azure DevOps uses Ubuntu 20.04 LTS which + no longer supports Python 2 and instead ships Python 3. - Prior to this change the checked-out line endings of batch files were - dependent on a user's git preferences. On Windows it is common for git - users to have automatic CRLF conversion enabled (core.autocrlf true), - but those users that don't would run into this behavior. + Closes #6678 + +- runtests.pl: kill processes locking test log files - For example a user has reported running the Visual Studio project - generator batch file (projects/generate.bat) and it looped forever. - Output showed that the Windows OS interpreter was occasionally jumping - to arbitrary points in the batch file and executing commands. This - resulted in unintended files being removed (a removal sequence called) - and looping forever. + Introduce a new runtests.pl command option: -rm - Ref: https://serverfault.com/q/429594 - Ref: https://stackoverflow.com/q/232651 - Ref: https://www.dostips.com/forum/viewtopic.php?t=8988 - Ref: https://git-scm.com/docs/gitattributes#_checking_out_and_checking_in - Ref: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_core_autocrlf + For now only required and implemented for Windows. + Ignore stunnel logs due to long running processes. - Bug: https://github.com/curl/curl/discussions/6427 - Reported-by: Ganesh Kamath + Requires Sysinternals handle[64].exe to be on PATH. - Closes https://github.com/curl/curl/pull/6442 - -Daniel Stenberg (14 Jan 2021) -- tool_operate: spellfix a comment - -- ROADMAP: refreshed + Reviewed-by: Jay Satiro - o removed HSTS - already implemented - o added HTTPS RR records - o mention HTTP/3 completion - -- http_chunks: remove Curl_ prefix from static functions - -- transfer: remove Curl_ prefix from static functions - -- tftp: remove Curl_ prefix from static functions - -- multi: remove Curl_ prefix from static functions - -- ldap: remove Curl_ prefix from static functions - -- doh: remove Curl_ prefix from static functions + Ref: #6058 + Closes #6179 -- asyn-ares: remove Curl_ prefix from static functions +- pathhelp.pm: fix use of pwd -L in Msys environment + + While Msys2 has a pwd binary which supports -L, + Msys1 only has a shell built-in with that feature. + + Reviewed-by: Jay Satiro + + Part of #6179 -- vtls: remove Curl_ prefix from static functions +Daniel Gustafsson (1 Mar 2021) +- ldap: use correct memory free function + + unescaped is coming from Curl_urldecode and not a unicode conversion + function, so reclaiming its memory should be performed with a normal + call to free rather than curlx_unicodefree. In reality, this is the + same thing as curlx_unicodefree is implemented as a call to free but + that's not guaranteed to always hold. Using the curlx macro present + issues with memory debugging as well. + + Closes #6671 + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg -- bearssl: remove Curl_ prefix from static functions +- url: fix typo in comment + + Correct a small typo which snuck in with a304051620. -- mbedtls: remove Curl_ prefix from static functions +Jay Satiro (28 Feb 2021) +- tool_help: Increase space between option and description + + - Increase the minimum number of spaces between the option and the + description from 1 to 2. + + Before: + ~~~ + -u, --user Server user and password + -A, --user-agent Send User-Agent to server + -v, --verbose Make the operation more talkative + -V, --version Show version number and quit + -w, --write-out Use output FORMAT after completion + --xattr Store metadata in extended file attributes + ~~~ + + After: + ~~~ + -u, --user Server user and password + -A, --user-agent Send User-Agent to server + -v, --verbose Make the operation more talkative + -V, --version Show version number and quit + -w, --write-out Use output FORMAT after completion + --xattr Store metadata in extended file attributes + ~~~ + + Closes https://github.com/curl/curl/pull/6674 -- wolfssl: remove Curl_ prefix from static functions +Daniel Stenberg (27 Feb 2021) +- curl: set CURLOPT_NEW_FILE_PERMS if requested + + The --create-file-mode code logic accepted the value but never actually + passed it on to libcurl! + + Follow-up to a7696c73436f (shipped in 7.75.0) + Reported-by: Johannes Lesr + Fixes #6657 + Closes #6666 -- nss: remove Curl_ prefix from static functions +- tool_operate: check argc before accessing argv[1] + + Follow-up to 09363500b + Reported-by: Emil Engler + Reviewed-by: Daniel Gustafsson + Closes #6668 -- gnutls: remove Curl_ prefix from static functions +Daniel Gustafsson (26 Feb 2021) +- [Jean-Philippe Menil brought this change] -- openssl: remove Curl_ prefix from static functions + openssl: remove get_ssl_version_txt in favor of SSL_get_version - ... as we reserve this prefix to library-wide functions. + openssl: use SSL_get_version to get connection protocol - Closes #6443 - -- nss: get the run-time version instead of build-time + Replace our bespoke get_ssl_version_txt in favor of SSL_get_version. + We can get rid of few lines of code, since SSL_get_version achieve + the exact same thing - Closes #6445 + Closes #6665 + Reviewed-by: Daniel Gustafsson + Signed-off-by: Jean-Philippe Menil -Jay Satiro (12 Jan 2021) -- tool_doswin: Restore original console settings on CTRL signal +- gnutls: Fix nettle discovery - - Move Windows terminal init code from tool_main to tool_doswin. + Commit e06fa7462ac258c removed support for libgcrypt leaving only + support for nettle which has been the default crypto library in + GnuTLS for a long time. There were however a few conditionals on + USE_GNUTLS_NETTLE which cause compilation errors in the metalink + code (as it used the gcrypt fallback instead as a result). See the + below autobuild for an example of the error: - - Restore the original console settings on CTRL+C and CTRL+BREAK. + https://curl.se/dev/log.cgi?id=20210225123226-30704#prob1 - Background: On Windows the curl tool changes the console settings to - enable virtual terminal processing (eg color output) if supported - (ie Win 10). The original settings are restored on exit but prior to - this change were not restored in the case of the CTRL signals. + This removes all uses of USE_GNUTLS_NETTLE and also removes the + gcrypt support from the metalink code while at it. - Windows VT behavior varies depending on console/powershell/terminal; - refer to the discussion in #6226. + Closes #6656 + Reviewed-by: Daniel Stenberg + +- cookies: Support multiple -b parameters - Assisted-by: Rich Turner + Previously only a single -b cookie parameter was supported with the last + one winning. This adds support for supplying multiple -b params to have + them serialized semicolon separated. Both cookiefiles and cookies can be + entered multiple times. - Closes https://github.com/curl/curl/pull/6226 + Closes #6649 + Reviewed-by: Daniel Stenberg -Daniel Stenberg (12 Jan 2021) -- gen.pl: fix perl syntax +Daniel Stenberg (25 Feb 2021) +- build: remove all traces of USE_BLOCKING_SOCKETS - Follow-up to 324cf1d2e + libcurl doesn't behave properly with the define set + + Closes #6655 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - help: update to current codebase - - This commit bumps the help to the current state of the project. +Daniel Gustafsson (25 Feb 2021) +- docs: Fix typos - Closes #6437 - -- [Emil Engler brought this change] + Random typos spotted when skimming docs. - docs: fix line length bug in gen.pl +- cookies: Use named parameters in header prototypes - The script warns if the length of $opt and $desc is > 78. However, these - two variables are on totally separate lines so the check makes no sense. - Also the $bitmask field is totally forgotten. Currently this leads to - two warnings within `--resolve` and `--aws-sigv4`. + Align header with project style of using named parameters in the + function prototypes to aid readability and self-documentation. - Closes #6438 - -- [Emil Engler brought this change] + Closes #6653 + Reviewed-by: Daniel Stenberg - docs: fix wrong documentation in help.d +Daniel Stenberg (24 Feb 2021) +- urldata: make 'actions[]' use unsigned char instead of int - curl does not list all categories when you invoke "--help" without any - parameters. + ... as it only needs a few bits per index anyway. - Closes #6436 + Reviewed-by: Daniel Gustafsson + Closes #6648 -- aws-sigv4.d: polish the wording - - Make it shorter and imperative form +- configure: fail if --with-quiche is used and quiche isn't found - Closes #6439 + Closes #6652 -- [Fabian Keil brought this change] +- [Gregor Jasny brought this change] - misc: fix typos + cmake: use CMAKE_INSTALL_INCLUDEDIR indirection - Bug: https://curl.se/mail/lib-2021-01/0063.html - Closes #6434 + Reviewed-by: Sergei Nikulov + Closes #6440 -- multi_runsingle: bail out early on data->conn == NULL +Viktor Szakats (23 Feb 2021) +- mingw: enable using strcasecmp() - As that's a significant error condition and scan-build warns for NULL - pointer dereferences if we don't. + This makes the 'Features:' list sorted case-insensitively, + bringing output in-line with *nix builds. - Closes #6433 + Reviewed-by: Jay Satiro + Closes #6644 -- multi: skip DONE state if there's no connection left for ftp wildcard +- build: delete unused feature guards - ... to avoid running in that state with data->conn being NULL. + - `HAVE_STRNCASECMP` + - `HAVE_TCGETATTR` + - `HAVE_TCSETATTR` + + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes #6645 -- libssh2: fix "Value stored to 'readdir_len' is never read" +Jay Satiro (23 Feb 2021) +- docs: add CURLOPT_CURLU to 'See also' in curl_url_ functions - Detected by scan-build + Closes https://github.com/curl/curl/pull/6639 -- connect: mark intentional ignores of setsockopt return values +Daniel Stenberg (23 Feb 2021) +- [Jacob Hoffman-Andrews brought this change] + + configure: make hyper opt-in, and fail if missing - Pointed out by Coverity + Previously, configure would look for hyper by default, and use it if + found; otherwise it would not use hyper, and not error. - Closes #6431 - -Jay Satiro (11 Jan 2021) -- http_proxy: Fix CONNECT chunked encoding race condition + Now, configure will not look for hyper unless --with-hyper is passed. If + configure looks for hyper and fails, it will error. - - During the end-of-headers response phase do not mark the tunnel - complete unless the response body was completely parsed/ignored. + Also, add -ld -lpthread -lm to Hyper's libs. I think they are required. - Prior to this change if the entirety of a CONNECT response with chunked - encoding was not received by the time the final header was parsed then - the connection would be marked done prematurely, before all the chunked - data could be read in and ignored (since this is what we do with any - CONNECT response body) and the connection could not be used. + Closes #6598 + +- multi: do once-per-transfer inits in before_perform in DID state - Bug: https://curl.se/mail/lib-2021-01/0033.html - Reported-by: Fabian Keil + ... since the state machine might go to RATELIMITING and then back to + PERFORMING doing once-per-transfer inits in that function is wrong and + it caused problems with receiving chunked HTTP and it set the + PRETRANSFER time much too often... - Closes https://github.com/curl/curl/pull/6432 + Regression from b68dc34af341805aeb7b3715 (shipped in 7.75.0) + + Reported-by: Amaury Denoyelle + Fixes #6640 + Closes #6641 -Daniel Stenberg (11 Jan 2021) - RELEASE-NOTES: synced -- url: if IDNA conversion fails, fallback to Transitional +- CODE_STYLE.md: fix broken link to INTERNALS - This improves IDNA2003 compatiblity. + ... the link would only work if browsed on GitHub, while this link now + takes the user to the website instead and thus should work on either. - Reported-by: Bubu on github - Fixes #6423 - Closes #6428 + Reported-by: David Demelier -- travis: make the Hyper build from its master branch +- curl_url_set.3: mention CURLU_PATH_AS_IS - Closes #6430 - -- http: make 'authneg' also work for Hyper + ... it has been supported since the URL API was added. - When doing a request with a request body expecting a 401/407 back, that - initial request is sent with a zero content-length. Test 177 and more. + Bug: https://curl.se/mail/lib-2021-02/0046.html - Closes #6424 + Closes #6638 -Jay Satiro (8 Jan 2021) -- cmake: Add an option to disable libidn2 +Viktor Szakats (21 Feb 2021) +- time: enable 64-bit time_t in supported mingw environments - New option USE_LIBIDN2 defaults to ON for libidn2 detection. Prior to - this change libidn2 detection could not be turned off in cmake builds. + (Unless 32-bit `time_t` is selected manually via the `_USE_32BIT_TIME_T` + mingw macro.) - Reported-by: William A Rowe Jr + Previously, 64-bit `time_t` was enabled on VS2005 and newer only, and + 32-bit `time_t` was used on all other Windows builds. - Fixes https://github.com/curl/curl/issues/6361 - Closes https://github.com/curl/curl/pull/6362 - -Daniel Stenberg (8 Jan 2021) -- HYPER: no longer needs the special branch + Assisted-by: Jay Satiro + Closes #6636 -- test179: use consistent header line endings +Jay Satiro (20 Feb 2021) +- test1188: Check for --fail HTTP status - ... to make "Hyper mode" work better. - -- file: don't provide content-length for directories + - Change the test to check for curl error on HTTP 404 Not Found. - ... as it is misleading. + test1188 tests "--write-out with %{onerror} and %{urlnum} to stderr". + Prior to this change it did that by specifying a non-existent host which + would cause an error. ISPs may hijack DNS and resolve non-existent hosts + so the test would not work if that was the case. - Ref #6379 - Closes #6421 - -- TODO: Directory listing for FILE: + Ref: https://en.wikipedia.org/wiki/DNS_hijacking#Manipulation_by_ISPs + Ref: https://github.com/curl/curl/issues/6621 + Ref: https://github.com/curl/curl/pull/6623 - Ref #6379 + Closes https://github.com/curl/curl/pull/6637 -- curl.h: add CURLPROTO_GOPHERS as own protocol identifier +- memdebug: close debug logfile explicitly on exit - Follow-up to a1f06f32b860, to make sure it can be handled separately - from plain gopher. + - Use atexit to register a dbg cleanup function that closes the logfile. - Closes #6418 - -- http: have CURLOPT_FAILONERROR fail after all headers + LeakSantizier (LSAN) calls _exit() instead of exit() when a leak is + detected on exit so the logfile must be closed explicitly or data could + be lost. Though _exit() does not call atexit handlers such as this, + LSAN's call to _exit() comes after the atexit handlers are called. - ... so that Retry-After and other meta-content can still be used. + Prior to this change the logfile was not explicitly closed so it was + possible that if LSAN detected a leak and called _exit (which does + not flush or close files like exit) then the logfile could be missing + data. That could then cause curl's memanalyze to report false leaks + (eg a malloc was recorded to the logfile but the corresponding free was + discarded from the buffer instead of written to the logfile, then + memanalyze reports that as a leak). - Added 1634 to verify. Adjusted test 194 and 281 since --fail now also - includes the header-terminating CRLF in the output before it exits. + Ref: https://github.com/google/sanitizers/issues/1374 - Fixes #6408 - Closes #6409 + Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541 + + Closes https://github.com/curl/curl/pull/6620 -- global_init: debug builds allocates a byte in init +- curl_multibyte: always return a heap-allocated copy of string - ... to make build tools/valgrind warn if no curl_global_cleanup is - called. + - Change the Windows char <-> UTF-8 conversion functions to return an + allocated copy of the passed in string instead of the original. - This is conditionally only done for debug builds with the env variable - CURL_GLOBAL_INIT set. + Prior to this change the curlx_convert_ functions would, as what I + assume was an optimization, not make a copy of the passed in string if + no conversion was required. No conversion is required in non-UNICODE + Windows builds since our tchar strings are type char and remain in + whatever the passed in encoding is, which is assumed to be UTF-8 but may + be other encoding. - Closes #6410 - -- lib/unit tests: add missing curl_global_cleanup() calls + In contrast the UNICODE Windows builds require conversion + (wchar <-> char) and do return a copy. That inconsistency could lead to + programming errors where the developer expects a copy, and does not + realize that won't happen in all cases. + + Closes https://github.com/curl/curl/pull/6602 -- travis: adapt to Hyper build change +Viktor Szakats (19 Feb 2021) +- http: add new files missed from referrer commit - Closes #6419 + Ref: 44872aefc2d54f297caf2b0cc887df321bc9d791 + Ref: #6591 -- pretransfer: setup the User-Agent header here +- http: add support to read and store the referrer header - ... and not in the connection setup, as for multiplexed transfers the - connection setup might be skipped and then the transfer would end up - without the set user-agent! + - add CURLINFO_REFERER libcurl option + - add --write-out '%{referer}' command-line option + - extend --xattr command-line option to fill user.xdg.referrer.url extended + attribute with the referrer (if there was any) - Reported-by: Flameborn on github - Assisted-by: Andrey Gursky - Assisted-by: Jay Satiro - Assisted-by: Mike Gelfand - Fixes #6312 - Closes #6417 + Closes #6591 -- test66: disable with Hyper +Daniel Stenberg (19 Feb 2021) +- urldata: remove the _ORIG suffix from string names - ...as Hyper doesn't support HTTP/0.9 + It doesn't provide any useful info but only makes the names longer. + + Closes #6624 -- c-hyper: poll the tasks until end correctly +- url: fix memory leak if OOM in the HSTS handling - ... makes test 36 work. + Reported-by: Viktor Szakats + Bug: https://github.com/curl/curl/pull/6627#issuecomment-781626205 - Closes #6412 - -- [Gergely Nagy brought this change] + Closes #6628 - mk-ca-bundle.pl: deterministic output when using -t +- gnutls: assume nettle crypto support - Printing trust purposes are now sorted, making the output deterministic - when running on the same input certdata.txt. + nettle has been the default crypto library with GnuTLS since 2010. By + dropping support for the previous libcrypto, we simplify code. - Closes #6413 + Closes #6625 -- KNOWN_BUGS: fixed "wolfSSL lacks support for renegotiation" +- asyn-ares: use consistent resolve error message - Fixed by #6411 - -- [Himanshu Gupta brought this change] - - wolfssl: add SECURE_RENEGOTIATION support + ... with the help of Curl_resolver_error() which now is moved from + asyn-thead.c and is provided globally for this purpose. - Closes #6411 - -- RELEASE-NOTES: synced - -- wolfssl: update copyright year range + Follow-up to 35ca04ce1b77636 - Follow-up to 7de2e96535e9 + Makes test 1188 work for c-ares builds + + Closes #6626 -- c-hyper: make CURLE_GOT_NOTHING work +Viktor Szakats (18 Feb 2021) +- ci: stop building on freebsd-12-1 - Test 30 + An updated freebsd-12-2 image was added a few months ago, and this + older one is consistently failing to go past `pkginstall`: + ``` + Newer FreeBSD version for package py37-mlt: + To ignore this error set IGNORE_OSVERSION=yes + - package: 1202000 + - running kernel: 1201000 + Ignore the mismatch and continue? [Y/n]: pkg: repository FreeBSD contains packages for wrong OS version: FreeBSD:12:amd64 + ``` - Closes #6407 + FreeBSD thread suggests that 12.1 is EOL, and best to avoid. + + Ref: https://forums.freebsd.org/threads/78856/ + + Reviewed-by: Daniel Stenberg + Closes #6622 -- http_proxy: make CONNECT work with the Hyper backend +Daniel Stenberg (18 Feb 2021) +- test1188: change error from connect to resolve error - Makes test 80 run + Using the %NOLISTENPORT to trigger a connection failure is somewhat + "risky" (since it isn't guaranteed to not be listened to) and caused + occasional CI problems. This fix changes the infused error to be a more + reliable one but still verifies the --write-out functionality properly - + which is the purpose of this test. - Closes #6406 + Reported-by: Jay Satiro + Fixes #6621 + Closes #6623 -- TODO: --fail-with-body perchance? +- url.c: use consistent error message for failed resolve -Jay Satiro (4 Jan 2021) -- tool_operate: fix the suppression logic of some error messages +- BUGS: language polish + +- wolfssl: don't store a NULL sessionid - - Fix the failed truncation and failed writing body error messages to - not be shown unless error messages are shown. (ie the user has - specified -sS, or has not specified -s). + This caused a memory leak as the session id cache entry was still + erroneously stored with a NULL sessionid and that would later be treated + as not needed to get freed. - - Also prefix same error messages with "curl: ", for example: - curl: (23) Failed to truncate, exiting + Reported-by: Gisle Vanem + Fixes #6616 + Closes #6617 + +- parse_proxy: fix a memory leak in the OOM path - Prior to this change the failed truncation error messages would be shown - if not -s, but did not account for -sS which should show. + Reported-by: Jay Satiro + Reviewed-by: Jay Satiro + Reviewed-by: Emil Engler - Prior to this change the failed writing body error messages would be - shown always. + Closes #6614 + Bug: https://github.com/curl/curl/pull/6591#issuecomment-780396541 + +Jay Satiro (17 Feb 2021) +- url: fix possible use-after-free in default protocol - Ref: https://curl.se/docs/manpage.html#-S + Prior to this change if the user specified a default protocol and a + separately allocated non-absolute URL was used then it was freed + prematurely, before it was then used to make the replacement URL. - Bug: https://curl.se/mail/archive-2020-12/0017.html - Reported-by: Hongyi Zhao + Bug: https://github.com/curl/curl/issues/6604#issuecomment-780138219 + Reported-by: arvids-kokins-bidstack@users.noreply.github.com - Closes https://github.com/curl/curl/pull/6402 + Closes https://github.com/curl/curl/pull/6613 -- wolfssl: Support wolfSSL builds missing TLS 1.1 +Daniel Stenberg (16 Feb 2021) +- multi: rename the multi transfer states - The wolfSSL TLS library defines NO_OLD_TLS in some of their build - configurations and that causes the library to be built without TLS 1.1. - For example if MD5 is explicitly disabled when building wolfSSL then - that defines NO_OLD_TLS and the library is built without TLS 1.1 [1]. + While working on documenting the states it dawned on me that step one is + to use more descriptive names on the states. This also changes prefix on + the states to make them shorter in the source. - Prior to this change attempting to build curl with a wolfSSL that was - built with NO_OLD_TLS would cause a build link error undefined reference - to wolfTLSv1_client_method. + State names NOT ending with *ing are transitional ones. - [1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366 + Closes #6612 + +Viktor Szakats (16 Feb 2021) +- http: do not add a referrer header with empty value - Bug: https://curl.se/mail/lib-2020-12/0121.html - Reported-by: Julian Montes + Previously an empty 'Referer:' header was added to the HTTP request when + passing `--referer ';auto'` or `--referer ''` on the command-line. This + patch makes `--referer` work like `--header 'Referer:'` and will only add + the header if it has a non-zero length value. - Closes https://github.com/curl/curl/pull/6388 + Reviewed-by: Jay Satiro + Closes #6610 -Daniel Stenberg (4 Jan 2021) -- test1633: set appropriate name +Daniel Stenberg (16 Feb 2021) +- lib: remove 'conn->data' completely - "--retry with a 429 response and Retry-After:" + The Curl_easy pointer struct entry in connectdata is now gone. Just + before commit 215db086e0 landed on January 8, 2021 there were 919 + references to conn->data. + + Closes #6608 -- travis: limit the tests with quiche builds to HTTPS and FTPS only +- openldap: pass 'data' to the callbacks instead of 'conn' + +Jay Satiro (15 Feb 2021) +- doh: Fix sharing user's resolve list with DOH handles - ... since it runs into the 50 minute time limit too often otherwise. + - Share the shared object from the user's easy handle with the DOH + handles. - Closes #6403 + Prior to this change if the user had set a shared object with shared + cached DNS (CURL_LOCK_DATA_DNS) for their easy handle then that wasn't + used by any associated DOH handles, since they used the multi's default + hostcache. + + This change means all the handles now use the same hostcache, which is + either the shared hostcache from the user created shared object if it + exists or if not then the multi's default hostcache. + + Reported-by: Manuj Bhatia + + Fixes https://github.com/curl/curl/issues/6589 + Closes https://github.com/curl/curl/pull/6607 -- HISTORY: added dates to early history +Daniel Stenberg (15 Feb 2021) +- http2: remove conn->data use - Mostly thanks to this archived web page for urlget: + ... but instead use a private alternative that points to the "driving + transfer" from the connection. We set the "user data" associated with + the connection to be the connectdata struct, but when we drive transfers + the code still needs to know the pointer to the transfer. We can change + the user data to become the Curl_easy handle, but with older nghttp2 + version we cannot dynamically update that pointer properly when + different transfers are used over the same connection. - https://web.archive.org/web/19980216125115/http://www.inf.ufrgs.br/~sagula/urlget.html + Closes #6520 -- httpauth: make multi-request auth work with custom port +- openssl: remove conn->data use - When doing HTTP authentication and a port number set with CURLOPT_PORT, - the code would previously have the URL's port number override as if it - had been a redirect to an absolute URL. + We still make the trace callback function get the connectdata struct + passed to it, since the callback is anchored on the connection. - Added test 1568 to verify. + Repeatedly updating the callback pointer to set 'data' with + SSL_CTX_set_msg_callback_arg() doesn't seem to work, probably because + there might already be messages in the queue with the old pointer. - Reported-by: UrsusArctos on github - Fixes #6397 - Closes #6400 + This code therefore makes sure to set the "logger" handle before using + OpenSSL calls so that the right easy handle gets used for tracing. + + Closes #6522 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - language: s/behaviour/behavior/g +Jay Satiro (14 Feb 2021) +- doh: add options to disable ssl verification - We currently use both spellings the british "behaviour" and the american - "behavior". However "behavior" is more used in the project so I think - it's worth dropping the british name. + - New libcurl options CURLOPT_DOH_SSL_VERIFYHOST, + CURLOPT_DOH_SSL_VERIFYPEER and CURLOPT_DOH_SSL_VERIFYSTATUS do the + same as their respective counterparts. - Closes #6395 + - New curl tool options --doh-insecure and --doh-cert-status do the same + as their respective counterparts. + + Prior to this change DOH SSL certificate verification settings for + verifyhost and verifypeer were supposed to be inherited respectively + from CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER, but due to a bug + were not. As a result DOH verification remained at the default, ie + enabled, and it was not possible to disable. This commit changes + behavior so that the DOH verification settings are independent and not + inherited. + + Ref: https://github.com/curl/curl/pull/4579#issuecomment-554723676 + + Fixes https://github.com/curl/curl/issues/4578 + Closes https://github.com/curl/curl/pull/6597 -- cmdline-opts/retry.d: mention response code 429 as well +- hostip: fix crash in sync resolver builds that use DOH - Reported-by: Cherish98 - Bug: https://curl.se/mail/archive-2020-12/0018.html + - Guard some Curl_async accesses with USE_CURL_ASYNC instead of + !CURLRES_SYNCH. + + This is another follow-up to 8335c64 which moved the async struct from + the connectdata struct into the Curl_easy struct. A previous follow-up + 6cd167a fixed building for sync resolver by guarding some async struct + accesses with !CURLRES_SYNCH. The problem is since DOH (DNS-over-HTTPS) + is available as an asynchronous secondary resolver the async struct may + be used even when libcurl is built for the sync resolver. That means + that CURLRES_SYNCH and USE_CURL_ASYNC may be defined at the same time. + + Closes https://github.com/curl/curl/pull/6603 -- docs/HYPER.md: mention outstanding issues +Daniel Stenberg (13 Feb 2021) +- KNOWN_BUGS: cannot enable LDAPS on Windows with cmake - To make it more obvious to users what doesn't work (yet) + Reported-by: Jack Boos Yu + Closes #6284 + +- KNOWN_BUGS: Excessive HTTP/2 packets with TCP_NODELAY - Closes #6389 + Reported-by: Alex Xu + Closes #6363 -- COPYING/configure: bump copyright year range +- http: use credentials from transfer, not connection + + HTTP auth "accidentally" worked before this cleanup since the code would + always overwrite the connection credentials with the credentials from + the most recent transfer and since HTTP auth is typically done first + thing, this has not been an issue. It was still wrong and subject to + possible race conditions or future breakage if the sequence of functions + would change. + + The data.set.str[] strings MUST remain unmodified exactly as set by the + user, and the credentials to use internally are instead set/updated in + state.aptr.* + + Added test 675 to verify different credentials used in two requests done + over a reused HTTP connection, which previously behaved wrongly. + + Fixes #6542 + Closes #6545 -- c-hyper: add timecondition to the request +- test433: clear some home dir env variables - Test 77-78 + Follow-up to bd6b54ba1f55b5 - Closes #6391 + ... so that XDG_CONFIG_HOME is the only home dir variable set and thus + used correctly in the test! + + Fixes #6599 + Closes #6600 -- c-hyper: make Digest and NTLM work +- RELEASE-NOTES: synced - Test 64, 65, 67, 68, 69, 70, 72 + bumped the version to 7.76.0 + +- travis: install libgsasl-dev to add that to the builds - Closes #6390 + Closes #6588 -- examples/curlgtk.c: fix the copyright year range +- urldata: don't touch data->set.httpversion at run-time - ... and make private functions static. + Rename it to 'httpwant' and make a cloned field in the state struct as + well for run-time updates. + + Also: refuse non-supported HTTP versions. Verified with test 129. + + Closes #6585 -- [Olaf Hering brought this change] +Viktor Szakats (11 Feb 2021) +- tests: disable .curlrc in more environments + + by also setting CURL_HOME and XDG_CONFIG_HOME envvars to the local + directory. + + Reviewed-by: Daniel Stenberg + Fixes #6595 + Closes #6596 - docs/examples: adjust prototypes for CURLOPT_READFUNCTION +- docs/Makefile.inc: format to be update-friendly - The type of the buffer in curl_read_callback is 'char *', not 'void *'. + - one source file per line + - convert tabs to spaces + - do not align line-continuation backslashes + - sort source files alphabetically - Signed-off-by: Olaf Hering - Closes #6392 + Reviewed-by: Daniel Stenberg + Closes #6593 -- examples: fix more empty expression statement has no effect +Daniel Stenberg (11 Feb 2021) +- curl: provide libgsasl version and feature info in -V output - Follow-up to 26e46617b9 + Closes #6592 -- cleanup: fix two empty expression statement has no effect +- gsasl: provide CURL_VERSION_GSASL if built-in - Follow-up to 26e46617b9 + To let applications know the feature is available. + + Closes #6592 -- configure: set -Wextra-semi-stmt for clang with --enable-debug +- curl: add --fail-with-body - To have it properly complain on empty statements with no effect. + Prevent both --fail and --fail-with-body on the same command line. - Ref: #6376 - Closes #6378 + Verify with test 349, 360 and 361. + + Closes #6449 -- tests/unit: fix empty statements with no effect +- TODO: remove HSTS - ... by making macros use "do {} while(0)" + Provided now since commit 7385610d0c74 -- [Paul Groke brought this change] +Jay Satiro (10 Feb 2021) +- tests: Fix tests failing due to change in curl --help + + Follow-up to parent 3183217 which added add missing argument to + --create-file-mode . + + Ref: https://github.com/curl/curl/issues/6590 - dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries +- tool_help: add missing argument for --create-file-mode - Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix - (similar to the existing '-' prefix for removing entries) to add - DNS cache entries that will time out just like entries that are added - by libcurl itself. + Prior to this change the required argument was not shown in curl --help. - Append " (non-permanent)" to info log message in case a non-permanent - entry is added. + before: + --create-file-mode File mode for created files - Adjust relevant comments to reflect the new behavior. + after: + --create-file-mode File mode (octal) for created files - Adjust documentation. + Reported-by: ZimCodes@users.noreply.github.com - Extend unit1607 to test the new functionality. + Fixes https://github.com/curl/curl/issues/6590 + +- create-file-mode.d: add missing Arg tag - Closes #6294 + Prior to this change the required argument was not shown. + + curl.1 before: --create-file-mode + curl.1 after: --create-file-mode + + Reported-by: ZimCodes@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/6590 -- schannel: fix "empty expression statement has no effect" +Viktor Szakats (10 Feb 2021) +- gsasl: fix errors/warnings building against libgsasl - Bug: https://github.com/curl/curl/commit/8ab78f720ae478d533e30b202baec4b451741579#commitcomment-45445950 - Reported-by: Gisle Vanem - Closes #6381 + - also fix an indentation + - make Curl_auth_gsasl_token() use CURLcode (by Daniel Stenberg) + + Ref: https://github.com/curl/curl/pull/6372#issuecomment-776118711 + Ref: https://github.com/curl/curl/pull/6588 + + Reviewed-by: Jay Satiro + Assisted-by: Daniel Stenberg + Reviewed-by: Simon Josefsson + Closes #6587 -- [Denis Laxalde brought this change] +- Makefile.m32: add support for libgsasl dependency + + Reviewed-by: Marcel Raad + Closes #6586 - docs: remove redundant "better" in --fail help +Marcel Raad (10 Feb 2021) +- ngtcp2: clarify calculation precedence - Closes #6385 + As suggested by Codacy/cppcheck. + + Closes https://github.com/curl/curl/pull/6576 -- [Kevin Ushey brought this change] +- server: remove redundant condition + + `end` is always non-null here. + + Closes https://github.com/curl/curl/pull/6576 - curl.1: fix typo microsft -> microsoft +- lib: remove redundant code - Closes #6380 + Closes https://github.com/curl/curl/pull/6576 -- [XhmikosR brought this change] +- mqttd: remove unused variable + + Closes https://github.com/curl/curl/pull/6576 - misc: assorted typo fixes +- tool_paramhlp: reduce variable scope - Closes #6375 + Closes https://github.com/curl/curl/pull/6576 -- RELEASE-NOTES: synced +- tests: reduce variable scopes + + Closes https://github.com/curl/curl/pull/6576 -- tool_operate: avoid NULL dereference of first_arg +- lib: reduce variable scopes - Follow-up to 6a5e020d4d2b04a - Identified by OSS-Fuzz - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28999 - Closes #6377 + Closes https://github.com/curl/curl/pull/6576 -- misc: fix "warning: empty expression statement has no effect" +- ftp: fix Codacy/cppcheck warning about null pointer arithmetic - Turned several macros into do-while(0) style to allow their use to work - find with semicolon. + Increment `bytes` only if it is non-null. - Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45433279 - Follow-up to 08e8455dddc5e4 - Reported-by: Gisle Vanem - Closes #6376 + Closes https://github.com/curl/curl/pull/6576 -- KNOWN_BUGS: 6.10 curl never completes Negotiate over HTTP +Daniel Stenberg (9 Feb 2021) +- ngtcp2: adapt to the new recv_datagram callback + +- quiche: fix build error: use 'int' for port number - Closes #5235 - Closes #6370 + Follow-up to cb2dc1ba8 -- writeout: fix NULL dereference for "this url" +- ftp: add 'list_only' to the transfer state struct - Detected by torture test 1029 + and rename it from 'ftp_list_only' since it is also used for SSH and + POP3. The state is updated internally for 'type=D' FTP URLs. - Follow-up to 7a90ddf88f5a + Added test case 1570 to verify. - Closes #6374 + Closes #6578 -- failf: remove newline from formatting strings +- ftp: add 'prefer_ascii' to the transfer state struct - ... as failf adds one itself. + ... and make sure the code never updates 'set.prefer_ascii' as it breaks + handle reuse which should use the setting as the user specified it. - Also: add an assert() to failf() that triggers on a newline in the - format string! + Added test 1569 to verify: it first makes an FTP transfer with ';type=A' + and then another without type on the same handle and the second should + then use binary. Previously, curl failed this. - Closes #6365 + Closes #6578 -- [XhmikosR brought this change] +- RELEASE-NOTES: synced - CI: fix warning with the latest versions +- [Jacob Hoffman-Andrews brought this change] + + vtls: initial implementation of rustls backend - `git checkout HEAD^2` is no longer needed + This adds a new TLS backend, rustls. It uses the C-to-rustls bindings + from https://github.com/abetterinternet/crustls. - Closes #6369 - -- INSTALL: update the list known OSes and CPU archs curl has run on + Rustls is at https://github.com/ctz/rustls/. - Closes #6366 + There is still a fair bit to be done, like sending CloseNotify on + connection shutdown, respecting CAPATH, and properly indicating features + like "supports TLS 1.3 ciphersuites." But it works well enough to make + requests and receive responses. + + Blog post for context: + https://www.abetterinternet.org/post/memory-safe-curl/ + + Closes #6350 -- [Cherish98 brought this change] +- [Simon Josefsson brought this change] - curl: fix handling of -q option - - The match of the "-q" option (short for "--disable") should: - a) allow concatenation with other single-letters; and - b) be case-sensitive, lest confusing with "-Q" ("--quote") + sasl: support SCRAM-SHA-1 and SCRAM-SHA-256 via libgsasl - Closes #6364 + Closes #6372 -- tests/badsymbols.pl: ignore stand-alone single hash lines +Jay Satiro (9 Feb 2021) +- lib: use int type for more port variables - Bug: https://curl.se/mail/lib-2020-12/0084.html - Reported-by: Dennis Clarke - Assisted-by: Jay Satiro + This is a follow-up to 764c6bd. Prior to that change port variables + were usually type long. - Closes #6355 + Closes https://github.com/curl/curl/pull/6553 -- curl_easy_pause.3: add multiplexed pause effects +- tool_writeout: refactor write-out and write-out json - and generally refresh and update. Remove details for ancient versions. + - Deduplicate the logic used by write-out and write-out json. - Reviewed-by: Jay Satiro - Closes #6360 - -Jay Satiro (22 Dec 2020) -- curl_easy_pause.3: fix man page reference + Rather than have separate writeLong, writeString, etc, logic for + each of write-out and write-out json instead have respective shared + functions that can output either format and a 'use_json' parameter to + indicate whether it is json that is output. - Follow-up to ac9a724 from earlier today. + This will make it easier to maintain. Rather than have to go through + two sets of logic now we only have to go through one. - Ref: https://github.com/curl/curl/pull/6359 - -Daniel Stenberg (22 Dec 2020) -- EXPERIMENTAL: add the Hyper backend to the list + - Support write-out %{errormsg} and %{exitcode} in json. - ... of current experimental features in curl. + - Clarify in the doc that %{exitcode} is the exit code of the transfer. + + Prior to this change it just said "The numerical exitcode" which + implies it's the exit code of the tool, and it's not necessarily that. + + Closes https://github.com/curl/curl/pull/6544 -- speedcheck: exclude paused transfers +- lib: drop USE_SOCKETPAIR in favor of CURL_DISABLE_SOCKETPAIR - Paused transfers should not be stopped due to slow speed even when - CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is - now reset when the transfer is unpaused - as otherwise it would easily - just trigger immediately after unpausing. + .. since the former is undocumented and they both do the same thing. - Reported-by: Harry Sintonen - Fixes #6358 - Closes #6359 + Closes https://github.com/curl/curl/pull/6517 -- h2: do not wait for RECV on paused transfers +- curl_multibyte: fall back to local code page stat/access on Windows - ... as the socket might be readable all the time when paused and thus - causing a busy-loop. + If libcurl is built with Unicode support for Windows then it is assumed + the filename string is Unicode in UTF-8 encoding and it is converted to + UTF-16 to be passed to the wide character version of the respective + function (eg wstat). However the filename string may actually be in the + local encoding so, even if it successfully converted to UTF-16, if it + could not be stat/accessed then try again using the local code page + version of the function (eg wstat fails try stat). - Reported-by: Harry Sintonen - Reviewed-by: Jay Satiro - Fixes #6356 - Closes #6357 + We already do this with fopen (ie wfopen fails try fopen), so I think it + makes sense to extend it to stat and access functions. + + Closes https://github.com/curl/curl/pull/6514 -- RELEASE-NOTES: synced +- [Stephan Szabo brought this change] -- cmdline-opts/gen.pl: return hard on errors - - ... as the warnings tend to go unnoticed otherwise! + file: Support unicode urls on windows - Closes #6354 + Closes https://github.com/curl/curl/pull/6501 -- examples/libtest: add .checksrc to dist - - ... so that (auto)builds from tarballs also get the correct instructions. - - Fixes #6176 - Closes #6353 +- [Vincent Torri brought this change] -- test: verify new --write-out variables + cmake: fix import library name for non-MS compiler on Windows - Extended test 1029 and added 1188 - -- test970: adapted to the new internal order of variables - -- curl: add variables to --write-out + - Use _imp.lib suffix only for Microsoft's compiler (MSVC). - In particular, these ones can help a user to create its own error - message when one or transfers fail. + Prior to this change library suffix _imp.lib was used for the import + library on Windows regardless of compiler. - writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg' + With this change the other compilers should now use their default + suffix which should be .dll.a. - onerror - lets a user only show the rest on non-zero exit codes + This change is motivated by the usage of pkg-config on MSYS2. + Indeed, when 'pkg-config --libs libcurl' is used, -lcurl is + passed to ld. The documentation of ld on Windows : - url - the input URL used for this transfer + https://sourceware.org/binutils/docs/ld/WIN32.html - urlnum - the numerical URL counter (0 indexed) for this transfer + lists, in the 'direct linking to a dll' section, the pattern + of the searched import library, and libcurl_imp.lib is not there. - exitcode - the numerical exit code for the transfer + Closes https://github.com/curl/curl/pull/6225 + +Daniel Stenberg (9 Feb 2021) +- urldata: move 'followlocation' to UrlState - errormsg - obvious + As this is a state variable it does not belong in UserDefined which is + used to store values set by the user. - Reported-by: Earnestly on github - Fixes #6199 - Closes #6207 + Closes #6582 -- [Matthias Gatto brought this change] +- [Ikko Ashimine brought this change] - tests: add very simple AWS HTTP v4 Signature test + http_proxy: fix typo in http_proxy.c - Signed-off-by: Matthias Gatto + settting -> setting + + Closes #6583 -- [Matthias Gatto brought this change] +- [Fabian Keil brought this change] - docs: add AWS HTTP v4 Signature + tests/server: Bump MAX_TAG_LEN to 200 + + This is useful for tests containing HTML inside of sections. + For tags it's not uncommon to be longer than the previous + limit of 79 bytes. + + An example of a previously problem-causing tag is: + + which is needed for a Privoxy test for the banners-by-size filter. + + Previously it caused server failures like: + 12:29:05.786961 ====> Client connect + 12:29:05.787116 accept_connection 3 returned 4 + 12:29:05.787194 accept_connection 3 returned 0 + 12:29:05.787285 Read 119 bytes + 12:29:05.787345 Process 119 bytes request + 12:29:05.787407 Got request: GET /banners-by-size/9 HTTP/1.1 + 12:29:05.787464 Requested test number 9 part 0 + 12:29:05.787686 getpart() failed with error: -2 + 12:29:05.787744 - request found to be complete (9) + 12:29:05.787912 getpart() failed with error: -2 + 12:29:05.788048 Wrote request (119 bytes) input to log/server.input + 12:29:05.788157 Send response test9 section + 12:29:05.788443 getpart() failed with error: -2 + 12:29:05.788498 instructed to close connection after server-reply + 12:29:05.788550 ====> Client disconnect 0 + 12:29:05.871448 exit_signal_handler: 15 + 12:29:05.871714 signalled to die + 12:29:05.872040 ========> IPv4 sws (port 21108 pid: 51758) exits with signal (15) -- [Matthias Gatto brought this change] +- [Fabian Keil brought this change] - tool: add AWS HTTP v4 Signature support - - Signed-off-by: Matthias Gatto + tests/badsymbols.pl: when opening '$incdir' fails include it in the error message -- [Matthias Gatto brought this change] +- [Fabian Keil brought this change] - http: Make the call to v4 signature - - This patch allow to call the v4 signature introduce in previous commit - - Signed-off-by: Matthias Gatto + runtests.1: document -o, -P, -L, and -E -- [Matthias Gatto brought this change] +- [Fabian Keil brought this change] - http: introduce AWS HTTP v4 Signature - - It is a security process for HTTP. - - It doesn't seems to be standard, but it is used by some cloud providers. + runtests.pl: add %TESTNUMBER variable to make copying tests more convenient + +- [Fabian Keil brought this change] + + runtests.pl: add an -o option to change internal variables - Aws: - https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html - Outscale: - https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request - GCP (I didn't test that this code work with GCP though): - https://cloud.google.com/storage/docs/access-control/signing-urls-manually + runtests.pl has lots of internal variables one might want to + change in certain situations, but adding a dedicated option + for every single one of them isn't practical. - most of the code is in lib/http_v4_signature.c + Usage: + ./runtests.pl -o TESTDIR=$privoxy_curl_test_dir -o HOSTIP=10.0.0.1 ... + +- [Fabian Keil brought this change] + + runtests.pl: cleanups - Information require by the algorithm: - - The URL - - Current time - - some prefix that are append to some of the signature parameters. + - show the summarized test result in the last line of the report + - do not use $_ after mapping it to a named variable + Doing that makes the code harder to follow. + - log the restraints sorted by the number of their occurrences + - fix language when logging restraints that only occured once + - let runhttpserver() use $TESTDIR instead of $srcdir + ... so it works if a non-default $TESTDIR is being used. + +- [Fabian Keil brought this change] + + runtests.pl: add an -E option to specify an exclude file - The data extracted from the URL are: the URI, the region, - the host and the API type + It can contain additional restraints for test numbers, + keywords and tools. - example: - https://api.eu-west-2.outscale.com/api/latest/ReadNets - ~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ - ^ ^ ^ - / \ URI - API type region + The idea is to let third parties like the Privoxy project + distribute an exclude file with their tarballs that specifies + which curl tests are not expected to work when using Privoxy + as a proxy, without having to fork the whole curl test suite. - Small description of the algorithm: - - make canonical header using content type, the host, and the date - - hash the post data - - make canonical_request using custom request, the URI, - the get data, the canonical header, the signed header - and post data hash - - hash canonical_request - - make str_to_sign using one of the prefix pass in parameter, - the date, the credential scope and the canonical_request hash - - compute hmac from date, using secret key as key. - - compute hmac from region, using above hmac as key - - compute hmac from api_type, using above hmac as key - - compute hmac from request_type, using above hmac as key - - compute hmac from str_to_sign using above hmac as key - - create Authorization header using above hmac, prefix pass in parameter, - the date, and above hash + The syntax could be changed to be extendable and maybe + more closely reflect the "curl test" syntax. Currently + it's a bunch of lines like these: - Signed-off-by: Matthias Gatto + test:$TESTNUMBER:Reason why this test with number $TESTNUMBER should be skipped + keyword:$KEYWORD:Reason why tests whose keywords contain the $KEYWORD should be skipped + tool:$TOOL:Reason why tests with tools that contain $TOOL should be skipped - Closes #5703 + To specify multiple $TESTNUMBERs, $KEYWORDs and $TOOLs + on a single line, split them with commas. -- [Matthias Gatto brought this change] +- [Fabian Keil brought this change] - http: add hmac support for sha256 - - It seems current hmac implementation use md5 for the hash, - V4 signature require sha256, so I've added the needed struct in - this commit. + runtests.pl: add -L parameter to require additional perl libraries - I've added the functions that do the hmac in v4 signature file - as a static function ,in the next patch of the serie, - because it's used only by this file. + This is useful to change the behaviour of the script without + having to modify the file itself, for example to use a custom + compareparts() function that ignores header differences that + are expected to occur when an external proxy is being used. - Signed-off-by: Matthias Gatto + Such differences are proxy-specific and thus the modifications + should be maintained together with the proxy. -- [Cristian Rodríguez brought this change] +- [Fabian Keil brought this change] - connect: on linux, enable reporting of all ICMP errors on UDP sockets + runtests.pl: add a -P option to specify an external proxy - The linux kernel does not report all ICMP errors back to userspace due - to historical reasons. + ... that should be used when executing the tests. - IP*_RECVERR sockopt must be turned on to have the correct behaviour - which is to pass all ICMP errors to userspace. + The assumption is that the proxy is an HTTP proxy. - See https://bugzilla.kernel.org/show_bug.cgi?id=202355 + This option should be used together with -L to provide + a customized compareparts() version that knows which + proxy-specific header differences should be ignored. - Closes #6341 + This option doesn't work for all test types yet. -- curl: add --create-file-mode [mode] - - This option sets the (octal) mode to use for the remote file when one is - created, using the SFTP, SCP or FILE protocols. When not set, the - default is 0644. - - Closes #6244 +- [Fabian Keil brought this change] -- c-hyper: fix compiler warnings - - Identified by clang on windows. - - Reported-by: Gisle Vanem - Bug: 58974d25d8173aec154e593ed9d866da566c9811 + tests: fixup several tests - Closes #6351 - -- KNOWN_BUGS: Remote recursive folder creation with SFTP + missing CRs and modified %hostip - Closes #5204 - -Jay Satiro (20 Dec 2020) -- badsymbols.pl: Add verbose mode -v + lib556/test556: use a real HTTP version to make test reuse more convenient - Use -v as the first option to enable verbose mode which will show source - input, extracted symbol and line info. For example: + make sure the weekday in Date headers matches the date - Source: ./../include/curl/typecheck-gcc.h - Symbol: curlcheck_socket_info(info) - Line #423: #define curlcheck_socket_info(info) \ + test61: replace stray "^M" (5e 4d) at the end of a cookie with a '^M' (0d) - Ref: https://curl.se/mail/lib-2020-12/0084.html + Gets the test working with external proxies like Privoxy again. - Closes https://github.com/curl/curl/pull/6349 + Closes #6463 -- KNOWN_BUGS: Secure Transport disabling hostname validation also disables SNI +- ftp: never set data->set.ftp_append outside setopt - That behavior is a limitation of Apple's Secure Transport. - - Reported-by: Cory Benfield - Reported-by: Ian Spence - Confirmed-by: Nick Zitzmann - - Ref: https://github.com/curl/curl/issues/998 + Since the set value then risks getting used like that when the easy + handle is reused by the application. - Closes https://github.com/curl/curl/issues/6347 - Closes https://github.com/curl/curl/pull/6348 - -Daniel Stenberg (18 Dec 2020) -- TODO: alt-svc should fallback if alt-svc doesn't work + Also: renamed the struct field from 'ftp_append' to 'remote_append' + since it is also used for SSH protocols. - Closes #4908 + Closes #6579 -- travis: restrict the openssl3 job to only run https and ftps tests +- urldata: remove the 'rtspversion' field - ... as it runs too long otherwise and the other tests are verified in - other builds anyway. + from struct connectdata and the corresponding code in http.c that set + it. It was never used for anything! - Closes #6345 + Closes #6581 -- build: repair http disabled but mqtt enabled build +- CURLOPT_QUOTE.3: clarify that libcurl doesn't parse what's sent - ... as the mqtt code reuses the "method" originally used for HTTP. + ... so passed in commands may confuse libcurl's knowledge of state. - Closes #6344 + Reported-by: Bodo Bergmann + Fixes #6577 + Closes #6580 -- [Jon Wilkes brought this change] +- [Jacob Hoffman-Andrews brought this change] - cookie: avoid the C1001 internal compiler error with MSVC 14 + vtls: factor out Curl_ssl_getsock to field of Curl_ssl - Fixes #6112 - Closes #6135 + Closes #6558 -- RELEASE-NOTES: synced +- RELEASE-PROCEDURE: remove old release dates, add new -- mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE - - Detected by OSS-Fuzz - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735 +- docs/SSL-PROBLEMS: enhanced - Added test 1916 and 1917 to verify. + Elaborate on the intermediate cert issue, and mention that anything + below TLS 1.2 is generally considered insecure these days. - Closes #6338 + Closes #6572 -- travis: add CI job for Hyper build +- THANKS: remove a Jon Rumsey dupe -- tests: updated tests for Hyper +Daniel Gustafsson (5 Feb 2021) +- [nimaje brought this change] -- lib: introduce c-hyper for using Hyper + docs: fix FILE example url in --metalink documentation - ... as an alternative HTTP backend within libcurl. - -- tool_setopt: provide helper output in debug builds + In a url after :// follows the possibly empty authority part + till the next /, so that url missed a /. - ... for when setopt() returns error. - -- setopt: adjust to Hyper and disabled HTTP builds - -- rtsp: disable if Hyper is used + Closes #6573 + Reviewed-by: Daniel Stenberg + Reviewed-by: Daniel Gustafsson -- getinfo: build with disabled HTTP support +Daniel Stenberg (5 Feb 2021) +- hostip: fix build with sync resolver + + Reported-by: David Goerger + Follow-up from 8335c6417 + Fixes #6566 + Closes #6568 -- version: include hyper version +- mailmap: Jon Rumsey -- docs: add HYPER.md +- [Jon Rumsey brought this change] -- configure: add --with-hyper + gskit: correct the gskit_send() prototype - As the first (optional) HTTP backend alternative instead of native + gskit_send() first paramater is a pointer to Curl_easy not connectdata + struct. - Close #6110 + Closes #6570 + Fixes #6569 -- test1522: add debug tracing - - I used this to track down some issues and I figured I could just as well - keep this extra logging in here for future needs. +- urldata: fix build without HTTP and MQTT - Closes #6331 + Reported-by: Joseph Chen + Fixes #6562 + Closes #6563 -- http: show the request as headers even when split-sending +- ftp: avoid SIZE when asking for a TYPE A file - When the initial request isn't possible to send in its entirety, the - remainder of request would be delivered to the debug callback as data - and would wrongly be counted internally as body-bytes sent. + ... as we ignore it anyway because servers don't report the correct size + and proftpd even blatantly returns a 550. - Extended test 1295 to verify. + Updates a set of tests accordingly. - Closes #6328 + Reported-by: awesomenode on github + Fixes #6564 + Closes #6565 -- multi: when erroring in TOOFAST state, act as for PERFORM +- pingpong: rename the curl_pp_transfer enum to use PP prefix - When failing in TOOFAST, the multi_done() wasn't called so the same - cleanup and handling wasn't done like when it fails in PERFORM, which in - the case of FTP could mean that the control connection wouldn't be - marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused - ftp_disconnect() to use it to send "QUIT", which could end up waiting - for a response a long time before giving up! - - Reported-by: Tomas Berger - Fixes #6333 - Closes #6337 + Using an FTP prefix for PP provided functionality was misleading. -- cmake: enable gophers correctly in curl-config +- RELEASE-NOTES: synced - Closes #6336 + ... and bump pending version to 7.75.1 (for now) -- test1198/9: add two mqtt publish tests without payload lengths +Jay Satiro (4 Feb 2021) +- build: fix --disable-http-auth - Closes #6335 - -- tests/mqttd: extract the client id from the correct offset + Broken since 215db08 (precedes 7.75.0). - Closes #6334 - -- TODO: Prevent terminal injection when writing to terminal + Reported-by: Benbuck Nason - Closes #6150 + Fixes https://github.com/curl/curl/issues/6567 -- Revert "CI/github: work-around for brew breakage on macOS" - - This reverts commit 4cbb17a2cbbbe6337142d39479e21c3990b9c22f. +- build: fix --disable-dateparse - ... as the work-around now causes failures. - - Closes #6332 - -- examples: remove superfluous asterisk uses + Broken since 215db08 (precedes 7.75.0). - ... for function pointers. Breaks in ancient compilers. + Bug: https://curl.se/mail/lib-2021-02/0008.html + Reported-by: Firefox OS -- RELEASE-NOTES: synced +Daniel Stenberg (4 Feb 2021) +- [Jon Rumsey brought this change] -- test1272: fix line ending + OS400: update for CURLOPT_AWS_SIGV4 - Follow-up to f24784f9143 - -- URL-SYNTAX: add gophers details + chkstrings fails because a new string option that could require codepage + conversion has been added. + + Closes #6561 + Fixes #6560 -- test1272: test gophers +- BUG-BOUNTY: removed the cooperation mention -- runtests: add support for gophers, gopher over TLS +Version 7.75.0 (3 Feb 2021) -- [parazyd brought this change] +Daniel Stenberg (3 Feb 2021) +- RELEASE-NOTES: synced - gopher: Implement secure gopher protocol. - - This commit introduces a "gophers" handler inside the gopher protocol if - USE_SSL is defined. This protocol is no different than the usual gopher - prococol, with the added TLS encapsulation upon connecting. The protocol - has been adopted in the gopher community, and many people have enabled - TLS in their gopher daemons like geomyidae(8), and clients, like clic(1) - and hurl(1). - - I have not implemented test units for this protocol because my knowledge - of Perl is sub-par. However, for someone more knowledgeable it might be - fairly trivial, because the same test that tests the plain gopher - protocol can be used for "gophers" just by adding a TLS listener. - - Signed-off-by: parazyd - - Closes #6208 +- THANKS: added contributors from 7.75.0 -- TODO: Package curl for Windows in a signed installer +- copyright: fix year ranges in need of updates + +- TODO: remove items for next SONAME bump etc - Closes #5424 + We want to avoid that completely, so we don't plan for things after such + an event. -- mqtt: deal with 0 byte reads correctly +- [Jay Satiro brought this change] + + ngtcp2: Fix build error due to change in ngtcp2_settings - OSS-Fuzz found it - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676 + - Separate ngtcp2_transport_params. - Closes #6327 - -- BUG-BOUNTY: minor language update + ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from + ngtcp2_settings. - ... and remove the wording about entries from before 2019 as the "within - 12 months" is still there and covers that. + ngtcp2 master is required to build curl with http3 support. - Closes #6318 + Closes #6554 -- tooĺ_writeout: fix the -w time output units +- vtls: remove md5sum - Fix regression from commit fc813f80e1bcac (#6248) that changed the unit - to microseconds instead of seconds with fractions + As it is not used anymore. - Reported-by: 不确定 - Fixes #6321 - Closes #6322 + Reported-by: Jacob Hoffman-Andrews + Bug: https://curl.se/mail/lib-2021-02/0000.html + + Closes #6557 -- quiche: remove fprintf() leftover +- [Alessandro Ghedini brought this change] -Jay Satiro (14 Dec 2020) -- KNOWN_BUGS: SHA-256 digest not supported in Windows SSPI builds + quiche: don't use primary_ip / primary_port - Closes https://github.com/curl/curl/issues/6302 + Closes #6555 -- digest_sspi: Show InitializeSecurityContext errors in verbose mode - - The error is shown with infof rather than failf so that the user will - see the extended error message information only in verbose mode, and - will still see the standard CURLE_AUTH_ERROR message. For example: - - --- - - * schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED - (0x8009030A) - The per-message Quality of Protection is not supported by - the security package - * multi_done - * Connection #1 to host 127.0.0.1 left intact - curl: (94) An authentication function returned an error - - --- +Alessandro Ghedini (1 Feb 2021) +- travis: enable quiche's FFI feature + +Daniel Stenberg (30 Jan 2021) +- [Dmitry Wagin brought this change] + + http: improve AWS HTTP v4 Signature auth - Ref: https://github.com/curl/curl/issues/6302 + - Add support services without region and service prefixes in + the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc) + by providing region and service parameters via aws-sigv4 option. + - Add [:region[:service]] suffix to aws-sigv4 option; + - Fix memory allocation errors. + - Refactor memory management. + - Use Curl_http_method instead() STRING_CUSTOMREQUEST. + - Refactor canonical headers generating. + - Remove repeated sha256_to_hex() usage. + - Add some docs fixes. + - Add some codestyle fixes. + - Add overloaded strndup() for debug - curl_dbg_strndup(). + - Update tests. - Closes https://github.com/curl/curl/pull/6315 + Closes #6524 -Daniel Stenberg (13 Dec 2020) -- URL-SYNTAX: add default port numbers and IDNA details +- hyper: fix CONNECT to set 'data' as userdata - Closes #6316 + Follow-up to 14e075d1a7fd -- URL-SYNTAX: mention how FILE:// access can access network on windows - - Closes #6314 +- [Layla brought this change] -Jay Satiro (12 Dec 2020) -- URL-SYNTAX: Document default SMTP port 25 - - Note that ports 25 and 587 are common ports for smtp, the former being - the default. + connect: fix compile errors in `Curl_conninfo_local` - Closes https://github.com/curl/curl/pull/6310 - -Daniel Stenberg (12 Dec 2020) -- CURLOPT_URL.3: remove scheme specific details + .. for the `#else` (`!HAVE_GETSOCKNAME`) case - ... that are now found in URL-SYNTAX.md + Fixes https://github.com/curl/curl/issues/6548 + Closes #6549 - Closes #6307 + Signed-off-by: Layla -Dan Fandrich (12 Dec 2020) -- docs: Fix some typos - - [skip ci] +- [Michał Antoniak brought this change] -Daniel Stenberg (12 Dec 2020) -- URL-SYNTAX: mention all supported schemes + transfer: fix GCC 10 warning with flag '-Wint-in-bool-context' - Closes #6311 + ... and return the error code from the Curl_mime_rewind call. + + Closes #6537 -- [Douglas R. Reno brought this change] +- [Michał Antoniak brought this change] - URL-SYNTAX.md: minor language improvements - - Closes #6308 + avoid warning: enum constant in boolean context -- docs/URL-SYNTAX: the URL syntax curl accepts and works with - - Closes #6285 +- copyright: fix missing year (range) updates -- [0xflotus brought this change] +- RELEASE-NOTES: synced - docs: enable syntax highlighting in several docs files +- openssl: lowercase the hostname before using it for SNI - ... for better readability + ... because it turns out several servers out there don't actually behave + correctly otherwise in spite of the fact that the SNI field is + specifically said to be case insensitive in RFC 6066 section 3. - Closes #6286 + Reported-by: David Earl + Fixes #6540 + Closes #6543 -- test1564/1565: require the 'wakeup' feature to run +- KNOWN_BUGS: cmake: ExternalProject_Add does not set CURL_CA_PATH - Fixes #6299 - Fixes #6300 - Closes #6301 - -- runtests: add 'wakeup' as a feature + Closes #6313 -- tests/server/disabled: add "wakeup" +- KNOWN_BUGS: Multi perform hangs waiting for threaded resolver - To allow the test suite to know if wakeup support is disabled in the - build. - -- lib1564/5: verify that curl_multi_wakeup returns OK + Closes #4852 -- tests: make --libcurl tests only test FTP options if ftp enabled +- KNOWN_BUGS: "pulseUI VPN client" is known to be buggy - Adjust six --libcurl tests to only check the FTP option if FTP is - actually present in the build. + First entry in the new section "applications" for known problems in + libcurl using applications. - Fixes #6303 - Closes #6305 + Closes #6306 -- runtests.pl: fix "uninitialized value" warning +- tool_writeout: make %{errormsg} blank for no errors - follow-up to e12825c642a88774 + Closes #6539 -- runtests: add support for %if [feature] conditions +Jay Satiro (27 Jan 2021) +- [Gisle Vanem brought this change] + + build: fix djgpp builds - ... to make tests run differently or expect different results depending - on what features that are present or not in curl. + - Update build instructions in packages/DOS/README - Bonus: initial minor 'Hyper' awareness but nothing is using that yet + - Extend 'VPATH' with 'vquic' and 'vssh'. - Closes #6304 - -- [Jon Rumsey brought this change] - - OS400: update ccsidcurl.c + - Allow 'Makefile.dist' to build both 'lib' and 'src'. - Add 'struct' to cast and declaration of cfcdata to fix compilation - error. + - Allow using the Windows hosted djgpp cross compiler to build for MSDOS + under Windows. - Fixes #6292 - Closes #6297 - -- ngtcp2: make it build it current master again + - 'USE_SSL' -> 'USE_OPENSSL' - Closes #6296 - -- [Cristian Rodríguez brought this change] - - connect: defer port selection until connect() time + - Added a 'link_EXE' macro. Etc, etc. - If supported, defer port selection until connect() time - if --interface is given and source port is 0. + - Linking 'curl.exe' needs '$(CURLX_CFILES)' too. - Reproducer: + - Do not pick-up '../lib/djgpp/*.o' files. Recompile locally. - * start fast webserver on port 80 - * starve system of ephemeral ports - $ sysctl net.ipv4.ip_local_port_range="60990 60999" + - Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'. - * start a curl/libcurl "crawler" - $curl --keepalive --parallel --parallel-immediate --head --interface - 127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt" + - Remove 'djgpp-clean' - current result: - (possible some successful data) - curl: (45) bind failed with errno 98: Address already in use + - Adapt to new C-ares directory structure - result after patch: - (complete success or few connections failing, higlhy depending on load) + - Use conditional variable assignments - Fail only when all the possible 4-tuple combinations are exhausted, - which is impossible to do when port is selected at bind() time becuse - the kernel does not know if socket will be listen()'ed on or connect'ed - yet. + Clarify the 'conditional variable assignment' in 'common.dj'. - Closes #6295 + Closes https://github.com/curl/curl/pull/6382 -- [Hans-Christian Noren Egtvedt brought this change] +Daniel Stenberg (27 Jan 2021) +- [Ikko Ashimine brought this change] - connect: zero variable on stack to silence valgrind complaint + hyper: fix typo in c-hyper.c - Valgrind will complain that ssrem buffer usage if not explicit - initialized, hence initialize it to zero. + settting -> setting - This completes the change intially started in commit 2c0d7212151 ('ftp: - retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has - a similar memset to zero. - - Signed-off-by: Hans-Christian Noren Egtvedt - Closes #6289 + Closes #6538 -- RELEASE-NOTES: synced +- libssh2: fix CURL_LIBSSH2_DEBUG-enabled build - start over on the next release cycle - -Version 7.74.0 (9 Dec 2020) - -Daniel Stenberg (9 Dec 2020) -- RELEASE-NOTES: synced + Follow-up to 2dcc940959772a - for 7.74.0 - -Jay Satiro (7 Dec 2020) -- [Jacob Hoffman-Andrews brought this change] + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/2dcc940959772a652f6813fb6bd3092095a4877b#commitcomment-46420088 - urldata: restore comment on ssl_connect_data.use - - This comment was originally on the `use` field, but was separated from - its field in 62a2534. +Jay Satiro (27 Jan 2021) +- asyn-thread: fix build for when getaddrinfo missing - Closes https://github.com/curl/curl/pull/6287 - -Daniel Stenberg (7 Dec 2020) -- VERSIONS: refreshed + This is a follow-up to 8315343 which several days ago moved the resolver + pointer into the async struct but did not update the code that uses it + when getaddrinfo is not present. - We always use the patch number these days: all releases are - "major.minor.patch" - -- [Jakub Zakrzewski brought this change] + Closes https://github.com/curl/curl/pull/6536 - cmake: don't use reserved target name 'test' +Daniel Stenberg (27 Jan 2021) +- urldata: move 'ints' to the end of 'connectdata' - CMake up to 3.10 always reserves this name + To optimize storage slightly. - Fixes #6257 - Closes #6258 + Closes #6534 -- openssl: make the OCSP verification verify the certificate id +- urldata: store ip version in a single byte - CVE-2020-8286 + Closes #6534 + +- urldata: remove duplicate 'upkeep_interval_ms' from connectdata - Reported by anonymous + ... and rely only on the value already set in Curl_easy. - Bug: https://curl.se/docs/CVE-2020-8286.html + Closes #6534 -- ftp: make wc_statemach loop instead of recurse +- urldata: remove 'local_ip' from the connectdata struct - CVE-2020-8285 + As the info is already stored in the transfer handle anyway, there's no + need to carry around a duplicate buffer for the life-time of the handle. - Fixes #6255 - Bug: https://curl.se/docs/CVE-2020-8285.html - Reported-by: xnynx on github + Closes #6534 -- ftp: CURLOPT_FTP_SKIP_PASV_IP by default +- urldata: remove duplicate port number storage - The command line tool also independently sets --ftp-skip-pasv-ip by - default. + ... and use 'int' for ports. We don't use 'unsigned short' since -1 is + still often used internally to signify "unknown value" and 0 - 65535 are + all valid port numbers. - Ten test cases updated to adapt the modified --libcurl output. + Closes #6534 + +- urldata: remove the duplicate 'ip_addr_str' field - Bug: https://curl.se/docs/CVE-2020-8284.html - CVE-2020-8284 + ... as the numerical IP address is already stored and kept in 'primary_ip'. - Reported-by: Varnavas Papaioannou + Closes #6534 -- urlapi: don't accept blank port number field without scheme +- select: convert Curl_select() to private static function - ... as it makes the URL parser accept "very-long-hostname://" as a valid - host name and we don't want that. The parser now only accepts a blank - (no digits) after the colon if the URL starts with a scheme. + The old function should not be used anywhere anymore (the only remaining + gskit use has to be fixed to instead use Curl_poll or none at all). - Reported-by: d4d on hackerone + The static function version is now called our_select() and is only built + if necessary. - Closes #6283 + Closes #6531 -- Revert "multi: implement wait using winsock events" - - This reverts commit d2a7d7c185f98df8f3e585e5620cbc0482e45fac. +- Curl_chunker: shrink the struct - This commit also reverts the subsequent follow-ups to that commit, which - were all done within windows #ifdefs that are removed in this - change. Marc helped me verify this. + ... by removing a field, converting the hex index into a byte and + rearranging the order. Cuts it down from 48 bytes to 32 on x86_64. - Fixes #6146 - Closes #6281 + Closes #6527 -- [Klaus Crusius brought this change] +- curl: include the file name in --xattr/--remote-time error msgs - ftp: retry getpeername for FTP with TCP_FASTOPEN - - In the case of TFO, the remote host name is not resolved at the - connetion time. +- curl: s/config->global/global/ in single_transfer() + +- curl: move fprintf outputs to warnf - For FTP that has lead to missing hostname for the secondary connection. - Therefore the name resolution is done at the time, when FTP requires it. + For setting and getting time of the download. To make the outputs + respect --silent etc. - Fixes #6252 - Closes #6265 - Closes #6282 + Reported-by: Viktor Szakats + Fixes #6533 + Closes #6535 -- [Thomas Danielsson brought this change] +- [Tatsuhiro Tsujikawa brought this change] - scripts/completion.pl: parse all opts - - For tab-completion it may be preferable to include all the - available options. + ngtcp2: Fix http3 upload stall - Closes #6280 + Closes #6521 -- RELEASE-NOTES: synced +- [Tatsuhiro Tsujikawa brought this change] -- openssl: use OPENSSL_init_ssl() with >= 1.1.0 + ngtcp2: Fix stack buffer overflow - Reported-by: Kovalkov Dmitrii and Per Nilsson - Fixes #6254 - Fixes #6256 - Closes #6260 + Closes #6521 -- SECURITY-PROCESS: disclose on hackerone - - Once a vulnerability has been published, the hackerone issue should be - disclosed. For tranparency. +- warnless.h: remove the prototype for curlx_ultosi - Closes #6275 + Follow-up to 217552503ff3 -Marc Hoersken (3 Dec 2020) -- tests/util.py: fix compatibility with Python 2 - - Backporting the Python 3 implementation of setStream - to ClosingFileHandler as a fallback within Python 2. +- warnless: remove curlx_ultosi - Reported-by: Jay Satiro + ... not used anywhere - Fixes #6259 - Closes #6270 + Closes #6530 -Daniel Gustafsson (3 Dec 2020) -- docs: fix typos and markup in ETag manpage sections - - Reported-by: emanruse on github - Fixes #6273 +- [Patrick Monnerat brought this change] -Daniel Stenberg (2 Dec 2020) -- quiche: close the connection + lib: remove conn->data uses - Reported-by: Junho Choi - Fixes #6213 - Closes #6217 + Closes #6515 -Jay Satiro (2 Dec 2020) -- ngtcp2: Fix build error due to symbol name change +- pingpong: remove the 'conn' struct member - - NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION + ... as it's superfluous now when Curl_easy is passed in and we can + derive the connection from that instead and avoid the duplicate copy. - ngtcp2/ngtcp2@76232e9 changed the name. + Closes #6525 + +- hostip/proxy: remove conn->data use - ngtcp2 master is required to build curl with http3 support. + Closes #6513 + +- url: reduce conn->data references - Closes https://github.com/curl/curl/pull/6271 + ... there are a few left but let's keep them to last + + Closes #6512 -Daniel Stenberg (1 Dec 2020) -- [Klaus Crusius brought this change] +- scripts/singleuse: add curl_easy_option* - cmake: check for linux/tcp.h +Jay Satiro (25 Jan 2021) +- test410: fix for windows - The HAVE_LINUX_TCP_H define was not set by cmake. + - Pass the very long request header via file instead of command line. - Closes #6252 + Prior to this change the 49k very long request header string was passed + via command line and on Windows that is too long so it was truncated and + the test would fail (specifically msys CI). + + Closes https://github.com/curl/curl/pull/6516 -- NEW-PROTOCOL: document what needs to be done to add one +Daniel Stenberg (25 Jan 2021) +- libssh2: move data from connection object to transfer object - Closes #6263 + Readdir data, filenames and attributes are strictly related to the + transfer and not the connection. This also reduces the total size of the + fixed connectdata struct. + + Closes #6519 -- splay: rename Curl_splayremovebyaddr to Curl_splayremove +- RELEASE-NOTES: synced + +- [Patrick Monnerat brought this change] + + lib: remove conn->data uses - ... and remove the old unused proto for the old Curl_splayremove - version. + Closes #6499 + +- hyper: remove the conn->data references - Closes #6269 + Closes #6508 -- openssl: free mem_buf in error path +- travis: build ngtcp2 --with-gnutls - To fix a memory-leak. + ... since they disable it by default since a few days back. - Closes #6267 + Closes #6506 + Fixes #6493 -- openssl: remove #if 0 leftover +- hostip: remove conn->data from resolver functions - Follow-up to 4c9768565ec3a9 (from Sep 2008) + This also moves the 'async' struct from the connectdata struct into the + Curl_easy struct, which seems like a better home for it. - Closes #6268 + Closes #6497 -- ntlm: avoid malloc(0) on zero length user and domain +Jay Satiro (22 Jan 2021) +- strerror: skip errnum >= 0 assertion on windows - ... and simplify the too-long checks somewhat. + On Windows an error number may be greater than INT_MAX and negative once + cast to int. - Detected by OSS-Fuzz + The assertion is checked only in debug builds. - Closes #6264 - -- RELEASE-NOTES: synced + Closes https://github.com/curl/curl/pull/6504 -Marc Hoersken (28 Nov 2020) -- tests/server/tftpd.c: close upload file in case of abort - - Commit c353207 removed the closing right after do_tftp - which covered the case of abort. This handles that case. +Daniel Stenberg (21 Jan 2021) +- doh: make Curl_doh_is_resolved survive a NULL pointer - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ... if Curl_doh() returned a NULL, this function gets called anyway as + in a asynch procedure. Then the doh struct pointer is NULL and signifies + an OOM situation. - Follow up to #6209 - Closes #6234 - -Daniel Stenberg (26 Nov 2020) -- [Daiki Ueno brought this change] + Follow-up to 6246a1d8c6776 - ngtcp2: use the minimal version of QUIC supported by ngtcp2 +- wolfssh: remove conn->data references - Closes #6250 - -- [Daiki Ueno brought this change] - - ngtcp2: advertise h3 ALPN unconditionally + ... and repair recent build breakage - Closes #6250 - -- [Daiki Ueno brought this change] + Closes #6507 - vquic/ngtcp2.h: define local_addr as sockaddr_storage - - This field needs to be wide enough to hold sockaddr_in6 when - connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the - packets because of the address mismatch: - I00000022 [...] con ignore packet from unknown path +- http: empty reply connection are not left intact - We can safely assume that struct sockaddr_storage is available, as it - is used in the public interface of ngtcp2. + ... so mark the connection as closed in this condition to prevent that + verbose message to wrongly appear. - Closes #6250 + Reported-by: Matt Holt + Bug: https://twitter.com/mholt6/status/1352130240265375744 + Closes #6503 -- socks: check for DNS entries with the right port number +- chunk/encoding: remove conn->data references - The resolve call is done with the right port number, but the subsequent - check used the wrong one, which then could find a previous resolve which - would return and leave the fresh resolve "incomplete" and leaking - memory. + ... by anchoring more functions on Curl_easy instead of connectdata - Fixes #6247 - Closes #6253 + Closes #6498 -- curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use - - ... so don't define it when instructed to use c-ares! +Jay Satiro (20 Jan 2021) +- [Erik Olsson brought this change] -- test506: make it not run in c-ares builds + lib: save a bit of space with some structure packing - As the asynch nature of it may trigger events in another order. A c-ares - upgrade made it break. + - Reorder some internal struct members so that less padding is used. - Reported-by: Marc Hörsken - Fixes #6247 - -- runtests: make 'c-ares' a "feature" to depend on + This is an attempt at saving a bit of space by packing some structs + (using pahole to find the holes) where it might make sense to do + so without losing readability. - ... also added to the docs. - -- tool_writeout: use off_t getinfo-types instead of doubles + I.e., I tried to avoid separating fields that seem grouped + together (like the cwd... fields in struct ftp_conn for instance). + Also abstained from touching fields behind conditional macros as + that quickly can get complicated. - Commit 3b80d3ca46b12e52342 (June 2017) introduced getinfo replacement - variables that use curl_off_t instead of doubles. Switch the --write-out - function over to use them. + Closes https://github.com/curl/curl/pull/6483 + +Daniel Stenberg (20 Jan 2021) +- INSTALL.md: fix typo - Closes #6248 + Found-by: Marcel Raad -- [Emil Engler brought this change] +- [Fabian Keil brought this change] - file: avoid duplicated code sequence + http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy - file_disconnect() is identical with file_do() except the function header - but as the arguments are unused anyway so why not just return file_do() - directly! + Added test 1613 to verify. - Reviewed-by: Daniel Stenberg - Closes #6249 + Closes #6490 -- [Rikard Falkeborn brought this change] +- Merge branch 'bagder/curl_range-data-conn' - infof/failf calls: fix format specifiers - - Update a few format specifiers to match what is being printed. - - Closes #6241 +- ftp: remove conn->data leftover -- docs/INTERNALS: remove reference to Curl_sendf() - - The function has been removed from common usage. Also removed comment in - gopher.c that still referenced it. +- curl_range: remove conn->data - Reported-by: Rikard Falkeborn - Fixes #6242 - Closes #6243 + Closes #6496 -- [Rikard Falkeborn brought this change] +- INSTALL: now at 85 operating systems - examples: update .gitignore - - Add files that are generated by 'make examples' and remove some that - have been renamed. - - The commits that renamed the programs are e9625c5bc6c046a (imap.c and - simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and - ad39e7ec01e7 (pop3slist.c and pop3s.c were renamed to pop3-list.c and - pop3-ssl.c). +- quiche: fix unused parameter ‘conn’ - Closes #6240 + Follow-up to 2bdec0b3 -- asyn: use 'struct thread_data *' instead of 'void *' +- transfer: fix ‘conn’ undeclared mistake for iconv build - To reduce use of types that can't be checked at compile time. Also - removes several typecasts. + Follow-up to 219d9f8620d + +- doh: allocate state struct on demand - ... and rename the struct field from 'os_specific' to 'tdata'. + ... instead of having it static within the Curl_easy struct. This takes + away 1176 bytes (18%) from the Curl_easy struct that aren't used very + often and instead makes the code allocate it when needed. - Closes #6239 - Reviewed-by: Jay Satiro + Closes #6492 -Viktor Szakats (23 Nov 2020) -- Makefile.m32: add support for UNICODE builds +- socks: use the download buffer instead - It requires the linker to support the `-municode` option. - This is available in more recent mingw-w64 releases. + The SOCKS code now uses the generic download buffer for temporary + storage during the connection procedure, instead of having its own + private 600 byte buffer that adds to the connectdata struct size. This + works fine because this point the buffer is allocated but is not use for + download yet since the connection hasn't completed. - Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html - Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847 + This reduces the connection struct size by 22% on a 64bit arch! - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad + The SOCKS buffer needs to be at least 600 bytes, and the download buffer + is guaranteed to never be smaller than 1000 bytes. - Closes #6228 + Closes #6491 -Daniel Stenberg (23 Nov 2020) -- urldata: remove 'void *protop' and create the union 'p' +- urldata: make magic be the first struct field - ... to avoid the use of 'void *' for the protocol specific structs done - per transfer. + By making the `magic` identifier the same size and at the same place + within the structs (easy, multi, share), libcurl will be able to more + reliably detect and safely error out if an application passes in the + wrong handle to APIs. Easier to detect and less likely to cause crashes + if done. - Closes #6238 + Such mixups can't be detected at compile-time due to them being + typedefed void pointers - unless `CURL_STRICTER` is defined. + + Closes #6484 -- winbuild: remove docs from Makefiles and refer to README.md +- http_chunks: correct and clarify a comment on hexnumber length - Reduce risk for conflicting docs and makes it to a single place to fix - and polish. + ... and also rename the define for max length. - add these missing options to the readme: + Closes #6489 + +- curl_path: remove conn->data use - ENABLE_OPENSSL_AUTO_LOAD_CONFIG and ENABLE_UNICODE + Closes #6487 + +- transfer: remove conn->data use - clarify ENABLE_SCHANNEL default varies + Closes #6486 + +- quic: remove conn->data use - Fixes #6216 - Closes #6227 - Co-Authored-by: Jay Satiro + Closes #6485 -- [Daiki Ueno brought this change] +- [Fabian Keil brought this change] - http3: use the master branch of GnuTLS for testing - - Closes #6235 + Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" -- KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation - - Closes #5839 +- [Fabian Keil brought this change] -- KNOWN_BUGS: wakeup socket disconnect causes havoc + Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive" - Closes #6132 - Closes #6133 - -- RELEASE-NOTES: synced + At the moment the test fails as curl sends two Proxy-Connection + headers. -- [Oliver Urbann brought this change] +- c-hyper: avoid duplicated Proxy-Connection headers - curl: add compatibility for Amiga and GCC 6.5 +- http: make providing Proxy-Connection header not cause duplicated headers - Changes are mainly reordering and adding of includes required - to compile with a more recent version of GCC. + Fixes test 1180 - Closes #6220 + Bug: https://curl.se/mail/lib-2021-01/0095.html + Reported-by: Fabian Keil + Closes #6472 -Marc Hoersken (20 Nov 2020) -- tests/server/tftpd.c: close upload file right after transfer - - Make sure uploaded file is no longer locked after the - transfer while waiting for the final ACK to be handled. +- runtests: preprocess DISABLED to allow conditionals - Assisted-by: Daniel Stenberg + ... with this function provided, we can disable tests for specific + environments and setups directly within this file. - Bug: #6058 - Closes #6209 + Closes #6477 -- CI/cirrus: simplify logic for disabled tests - - The OpenSSH server instance for the testsuite cannot - be started on FreeBSD, therefore the SFTP and SCP - tests are disabled right away from the beginning. +- runtests: turn preprocessing into a separate function - The previous OS version specific logic for SKIP_TESTS - is no longer needed/used and can therefore be removed. + ... and remove all other variable substitutions as they're now done once + and for all in the preprocessor. + +- lib/Makefile.inc: convert to listing each file on its own line - Reviewed-by: Daniel Stenberg + ... to make it diff friendlier and easier to read. - Follow up to #6211 - Closes #6229 + Closes #6448 -Daniel Gustafsson (20 Nov 2020) -- mailmap: Daniel Hwang - - Add Daniel Hwang to the mailmap to cover the alternative spelling - Daniel Lee Hwang which was used in one commit. +- ftplistparser: remove use of conn->data - Closes #6230 - Reviewed-by: Daniel Stenberg + Closes #6482 -- openssl: guard against OOM on context creation - - EVP_MD_CTX_create will allocate memory for the context and returns - NULL in case the allocation fails. Make sure to catch any allocation - failures and exit early if so. - - In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex - as the latter is intended for ENGINE selection which we don't do. +- lib: more conn->data cleanups - Closes #6224 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler + Closes #6479 -Daniel Stenberg (19 Nov 2020) -- [Vincent Torri brought this change] +- [Patrick Monnerat brought this change] - cmake: use libcurl.rc in all Windows builds + vtls: reduce conn->data use - Reviewed-by: Marcel Raad - Closes #6215 - -- [Cristian Morales Vega brought this change] + Closes #6474 - cmake: make CURL_ZLIB a tri-state variable +- hyper: deliver data to application with Curl_client_write - By differentiating between ON and AUTO it can make a missing zlib - library a hard error when CURL_ZLIB=ON is used. + ... just as the native code path does. Avoids sending too large data + chunks in the callback and more. - Reviewed-by: Jakub Zakrzewski - Closes #6221 - Fixes #6173 + Reported-by: Gisle Vanem + Fixes #6462 + Closes #6473 -- quiche: remove 'static' from local buffer - - For thread-safety - - Closes #6223 +- gopher: remove accidental conn->data leftover -- KNOWN_BUGS: cmake: libspsl is not supported +- libssh: avoid plain free() of libssh-memory - Closes #6214 - -- KNOWN_BUGS: cmake autodetects cert paths when cross-compiling + Since curl's own memory debugging system redefines free() calls to track + and fiddle with memory, it cannot be used on memory allocated by 3rd + party libraries. - Closes #6178 - -- KNOWN_BUGS: cmake build doesn't fail if zlib not found + Third party libraries SHOULD NOT require free() to release allocated + resources for this reason - and libs can use separate healp allocators + on some systems (like Windows) so free() doesn't necessarily work + anyway. - Closes #6173 - -- KNOWN_BUGS: cmake libcurl.pc uses absolute library paths + Filed as an issue with libssh: https://bugs.libssh.org/T268 - Closes #6169 + Closes #6481 -- KNOWN_BUGS: cmake: generated .pc file contains strange entries +- send: assert that Curl_write_plain() has a ->conn when called - Closes #6167 - -- KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads + To help catch bad invokes. - Closes #6166 + Closes #6476 -- KNOWN_BUGS: cmake build in Linux links libcurl to libdl +- test410: verify HTTPS GET with a 49K request header - Closes #6165 + skip test 410 for mesalink in the CI as it otherwise hangs "forever" -- KNOWN_BUGS: make a new section for cmake topics +- lib: pass in 'struct Curl_easy *' to most functions - Closes #6219 - -- [Emil Engler brought this change] - - cirrus: build with FreeBSD 12.2 in CirrusCI + ... in most cases instead of 'struct connectdata *' but in some cases in + addition to. - Closes #6211 - -Marc Hoersken (14 Nov 2020) -- tests/*server.py: close log file after each log line + - We mostly operate on transfers and not connections. - Make sure the log file is not locked once a test has - finished and align with the behavior of our logmsg. + - We need the transfer handle to log, store data and more. Everything in + libcurl is driven by a transfer (the CURL * in the public API). - Rename curl_test_data.py to be a general util.py. - Format and sort Python imports with isort/VSCode. + - This work clarifies and separates the transfers from the connections + better. - Bug: #6058 - Closes #6206 + - We should avoid "conn->data". Since individual connections can be used + by many transfers when multiplexing, making sure that conn->data + points to the current and correct transfer at all times is difficult + and has been notoriously error-prone over the years. The goal is to + ultimately remove the conn->data pointer for this reason. + + Closes #6425 -Daniel Stenberg (13 Nov 2020) -- CURLOPT_HSTS.3: document the file format +Emil Engler (17 Jan 2021) +- docs: fix typos in NEW-PROTOCOL.md - Closes #6205 + This fixes a misspelled "it" and a grammatically wrong "-ing" suffix. + + Closes #6471 +Daniel Stenberg (16 Jan 2021) - RELEASE-NOTES: synced -- release-notes.pl: detect #[number] better for Ref: etc +Jay Satiro (16 Jan 2021) +- [Razvan Cojocaru brought this change] -- curl: only warn not fail, if not finding the home dir - - ... as there's no good reason to error out completely. + cmake: expose CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG - Reported-by: Andreas Fischer - Fixes #6200 - Closes #6201 - -- httpput-postfields.c: new example doing PUT with POSTFIELDS + This does for cmake builds what --disable-openssl-auto-load-config + does for autoconf builds. - Proposed-by: Jeroen Ooms - Ref: #6186 - Closes #6188 - -- [Tobias Hieta brought this change] + Closes https://github.com/curl/curl/pull/6435 - cmake: correctly handle linker flags for static libs +Daniel Stenberg (15 Jan 2021) +- test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id() - curl CMake was setting the the EXE flags for static libraries which made - the /manifest:no flag ended up when linking the static library, which is - not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit - with an error. + ... and as a practical side-effect, make sure that the + Curl_easyopts_check() function is asserted in debug builds, which we + want to detect mismatches between the options list in easyoptions.c and + the options in curl.h - The better way to handle this is to make sure that we pass the correct - linker flags to CMAKE_STATIC_LINKER_FLAGS instead. + Found-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45991815 - Reviewed-by: Jakub Zakrzewski - Closes #6195 + Closes #6461 -- [Tobias Hieta brought this change] +- [Gisle Vanem brought this change] - cmake: don't pass -fvisibility=hidden to clang-cl on Windows + easyoptions: add the missing AWS_SIGV4 - When using clang-cl on windows -fvisibility=hidden is not an known - argument. Instead it behaves exactly like MSVC in this case. So let's - make sure we take that path. + Follow-up from AWS_SIGV4 + +- schannel_verify: fix safefree call typo - In CMake clang-cl sets both CMAKE_C_COMPILER_ID=clang and MSVC get's - defined since clang-cl is basically a MSVC emulator. So guarding like we - do in this patch seems logical. + Follow-up from e87ad71d1ba00519 - Reviewed-by: Jakub Zakrzewski - Closes #6194 + Closes #6459 -- http_proxy: use enum with state names for 'keepon' +- mime: make sure setting MIMEPOST to NULL resets properly - To make the code clearer, change the 'keepon' from an int to an enum - with better state names. + ... so that a function can first use MIMEPOST and then set it to NULL to + reset it back to a blank POST. - Reported-by: Niranjan Hasabnis - Bug: https://curl.se/mail/lib-2020-11/0026.html - Closes #6193 - -- curl_easy_escape: limit output string length to 3 * max input + Added test 584 to verify the fix. - ... instead of the limiting it to just the max input size. As every - input byte can be expanded to 3 output bytes, this could limit the input - string to 2.66 MB instead of the intended 8 MB. + Reported-by: Christoph M. Becker - Reported-by: Marc Schlatter - Closes #6192 + Fixes #6455 + Closes #6456 -- docs: document the 8MB input string limit +- multi: set the PRETRANSFER time-stamp when we switch to PERFORM - for curl_easy_escape and curl_easy_setopt() + ... instead of at end of the DO state. This makes the timer more + accurate for the protocols that use the DOING state (such as FTP), and + simplifies how the function (now called init_perform) is called. - The limit is there to catch mistakes and abuse. It is meant to be large - enough to allow virtually all "fine" use cases. + The timer will then include the entire procedure up to PERFORM - + including all instructions for getting the transfer started. - Reported-by: Marc Schlatter - Fixes #6190 - Closes #6191 + Closes #6454 -- mqttd: fclose test file when done +- CURLINFO_PRETRANSFER_TIME.3: clarify - Reported-by: Marc Hörsken - Reviewed-by: Jay Satiro - Bug: #6058 - Closes #6189 - -- RELEASE-NOTES: synced + ... the timer *does* include the instructions for getting the remote + file. + + Ref: #6452 + Closes #6453 -- THANKS-filter: ignore autobuild links +- [Gisle Vanem brought this change] -- Revert "libcurl.pc: make it relocatable" - - This reverts commit 3862c37b6373a55ca704171d45ba5ee91dec2c9f. + schannel: plug a memory-leak - That fix should either be done differently or with an option. + ... when built without -DUNICODE. - Reported-by: asavah on github - Fixes #6157 - Closes #6183 + Closes #6457 -- examples/httpput: remove use of CURLOPT_PUT +Jay Satiro (14 Jan 2021) +- gitattributes: Set batch files to CRLF line endings on checkout - It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD. + If a batch file is run without CRLF line endings (ie LF-only) then + arbitrary behavior may occur. I consider that a bug in Windows, however + the effects can be serious enough (eg unintended code executed) that + we're fixing it in the repo by requiring CRLF line endings for batch + files on checkout. - Reported-by: Jeroen Ooms - Fixes #6186 - Closes #6187 - -- Curl_pgrsStartNow: init speed limit time stamps at start + Prior to this change the checked-out line endings of batch files were + dependent on a user's git preferences. On Windows it is common for git + users to have automatic CRLF conversion enabled (core.autocrlf true), + but those users that don't would run into this behavior. - By setting the speed limit time stamps unconditionally at transfer - start, we can start off a transfer without speed limits and yet allow - them to get set during transfer and have an effect. + For example a user has reported running the Visual Studio project + generator batch file (projects/generate.bat) and it looped forever. + Output showed that the Windows OS interpreter was occasionally jumping + to arbitrary points in the batch file and executing commands. This + resulted in unintended files being removed (a removal sequence called) + and looping forever. - Reported-by: Kael1117 on github - Fixes #6162 - Closes #6184 - -- ngtcp2: adapt to recent nghttp3 updates + Ref: https://serverfault.com/q/429594 + Ref: https://stackoverflow.com/q/232651 + Ref: https://www.dostips.com/forum/viewtopic.php?t=8988 + Ref: https://git-scm.com/docs/gitattributes#_checking_out_and_checking_in + Ref: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_core_autocrlf - 'reset_stream' was added to the nghttp3_conn_callbacks struct + Bug: https://github.com/curl/curl/discussions/6427 + Reported-by: Ganesh Kamath - Closes #6185 + Closes https://github.com/curl/curl/pull/6442 -- configure: pass -pthread to Libs.private for pkg-config - - Reported-by: Cristian Morales Vega - Fixes #6168 - Closes #6181 +Daniel Stenberg (14 Jan 2021) +- tool_operate: spellfix a comment -- altsvc: minimize variable scope and avoid "DEAD_STORE" +- ROADMAP: refreshed - Closes #6182 + o removed HSTS - already implemented + o added HTTPS RR records + o mention HTTP/3 completion -- FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?" - - This hasn't been the case for a while now, remove. +- http_chunks: remove Curl_ prefix from static functions -- FAQ: refresh "Why do I get "certificate verify failed" - - Add more details, remove references to ancient curl version. +- transfer: remove Curl_ prefix from static functions -- test493: verify --hsts upgrade and that %{url_effective} reflects that - - Closes #6175 +- tftp: remove Curl_ prefix from static functions -- url: make sure an HSTS upgrade updates URL and scheme correctly - - Closes #6175 +- multi: remove Curl_ prefix from static functions -- tool_operate: set HSTS with CURLOPT_HSTS to pass on filename - - Closes #6175 +- ldap: remove Curl_ prefix from static functions -- hsts: remove debug code leftovers - - Closes #6175 +- doh: remove Curl_ prefix from static functions -- FAQ: refreshed - - - remove a few ancient questions - - add configure with static libs question - - updated wording in several places - - lowercased curl - - Closes #6177 +- asyn-ares: remove Curl_ prefix from static functions -Daniel Gustafsson (5 Nov 2020) -- examples: fix comment syntax - - Commit ac0a88fd2 accidentally added a stray character outside of the - comment which broke compilation. Fix by removing. - - Reported-by: autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742 +- vtls: remove Curl_ prefix from static functions -- hsts: Remove pointless call to free in errorpath - - The line variable will always be NULL in the error path, so remove - the free call since it's pointless. - - Closes #6170 - Reviewed-by: Daniel Stenberg +- bearssl: remove Curl_ prefix from static functions -- docs: Fix various typos in documentation - - Closes #6171 - Reviewed-by: Daniel Stenberg +- mbedtls: remove Curl_ prefix from static functions -Daniel Stenberg (5 Nov 2020) -- copyright: fix year ranges - - Follow-up from 4d2f8006777 +- wolfssl: remove Curl_ prefix from static functions -- HISTORY: the new domain +- nss: remove Curl_ prefix from static functions -- curl.se: new home - - Closes #6172 +- gnutls: remove Curl_ prefix from static functions -- KNOWN_BUGS: FTPS with Schannel times out file list operation +- openssl: remove Curl_ prefix from static functions - Reported-by: bobmitchell1956 on github - Closes #5284 - -- KNOWN_BUGS: SMB tests fail with Python 2 + ... as we reserve this prefix to library-wide functions. - Reported-by: Jay Satiro - Closes #5983 + Closes #6443 -- KNOWN_BUGS: LDAPS with NSS is slow +- nss: get the run-time version instead of build-time - Reported-by: nosajsnikta on github - Closes #5874 + Closes #6445 -Sergei Nikulov (4 Nov 2020) -- travis: use ninja-build for CMake builds +Jay Satiro (12 Jan 2021) +- tool_doswin: Restore original console settings on CTRL signal - Added package ninja-build to environment - Use ninja to speed up CMake builds + - Move Windows terminal init code from tool_main to tool_doswin. - Closes #6077 - -Daniel Stenberg (4 Nov 2020) -- [Harry Sintonen brought this change] - - rtsp: error out on empty Session ID, unified the code - -- [Harry Sintonen brought this change] - - rtsp: fixed the RTST Session ID mismatch in test 570 + - Restore the original console settings on CTRL+C and CTRL+BREAK. - Closes #6161 - -- [Harry Sintonen brought this change] - - rtsp: fixed Session ID comparison to refuse prefix + Background: On Windows the curl tool changes the console settings to + enable virtual terminal processing (eg color output) if supported + (ie Win 10). The original settings are restored on exit but prior to + this change were not restored in the case of the CTRL signals. - Closes #6161 - -- RELEASE-NOTES: synced + Windows VT behavior varies depending on console/powershell/terminal; + refer to the discussion in #6226. - (forgot to update the list of contributors) + Assisted-by: Rich Turner + + Closes https://github.com/curl/curl/pull/6226 -- RELEASE-NOTES: synced +Daniel Stenberg (12 Jan 2021) +- gen.pl: fix perl syntax + + Follow-up to 324cf1d2e -- curlver: bumped to 7.74.0 +- [Emil Engler brought this change] -- hsts: add read/write callbacks + help: update to current codebase - - read/write callback options - - man pages for the 4 new setopts - - test 1915 verifies the callbacks + This commit bumps the help to the current state of the project. - Closes #5896 + Closes #6437 -- hsts: add support for Strict-Transport-Security +- [Emil Engler brought this change] + + docs: fix line length bug in gen.pl - - enable in the build (configure) - - header parsing - - host name lookup - - unit tests for the above - - CI build - - CURL_VERSION_HSTS bit - - curl_version_info support - - curl -V output - - curl-config --features - - CURLOPT_HSTS_CTRL - - man page for CURLOPT_HSTS_CTRL - - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) - - man page for --hsts - - save cache to disk - - load cache from disk - - CURLOPT_HSTS - - man page for CURLOPT_HSTS - - added docs/HSTS.md - - fixed --version docs - - adjusted curl_easy_duphandle + The script warns if the length of $opt and $desc is > 78. However, these + two variables are on totally separate lines so the check makes no sense. + Also the $bitmask field is totally forgotten. Currently this leads to + two warnings within `--resolve` and `--aws-sigv4`. - Closes #5896 + Closes #6438 -- [Sergei Nikulov brought this change] +- [Emil Engler brought this change] - CI/tests: enable test target on TravisCI for CMake builds - - Added test-nonflaky target to CMake builds + docs: fix wrong documentation in help.d - Disabled test 1139 because the cmake build doesn't create docs/curl.1 + curl does not list all categories when you invoke "--help" without any + parameters. - Closes #6074 + Closes #6436 -- tool_debug_cb: do not assume zero-terminated data +- aws-sigv4.d: polish the wording - Follow-up to d70a5b5a0f5e3 - -- sendf: move the verbose-check into Curl_debug + Make it shorter and imperative form - Saves us from having the same check done everywhere. + Closes #6439 + +- [Fabian Keil brought this change] + + misc: fix typos - Closes #6159 + Bug: https://curl.se/mail/lib-2021-01/0063.html + Closes #6434 -- travis: use valgrind when running tests for debug builds +- multi_runsingle: bail out early on data->conn == NULL - Except the non-x86 and sanitizer builds + As that's a significant error condition and scan-build warns for NULL + pointer dereferences if we don't. - Closes #6154 + Closes #6433 -- header.d: fix syntax mistake +- multi: skip DONE state if there's no connection left for ftp wildcard - follow-up from 1144886f38fd0 - -- [Harry Sintonen brought this change] + ... to avoid running in that state with data->conn being NULL. - gnutls: fix memory leaks (certfields memory wasn't released) +- libssh2: fix "Value stored to 'readdir_len' is never read" - Closes #6153 + Detected by scan-build -- tests: add missing global_init/cleanup calls +- connect: mark intentional ignores of setsockopt return values - Without the cleanup call in these test files, the mbedTLS backend leaks - memory. + Pointed out by Coverity - Closes #6156 + Closes #6431 -- tool_operate: --retry for HTTP 408 responses too - - This was inadvertently dropped from the code when the parallel support - was added. +Jay Satiro (11 Jan 2021) +- http_proxy: Fix CONNECT chunked encoding race condition - Regression since b88940850 (7.66.0) + - During the end-of-headers response phase do not mark the tunnel + complete unless the response body was completely parsed/ignored. - Reviewed-by: Jay Satiro - Closes #6155 + Prior to this change if the entirety of a CONNECT response with chunked + encoding was not received by the time the final header was parsed then + the connection would be marked done prematurely, before all the chunked + data could be read in and ignored (since this is what we do with any + CONNECT response body) and the connection could not be used. + + Bug: https://curl.se/mail/lib-2021-01/0033.html + Reported-by: Fabian Keil + + Closes https://github.com/curl/curl/pull/6432 -- http: pass correct header size to debug callback for chunked post +Daniel Stenberg (11 Jan 2021) +- RELEASE-NOTES: synced + +- url: if IDNA conversion fails, fallback to Transitional - ... when the chunked framing was added, the size of the "body part" of - the data was calculated wrongly so the debug callback would get told a - header chunk a few bytes too big that would also contain the first few - bytes of the request body. + This improves IDNA2003 compatiblity. - Reported-by: Dirk Wetter - Ref: #6144 - Closes #6147 + Reported-by: Bubu on github + Fixes #6423 + Closes #6428 -- header.d: mention the "Transfer-Encoding: chunked" handling +- travis: make the Hyper build from its master branch - Ref: #6144 - Closes #6148 + Closes #6430 -- acinclude: detect manually set minimum macos/ipod version +- http: make 'authneg' also work for Hyper - ... even if set in the CC or IPHONEOS/MACOSX_DEPLOYMENT_TARGET - variables. + When doing a request with a request body expecting a 401/407 back, that + initial request is sent with a zero content-length. Test 177 and more. - Reported-by: hamstergene on github - Fixes #6138 - Closes #6140 + Closes #6424 -Jay Satiro (29 Oct 2020) -- tests: fix some http/2 tests for older versions of nghttpx - - - Add regex that strips http/2 server header name to those http/2 tests - that don't already have it. +Jay Satiro (8 Jan 2021) +- cmake: Add an option to disable libidn2 - - Improve that regex in all http/2 tests. + New option USE_LIBIDN2 defaults to ON for libidn2 detection. Prior to + this change libidn2 detection could not be turned off in cmake builds. - Tests 358 and 359 were failing for me before this change on a system - that uses an older version of nghttpx which includes its version number - in the server header. + Reported-by: William A Rowe Jr - Closes https://github.com/curl/curl/pull/6139 - -Daniel Stenberg (30 Oct 2020) -- RELEASE-NOTES: synced + Fixes https://github.com/curl/curl/issues/6361 + Closes https://github.com/curl/curl/pull/6362 -- [Cristian Morales Vega brought this change] +Daniel Stenberg (8 Jan 2021) +- HYPER: no longer needs the special branch - configure: use pkgconfig to find openSSL when cross-compiling - - This reverts 736a40fec (November 2004), which doesn't explain why it was - done. +- test179: use consistent header line endings - Closes #6145 + ... to make "Hyper mode" work better. -- tool_operate: bail out proper on errors for parallel setup +- file: don't provide content-length for directories - ... otherwise for example trying to upload a missing file just causes a - loop. + ... as it is misleading. - Reported-by: BrumBrum on hackerone - Closes #6141 + Ref #6379 + Closes #6421 -- [Sergei Nikulov brought this change] +- TODO: Directory listing for FILE: + + Ref #6379 - CMake: make BUILD_TESTING dependent option +- curl.h: add CURLPROTO_GOPHERS as own protocol identifier - CMake will now handle BUILD_TESTING depending on PERL_FOUND and - CURL_DISABLE_TESTING + Follow-up to a1f06f32b860, to make sure it can be handled separately + from plain gopher. - Ref: #6036 - Closes #6072 + Closes #6418 -- libssh2: fix transport over HTTPS proxy +- http: have CURLOPT_FAILONERROR fail after all headers - The fix in #6021 was not enough. This fix makes sure SCP/SFTP content - can also be transfered over a HTTPS proxy. + ... so that Retry-After and other meta-content can still be used. - Fixes #6113 - Closes #6128 + Added 1634 to verify. Adjusted test 194 and 281 since --fail now also + includes the header-terminating CRLF in the output before it exits. + + Fixes #6408 + Closes #6409 -- curl.1: add an "OUTPUT" section at the top of the manpage +- global_init: debug builds allocates a byte in init - Explain the basic concepts behind curl output. + ... to make build tools/valgrind warn if no curl_global_cleanup is + called. - Inspired by #6124 + This is conditionally only done for debug builds with the env variable + CURL_GLOBAL_INIT set. - Closes #6134 + Closes #6410 -- mailmap: set Viktor Szakats's email +- lib/unit tests: add missing curl_global_cleanup() calls -- runtests: show keywords when no tests ran - - To help out future debugging, runtests now outputs the list of keywords - when it fails because no tests ran. +- travis: adapt to Hyper build change - Ref: #6120 - Closes #6126 + Closes #6419 -Jay Satiro (26 Oct 2020) -- CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo +- pretransfer: setup the User-Agent header here - Reported-by: Rui LIU + ... and not in the connection setup, as for multiplexed transfers the + connection setup might be skipped and then the transfer would end up + without the set user-agent! - Closes https://github.com/curl/curl/issues/6131 + Reported-by: Flameborn on github + Assisted-by: Andrey Gursky + Assisted-by: Jay Satiro + Assisted-by: Mike Gelfand + Fixes #6312 + Closes #6417 -- range.d: fix typo +- test66: disable with Hyper - Follow-up to 15ae039 from earlier today. + ...as Hyper doesn't support HTTP/0.9 -Daniel Stenberg (26 Oct 2020) -- CI/github: work-around for brew breakage on macOS +- c-hyper: poll the tasks until end correctly - ... and make it use OpenSSL 1.1 properly + ... makes test 36 work. - Fixes #6130 - Closes #6129 + Closes #6412 -- [José Joaquín Atria brought this change] +- [Gergely Nagy brought this change] - range.d: clarify that curl will not parse multipart responses + mk-ca-bundle.pl: deterministic output when using -t - Closes #6127 - Fixes #6124 + Printing trust purposes are now sorted, making the output deterministic + when running on the same input certdata.txt. + + Closes #6413 -- RELEASE-NOTES: synced +- KNOWN_BUGS: fixed "wolfSSL lacks support for renegotiation" + + Fixed by #6411 -- [Baruch Siach brought this change] +- [Himanshu Gupta brought this change] - libssh2: fix build with disabled proxy support + wolfssl: add SECURE_RENEGOTIATION support - Build breaks because the http_proxy field is missing: + Closes #6411 + +- RELEASE-NOTES: synced + +- wolfssl: update copyright year range - vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy' + Follow-up to 7de2e96535e9 + +- c-hyper: make CURLE_GOT_NOTHING work - Regression from #6021, shipped in curl 7.73.0 + Test 30 - Closes #6125 + Closes #6407 -- alt-svc: enable by default - - Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported. +- http_proxy: make CONNECT work with the Hyper backend - alt-svc support in curl is no longer considered experimental + Makes test 80 run - Closes #5868 - -- CI/appveyor: remove (unused) runtests.pl -b option + Closes #6406 -- [Emil Engler brought this change] +- TODO: --fail-with-body perchance? - tool_help: make "output" description less confusing +Jay Satiro (4 Jan 2021) +- tool_operate: fix the suppression logic of some error messages - Currently the description of "output" is misleading when comparing it - "verbose". + - Fix the failed truncation and failed writing body error messages to + not be shown unless error messages are shown. (ie the user has + specified -sS, or has not specified -s). - Closes #6118 - -- CI/appveyor: disable test 571 in two cmake builds + - Also prefix same error messages with "curl: ", for example: + curl: (23) Failed to truncate, exiting - ... they're simply too flaky there. + Prior to this change the failed truncation error messages would be shown + if not -s, but did not account for -sS which should show. - Closes #6119 - -- cmake: set the unicode feature in curl-config on Windows + Prior to this change the failed writing body error messages would be + shown always. - ... if built that way. To make it match curl -V output. + Ref: https://curl.se/docs/manpage.html#-S - Reviewed-by: Marcel Raad - Closes #6117 + Bug: https://curl.se/mail/archive-2020-12/0017.html + Reported-by: Hongyi Zhao + + Closes https://github.com/curl/curl/pull/6402 -- libssh2: require version 1.0 or later +- wolfssl: Support wolfSSL builds missing TLS 1.1 - ... and simplify the code accordingly. libssh2 version 1.0 was released - in April 2009. + The wolfSSL TLS library defines NO_OLD_TLS in some of their build + configurations and that causes the library to be built without TLS 1.1. + For example if MD5 is explicitly disabled when building wolfSSL then + that defines NO_OLD_TLS and the library is built without TLS 1.1 [1]. - Closes #6116 - -- KNOWN_BUGS: mention the individual cmake issues + Prior to this change attempting to build curl with a wolfSSL that was + built with NO_OLD_TLS would cause a build link error undefined reference + to wolfTLSv1_client_method. - ... to make them easier to refer to and address separately and - one-by-one. - -- CMake: store IDN2 information in curl_config.h + [1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366 - This allows the build to enable IDN properly and it makes test 1014 - happier. + Bug: https://curl.se/mail/lib-2020-12/0121.html + Reported-by: Julian Montes - Ref: #6074 - Closes #6108 + Closes https://github.com/curl/curl/pull/6388 -- CMake: call the feature unixsockets without dash +Daniel Stenberg (4 Jan 2021) +- test1633: set appropriate name - ... so that curl-config gets correct and makes test 1014 happy! - - Ref: #6074 - Closes #6108 + "--retry with a 429 response and Retry-After:" -- CI/travis: add brotli and zstd to the libssh2 build +- travis: limit the tests with quiche builds to HTTPS and FTPS only - ... to make sure such tests are run with valgrind. Suppress the zstd - valgrind warnings we get with version 1.3.3 on Ubuntu 18.04 (for debug - and non-debug builds). + ... since it runs into the 50 minute time limit too often otherwise. - Closes #6105 + Closes #6403 -- runtests: revert the mistaken edit of $CURL +- HISTORY: added dates to early history - Regression from c4693adc62 - -- RELEASE-NOTES: synced - -- curl_url_set.3: fix typo in the RETURN VALUE section + Mostly thanks to this archived web page for urlget: - Reported-by: Basuke Suzuki - Fixes #6102 - -Jay Satiro (17 Oct 2020) -- [Daniel Stenberg brought this change] + https://web.archive.org/web/19980216125115/http://www.inf.ufrgs.br/~sagula/urlget.html - packages/OS400: make the source code-style compliant +- httpauth: make multi-request auth work with custom port - ... and make sure 'make checksrc' in the root dir also verifies the - packages/OS400 sources. + When doing HTTP authentication and a port number set with CURLOPT_PORT, + the code would previously have the URL's port number override as if it + had been a redirect to an absolute URL. - Closes https://github.com/curl/curl/pull/6085 - -- os400: Sync libcurl API options + Added test 1568 to verify. - This fixes the OS400 build and also an incorrect entry for - CURLINFO_APPCONNECT_TIME_T where it was treated as - CURLINFO_STARTTRANSFER_TIME_T. + Reported-by: UrsusArctos on github + Fixes #6397 + Closes #6400 + +- [Emil Engler brought this change] + + language: s/behaviour/behavior/g - Reported-by: Jon Rumsey + We currently use both spellings the british "behaviour" and the american + "behavior". However "behavior" is more used in the project so I think + it's worth dropping the british name. - Fixes https://github.com/curl/curl/issues/6083 - Closes https://github.com/curl/curl/pull/6084 + Closes #6395 -Daniel Stenberg (16 Oct 2020) -- CURLOPT_NOBODY.3: fix typo +- cmdline-opts/retry.d: mention response code 429 as well - Reported-by: Basuke Suzuki - Fixes #6097 + Reported-by: Cherish98 + Bug: https://curl.se/mail/archive-2020-12/0018.html -Marc Hoersken (16 Oct 2020) -- CI/azure: improve on flakiness by avoiding libtool wrappers - - Install curl binaries into MinGW bin folder and use that - for the tests in order to avoid libtool wrapper binaries. - - The libtool wrapper binaries (not scripts) on Windows seem - to be one of the possible causes for the following issues: +- docs/HYPER.md: mention outstanding issues - 1. Process output can be lost in the wrapper process chain. - 2. Killing the wrapper process does not kill the actual one. + To make it more obvious to users what doesn't work (yet) - Derived from #5904 - Closes #6049 - -Daniel Stenberg (16 Oct 2020) -- CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well + Closes #6389 -- [Zenju brought this change] +- COPYING/configure: bump copyright year range - CURLOPT_TCP_NODELAY.3: fix comment in example code +- c-hyper: add timecondition to the request - Closes #6096 - -- openssl: acknowledge SRP disabling in configure properly + Test 77-78 - Follow-up to 68a513247409 + Closes #6391 + +- c-hyper: make Digest and NTLM work - Use a new separate define that is the combination of both - HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP + Test 64, 65, 67, 68, 69, 70, 72 - Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html + Closes #6390 + +- examples/curlgtk.c: fix the copyright year range - Closes #6094 + ... and make private functions static. -Viktor Szakats (16 Oct 2020) -- http3: fix two build errors, silence warnings +- [Olaf Hering brought this change] + + docs/examples: adjust prototypes for CURLOPT_READFUNCTION - * fix two build errors due to mismatch between function - declarations and their definitions - * silence two mismatched signs warnings via casts + The type of the buffer in curl_read_callback is 'char *', not 'void *'. - Approved-by: Daniel Stenberg - Closes #6093 + Signed-off-by: Olaf Hering + Closes #6392 -- Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 +- examples: fix more empty expression statement has no effect - Approved-by: Daniel Stenberg - Closes #6092 + Follow-up to 26e46617b9 -Daniel Stenberg (16 Oct 2020) -- tool_operate: fix compiler warning when --libcurl is disabled +- cleanup: fix two empty expression statement has no effect - Closes #6095 + Follow-up to 26e46617b9 -- checksrc: warn on empty line before open brace +- configure: set -Wextra-semi-stmt for clang with --enable-debug - ... and fix a few occurances + To have it properly complain on empty statements with no effect. - Closes #6088 + Ref: #6376 + Closes #6378 -- urlapi: URL encode a '+' in the query part - - ... when asked to with CURLU_URLENCODE. +- tests/unit: fix empty statements with no effect - Extended test 1560 to verify. - Reported-by: Dietmar Hauser - Fixes #6086 - Closes #6087 + ... by making macros use "do {} while(0)" -- [Cristian Morales Vega brought this change] +- [Paul Groke brought this change] - libcurl.pc: make it relocatable + dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries - It supposes when people specify the libdir/includedir they do it to - change where under prefix/exec_prefix it should be, not to make it - independent of prefix/exec_prefix. + Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix + (similar to the existing '-' prefix for removing entries) to add + DNS cache entries that will time out just like entries that are added + by libcurl itself. - Closes #6061 - -- runtests: return error if no tests ran + Append " (non-permanent)" to info log message in case a non-permanent + entry is added. - ... and make TESTFAIL stand out a little better by adding newlines - before and after. + Adjust relevant comments to reflect the new behavior. - Reported-by: Marc Hörsken - Issue: #6052 - Closes #6053 - -- docs/FEATURE: convert to markdown + Adjust documentation. - ... and clean it up a bit. + Extend unit1607 to test the new functionality. - Closes #6067 + Closes #6294 -- [Philipp Klaus Krause brought this change] +- schannel: fix "empty expression statement has no effect" + + Bug: https://github.com/curl/curl/commit/8ab78f720ae478d533e30b202baec4b451741579#commitcomment-45445950 + Reported-by: Gisle Vanem + Closes #6381 - strerror: use 'const' as the string should never be modified +- [Denis Laxalde brought this change] + + docs: remove redundant "better" in --fail help - Closes #6068 + Closes #6385 -- [Jay Satiro brought this change] +- [Kevin Ushey brought this change] - connect: repair build without ipv6 availability + curl.1: fix typo microsft -> microsoft - Assisted-by: Daniel Stenberg - Reported-by: Tom G. Christensen + Closes #6380 + +- [XhmikosR brought this change] + + misc: assorted typo fixes - Fixes https://github.com/curl/curl/issues/6069 - Closes https://github.com/curl/curl/pull/6071 + Closes #6375 - RELEASE-NOTES: synced + +- tool_operate: avoid NULL dereference of first_arg - Started over for the journey to next release. + Follow-up to 6a5e020d4d2b04a + Identified by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28999 + Closes #6377 -- src/tool_filetime: disable -Wformat on mingw for this file +- misc: fix "warning: empty expression statement has no effect" - With gcc 10 on mingw we otherwise get this warning: + Turned several macros into do-while(0) style to allow their use to work + find with semicolon. - error: ISO C does not support the 'I' printf flag [-Werror=format=] + Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45433279 + Follow-up to 08e8455dddc5e4 + Reported-by: Gisle Vanem + Closes #6376 + +- KNOWN_BUGS: 6.10 curl never completes Negotiate over HTTP - Fixes #6079 - Closes #6082 + Closes #5235 + Closes #6370 -- test122[12]: remove these two tests +- writeout: fix NULL dereference for "this url" - ... and remove the objnames scripts they tested. They're not used for - anything anymore so testing them serves no purpose! + Detected by torture test 1029 - Reported-by: Marc Hörsken - Fixes #6080 - Closes #6081 - -Version 7.73.0 (14 Oct 2020) + Follow-up to 7a90ddf88f5a + + Closes #6374 -Daniel Stenberg (14 Oct 2020) -- RELEASE-NOTES: synced +- failf: remove newline from formatting strings - for 7.73.0 + ... as failf adds one itself. + + Also: add an assert() to failf() that triggers on a newline in the + format string! + + Closes #6365 -- THANKS: from 7.73.0 and .mailmap fixes +- [XhmikosR brought this change] -- mailmap: fixups of some contributors + CI: fix warning with the latest versions + + `git checkout HEAD^2` is no longer needed + + Closes #6369 -- projects/build-wolfssl.bat: fix the copyright year range +- INSTALL: update the list known OSes and CPU archs curl has run on + + Closes #6366 -Marc Hoersken (14 Oct 2020) -- [Sergei Nikulov brought this change] +- [Cherish98 brought this change] - CI/tests: fix invocation of tests for CMake builds + curl: fix handling of -q option - Update appveyor.yml to set env variable TFLAGS and run tests - Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) - Move testdeps build to build step (per review comments) + The match of the "-q" option (short for "--disable") should: + a) allow concatenation with other single-letters; and + b) be case-sensitive, lest confusing with "-Q" ("--quote") - Reviewed-by: Marc Hörsken + Closes #6364 + +- tests/badsymbols.pl: ignore stand-alone single hash lines - Closes #6066 - Fixes #6052 + Bug: https://curl.se/mail/lib-2020-12/0084.html + Reported-by: Dennis Clarke + Assisted-by: Jay Satiro + + Closes #6355 -- tests/server/util.c: fix support for Windows Unicode builds +- curl_easy_pause.3: add multiplexed pause effects - Detected via #6066 - Closes #6070 + and generally refresh and update. Remove details for ancient versions. + + Reviewed-by: Jay Satiro + Closes #6360 -Daniel Stenberg (13 Oct 2020) -- [Jay Satiro brought this change] +Jay Satiro (22 Dec 2020) +- curl_easy_pause.3: fix man page reference + + Follow-up to ac9a724 from earlier today. + + Ref: https://github.com/curl/curl/pull/6359 - strerror: Revert to local codepage for Windows error string +Daniel Stenberg (22 Dec 2020) +- EXPERIMENTAL: add the Hyper backend to the list - - Change get_winapi_error() to return the error string in the local - codepage instead of UTF-8 encoding. + ... of current experimental features in curl. + +- speedcheck: exclude paused transfers - Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it - also changed the error string's encoding from local codepage to UTF-8. + Paused transfers should not be stopped due to slow speed even when + CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is + now reset when the transfer is unpaused - as otherwise it would easily + just trigger immediately after unpausing. - We return the local codepage version of the error string because if it - is output to the user's terminal it will likely be with functions which - expect the local codepage (eg fprintf, failf, infof). + Reported-by: Harry Sintonen + Fixes #6358 + Closes #6359 + +- h2: do not wait for RECV on paused transfers - This is essentially a partial revert of bed5f84. The support for xbox - remains but the error string is reverted back to local codepage. + ... as the socket might be readable all the time when paused and thus + causing a busy-loop. - Ref: https://github.com/curl/curl/pull/6005 + Reported-by: Harry Sintonen + Reviewed-by: Jay Satiro + Fixes #6356 + Closes #6357 + +- RELEASE-NOTES: synced + +- cmdline-opts/gen.pl: return hard on errors - Reviewed-by: Marcel Raad - Closes #6065 + ... as the warnings tend to go unnoticed otherwise! + + Closes #6354 -Marc Hoersken (13 Oct 2020) -- CI/tests: use verification curl for test reporting APIs +- examples/libtest: add .checksrc to dist - Avoid using our own, potentially installed, curl for - the test reporting APIs in case it is broken. + ... so that (auto)builds from tarballs also get the correct instructions. - Reviewed-by: Daniel Stenberg + Fixes #6176 + Closes #6353 + +- test: verify new --write-out variables - Preparation for #6049 - Closes #6063 + Extended test 1029 and added 1188 -Viktor Szakats (12 Oct 2020) -- windows: fix comparison of mismatched types warning +- test970: adapted to the new internal order of variables + +- curl: add variables to --write-out - clang 10, mingw-w64: - ``` - vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') - [-Wsign-compare] - if(GetLastError() != CRYPT_E_NOT_FOUND) - ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ - ``` + In particular, these ones can help a user to create its own error + message when one or transfers fail. - Approved-by: Daniel Stenberg - Closes #6062 + writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg' + + onerror - lets a user only show the rest on non-zero exit codes + + url - the input URL used for this transfer + + urlnum - the numerical URL counter (0 indexed) for this transfer + + exitcode - the numerical exit code for the transfer + + errormsg - obvious + + Reported-by: Earnestly on github + Fixes #6199 + Closes #6207 -Daniel Stenberg (11 Oct 2020) -- [Viktor Szakats brought this change] +- [Matthias Gatto brought this change] - src/Makefile.m32: fix undefined curlx_dyn_* errors + tests: add very simple AWS HTTP v4 Signature test - by linking `lib/dynbuf.c` when building a static curl binary. - Previously this source file was only included when building - a dynamic curl binary. This was likely possibly because no - functions from the `src/Makefile.inc` / `CURLX_CFILES` sources - were actually required for a curl tool build. This has - recently changed with the introduction of `curlx_dyn_*()` - memory functions and their use by the tool sources. + Signed-off-by: Matthias Gatto + +- [Matthias Gatto brought this change] + + docs: add AWS HTTP v4 Signature + +- [Matthias Gatto brought this change] + + tool: add AWS HTTP v4 Signature support - Closes #6060 + Signed-off-by: Matthias Gatto -- HISTORY: curl verifies SSL certs by default since version 7.10 +- [Matthias Gatto brought this change] -Marc Hoersken (8 Oct 2020) -- runtests.pl: use $LIBDIR variable instead of hardcoded path + http: Make the call to v4 signature - Reviewed-by: Daniel Stenberg - Closes #6051 + This patch allow to call the v4 signature introduce in previous commit + + Signed-off-by: Matthias Gatto -Daniel Stenberg (7 Oct 2020) -- checksrc: detect // comments on column 0 +- [Matthias Gatto brought this change] + + http: introduce AWS HTTP v4 Signature - Spotted while working on #6045 + It is a security process for HTTP. - Closes #6048 + It doesn't seems to be standard, but it is used by some cloud providers. + + Aws: + https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html + Outscale: + https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request + GCP (I didn't test that this code work with GCP though): + https://cloud.google.com/storage/docs/access-control/signing-urls-manually + + most of the code is in lib/http_v4_signature.c + + Information require by the algorithm: + - The URL + - Current time + - some prefix that are append to some of the signature parameters. + + The data extracted from the URL are: the URI, the region, + the host and the API type + + example: + https://api.eu-west-2.outscale.com/api/latest/ReadNets + ~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ + ^ ^ ^ + / \ URI + API type region + + Small description of the algorithm: + - make canonical header using content type, the host, and the date + - hash the post data + - make canonical_request using custom request, the URI, + the get data, the canonical header, the signed header + and post data hash + - hash canonical_request + - make str_to_sign using one of the prefix pass in parameter, + the date, the credential scope and the canonical_request hash + - compute hmac from date, using secret key as key. + - compute hmac from region, using above hmac as key + - compute hmac from api_type, using above hmac as key + - compute hmac from request_type, using above hmac as key + - compute hmac from str_to_sign using above hmac as key + - create Authorization header using above hmac, prefix pass in parameter, + the date, and above hash + + Signed-off-by: Matthias Gatto + + Closes #5703 -- [Frederik Wedel-Heinen brought this change] +- [Matthias Gatto brought this change] - mbedtls: add missing header when defining MBEDTLS_DEBUG + http: add hmac support for sha256 - Closes #6045 + It seems current hmac implementation use md5 for the hash, + V4 signature require sha256, so I've added the needed struct in + this commit. + + I've added the functions that do the hmac in v4 signature file + as a static function ,in the next patch of the serie, + because it's used only by this file. + + Signed-off-by: Matthias Gatto -- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long +- [Cristian Rodríguez brought this change] + + connect: on linux, enable reporting of all ICMP errors on UDP sockets - Previously, it would pass on a define (int) which could make libcurl - read junk as a value - which prevented the CURLOPT_IPRESOLVE option to - "take". This could then make test 2100 do two DoH requests instead of - one! + The linux kernel does not report all ICMP errors back to userspace due + to historical reasons. - Fixes #6042 - Closes #6043 + IP*_RECVERR sockopt must be turned on to have the correct behaviour + which is to pass all ICMP errors to userspace. + + See https://bugzilla.kernel.org/show_bug.cgi?id=202355 + + Closes #6341 -- RELEASE-NOTES: synced +- curl: add --create-file-mode [mode] + + This option sets the (octal) mode to use for the remote file when one is + created, using the SFTP, SCP or FILE protocols. When not set, the + default is 0644. + + Closes #6244 -- scripts/release-notes.pl: don't "embed" $ in format string for printf() +- c-hyper: fix compiler warnings - ... since they might contain %-codes that mess up the output! + Identified by clang on windows. + + Reported-by: Gisle Vanem + Bug: 58974d25d8173aec154e593ed9d866da566c9811 + + Closes #6351 -Jay Satiro (5 Oct 2020) -- [M.R.T brought this change] +- KNOWN_BUGS: Remote recursive folder creation with SFTP + + Closes #5204 - build-wolfssl: fix build with Visual Studio 2019 +Jay Satiro (20 Dec 2020) +- badsymbols.pl: Add verbose mode -v - Closes https://github.com/curl/curl/pull/6033 + Use -v as the first option to enable verbose mode which will show source + input, extracted symbol and line info. For example: + + Source: ./../include/curl/typecheck-gcc.h + Symbol: curlcheck_socket_info(info) + Line #423: #define curlcheck_socket_info(info) \ + + Ref: https://curl.se/mail/lib-2020-12/0084.html + + Closes https://github.com/curl/curl/pull/6349 -Daniel Stenberg (4 Oct 2020) -- runtests: add %repeat[]% for test files +- KNOWN_BUGS: Secure Transport disabling hostname validation also disables SNI - ... and use this new keywords in all the test files larger than 50K to reduce - their sizes and make them a lot easier to read and understand. + That behavior is a limitation of Apple's Secure Transport. - Closes #6040 + Reported-by: Cory Benfield + Reported-by: Ian Spence + Confirmed-by: Nick Zitzmann + + Ref: https://github.com/curl/curl/issues/998 + + Closes https://github.com/curl/curl/issues/6347 + Closes https://github.com/curl/curl/pull/6348 -- [Emil Engler brought this change] +Daniel Stenberg (18 Dec 2020) +- TODO: alt-svc should fallback if alt-svc doesn't work + + Closes #4908 - --help: move two options from the misc category +- travis: restrict the openssl3 job to only run https and ftps tests - The cmdline opts delegation and suppress-connect-headers - fit better into auth and proxy rather than misc. + ... as it runs too long otherwise and the other tests are verified in + other builds anyway. - Follow-up to aa8777f63febc - Closes #6038 - -- [Samanta Navarro brought this change] + Closes #6345 - docs/opts: fix typos in two manual pages +- build: repair http disabled but mqtt enabled build - Closes #6039 + ... as the mqtt code reuses the "method" originally used for HTTP. + + Closes #6344 -- ldap: reduce the amount of #ifdefs needed +- [Jon Wilkes brought this change] + + cookie: avoid the C1001 internal compiler error with MSVC 14 - Closes #6035 + Fixes #6112 + Closes #6135 -- runtests: provide curl's version string as %VERSION for tests +- RELEASE-NOTES: synced + +- mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE - ... so that we can check HTTP requests for User-Agent: curl/%VERSION + Detected by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735 - Update 600+ test cases accordingly. + Added test 1916 and 1917 to verify. - Closes #6037 + Closes #6338 -- checksrc: warn on space after exclamation mark +- travis: add CI job for Hyper build + +- tests: updated tests for Hyper + +- lib: introduce c-hyper for using Hyper - Closes #6034 + ... as an alternative HTTP backend within libcurl. -- test1465: verify --libcurl with binary POST data +- tool_setopt: provide helper output in debug builds + + ... for when setopt() returns error. -- runtests: allow generating a binary sequence from hex +- setopt: adjust to Hyper and disabled HTTP builds -- tool_setopt: escape binary data to hex, not octal +- rtsp: disable if Hyper is used -- curl: make --libcurl show binary posts correctly +- getinfo: build with disabled HTTP support + +- version: include hyper version + +- docs: add HYPER.md + +- configure: add --with-hyper - Reported-by: Stephan Mühlstrasser - Fixes #6031 - Closes #6032 + As the first (optional) HTTP backend alternative instead of native + + Close #6110 -Jay Satiro (1 Oct 2020) -- strerror: fix null deref on winapi out-of-memory +- test1522: add debug tracing - Follow-up to bed5f84 from several days ago. + I used this to track down some issues and I figured I could just as well + keep this extra logging in here for future needs. - Ref: https://github.com/curl/curl/pull/6005 + Closes #6331 -Daniel Stenberg (1 Oct 2020) -- [Kamil Dudka brought this change] +- http: show the request as headers even when split-sending + + When the initial request isn't possible to send in its entirety, the + remainder of request would be delivered to the debug callback as data + and would wrongly be counted internally as body-bytes sent. + + Extended test 1295 to verify. + + Closes #6328 - vtls: deduplicate some DISABLE_PROXY ifdefs +- multi: when erroring in TOOFAST state, act as for PERFORM - ... in the code of gtls, nss, and openssl + When failing in TOOFAST, the multi_done() wasn't called so the same + cleanup and handling wasn't done like when it fails in PERFORM, which in + the case of FTP could mean that the control connection wouldn't be + marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused + ftp_disconnect() to use it to send "QUIT", which could end up waiting + for a response a long time before giving up! - Closes #5735 + Reported-by: Tomas Berger + Fixes #6333 + Closes #6337 -- RELEASE-NOTES: synced +- cmake: enable gophers correctly in curl-config + + Closes #6336 -- [Emil Engler brought this change] +- test1198/9: add two mqtt publish tests without payload lengths + + Closes #6335 - TODO: Add OpenBSD libtool notice +- tests/mqttd: extract the client id from the correct offset - See #5862 - Closes #6030 + Closes #6334 -- tests/unit/README: convert to markdown +- TODO: Prevent terminal injection when writing to terminal - ... and add to dist! + Closes #6150 + +- Revert "CI/github: work-around for brew breakage on macOS" - Closes #6028 + This reverts commit 4cbb17a2cbbbe6337142d39479e21c3990b9c22f. + + ... as the work-around now causes failures. + + Closes #6332 -- tests/README: convert to markdown +- examples: remove superfluous asterisk uses - Closes #6028 + ... for function pointers. Breaks in ancient compilers. -- include/README: convert to markdown +- RELEASE-NOTES: synced + +- test1272: fix line ending - Closes #6028 + Follow-up to f24784f9143 -- examples/README: convert to markdown +- URL-SYNTAX: add gophers details + +- test1272: test gophers + +- runtests: add support for gophers, gopher over TLS + +- [parazyd brought this change] + + gopher: Implement secure gopher protocol. - Closes #6028 + This commit introduces a "gophers" handler inside the gopher protocol if + USE_SSL is defined. This protocol is no different than the usual gopher + prococol, with the added TLS encapsulation upon connecting. The protocol + has been adopted in the gopher community, and many people have enabled + TLS in their gopher daemons like geomyidae(8), and clients, like clic(1) + and hurl(1). + + I have not implemented test units for this protocol because my knowledge + of Perl is sub-par. However, for someone more knowledgeable it might be + fairly trivial, because the same test that tests the plain gopher + protocol can be used for "gophers" just by adding a TLS listener. + + Signed-off-by: parazyd + + Closes #6208 -- configure: don't say HTTPS-proxy is enabled when disabled! +- TODO: Package curl for Windows in a signed installer - Reported-by: Kamil Dudka - Reviewed-by: Kamil Dudka - Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 - Closes #6029 + Closes #5424 -Daniel Gustafsson (30 Sep 2020) -- src: Consistently spell whitespace without whitespace +- mqtt: deal with 0 byte reads correctly - Whitespace is spelled without a space between white and space, so - make sure to consistently spell it that way across the codebase. + OSS-Fuzz found it + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676 - Closes #6023 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler + Closes #6327 -- MANUAL: update examples to resolve without redirects +- BUG-BOUNTY: minor language update + + ... and remove the wording about entries from before 2019 as the "within + 12 months" is still there and covers that. + + Closes #6318 + +- tooĺ_writeout: fix the -w time output units + + Fix regression from commit fc813f80e1bcac (#6248) that changed the unit + to microseconds instead of seconds with fractions + + Reported-by: 不确定 + Fixes #6321 + Closes #6322 + +- quiche: remove fprintf() leftover + +Jay Satiro (14 Dec 2020) +- KNOWN_BUGS: SHA-256 digest not supported in Windows SSPI builds + + Closes https://github.com/curl/curl/issues/6302 + +- digest_sspi: Show InitializeSecurityContext errors in verbose mode + + The error is shown with infof rather than failf so that the user will + see the extended error message information only in verbose mode, and + will still see the standard CURLE_AUTH_ERROR message. For example: + + --- + + * schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED + (0x8009030A) - The per-message Quality of Protection is not supported by + the security package + * multi_done + * Connection #1 to host 127.0.0.1 left intact + curl: (94) An authentication function returned an error - www.netscape.com is redirecting to a cookie consent form on Aol, and - cool.haxx.se isn't responding to FTP anymore. Replace with examples - that resolves in case users try out the commands when reading the - manual. + --- - Closes #6024 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler + Ref: https://github.com/curl/curl/issues/6302 + + Closes https://github.com/curl/curl/pull/6315 -Daniel Stenberg (30 Sep 2020) -- HISTORY: add some 2020 events +Daniel Stenberg (13 Dec 2020) +- URL-SYNTAX: add default port numbers and IDNA details + + Closes #6316 -- sectransp: make it build with --disable-proxy +- URL-SYNTAX: mention how FILE:// access can access network on windows - Follow-up from #5466 and f3d501dc678d80 - Reported-by: Javier Navarro - Fixes #6025 - Closes #6026 + Closes #6314 -- ECH: renamed from ESNI in docs and configure +Jay Satiro (12 Dec 2020) +- URL-SYNTAX: Document default SMTP port 25 - Encrypted Client Hello (ECH) is the current name. + Note that ports 25 and 587 are common ports for smtp, the former being + the default. - Closes #6022 + Closes https://github.com/curl/curl/pull/6310 -- configure: use "no" instead of "disabled" for the end summary +Daniel Stenberg (12 Dec 2020) +- CURLOPT_URL.3: remove scheme specific details - ... for consistency but also to make them more distinctly stand out next - to the "enabled" lines. + ... that are now found in URL-SYNTAX.md + + Closes #6307 -- TODO: SSH over HTTPS proxy with more backends +Dan Fandrich (12 Dec 2020) +- docs: Fix some typos - ... as right now only the libssh2 backend supports it. + [skip ci] -- libssh2: handle the SSH protocols done over HTTPS proxy +Daniel Stenberg (12 Dec 2020) +- URL-SYNTAX: mention all supported schemes - Reported-by: Robin Douine - Fixes #4295 - Closes #6021 + Closes #6311 -- [Emil Engler brought this change] +- [Douglas R. Reno brought this change] - memdebug: remove 9 year old unused debug function - - There used to be a way to have memdebug fill allocated memory. 9 years - later this has no value there (valgrind and ASAN etc are way better). If - people need to know about it they can have a look at VCS logs. + URL-SYNTAX.md: minor language improvements - Closes #5973 + Closes #6308 -- sendf: move Curl_sendf to dict.c and make it static - - ... as the only remaining user of that function. Also fix gopher.c to - instead use Curl_write() +- docs/URL-SYNTAX: the URL syntax curl accepts and works with - Closes #6020 + Closes #6285 -- ROADMAP: updates and cleanups +- [0xflotus brought this change] + + docs: enable syntax highlighting in several docs files - Fix the HSTS PR + ... for better readability - Remove DoT, thread-safe init and hard-coded localhost. I feel very - little interest for these with users so I downgrade them to plain "TODO" - entries again. + Closes #6286 -- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root - - This matches what is returned in other TLS backends in the same - situation. +- test1564/1565: require the 'wakeup' feature to run - Reviewed-by: Jay Satiro - Reviewed-by: Emil Engler - Follow-up to 5a3efb1 - Reported-by: iammrtau on github - Fixes #6003 - Closes #6018 + Fixes #6299 + Fixes #6300 + Closes #6301 -- RELEASE-NOTES: synced +- runtests: add 'wakeup' as a feature -- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL - - Added test 348 to verify. Added a 'STOR' command to the test FTP - server to enable test 348. Documented the command in FILEFORMAT.md +- tests/server/disabled: add "wakeup" - Reported-by: Duncan Wilcox - Fixes #6016 - Closes #6017 + To allow the test suite to know if wakeup support is disabled in the + build. -- pause: only trigger a reread if the unpause sticks - - As an unpause might itself get paused again and then triggering another - reread doesn't help. - - Follow-up from e040146f22608fd9 (shipped since 7.69.1) - - Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html - Patch-by: Kunal Chandarana - Fixes #5988 - Closes #6013 +- lib1564/5: verify that curl_multi_wakeup returns OK -- test163[12]: require http to be built-in to run +- tests: make --libcurl tests only test FTP options if ftp enabled - ... as speaking over an HTTPS proxy implies http! + Adjust six --libcurl tests to only check the FTP option if FTP is + actually present in the build. - Closes #6014 + Fixes #6303 + Closes #6305 -- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define +- runtests.pl: fix "uninitialized value" warning - Closes #6012 - -- [Javier Blazquez brought this change] + follow-up to e12825c642a88774 - strerror: honor Unicode API choice on Windows +- runtests: add support for %if [feature] conditions - Closes #6005 - -- imap: make imap_send use dynbuf for the send buffer management + ... to make tests run differently or expect different results depending + on what features that are present or not in curl. - Reuses the buffer and thereby reduces number of mallocs over a transfer. + Bonus: initial minor 'Hyper' awareness but nothing is using that yet - Closes #6010 + Closes #6304 -- Curl_send: return error when pre_receive_plain can't malloc - - ... will probably trigger some false DEAD CODE positives on non-windows - code analyzers for the conditional code. - - Closes #6011 +- [Jon Rumsey brought this change] -- ftp: separate FTPS from FTP over "HTTPS proxy" + OS400: update ccsidcurl.c - When using HTTPS proxy, SSL is used but not in the view of the FTP - protocol handler itself so separate the connection's use of SSL from the - FTP control connection's sue. + Add 'struct' to cast and declaration of cfcdata to fix compilation + error. - Reported-by: Mingtao Yang - Fixes #5523 - Closes #6006 + Fixes #6292 + Closes #6297 -Dan Fandrich (23 Sep 2020) -- tests/data: Fix some mismatched XML tags in test cases +- ngtcp2: make it build it current master again - This allows these test files to pass xmllint. + Closes #6296 -Daniel Stenberg (23 Sep 2020) -- pingpong: use a dynbuf for the *_pp_sendf() function - - ... reuses the same dynamic buffer instead of doing repeated malloc/free - cycles. +- [Cristian Rodríguez brought this change] + + connect: defer port selection until connect() time - Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls - after this change in my test setup (132 => 125), curl 7.72.0 needed 140 - calls for this. + If supported, defer port selection until connect() time + if --interface is given and source port is 0. - Test case 103 makes 9 less allocations now (130). Down from 149 in - 7.72.0. + Reproducer: - Closes #6004 - -- dynbuf: add Curl_dyn_vaddf + * start fast webserver on port 80 + * starve system of ephemeral ports + $ sysctl net.ipv4.ip_local_port_range="60990 60999" - Closes #6004 - -- dynbuf: make *addf() not require extra mallocs + * start a curl/libcurl "crawler" + $curl --keepalive --parallel --parallel-immediate --head --interface + 127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt" - ... by introducing a printf() function that appends directly into a - dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if - the buffer is already big enough it can just printf directly into it. + current result: + (possible some successful data) + curl: (45) bind failed with errno 98: Address already in use - Since this less-malloc version requires tthe use of a library internal - printf function, we only provide this version when building libcurl and - not for the dynbuf code that is used when building the curl tool. + result after patch: + (complete success or few connections failing, higlhy depending on load) - Closes #5998 - -- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport + Fail only when all the possible 4-tuple combinations are exhausted, + which is impossible to do when port is selected at bind() time becuse + the kernel does not know if socket will be listen()'ed on or connect'ed + yet. - Closes #5403 + Closes #6295 -- pingpong: remove a malloc per Curl_pp_vsendf call - - This typically makes 7-9 fewer mallocs per FTP transfer. - - Closes #5997 +- [Hans-Christian Noren Egtvedt brought this change] -- symbian: drop support + connect: zero variable on stack to silence valgrind complaint - The OS is deprecated. I see no traces of anyone having actually built - curl for Symbian after 2012. + Valgrind will complain that ssrem buffer usage if not explicit + initialized, hence initialize it to zero. - The public headers are unmodified. + This completes the change intially started in commit 2c0d7212151 ('ftp: + retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has + a similar memset to zero. - Closes #5989 + Signed-off-by: Hans-Christian Noren Egtvedt + Closes #6289 - RELEASE-NOTES: synced - -- curl_krb5.h: rename from krb5.h - - Follow-up from f4873ebd0be32cf - Turns out some older openssl installations go bananas otherwise. - Reported-by: Tom van der Woerdt - Fixes #5995 - Closes #5996 + start over on the next release cycle -- test1297: verify GOT_NOTHING with http proxy tunnel +Version 7.74.0 (9 Dec 2020) -- http_proxy: do not count proxy headers in the header bytecount - - ... as that counter is subsequently used to detect if nothing was - returned from the peer. This made curl return CURLE_OK when it should - have returned CURLE_GOT_NOTHING. +Daniel Stenberg (9 Dec 2020) +- RELEASE-NOTES: synced - Fixes #5992 - Reported-by: Tom van der Woerdt - Closes #5994 + for 7.74.0 + +Jay Satiro (7 Dec 2020) +- [Jacob Hoffman-Andrews brought this change] -- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument + urldata: restore comment on ssl_connect_data.use - Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the - option is, yeah, not known. Clarified this in the setopt man page too. + This comment was originally on the `use` field, but was separated from + its field in 62a2534. - Closes #5993 + Closes https://github.com/curl/curl/pull/6287 -- krb5: merged security.c and krb specific FTP functions in here +Daniel Stenberg (7 Dec 2020) +- VERSIONS: refreshed - These two files were always tightly connected and it was hard to - understand what went into which. This also allows us to make the - ftpsend() function static (moved from ftp.c). + We always use the patch number these days: all releases are + "major.minor.patch" + +- [Jakub Zakrzewski brought this change] + + cmake: don't use reserved target name 'test' - Removed security.c - Renamed curl_sec.h to krb5.h + CMake up to 3.10 always reserves this name - Closes #5987 + Fixes #6257 + Closes #6258 -- Curl_handler: add 'family' to each protocol +- openssl: make the OCSP verification verify the certificate id - Makes get_protocol_family() faster and it moves the knowledge about the - "families" to each protocol handler, where it belongs. + CVE-2020-8286 - Closes #5986 + Reported by anonymous + + Bug: https://curl.se/docs/CVE-2020-8286.html -- parsedate: tune the date to epoch conversion +- ftp: make wc_statemach loop instead of recurse - By avoiding an unnecessary error check and the temp use of the tm - struct, the time2epoch conversion function gets a little bit faster. - When repeating test 517, the updated version is perhaps 1% faster (on - one particular build on one particular architecture). + CVE-2020-8285 - Closes #5985 + Fixes #6255 + Bug: https://curl.se/docs/CVE-2020-8285.html + Reported-by: xnynx on github -- cmake: remove scary warning +- ftp: CURLOPT_FTP_SKIP_PASV_IP by default - Remove the text saying + The command line tool also independently sets --ftp-skip-pasv-ip by + default. - "the curl cmake build system is poorly maintained. Be aware" + Ten test cases updated to adapt the modified --libcurl output. - ... not because anything changed just now, but to encourage users to use - it and subsequently improve it. + Bug: https://curl.se/docs/CVE-2020-8284.html + CVE-2020-8284 - Closes #5984 - -- docs/MQTT: remove outdated paaragraphs + Reported-by: Varnavas Papaioannou -- docs/MQTT: not experimental anymore +- urlapi: don't accept blank port number field without scheme - Follow-up to e37e4468688d8f - -- docs/RESOURCES: remove + ... as it makes the URL parser accept "very-long-hostname://" as a valid + host name and we don't want that. The parser now only accepts a blank + (no digits) after the colon if the URL starts with a scheme. - This document is not maintained and rather than trying to refresh it, - let's kill it. A more up-to-date document with relevant RFCs is this - page on the curl website: https://curl.haxx.se/rfc/ + Reported-by: d4d on hackerone - Closes #5980 + Closes #6283 -- docs/TheArtOfHttpScripting: convert to markdown +- Revert "multi: implement wait using winsock events" - Makes it easier to browse on github etc. Offers (better) links. + This reverts commit d2a7d7c185f98df8f3e585e5620cbc0482e45fac. - It should be noted that this document is already mostly outdated and - "Everything curl" at https://ec.haxx.se/ is a better resource and - tutorial. + This commit also reverts the subsequent follow-ups to that commit, which + were all done within windows #ifdefs that are removed in this + change. Marc helped me verify this. - Closes #5981 + Fixes #6146 + Closes #6281 -- BUGS: convert document to markdown - - Closes #5979 +- [Klaus Crusius brought this change] -- --help: strdup the category + ftp: retry getpeername for FTP with TCP_FASTOPEN - ... since it is converted and the original pointer is freed on Windows - unicode handling. + In the case of TFO, the remote host name is not resolved at the + connetion time. - Follow-up to aa8777f63febc - Fixes #5977 - Closes #5978 - Reported-by: xwxbug on github + For FTP that has lead to missing hostname for the secondary connection. + Therefore the name resolution is done at the time, when FTP requires it. + + Fixes #6252 + Closes #6265 + Closes #6282 -- CHECKSRC: document two missing warnings +- [Thomas Danielsson brought this change] + + scripts/completion.pl: parse all opts + + For tab-completion it may be preferable to include all the + available options. + + Closes #6280 - RELEASE-NOTES: synced -- ftp: avoid risk of reading uninitialized integers - - If the received PASV response doesn't match the expected pattern, we - could end up reading uninitialized integers for IP address and port - number. +- openssl: use OPENSSL_init_ssl() with >= 1.1.0 - Issue pointed out by muse.dev - Closes #5972 + Reported-by: Kovalkov Dmitrii and Per Nilsson + Fixes #6254 + Fixes #6256 + Closes #6260 -- [Quentin Balland brought this change] +- SECURITY-PROCESS: disclose on hackerone + + Once a vulnerability has been published, the hackerone issue should be + disclosed. For tranparency. + + Closes #6275 - easy_reset: clear retry counter +Marc Hoersken (3 Dec 2020) +- tests/util.py: fix compatibility with Python 2 - Closes #5975 - Fixes #5974 + Backporting the Python 3 implementation of setStream + to ClosingFileHandler as a fallback within Python 2. + + Reported-by: Jay Satiro + + Fixes #6259 + Closes #6270 -- ftp: get rid of the PPSENDF macro +Daniel Gustafsson (3 Dec 2020) +- docs: fix typos and markup in ETag manpage sections - The use of such a macro hides some of what's actually going on to the - reader and is generally disapproved of in the project. + Reported-by: emanruse on github + Fixes #6273 + +Daniel Stenberg (2 Dec 2020) +- quiche: close the connection - Closes #5971 + Reported-by: Junho Choi + Fixes #6213 + Closes #6217 -- man pages: switch to https://example.com URLs +Jay Satiro (2 Dec 2020) +- ngtcp2: Fix build error due to symbol name change - Since HTTPS is "the new normal", this update changes a lot of man page - examples to use https://example.com instead of the previous "http://..." + - NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION - Closes #5969 + ngtcp2/ngtcp2@76232e9 changed the name. + + ngtcp2 master is required to build curl with http3 support. + + Closes https://github.com/curl/curl/pull/6271 -- github: remove the duplicate "Security vulnerability" entry +Daniel Stenberg (1 Dec 2020) +- [Klaus Crusius brought this change] + + cmake: check for linux/tcp.h - ... since github adds an entry automatically by itself. + The HAVE_LINUX_TCP_H define was not set by cmake. - Closes #5970 + Closes #6252 -- [Emil Engler brought this change] +- NEW-PROTOCOL: document what needs to be done to add one + + Closes #6263 - github: use new issue template feature +- splay: rename Curl_splayremovebyaddr to Curl_splayremove - This helps us to avoid getting feature requests as well as security - bugs reported into the issue tracker. + ... and remove the old unused proto for the old Curl_splayremove + version. - Closes #5936 + Closes #6269 -- [Emil Engler brought this change] +- openssl: free mem_buf in error path + + To fix a memory-leak. + + Closes #6267 - urlapi: use more Curl_safefree +- openssl: remove #if 0 leftover - Closes #5968 + Follow-up to 4c9768565ec3a9 (from Sep 2008) + + Closes #6268 -Marc Hoersken (17 Sep 2020) -- multi: align WinSock mask variables in Curl_multi_wait +- ntlm: avoid malloc(0) on zero length user and domain - Also skip pre-checking sockets to set timeout_ms to 0 - after the first socket has been detected to be ready. + ... and simplify the too-long checks somewhat. - Reviewed-by: rcombs on github - Reviewed-by: Daniel Stenberg + Detected by OSS-Fuzz - Follow up to #5886 + Closes #6264 -- multi: reuse WinSock events variable in Curl_multi_wait +- RELEASE-NOTES: synced + +Marc Hoersken (28 Nov 2020) +- tests/server/tftpd.c: close upload file in case of abort - Since the struct is quite large (1 long and 10 ints) we - declare it once at the beginning of the function instead - of multiple times inside loops to avoid stack movements. + Commit c353207 removed the closing right after do_tftp + which covered the case of abort. This handles that case. - Reviewed-by: Viktor Szakats + Reviewed-by: Jay Satiro Reviewed-by: Daniel Stenberg - Closes #5886 + Follow up to #6209 + Closes #6234 -Daniel Stenberg (16 Sep 2020) -- TODO: dynamically decide to use socketpair +Daniel Stenberg (26 Nov 2020) +- [Daiki Ueno brought this change] + + ngtcp2: use the minimal version of QUIC supported by ngtcp2 - Suggested-by: Anders Bakken + Closes #6250 + +- [Daiki Ueno brought this change] + + ngtcp2: advertise h3 ALPN unconditionally - Closes #4829 + Closes #6250 -- TODO: add PR reference for native IDN support on macOS +- [Daiki Ueno brought this change] + + vquic/ngtcp2.h: define local_addr as sockaddr_storage - As there was work started on this that never got completed. + This field needs to be wide enough to hold sockaddr_in6 when + connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the + packets because of the address mismatch: + I00000022 [...] con ignore packet from unknown path - Closes #5371 - -- tool_help.h: update copyright year range + We can safely assume that struct sockaddr_storage is available, as it + is used in the public interface of ngtcp2. - Follow-up from aa8777f63febca + Closes #6250 -- CI/azure: disable test 571 in the msys2 builds +- socks: check for DNS entries with the right port number - It's just too flaky there + The resolve call is done with the right port number, but the subsequent + check used the wrong one, which then could find a previous resolve which + would return and leave the fresh resolve "incomplete" and leaking + memory. - Reviewed-by: Marc Hoersken - Closes #5954 + Fixes #6247 + Closes #6253 -- tool_writeout: protect fputs() from NULL +- curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use - When the code was changed to do fputs() instead of fprintf() it got - sensitive for NULL pointers; add checks for that. + ... so don't define it when instructed to use c-ares! + +- test506: make it not run in c-ares builds - Follow-up from 0c1e767e83ec66 + As the asynch nature of it may trigger events in another order. A c-ares + upgrade made it break. - Closes #5963 + Reported-by: Marc Hörsken + Fixes #6247 -- test3015: verify stdout "as text" - - Follow-up from 0c1e767e83e to please win32 tests +- runtests: make 'c-ares' a "feature" to depend on - Closes #5962 + ... also added to the docs. -- travis: use libressl v3.1.4 instead of master +- tool_writeout: use off_t getinfo-types instead of doubles - ... as their git master seems too fragile to use (and 3.2.1 which is the - latest has a build failure). + Commit 3b80d3ca46b12e52342 (June 2017) introduced getinfo replacement + variables that use curl_off_t instead of doubles. Switch the --write-out + function over to use them. - Closes #5964 + Closes #6248 -- tests/FILEFORMAT: document type=shell for +- [Emil Engler brought this change] -- tests/FILEFORMAT: document nonewline support for + file: avoid duplicated code sequence - The one in , that creates files. + file_disconnect() is identical with file_do() except the function header + but as the arguments are unused anyway so why not just return file_do() + directly! - Follow-up from b83947c8df7 + Reviewed-by: Daniel Stenberg + Closes #6249 -- [anio brought this change] +- [Rikard Falkeborn brought this change] - tool_writeout: add new writeout variable, %{num_headers} + infof/failf calls: fix format specifiers - This variable gives the number of headers. + Update a few format specifiers to match what is being printed. - Closes #5947 + Closes #6241 -- tool_urlglob: fix compiler warning "unreachable code" +- docs/INTERNALS: remove reference to Curl_sendf() - (On Windows builds.) + The function has been removed from common usage. Also removed comment in + gopher.c that still referenced it. - Follow-up to 70a3b003d9 + Reported-by: Rikard Falkeborn + Fixes #6242 + Closes #6243 -- [Gergely Nagy brought this change] +- [Rikard Falkeborn brought this change] - vtls: deduplicate client certificates in ssl_config_data + examples: update .gitignore - Closes #5629 - -- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND + Add files that are generated by 'make examples' and remove some that + have been renamed. - This is primarily interesting for cases where CURLOPT_NOBODY is set as - previously curl would not return an error for this case. + The commits that renamed the programs are e9625c5bc6c046a (imap.c and + simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and + ad39e7ec01e7 (pop3slist.c and pop3s.c were renamed to pop3-list.c and + pop3-ssl.c). - MDTM getting 550 now also returns this error (it returned - CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for - missing files across protocols and specific FTP commands. + Closes #6240 + +- asyn: use 'struct thread_data *' instead of 'void *' - libcurl already returns error on a 550 as a MDTM response (when - CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would - happen subsequently anyway since the RETR command would fail. + To reduce use of types that can't be checked at compile time. Also + removes several typecasts. - Add test 1913 and 1914 to verify. Updated several tests accordingly due - to the updated SIZE behavior. + ... and rename the struct field from 'os_specific' to 'tdata'. - Reported-by: Tomas Berger - Fixes #5953 - Closes #5957 + Closes #6239 + Reviewed-by: Jay Satiro -- curl: make checkpasswd use dynbuf +Viktor Szakats (23 Nov 2020) +- Makefile.m32: add support for UNICODE builds - Closes #5952 - -- curl: make glob_match_url use dynbuf + It requires the linker to support the `-municode` option. + This is available in more recent mingw-w64 releases. - Closes #5952 - -- curl: make file2memory use dynbuf + Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html + Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847 - Closes #5952 - -- curl: make file2string use dynbuf + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad - Closes #5952 - -- [Antarpreet Singh brought this change] + Closes #6228 - imap: set cselect_bits to CURL_CSELECT_IN initially - - ... when continuing a transfer from a FETCH response. +Daniel Stenberg (23 Nov 2020) +- urldata: remove 'void *protop' and create the union 'p' - When the size of the file was small enough that the entirety of the - transfer happens in a single go and schannel buffers holds the entire - data. However, it wasn't completely read in Curl_pp_readresp since a - line break was found before that could happen. So, by the time we are in - imap_state_fetch_resp - there's data in buffers that needs to be read - via Curl_read but nothing to read from the socket. After we setup a - transfer (Curl_setup_transfer), curl just waits on the socket state to - change - which doesn't happen since no new data ever comes. + ... to avoid the use of 'void *' for the protocol specific structs done + per transfer. - Closes #5961 - -- RELEASE-NOTES: synced + Closes #6238 -- test434: test -K use in a single line without newline +- winbuild: remove docs from Makefiles and refer to README.md - Closes #5946 - -- runtests: allow creating files without newlines + Reduce risk for conflicting docs and makes it to a single place to fix + and polish. - Closes #5946 - -- curl: use curlx_dynbuf for realloc when loading config files + add these missing options to the readme: - ... fixes an integer overflow at the same time. + ENABLE_OPENSSL_AUTO_LOAD_CONFIG and ENABLE_UNICODE - Reported-by: ihsinme on github - Assisted-by: Jay Satiro + clarify ENABLE_SCHANNEL default varies - Closes #5946 + Fixes #6216 + Closes #6227 + Co-Authored-by: Jay Satiro -- dynbuf: provide curlx_ names for reuse by the curl tool - - Closes #5946 +- [Daiki Ueno brought this change] -- dynbuf: make sure Curl_dyn_tail() zero terminates + http3: use the master branch of GnuTLS for testing - Closes #5959 + Closes #6235 -- tests: add test1912 to the dist +- KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation - Follow-up to 70984ce1be4cab6c + Closes #5839 -- docs/LICENSE-MIXING: remove - - This document is not maintained and I feel that it doesn't provide much - value to users anymore (if it ever did). +- KNOWN_BUGS: wakeup socket disconnect causes havoc - Closes #5955 + Closes #6132 + Closes #6133 -- [Laramie Leavitt brought this change] +- RELEASE-NOTES: synced - http: consolidate nghttp2_session_mem_recv() call paths - - Previously there were several locations that called - nghttp2_session_mem_recv and handled responses slightly differently. - Those have been converted to call the existing - h2_process_pending_input() function. - - Moved the end-of-session check to h2_process_pending_input() since the - only place the end-of-session state can change is after nghttp2 - processes additional input frames. - - This will likely fix the fuzzing error. While I don't have a root cause - the out-of-bounds read seems like a use after free, so moving the - nghttp2_session_check_request_allowed() call to a location with a - guaranteed nghttp2 session seems reasonable. +- [Oliver Urbann brought this change] + + curl: add compatibility for Amiga and GCC 6.5 - Also updated a few nghttp2 callsites to include error messages and added - a few additional error checks. + Changes are mainly reordering and adding of includes required + to compile with a more recent version of GCC. - Closes #5648 + Closes #6220 -- HISTORY: mention alt-svc added in 2019 +Marc Hoersken (20 Nov 2020) +- tests/server/tftpd.c: close upload file right after transfer - ... and make 1996 the first year subtitle - -- base64: also build for pop3 and imap + Make sure uploaded file is no longer locked after the + transfer while waiting for the final ACK to be handled. - Follow-up to the fix in 20417a13fb8f83 + Assisted-by: Daniel Stenberg - Reported-by: Michael Olbrich - Fixes #5937 - Closes #5948 + Bug: #6058 + Closes #6209 -- base64: enable in build with SMTP +- CI/cirrus: simplify logic for disabled tests - The oauth2 support is used with SMTP and it uses base64 functions. + The OpenSSH server instance for the testsuite cannot + be started on FreeBSD, therefore the SFTP and SCP + tests are disabled right away from the beginning. - Reported-by: Michael Olbrich - Fixes #5937 - Closes #5938 - -- curl_mime_headers.3: fix the example's use of curl_slist_append + The previous OS version specific logic for SKIP_TESTS + is no longer needed/used and can therefore be removed. - Reported-by: sofaboss on github - Fixes #5942 - Closes #5943 - -- lib583: fix enum mixup + Reviewed-by: Daniel Stenberg - grrr the previous follow-up to 17fcdf6a31 was wrong + Follow up to #6211 + Closes #6229 -- libtest: fix build errors +Daniel Gustafsson (20 Nov 2020) +- mailmap: Daniel Hwang - Follow-up from 17fcdf6a310d4c8076 + Add Daniel Hwang to the mailmap to cover the alternative spelling + Daniel Lee Hwang which was used in one commit. + + Closes #6230 + Reviewed-by: Daniel Stenberg -- lib: fix -Wassign-enum warnings +- openssl: guard against OOM on context creation - configure --enable-debug now enables -Wassign-enum with clang, - identifying several enum "abuses" also fixed. + EVP_MD_CTX_create will allocate memory for the context and returns + NULL in case the allocation fails. Make sure to catch any allocation + failures and exit early if so. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 + In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex + as the latter is intended for ENGINE selection which we don't do. - Closes #5929 - -- RELEASE-NOTES: synced + Closes #6224 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- [Diven Qi brought this change] +Daniel Stenberg (19 Nov 2020) +- [Vincent Torri brought this change] - url: use blank credentials when using proxy w/o username and password - - Fixes proxy regression brought in commit ad829b21ae (7.71.0) + cmake: use libcurl.rc in all Windows builds - Fixed #5911 - Closes #5914 + Reviewed-by: Marcel Raad + Closes #6215 -- travis: add a build using libressl (from git master) +- [Cristian Morales Vega brought this change] + + cmake: make CURL_ZLIB a tri-state variable - The v3.2.1 tag (latest release atm) results in a broken build. + By differentiating between ON and AUTO it can make a missing zlib + library a hard error when CURL_ZLIB=ON is used. - Closes #5932 + Reviewed-by: Jakub Zakrzewski + Closes #6221 + Fixes #6173 -- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 +- quiche: remove 'static' from local buffer - Unfortunately, this option is not detecting the same issues as clang's - -Wassign-enum flag, but should still be useful to detect future - mistakes. + For thread-safety - Closes #5930 + Closes #6223 -- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - - If the error reason from the lib is - SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return - CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. +- KNOWN_BUGS: cmake: libspsl is not supported - This unifies the libcurl return code and makes libressl run test 313 - (CRL testing) fine. + Closes #6214 + +- KNOWN_BUGS: cmake autodetects cert paths when cross-compiling - Closes #5934 + Closes #6178 -- FAQ: refreshed some very old language +- KNOWN_BUGS: cmake build doesn't fail if zlib not found + + Closes #6173 -- cmake: make HTTP_ONLY also disable MQTT +- KNOWN_BUGS: cmake libcurl.pc uses absolute library paths - ... and alphasort the order of disabling protocols to make it easier to - browse. + Closes #6169 + +- KNOWN_BUGS: cmake: generated .pc file contains strange entries - Closes #5931 + Closes #6167 -- libtest: remove lib1541 leftovers +- KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads - Caused automake errors. + Closes #6166 + +- KNOWN_BUGS: cmake build in Linux links libcurl to libdl - Follow-up to 8ca54a03ea08a + Closes #6165 -- tests/libtests: remove test 1900 and 2033 +- KNOWN_BUGS: make a new section for cmake topics - We already remove the test files, now remove the libtest codes as well. + Closes #6219 + +- [Emil Engler brought this change] + + cirrus: build with FreeBSD 12.2 in CirrusCI - Follow-up to e50a877df74 + Closes #6211 -Marc Hoersken (7 Sep 2020) -- CI/azure: add test number to title for display in analytics +Marc Hoersken (14 Nov 2020) +- tests/*server.py: close log file after each log line - To ease identification of tests the test number is added to - the test case title in order to have it on the Azure DevOps - Analytics pages and reports which currently do not show it. + Make sure the log file is not locked once a test has + finished and align with the behavior of our logmsg. - Bump test case revision to make Azure DevOps update titles. + Rename curl_test_data.py to be a general util.py. + Format and sort Python imports with isort/VSCode. - Closes #5927 + Bug: #6058 + Closes #6206 -Daniel Stenberg (6 Sep 2020) -- altsvc: clone setting in curl_easy_duphandle - - The cache content is not duplicated, like other caches, but the setting - and specified file name are. - - Test 1908 is extended to verify this somewhat. Since the duplicated - handle gets the same file name, the test unfortunately overwrites the - same file twice (with different contents) which makes it hard to check - automatically. +Daniel Stenberg (13 Nov 2020) +- CURLOPT_HSTS.3: document the file format - Closes #5923 + Closes #6205 -- test1541: remove since it is a known bug +- RELEASE-NOTES: synced + +- release-notes.pl: detect #[number] better for Ref: etc + +- curl: only warn not fail, if not finding the home dir - A shared connection cache is not thread-safe is a known issue. Stop - testing this until we believe this issue is addressed. Reduces - occasional test failures we don't care about. + ... as there's no good reason to error out completely. - The test code in lib1541.c is left in git to allow us to restore it when - we get to fix this. + Reported-by: Andreas Fischer + Fixes #6200 + Closes #6201 + +- httpput-postfields.c: new example doing PUT with POSTFIELDS - Closes #5922 + Proposed-by: Jeroen Ooms + Ref: #6186 + Closes #6188 -- tests: remove pipelining tests +- [Tobias Hieta brought this change] + + cmake: correctly handle linker flags for static libs - Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were - previously disabled. + curl CMake was setting the the EXE flags for static libraries which made + the /manifest:no flag ended up when linking the static library, which is + not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit + with an error. - The Pipelining code was removed from curl in commit 2f44e94efb3df8e, - April 2019. + The better way to handle this is to make sure that we pass the correct + linker flags to CMAKE_STATIC_LINKER_FLAGS instead. - Closes #5921 + Reviewed-by: Jakub Zakrzewski + Closes #6195 -- curl: retry delays in parallel mode no longer sleeps blocking - - The previous sleep for retries would block all other concurrent - transfers. Starting now, the retry will instead be properly marked to - not get restarted until after the delay time but other transfers can - still continue in the mean time. - - Closes #5917 +- [Tobias Hieta brought this change] -- curl:parallel_transfers: make sure retry readds the transfer + cmake: don't pass -fvisibility=hidden to clang-cl on Windows - Reported-by: htasta on github - Fixes #5905 - Closes #5917 - -- build: drop support for building with Watcom + When using clang-cl on windows -fvisibility=hidden is not an known + argument. Instead it behaves exactly like MSVC in this case. So let's + make sure we take that path. - These files are not maintained, they seem to have no users, Watcom - compilers look like not having users nor releases anymore. + In CMake clang-cl sets both CMAKE_C_COMPILER_ID=clang and MSVC get's + defined since clang-cl is basically a MSVC emulator. So guarding like we + do in this patch seems logical. - Closes #5918 + Reviewed-by: Jakub Zakrzewski + Closes #6194 -- winbuild/rundebug.cmd: remove +- http_proxy: use enum with state names for 'keepon' - Seems to have been added by mistake? Not included in dists. + To make the code clearer, change the 'keepon' from an int to an enum + with better state names. - Closes #5919 + Reported-by: Niranjan Hasabnis + Bug: https://curl.se/mail/lib-2020-11/0026.html + Closes #6193 -- curl: in retry output don't call all problems "transient" +- curl_easy_escape: limit output string length to 3 * max input - ... because when --retry-all-errors is used, the error isn't necessarily - transient at all. + ... instead of the limiting it to just the max input size. As every + input byte can be expanded to 3 output bytes, this could limit the input + string to 2.66 MB instead of the intended 8 MB. - Closes #5916 + Reported-by: Marc Schlatter + Closes #6192 -- easygetopt: pass a valid enum to avoid compiler warning +- docs: document the 8MB input string limit - "integer constant not in range of enumerated type 'CURLoption'" + for curl_easy_escape and curl_easy_setopt() - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 + The limit is there to catch mistakes and abuse. It is meant to be large + enough to allow virtually all "fine" use cases. - Closes #5915 - -- [Emil Engler brought this change] + Reported-by: Marc Schlatter + Fixes #6190 + Closes #6191 - tests: Add tests for new --help - - This commit is a part of "--help me if you can" +- mqttd: fclose test file when done - Closes #5680 + Reported-by: Marc Hörsken + Reviewed-by: Jay Satiro + Bug: #6058 + Closes #6189 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - tool: update --help with categories +- THANKS-filter: ignore autobuild links + +- Revert "libcurl.pc: make it relocatable" - This commit is a part of "--help me if you can" + This reverts commit 3862c37b6373a55ca704171d45ba5ee91dec2c9f. - Closes #5680 + That fix should either be done differently or with an option. + + Reported-by: asavah on github + Fixes #6157 + Closes #6183 -- [Emil Engler brought this change] +- examples/httpput: remove use of CURLOPT_PUT + + It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD. + + Reported-by: Jeroen Ooms + Fixes #6186 + Closes #6187 - docs: add categories to all cmdline opts - - Adapted gen.pl with 'listcats' +- Curl_pgrsStartNow: init speed limit time stamps at start - This commit is a part of "--help me if you can" + By setting the speed limit time stamps unconditionally at transfer + start, we can start off a transfer without speed limits and yet allow + them to get set during transfer and have an effect. - Closes #5680 - -- RELEASE-NOTES: synced - -- [ihsinme brought this change] + Reported-by: Kael1117 on github + Fixes #6162 + Closes #6184 - connect.c: remove superfluous 'else' in Curl_getconnectinfo +- ngtcp2: adapt to recent nghttp3 updates - Closes #5912 + 'reset_stream' was added to the nghttp3_conn_callbacks struct + + Closes #6185 -- [Samuel Marks brought this change] +- configure: pass -pthread to Libs.private for pkg-config + + Reported-by: Cristian Morales Vega + Fixes #6168 + Closes #6181 - CMake: remove explicit `CMAKE_ANSI_CFLAGS` +- altsvc: minimize variable scope and avoid "DEAD_STORE" - This variable was removed from cmake in commit - https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later - CMake commit removes the variable from the tests, claiming that it was - removed in CMake 2.6 + Closes #6182 + +- FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?" - Reviewed-By: Peter Wu - Closes #5439 + This hasn't been the case for a while now, remove. -- [cbe brought this change] +- FAQ: refresh "Why do I get "certificate verify failed" + + Add more details, remove references to ancient curl version. - libssh2: pass on the error from ssh_force_knownhost_key_type +- test493: verify --hsts upgrade and that %{url_effective} reflects that - Closes #5909 + Closes #6175 -- scripts/delta: add diffstat summary +- url: make sure an HSTS upgrade updates URL and scheme correctly - ... and make output more table-like + Closes #6175 -- [Martin Bašti brought this change] +- tool_operate: set HSTS with CURLOPT_HSTS to pass on filename + + Closes #6175 - http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set +- hsts: remove debug code leftovers - ... in case NO_PROXY takes an effect + Closes #6175 + +- FAQ: refreshed - Without this patch, the following command crashes: + - remove a few ancient questions + - add configure with static libs question + - updated wording in several places + - lowercased curl - $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ - git clone https://github.com/curl/curl.git + Closes #6177 + +Daniel Gustafsson (5 Nov 2020) +- examples: fix comment syntax - Minimal libcurl-based reproducer: + Commit ac0a88fd2 accidentally added a stray character outside of the + comment which broke compilation. Fix by removing. - #include + Reported-by: autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742 + +- hsts: Remove pointless call to free in errorpath - int main() { - CURL *curl = curl_easy_init(); - if(curl) { - CURLcode ret; - curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); - curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); - /* set the proxy type */ - curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); - curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); - curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); - ret = curl_easy_perform(curl); - curl_easy_cleanup(curl); - return ret; - } - return -1; - } + The line variable will always be NULL in the error path, so remove + the free call since it's pointless. - Assisted-by: Kamil Dudka - Bug: https://bugzilla.redhat.com/1873327 - Closes #5902 + Closes #6170 + Reviewed-by: Daniel Stenberg -- travis: add a CI job with openssl3 (from git master) +- docs: Fix various typos in documentation - Closes #5908 + Closes #6171 + Reviewed-by: Daniel Stenberg -- openssl: avoid error conditions when importing native CA - - The code section that is OpenSSL 3+ specific now uses the same logic as - is used in the version < 3 section. It caused a compiler error without - it. +Daniel Stenberg (5 Nov 2020) +- copyright: fix year ranges - Closes #5907 + Follow-up from 4d2f8006777 -- setopt: avoid curl_ on local variable - - Closes #5906 +- HISTORY: the new domain -- mqtt.c: avoid curl_ prefix on local variable +- curl.se: new home - Closes #5906 + Closes #6172 -- wildcard: strip "curl_" prefix from private symbols +- KNOWN_BUGS: FTPS with Schannel times out file list operation - Closes #5906 + Reported-by: bobmitchell1956 on github + Closes #5284 -- vtls: make it 'struct Curl_ssl_session' +- KNOWN_BUGS: SMB tests fail with Python 2 - Use uppercase C for internal symbols. + Reported-by: Jay Satiro + Closes #5983 + +- KNOWN_BUGS: LDAPS with NSS is slow - Closes #5906 + Reported-by: nosajsnikta on github + Closes #5874 -- curl_threads: make it 'struct Curl_actual_call' +Sergei Nikulov (4 Nov 2020) +- travis: use ninja-build for CMake builds - Internal names should not be prefixed "curl_" + Added package ninja-build to environment + Use ninja to speed up CMake builds - Closes #5906 + Closes #6077 -- schannel: make it 'struct Curl_schannel*' - - As internal global names should use captical C. +Daniel Stenberg (4 Nov 2020) +- [Harry Sintonen brought this change] + + rtsp: error out on empty Session ID, unified the code + +- [Harry Sintonen brought this change] + + rtsp: fixed the RTST Session ID mismatch in test 570 - Closes #5906 + Closes #6161 -- hash: make it 'struct Curl_hash' +- [Harry Sintonen brought this change] + + rtsp: fixed Session ID comparison to refuse prefix - As internal global names should use captical C. + Closes #6161 + +- RELEASE-NOTES: synced - Closes #5906 + (forgot to update the list of contributors) -- llist: make it "struct Curl_llist" +- RELEASE-NOTES: synced + +- curlver: bumped to 7.74.0 + +- hsts: add read/write callbacks - As internal global names should use captical C. + - read/write callback options + - man pages for the 4 new setopts + - test 1915 verifies the callbacks - Closes #5906 + Closes #5896 -Marc Hoersken (2 Sep 2020) -- telnet.c: depend on static requirement of WinSock version 2 - - Drop dynamic loading of ws2_32.dll and instead rely on the - imported version which is now required to be at least 2.2. +- hsts: add support for Strict-Transport-Security - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Reviewed-by: Viktor Szakats + - enable in the build (configure) + - header parsing + - host name lookup + - unit tests for the above + - CI build + - CURL_VERSION_HSTS bit + - curl_version_info support + - curl -V output + - curl-config --features + - CURLOPT_HSTS_CTRL + - man page for CURLOPT_HSTS_CTRL + - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) + - man page for --hsts + - save cache to disk + - load cache from disk + - CURLOPT_HSTS + - man page for CURLOPT_HSTS + - added docs/HSTS.md + - fixed --version docs + - adjusted curl_easy_duphandle - Closes #5854 + Closes #5896 -- win32: drop support for WinSock version 1, require version 2 +- [Sergei Nikulov brought this change] + + CI/tests: enable test target on TravisCI for CMake builds - IPv6, telnet and now also the multi API require WinSock - version 2 which is available starting with Windows 95. + Added test-nonflaky target to CMake builds - Therefore we think it is time to drop support for version 1. + Disabled test 1139 because the cmake build doesn't create docs/curl.1 - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Reviewed-by: Viktor Szakats + Closes #6074 + +- tool_debug_cb: do not assume zero-terminated data - Follow up to #5634 - Closes #5854 + Follow-up to d70a5b5a0f5e3 -- select: align poll emulation to return all relevant events +- sendf: move the verbose-check into Curl_debug - The poll emulation via select already consumes POLLRDNORM, - POLLWRNORM and POLLRDBAND as input events. Therefore it - should also return them as output events if signaled. + Saves us from having the same check done everywhere. - Also fix indentation in input event handling block. + Closes #6159 + +- travis: use valgrind when running tests for debug builds - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Except the non-x86 and sanitizer builds - Replaces #5852 - Closes #5883 + Closes #6154 -- CI/azure: MQTT is now enabled by default - - Reviewed-by: Daniel Stenberg +- header.d: fix syntax mistake - Follow up to #5858 - Closes #5903 + follow-up from 1144886f38fd0 -Daniel Stenberg (2 Sep 2020) -- copyright.pl: ignore buildconf +- [Harry Sintonen brought this change] -- test971: show test mismatches "inline" + gnutls: fix memory leaks (certfields memory wasn't released) + + Closes #6153 -- lib/Makefile.am: bump VERSIONINFO due to new functions +- tests: add missing global_init/cleanup calls - ... we're generally bad at this, but we are adding new functions for - this release. + Without the cleanup call in these test files, the mbedTLS backend leaks + memory. - Closes #5899 + Closes #6156 -- optiontable: use DEBUGBUILD +- tool_operate: --retry for HTTP 408 responses too - Follow-up to commit 6e18568ba38 (#5877) - -- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 + This was inadvertently dropped from the code when the parallel support + was added. - If there are more than two items in the list, use commas for all but the - last separator which is set to 'and'. Reads better. + Regression since b88940850 (7.66.0) - Closes #5898 + Reviewed-by: Jay Satiro + Closes #6155 -- curl.1: add see also no-progress-meter on two spots +- http: pass correct header size to debug callback for chunked post - Ref: #5894 + ... when the chunked framing was added, the size of the "body part" of + the data was calculated wrongly so the debug callback would get told a + header chunk a few bytes too big that would also contain the first few + bytes of the request body. - Closes #5897 + Reported-by: Dirk Wetter + Ref: #6144 + Closes #6147 -- RELEASE-NOTES: synced +- header.d: mention the "Transfer-Encoding: chunked" handling + + Ref: #6144 + Closes #6148 -- mqtt: enable by default +- acinclude: detect manually set minimum macos/ipod version - No longer considered experimental. + ... even if set in the CC or IPHONEOS/MACOSX_DEPLOYMENT_TARGET + variables. - Closes #5858 - -- [Michael Baentsch brought this change] + Reported-by: hamstergene on github + Fixes #6138 + Closes #6140 - tls: add CURLOPT_SSL_EC_CURVES and --curves +Jay Satiro (29 Oct 2020) +- tests: fix some http/2 tests for older versions of nghttpx - Closes #5892 + - Add regex that strips http/2 server header name to those http/2 tests + that don't already have it. + + - Improve that regex in all http/2 tests. + + Tests 358 and 359 were failing for me before this change on a system + that uses an older version of nghttpx which includes its version number + in the server header. + + Closes https://github.com/curl/curl/pull/6139 -- url: remove funny embedded comments in Curl_disonnect calls +Daniel Stenberg (30 Oct 2020) +- RELEASE-NOTES: synced -- [Chris Paulson-Ellis brought this change] +- [Cristian Morales Vega brought this change] - conn: check for connection being dead before reuse - - Prevents incorrect reuse of an HTTP connection that has been prematurely - shutdown() by the server. + configure: use pkgconfig to find openSSL when cross-compiling - Partial revert of 755083d00deb16 + This reverts 736a40fec (November 2004), which doesn't explain why it was + done. - Fixes #5884 - Closes #5893 + Closes #6145 -Marc Hoersken (29 Aug 2020) -- buildconf: exec autoreconf to avoid additional process - - Also make buildconf exit with the return code of autoreconf. +- tool_operate: bail out proper on errors for parallel setup - Reviewed-by: Daniel Stenberg + ... otherwise for example trying to upload a missing file just causes a + loop. - Follow up to #5853 - Closes #5890 + Reported-by: BrumBrum on hackerone + Closes #6141 -- CI/azure: no longer ignore results of test 1013 - - Follow up to #5771 - Closes #5889 +- [Sergei Nikulov brought this change] -- docs: add description about CI platforms to CONTRIBUTE.md + CMake: make BUILD_TESTING dependent option - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro + CMake will now handle BUILD_TESTING depending on PERL_FOUND and + CURL_DISABLE_TESTING - Closes #5882 + Ref: #6036 + Closes #6072 -Daniel Stenberg (29 Aug 2020) -- tests/getpart: use MIME::Base64 instead of home-cooked - - Since we already use the base64 package since a while back, we can just - as well switch to that here too. +- libssh2: fix transport over HTTPS proxy - It also happens to use the exact same function name, which otherwise - causes a run-time warning. + The fix in #6021 was not enough. This fix makes sure SCP/SFTP content + can also be transfered over a HTTPS proxy. - Reported-by: Marc Hörsken - Fixes #5885 - Closes #5887 + Fixes #6113 + Closes #6128 -Marcel Raad (29 Aug 2020) -- ntlm: fix condition for curl_ntlm_core usage +- curl.1: add an "OUTPUT" section at the top of the manpage - `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES - backend is fine, but was excluded before. + Explain the basic concepts behind curl output. - This also fixes test 1013 as the condition for SMB support in - configure.ac didn't match the condition in the source code. Now it - does. + Inspired by #6124 - Fixes https://github.com/curl/curl/issues/1262 - Closes https://github.com/curl/curl/pull/5771 + Closes #6134 -- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode +- mailmap: set Viktor Szakats's email + +- runtests: show keywords when no tests ran - The Schannel builds are the most useful to verify as they make the most - use of the Windows API. Classic MinGW doesn't support Unicode at all, - only MinGW-w64 and MSVC do. + To help out future debugging, runtests now outputs the list of keywords + when it fails because no tests ran. - Closes https://github.com/curl/curl/pull/5843 + Ref: #6120 + Closes #6126 -- CMake: add option to enable Unicode on Windows +Jay Satiro (26 Oct 2020) +- CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo - As already existing for winbuild. + Reported-by: Rui LIU - Closes https://github.com/curl/curl/pull/5843 + Closes https://github.com/curl/curl/issues/6131 -Marc Hoersken (29 Aug 2020) -- select: simplify return code handling for poll and select +- range.d: fix typo - poll and select already return -1 on error according to POSIX, - so there is no need to perform a <0 to -1 conversion in code. + Follow-up to 15ae039 from earlier today. + +Daniel Stenberg (26 Oct 2020) +- CI/github: work-around for brew breakage on macOS - Also we can just use one check with <= 0 on the return code. + ... and make it use OpenSSL 1.1 properly - Assisted-by: Daniel Stenberg - Reviewed-by: Jay Satiro + Fixes #6130 + Closes #6129 + +- [José Joaquín Atria brought this change] + + range.d: clarify that curl will not parse multipart responses - Replaces #5852 - Closes #5880 + Closes #6127 + Fixes #6124 -Daniel Stenberg (28 Aug 2020) - RELEASE-NOTES: synced -- [Jeroen Ooms brought this change] +- [Baruch Siach brought this change] - tests: add test1912 with typechecks + libssh2: fix build with disabled proxy support - Validates that gcc-typecheck macros match the new option type API. + Build breaks because the http_proxy field is missing: - Closes #5873 + vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy' + + Regression from #6021, shipped in curl 7.73.0 + + Closes #6125 -- easyoptions: provide debug function when DEBUGBUILD +- alt-svc: enable by default - ... not CURLDEBUG as they're not always set in conjunction. + Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported. - Follow-up to 6ebe63fac23f38df + alt-svc support in curl is no longer considered experimental - Fixes #5877 - Closes #5878 + Closes #5868 -Marc Hoersken (28 Aug 2020) -- sockfilt: handle FD_CLOSE winsock event on write socket +- CI/appveyor: remove (unused) runtests.pl -b option + +- [Emil Engler brought this change] + + tool_help: make "output" description less confusing - Learn from the way Cygwin handles and maps the WinSock events - to simulate correct and complete poll and select behaviour - according to Richard W. Stevens Network Programming book. + Currently the description of "output" is misleading when comparing it + "verbose". - Follow up to #5867 - Closes #5879 + Closes #6118 -- multi: handle connection state winsock events +- CI/appveyor: disable test 571 in two cmake builds - Learn from the way Cygwin handles and maps the WinSock events - to simulate correct and complete poll and select behaviour - according to Richard W. Stevens Network Programming book. + ... they're simply too flaky there. + + Closes #6119 + +- cmake: set the unicode feature in curl-config on Windows + + ... if built that way. To make it match curl -V output. - Reviewed-by: Jay Satiro Reviewed-by: Marcel Raad + Closes #6117 + +- libssh2: require version 1.0 or later - Follow up to #5634 - Closes #5867 + ... and simplify the code accordingly. libssh2 version 1.0 was released + in April 2009. + + Closes #6116 -Daniel Stenberg (28 Aug 2020) -- Curl_pgrsTime - return new time to avoid timeout integer overflow +- KNOWN_BUGS: mention the individual cmake issues - Setting a timeout to INT_MAX could cause an immediate error to get - returned as timeout because of an overflow when different values of - 'now' were used. + ... to make them easier to refer to and address separately and + one-by-one. + +- CMake: store IDN2 information in curl_config.h - This is primarily fixed by having Curl_pgrsTime() return the "now" when - TIMER_STARTSINGLE is set so that the parent function will continue using - that time. + This allows the build to enable IDN properly and it makes test 1014 + happier. - Reported-by: Ionuț-Francisc Oancea - Fixes #5583 - Closes #5847 + Ref: #6074 + Closes #6108 -- TLS: fix SRP detection by using the proper #ifdefs +- CMake: call the feature unixsockets without dash - USE_TLS_SRP will be true if *any* selected TLS backend can use SRP + ... so that curl-config gets correct and makes test 1014 happy! - HAVE_OPENSSL_SRP is defined when OpenSSL can use it + Ref: #6074 + Closes #6108 + +- CI/travis: add brotli and zstd to the libssh2 build - HAVE_GNUTLS_SRP is defined when GnuTLS can use it + ... to make sure such tests are run with valgrind. Suppress the zstd + valgrind warnings we get with version 1.3.3 on Ubuntu 18.04 (for debug + and non-debug builds). - Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is - set if at least one of the supported backends offers SRP. + Closes #6105 + +- runtests: revert the mistaken edit of $CURL - Reported-by: Stefan Strogin - Fixes #5865 - Closes #5870 + Regression from c4693adc62 -- [Dan Kenigsberg brought this change] +- RELEASE-NOTES: synced - docs: SSLCERTS: fix English syntax - - Signed-off-by: Dan Kenigsberg +- curl_url_set.3: fix typo in the RETURN VALUE section - Closes #5876 + Reported-by: Basuke Suzuki + Fixes #6102 -- [Alessandro Ghedini brought this change] +Jay Satiro (17 Oct 2020) +- [Daniel Stenberg brought this change] - docs: non-existing macros in man pages - - As reported by man(1) when invoked as: + packages/OS400: make the source code-style compliant - man --warnings -E UTF-8 -l -Tutf8 -Z >/dev/null + ... and make sure 'make checksrc' in the root dir also verifies the + packages/OS400 sources. - Closes #5846 - -- [Alessandro Ghedini brought this change] + Closes https://github.com/curl/curl/pull/6085 - curl.1: fix typo invokved -> invoked +- os400: Sync libcurl API options - Closes #5846 - -- buildconf: invoke 'autoreconf -fi' instead + This fixes the OS400 build and also an incorrect entry for + CURLINFO_APPCONNECT_TIME_T where it was treated as + CURLINFO_STARTTRANSFER_TIME_T. - The custom script isn't necessary anymore - but remains for simplicity - and just invokes autoreconf. + Reported-by: Jon Rumsey - Closes #5853 + Fixes https://github.com/curl/curl/issues/6083 + Closes https://github.com/curl/curl/pull/6084 -- [Emil Engler brought this change] +Daniel Stenberg (16 Oct 2020) +- CURLOPT_NOBODY.3: fix typo + + Reported-by: Basuke Suzuki + Fixes #6097 - lib: make Curl_gethostname accept a const pointer +Marc Hoersken (16 Oct 2020) +- CI/azure: improve on flakiness by avoiding libtool wrappers - The address of that variable never gets changed, only the data in it so - why not make it a "char * const"? + Install curl binaries into MinGW bin folder and use that + for the tests in order to avoid libtool wrapper binaries. - Closes #5866 - -- docs/libcurl: update "Added in" version for curl_easy_option* + The libtool wrapper binaries (not scripts) on Windows seem + to be one of the possible causes for the following issues: - Follow-up to 6ebe63fac23f38 - -- scripts: improve the "get latest curl release tag" logic + 1. Process output can be lost in the wrapper process chain. + 2. Killing the wrapper process does not kill the actual one. - ... by insiting on it matching "^curl-". + Derived from #5904 + Closes #6049 -- configure: added --disable-get-easy-options - - To allow disabling of the curl_easy_option APIs in a build. +Daniel Stenberg (16 Oct 2020) +- CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well + +- [Zenju brought this change] + + CURLOPT_TCP_NODELAY.3: fix comment in example code - Closes #5365 + Closes #6096 -- options: API for meta-data about easy options +- openssl: acknowledge SRP disabling in configure properly - const struct curl_easyoption *curl_easy_option_by_name(const char *name); + Follow-up to 68a513247409 - const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); + Use a new separate define that is the combination of both + HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP - const struct curl_easyoption * - curl_easy_option_next(const struct curl_easyoption *prev); + Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html - The purpose is to provide detailed enough information to allow for - example libcurl bindings to get option information at run-time about - what easy options that exist and what arguments they expect. + Closes #6094 + +Viktor Szakats (16 Oct 2020) +- http3: fix two build errors, silence warnings - Assisted-by: Jeroen Ooms - Closes #5365 + * fix two build errors due to mismatch between function + declarations and their definitions + * silence two mismatched signs warnings via casts + + Approved-by: Daniel Stenberg + Closes #6093 -- [Eric Curtin brought this change] +- Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 + + Approved-by: Daniel Stenberg + Closes #6092 - HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 +Daniel Stenberg (16 Oct 2020) +- tool_operate: fix compiler warning when --libcurl is disabled - Closes #5871 + Closes #6095 -- RELEASE-NOTES: synced +- checksrc: warn on empty line before open brace + + ... and fix a few occurances + + Closes #6088 -Jay Satiro (26 Aug 2020) -- openssl: Fix wincrypt symbols conflict with BoringSSL +- urlapi: URL encode a '+' in the query part - OpenSSL undefines the conflicting symbols but BoringSSL does not so we - must do it ourselves. + ... when asked to with CURLU_URLENCODE. - Reported-by: Samuel Tranchet - Assisted-by: Javier Blazquez + Extended test 1560 to verify. + Reported-by: Dietmar Hauser + Fixes #6086 + Closes #6087 + +- [Cristian Morales Vega brought this change] + + libcurl.pc: make it relocatable - Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 - Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 + It supposes when people specify the libdir/includedir they do it to + change where under prefix/exec_prefix it should be, not to make it + independent of prefix/exec_prefix. - Fixes https://github.com/curl/curl/issues/5669 - Closes https://github.com/curl/curl/pull/5857 + Closes #6061 -Daniel Stenberg (26 Aug 2020) -- socketpair: allow CURL_DISABLE_SOCKETPAIR +- runtests: return error if no tests ran - ... to completely disable the use of socketpair + ... and make TESTFAIL stand out a little better by adding newlines + before and after. - Closes #5850 + Reported-by: Marc Hörsken + Issue: #6052 + Closes #6053 -- curl_get_line: build only if cookies or alt-svc are enabled +- docs/FEATURE: convert to markdown - Closes #5851 + ... and clean it up a bit. + + Closes #6067 -- [fullincome brought this change] +- [Philipp Klaus Krause brought this change] - schannel: fix memory leak when using get_cert_location - - The get_cert_location function allocates memory only on success. - Previously get_cert_location was able to allocate memory and return - error. It wasn't obvious and in this case the memory wasn't - released. + strerror: use 'const' as the string should never be modified - Fixes #5855 - Closes #5860 + Closes #6068 -- [Emil Engler brought this change] +- [Jay Satiro brought this change] - git: ignore libtests in 3XXX area + connect: repair build without ipv6 availability - Currently the file tests/libtest/lib3010 is not getting - ignored by git. This fixes it by adding the 3XXX area to - the according .gitignore file. + Assisted-by: Daniel Stenberg + Reported-by: Tom G. Christensen - Closes #5859 + Fixes https://github.com/curl/curl/issues/6069 + Closes https://github.com/curl/curl/pull/6071 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced + + Started over for the journey to next release. - doh: add error message for DOH_DNS_NAME_TOO_LONG +- src/tool_filetime: disable -Wformat on mingw for this file - When this error code was introduced in b6a53fff6c1d07e8a9, it was - forgotten to be added in the errors array and doh_strerror function. + With gcc 10 on mingw we otherwise get this warning: - Closes #5863 + error: ISO C does not support the 'I' printf flag [-Werror=format=] + + Fixes #6079 + Closes #6082 -- ngtcp2: adapt to the new pkt_info arguments +- test122[12]: remove these two tests - Guidance-by: Tatsuhiro Tsujikawa + ... and remove the objnames scripts they tested. They're not used for + anything anymore so testing them serves no purpose! - Closes #5864 + Reported-by: Marc Hörsken + Fixes #6080 + Closes #6081 -- winbuild/README.md: make visible - - Follow-up to be753add31c2d8c +Version 7.73.0 (14 Oct 2020) -- winbuild: convert the instruction text to README.md +Daniel Stenberg (14 Oct 2020) +- RELEASE-NOTES: synced - Closes #5861 + for 7.73.0 -- lib1560: verify "redirect" to double-slash leading URL - - Closes #5849 +- THANKS: from 7.73.0 and .mailmap fixes -Marc Hoersken (25 Aug 2020) -- multi: expand pre-check for socket readiness +- mailmap: fixups of some contributors + +- projects/build-wolfssl.bat: fix the copyright year range + +Marc Hoersken (14 Oct 2020) +- [Sergei Nikulov brought this change] + + CI/tests: fix invocation of tests for CMake builds + + Update appveyor.yml to set env variable TFLAGS and run tests + Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) + Move testdeps build to build step (per review comments) - Check readiness of all sockets before waiting on them - to avoid locking in case the one-time event FD_WRITE - was already consumed by a previous wait operation. + Reviewed-by: Marc Hörsken - More information about WinSock network events: - https://docs.microsoft.com/en-us/windows/win32/api/ - winsock2/nf-winsock2-wsaeventselect#return-value + Closes #6066 + Fixes #6052 + +- tests/server/util.c: fix support for Windows Unicode builds - Closes #5634 + Detected via #6066 + Closes #6070 -- [rcombs brought this change] +Daniel Stenberg (13 Oct 2020) +- [Jay Satiro brought this change] - multi: implement wait using winsock events + strerror: Revert to local codepage for Windows error string - This avoids using a pair of TCP ports to provide wakeup functionality - for every multi instance on Windows, where socketpair() is emulated - using a TCP socket on loopback which could in turn lead to socket - resource exhaustion. + - Change get_winapi_error() to return the error string in the local + codepage instead of UTF-8 encoding. - A previous version of this patch failed to account for how in WinSock, - FD_WRITE is set only once when writing becomes possible and not again - until after a send has failed due to the buffer filling. This contrasts - to how FD_READ and FD_OOB continue to be set until the conditions they - refer to no longer apply. This meant that if a user wrote some data to - a socket, but not enough data to completely fill its send buffer, then - waited on that socket to become writable, we'd erroneously stall until - their configured timeout rather than returning immediately. + Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it + also changed the error string's encoding from local codepage to UTF-8. - This version of the patch addresses that issue by checking each socket - we're waiting on to become writable with select() before the wait, and - zeroing the timeout if it's already writable. + We return the local codepage version of the error string because if it + is output to the user's terminal it will likely be with functions which + expect the local codepage (eg fprintf, failf, infof). - Assisted-by: Marc Hörsken - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Tested-by: Gergely Nagy - Tested-by: Rasmus Melchior Jacobsen - Tested-by: Tomas Berger + This is essentially a partial revert of bed5f84. The support for xbox + remains but the error string is reverted back to local codepage. - Replaces #5397 - Reverts #5632 - Closes #5634 + Ref: https://github.com/curl/curl/pull/6005 + + Reviewed-by: Marcel Raad + Closes #6065 -- select: reduce duplication of Curl_poll in Curl_socket_check +Marc Hoersken (13 Oct 2020) +- CI/tests: use verification curl for test reporting APIs - Change Curl_socket_check to use select-fallback in Curl_poll - instead of implementing it in Curl_socket_check and Curl_poll. + Avoid using our own, potentially installed, curl for + the test reporting APIs in case it is broken. Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - Replaces #5262 and #5492 - Closes #5707 + Preparation for #6049 + Closes #6063 -- select: fix poll-based check not detecting connect failure - - This commit changes Curl_socket_check to use POLLPRI to - check for connect failure on the write socket, because - POLLPRI maps to fds_err. This is in line with select(2). +Viktor Szakats (12 Oct 2020) +- windows: fix comparison of mismatched types warning - The select-based socket check correctly checks for connect - failures by adding the write socket also to fds_err. + clang 10, mingw-w64: + ``` + vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') + [-Wsign-compare] + if(GetLastError() != CRYPT_E_NOT_FOUND) + ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ + ``` - The poll-based implementation (which internally can itself - fallback to select again) did not previously check for - connect failure by using POLLPRI with the write socket. + Approved-by: Daniel Stenberg + Closes #6062 + +Daniel Stenberg (11 Oct 2020) +- [Viktor Szakats brought this change] + + src/Makefile.m32: fix undefined curlx_dyn_* errors - See the follow up commit to this for more information. + by linking `lib/dynbuf.c` when building a static curl binary. + Previously this source file was only included when building + a dynamic curl binary. This was likely possibly because no + functions from the `src/Makefile.inc` / `CURLX_CFILES` sources + were actually required for a curl tool build. This has + recently changed with the introduction of `curlx_dyn_*()` + memory functions and their use by the tool sources. - This commit makes sure connect failures can be detected - and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. + Closes #6060 + +- HISTORY: curl verifies SSL certs by default since version 7.10 + +Marc Hoersken (8 Oct 2020) +- runtests.pl: use $LIBDIR variable instead of hardcoded path Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro - - Replaces #5509 - Prepares #5707 + Closes #6051 -- select.h: make socket validation macros test for INVALID_SOCKET - - With Winsock the valid range is [0..INVALID_SOCKET-1] according to - https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 +Daniel Stenberg (7 Oct 2020) +- checksrc: detect // comments on column 0 - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg + Spotted while working on #6045 - Closes #5760 + Closes #6048 -Daniel Stenberg (24 Aug 2020) -- docs: --output-dir is added in 7.73.0, nothing else - - Follow-up to 5620d2cc78c0 +- [Frederik Wedel-Heinen brought this change] -- curl: add --output-dir - - Works with --create-dirs and with -J - - Add test 3008, 3009, 3011, 3012 and 3013 to verify. + mbedtls: add missing header when defining MBEDTLS_DEBUG - Closes #5637 + Closes #6045 -- configure: fix pkg-config detecting wolfssl +- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long - When amending the include path with "/wolfssl", this now properly strips - off all whitespace from the path variable! Previously this would lead to - pkg-config builds creating bad command lines. + Previously, it would pass on a define (int) which could make libcurl + read junk as a value - which prevented the CURLOPT_IPRESOLVE option to + "take". This could then make test 2100 do two DoH requests instead of + one! - Closes #5848 + Fixes #6042 + Closes #6043 -- [Michael Musset brought this change] +- RELEASE-NOTES: synced - sftp: add the option CURLKHSTAT_FINE_REPLACE +- scripts/release-notes.pl: don't "embed" $ in format string for printf() - Replace the old fingerprint of the host with a new. + ... since they might contain %-codes that mess up the output! + +Jay Satiro (5 Oct 2020) +- [M.R.T brought this change] + + build-wolfssl: fix build with Visual Studio 2019 - Closes #5685 + Closes https://github.com/curl/curl/pull/6033 -- RELEASE-NOTES: synced +Daniel Stenberg (4 Oct 2020) +- runtests: add %repeat[]% for test files - The next release is now to become 7.73.0 + ... and use this new keywords in all the test files larger than 50K to reduce + their sizes and make them a lot easier to read and understand. + + Closes #6040 -- checksrc: verify do-while and spaces between the braces +- [Emil Engler brought this change] + + --help: move two options from the misc category - Updated mprintf.c to comply + The cmdline opts delegation and suppress-connect-headers + fit better into auth and proxy rather than misc. - Closes #5845 + Follow-up to aa8777f63febc + Closes #6038 -- curl: support XDG_CONFIG_HOME to find .curlrc +- [Samanta Navarro brought this change] + + docs/opts: fix typos in two manual pages - Added test433 to verify. Updated documentation. + Closes #6039 + +- ldap: reduce the amount of #ifdefs needed - Reviewed-by: Jay Satiro - Suggested-by: Eli Schwartz - Fixes #5829 - Closes #5837 + Closes #6035 -- etag: save and use the full received contents +- runtests: provide curl's version string as %VERSION for tests - ... which makes it support weak tags and non-standard etags too! + ... so that we can check HTTP requests for User-Agent: curl/%VERSION - Added test case 347 to verify blank incoming ETag: + Update 600+ test cases accordingly. - Fixes #5610 - Closes #5833 + Closes #6037 -- setopt: if the buffer exists, refuse the new BUFFERSIZE - - The buffer only exists during transfer and then we shouldn't change the - size (the setopt is not documented to work then). +- checksrc: warn on space after exclamation mark - Reported-by: Harry Sintonen - Closes #5842 + Closes #6034 -- [COFFEETALES brought this change] +- test1465: verify --libcurl with binary POST data - sftp: add new quote commands 'atime' and 'mtime' - - Closes #5810 +- runtests: allow generating a binary sequence from hex -- CURLE_PROXY: new error code +- tool_setopt: escape binary data to hex, not octal + +- curl: make --libcurl show binary posts correctly - Failures clearly returned from a (SOCKS) proxy now causes this return - code. Previously the situation was not very clear as what would be - returned and when. + Reported-by: Stephan Mühlstrasser + Fixes #6031 + Closes #6032 + +Jay Satiro (1 Oct 2020) +- strerror: fix null deref on winapi out-of-memory - In addition: when this error code is returned, an application can use - CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then - returns a value from the new 'CURLproxycode' enum. + Follow-up to bed5f84 from several days ago. - Closes #5770 + Ref: https://github.com/curl/curl/pull/6005 -- runtests: make cleardir() erase dot files too +Daniel Stenberg (1 Oct 2020) +- [Kamil Dudka brought this change] + + vtls: deduplicate some DISABLE_PROXY ifdefs - Because test cases might use dot files. + ... in the code of gtls, nss, and openssl - Closes #5838 + Closes #5735 + +- RELEASE-NOTES: synced + +- [Emil Engler brought this change] + + TODO: Add OpenBSD libtool notice + + See #5862 + Closes #6030 -- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses +- tests/unit/README: convert to markdown - Also: the current behavior is now documented in the curl.1 and - CURLOPT_NOPROXY.3 man pages. + ... and add to dist! - Reported-by: Andrew Barnes - Closes #5745 - Closes #5841 + Closes #6028 -Viktor Szakats (22 Aug 2020) -- Makefile.m32: add ability to override zstd libs [ci skip] +- tests/README: convert to markdown - Similarly to brotli, where this was already possible. - E.g. it allows to link zstd statically to libcurl.dll. + Closes #6028 + +- include/README: convert to markdown - Ref: https://github.com/curl/curl-for-win/issues/12 - Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 + Closes #6028 + +- examples/README: convert to markdown - Closes https://github.com/curl/curl/pull/5840 + Closes #6028 -Daniel Stenberg (21 Aug 2020) -- runtests: avoid 'fail to start' repeated messages in attempt loops +- configure: don't say HTTPS-proxy is enabled when disabled! - Closes #5834 + Reported-by: Kamil Dudka + Reviewed-by: Kamil Dudka + Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 + Closes #6029 -- runtests: clear pid variables when failing to start a server +Daniel Gustafsson (30 Sep 2020) +- src: Consistently spell whitespace without whitespace - ... as otherwise the parent doesn't detect the failure and believe it - actually worked to start. + Whitespace is spelled without a space between white and space, so + make sure to consistently spell it that way across the codebase. - Reported-by: Christian Weisgerber - Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html - Closes #5834 + Closes #6023 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- TODO: Virtual external sockets +- MANUAL: update examples to resolve without redirects - Closes #5835 + www.netscape.com is redirecting to a cookie consent form on Aol, and + cool.haxx.se isn't responding to FTP anymore. Replace with examples + that resolves in case users try out the commands when reading the + manual. + + Closes #6024 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- [Don J Olmstead brought this change] +Daniel Stenberg (30 Sep 2020) +- HISTORY: add some 2020 events - dist: add missing CMake Find modules to the distribution +- sectransp: make it build with --disable-proxy - Closes #5836 + Follow-up from #5466 and f3d501dc678d80 + Reported-by: Javier Navarro + Fixes #6025 + Closes #6026 -- RELEASE-NOTES: synced +- ECH: renamed from ESNI in docs and configure - ... and version bumped to 7.72.1 - -- tls: provide the CApath verbose log on its own line + Encrypted Client Hello (ECH) is the current name. - ... not newline separated from the previous line. This makes it output - asterisk prefixed properly like other verbose putput! + Closes #6022 + +- configure: use "no" instead of "disabled" for the end summary - Reported-by: jmdavitt on github - Fixes #5826 - Closes #5827 + ... for consistency but also to make them more distinctly stand out next + to the "enabled" lines. -Version 7.72.0 (19 Aug 2020) +- TODO: SSH over HTTPS proxy with more backends + + ... as right now only the libssh2 backend supports it. -Daniel Stenberg (19 Aug 2020) -- RELEASE-NOTES: synced +- libssh2: handle the SSH protocols done over HTTPS proxy - The curl 7.72.0 release + Reported-by: Robin Douine + Fixes #4295 + Closes #6021 -- THANKS: add names from curl 7.72.0 release +- [Emil Engler brought this change] -Jay Satiro (18 Aug 2020) -- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions + memdebug: remove 9 year old unused debug function - Reported-by: plujon@users.noreply.github.com + There used to be a way to have memdebug fill allocated memory. 9 years + later this has no value there (valgrind and ASAN etc are way better). If + people need to know about it they can have a look at VCS logs. - Closes https://github.com/curl/curl/issues/5488 + Closes #5973 -Daniel Stenberg (17 Aug 2020) -- Curl_easy: remember last connection by id, not by pointer +- sendf: move Curl_sendf to dict.c and make it static - CVE-2020-8231 + ... as the only remaining user of that function. Also fix gopher.c to + instead use Curl_write() - Bug: https://curl.haxx.se/docs/CVE-2020-8231.html + Closes #6020 + +- ROADMAP: updates and cleanups - Reported-by: Marc Aldorasi - Closes #5824 + Fix the HSTS PR + + Remove DoT, thread-safe init and hard-coded localhost. I feel very + little interest for these with users so I downgrade them to plain "TODO" + entries again. -- examples/rtsp.c: correct the copyright year +- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root + + This matches what is returned in other TLS backends in the same + situation. + + Reviewed-by: Jay Satiro + Reviewed-by: Emil Engler + Follow-up to 5a3efb1 + Reported-by: iammrtau on github + Fixes #6003 + Closes #6018 -- RELEASE-PROCEDURE.md: add more future release dates +- RELEASE-NOTES: synced -- [H3RSKO brought this change] +- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL + + Added test 348 to verify. Added a 'STOR' command to the test FTP + server to enable test 348. Documented the command in FILEFORMAT.md + + Reported-by: Duncan Wilcox + Fixes #6016 + Closes #6017 - docs: change "web site" to "website" +- pause: only trigger a reread if the unpause sticks - According to wikipedia: + As an unpause might itself get paused again and then triggering another + reread doesn't help. - While "web site" was the original spelling, this variant has become - rarely used, and "website" has become the standard spelling + Follow-up from e040146f22608fd9 (shipped since 7.69.1) - Closes #5822 - -- [Bevan Weiss brought this change] + Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html + Patch-by: Kunal Chandarana + Fixes #5988 + Closes #6013 - CMake: don't complain about missing nroff +- test163[12]: require http to be built-in to run - The curl_nroff_check() was always being called, and complaining if - *NROFF wasn't found, even when not making the manual. + ... as speaking over an HTTPS proxy implies http! - Only check for nroff (and complain) if actually making the manual + Closes #6014 + +- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define - Closes #5817 + Closes #6012 -- [Brian Inglis brought this change] +- [Javier Blazquez brought this change] - libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin + strerror: honor Unicode API choice on Windows - copy the LDFLAGS approach for adding same option with `libhostname` in - `libtest/Makefile.am`: + Closes #6005 + +- imap: make imap_send use dynbuf for the send buffer management - - init `libstubgss_la_LDFLAGS_EXTRA` variable, - - add option to variable inside conditional, - - use variable in `libstubgss_la_LDFLAGS` + Reuses the buffer and thereby reduces number of mallocs over a transfer. - Fixes #5819 - Closes #5820 + Closes #6010 -- docs: clarify MAX_SEND/RECV_SPEED functionality +- Curl_send: return error when pre_receive_plain can't malloc - ... in particular what happens if the maximum speed limit is set to a - value that's smaller than the transfer buffer size in use. + ... will probably trigger some false DEAD CODE positives on non-windows + code analyzers for the conditional code. - Reported-by: Tomas Berger - Fixes #5788 - Closes #5813 + Closes #6011 -- test1140: compare stdout +- ftp: separate FTPS from FTP over "HTTPS proxy" - To make problems more immediately obvious when tests fail. + When using HTTPS proxy, SSL is used but not in the view of the FTP + protocol handler itself so separate the connection's use of SSL from the + FTP control connection's sue. - Closes #5814 + Reported-by: Mingtao Yang + Fixes #5523 + Closes #6006 -- asyn-ares: correct some bad comments +Dan Fandrich (23 Sep 2020) +- tests/data: Fix some mismatched XML tags in test cases - Closes #5812 - -- [Emil Engler brought this change] + This allows these test files to pass xmllint. - docs: Add video link to docs/CONTRIBUTE.md +Daniel Stenberg (23 Sep 2020) +- pingpong: use a dynbuf for the *_pp_sendf() function - Closes #5811 - -- curl-config: ignore REQUIRE_LIB_DEPS in --libs output + ... reuses the same dynamic buffer instead of doing repeated malloc/free + cycles. - Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is - not considered for the --libs output. + Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls + after this change in my test setup (132 => 125), curl 7.72.0 needed 140 + calls for this. - Reported-by: ramsay-jones on github - Assisted-by: Brian Inglis and Ken Brown - Fixes #5793 - Closes #5808 + Test case 103 makes 9 less allocations now (130). Down from 149 in + 7.72.0. + + Closes #6004 -- copyright: update/correct the year range on a few files +- dynbuf: add Curl_dyn_vaddf + + Closes #6004 -- scripts/copyright.pl: ignore .muse files +- dynbuf: make *addf() not require extra mallocs + + ... by introducing a printf() function that appends directly into a + dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if + the buffer is already big enough it can just printf directly into it. + + Since this less-malloc version requires tthe use of a library internal + printf function, we only provide this version when building libcurl and + not for the dynbuf code that is used when building the curl tool. + + Closes #5998 -- [Emil Engler brought this change] +- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport + + Closes #5403 - multi: Remove 10-year old out-commented code +- pingpong: remove a malloc per Curl_pp_vsendf call - The code hasn't been touched since 2010-08-18 + This typically makes 7-9 fewer mallocs per FTP transfer. - Closes #5805 + Closes #5997 -- KNOWN_BUGS: A shared connection cache is not thread-safe +- symbian: drop support - Closes #4915 - Closes #5802 - -- CONTRIBUTE: extend git commit message description + The OS is deprecated. I see no traces of anyone having actually built + curl for Symbian after 2012. - In particular how the first line works. + The public headers are unmodified. - Closes #5803 + Closes #5989 - RELEASE-NOTES: synced -- [Stefan Yohansson brought this change] - - transfer: move retrycount from connect struct to easy handle +- curl_krb5.h: rename from krb5.h - This flag was applied to the connection struct that is released on - retry. These changes move the retry counter into Curl_easy struct that - lives across retries and retains the new connection. + Follow-up from f4873ebd0be32cf - Reported-by: Cherish98 on github - Fixes #5794 - Closes #5800 + Turns out some older openssl installations go bananas otherwise. + Reported-by: Tom van der Woerdt + Fixes #5995 + Closes #5996 -- libssh2: s/ssherr/sftperr/ +- test1297: verify GOT_NOTHING with http proxy tunnel + +- http_proxy: do not count proxy headers in the header bytecount - The debug output used ssherr instead of sftperr which not only outputs - the wrong error code but also casues a warning on Windows. + ... as that counter is subsequently used to detect if nothing was + returned from the peer. This made curl return CURLE_OK when it should + have returned CURLE_GOT_NOTHING. - Follow-up to 7370b4e39f1 + Fixes #5992 + Reported-by: Tom van der Woerdt + Closes #5994 + +- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 - Closes #5799 + Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the + option is, yeah, not known. Clarified this in the setopt man page too. + + Closes #5993 -- ftp: don't do ssl_shutdown instead of ssl_close +- krb5: merged security.c and krb specific FTP functions in here - The shutdown function is for downgrading a connection from TLS to plain, - and this is not requested here. + These two files were always tightly connected and it was hard to + understand what went into which. This also allows us to make the + ftpsend() function static (moved from ftp.c). - Have ssl_close reset the TLS connection state. + Removed security.c + Renamed curl_sec.h to krb5.h - This partially reverts commit f002c850d98d + Closes #5987 + +- Curl_handler: add 'family' to each protocol - Reported-by: Rasmus Melchior Jacobsen - Reported-by: Denis Goleshchikhin - Fixes #5797 + Makes get_protocol_family() faster and it moves the knowledge about the + "families" to each protocol handler, where it belongs. + + Closes #5986 -Marc Hoersken (9 Aug 2020) -- CI/azure: fix test outcome values and use latest API version +- parsedate: tune the date to epoch conversion - This makes sure that tests ignored or skipped are not shown - just in the category "Other", but with their correct state. + By avoiding an unnecessary error check and the temp use of the tm + struct, the time2epoch conversion function gets a little bit faster. + When repeating test 517, the updated version is perhaps 1% faster (on + one particular build on one particular architecture). - Closes #5796 + Closes #5985 -- CI/azure: show runtime stats to investigate slowness +- cmake: remove scary warning - Also avoid naming conflict of TFLAGS env and tflags variables. + Remove the text saying - Closes #5776 + "the curl cmake build system is poorly maintained. Be aware" + + ... not because anything changed just now, but to encourage users to use + it and subsequently improve it. + + Closes #5984 -Daniel Stenberg (8 Aug 2020) -- TLS naming: fix more Winssl and Darwinssl leftovers +- docs/MQTT: remove outdated paaragraphs + +- docs/MQTT: not experimental anymore - The CMake option is now called CMAKE_USE_SCHANNEL + Follow-up to e37e4468688d8f + +- docs/RESOURCES: remove - The winbuild flag is USE_SCHANNEL + This document is not maintained and rather than trying to refresh it, + let's kill it. A more up-to-date document with relevant RFCs is this + page on the curl website: https://curl.haxx.se/rfc/ - The CI jobs and build scripts only use the new names and the new name - options + Closes #5980 + +- docs/TheArtOfHttpScripting: convert to markdown - Tests now require 'Schannel' (when necessary) + Makes it easier to browse on github etc. Offers (better) links. - Closes #5795 + It should be noted that this document is already mostly outdated and + "Everything curl" at https://ec.haxx.se/ is a better resource and + tutorial. + + Closes #5981 -- smtp_parse_address: handle blank input string properly +- BUGS: convert document to markdown - Closes #5792 + Closes #5979 -- runtests: run the DICT server on a random port number +- --help: strdup the category - Removed support for -b (base port number) + ... since it is converted and the original pointer is freed on Windows + unicode handling. - Closes #5783 + Follow-up to aa8777f63febc + Fixes #5977 + Closes #5978 + Reported-by: xwxbug on github + +- CHECKSRC: document two missing warnings - RELEASE-NOTES: synced -- runtests: move the TELNET server to a dynamic port +- ftp: avoid risk of reading uninitialized integers - Rename the port variable to TELNETPORT to better match the existing - pattern. + If the received PASV response doesn't match the expected pattern, we + could end up reading uninitialized integers for IP address and port + number. - Closes #5785 + Issue pointed out by muse.dev + Closes #5972 -- ngtcp2: adapt to error code rename - - Closes #5786 +- [Quentin Balland brought this change] -- runtests: move the smbserver to use a dynamic port number + easy_reset: clear retry counter - Closes #5782 + Closes #5975 + Fixes #5974 -- runtests: run the http2 tests on a random port number +- ftp: get rid of the PPSENDF macro - Closes #5779 - -- gtls: survive not being able to get name/issuer + The use of such a macro hides some of what's actually going on to the + reader and is generally disapproved of in the project. - Closes #5778 + Closes #5971 -- runtests: move the gnutls-serv tests to a dynamic port +- man pages: switch to https://example.com URLs - Affects test 320, 321, 322 and 324. + Since HTTPS is "the new normal", this update changes a lot of man page + examples to use https://example.com instead of the previous "http://..." - Closes #5778 + Closes #5969 -- runtests: support dynamicly base64 encoded sections in tests +- github: remove the duplicate "Security vulnerability" entry - This allows us to make test cases to use base64 at run-time and still - use and verify information determined at run-time, such as the IMAP test - server's port number in test 842. + ... since github adds an entry automatically by itself. - This change makes 12 tests run again that basically never ran since we - moved to dynamic port numbers. + Closes #5970 + +- [Emil Engler brought this change] + + github: use new issue template feature - ftpserver.pl is adjusted to load test instructions and test number from - the preprocessed test file. + This helps us to avoid getting feature requests as well as security + bugs reported into the issue tracker. - FILEFORMAT.md now documents the new base64 encoding syntax. + Closes #5936 + +- [Emil Engler brought this change] + + urlapi: use more Curl_safefree - Reported-by: Marcel Raad - Fixes #5761 - Closes #5775 + Closes #5968 -- curl.1: add a few missing valid exit codes +Marc Hoersken (17 Sep 2020) +- multi: align WinSock mask variables in Curl_multi_wait - 93 - 96 can be returned as well. + Also skip pre-checking sockets to set timeout_ms to 0 + after the first socket has been detected to be ready. - Closes #5777 - -- TODO: Use multiple parallel transfers for a single download + Reviewed-by: rcombs on github + Reviewed-by: Daniel Stenberg - Closes #5774 + Follow up to #5886 -- TODO: Set the modification date on an uploaded file +- multi: reuse WinSock events variable in Curl_multi_wait - Closes #5768 - -- [Thomas M. DuBuisson brought this change] - - CI: Add muse CI config + Since the struct is quite large (1 long and 10 ints) we + declare it once at the beginning of the function instead + of multiple times inside loops to avoid stack movements. - Closes #5772 + Reviewed-by: Viktor Szakats + Reviewed-by: Daniel Stenberg + + Closes #5886 -- [Thomas M. DuBuisson brought this change] +Daniel Stenberg (16 Sep 2020) +- TODO: dynamically decide to use socketpair + + Suggested-by: Anders Bakken + + Closes #4829 - travis/script.sh: fix use of `-n' with unquoted envvar +- TODO: add PR reference for native IDN support on macOS - Shellcheck tells us "-n doesn't work with unquoted arguments. quote or - use [[ ]]." + As there was work started on this that never got completed. - And testing shows: + Closes #5371 + +- tool_help.h: update copyright year range - ``` - docker run --rm -it ubuntu bash - root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" - I ran - root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" - root@fe85ce156856:/# - ``` + Follow-up from aa8777f63febca + +- CI/azure: disable test 571 in the msys2 builds + + It's just too flaky there - Closes #5773 + Reviewed-by: Marc Hoersken + Closes #5954 -- h2: repair trailer handling +- tool_writeout: protect fputs() from NULL - The previous h2 trailer fix in 54a2b63 was wrong and caused a - regression: it cannot deal with trailers immediately when read since - they may be read off the connection by the wrong 'data' owner. + When the code was changed to do fputs() instead of fprintf() it got + sensitive for NULL pointers; add checks for that. - This change reverts the logic back to gathering all trailers into a - single buffer, like before 54a2b63. + Follow-up from 0c1e767e83ec66 - Reported-by: Tadej Vengust - Fixes #5663 - Closes #5769 + Closes #5963 -Viktor Szakats (3 Aug 2020) -- windows: disable Unix Sockets for old mingw - - Classic mingw and 10y+ old versions of mingw-w64 don't ship with - Windows headers having the typedef necessary for Unix Sockets - support, so try detecting these environments to disable this - feature. - - Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ +- test3015: verify stdout "as text" - Reviewed-by: Daniel Stenberg + Follow-up from 0c1e767e83e to please win32 tests - Fixes #5674 - Closes #5758 + Closes #5962 -Marcel Raad (3 Aug 2020) -- test1908: treat file as text +- travis: use libressl v3.1.4 instead of master - Fixes the line endings on Windows. + ... as their git master seems too fragile to use (and 3.2.1 which is the + latest has a build failure). - Closes https://github.com/curl/curl/pull/5767 + Closes #5964 -- TrackMemory tests: ignore realloc and free in getenv.c +- tests/FILEFORMAT: document type=shell for + +- tests/FILEFORMAT: document nonewline support for - These are only called for WIN32. + The one in , that creates files. - Closes https://github.com/curl/curl/pull/5767 - -Daniel Stenberg (3 Aug 2020) -- tests/FILEFORMAT.md: mention %HTTP2PORT + Follow-up from b83947c8df7 -- RELEASE-NOTES: synced +- [anio brought this change] -- tlsv1.3.d. only for TLS-using connections + tool_writeout: add new writeout variable, %{num_headers} - ... and rephrase that "not all" TLS backends support it. + This variable gives the number of headers. - Closes #5764 + Closes #5947 -- tls-max.d: this option is only for TLS-using connections +- tool_urlglob: fix compiler warning "unreachable code" - Ref: #5763 - Closes #5764 + (On Windows builds.) + + Follow-up to 70a3b003d9 -Marcel Raad (2 Aug 2020) -- [Cameron Cawley brought this change] +- [Gergely Nagy brought this change] - tool_doswin: Simplify Windows version detection + vtls: deduplicate client certificates in ssl_config_data - Closes https://github.com/curl/curl/pull/5754 - -- [Cameron Cawley brought this change] + Closes #5629 - win32: Add Curl_verify_windows_version() to curlx +- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND - Closes https://github.com/curl/curl/pull/5754 + This is primarily interesting for cases where CURLOPT_NOBODY is set as + previously curl would not return an error for this case. + + MDTM getting 550 now also returns this error (it returned + CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for + missing files across protocols and specific FTP commands. + + libcurl already returns error on a 550 as a MDTM response (when + CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would + happen subsequently anyway since the RETR command would fail. + + Add test 1913 and 1914 to verify. Updated several tests accordingly due + to the updated SIZE behavior. + + Reported-by: Tomas Berger + Fixes #5953 + Closes #5957 -- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL +- curl: make checkpasswd use dynbuf - This makes the tests that require the OpenSSL feature also run for - those two compatible libraries. + Closes #5952 + +- curl: make glob_match_url use dynbuf - Closes https://github.com/curl/curl/pull/5762 + Closes #5952 -Daniel Stenberg (1 Aug 2020) -- multi: Condition 'extrawait' is always true +- curl: make file2memory use dynbuf - Reported by Codacy. + Closes #5952 + +- curl: make file2string use dynbuf - Reviewed-by: Marcel Raad - Closes #5759 + Closes #5952 -Marcel Raad (1 Aug 2020) -- openssl: fix build with LibreSSL < 2.9.1 +- [Antarpreet Singh brought this change] + + imap: set cselect_bits to CURL_CSELECT_IN initially - `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were - introduced in LibreSSL 2.9.1 [0]. + ... when continuing a transfer from a FETCH response. - [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef + When the size of the file was small enough that the entirety of the + transfer happens in a single go and schannel buffers holds the entire + data. However, it wasn't completely read in Curl_pp_readresp since a + line break was found before that could happen. So, by the time we are in + imap_state_fetch_resp - there's data in buffers that needs to be read + via Curl_read but nothing to read from the socket. After we setup a + transfer (Curl_setup_transfer), curl just waits on the socket state to + change - which doesn't happen since no new data ever comes. - Closes https://github.com/curl/curl/pull/5757 + Closes #5961 -Daniel Stenberg (1 Aug 2020) -- [Marc Aldorasi brought this change] +- RELEASE-NOTES: synced - multi_remove_handle: close unused connect-only connections +- test434: test -K use in a single line without newline - Previously any connect-only connections in a multi handle would be kept - alive until the multi handle was closed. Since these connections cannot - be re-used, they can be marked for closure when the associated easy - handle is removed from the multi handle. + Closes #5946 + +- runtests: allow creating files without newlines - Closes #5749 + Closes #5946 -- checksrc: invoke script with -D to find .checksrc proper +- curl: use curlx_dynbuf for realloc when loading config files - Without the -D command line option, checksrc.pl won't know which - directory to load the ".checksrc" file from when building out of the - source tree. + ... fixes an integer overflow at the same time. - Reported-by: Marcel Raad - Fixes #5715 - Closes #5755 + Reported-by: ihsinme on github + Assisted-by: Jay Satiro + + Closes #5946 -- [Carlo Marcelo Arenas Belón brought this change] +- dynbuf: provide curlx_ names for reuse by the curl tool + + Closes #5946 - buildconf: retire ares buildconf invocation +- dynbuf: make sure Curl_dyn_tail() zero terminates - no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 + Closes #5959 -- [Carlo Marcelo Arenas Belón brought this change] +- tests: add test1912 to the dist + + Follow-up to 70984ce1be4cab6c - buildconf: excempt defunct reference to ACLOCAL_FLAGS +- docs/LICENSE-MIXING: remove - retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as - the name is generic enough that it might be in use and relied upon from - the environment. + This document is not maintained and I feel that it doesn't provide much + value to users anymore (if it ever did). + + Closes #5955 -- [Carlo Marcelo Arenas Belón brought this change] +- [Laramie Leavitt brought this change] - buildconf: avoid array concatenation in die() - - reported as error SC2145[1] by shellcheck, but not expected to cause - any behavioural differences otherwise. + http: consolidate nghttp2_session_mem_recv() call paths - [1] https://github.com/koalaman/shellcheck/wiki/SC2145 + Previously there were several locations that called + nghttp2_session_mem_recv and handled responses slightly differently. + Those have been converted to call the existing + h2_process_pending_input() function. - Closes #5701 - -- travis: add ppc64le and s390x builds + Moved the end-of-session check to h2_process_pending_input() since the + only place the end-of-session state can change is after nghttp2 + processes additional input frames. - Closes #5752 - -Marc Hoersken (31 Jul 2020) -- connect: remove redundant message about connect failure + This will likely fix the fuzzing error. While I don't have a root cause + the out-of-bounds read seems like a use after free, so moving the + nghttp2_session_check_request_allowed() call to a location with a + guaranteed nghttp2 session seems reasonable. - Reviewed-by: Daniel Stenberg + Also updated a few nghttp2 callsites to include error messages and added + a few additional error checks. - Closes #5708 + Closes #5648 -- tests/sshserver.pl: fix compatibility with OpenSSH for Windows +- HISTORY: mention alt-svc added in 2019 - Follow up to #5721 + ... and make 1996 the first year subtitle -- CI/azure: install libssh2 for use with msys2-based builds - - This enables building and running the SFTP tests. - Unfortunately OpenSSH for Windows does not support SCP (yet). +- base64: also build for pop3 and imap - Reviewed-by: Daniel Stenberg + Follow-up to the fix in 20417a13fb8f83 - Closes #5721 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5948 -- CI/azure: increase Windows job timeout once again +- base64: enable in build with SMTP - Avoid aborted jobs due to performance issues on Azure DevOps. + The oauth2 support is used with SMTP and it uses base64 functions. - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5938 + +- curl_mime_headers.3: fix the example's use of curl_slist_append - Closes #5738 + Reported-by: sofaboss on github + Fixes #5942 + Closes #5943 -Jay Satiro (30 Jul 2020) -- TODO: Schannel: 'Add option to allow abrupt server closure' +- lib583: fix enum mixup - We should offer an option to allow abrupt server closures (server closes - SSL transfer without sending a known termination point such as length of - transfer or close_notify alert). Abrupt server closures are usually - because of misconfigured or very old servers. + grrr the previous follow-up to 17fcdf6a31 was wrong + +- libtest: fix build errors - Closes https://github.com/curl/curl/issues/4427 + Follow-up from 17fcdf6a310d4c8076 -- url: fix CURLU and location following +- lib: fix -Wassign-enum warnings - Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was - incorrectly used for the location follow, resulting in infinite requests - to the original location. + configure --enable-debug now enables -Wassign-enum with clang, + identifying several enum "abuses" also fixed. - Reported-by: sspiri@users.noreply.github.com + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 - Fixes https://github.com/curl/curl/issues/5709 - Closes https://github.com/curl/curl/pull/5713 + Closes #5929 -Daniel Stenberg (30 Jul 2020) - RELEASE-NOTES: synced -- [divinity76 brought this change] +- [Diven Qi brought this change] - docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions + url: use blank credentials when using proxy w/o username and password - it helps make it obvious that most developers don't have to care about - the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 - years old, November 4 2009) + Fixes proxy regression brought in commit ad829b21ae (7.71.0) - Closes #5744 + Fixed #5911 + Closes #5914 -Jay Satiro (29 Jul 2020) -- tool_cb_wrt: fix outfile mode flags for Windows - - - Use S_IREAD and S_IWRITE mode permission flags to create the file - on Windows instead of S_IRUSR, S_IWUSR, etc. +- travis: add a build using libressl (from git master) - Windows only accepts a combination of S_IREAD and S_IWRITE. It does not - acknowledge other combinations, for which it may generate an assertion. + The v3.2.1 tag (latest release atm) results in a broken build. - This is a follow-up to 81b4e99 from yesterday, which improved the - existing file check with -J. + Closes #5932 + +- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 - Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks - Ref: https://github.com/curl/curl/pull/5731 + Unfortunately, this option is not detecting the same issues as clang's + -Wassign-enum flag, but should still be useful to detect future + mistakes. - Closes https://github.com/curl/curl/pull/5742 + Closes #5930 -Daniel Stenberg (28 Jul 2020) -- checksrc: ban gmtime/localtime +- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - They're not thread-safe so they should not be used in libcurl code. + If the error reason from the lib is + SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return + CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. - Explictly enabled when deemed necessary and in examples and tests + This unifies the libcurl return code and makes libressl run test 313 + (CRL testing) fine. - Reviewed-by: Nicolas Sterchele - Closes #5732 + Closes #5934 -- transfer: fix data_pending for builds with both h2 and h3 enabled - - Closes #5734 +- FAQ: refreshed some very old language -- curl_multi_setopt: fix compiler warning "result is always false" +- cmake: make HTTP_ONLY also disable MQTT - On systems with 32 bit long the expression is always false. Avoid - the warning. + ... and alphasort the order of disabling protocols to make it easier to + browse. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 - Closes #5736 + Closes #5931 -- curl: improve the existing file check with -J +- libtest: remove lib1541 leftovers - Previously a file that isn't user-readable but is user-writable would - not be properly avoided and would get overwritten. + Caused automake errors. - Reported-by: BrumBrum on hackerone - Assisted-by: Jay Satiro - Bug: https://hackerone.com/reports/926638 - Closes #5731 - -- [Jonathan Nieder brought this change] + Follow-up to 8ca54a03ea08a - multi: update comment to say easyp list is linear +- tests/libtests: remove test 1900 and 2033 - Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, - 2013-08-02), the easy handle list is not circular but ends with - ->next pointing to NULL. + We already remove the test files, now remove the libtest codes as well. - Reported-by: Masaya Suzuki - Closes #5737 + Follow-up to e50a877df74 -- CURLOPT_NOBODY.3: fix the syntax for referring to options +Marc Hoersken (7 Sep 2020) +- CI/azure: add test number to title for display in analytics - As test 1140 fails otherwise! + To ease identification of tests the test number is added to + the test case title in order to have it on the Azure DevOps + Analytics pages and reports which currently do not show it. - Follow-up to e1bac81cc815 - -- ngtcp2: store address in sockaddr_storage + Bump test case revision to make Azure DevOps update titles. - Reported-by: Tatsuhiro Tsujikawa - Closes #5733 + Closes #5927 -- CURLOPT_NOBODY.3: clarify what setting to 0 means +Daniel Stenberg (6 Sep 2020) +- altsvc: clone setting in curl_easy_duphandle - ... and mention that HTTP with other methods than HEAD might get a body and - there's no option available to stop that. + The cache content is not duplicated, like other caches, but the setting + and specified file name are. - Closes #5729 + Test 1908 is extended to verify this somewhat. Since the duplicated + handle gets the same file name, the test unfortunately overwrites the + same file twice (with different contents) which makes it hard to check + automatically. + + Closes #5923 -- setopt: unset NOBODY switches to GET if still HEAD +- test1541: remove since it is a known bug - Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented - action but before 7.71.0 that used to switch back to GET and with this - change (assuming the method is still set to HEAD) this behavior is - brought back. + A shared connection cache is not thread-safe is a known issue. Stop + testing this until we believe this issue is addressed. Reduces + occasional test failures we don't care about. - Reported-by: causal-agent on github - Fixes #5725 - Closes #5728 - -- [Ehren Bendler brought this change] + The test code in lib1541.c is left in git to allow us to restore it when + we get to fix this. + + Closes #5922 - configure: cleanup wolfssl + pkg-config conflicts when cross compiling. +- tests: remove pipelining tests - Also choose a different wolfSSL function to test for NTLM support. + Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were + previously disabled. - Fixes #5605 - Closes #5682 - -- configure: show zstd "no" in summary when built without it + The Pipelining code was removed from curl in commit 2f44e94efb3df8e, + April 2019. - Reported-by: Marc Hörsken - Fixes #5720 - Closes #5730 + Closes #5921 -- quiche: handle calling disconnect twice +- curl: retry delays in parallel mode no longer sleeps blocking - Reported-by: lilongyan-huawei on github - Fixes #5726 - Closes #5727 + The previous sleep for retries would block all other concurrent + transfers. Starting now, the retry will instead be properly marked to + not get restarted until after the delay time but other transfers can + still continue in the mean time. + + Closes #5917 -- [Nicolas Sterchele brought this change] +- curl:parallel_transfers: make sure retry readds the transfer + + Reported-by: htasta on github + Fixes #5905 + Closes #5917 - getinfo: reset retry-after value in initinfo +- build: drop support for building with Watcom - - Avoid re-using retry_after value from preceding request - - Add libtest 3010 to verify + These files are not maintained, they seem to have no users, Watcom + compilers look like not having users nor releases anymore. - Reported-by: joey-l-us on github - Fixes #5661 - Closes #5672 + Closes #5918 -Marcel Raad (27 Jul 2020) -- WIN32: stop forcing narrow-character API +- winbuild/rundebug.cmd: remove - Except where the results are only used for character output. - getenv is not touched because it's part of the public API, and having - it return UTF-8 instead of ANSI would be a breaking change. + Seems to have been added by mistake? Not included in dists. - Fixes https://github.com/curl/curl/issues/5658 - Fixes https://github.com/curl/curl/issues/5712 - Closes https://github.com/curl/curl/pull/5718 - -Jay Satiro (27 Jul 2020) -- [Tobias Stoeckmann brought this change] + Closes #5919 - mprintf: Fix stack overflows +- curl: in retry output don't call all problems "transient" - Stack overflows can occur with precisions for integers and floats. + ... because when --retry-all-errors is used, the error isn't necessarily + transient at all. - Proof of concepts: - - curl_mprintf("%d, %.*1$d", 500, 1); - - curl_mprintf("%d, %+0500.*1$f", 500, 1); + Closes #5916 + +- easygetopt: pass a valid enum to avoid compiler warning - Ideally, compile with -fsanitize=address which makes this undefined - behavior a bit more defined for debug purposes. + "integer constant not in range of enumerated type 'CURLoption'" - The format strings are valid. The overflows occur due to invalid - arguments. If these arguments are variables with contents controlled - by an attacker, the function's stack can be corrupted. + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 - Also see CVE-2016-9586 which partially fixed the float aspect. + Closes #5915 + +- [Emil Engler brought this change] + + tests: Add tests for new --help - Signed-off-by: Tobias Stoeckmann + This commit is a part of "--help me if you can" - Closes https://github.com/curl/curl/pull/5722 + Closes #5680 -- [Tobias Stoeckmann brought this change] +- [Emil Engler brought this change] - mprintf: Fix dollar string handling + tool: update --help with categories - Verify that specified parameters are in range. If parameters are too - large, fail early on and avoid out of boundary accesses. + This commit is a part of "--help me if you can" - Also do not read behind boundaries of illegal format strings. + Closes #5680 + +- [Emil Engler brought this change] + + docs: add categories to all cmdline opts - These are defensive measures since it is expected that format strings - are well-formed. Format strings should not be modifiable by user - input due to possible generic format string attacks. + Adapted gen.pl with 'listcats' - Closes https://github.com/curl/curl/pull/5722 + This commit is a part of "--help me if you can" + + Closes #5680 -Daniel Stenberg (26 Jul 2020) -- ntlm: free target_info before (re-)malloc +- RELEASE-NOTES: synced + +- [ihsinme brought this change] + + connect.c: remove superfluous 'else' in Curl_getconnectinfo - OSS-Fuzz found a way this could get called again with the pointer still - pointing to a malloc'ed memory, leading to a leak. + Closes #5912 + +- [Samuel Marks brought this change] + + CMake: remove explicit `CMAKE_ANSI_CFLAGS` - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 + This variable was removed from cmake in commit + https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later + CMake commit removes the variable from the tests, claiming that it was + removed in CMake 2.6 - Closes #5724 + Reviewed-By: Peter Wu + Closes #5439 -Marcel Raad (26 Jul 2020) -- CI/macos: set minimum macOS version +- [cbe brought this change] + + libssh2: pass on the error from ssh_force_knownhost_key_type - This enables some deprecation warnings. - Previously, autotools defaulted to 10.8. + Closes #5909 + +- scripts/delta: add diffstat summary - Closes https://github.com/curl/curl/pull/5723 + ... and make output more table-like -Daniel Stenberg (26 Jul 2020) -- RELEASE-NOTES: synced +- [Martin Bašti brought this change] -Marcel Raad (25 Jul 2020) -- CI/macos: enable warnings as errors for CMake builds + http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set - Closes https://github.com/curl/curl/pull/5716 - -- CMake: fix test for warning suppressions + ... in case NO_PROXY takes an effect - GCC doesn't warn for unknown `-Wno-` options, except if there are other - warnings or errors [0]. This was problematic with `CURL_WERROR` as that - warning-as-error cannot be suppressed. Notably, this always happened - with `-Wno-pedantic-ms-format` when not targeting Windows. So test for - the positive form of the warning instead, which should always result in - a diagnostic if unknown. + Without this patch, the following command crashes: - [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html + $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ + git clone https://github.com/curl/curl.git - Closes https://github.com/curl/curl/pull/5714 - -Jay Satiro (23 Jul 2020) -- curl.h: update CURLINFO_LASTONE + Minimal libcurl-based reproducer: - CURLINFO_LASTONE should have been updated when - CURLINFO_EFFECTIVE_METHOD was added. + #include - Reported-by: xwxbug@users.noreply.github.com + int main() { + CURL *curl = curl_easy_init(); + if(curl) { + CURLcode ret; + curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); + curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); + /* set the proxy type */ + curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); + curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); + curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); + ret = curl_easy_perform(curl); + curl_easy_cleanup(curl); + return ret; + } + return -1; + } - Fixes https://github.com/curl/curl/issues/5711 + Assisted-by: Kamil Dudka + Bug: https://bugzilla.redhat.com/1873327 + Closes #5902 -Marc Hoersken (22 Jul 2020) -- CI/azure: unconditionally enable warnings-as-errors with autotools +- travis: add a CI job with openssl3 (from git master) - Reviewed-by: Marcel Raad + Closes #5908 + +- openssl: avoid error conditions when importing native CA - Follow up to #5694 - Closes #5706 + The code section that is OpenSSL 3+ specific now uses the same logic as + is used in the version < 3 section. It caused a compiler error without + it. + + Closes #5907 -Marcel Raad (21 Jul 2020) -- doh: remove redundant cast +- setopt: avoid curl_ on local variable - Closes https://github.com/curl/curl/pull/5704 + Closes #5906 -- CI/macos: unconditionally enable warnings-as-errors with autotools +- mqtt.c: avoid curl_ prefix on local variable - Previously, warnings were only visible in the output for most jobs. + Closes #5906 + +- wildcard: strip "curl_" prefix from private symbols - Closes https://github.com/curl/curl/pull/5694 + Closes #5906 -- util: silence conversion warnings +- vtls: make it 'struct Curl_ssl_session' - timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might - be a 64-bit integer. This is the case when building for recent macOS - versions, for example. Just treat tv_usec as an int, which should - hopefully always be sufficient on systems with - `HAVE_CLOCK_GETTIME_MONOTONIC`. + Use uppercase C for internal symbols. - Closes https://github.com/curl/curl/pull/5695 + Closes #5906 -- md(4|5): don't use deprecated macOS functions +- curl_threads: make it 'struct Curl_actual_call' - They are marked as deprecated for -mmacosx-version-min >= 10.15, - which might result in warnings-as-errors. + Internal names should not be prefixed "curl_" - Closes https://github.com/curl/curl/pull/5695 + Closes #5906 -Daniel Stenberg (18 Jul 2020) -- strdup: remove the odd strlen check +- schannel: make it 'struct Curl_schannel*' - It confuses code analyzers with its use of -1 for unsigned value. Also, - a check that's not normally used in strdup() code - and not necessary. + As internal global names should use captical C. - Closes #5697 - -- [Alessandro Ghedini brought this change] + Closes #5906 - travis: update quiche builds for new boringssl layout - - This is required after https://github.com/cloudflare/quiche/pull/593 - moved BoringSSL around slightly. +- hash: make it 'struct Curl_hash' - This also means that Go is not needed to build BoringSSL anymore (the - one provided by quiche anyway). + As internal global names should use captical C. - Closes #5691 + Closes #5906 -Marcel Raad (17 Jul 2020) -- configure: allow disabling warnings +- llist: make it "struct Curl_llist" - When using `--enable-warnings`, it was not possible to disable warnings - via CFLAGS that got explicitly enabled. Now warnings are not enabled - anymore if they are explicitly disabled (or enabled) in CFLAGS. This - works for at least GCC, clang, and TCC as they have corresponding - `-Wno-` options for every warning. + As internal global names should use captical C. - Closes https://github.com/curl/curl/pull/5689 + Closes #5906 -Daniel Stenberg (16 Jul 2020) -- ngtcp2: adjust to recent sockaddr updates +Marc Hoersken (2 Sep 2020) +- telnet.c: depend on static requirement of WinSock version 2 - Closes #5690 - -- page-header: provide protocol details in the curl.1 man page + Drop dynamic loading of ws2_32.dll and instead rely on the + imported version which is now required to be at least 2.2. - Add protocol and version specific information about all protocols curl - supports. + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats - Fixes #5679 - Reported-by: tbugfinder on github - Closes #5686 + Closes #5854 -Daniel Gustafsson (16 Jul 2020) -- docs: Update a few leftover mentions of DarwinSSL +- win32: drop support for WinSock version 1, require version 2 - Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the - more correct/common name Secure Transport, but a few mentions in the docs - remained. + IPv6, telnet and now also the multi API require WinSock + version 2 which is available starting with Windows 95. - Closes #5688 - Reviewed-by: Daniel Stenberg + Therefore we think it is time to drop support for version 1. + + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats + + Follow up to #5634 + Closes #5854 -Daniel Stenberg (16 Jul 2020) -- file2memory: use a define instead of -1 unsigned value +- select: align poll emulation to return all relevant events + + The poll emulation via select already consumes POLLRDNORM, + POLLWRNORM and POLLRDBAND as input events. Therefore it + should also return them as output events if signaled. - ... to use the maximum value for 'size_t' when detecting integer overflow. - Changed the limit to max/4 as already that seems unreasonably large. + Also fix indentation in input event handling block. - Codacy didn't like the previous approach. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5683 + Replaces #5852 + Closes #5883 -- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream +- CI/azure: MQTT is now enabled by default - ... by adding support for a new dedicated return code. + Reviewed-by: Daniel Stenberg - Suggested-by: Jonathan Cardoso - Assisted-by: Erik Johansson - URL: https://curl.haxx.se/mail/lib-2020-06/0099.html - Closes #5636 + Follow up to #5858 + Closes #5903 -- [Baruch Siach brought this change] +Daniel Stenberg (2 Sep 2020) +- copyright.pl: ignore buildconf + +- test971: show test mismatches "inline" - nss: fix build with disabled proxy support +- lib/Makefile.am: bump VERSIONINFO due to new functions - Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is - defined. + ... we're generally bad at this, but we are adding new functions for + this release. - Closes #5667 - -- test1139: make it display the difference on test failures + Closes #5899 -- test1119: verify stdout in the test - - So that failures will be displayed in the terminal, as it makes test failures - visually displayed easier and faster. +- optiontable: use DEBUGBUILD - Closes #5644 + Follow-up to commit 6e18568ba38 (#5877) -- curl: add %{method} to the -w variables +- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 - Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. + If there are more than two items in the list, use commas for all but the + last separator which is set to 'and'. Reads better. - Added test 1197 to verify. + Closes #5898 -- CURLINFO_EFFECTIVE_METHOD: added +- curl.1: add see also no-progress-meter on two spots - Provide the HTTP method that was used on the latest request, which might - be relevant for users when there was one or more redirects involved. + Ref: #5894 - Closes #5511 + Closes #5897 -Viktor Szakats (14 Jul 2020) -- windows: add unicode to feature list +- RELEASE-NOTES: synced + +- mqtt: enable by default - Reviewed-by: Marcel Raad - Reviewed-by: Marc Hörsken + No longer considered experimental. - Closes #5491 + Closes #5858 + +- [Michael Baentsch brought this change] -Daniel Stenberg (14 Jul 2020) -- multi: remove two checks always true + tls: add CURLOPT_SSL_EC_CURVES and --curves - Detected by Codacy - Closes #5676 + Closes #5892 + +- url: remove funny embedded comments in Curl_disonnect calls + +- [Chris Paulson-Ellis brought this change] -Marc Hoersken (13 Jul 2020) -- workflows: limit what branches to run CodeQL on + conn: check for connection being dead before reuse - Align CodeQL action with existing CI actions: - - Update branch filter to avoid duplicate CI runs. - - Shorten workflow name due to informative job name. + Prevents incorrect reuse of an HTTP connection that has been prematurely + shutdown() by the server. - Reviewed-by: Daniel Stenberg + Partial revert of 755083d00deb16 - Closes #5660 + Fixes #5884 + Closes #5893 -- appveyor: collect libcurl.dll variants with prefix or suffix +Marc Hoersken (29 Aug 2020) +- buildconf: exec autoreconf to avoid additional process - On some platforms libcurl is build with a platform-specific - prefix and/or a version number suffix. + Also make buildconf exit with the return code of autoreconf. - Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5659 + Follow up to #5853 + Closes #5890 -Daniel Stenberg (12 Jul 2020) -- [ihsinme brought this change] +- CI/azure: no longer ignore results of test 1013 + + Follow up to #5771 + Closes #5889 - socks: use size_t for size variable +- docs: add description about CI platforms to CONTRIBUTE.md - Use the unsigned type (size_t) in the arithmetic of pointers. In this - context, the signed type (ssize_t) is used unnecessarily. + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro - Authored-by: ihsinme on github - Closes #5654 + Closes #5882 -- RELEASE-NOTES: synced +Daniel Stenberg (29 Aug 2020) +- tests/getpart: use MIME::Base64 instead of home-cooked - ... and bumped to 7.72.0 as the next release version number - -- [Gilles Vollant brought this change] - - content_encoding: add zstd decoding support + Since we already use the base64 package since a while back, we can just + as well switch to that here too. - include zstd curl patch for Makefile.m32 from vszakats - and include Add CMake support for zstd from Peter Wu + It also happens to use the exact same function name, which otherwise + causes a run-time warning. - Helped-by: Viktor Szakats - Helped-by: Peter Wu - Closes #5453 + Reported-by: Marc Hörsken + Fixes #5885 + Closes #5887 -- asyn.h: remove the Curl_resolver_getsock define +Marcel Raad (29 Aug 2020) +- ntlm: fix condition for curl_ntlm_core usage - - not used - - used the wrong number of arguments - - confused the Codeacy code analyzer + `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES + backend is fine, but was excluded before. - Closes #5647 - -- [Nicolas Sterchele brought this change] + This also fixes test 1013 as the condition for SMB support in + configure.ac didn't match the condition in the source code. Now it + does. + + Fixes https://github.com/curl/curl/issues/1262 + Closes https://github.com/curl/curl/pull/5771 - configure.ac: Sort features name in summary +- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode - - Same as protocols + The Schannel builds are the most useful to verify as they make the most + use of the Windows API. Classic MinGW doesn't support Unicode at all, + only MinGW-w64 and MSVC do. - Closes #5656 - -- [Matthias Naegler brought this change] + Closes https://github.com/curl/curl/pull/5843 - cmake: fix windows xp build +- CMake: add option to enable Unicode on Windows - Reviewed-by: Marcel Raad - Closes #5662 - -- ngtcp2: update to modified qlog callback prototype + As already existing for winbuild. - Closes #5675 + Closes https://github.com/curl/curl/pull/5843 -- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle +Marc Hoersken (29 Aug 2020) +- select: simplify return code handling for poll and select - Added test case 674 to reproduce and verify the bug report. + poll and select already return -1 on error according to POSIX, + so there is no need to perform a <0 to -1 conversion in code. - Fixes #5665 - Reported-by: NobodyXu on github - Closes #5673 - -- [Baruch Siach brought this change] - - bearssl: fix build with disabled proxy support + Also we can just use one check with <= 0 on the return code. - Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is - defined. + Assisted-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Reviewed-by: Nicolas Sterchele - Closes #5666 + Replaces #5852 + Closes #5880 +Daniel Stenberg (28 Aug 2020) - RELEASE-NOTES: synced -Jay Satiro (11 Jul 2020) -- [Carlo Marcelo Arenas Belón brought this change] +- [Jeroen Ooms brought this change] - cirrus-ci: upgrade 11-STABLE to 11.4 + tests: add test1912 with typechecks - Meant to be the last of the 11 series and so make sure that all - other references reflect all 11 versions so they can be retired - together later. + Validates that gcc-typecheck macros match the new option type API. - Closes https://github.com/curl/curl/pull/5668 - -- [Filip Salomonsson brought this change] + Closes #5873 - CURLINFO_CERTINFO.3: fix typo +- easyoptions: provide debug function when DEBUGBUILD - Closes https://github.com/curl/curl/pull/5655 - -Daniel Stenberg (4 Jul 2020) -- http2: only do the *done() cleanups for HTTP + ... not CURLDEBUG as they're not always set in conjunction. - Follow-up to ef86daf4d3 + Follow-up to 6ebe63fac23f38df - Closes #5650 - Fixes #5646 - -- [Alex Kiernan brought this change] + Fixes #5877 + Closes #5878 - gnutls: repair the build with `CURL_DISABLE_PROXY` - - `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` - if `CURL_DISABLE_PROXY` is enabled. Repair the build with that - configuration. +Marc Hoersken (28 Aug 2020) +- sockfilt: handle FD_CLOSE winsock event on write socket - Signed-off-by: Alex Kiernan - Closes #5645 - -Alex Kiernan (3 Jul 2020) -- gnutls: Fetch backend when using proxy + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") - Signed-off-by: Alex Kiernan - -Daniel Stenberg (3 Jul 2020) -- [Laramie Leavitt brought this change] + Follow up to #5867 + Closes #5879 - http2: close the http2 connection when no more requests may be sent - - Well-behaving HTTP2 servers send two GOAWAY messages. The first - message is a warning that indicates that the server is going to - stop accepting streams. The second one actually closes the stream. - - nghttp2 reports this state (and the other state of no more stream - identifiers) via the call nghttp2_session_check_request_allowed(). - In this state the client should not create more streams on the - session (tcp connection), and in curl this means that the server - has requested that the connection is closed. - - It would be also be possible to put the connclose() call into the - on_http2_frame_recv() function that triggers on the GOAWAY message. - - This fixes a bug seen when the client sees the following sequence of - frames: +- multi: handle connection state winsock events - // advisory GOAWAY - HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] - ... some additional frames + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - // final GOAWAY - HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad - Before this change, curl will attempt to reuse the connection even - after the last stream, will encounter this error: + Follow up to #5634 + Closes #5867 + +Daniel Stenberg (28 Aug 2020) +- Curl_pgrsTime - return new time to avoid timeout integer overflow - * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] - * Re-using existing connection! (#0) with host localhost - * Connected to localhost (::1) port 10443 (#0) - * Using Stream ID: 9 (easy handle 0x5595f0a72e30) - > GET /index.html?5 HTTP/2 - > Host: localhost:10443 - > user-agent: curl/7.68.0 - > accept: */* - > - * stopped the pause stream! - * Connection #0 to host localhost left intact - curl: (16) Error in the HTTP2 framing layer + Setting a timeout to INT_MAX could cause an immediate error to get + returned as timeout because of an overflow when different values of + 'now' were used. - This error may posion the connection cache, causing future requests - which resolve to the same curl connection to go through the same error - path. + This is primarily fixed by having Curl_pgrsTime() return the "now" when + TIMER_STARTSINGLE is set so that the parent function will continue using + that time. - Closes #5643 + Reported-by: Ionuț-Francisc Oancea + Fixes #5583 + Closes #5847 -- ftpserver: don't verify SMTP MAIL FROM names - - Rely on tests asking the names to get refused instead - test servers - should be as dumb as possible. Edited test 914, 955 and 959 accordingly. +- TLS: fix SRP detection by using the proper #ifdefs - Closes #5639 - -- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated + USE_TLS_SRP will be true if *any* selected TLS backend can use SRP - This came up in #5640. It make sense to clarify this in the docs! + HAVE_OPENSSL_SRP is defined when OpenSSL can use it - Reminded-by: Kamil Dudka - Closes #5642 - -Kamil Dudka (3 Jul 2020) -- tool_getparam: make --krb option work again + HAVE_GNUTLS_SRP is defined when GnuTLS can use it - It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. + Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is + set if at least one of the supported backends offers SRP. - Bug: https://bugzilla.redhat.com/1833193 - Closes #5640 + Reported-by: Stefan Strogin + Fixes #5865 + Closes #5870 -Daniel Stenberg (2 Jul 2020) -- [Jeremy Maitin-Shepard brought this change] +- [Dan Kenigsberg brought this change] - http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages - - Confusingly, nghttp2 has two different error code enums: + docs: SSLCERTS: fix English syntax - - nghttp2_error, to be used with nghttp2_strerror - - nghttp2_error_code, to be used with nghttp2_http2_strerror + Signed-off-by: Dan Kenigsberg - Closes #5641 + Closes #5876 + +- [Alessandro Ghedini brought this change] -Marcel Raad (2 Jul 2020) -- url: silence MSVC warning + docs: non-existing macros in man pages - Since commit f3d501dc678, if proxy support is disabled, MSVC warns: - url.c : warning C4701: potentially uninitialized local variable - 'hostaddr' used - url.c : error C4703: potentially uninitialized local pointer variable - 'hostaddr' used + As reported by man(1) when invoked as: - That could actually only happen if both `conn->bits.proxy` and - `CURL_DISABLE_PROXY` were enabled. - Initialize it to NULL to silence the warning. + man --warnings -E UTF-8 -l -Tutf8 -Z >/dev/null - Closes https://github.com/curl/curl/pull/5638 - -Daniel Stenberg (1 Jul 2020) -- RELEASE-NOTES: synced - -Version 7.71.1 (30 Jun 2020) - -Daniel Stenberg (30 Jun 2020) -- RELEASE-NOTES: curl 7.71.1 - -- THANKS: add contributors to 7.71.1 + Closes #5846 -- scripts/copyright.pl: skip .dcignore +- [Alessandro Ghedini brought this change] -- Revert "multi: implement wait using winsock events" + curl.1: fix typo invokved -> invoked - This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. + Closes #5846 + +- buildconf: invoke 'autoreconf -fi' instead - That commit (from #5397) introduced a regression in 7.71.0. + The custom script isn't necessary anymore - but remains for simplicity + and just invokes autoreconf. - Reported-by: tmkk on github - Fixes #5631 - Closes #5632 - -- TODO: Add flag to specify download directory + Closes #5853 -- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection +- [Emil Engler brought this change] -- cirrus-ci: disable FreeBSD 13 (again) + lib: make Curl_gethostname accept a const pointer - It has been failing for a good while again. This time we better leave it - disabled until we have more reason to believe it behaves. + The address of that variable never gets changed, only the data in it so + why not make it a "char * const"? - Closes #5628 + Closes #5866 -- ngtcp2: sync with current master - - ngtcp2 added two new callbacks +- docs/libcurl: update "Added in" version for curl_easy_option* - Reported-by: Lucien Zürcher - Fixes #5624 - Closes #5627 + Follow-up to 6ebe63fac23f38 -- examples/multithread.c: call curl_global_cleanup() +- scripts: improve the "get latest curl release tag" logic - Reported-by: qiandu2006 on github - Fixes #5622 - Closes #5623 + ... by insiting on it matching "^curl-". -- vtls: compare cert blob when finding a connection to reuse +- configure: added --disable-get-easy-options - Reported-by: Gergely Nagy - Fixes #5617 - Closes #5619 - -- RELEASE-NOTES: synced + To allow disabling of the curl_easy_option APIs in a build. + + Closes #5365 -- terminology: call them null-terminated strings +- options: API for meta-data about easy options - Updated terminology in docs, comments and phrases to refer to C strings - as "null-terminated". Done to unify with how most other C oriented docs - refer of them and what users in general seem to prefer (based on a - single highly unscientific poll on twitter). + const struct curl_easyoption *curl_easy_option_by_name(const char *name); - Reported-by: coinhubs on github - Fixes #5598 - Closes #5608 - -- http: fix proxy auth with blank password + const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); - Regression in 7.71.0 + const struct curl_easyoption * + curl_easy_option_next(const struct curl_easyoption *prev); - Added test case 346 to verify. + The purpose is to provide detailed enough information to allow for + example libcurl bindings to get option information at run-time about + what easy options that exist and what arguments they expect. - Reported-by: Kristoffer Gleditsch - Fixes #5613 - Closes #5616 + Assisted-by: Jeroen Ooms + Closes #5365 + +- [Eric Curtin brought this change] -- .dcignore: ignore tests and docs directories + HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 - This is a config file for deepcode.ai, a static code analyzer. + Closes #5871 + +- RELEASE-NOTES: synced -Jay Satiro (26 Jun 2020) -- tool_cb_hdr: Fix etag warning output and return code +Jay Satiro (26 Aug 2020) +- openssl: Fix wincrypt symbols conflict with BoringSSL - - Return 'failure' on failure, to follow the existing style. + OpenSSL undefines the conflicting symbols but BoringSSL does not so we + must do it ourselves. - - Put Warning: and the warning message on the same line. + Reported-by: Samuel Tranchet + Assisted-by: Javier Blazquez - Ref: https://github.com/curl/curl/issues/5610 + Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 + Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 - Closes https://github.com/curl/curl/pull/5612 + Fixes https://github.com/curl/curl/issues/5669 + Closes https://github.com/curl/curl/pull/5857 -Daniel Stenberg (26 Jun 2020) -- CURLOPT_READFUNCTION.3: provide the upload data size up front +Daniel Stenberg (26 Aug 2020) +- socketpair: allow CURL_DISABLE_SOCKETPAIR - Assisted-by: Jay Satiro - Closes #5607 - -- test1539: do a HTTP 1.0 POST without a set size (fails) + ... to completely disable the use of socketpair - Attempt to reproduce #5593. Test case 1514 is very similar but uses - HTTP/1.1 and thus switches to chunked. + Closes #5850 + +- curl_get_line: build only if cookies or alt-svc are enabled - Closes #5595 + Closes #5851 -- [Baruch Siach brought this change] +- [fullincome brought this change] - mbedtls: fix build with disabled proxy support - - Don't reference fields that do not exist. Fixes build failure: - - vtls/mbedtls.c: In function 'mbed_connect_step1': - vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' + schannel: fix memory leak when using get_cert_location - Closes #5615 - -- codeql-analysis.yml: fix the 'languages' setting + The get_cert_location function allocates memory only on success. + Previously get_cert_location was able to allocate memory and return + error. It wasn't obvious and in this case the memory wasn't + released. - It needs a 'with:' in front of it. - -GitHub (26 Jun 2020) -- [Daniel Stenberg brought this change] + Fixes #5855 + Closes #5860 - gtihub: codeql-analysis.yml - - enables code security scanning with github actions +- [Emil Engler brought this change] -Daniel Stenberg (25 Jun 2020) -- tests: verify newline in username and password for HTTP + git: ignore libtests in 3XXX area - test 1296 is a simply command line test + Currently the file tests/libtest/lib3010 is not getting + ignored by git. This fixes it by adding the 3XXX area to + the according .gitignore file. - test 1910 is a libcurl test including a redirect + Closes #5859 -- url: allow user + password to contain "control codes" for HTTP(S) - - Reported-by: Jon Johnson Jr - Fixes #5582 - Closes #5592 +- [Emil Engler brought this change] -- escape: make the URL decode able to reject only %00 bytes + doh: add error message for DOH_DNS_NAME_TOO_LONG - ... or all "control codes" or nothing. + When this error code was introduced in b6a53fff6c1d07e8a9, it was + forgotten to be added in the errors array and doh_strerror function. - Assisted-by: Nicolas Sterchele + Closes #5863 -- http2: set the correct URL in pushed transfers +- ngtcp2: adapt to the new pkt_info arguments - ...previously CURLINFO_EFFECTIVE_URL would report the URL of the - original "mother transfer", not the actually pushed resource. + Guidance-by: Tatsuhiro Tsujikawa - Reported-by: Jonathan Cardoso Machado - Fixes #5589 - Closes #5591 - -Jay Satiro (25 Jun 2020) -- [Javier Blazquez brought this change] + Closes #5864 - openssl: Fix compilation on Windows when ngtcp2 is enabled - - - Include wincrypt before OpenSSL includes so that the latter can - properly handle any conflicts between the two. +- winbuild/README.md: make visible - Closes https://github.com/curl/curl/pull/5606 + Follow-up to be753add31c2d8c -Daniel Stenberg (25 Jun 2020) -- test543: extended to verify zero length input +- winbuild: convert the instruction text to README.md - As was reported in #5601 + Closes #5861 -- escape: zero length input should return a zero length output - - Regression added in 7.71.0. +- lib1560: verify "redirect" to double-slash leading URL - Fixes #5601 - Reported-by: Kristoffer Gleditsch - Closes #5602 + Closes #5849 -- Curl_inet_ntop: always check the return code +Marc Hoersken (25 Aug 2020) +- multi: expand pre-check for socket readiness - Reported-by: Siva Sivaraman - Fixes #5412 - Closes #5597 - -- sendf: improve the message on client write errors + Check readiness of all sockets before waiting on them + to avoid locking in case the one-time event FD_WRITE + was already consumed by a previous wait operation. - Replace "Failed writing body (X != Y)" with - "Failure writing output to destination". Possibly slightly less cryptic. + More information about WinSock network events: + https://docs.microsoft.com/en-us/windows/win32/api/ + winsock2/nf-winsock2-wsaeventselect#return-value - Reported-by: coinhubs on github - Fixes #5594 - Closes #5596 - -- RELEASE-NOTES: synced - -- curlver: start working on 7.71.1 + Closes #5634 -- [Denis Baručić brought this change] +- [rcombs brought this change] - DYNBUF.md: fix a typo: trail => tail + multi: implement wait using winsock events - Closes #5599 - -Version 7.71.0 (23 Jun 2020) - -Daniel Stenberg (23 Jun 2020) -- RELEASE-NOTES: curl 7.71.0 release - -- THANKS: curl 7.71.0 additions - -- url: make sure pushed streams get an allocated download buffer + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. + + A previous version of this patch failed to account for how in WinSock, + FD_WRITE is set only once when writing becomes possible and not again + until after a send has failed due to the buffer filling. This contrasts + to how FD_READ and FD_OOB continue to be set until the conditions they + refer to no longer apply. This meant that if a user wrote some data to + a socket, but not enough data to completely fill its send buffer, then + waited on that socket to become writable, we'd erroneously stall until + their configured timeout rather than returning immediately. - Follow-up to c4e6968127e876b0 + This version of the patch addresses that issue by checking each socket + we're waiting on to become writable with select() before the wait, and + zeroing the timeout if it's already writable. - When a new transfer is created, as a resuly of an acknowledged push, - that transfer needs a download buffer allocated. + Assisted-by: Marc Hörsken + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Tested-by: Gergely Nagy + Tested-by: Rasmus Melchior Jacobsen + Tested-by: Tomas Berger - Closes #5590 + Replaces #5397 + Reverts #5632 + Closes #5634 -Jay Satiro (22 Jun 2020) -- openssl: Don't ignore CA paths when using Windows CA store +- select: reduce duplication of Curl_poll in Curl_socket_check - This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does - not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default - locations. Instead the CA store can now be used at the same time. + Change Curl_socket_check to use select-fallback in Curl_poll + instead of implementing it in Curl_socket_check and Curl_poll. - The change is due to the impending release. The issue is still being - discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and - is now documented as experimental. + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Ref: bc052cc (parent commit) - Ref: https://github.com/curl/curl/issues/5585 + Replaces #5262 and #5492 + Closes #5707 -- tool_operate: Don't use Windows CA store as a fallback +- select: fix poll-based check not detecting connect failure - Background: + This commit changes Curl_socket_check to use POLLPRI to + check for connect failure on the write socket, because + POLLPRI maps to fds_err. This is in line with select(2). - 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate - store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides - CURLOPT_CAINFO if both are set. The curl tool will fall back to - CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set - via CURLOPT_CAINFO. + The select-based socket check correctly checks for connect + failures by adding the write socket also to fds_err. - Problem: + The poll-based implementation (which internally can itself + fallback to select again) did not previously check for + connect failure by using POLLPRI with the write socket. - libcurl may be built with hardcoded paths to a certificate bundle or - directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are - ignored. + See the follow up commit to this for more information. - Solution: + This commit makes sure connect failures can be detected + and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. - A solution is still being discussed but since there's an impending - release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Ref: https://github.com/curl/curl/issues/5585 + Replaces #5509 + Prepares #5707 -- openssl: Fix CA fallback logic for OpenSSL 3.0 build +- select.h: make socket validation macros test for INVALID_SOCKET + + With Winsock the valid range is [0..INVALID_SOCKET-1] according to + https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 - Prior to this change I assume a build error would occur when - CURL_CA_FALLBACK was used. + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5587 - -Daniel Stenberg (22 Jun 2020) -- copyright: update mismatched copyright years - -- test1460: verify that -Ji is not ok + Closes #5760 -- tool_getparam: -i is not OK if -J is used +Daniel Stenberg (24 Aug 2020) +- docs: --output-dir is added in 7.73.0, nothing else - Reported-by: sn on hackerone - Bug: https://curl.haxx.se/docs/CVE-2020-8177.html - -- [Peter Wu brought this change] + Follow-up to 5620d2cc78c0 - CMake: ignore INTERFACE_LIBRARY targets for pkg-config file +- curl: add --output-dir - Reviewed-by: Marcel Raad - Fixes #5512 - Closes #5517 - -- [Valentyn Korniienko brought this change] - - multibyte: Fixed access-> waccess to file for Windows Plarform + Works with --create-dirs and with -J - Reviewed-by: Marcel Raad - Closes #5580 - -- altsvc: bump to h3-29 + Add test 3008, 3009, 3011, 3012 and 3013 to verify. - Closes #5584 + Closes #5637 -- urlglob: treat literal IPv6 addresses with zone IDs as a host name +- configure: fix pkg-config detecting wolfssl - ... and not as a "glob". Now done by passing the supposed host to the - URL parser which supposedly will do a better job at identifying "real" - numerical IPv6 addresses. + When amending the include path with "/wolfssl", this now properly strips + off all whitespace from the path variable! Previously this would lead to + pkg-config builds creating bad command lines. - Reported-by: puckipedia on github - Fixes #5576 - Closes #5579 + Closes #5848 -- test1179: verify error message for non-existing cmdline option +- [Michael Musset brought this change] -- tool_getparam: repair the error message for unknown flag + sftp: add the option CURLKHSTAT_FINE_REPLACE - Follow-up to 9e5669f3880674 - Detected by Coverity CID 1464582 ("Logically dead code") + Replace the old fingerprint of the host with a new. - Closes #5577 + Closes #5685 -- FILEFORMAT: describe verify/stderr +- RELEASE-NOTES: synced + + The next release is now to become 7.73.0 -- connect: improve happy eyeballs handling +- checksrc: verify do-while and spaces between the braces - For QUIC but also for regular TCP when the second family runs out of IPs - with a failure while the first family is still trying to connect. + Updated mprintf.c to comply - Separated the timeout handling for IPv4 and IPv6 connections when they - both have a number of addresses to iterate over. - -- ngtcp2: never call fprintf() in lib code in release version + Closes #5845 -- ngtcp2: fix happy eyeballs quic connect crash +- curl: support XDG_CONFIG_HOME to find .curlrc - Reported-by: Peter Wu - Fixes #5565 - Closes #5568 - -- select: remove the unused ELAPSED_MS() macro + Added test433 to verify. Updated documentation. - Closes #5573 - -Marc Hoersken (17 Jun 2020) -- [rcombs brought this change] + Reviewed-by: Jay Satiro + Suggested-by: Eli Schwartz + Fixes #5829 + Closes #5837 - multi: implement wait using winsock events +- etag: save and use the full received contents - This avoids using a pair of TCP ports to provide wakeup functionality - for every multi instance on Windows, where socketpair() is emulated - using a TCP socket on loopback which could in turn lead to socket - resource exhaustion. + ... which makes it support weak tags and non-standard etags too! - Reviewed-by: Gergely Nagy - Reviewed-by: Marc Hörsken + Added test case 347 to verify blank incoming ETag: - Closes #5397 + Fixes #5610 + Closes #5833 -Daniel Stenberg (17 Jun 2020) -- manpage: add three missing environment variables +- setopt: if the buffer exists, refuse the new BUFFERSIZE - CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE + The buffer only exists during transfer and then we shouldn't change the + size (the setopt is not documented to work then). - Closes #5571 + Reported-by: Harry Sintonen + Closes #5842 -- RELEASE-NOTES: synced +- [COFFEETALES brought this change] -- configure: for wolfSSL, check for the DES func needed for NTLM + sftp: add new quote commands 'atime' and 'mtime' - Also adds pkg-config support for the wolfSSL detection. - -- [Ruurd Beerstra brought this change] + Closes #5810 - ntlm: enable NTLM support with wolfSSL - - When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* - functions that OpenSSL has. This change take advantage of that. +- CURLE_PROXY: new error code - Co-authored-by: Daniel Stenberg - Closes #5556 - Fixes #5548 - -- http: move header storage to Curl_easy from connectdata + Failures clearly returned from a (SOCKS) proxy now causes this return + code. Previously the situation was not very clear as what would be + returned and when. - Since the connection can be used by many independent requests (using - HTTP/2 or HTTP/3), things like user-agent and other transfer-specific - data MUST NOT be kept connection oriented as it could lead to requests - getting the wrong string for their requests. This struct data was - lingering like this due to old HTTP1 legacy thinking where it didn't - mattered.. + In addition: when this error code is returned, an application can use + CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then + returns a value from the new 'CURLproxycode' enum. - Fixes #5566 - Closes #5567 + Closes #5770 -- CODE_REVIEW.md: how to do code reviews in curl +- runtests: make cleardir() erase dot files too - Assisted-by: Daniel Gustafsson - Assisted-by: Rich Salz - Assisted-by: Hugo van Kemenade - Assisted-by: James Fuller - Assisted-by: Marc Hörsken - Assisted-by: Jay Satiro + Because test cases might use dot files. - Closes #5555 + Closes #5838 -- altsvc: remove the num field from the altsvc struct +- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses - It was superfluous since we have the list.size alredy + Also: the current behavior is now documented in the curl.1 and + CURLOPT_NOPROXY.3 man pages. - Reported-by: Jay Satiro - Fixes #5553 - Closes #5563 + Reported-by: Andrew Barnes + Closes #5745 + Closes #5841 -- version.d: expanded and alpha-sorted +Viktor Szakats (22 Aug 2020) +- Makefile.m32: add ability to override zstd libs [ci skip] - Added a few missing features not previously mentioned. Ordered them - alphabetically. + Similarly to brotli, where this was already possible. + E.g. it allows to link zstd statically to libcurl.dll. - Closes #5558 - -- ABI.md: rename to .md and polish the markdown + Ref: https://github.com/curl/curl-for-win/issues/12 + Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 - Closes #5562 + Closes https://github.com/curl/curl/pull/5840 -- HELP-US: add a section for "smaller tasks" +Daniel Stenberg (21 Aug 2020) +- runtests: avoid 'fail to start' repeated messages in attempt loops - The point of this section is to meet the CII Best Practices gold level - critera: + Closes #5834 + +- runtests: clear pid variables when failing to start a server - "The project MUST clearly identify small tasks that can be performed by - new or casual contributors" + ... as otherwise the parent doesn't detect the failure and believe it + actually worked to start. - Closes #5560 + Reported-by: Christian Weisgerber + Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html + Closes #5834 -- TODO: retry on the redirected-to URL +- TODO: Virtual external sockets - Closes #5462 - -- mailmap: Nicolas Sterchele + Closes #5835 -- [Nicolas Sterchele brought this change] +- [Don J Olmstead brought this change] - TODO: remove 19.3 section title - - Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on - curl documentation website + dist: add missing CMake Find modules to the distribution - Closes #5561 - -- [Martin V brought this change] + Closes #5836 - test1560: avoid possibly negative association in wording +- RELEASE-NOTES: synced - Closes #5549 + ... and version bumped to 7.72.1 -- share: don't set the share flag it something fails +- tls: provide the CApath verbose log on its own line - When asking for a specific feature to be shared in the share object, - that bit was previously set unconditionally even if the shared feature - failed or otherwise wouldn't work. + ... not newline separated from the previous line. This makes it output + asterisk prefixed properly like other verbose putput! - Closes #5554 + Reported-by: jmdavitt on github + Fixes #5826 + Closes #5827 -- buildconf: remove -print from the find command that removes files - - It's just too annoying and unnecessary to get a long list of files shown +Version 7.72.0 (19 Aug 2020) +Daniel Stenberg (19 Aug 2020) - RELEASE-NOTES: synced + + The curl 7.72.0 release -- wording: avoid blacklist/whitelist stereotypes +- THANKS: add names from curl 7.72.0 release + +Jay Satiro (18 Aug 2020) +- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions - Instead of discussing if there's value or meaning (implied or not) in - the colors, let's use words without the same possibly negative - associations. + Reported-by: plujon@users.noreply.github.com - Closes #5546 + Closes https://github.com/curl/curl/issues/5488 -Jay Satiro (9 Jun 2020) -- tool_getparam: fix memory leak in parse_args +Daniel Stenberg (17 Aug 2020) +- Curl_easy: remember last connection by id, not by pointer - Prior to this change in Windows Unicode builds most parsed options would - not be freed. + CVE-2020-8231 - Found using _CrtDumpMemoryLeaks(). + Bug: https://curl.haxx.se/docs/CVE-2020-8231.html - Ref: https://github.com/curl/curl/issues/5545 + Reported-by: Marc Aldorasi + Closes #5824 + +- examples/rtsp.c: correct the copyright year + +- RELEASE-PROCEDURE.md: add more future release dates -Daniel Stenberg (8 Jun 2020) -- socks: detect connection close during handshake +- [H3RSKO brought this change] + + docs: change "web site" to "website" + + According to wikipedia: - The SOCKS4/5 state machines weren't properly terminated when the proxy - connection got closed, leading to a busy-loop. + While "web site" was the original spelling, this variant has become + rarely used, and "website" has become the standard spelling - Reported-By: zloi-user on github - Fixes #5532 - Closes #5542 + Closes #5822 -- [James Fuller brought this change] +- [Bevan Weiss brought this change] - multi: add defensive check on data->multi->num_alive + CMake: don't complain about missing nroff - Closes #5540 - -- Curl_addrinfo: use one malloc instead of three + The curl_nroff_check() was always being called, and complaining if + *NROFF wasn't found, even when not making the manual. - To reduce the amount of allocations needed for creating a Curl_addrinfo - struct, make a single larger malloc instead of three separate smaller - ones. + Only check for nroff (and complain) if actually making the manual - Closes #5533 + Closes #5817 -- [Alessandro Ghedini brought this change] +- [Brian Inglis brought this change] - quiche: update SSLKEYLOGFILE support + libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin + + copy the LDFLAGS approach for adding same option with `libhostname` in + `libtest/Makefile.am`: - quiche now requires the application to explicitly set the keylog path - for each connection, rather than reading the environment variable - itself. + - init `libstubgss_la_LDFLAGS_EXTRA` variable, + - add option to variable inside conditional, + - use variable in `libstubgss_la_LDFLAGS` - Closes #5541 + Fixes #5819 + Closes #5820 -- tests: add two simple tests for --login-options +- docs: clarify MAX_SEND/RECV_SPEED functionality - Test 895 and 896 - as a follow-up to a3e972313b + ... in particular what happens if the maximum speed limit is set to a + value that's smaller than the transfer buffer size in use. - Closes #5539 + Reported-by: Tomas Berger + Fixes #5788 + Closes #5813 -- ngtcp2: update with recent API changes +- test1140: compare stdout - Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. + To make problems more immediately obvious when tests fail. - Assisted-by: Tatsuhiro Tsujikawa - Closes #5538 - -- [James Fuller brought this change] + Closes #5814 - socks: remove unreachable breaks in socks.c and mime.c +- asyn-ares: correct some bad comments - Closes #5537 + Closes #5812 -- tool_cfgable: free login_options at exit - - Memory leak - Reported-by: Geeknik Labs - Fixes #5535 - Closes #5536 +- [Emil Engler brought this change] -- libssh2: keep sftp errors as 'unsigned long' - - Remove weird work-around for storing the SFTP errors as int instead of - the "unsigned long" that libssh2 actually returns for SFTP errors. + docs: Add video link to docs/CONTRIBUTE.md - Closes #5534 + Closes #5811 -Marc Hoersken (6 Jun 2020) -- timeouts: move ms timeouts to timediff_t from int and long - - Now that all functions in select.[ch] take timediff_t instead - of the limited int or long, we can remove type conversions - and related preprocessor checks to silence compiler warnings. +- curl-config: ignore REQUIRE_LIB_DEPS in --libs output - Avoiding conversions from time_t was already done in 842f73de. + Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is + not considered for the --libs output. - Based upon #5262 - Supersedes #5214, #5220 and #5221 - Follow up to #5343 and #5479 - Closes #5490 + Reported-by: ramsay-jones on github + Assisted-by: Brian Inglis and Ken Brown + Fixes #5793 + Closes #5808 + +- copyright: update/correct the year range on a few files -Daniel Stenberg (6 Jun 2020) -- [François Rigault brought this change] +- scripts/copyright.pl: ignore .muse files - openssl: set FLAG_TRUSTED_FIRST unconditionally - - On some systems, openssl 1.0.x is still the default, but it has been - patched to contain all the recent security fixes. As a result of this - patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be - defined, while the previous behavior of openssl to not look at trusted - chains first, remains. +- [Emil Engler brought this change] + + multi: Remove 10-year old out-commented code - Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to - probe for the behavior of openssl based on the existence ofmacros. + The code hasn't been touched since 2010-08-18 - Closes #5530 + Closes #5805 -- server/util: fix logmsg format using curl_off_t argument +- KNOWN_BUGS: A shared connection cache is not thread-safe - ... this caused segfaults on armv7. + Closes #4915 + Closes #5802 + +- CONTRIBUTE: extend git commit message description - Regression added in dd0365d560aea5a (7.70.0) + In particular how the first line works. - Reviewed-by: Jay Satiro - Closes #5529 + Closes #5803 - RELEASE-NOTES: synced -- [Cherish98 brought this change] +- [Stefan Yohansson brought this change] - socks: fix expected length of SOCKS5 reply - - Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the - reply ATYP is X'01'. This resulted in erroneously expecting more bytes - when the request length is greater than the reply length (e.g., when - remotely resolving the hostname). + transfer: move retrycount from connect struct to easy handle - Closes #5527 - -Marc Hoersken (5 Jun 2020) -- .gitignore: add directory containing the stats repo + This flag was applied to the connection struct that is released on + retry. These changes move the retry counter into Curl_easy struct that + lives across retries and retains the new connection. - Since the new curl/stats repository is designed to be - checked out into the curl repository working tree as stats/ - it should be on the ignore list to aid in commit staging. - -Daniel Stenberg (5 Jun 2020) -- [Adnan Khan brought this change] + Reported-by: Cherish98 on github + Fixes #5794 + Closes #5800 - HTTP3.md: clarify cargo build directory +- libssh2: s/ssherr/sftperr/ - Cargo needs to be called from within the 'quiche' directory. + The debug output used ssherr instead of sftperr which not only outputs + the wrong error code but also casues a warning on Windows. - Closes #5522 - -- user-agent.d: spell out what happens given a blank argument + Follow-up to 7370b4e39f1 - Closes #5525 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 + Closes #5799 -- trailers: switch h1-trailer logic to use dynbuf +- ftp: don't do ssl_shutdown instead of ssl_close - In the continued effort to remove "manual" realloc schemes. + The shutdown function is for downgrading a connection from TLS to plain, + and this is not requested here. - Closes #5524 - -- CURLINFO_ACTIVESOCKET.3: clarify the description + Have ssl_close reset the TLS connection state. - Reported-by: Jay Satiro - Fixes #5299 - Closes #5520 - -- mailmap: Don J Olmstead - -- configure: only strip first -L from LDFLAGS + This partially reverts commit f002c850d98d - In the logic that works out if a given OpenSSL path works, it stripped - off a possibly leading -L flag using an incorrect sed pattern which - would remove all instances of -L in the string, including if the path - itself contained that two-letter sequence! + Reported-by: Rasmus Melchior Jacobsen + Reported-by: Denis Goleshchikhin + Fixes #5797 + +Marc Hoersken (9 Aug 2020) +- CI/azure: fix test outcome values and use latest API version - The same pattern was used and is now updated in multiple places. Now it - only removes -L if it starts the strings. + This makes sure that tests ignored or skipped are not shown + just in the category "Other", but with their correct state. - Reported-by: Mohamed Osama - Fixes #5519 - Closes #5521 + Closes #5796 -Peter Wu (4 Jun 2020) -- quiche: advertise draft 28 support +- CI/azure: show runtime stats to investigate slowness - Fix the verbose message while at it, quiche currently supports draft - 27 and draft 28 simultaneously. + Also avoid naming conflict of TFLAGS env and tflags variables. - Closes #5518 + Closes #5776 -Daniel Stenberg (4 Jun 2020) -- KNOWN_BUGS: RTSP authentication breaks without redirect support +Daniel Stenberg (8 Aug 2020) +- TLS naming: fix more Winssl and Darwinssl leftovers - Closes #4750 - -Jay Satiro (4 Jun 2020) -- projects: Add crypt32.lib to dependencies for all OpenSSL configs + The CMake option is now called CMAKE_USE_SCHANNEL - Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO - need crypt32. + The winbuild flag is USE_SCHANNEL - Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. + The CI jobs and build scripts only use the new names and the new name + options - The changes that are in this commit were made by script. + Tests now require 'Schannel' (when necessary) - Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 + Closes #5795 + +- smtp_parse_address: handle blank input string properly - Closes https://github.com/curl/curl/pull/5516 + Closes #5792 -Marc Hoersken (3 Jun 2020) -- CI/macos: fix 'is already installed' errors by using bundle +- runtests: run the DICT server on a random port number - Avoid failing CI builds due to nghttp2 being already installed. + Removed support for -b (base port number) - Closes #5513 - -Daniel Stenberg (3 Jun 2020) -- altsvc: fix 'dsthost' may be used uninitialized in this function + Closes #5783 - RELEASE-NOTES: synced -- urldata: let the HTTP method be in the set.* struct - - When the method is updated inside libcurl we must still not change the - method as set by the user as then repeated transfers with that same - handle might not execute the same operation anymore! - - This fixes the libcurl part of #5462 +- runtests: move the TELNET server to a dynamic port - Test 1633 added to verify. + Rename the port variable to TELNETPORT to better match the existing + pattern. - Closes #5499 + Closes #5785 -- hostip: fix the memory-leak introduced in 67d2802 +- ngtcp2: adapt to error code rename - Fixes #5503 - Closes #5504 + Closes #5786 -- test970: make it require proxy support - - This test verifies the -w %json output and the test case includes a full - generated "blob". If there's no proxy support built into libcurl, it - will return an error for proxy related info variables and they will not - be included in the json, thus causing a mismatch and this test fails. +- runtests: move the smbserver to use a dynamic port number - Reported-by: Marc Hörsken - Fixes #5501 - Closes #5502 - -- [Radoslav Georgiev brought this change] + Closes #5782 - examples/http2-down/upload: add error checks - - If `index.html` does not exist in the directory from which the example - is invoked, the fopen(upload, "rb") invocation in `setup` would fail, - returning NULL. This value is subsequently passed as the FILE* argument - of the `fread` invocation in the `read_callback` function, which is the - actual cause of the crash (apparently `fread` assumes that argument to - be non-null). - - In addition, mitigate some possible crashes of similar origin. +- runtests: run the http2 tests on a random port number - Closes #5463 - -- [kotoriのねこ brought this change] + Closes #5779 - examples/ephiperfifo: turn off interval when setting timerfd +- gtls: survive not being able to get name/issuer - Reported-by: therealhirudo on github - Fixes #5485 - Closes #5497 - -- [Saleem Abdulrasool brought this change] + Closes #5778 - vtls: repair the build with `CURL_DISABLE_PROXY` - - `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` - is enabled. Repair the build with that configuration. +- runtests: move the gnutls-serv tests to a dynamic port - Follow-up to f3d501dc67 + Affects test 320, 321, 322 and 324. - Closes #5498 + Closes #5778 -- transfer: remove k->str NULL check - - "Null-checking k->str suggests that it may be null, but it has already - been dereferenced on all paths leading to the check" - and it can't - legally be NULL at this point. Remove check. - - Detected by Coverity CID 1463884 +- runtests: support dynamicly base64 encoded sections in tests - Closes #5495 - -Marc Hoersken (1 Jun 2020) -- select: always use Sleep in Curl_wait_ms on Win32 + This allows us to make test cases to use base64 at run-time and still + use and verify information determined at run-time, such as the IMAP test + server's port number in test 842. - Since Win32 almost always will also have USE_WINSOCK, - we can reduce complexity and always use Sleep there. + This change makes 12 tests run again that basically never ran since we + moved to dynamic port numbers. - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ftpserver.pl is adjusted to load test instructions and test number from + the preprocessed test file. - Follow up to #5343 - Closes #5489 - -Daniel Stenberg (31 May 2020) -- conncache: download buffer needs +1 size for trailing zero + FILEFORMAT.md now documents the new base64 encoding syntax. - Follow-up to c4e6968127e - Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 + Reported-by: Marcel Raad + Fixes #5761 + Closes #5775 -Marc Hoersken (31 May 2020) -- azure: use matrix strategy to avoid configuration redundancy - - This also includes the following changes: - - - Use the same timeout for all jobs on Linux (60 minutes) - and Windows (90 minutes) - - Use CLI stable apt-get install -y instead of apt install - which warns about that and run apt-get update first - - Enable MQTT for Windows msys2 builds instead of - legacy msys1 builds - - Add ./configure --prefix parameter to the msys2 builds - - The MSYSTEM environment variable is now preset inside - the container images for the msys2 builds +- curl.1: add a few missing valid exit codes - Note: on Azure Pipelines the matrix strategy is basically - just a simple list of job copies and not really a matrix. + 93 - 96 can be returned as well. - Closes #5468 + Closes #5777 -Daniel Stenberg (30 May 2020) -- build: disable more code/data when built without proxy support - - Added build to travis to verify +- TODO: Use multiple parallel transfers for a single download - Closes #5466 + Closes #5774 -- url: alloc the download buffer at transfer start - - ... and free it as soon as the transfer is done. It removes the extra - alloc when a new size is set with setopt() and reduces memory for unused - easy handles. - - In addition: the closure_handle now doesn't use an allocated buffer at - all but the smallest supported size as a stack based one. +- TODO: Set the modification date on an uploaded file - Closes #5472 + Closes #5768 -- timeouts: change millisecond timeouts to timediff_t from time_t - - For millisecond timers we like timediff_t better. Also, time_t can be - unsigned so returning a negative value doesn't work then. +- [Thomas M. DuBuisson brought this change] + + CI: Add muse CI config - Closes #5479 + Closes #5772 + +- [Thomas M. DuBuisson brought this change] -Marc Hoersken (30 May 2020) -- select: add overflow checks for timeval conversions + travis/script.sh: fix use of `-n' with unquoted envvar - Using time_t and suseconds_t if suseconds_t is available, - long on Windows (maybe others in the future) and int elsewhere. + Shellcheck tells us "-n doesn't work with unquoted arguments. quote or + use [[ ]]." - Also handle case of ULONG_MAX being greater or equal to INFINITE. + And testing shows: - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + ``` + docker run --rm -it ubuntu bash + root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" + I ran + root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" + root@fe85ce156856:/# + ``` - Part of #5343 + Closes #5773 -- select: use timediff_t instead of time_t and int for timeout_ms +- h2: repair trailer handling - Make all functions in select.[ch] take timeout_ms as timediff_t - which should always be large enough and signed on all platforms - to take all possible timeout values and avoid type conversions. + The previous h2 trailer fix in 54a2b63 was wrong and caused a + regression: it cannot deal with trailers immediately when read since + they may be read off the connection by the wrong 'data' owner. - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + This change reverts the logic back to gathering all trailers into a + single buffer, like before 54a2b63. - Replaces #5107 and partially #5262 - Related to #5240 and #5286 - Closes #5343 + Reported-by: Tadej Vengust + Fixes #5663 + Closes #5769 -- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' +Viktor Szakats (3 Aug 2020) +- windows: disable Unix Sockets for old mingw - GCC 10 warns about this with warning: implicit conversion - from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] + Classic mingw and 10y+ old versions of mingw-w64 don't ship with + Windows headers having the typedef necessary for Unix Sockets + support, so try detecting these environments to disable this + feature. - Since 'expected_result' is not really of type 'CURLcode' and - it is not exposed in any way, we can just use 'SANITIZEcode'. + Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #5476 + Fixes #5674 + Closes #5758 -- tests/libtest: fix undefined reference to 'curlx_win32_fopen' - - Since curl_setup.h now makes use of curlx_win32_fopen for Win32 - builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, - we need to include the relevant files for tests using fopen, - because the libtest sources are also including curl_setup.h +Marcel Raad (3 Aug 2020) +- test1908: treat file as text - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg + Fixes the line endings on Windows. - Follow up to #3784 (ffdddb45d9) - Closes #5475 + Closes https://github.com/curl/curl/pull/5767 -- appveyor: add non-debug plain autotools-based build - - This should enable us to catch linking issues with the - testsuite early, like the one described/fixed in #5475. +- TrackMemory tests: ignore realloc and free in getenv.c - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad + These are only called for WIN32. - Closes #5477 + Closes https://github.com/curl/curl/pull/5767 + +Daniel Stenberg (3 Aug 2020) +- tests/FILEFORMAT.md: mention %HTTP2PORT -Daniel Stenberg (29 May 2020) - RELEASE-NOTES: synced -- Revert "buildconf: use find -execdir" - - This partially reverts commit c712009838f44211958854de431315586995bc61. +- tlsv1.3.d. only for TLS-using connections - Keep the ares_ files removed but bring back the older way to run find, - to make it work with busybox's find, as apparently that's being used. + ... and rephrase that "not all" TLS backends support it. - Reported-by: Max Peal - Fixes #5483 - Closes #5484 - -- server/sws: fix asan warning on use of uninitialized variable + Closes #5764 -- libssh2: improved error output for wrong quote syntax - - Reported-by: Werner Stolz +- tls-max.d: this option is only for TLS-using connections - Closes #5474 + Ref: #5763 + Closes #5764 -- mk-lib1521: generate code for testing BLOB options as well - - Follow-up to cac5374298b3 - - Closes #5478 +Marcel Raad (2 Aug 2020) +- [Cameron Cawley brought this change] -- configure: repair the check if argv can be written to - - Due to bad escaping of the test code, the test wouldn't build and thus - result in a negative test result, which would lead to the unconditional - assumption that overwriting the arguments doesn't work and thus curl - would never hide credentials given in the command line, even when it - would otherwise be possible. - - Regression from commit 2d4c2152c (7.60.0) + tool_doswin: Simplify Windows version detection - Reported-by: huzunhao on github - Fixes #5470 - Closes #5471 + Closes https://github.com/curl/curl/pull/5754 -Peter Wu (28 May 2020) -- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - - Otherwise the build might fail due to missing source files, as - demonstrated by the recent keylog.c addition on an existing build dir. - - Closes #5469 +- [Cameron Cawley brought this change] -Daniel Stenberg (28 May 2020) -- urldata: fix comments: Curl_done() is called multi_done() now + win32: Add Curl_verify_windows_version() to curlx - ... since 575e885db + Closes https://github.com/curl/curl/pull/5754 -Peter Wu (27 May 2020) -- ngtcp2: use common key log routine for better thread-safety +- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL - Tested with ngtcp2 built against the OpenSSL library. Additionally - tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + This makes the tests that require the OpenSSL feature also run for + those two compatible libraries. - The TLS backend (independent of QUIC) may or may not already have opened - the keylog file before. Therefore Curl_tls_keylog_open is always called - to ensure the file is open. + Closes https://github.com/curl/curl/pull/5762 -- wolfssl: add SSLKEYLOGFILE support - - Tested following the same curl and tshark commands as in commit - "vtls: Extract and simplify key log file handling from OpenSSL" using - WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with - `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. +Daniel Stenberg (1 Aug 2020) +- multi: Condition 'extrawait' is always true - Full support for this feature requires certain wolfSSL build options, - see "Availability note" in lib/vtls/wolfssl.c for details. + Reported by Codacy. - Closes #5327 + Reviewed-by: Marcel Raad + Closes #5759 -- vtls: Extract and simplify key log file handling from OpenSSL +Marcel Raad (1 Aug 2020) +- openssl: fix build with LibreSSL < 2.9.1 - Create a set of routines for TLS key log file handling to enable reuse - with other TLS backends. Simplify the OpenSSL backend as follows: + `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were + introduced in LibreSSL 2.9.1 [0]. - - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. - - Do not perform dynamic memory allocation when preparing a log entry. - Unless the TLS specifications change we can suffice with a reasonable - fixed-size buffer. - - Simplify state tracking when SSL_CTX_set_keylog_callback is - unavailable. My original sslkeylog.c code included this tracking in - order to handle multiple calls to SSL_connect and detect new keys - after renegotiation (via SSL_read/SSL_write). For curl however we can - be sure that a single master secret eventually becomes available - after SSL_connect, so a simple flag is sufficient. An alternative to - the flag is examining SSL_state(), but this seems more complex and is - not pursued. Capturing keys after server renegotiation was already - unsupported in curl and remains unsupported. + [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef - Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f - (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) - against an OpenSSL 1.1.1f server configured with: + Closes https://github.com/curl/curl/pull/5757 + +Daniel Stenberg (1 Aug 2020) +- [Marc Aldorasi brought this change] + + multi_remove_handle: close unused connect-only connections - # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 - openssl s_server -www -tls1 - # Likewise, but fail the server handshake. - openssl s_server -www -tls1 -Verify 2 - # TLS 1.3 test. No need to test the failing server handshake. - openssl s_server -www -tls1_3 + Previously any connect-only connections in a multi handle would be kept + alive until the multi handle was closed. Since these connections cannot + be re-used, they can be marked for closure when the associated easy + handle is removed from the multi handle. - Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly - written using Wireshark. For the first and third case, expect four - matches per connection (decrypted Server Finished, Client Finished, HTTP - Request, HTTP Response). For the second case where the handshake fails, - expect a decrypted Server Finished only. + Closes #5749 + +- checksrc: invoke script with -D to find .checksrc proper - tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ - -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ - -dtls.port==4433,http -ohttp.desegment_body:FALSE \ - -Y 'tls.handshake.verify_data or http' + Without the -D command line option, checksrc.pl won't know which + directory to load the ".checksrc" file from when building out of the + source tree. - A single connection can easily be identified via the `tcp.stream` field. - -Daniel Stenberg (27 May 2020) -- FILEFORMAT: add more features that tests can depend on + Reported-by: Marcel Raad + Fixes #5715 + Closes #5755 -- [Michael Kaufmann brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - transfer: close connection after excess data has been read - - For HTTP 1.x, it's a protocol error when the server sends more bytes - than announced. If this happens, don't reuse the connection, because the - start position of the next response is undefined. + buildconf: retire ares buildconf invocation - Closes #5440 + no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 -- [Estanislau Augé-Pujadas brought this change] +- [Carlo Marcelo Arenas Belón brought this change] - Revert "ssh: ignore timeouts during disconnect" - - This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in - curl 7.54.1. + buildconf: excempt defunct reference to ACLOCAL_FLAGS - Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html - Closes #5465 + retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as + the name is generic enough that it might be in use and relied upon from + the environment. -- urldata: connect related booleans live in struct ConnectBits - - And remove a few unused booleans! - - Closes #5461 +- [Carlo Marcelo Arenas Belón brought this change] -- hostip: on macOS avoid DoH when given a numerical IP address + buildconf: avoid array concatenation in die() - When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that - numerical IP addresses still need to get "resolved" - but not with DoH. + reported as error SC2145[1] by shellcheck, but not expected to cause + any behavioural differences otherwise. - Reported-by: Viktor Szakats - Fixes #5454 - Closes #5459 + [1] https://github.com/koalaman/shellcheck/wiki/SC2145 + + Closes #5701 -- ngtcp2: cleanup memory when failing to connect +- travis: add ppc64le and s390x builds - Reported-by: Peter Wu - Fixes #5447 (the ngtcp2 side of it) - Closes #5451 + Closes #5752 -- quiche: clean up memory properly when failing to connect +Marc Hoersken (31 Jul 2020) +- connect: remove redundant message about connect failure + + Reviewed-by: Daniel Stenberg - Addresses the quiche side of #5447 - Reported-by: Peter Wu - Closes #5450 + Closes #5708 -- cleanup: use a single space after equals sign in assignments +- tests/sshserver.pl: fix compatibility with OpenSSH for Windows + + Follow up to #5721 -- url: accept "any length" credentials for proxy auth +- CI/azure: install libssh2 for use with msys2-based builds - They're only limited to the maximum string input restrictions, not to - 256 bytes. + This enables building and running the SFTP tests. + Unfortunately OpenSSH for Windows does not support SCP (yet). - Added test 1178 to verify + Reviewed-by: Daniel Stenberg - Reported-by: Will Roberts - Fixes #5448 - Closes #5449 - -- [Maksim Stsepanenka brought this change] + Closes #5721 - test1167: fixes in badsymbols.pl +- CI/azure: increase Windows job timeout once again + + Avoid aborted jobs due to performance issues on Azure DevOps. + + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Closes #5442 + Closes #5738 -- altsvc: fix parser for lines ending with CRLF - - Fixed the alt-svc parser to treat a newline as end of line. +Jay Satiro (30 Jul 2020) +- TODO: Schannel: 'Add option to allow abrupt server closure' - The unit tests in test 1654 were done without CRLF and thus didn't quite - match the real world. Now they use CRLF as well. + We should offer an option to allow abrupt server closures (server closes + SSL transfer without sending a known termination point such as length of + transfer or close_notify alert). Abrupt server closures are usually + because of misconfigured or very old servers. - Reported-by: Peter Wu - Assisted-by: Peter Wu - Assisted-by: Jay Satiro - Fixes #5445 - Closes #5446 + Closes https://github.com/curl/curl/issues/4427 -Viktor Szakats (25 May 2020) -- all: fix codespell errors +- url: fix CURLU and location following - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5452 - -Peter Wu (25 May 2020) -- ngtcp2: fix build with current ngtcp2 master implementing draft 28 + Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was + incorrectly used for the location follow, resulting in infinite requests + to the original location. - Based on client.cc changes from ngtcp2. Tested with current git master, - ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. + Reported-by: sspiri@users.noreply.github.com - Fixes #5444 - Closes #5443 + Fixes https://github.com/curl/curl/issues/5709 + Closes https://github.com/curl/curl/pull/5713 -Daniel Stenberg (25 May 2020) +Daniel Stenberg (30 Jul 2020) - RELEASE-NOTES: synced - - moved the new setopts up to a "change" -- RELEASE-NOTES: synced +- [divinity76 brought this change] -- copyright: updated year ranges out of sync + docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions - ... and whitelisted a few more files in the the copyright.pl script. - -- [Gilles Vollant brought this change] - - setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency + it helps make it obvious that most developers don't have to care about + the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 + years old, November 4 2009) - Closes #5431 + Closes #5744 -- curl: remove -J "informational" written on stdout +Jay Satiro (29 Jul 2020) +- tool_cb_wrt: fix outfile mode flags for Windows - curl would previously show "curl: Saved to filename 'name from header'" - if -J was used and a name was picked from the Content-Disposition - header. That output could interfer with other stdout output, such as -w. + - Use S_IREAD and S_IWRITE mode permission flags to create the file + on Windows instead of S_IRUSR, S_IWUSR, etc. - This commit removes that output line. - Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html - Reported-by: Коваленко Анатолий Викторович - Closes #5435 - -Peter Wu (22 May 2020) -- travis: simplify quiche build instructions wrt boringssl + Windows only accepts a combination of S_IREAD and S_IWRITE. It does not + acknowledge other combinations, for which it may generate an assertion. - quiche builds boringssl as static library, reuse that instead of - building another shared library. + This is a follow-up to 81b4e99 from yesterday, which improved the + existing file check with -J. - Closes #5438 - -- configure: fix pthread check with static boringssl + Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks + Ref: https://github.com/curl/curl/pull/5731 - A shared boringssl/OpenSSL library requires -lcrypto only for linking. - A static build additionally requires `-ldl -lpthread`. In the latter - case `-lpthread` is added to LIBS which prevented `-pthread` from being - added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. + Closes https://github.com/curl/curl/pull/5742 -Daniel Stenberg (22 May 2020) -- Revert "sendf: make failf() use the mvsnprintf() return code" - - This reverts commit 74623551f306990e70c7c5515b88972005604a74. +Daniel Stenberg (28 Jul 2020) +- checksrc: ban gmtime/localtime - Instead mark the function call with (void). Getting the return code and - using it instead triggered Coverity warning CID 1463596 because - snprintf() can return a negative value... + They're not thread-safe so they should not be used in libcurl code. - Closes #5441 - -- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' + Explictly enabled when deemed necessary and in examples and tests - Reported-by: Billyzou0741326 on github - Fixes #5432 - Closes #5436 + Reviewed-by: Nicolas Sterchele + Closes #5732 -- tests/server/util.h: add extern to silence compiler warning +- transfer: fix data_pending for builds with both h2 and h3 enabled - Follow-up from a3b0699d5c1 + Closes #5734 -- typecheck-gcc.h: fix the OFF_T check - - The option number also needs to be less than CURLOPTTYPE_BLOB. +- curl_multi_setopt: fix compiler warning "result is always false" - Follow-up to cac5374298 - Reported-by: Jeroen Ooms - Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 - -- TODO: --dry-run + On systems with 32 bit long the expression is always false. Avoid + the warning. - Closes #5426 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 + Closes #5736 -- TODO: Ratelimit or wait between serial requests +- curl: improve the existing file check with -J - Closes #5406 - -- tool_paramhlp: fixup C89 mistake + Previously a file that isn't user-readable but is user-writable would + not be properly avoided and would get overwritten. - Follow-up to c5f0a9db22. + Reported-by: BrumBrum on hackerone + Assisted-by: Jay Satiro + Bug: https://hackerone.com/reports/926638 + Closes #5731 -- [Siva Sivaraman brought this change] +- [Jonathan Nieder brought this change] - tool_paramhlp: fixed potentially uninitialized strtol() variable + multi: update comment to say easyp list is linear - Seems highly unlikely to actually be possible, but better safe than - sorry. + Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, + 2013-08-02), the easy handle list is not circular but ends with + ->next pointing to NULL. - Closes #5417 - -- [Siva Sivaraman brought this change] + Reported-by: Masaya Suzuki + Closes #5737 - tool_operate: fixed potentially uninitialized variables +- CURLOPT_NOBODY.3: fix the syntax for referring to options - ... in curl_easy_getinfo() calls. They're harmless but clearing the - variables makes the code safer and comforts the reader. + As test 1140 fails otherwise! - Closes #5416 + Follow-up to e1bac81cc815 -- sha256: move assign to the declaration line +- ngtcp2: store address in sockaddr_storage - Follow-up to fae30656. Should've been squashed with that commit... - -- [Siva Sivaraman brought this change] + Reported-by: Tatsuhiro Tsujikawa + Closes #5733 - sha256: fixed potentially uninitialized variable +- CURLOPT_NOBODY.3: clarify what setting to 0 means - Closes #5414 + ... and mention that HTTP with other methods than HEAD might get a body and + there's no option available to stop that. + + Closes #5729 -- sendf: make failf() use the mvsnprintf() return code +- setopt: unset NOBODY switches to GET if still HEAD - ... and avoid a strlen() call. Fixes a MonocleAI warning. + Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented + action but before 7.71.0 that used to switch back to GET and with this + change (assuming the method is still set to HEAD) this behavior is + brought back. - Reported-by: MonocleAI - Fixes #5413 - Closes #5420 + Reported-by: causal-agent on github + Fixes #5725 + Closes #5728 -- hostip: make Curl_printable_address not return anything +- [Ehren Bendler brought this change] + + configure: cleanup wolfssl + pkg-config conflicts when cross compiling. - It was not used much anyway and instead we let it store a blank buffer - in case of failure. + Also choose a different wolfSSL function to test for NTLM support. - Reported-by: MonocleAI - Fixes #5411 - Closes #5418 + Fixes #5605 + Closes #5682 -- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) +- configure: show zstd "no" in summary when built without it - They're done on purpose, make that visible in the code. - Reported-by: MonocleAI - Fixes #5412 - Closes #549 + Reported-by: Marc Hörsken + Fixes #5720 + Closes #5730 -- TODO: forbid TLS post-handshake auth and do TLS record padding +- quiche: handle calling disconnect twice - Closes #5396 - Closes #5398 + Reported-by: lilongyan-huawei on github + Fixes #5726 + Closes #5727 -- RELEASE-NOTES: synced +- [Nicolas Sterchele brought this change] -- dynbuf: return NULL when there's no buffer length + getinfo: reset retry-after value in initinfo - ... as returning a "" is not a good idea as the string is supposed to be - allocated and returning a const string will cause issues. + - Avoid re-using retry_after value from preceding request + - Add libtest 3010 to verify - Reported-by: Brian Carpenter - Follow-up to ed35d6590e72c - Closes #5405 + Reported-by: joey-l-us on github + Fixes #5661 + Closes #5672 -Peter Wu (16 May 2020) -- travis: upgrade to bionic, clang-9, improve readability - - Changes, partially to reduce build failures from external dependencies: - - Upgrade Ubuntu and drop unnecessary third-party repos. - - Properly clone apt config to ensure retries. - - Upgrade to clang-9 from the standard repos. - - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey - fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use - focal everywhere yet since Travis CI has not documented this option. - In focal, python-impacket (Py2.7) has been removed, leaving only - python3-impacket. Since it is only needed for SMB tests and not SSH, - skip it for the libssh job since it might need more work. - - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. +Marcel Raad (27 Jul 2020) +- WIN32: stop forcing narrow-character API - Non-functional cleanups: - - Simplify test matrix, drop redundant os and compiler keys. - - Deprecation fixes: remove sudo, rename matrix -> jobs. - - Every job has an 'env' key, put this key first in a list item. + Except where the results are only used for character output. + getenv is not touched because it's part of the public API, and having + it return UTF-8 instead of ANSI would be a breaking change. - Closes #5370 + Fixes https://github.com/curl/curl/issues/5658 + Fixes https://github.com/curl/curl/issues/5712 + Closes https://github.com/curl/curl/pull/5718 -- travis: whitespace-only changes for consistency - - Automatically apply a consistent indentation with: +Jay Satiro (27 Jul 2020) +- [Tobias Stoeckmann brought this change] + + mprintf: Fix stack overflows - python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' + Stack overflows can occur with precisions for integers and floats. - followed by manually re-indenting three comments. + Proof of concepts: + - curl_mprintf("%d, %.*1$d", 500, 1); + - curl_mprintf("%d, %+0500.*1$f", 500, 1); - Closes #5370 - -- CMake: add libssh build support + Ideally, compile with -fsanitize=address which makes this undefined + behavior a bit more defined for debug purposes. - Closes #5372 - -Daniel Stenberg (15 May 2020) -- KNOWN_BUGS: wolfssh: publickey auth doesn't work + The format strings are valid. The overflows occur due to invalid + arguments. If these arguments are variables with contents controlled + by an attacker, the function's stack can be corrupted. - Closes #4820 - -- KNOWN_BUGS: OS400 port requires deprecated IBM library + Also see CVE-2016-9586 which partially fixed the float aspect. - Closes #5176 - -- [Vyron Tsingaras brought this change] - - http2: keep trying to send pending frames after req.upload_done + Signed-off-by: Tobias Stoeckmann - Fixes #1410 - Closes #5401 + Closes https://github.com/curl/curl/pull/5722 -- [Gilles Vollant brought this change] +- [Tobias Stoeckmann brought this change] - setopt: support certificate options in memory with struct curl_blob + mprintf: Fix dollar string handling - This change introduces a generic way to provide binary data in setopt - options, called BLOBs. + Verify that specified parameters are in range. If parameters are too + large, fail early on and avoid out of boundary accesses. - This change introduces these new setopts: + Also do not read behind boundaries of illegal format strings. - CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, - CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. + These are defensive measures since it is expected that format strings + are well-formed. Format strings should not be modifiable by user + input due to possible generic format string attacks. - Reviewed-by: Daniel Stenberg - Closes #5357 + Closes https://github.com/curl/curl/pull/5722 -- source cleanup: remove all custom typedef structs - - - Stick to a single unified way to use structs - - Make checksrc complain on 'typedef struct {' - - Allow them in tests, public headers and examples +Daniel Stenberg (26 Jul 2020) +- ntlm: free target_info before (re-)malloc - - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually - typedef different types/structs depending on build conditions. + OSS-Fuzz found a way this could get called again with the pointer still + pointing to a malloc'ed memory, leading to a leak. - Closes #5338 - -- travis: remove the .checksrc fiddling - -- ftp: make domore_getsock() return the secondary socket properly + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 - Previously, after PASV and immediately after the data connection has - connected, the function would only return the control socket to wait for - which then made the data connection simply timeout and not get polled - correctly. This become obvious when running test 1631 and 1632 event- - based. - -- test1632: verify FTP through HTTPS-proxy with connection re-use - -- test1631: verify FTP download through HTTPS-proxy + Closes #5724 -- sws: as last resort, get test number from server cmd file +Marcel Raad (26 Jul 2020) +- CI/macos: set minimum macOS version - If it can't be found in the request. Also support --cmdfile to set it to - a custom file name. + This enables some deprecation warnings. + Previously, autotools defaulted to 10.8. - runtests.pl always writes this file with the test number in it since a - while back. + Closes https://github.com/curl/curl/pull/5723 -- ftp: shut down the secondary connection properly when SSL is used - - Reported-by: Neal Poole - Fixes #5340 - Closes #5385 +Daniel Stenberg (26 Jul 2020) +- RELEASE-NOTES: synced -Marcel Raad (14 May 2020) -- KNOWN_BUGS: adapt 5.5 to recent changes - - It only applies to non-Unicode builds now. - Also merge 5.10 into it as it's effectively a duplicate. +Marcel Raad (25 Jul 2020) +- CI/macos: enable warnings as errors for CMake builds - Closes https://github.com/curl/curl/pull/3784 + Closes https://github.com/curl/curl/pull/5716 -- curl_setup: support Unicode functions to open files on Windows +- CMake: fix test for warning suppressions + + GCC doesn't warn for unknown `-Wno-` options, except if there are other + warnings or errors [0]. This was problematic with `CURL_WERROR` as that + warning-as-error cannot be suppressed. Notably, this always happened + with `-Wno-pedantic-ms-format` when not targeting Windows. So test for + the positive form of the warning instead, which should always result in + a diagnostic if unknown. - Use them only if `_UNICODE` is defined, in which case command-line - arguments have been converted to UTF-8. + [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html - Closes https://github.com/curl/curl/pull/3784 + Closes https://github.com/curl/curl/pull/5714 -- tool: support UTF-16 command line on Windows - - - use `wmain` instead of `main` when `_UNICODE` is defined [0] - - define `argv_item_t` as `wchar_t *` in this case - - use the curl_multibyte gear to convert the command-line arguments to - UTF-8 +Jay Satiro (23 Jul 2020) +- curl.h: update CURLINFO_LASTONE - This makes it possible to pass parameters with characters outside of - the current locale on Windows, which is required for some tests, e.g. - the IDN tests. Out of the box, this currently only works with the - Visual Studio project files, which default to Unicode, and winbuild - with the `ENABLE_UNICODE` option. + CURLINFO_LASTONE should have been updated when + CURLINFO_EFFECTIVE_METHOD was added. - [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + Reported-by: xwxbug@users.noreply.github.com - Ref: https://github.com/curl/curl/issues/3747 - Closes https://github.com/curl/curl/pull/3784 + Fixes https://github.com/curl/curl/issues/5711 -- curl_multibyte: add to curlx +Marc Hoersken (22 Jul 2020) +- CI/azure: unconditionally enable warnings-as-errors with autotools - This will also be needed in the tool and tests. + Reviewed-by: Marcel Raad - Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 - Closes https://github.com/curl/curl/pull/3784 + Follow up to #5694 + Closes #5706 -Daniel Stenberg (14 May 2020) -- url: make the updated credentials URL-encoded in the URL +Marcel Raad (21 Jul 2020) +- doh: remove redundant cast - Found-by: Gregory Jefferis - Reported-by: Jeroen Ooms - Added test 1168 to verify. Bug spotted when doing a redirect. - Bug: https://github.com/jeroen/curl/issues/224 - Closes #5400 + Closes https://github.com/curl/curl/pull/5704 -- tests: add https-proxy support to the test suite +- CI/macos: unconditionally enable warnings-as-errors with autotools - Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like - HTTP proxy but with a full TLS connection to the proxy. + Previously, warnings were only visible in the output for most jobs. - Closes #5399 - -- mailmap: James Fuller - -- [Major_Tom brought this change] + Closes https://github.com/curl/curl/pull/5694 - vauth/cleartext: fix theoretical integer overflow - - Fix theoretical integer overflow in Curl_auth_create_plain_message. +- util: silence conversion warnings - The security impact of the overflow was discussed on hackerone. We - agreed this is more of a theoretical vulnerability, as the integer - overflow would only be triggerable on systems using 32-bits size_t with - over 4GB of available memory space for the process. + timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might + be a 64-bit integer. This is the case when building for recent macOS + versions, for example. Just treat tv_usec as an int, which should + hopefully always be sufficient on systems with + `HAVE_CLOCK_GETTIME_MONOTONIC`. - Closes #5391 + Closes https://github.com/curl/curl/pull/5695 -Jay Satiro (13 May 2020) -- curl.1: Quote globbed URLs +- md(4|5): don't use deprecated macOS functions + + They are marked as deprecated for -mmacosx-version-min >= 10.15, + which might result in warnings-as-errors. - - Quote the globbing example URLs that contain characters [] {} since - otherwise they may be interpreted as shell metacharacters. + Closes https://github.com/curl/curl/pull/5695 + +Daniel Stenberg (18 Jul 2020) +- strdup: remove the odd strlen check - Bug: https://github.com/curl/curl/issues/5388 - Reported-by: John Simpson + It confuses code analyzers with its use of -1 for unsigned value. Also, + a check that's not normally used in strdup() code - and not necessary. - Closes https://github.com/curl/curl/pull/5394 + Closes #5697 -Daniel Stenberg (14 May 2020) -- checksrc: enhance the ASTERISKSPACE and update code accordingly +- [Alessandro Ghedini brought this change] + + travis: update quiche builds for new boringssl layout - Fine: "struct hello *world" + This is required after https://github.com/cloudflare/quiche/pull/593 + moved BoringSSL around slightly. - Not fine: "struct hello* world" (and variations) + This also means that Go is not needed to build BoringSSL anymore (the + one provided by quiche anyway). - Closes #5386 + Closes #5691 -- docs/options-in-versions: which version added each cmdline option +Marcel Raad (17 Jul 2020) +- configure: allow disabling warnings - Added test 971 to verify that the list is in sync with the files in - cmdline-opts. The check also verifies that .d-files that uses Added: - specify the same version number as the options-in-versions file does. + When using `--enable-warnings`, it was not possible to disable warnings + via CFLAGS that got explicitly enabled. Now warnings are not enabled + anymore if they are explicitly disabled (or enabled) in CFLAGS. This + works for at least GCC, clang, and TCC as they have corresponding + `-Wno-` options for every warning. - Closes #5381 + Closes https://github.com/curl/curl/pull/5689 -- docs: unify protocol lists - - We boast support for 25 transfer protocols. Make sure the lists are - consistent +Daniel Stenberg (16 Jul 2020) +- ngtcp2: adjust to recent sockaddr updates - Closes #5384 + Closes #5690 diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index 69c3c11dca..91a9f3c396 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -45,6 +45,7 @@ Alan Pinstein Albert Chin-A-Young Albert Choy Alejandro Alvarez Ayllon +Alejandro Colomar Alejandro R. Sedeño Aleksandar Milivojevic Aleksey Tulinov @@ -72,6 +73,7 @@ Alex Rousskov Alex Samorukov Alex Suykov Alex Vinnik +Alex Xu Alexander Beedie Alexander Dyagilev Alexander Elgert @@ -98,6 +100,7 @@ Alfonso Martone Alfred Gebert Allen Pulsifer Alona Rossen +Amaury Denoyelle amishmm on github Amit Katyal Amol Pattekar @@ -129,6 +132,7 @@ Andreas Schuldei Andreas Streichardt Andreas Wurf Andrei Benea +Andrei Bica Andrei Cipu Andrei Karas Andrei Kurushin @@ -169,6 +173,7 @@ Anthon Pang Anthony Avina Anthony Bryan Anthony G. Basile +Anthony Ramine Antoine Aubert Antoine Calando Anton Bychkov @@ -193,6 +198,7 @@ Artak Galoyan Arthur Murray Arve Knudsen Arvid Norberg +arvids-kokins-bidstack on github asavah on github Ashish Shukla Ashwin Metpalli @@ -202,6 +208,7 @@ Ates Goral Augustus Saunders Austin Green Avery Fay +awesomenode on github Axel Tillequin Ayoub Boudhar b9a1 on github @@ -269,6 +276,7 @@ Bob Relyea Bob Richmond Bob Schader bobmitchell1956 on github +Bodo Bergmann Bogdan Nicula Brad Burdick Brad Fitzpatrick @@ -321,6 +329,7 @@ Camille Moncelier Caolan McNamara Captain Basil Carie Pointer +Carl Zogheib Carlo Cannas Carlo Marcelo Arenas Belón Carlo Teubner @@ -491,11 +500,14 @@ David Binderman David Blaikie David Byron David Cohen +David Demelier David E. Narváez David Earl David Eriksson David Garske +David Goerger David Houlder +David Hu David Hull David J Meyer David James @@ -610,6 +622,7 @@ Dániel Bakai Early Ehlinger Earnestly on github Eason-Yu on github +ebejan on github Ebenezer Ikonne Ed Morley Edgaras Janušauskas @@ -706,6 +719,7 @@ Felix Yan Feng Tu Fernando Muñoz Filip Salomonsson +Firefox OS Flameborn on github Flavio Medeiros Florian Pritz @@ -791,6 +805,7 @@ Greg Onufer Greg Pratt Greg Rowe Greg Zavertnik +Gregor Jasny Gregory Jefferis Gregory Nicholls Gregory Szorc @@ -901,6 +916,7 @@ Ivan Avdeev IvanoG on github Ivo Bellin Salarin iz8mbw on github +Jack Boos Yu Jack Zhang Jackarain on github Jacky Lam @@ -972,6 +988,7 @@ Jean-Louis Lemaire Jean-Marc Ranger Jean-Noël Rouvignac Jean-Philippe Barrette-LaPierre +Jean-Philippe Menil Jeff Connelly Jeff Hodges Jeff Johnson @@ -1026,6 +1043,7 @@ Joe Malicki Joe Mason Joel Chen Joel Depooter +Joel Teichroeb joey-l-us on github Jofell Gallardo Johan Anderson @@ -1035,6 +1053,7 @@ Johan van Selst Johannes Bauer Johannes Ernst Johannes G. Kristinsson +Johannes Lesr Johannes Schindelin John A. Bristor John Bradshaw @@ -1092,13 +1111,15 @@ Jonathan Cardoso Machado Jonathan Hseu Jonathan Moerman Jonathan Nieder +Jonathan Watt Jongki Suwandi -jonrumsey on github Joombalaya on github Joonas Kuorilehto +Jordan Brown Jose Alf Jose Kahan Josef Wolf +Joseph Chen Josh Bialkowski Josh Kapell joshhe on github @@ -1127,6 +1148,7 @@ Julien Chaffraix Julien Nabet Julien Royer Jun-ichiro itojun Hagino +Jun-ya Kato jungle-boogie on github Junho Choi Jurij Smakov @@ -1194,6 +1216,7 @@ Klaus Stein Klevtsov Vadim Kobi Gurkan Koen Dergent +kokke on github Konstantin Isakov Konstantin Kushnir kotoriのねこ @@ -1239,6 +1262,7 @@ Laurent Bonnans Laurent Rabret Lauri Kasanen Laurie Clark-Michalek +Lawrence Gripper Lawrence Matthews Lawrence Wagerfield Legoff Vincent @@ -1255,6 +1279,7 @@ Leon Breedt Leon Winter Leonardo Rosati Leonardo Taccari +Li Xinwei Liam Healy lijian996 on github Lijo Antony @@ -1312,6 +1337,7 @@ Mandy Wu Manfred Schwarb MAntoniak on github Manuel Massing +Manuj Bhatia Marc Aldorasi Marc Boucher Marc Deslauriers @@ -1441,6 +1467,7 @@ Michael Anti Michael Baentsch Michael Benedict Michael Brehm +Michael Brown Michael Calmer Michael Cronenworth Michael Curtis @@ -1449,6 +1476,7 @@ Michael Felt Michael Forney Michael Gmelin Michael Goffioul +Michael Hordijk Michael Jahn Michael Jerris Michael Kalinin @@ -1685,6 +1713,7 @@ Pedro Monreal Pedro Neves pendrek at hackerone Peng Li +Per Jensen Per Lundberg Per Malmberg Per Nilsson @@ -1860,6 +1889,7 @@ Robert Kolcun Robert Linden Robert Olson Robert Prag +Robert Ronto Robert Schumann Robert Weaver Robert Wruck @@ -2239,6 +2269,7 @@ Vlad Ureche Vladimir Grishchenko Vladimir Kotal Vladimir Lazarenko +Vladimir Varlamov Vlastimil Ovčáčík Vojtech Janota Vojtech Minarik @@ -2315,14 +2346,17 @@ Zhao Yisha Zhaoyang Wu Zhibiao Wu Zhouyihai Ding +ZimCodes on github zloi-user on github Zmey Petroff Zvi Har'El zzq1015 on github +Ádler Jonas Gross İsmail Dönmez Łukasz Domeradzki Štefan Kremeň Коваленко Анатолий Викторович Никита Дорохин +ウさん 不确定 加藤郁之 -- cgit v1.2.3