From 2f0eaffffe7e5d3ac074797f880480494dbcc6f3 Mon Sep 17 00:00:00 2001 From: dartraiden Date: Sun, 7 Feb 2021 18:49:47 +0300 Subject: libcurl: update to 7.75.0 --- libs/libcurl/docs/CHANGES | 10055 ++++++++++++++++++++++---------------------- libs/libcurl/docs/COPYING | 2 +- libs/libcurl/docs/THANKS | 30 + 3 files changed, 5062 insertions(+), 5025 deletions(-) (limited to 'libs/libcurl/docs') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 56859b4993..f3439fd046 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,4462 +6,4653 @@ Changelog -Version 7.74.0 (9 Dec 2020) +Version 7.75.0 (3 Feb 2021) -Daniel Stenberg (9 Dec 2020) +Daniel Stenberg (3 Feb 2021) - RELEASE-NOTES: synced - - for 7.74.0 -Jay Satiro (7 Dec 2020) -- [Jacob Hoffman-Andrews brought this change] +- THANKS: added contributors from 7.75.0 - urldata: restore comment on ssl_connect_data.use - - This comment was originally on the `use` field, but was separated from - its field in 62a2534. - - Closes https://github.com/curl/curl/pull/6287 +- copyright: fix year ranges in need of updates -Daniel Stenberg (7 Dec 2020) -- VERSIONS: refreshed +- TODO: remove items for next SONAME bump etc - We always use the patch number these days: all releases are - "major.minor.patch" + We want to avoid that completely, so we don't plan for things after such + an event. -- [Jakub Zakrzewski brought this change] +- [Jay Satiro brought this change] - cmake: don't use reserved target name 'test' - - CMake up to 3.10 always reserves this name + ngtcp2: Fix build error due to change in ngtcp2_settings - Fixes #6257 - Closes #6258 - -- openssl: make the OCSP verification verify the certificate id + - Separate ngtcp2_transport_params. - CVE-2020-8286 + ngtcp2/ngtcp2@05d7adc made ngtcp2_transport_params separate from + ngtcp2_settings. - Reported by anonymous + ngtcp2 master is required to build curl with http3 support. - Bug: https://curl.se/docs/CVE-2020-8286.html + Closes #6554 -- ftp: make wc_statemach loop instead of recurse +- vtls: remove md5sum - CVE-2020-8285 + As it is not used anymore. - Fixes #6255 - Bug: https://curl.se/docs/CVE-2020-8285.html - Reported-by: xnynx on github + Reported-by: Jacob Hoffman-Andrews + Bug: https://curl.se/mail/lib-2021-02/0000.html + + Closes #6557 -- ftp: CURLOPT_FTP_SKIP_PASV_IP by default +- [Alessandro Ghedini brought this change] + + quiche: don't use primary_ip / primary_port - The command line tool also independently sets --ftp-skip-pasv-ip by - default. + Closes #6555 + +Alessandro Ghedini (1 Feb 2021) +- travis: enable quiche's FFI feature + +Daniel Stenberg (30 Jan 2021) +- [Dmitry Wagin brought this change] + + http: improve AWS HTTP v4 Signature auth - Ten test cases updated to adapt the modified --libcurl output. + - Add support services without region and service prefixes in + the URL endpoint (ex. Min.IO, GCP, Yandex Cloud, Mail.Ru Cloud Solutions, etc) + by providing region and service parameters via aws-sigv4 option. + - Add [:region[:service]] suffix to aws-sigv4 option; + - Fix memory allocation errors. + - Refactor memory management. + - Use Curl_http_method instead() STRING_CUSTOMREQUEST. + - Refactor canonical headers generating. + - Remove repeated sha256_to_hex() usage. + - Add some docs fixes. + - Add some codestyle fixes. + - Add overloaded strndup() for debug - curl_dbg_strndup(). + - Update tests. - Bug: https://curl.se/docs/CVE-2020-8284.html - CVE-2020-8284 + Closes #6524 + +- hyper: fix CONNECT to set 'data' as userdata - Reported-by: Varnavas Papaioannou + Follow-up to 14e075d1a7fd -- urlapi: don't accept blank port number field without scheme +- [Layla brought this change] + + connect: fix compile errors in `Curl_conninfo_local` - ... as it makes the URL parser accept "very-long-hostname://" as a valid - host name and we don't want that. The parser now only accepts a blank - (no digits) after the colon if the URL starts with a scheme. + .. for the `#else` (`!HAVE_GETSOCKNAME`) case - Reported-by: d4d on hackerone + Fixes https://github.com/curl/curl/issues/6548 + Closes #6549 - Closes #6283 + Signed-off-by: Layla -- Revert "multi: implement wait using winsock events" - - This reverts commit d2a7d7c185f98df8f3e585e5620cbc0482e45fac. +- [Michał Antoniak brought this change] + + transfer: fix GCC 10 warning with flag '-Wint-in-bool-context' - This commit also reverts the subsequent follow-ups to that commit, which - were all done within windows #ifdefs that are removed in this - change. Marc helped me verify this. + ... and return the error code from the Curl_mime_rewind call. - Fixes #6146 - Closes #6281 + Closes #6537 -- [Klaus Crusius brought this change] +- [Michał Antoniak brought this change] - ftp: retry getpeername for FTP with TCP_FASTOPEN + avoid warning: enum constant in boolean context + +- copyright: fix missing year (range) updates + +- RELEASE-NOTES: synced + +- openssl: lowercase the hostname before using it for SNI - In the case of TFO, the remote host name is not resolved at the - connetion time. + ... because it turns out several servers out there don't actually behave + correctly otherwise in spite of the fact that the SNI field is + specifically said to be case insensitive in RFC 6066 section 3. - For FTP that has lead to missing hostname for the secondary connection. - Therefore the name resolution is done at the time, when FTP requires it. + Reported-by: David Earl + Fixes #6540 + Closes #6543 + +- KNOWN_BUGS: cmake: ExternalProject_Add does not set CURL_CA_PATH - Fixes #6252 - Closes #6265 - Closes #6282 + Closes #6313 -- [Thomas Danielsson brought this change] +- KNOWN_BUGS: Multi perform hangs waiting for threaded resolver + + Closes #4852 - scripts/completion.pl: parse all opts +- KNOWN_BUGS: "pulseUI VPN client" is known to be buggy - For tab-completion it may be preferable to include all the - available options. + First entry in the new section "applications" for known problems in + libcurl using applications. - Closes #6280 + Closes #6306 -- RELEASE-NOTES: synced - -- openssl: use OPENSSL_init_ssl() with >= 1.1.0 +- tool_writeout: make %{errormsg} blank for no errors - Reported-by: Kovalkov Dmitrii and Per Nilsson - Fixes #6254 - Fixes #6256 - Closes #6260 + Closes #6539 -- SECURITY-PROCESS: disclose on hackerone +Jay Satiro (27 Jan 2021) +- [Gisle Vanem brought this change] + + build: fix djgpp builds - Once a vulnerability has been published, the hackerone issue should be - disclosed. For tranparency. + - Update build instructions in packages/DOS/README - Closes #6275 - -Marc Hoersken (3 Dec 2020) -- tests/util.py: fix compatibility with Python 2 + - Extend 'VPATH' with 'vquic' and 'vssh'. - Backporting the Python 3 implementation of setStream - to ClosingFileHandler as a fallback within Python 2. + - Allow 'Makefile.dist' to build both 'lib' and 'src'. - Reported-by: Jay Satiro + - Allow using the Windows hosted djgpp cross compiler to build for MSDOS + under Windows. - Fixes #6259 - Closes #6270 - -Daniel Gustafsson (3 Dec 2020) -- docs: fix typos and markup in ETag manpage sections + - 'USE_SSL' -> 'USE_OPENSSL' - Reported-by: emanruse on github - Fixes #6273 - -Daniel Stenberg (2 Dec 2020) -- quiche: close the connection + - Added a 'link_EXE' macro. Etc, etc. - Reported-by: Junho Choi - Fixes #6213 - Closes #6217 - -Jay Satiro (2 Dec 2020) -- ngtcp2: Fix build error due to symbol name change + - Linking 'curl.exe' needs '$(CURLX_CFILES)' too. - - NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION + - Do not pick-up '../lib/djgpp/*.o' files. Recompile locally. - ngtcp2/ngtcp2@76232e9 changed the name. + - Generate a gzipped 'tool_hugehelp.c' if 'USE_ZLIB=1'. - ngtcp2 master is required to build curl with http3 support. + - Remove 'djgpp-clean' - Closes https://github.com/curl/curl/pull/6271 + - Adapt to new C-ares directory structure + + - Use conditional variable assignments + + Clarify the 'conditional variable assignment' in 'common.dj'. + + Closes https://github.com/curl/curl/pull/6382 -Daniel Stenberg (1 Dec 2020) -- [Klaus Crusius brought this change] +Daniel Stenberg (27 Jan 2021) +- [Ikko Ashimine brought this change] - cmake: check for linux/tcp.h + hyper: fix typo in c-hyper.c - The HAVE_LINUX_TCP_H define was not set by cmake. + settting -> setting - Closes #6252 + Closes #6538 -- NEW-PROTOCOL: document what needs to be done to add one +- libssh2: fix CURL_LIBSSH2_DEBUG-enabled build - Closes #6263 + Follow-up to 2dcc940959772a + + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/2dcc940959772a652f6813fb6bd3092095a4877b#commitcomment-46420088 -- splay: rename Curl_splayremovebyaddr to Curl_splayremove +Jay Satiro (27 Jan 2021) +- asyn-thread: fix build for when getaddrinfo missing - ... and remove the old unused proto for the old Curl_splayremove - version. + This is a follow-up to 8315343 which several days ago moved the resolver + pointer into the async struct but did not update the code that uses it + when getaddrinfo is not present. - Closes #6269 + Closes https://github.com/curl/curl/pull/6536 -- openssl: free mem_buf in error path +Daniel Stenberg (27 Jan 2021) +- urldata: move 'ints' to the end of 'connectdata' - To fix a memory-leak. + To optimize storage slightly. - Closes #6267 + Closes #6534 -- openssl: remove #if 0 leftover +- urldata: store ip version in a single byte - Follow-up to 4c9768565ec3a9 (from Sep 2008) + Closes #6534 + +- urldata: remove duplicate 'upkeep_interval_ms' from connectdata - Closes #6268 + ... and rely only on the value already set in Curl_easy. + + Closes #6534 -- ntlm: avoid malloc(0) on zero length user and domain +- urldata: remove 'local_ip' from the connectdata struct - ... and simplify the too-long checks somewhat. + As the info is already stored in the transfer handle anyway, there's no + need to carry around a duplicate buffer for the life-time of the handle. - Detected by OSS-Fuzz + Closes #6534 + +- urldata: remove duplicate port number storage - Closes #6264 + ... and use 'int' for ports. We don't use 'unsigned short' since -1 is + still often used internally to signify "unknown value" and 0 - 65535 are + all valid port numbers. + + Closes #6534 -- RELEASE-NOTES: synced +- urldata: remove the duplicate 'ip_addr_str' field + + ... as the numerical IP address is already stored and kept in 'primary_ip'. + + Closes #6534 -Marc Hoersken (28 Nov 2020) -- tests/server/tftpd.c: close upload file in case of abort +- select: convert Curl_select() to private static function - Commit c353207 removed the closing right after do_tftp - which covered the case of abort. This handles that case. + The old function should not be used anywhere anymore (the only remaining + gskit use has to be fixed to instead use Curl_poll or none at all). - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + The static function version is now called our_select() and is only built + if necessary. - Follow up to #6209 - Closes #6234 + Closes #6531 -Daniel Stenberg (26 Nov 2020) -- [Daiki Ueno brought this change] - - ngtcp2: use the minimal version of QUIC supported by ngtcp2 +- Curl_chunker: shrink the struct - Closes #6250 + ... by removing a field, converting the hex index into a byte and + rearranging the order. Cuts it down from 48 bytes to 32 on x86_64. + + Closes #6527 -- [Daiki Ueno brought this change] +- curl: include the file name in --xattr/--remote-time error msgs - ngtcp2: advertise h3 ALPN unconditionally +- curl: s/config->global/global/ in single_transfer() + +- curl: move fprintf outputs to warnf - Closes #6250 + For setting and getting time of the download. To make the outputs + respect --silent etc. + + Reported-by: Viktor Szakats + Fixes #6533 + Closes #6535 -- [Daiki Ueno brought this change] +- [Tatsuhiro Tsujikawa brought this change] - vquic/ngtcp2.h: define local_addr as sockaddr_storage + ngtcp2: Fix http3 upload stall - This field needs to be wide enough to hold sockaddr_in6 when - connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the - packets because of the address mismatch: - I00000022 [...] con ignore packet from unknown path + Closes #6521 + +- [Tatsuhiro Tsujikawa brought this change] + + ngtcp2: Fix stack buffer overflow - We can safely assume that struct sockaddr_storage is available, as it - is used in the public interface of ngtcp2. + Closes #6521 + +- warnless.h: remove the prototype for curlx_ultosi - Closes #6250 + Follow-up to 217552503ff3 -- socks: check for DNS entries with the right port number +- warnless: remove curlx_ultosi - The resolve call is done with the right port number, but the subsequent - check used the wrong one, which then could find a previous resolve which - would return and leave the fresh resolve "incomplete" and leaking - memory. + ... not used anywhere - Fixes #6247 - Closes #6253 + Closes #6530 -- curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use +- [Patrick Monnerat brought this change] + + lib: remove conn->data uses - ... so don't define it when instructed to use c-ares! + Closes #6515 -- test506: make it not run in c-ares builds +- pingpong: remove the 'conn' struct member - As the asynch nature of it may trigger events in another order. A c-ares - upgrade made it break. + ... as it's superfluous now when Curl_easy is passed in and we can + derive the connection from that instead and avoid the duplicate copy. - Reported-by: Marc Hörsken - Fixes #6247 + Closes #6525 -- runtests: make 'c-ares' a "feature" to depend on +- hostip/proxy: remove conn->data use - ... also added to the docs. + Closes #6513 -- tool_writeout: use off_t getinfo-types instead of doubles +- url: reduce conn->data references - Commit 3b80d3ca46b12e52342 (June 2017) introduced getinfo replacement - variables that use curl_off_t instead of doubles. Switch the --write-out - function over to use them. + ... there are a few left but let's keep them to last - Closes #6248 + Closes #6512 -- [Emil Engler brought this change] +- scripts/singleuse: add curl_easy_option* - file: avoid duplicated code sequence +Jay Satiro (25 Jan 2021) +- test410: fix for windows - file_disconnect() is identical with file_do() except the function header - but as the arguments are unused anyway so why not just return file_do() - directly! + - Pass the very long request header via file instead of command line. - Reviewed-by: Daniel Stenberg - Closes #6249 + Prior to this change the 49k very long request header string was passed + via command line and on Windows that is too long so it was truncated and + the test would fail (specifically msys CI). + + Closes https://github.com/curl/curl/pull/6516 -- [Rikard Falkeborn brought this change] +Daniel Stenberg (25 Jan 2021) +- libssh2: move data from connection object to transfer object + + Readdir data, filenames and attributes are strictly related to the + transfer and not the connection. This also reduces the total size of the + fixed connectdata struct. + + Closes #6519 - infof/failf calls: fix format specifiers +- RELEASE-NOTES: synced + +- [Patrick Monnerat brought this change] + + lib: remove conn->data uses - Update a few format specifiers to match what is being printed. + Closes #6499 + +- hyper: remove the conn->data references - Closes #6241 + Closes #6508 -- docs/INTERNALS: remove reference to Curl_sendf() +- travis: build ngtcp2 --with-gnutls - The function has been removed from common usage. Also removed comment in - gopher.c that still referenced it. + ... since they disable it by default since a few days back. - Reported-by: Rikard Falkeborn - Fixes #6242 - Closes #6243 + Closes #6506 + Fixes #6493 -- [Rikard Falkeborn brought this change] +- hostip: remove conn->data from resolver functions + + This also moves the 'async' struct from the connectdata struct into the + Curl_easy struct, which seems like a better home for it. + + Closes #6497 - examples: update .gitignore +Jay Satiro (22 Jan 2021) +- strerror: skip errnum >= 0 assertion on windows - Add files that are generated by 'make examples' and remove some that - have been renamed. + On Windows an error number may be greater than INT_MAX and negative once + cast to int. - The commits that renamed the programs are e9625c5bc6c046a (imap.c and - simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and - ad39e7ec01e7 (pop3slist.c and pop3s.c were renamed to pop3-list.c and - pop3-ssl.c). + The assertion is checked only in debug builds. - Closes #6240 + Closes https://github.com/curl/curl/pull/6504 -- asyn: use 'struct thread_data *' instead of 'void *' +Daniel Stenberg (21 Jan 2021) +- doh: make Curl_doh_is_resolved survive a NULL pointer - To reduce use of types that can't be checked at compile time. Also - removes several typecasts. - - ... and rename the struct field from 'os_specific' to 'tdata'. + ... if Curl_doh() returned a NULL, this function gets called anyway as + in a asynch procedure. Then the doh struct pointer is NULL and signifies + an OOM situation. - Closes #6239 - Reviewed-by: Jay Satiro + Follow-up to 6246a1d8c6776 -Viktor Szakats (23 Nov 2020) -- Makefile.m32: add support for UNICODE builds +- wolfssh: remove conn->data references - It requires the linker to support the `-municode` option. - This is available in more recent mingw-w64 releases. + ... and repair recent build breakage - Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html - Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847 + Closes #6507 + +- http: empty reply connection are not left intact - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad + ... so mark the connection as closed in this condition to prevent that + verbose message to wrongly appear. - Closes #6228 + Reported-by: Matt Holt + Bug: https://twitter.com/mholt6/status/1352130240265375744 + Closes #6503 -Daniel Stenberg (23 Nov 2020) -- urldata: remove 'void *protop' and create the union 'p' +- chunk/encoding: remove conn->data references - ... to avoid the use of 'void *' for the protocol specific structs done - per transfer. + ... by anchoring more functions on Curl_easy instead of connectdata - Closes #6238 + Closes #6498 -- winbuild: remove docs from Makefiles and refer to README.md +Jay Satiro (20 Jan 2021) +- [Erik Olsson brought this change] + + lib: save a bit of space with some structure packing - Reduce risk for conflicting docs and makes it to a single place to fix - and polish. + - Reorder some internal struct members so that less padding is used. - add these missing options to the readme: + This is an attempt at saving a bit of space by packing some structs + (using pahole to find the holes) where it might make sense to do + so without losing readability. - ENABLE_OPENSSL_AUTO_LOAD_CONFIG and ENABLE_UNICODE + I.e., I tried to avoid separating fields that seem grouped + together (like the cwd... fields in struct ftp_conn for instance). + Also abstained from touching fields behind conditional macros as + that quickly can get complicated. - clarify ENABLE_SCHANNEL default varies + Closes https://github.com/curl/curl/pull/6483 + +Daniel Stenberg (20 Jan 2021) +- INSTALL.md: fix typo - Fixes #6216 - Closes #6227 - Co-Authored-by: Jay Satiro + Found-by: Marcel Raad -- [Daiki Ueno brought this change] +- [Fabian Keil brought this change] - http3: use the master branch of GnuTLS for testing + http: get CURLOPT_REQUEST_TARGET working with a HTTP proxy - Closes #6235 - -- KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation + Added test 1613 to verify. - Closes #5839 + Closes #6490 -- KNOWN_BUGS: wakeup socket disconnect causes havoc +- Merge branch 'bagder/curl_range-data-conn' + +- ftp: remove conn->data leftover + +- curl_range: remove conn->data - Closes #6132 - Closes #6133 + Closes #6496 -- RELEASE-NOTES: synced +- INSTALL: now at 85 operating systems -- [Oliver Urbann brought this change] +- quiche: fix unused parameter ‘conn’ + + Follow-up to 2bdec0b3 - curl: add compatibility for Amiga and GCC 6.5 +- transfer: fix ‘conn’ undeclared mistake for iconv build - Changes are mainly reordering and adding of includes required - to compile with a more recent version of GCC. + Follow-up to 219d9f8620d + +- doh: allocate state struct on demand - Closes #6220 + ... instead of having it static within the Curl_easy struct. This takes + away 1176 bytes (18%) from the Curl_easy struct that aren't used very + often and instead makes the code allocate it when needed. + + Closes #6492 -Marc Hoersken (20 Nov 2020) -- tests/server/tftpd.c: close upload file right after transfer +- socks: use the download buffer instead - Make sure uploaded file is no longer locked after the - transfer while waiting for the final ACK to be handled. + The SOCKS code now uses the generic download buffer for temporary + storage during the connection procedure, instead of having its own + private 600 byte buffer that adds to the connectdata struct size. This + works fine because this point the buffer is allocated but is not use for + download yet since the connection hasn't completed. - Assisted-by: Daniel Stenberg + This reduces the connection struct size by 22% on a 64bit arch! - Bug: #6058 - Closes #6209 - -- CI/cirrus: simplify logic for disabled tests + The SOCKS buffer needs to be at least 600 bytes, and the download buffer + is guaranteed to never be smaller than 1000 bytes. - The OpenSSH server instance for the testsuite cannot - be started on FreeBSD, therefore the SFTP and SCP - tests are disabled right away from the beginning. + Closes #6491 + +- urldata: make magic be the first struct field - The previous OS version specific logic for SKIP_TESTS - is no longer needed/used and can therefore be removed. + By making the `magic` identifier the same size and at the same place + within the structs (easy, multi, share), libcurl will be able to more + reliably detect and safely error out if an application passes in the + wrong handle to APIs. Easier to detect and less likely to cause crashes + if done. - Reviewed-by: Daniel Stenberg + Such mixups can't be detected at compile-time due to them being + typedefed void pointers - unless `CURL_STRICTER` is defined. - Follow up to #6211 - Closes #6229 + Closes #6484 -Daniel Gustafsson (20 Nov 2020) -- mailmap: Daniel Hwang +- http_chunks: correct and clarify a comment on hexnumber length - Add Daniel Hwang to the mailmap to cover the alternative spelling - Daniel Lee Hwang which was used in one commit. + ... and also rename the define for max length. - Closes #6230 - Reviewed-by: Daniel Stenberg + Closes #6489 -- openssl: guard against OOM on context creation +- curl_path: remove conn->data use - EVP_MD_CTX_create will allocate memory for the context and returns - NULL in case the allocation fails. Make sure to catch any allocation - failures and exit early if so. + Closes #6487 + +- transfer: remove conn->data use - In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex - as the latter is intended for ENGINE selection which we don't do. + Closes #6486 + +- quic: remove conn->data use - Closes #6224 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler + Closes #6485 -Daniel Stenberg (19 Nov 2020) -- [Vincent Torri brought this change] +- [Fabian Keil brought this change] - cmake: use libcurl.rc in all Windows builds + Add test1181: Proxy request with --proxy-header "Connection: Keep-Alive" + +- [Fabian Keil brought this change] + + Add test1180: Proxy request with -H "Proxy-Connection: Keep-Alive" - Reviewed-by: Marcel Raad - Closes #6215 + At the moment the test fails as curl sends two Proxy-Connection + headers. -- [Cristian Morales Vega brought this change] +- c-hyper: avoid duplicated Proxy-Connection headers - cmake: make CURL_ZLIB a tri-state variable +- http: make providing Proxy-Connection header not cause duplicated headers - By differentiating between ON and AUTO it can make a missing zlib - library a hard error when CURL_ZLIB=ON is used. + Fixes test 1180 - Reviewed-by: Jakub Zakrzewski - Closes #6221 - Fixes #6173 + Bug: https://curl.se/mail/lib-2021-01/0095.html + Reported-by: Fabian Keil + Closes #6472 -- quiche: remove 'static' from local buffer +- runtests: preprocess DISABLED to allow conditionals - For thread-safety + ... with this function provided, we can disable tests for specific + environments and setups directly within this file. - Closes #6223 + Closes #6477 -- KNOWN_BUGS: cmake: libspsl is not supported +- runtests: turn preprocessing into a separate function - Closes #6214 + ... and remove all other variable substitutions as they're now done once + and for all in the preprocessor. -- KNOWN_BUGS: cmake autodetects cert paths when cross-compiling +- lib/Makefile.inc: convert to listing each file on its own line - Closes #6178 - -- KNOWN_BUGS: cmake build doesn't fail if zlib not found + ... to make it diff friendlier and easier to read. - Closes #6173 + Closes #6448 -- KNOWN_BUGS: cmake libcurl.pc uses absolute library paths +- ftplistparser: remove use of conn->data - Closes #6169 + Closes #6482 -- KNOWN_BUGS: cmake: generated .pc file contains strange entries +- lib: more conn->data cleanups - Closes #6167 + Closes #6479 -- KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads +- [Patrick Monnerat brought this change] + + vtls: reduce conn->data use - Closes #6166 + Closes #6474 -- KNOWN_BUGS: cmake build in Linux links libcurl to libdl +- hyper: deliver data to application with Curl_client_write - Closes #6165 + ... just as the native code path does. Avoids sending too large data + chunks in the callback and more. + + Reported-by: Gisle Vanem + Fixes #6462 + Closes #6473 -- KNOWN_BUGS: make a new section for cmake topics +- gopher: remove accidental conn->data leftover + +- libssh: avoid plain free() of libssh-memory - Closes #6219 + Since curl's own memory debugging system redefines free() calls to track + and fiddle with memory, it cannot be used on memory allocated by 3rd + party libraries. + + Third party libraries SHOULD NOT require free() to release allocated + resources for this reason - and libs can use separate healp allocators + on some systems (like Windows) so free() doesn't necessarily work + anyway. + + Filed as an issue with libssh: https://bugs.libssh.org/T268 + + Closes #6481 -- [Emil Engler brought this change] +- send: assert that Curl_write_plain() has a ->conn when called + + To help catch bad invokes. + + Closes #6476 - cirrus: build with FreeBSD 12.2 in CirrusCI +- test410: verify HTTPS GET with a 49K request header - Closes #6211 + skip test 410 for mesalink in the CI as it otherwise hangs "forever" -Marc Hoersken (14 Nov 2020) -- tests/*server.py: close log file after each log line +- lib: pass in 'struct Curl_easy *' to most functions - Make sure the log file is not locked once a test has - finished and align with the behavior of our logmsg. + ... in most cases instead of 'struct connectdata *' but in some cases in + addition to. - Rename curl_test_data.py to be a general util.py. - Format and sort Python imports with isort/VSCode. + - We mostly operate on transfers and not connections. - Bug: #6058 - Closes #6206 + - We need the transfer handle to log, store data and more. Everything in + libcurl is driven by a transfer (the CURL * in the public API). + + - This work clarifies and separates the transfers from the connections + better. + + - We should avoid "conn->data". Since individual connections can be used + by many transfers when multiplexing, making sure that conn->data + points to the current and correct transfer at all times is difficult + and has been notoriously error-prone over the years. The goal is to + ultimately remove the conn->data pointer for this reason. + + Closes #6425 -Daniel Stenberg (13 Nov 2020) -- CURLOPT_HSTS.3: document the file format +Emil Engler (17 Jan 2021) +- docs: fix typos in NEW-PROTOCOL.md - Closes #6205 + This fixes a misspelled "it" and a grammatically wrong "-ing" suffix. + + Closes #6471 +Daniel Stenberg (16 Jan 2021) - RELEASE-NOTES: synced -- release-notes.pl: detect #[number] better for Ref: etc +Jay Satiro (16 Jan 2021) +- [Razvan Cojocaru brought this change] -- curl: only warn not fail, if not finding the home dir + cmake: expose CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG - ... as there's no good reason to error out completely. + This does for cmake builds what --disable-openssl-auto-load-config + does for autoconf builds. - Reported-by: Andreas Fischer - Fixes #6200 - Closes #6201 + Closes https://github.com/curl/curl/pull/6435 -- httpput-postfields.c: new example doing PUT with POSTFIELDS +Daniel Stenberg (15 Jan 2021) +- test1918: verify curl_easy_option_by_name() and curl_easy_option_by_id() - Proposed-by: Jeroen Ooms - Ref: #6186 - Closes #6188 + ... and as a practical side-effect, make sure that the + Curl_easyopts_check() function is asserted in debug builds, which we + want to detect mismatches between the options list in easyoptions.c and + the options in curl.h + + Found-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45991815 + + Closes #6461 -- [Tobias Hieta brought this change] +- [Gisle Vanem brought this change] - cmake: correctly handle linker flags for static libs + easyoptions: add the missing AWS_SIGV4 - curl CMake was setting the the EXE flags for static libraries which made - the /manifest:no flag ended up when linking the static library, which is - not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit - with an error. + Follow-up from AWS_SIGV4 + +- schannel_verify: fix safefree call typo - The better way to handle this is to make sure that we pass the correct - linker flags to CMAKE_STATIC_LINKER_FLAGS instead. + Follow-up from e87ad71d1ba00519 - Reviewed-by: Jakub Zakrzewski - Closes #6195 + Closes #6459 -- [Tobias Hieta brought this change] +- mime: make sure setting MIMEPOST to NULL resets properly + + ... so that a function can first use MIMEPOST and then set it to NULL to + reset it back to a blank POST. + + Added test 584 to verify the fix. + + Reported-by: Christoph M. Becker + + Fixes #6455 + Closes #6456 - cmake: don't pass -fvisibility=hidden to clang-cl on Windows +- multi: set the PRETRANSFER time-stamp when we switch to PERFORM - When using clang-cl on windows -fvisibility=hidden is not an known - argument. Instead it behaves exactly like MSVC in this case. So let's - make sure we take that path. + ... instead of at end of the DO state. This makes the timer more + accurate for the protocols that use the DOING state (such as FTP), and + simplifies how the function (now called init_perform) is called. - In CMake clang-cl sets both CMAKE_C_COMPILER_ID=clang and MSVC get's - defined since clang-cl is basically a MSVC emulator. So guarding like we - do in this patch seems logical. + The timer will then include the entire procedure up to PERFORM - + including all instructions for getting the transfer started. - Reviewed-by: Jakub Zakrzewski - Closes #6194 + Closes #6454 -- http_proxy: use enum with state names for 'keepon' +- CURLINFO_PRETRANSFER_TIME.3: clarify - To make the code clearer, change the 'keepon' from an int to an enum - with better state names. + ... the timer *does* include the instructions for getting the remote + file. - Reported-by: Niranjan Hasabnis - Bug: https://curl.se/mail/lib-2020-11/0026.html - Closes #6193 + Ref: #6452 + Closes #6453 -- curl_easy_escape: limit output string length to 3 * max input +- [Gisle Vanem brought this change] + + schannel: plug a memory-leak - ... instead of the limiting it to just the max input size. As every - input byte can be expanded to 3 output bytes, this could limit the input - string to 2.66 MB instead of the intended 8 MB. + ... when built without -DUNICODE. - Reported-by: Marc Schlatter - Closes #6192 + Closes #6457 -- docs: document the 8MB input string limit +Jay Satiro (14 Jan 2021) +- gitattributes: Set batch files to CRLF line endings on checkout - for curl_easy_escape and curl_easy_setopt() + If a batch file is run without CRLF line endings (ie LF-only) then + arbitrary behavior may occur. I consider that a bug in Windows, however + the effects can be serious enough (eg unintended code executed) that + we're fixing it in the repo by requiring CRLF line endings for batch + files on checkout. - The limit is there to catch mistakes and abuse. It is meant to be large - enough to allow virtually all "fine" use cases. + Prior to this change the checked-out line endings of batch files were + dependent on a user's git preferences. On Windows it is common for git + users to have automatic CRLF conversion enabled (core.autocrlf true), + but those users that don't would run into this behavior. - Reported-by: Marc Schlatter - Fixes #6190 - Closes #6191 + For example a user has reported running the Visual Studio project + generator batch file (projects/generate.bat) and it looped forever. + Output showed that the Windows OS interpreter was occasionally jumping + to arbitrary points in the batch file and executing commands. This + resulted in unintended files being removed (a removal sequence called) + and looping forever. + + Ref: https://serverfault.com/q/429594 + Ref: https://stackoverflow.com/q/232651 + Ref: https://www.dostips.com/forum/viewtopic.php?t=8988 + Ref: https://git-scm.com/docs/gitattributes#_checking_out_and_checking_in + Ref: https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration#_core_autocrlf + + Bug: https://github.com/curl/curl/discussions/6427 + Reported-by: Ganesh Kamath + + Closes https://github.com/curl/curl/pull/6442 -- mqttd: fclose test file when done +Daniel Stenberg (14 Jan 2021) +- tool_operate: spellfix a comment + +- ROADMAP: refreshed - Reported-by: Marc Hörsken - Reviewed-by: Jay Satiro - Bug: #6058 - Closes #6189 + o removed HSTS - already implemented + o added HTTPS RR records + o mention HTTP/3 completion -- RELEASE-NOTES: synced +- http_chunks: remove Curl_ prefix from static functions -- THANKS-filter: ignore autobuild links +- transfer: remove Curl_ prefix from static functions -- Revert "libcurl.pc: make it relocatable" +- tftp: remove Curl_ prefix from static functions + +- multi: remove Curl_ prefix from static functions + +- ldap: remove Curl_ prefix from static functions + +- doh: remove Curl_ prefix from static functions + +- asyn-ares: remove Curl_ prefix from static functions + +- vtls: remove Curl_ prefix from static functions + +- bearssl: remove Curl_ prefix from static functions + +- mbedtls: remove Curl_ prefix from static functions + +- wolfssl: remove Curl_ prefix from static functions + +- nss: remove Curl_ prefix from static functions + +- gnutls: remove Curl_ prefix from static functions + +- openssl: remove Curl_ prefix from static functions - This reverts commit 3862c37b6373a55ca704171d45ba5ee91dec2c9f. + ... as we reserve this prefix to library-wide functions. - That fix should either be done differently or with an option. + Closes #6443 + +- nss: get the run-time version instead of build-time - Reported-by: asavah on github - Fixes #6157 - Closes #6183 + Closes #6445 -- examples/httpput: remove use of CURLOPT_PUT +Jay Satiro (12 Jan 2021) +- tool_doswin: Restore original console settings on CTRL signal - It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD. + - Move Windows terminal init code from tool_main to tool_doswin. - Reported-by: Jeroen Ooms - Fixes #6186 - Closes #6187 - -- Curl_pgrsStartNow: init speed limit time stamps at start + - Restore the original console settings on CTRL+C and CTRL+BREAK. - By setting the speed limit time stamps unconditionally at transfer - start, we can start off a transfer without speed limits and yet allow - them to get set during transfer and have an effect. + Background: On Windows the curl tool changes the console settings to + enable virtual terminal processing (eg color output) if supported + (ie Win 10). The original settings are restored on exit but prior to + this change were not restored in the case of the CTRL signals. - Reported-by: Kael1117 on github - Fixes #6162 - Closes #6184 - -- ngtcp2: adapt to recent nghttp3 updates + Windows VT behavior varies depending on console/powershell/terminal; + refer to the discussion in #6226. - 'reset_stream' was added to the nghttp3_conn_callbacks struct + Assisted-by: Rich Turner - Closes #6185 + Closes https://github.com/curl/curl/pull/6226 -- configure: pass -pthread to Libs.private for pkg-config +Daniel Stenberg (12 Jan 2021) +- gen.pl: fix perl syntax - Reported-by: Cristian Morales Vega - Fixes #6168 - Closes #6181 + Follow-up to 324cf1d2e -- altsvc: minimize variable scope and avoid "DEAD_STORE" - - Closes #6182 +- [Emil Engler brought this change] -- FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?" + help: update to current codebase - This hasn't been the case for a while now, remove. - -- FAQ: refresh "Why do I get "certificate verify failed" + This commit bumps the help to the current state of the project. - Add more details, remove references to ancient curl version. + Closes #6437 -- test493: verify --hsts upgrade and that %{url_effective} reflects that +- [Emil Engler brought this change] + + docs: fix line length bug in gen.pl - Closes #6175 + The script warns if the length of $opt and $desc is > 78. However, these + two variables are on totally separate lines so the check makes no sense. + Also the $bitmask field is totally forgotten. Currently this leads to + two warnings within `--resolve` and `--aws-sigv4`. + + Closes #6438 -- url: make sure an HSTS upgrade updates URL and scheme correctly +- [Emil Engler brought this change] + + docs: fix wrong documentation in help.d - Closes #6175 + curl does not list all categories when you invoke "--help" without any + parameters. + + Closes #6436 -- tool_operate: set HSTS with CURLOPT_HSTS to pass on filename +- aws-sigv4.d: polish the wording - Closes #6175 + Make it shorter and imperative form + + Closes #6439 -- hsts: remove debug code leftovers +- [Fabian Keil brought this change] + + misc: fix typos - Closes #6175 + Bug: https://curl.se/mail/lib-2021-01/0063.html + Closes #6434 -- FAQ: refreshed +- multi_runsingle: bail out early on data->conn == NULL - - remove a few ancient questions - - add configure with static libs question - - updated wording in several places - - lowercased curl + As that's a significant error condition and scan-build warns for NULL + pointer dereferences if we don't. - Closes #6177 + Closes #6433 -Daniel Gustafsson (5 Nov 2020) -- examples: fix comment syntax +- multi: skip DONE state if there's no connection left for ftp wildcard - Commit ac0a88fd2 accidentally added a stray character outside of the - comment which broke compilation. Fix by removing. + ... to avoid running in that state with data->conn being NULL. + +- libssh2: fix "Value stored to 'readdir_len' is never read" - Reported-by: autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742 + Detected by scan-build -- hsts: Remove pointless call to free in errorpath +- connect: mark intentional ignores of setsockopt return values - The line variable will always be NULL in the error path, so remove - the free call since it's pointless. + Pointed out by Coverity - Closes #6170 - Reviewed-by: Daniel Stenberg + Closes #6431 -- docs: Fix various typos in documentation +Jay Satiro (11 Jan 2021) +- http_proxy: Fix CONNECT chunked encoding race condition - Closes #6171 - Reviewed-by: Daniel Stenberg - -Daniel Stenberg (5 Nov 2020) -- copyright: fix year ranges + - During the end-of-headers response phase do not mark the tunnel + complete unless the response body was completely parsed/ignored. - Follow-up from 4d2f8006777 + Prior to this change if the entirety of a CONNECT response with chunked + encoding was not received by the time the final header was parsed then + the connection would be marked done prematurely, before all the chunked + data could be read in and ignored (since this is what we do with any + CONNECT response body) and the connection could not be used. + + Bug: https://curl.se/mail/lib-2021-01/0033.html + Reported-by: Fabian Keil + + Closes https://github.com/curl/curl/pull/6432 -- HISTORY: the new domain +Daniel Stenberg (11 Jan 2021) +- RELEASE-NOTES: synced -- curl.se: new home +- url: if IDNA conversion fails, fallback to Transitional - Closes #6172 - -- KNOWN_BUGS: FTPS with Schannel times out file list operation + This improves IDNA2003 compatiblity. - Reported-by: bobmitchell1956 on github - Closes #5284 + Reported-by: Bubu on github + Fixes #6423 + Closes #6428 -- KNOWN_BUGS: SMB tests fail with Python 2 +- travis: make the Hyper build from its master branch - Reported-by: Jay Satiro - Closes #5983 + Closes #6430 -- KNOWN_BUGS: LDAPS with NSS is slow +- http: make 'authneg' also work for Hyper - Reported-by: nosajsnikta on github - Closes #5874 + When doing a request with a request body expecting a 401/407 back, that + initial request is sent with a zero content-length. Test 177 and more. + + Closes #6424 -Sergei Nikulov (4 Nov 2020) -- travis: use ninja-build for CMake builds +Jay Satiro (8 Jan 2021) +- cmake: Add an option to disable libidn2 - Added package ninja-build to environment - Use ninja to speed up CMake builds + New option USE_LIBIDN2 defaults to ON for libidn2 detection. Prior to + this change libidn2 detection could not be turned off in cmake builds. - Closes #6077 - -Daniel Stenberg (4 Nov 2020) -- [Harry Sintonen brought this change] - - rtsp: error out on empty Session ID, unified the code - -- [Harry Sintonen brought this change] - - rtsp: fixed the RTST Session ID mismatch in test 570 + Reported-by: William A Rowe Jr - Closes #6161 + Fixes https://github.com/curl/curl/issues/6361 + Closes https://github.com/curl/curl/pull/6362 -- [Harry Sintonen brought this change] +Daniel Stenberg (8 Jan 2021) +- HYPER: no longer needs the special branch - rtsp: fixed Session ID comparison to refuse prefix +- test179: use consistent header line endings - Closes #6161 + ... to make "Hyper mode" work better. -- RELEASE-NOTES: synced +- file: don't provide content-length for directories - (forgot to update the list of contributors) + ... as it is misleading. + + Ref #6379 + Closes #6421 -- RELEASE-NOTES: synced +- TODO: Directory listing for FILE: + + Ref #6379 -- curlver: bumped to 7.74.0 +- curl.h: add CURLPROTO_GOPHERS as own protocol identifier + + Follow-up to a1f06f32b860, to make sure it can be handled separately + from plain gopher. + + Closes #6418 -- hsts: add read/write callbacks +- http: have CURLOPT_FAILONERROR fail after all headers - - read/write callback options - - man pages for the 4 new setopts - - test 1915 verifies the callbacks + ... so that Retry-After and other meta-content can still be used. - Closes #5896 + Added 1634 to verify. Adjusted test 194 and 281 since --fail now also + includes the header-terminating CRLF in the output before it exits. + + Fixes #6408 + Closes #6409 -- hsts: add support for Strict-Transport-Security +- global_init: debug builds allocates a byte in init - - enable in the build (configure) - - header parsing - - host name lookup - - unit tests for the above - - CI build - - CURL_VERSION_HSTS bit - - curl_version_info support - - curl -V output - - curl-config --features - - CURLOPT_HSTS_CTRL - - man page for CURLOPT_HSTS_CTRL - - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) - - man page for --hsts - - save cache to disk - - load cache from disk - - CURLOPT_HSTS - - man page for CURLOPT_HSTS - - added docs/HSTS.md - - fixed --version docs - - adjusted curl_easy_duphandle + ... to make build tools/valgrind warn if no curl_global_cleanup is + called. - Closes #5896 + This is conditionally only done for debug builds with the env variable + CURL_GLOBAL_INIT set. + + Closes #6410 -- [Sergei Nikulov brought this change] +- lib/unit tests: add missing curl_global_cleanup() calls - CI/tests: enable test target on TravisCI for CMake builds +- travis: adapt to Hyper build change - Added test-nonflaky target to CMake builds + Closes #6419 + +- pretransfer: setup the User-Agent header here - Disabled test 1139 because the cmake build doesn't create docs/curl.1 + ... and not in the connection setup, as for multiplexed transfers the + connection setup might be skipped and then the transfer would end up + without the set user-agent! - Closes #6074 + Reported-by: Flameborn on github + Assisted-by: Andrey Gursky + Assisted-by: Jay Satiro + Assisted-by: Mike Gelfand + Fixes #6312 + Closes #6417 -- tool_debug_cb: do not assume zero-terminated data +- test66: disable with Hyper - Follow-up to d70a5b5a0f5e3 + ...as Hyper doesn't support HTTP/0.9 -- sendf: move the verbose-check into Curl_debug +- c-hyper: poll the tasks until end correctly - Saves us from having the same check done everywhere. + ... makes test 36 work. - Closes #6159 + Closes #6412 -- travis: use valgrind when running tests for debug builds +- [Gergely Nagy brought this change] + + mk-ca-bundle.pl: deterministic output when using -t - Except the non-x86 and sanitizer builds + Printing trust purposes are now sorted, making the output deterministic + when running on the same input certdata.txt. - Closes #6154 + Closes #6413 -- header.d: fix syntax mistake +- KNOWN_BUGS: fixed "wolfSSL lacks support for renegotiation" - follow-up from 1144886f38fd0 + Fixed by #6411 -- [Harry Sintonen brought this change] +- [Himanshu Gupta brought this change] - gnutls: fix memory leaks (certfields memory wasn't released) + wolfssl: add SECURE_RENEGOTIATION support - Closes #6153 + Closes #6411 -- tests: add missing global_init/cleanup calls - - Without the cleanup call in these test files, the mbedTLS backend leaks - memory. - - Closes #6156 +- RELEASE-NOTES: synced -- tool_operate: --retry for HTTP 408 responses too +- wolfssl: update copyright year range - This was inadvertently dropped from the code when the parallel support - was added. + Follow-up to 7de2e96535e9 + +- c-hyper: make CURLE_GOT_NOTHING work - Regression since b88940850 (7.66.0) + Test 30 - Reviewed-by: Jay Satiro - Closes #6155 + Closes #6407 -- http: pass correct header size to debug callback for chunked post +- http_proxy: make CONNECT work with the Hyper backend - ... when the chunked framing was added, the size of the "body part" of - the data was calculated wrongly so the debug callback would get told a - header chunk a few bytes too big that would also contain the first few - bytes of the request body. + Makes test 80 run - Reported-by: Dirk Wetter - Ref: #6144 - Closes #6147 + Closes #6406 -- header.d: mention the "Transfer-Encoding: chunked" handling - - Ref: #6144 - Closes #6148 +- TODO: --fail-with-body perchance? -- acinclude: detect manually set minimum macos/ipod version +Jay Satiro (4 Jan 2021) +- tool_operate: fix the suppression logic of some error messages - ... even if set in the CC or IPHONEOS/MACOSX_DEPLOYMENT_TARGET - variables. + - Fix the failed truncation and failed writing body error messages to + not be shown unless error messages are shown. (ie the user has + specified -sS, or has not specified -s). - Reported-by: hamstergene on github - Fixes #6138 - Closes #6140 - -Jay Satiro (29 Oct 2020) -- tests: fix some http/2 tests for older versions of nghttpx + - Also prefix same error messages with "curl: ", for example: + curl: (23) Failed to truncate, exiting - - Add regex that strips http/2 server header name to those http/2 tests - that don't already have it. + Prior to this change the failed truncation error messages would be shown + if not -s, but did not account for -sS which should show. - - Improve that regex in all http/2 tests. + Prior to this change the failed writing body error messages would be + shown always. - Tests 358 and 359 were failing for me before this change on a system - that uses an older version of nghttpx which includes its version number - in the server header. + Ref: https://curl.se/docs/manpage.html#-S - Closes https://github.com/curl/curl/pull/6139 - -Daniel Stenberg (30 Oct 2020) -- RELEASE-NOTES: synced - -- [Cristian Morales Vega brought this change] + Bug: https://curl.se/mail/archive-2020-12/0017.html + Reported-by: Hongyi Zhao + + Closes https://github.com/curl/curl/pull/6402 - configure: use pkgconfig to find openSSL when cross-compiling +- wolfssl: Support wolfSSL builds missing TLS 1.1 - This reverts 736a40fec (November 2004), which doesn't explain why it was - done. + The wolfSSL TLS library defines NO_OLD_TLS in some of their build + configurations and that causes the library to be built without TLS 1.1. + For example if MD5 is explicitly disabled when building wolfSSL then + that defines NO_OLD_TLS and the library is built without TLS 1.1 [1]. - Closes #6145 - -- tool_operate: bail out proper on errors for parallel setup + Prior to this change attempting to build curl with a wolfSSL that was + built with NO_OLD_TLS would cause a build link error undefined reference + to wolfTLSv1_client_method. - ... otherwise for example trying to upload a missing file just causes a - loop. + [1]: https://github.com/wolfSSL/wolfssl/blob/v4.5.0-stable/configure.ac#L2366 - Reported-by: BrumBrum on hackerone - Closes #6141 + Bug: https://curl.se/mail/lib-2020-12/0121.html + Reported-by: Julian Montes + + Closes https://github.com/curl/curl/pull/6388 -- [Sergei Nikulov brought this change] +Daniel Stenberg (4 Jan 2021) +- test1633: set appropriate name + + "--retry with a 429 response and Retry-After:" - CMake: make BUILD_TESTING dependent option +- travis: limit the tests with quiche builds to HTTPS and FTPS only - CMake will now handle BUILD_TESTING depending on PERL_FOUND and - CURL_DISABLE_TESTING + ... since it runs into the 50 minute time limit too often otherwise. - Ref: #6036 - Closes #6072 + Closes #6403 -- libssh2: fix transport over HTTPS proxy +- HISTORY: added dates to early history - The fix in #6021 was not enough. This fix makes sure SCP/SFTP content - can also be transfered over a HTTPS proxy. + Mostly thanks to this archived web page for urlget: - Fixes #6113 - Closes #6128 + https://web.archive.org/web/19980216125115/http://www.inf.ufrgs.br/~sagula/urlget.html -- curl.1: add an "OUTPUT" section at the top of the manpage +- httpauth: make multi-request auth work with custom port - Explain the basic concepts behind curl output. + When doing HTTP authentication and a port number set with CURLOPT_PORT, + the code would previously have the URL's port number override as if it + had been a redirect to an absolute URL. - Inspired by #6124 + Added test 1568 to verify. - Closes #6134 + Reported-by: UrsusArctos on github + Fixes #6397 + Closes #6400 -- mailmap: set Viktor Szakats's email +- [Emil Engler brought this change] -- runtests: show keywords when no tests ran + language: s/behaviour/behavior/g - To help out future debugging, runtests now outputs the list of keywords - when it fails because no tests ran. + We currently use both spellings the british "behaviour" and the american + "behavior". However "behavior" is more used in the project so I think + it's worth dropping the british name. - Ref: #6120 - Closes #6126 + Closes #6395 -Jay Satiro (26 Oct 2020) -- CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo - - Reported-by: Rui LIU +- cmdline-opts/retry.d: mention response code 429 as well - Closes https://github.com/curl/curl/issues/6131 + Reported-by: Cherish98 + Bug: https://curl.se/mail/archive-2020-12/0018.html -- range.d: fix typo +- docs/HYPER.md: mention outstanding issues - Follow-up to 15ae039 from earlier today. + To make it more obvious to users what doesn't work (yet) + + Closes #6389 -Daniel Stenberg (26 Oct 2020) -- CI/github: work-around for brew breakage on macOS +- COPYING/configure: bump copyright year range + +- c-hyper: add timecondition to the request - ... and make it use OpenSSL 1.1 properly + Test 77-78 - Fixes #6130 - Closes #6129 - -- [José Joaquín Atria brought this change] + Closes #6391 - range.d: clarify that curl will not parse multipart responses +- c-hyper: make Digest and NTLM work - Closes #6127 - Fixes #6124 + Test 64, 65, 67, 68, 69, 70, 72 + + Closes #6390 -- RELEASE-NOTES: synced +- examples/curlgtk.c: fix the copyright year range + + ... and make private functions static. -- [Baruch Siach brought this change] +- [Olaf Hering brought this change] - libssh2: fix build with disabled proxy support + docs/examples: adjust prototypes for CURLOPT_READFUNCTION - Build breaks because the http_proxy field is missing: + The type of the buffer in curl_read_callback is 'char *', not 'void *'. - vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy' + Signed-off-by: Olaf Hering + Closes #6392 + +- examples: fix more empty expression statement has no effect - Regression from #6021, shipped in curl 7.73.0 + Follow-up to 26e46617b9 + +- cleanup: fix two empty expression statement has no effect - Closes #6125 + Follow-up to 26e46617b9 -- alt-svc: enable by default +- configure: set -Wextra-semi-stmt for clang with --enable-debug - Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported. + To have it properly complain on empty statements with no effect. - alt-svc support in curl is no longer considered experimental + Ref: #6376 + Closes #6378 + +- tests/unit: fix empty statements with no effect - Closes #5868 + ... by making macros use "do {} while(0)" -- CI/appveyor: remove (unused) runtests.pl -b option +- [Paul Groke brought this change] -- [Emil Engler brought this change] - - tool_help: make "output" description less confusing + dns: extend CURLOPT_RESOLVE syntax for adding non-permanent entries - Currently the description of "output" is misleading when comparing it - "verbose". + Extend the syntax of CURLOPT_RESOLVE strings: allow using a '+' prefix + (similar to the existing '-' prefix for removing entries) to add + DNS cache entries that will time out just like entries that are added + by libcurl itself. - Closes #6118 + Append " (non-permanent)" to info log message in case a non-permanent + entry is added. + + Adjust relevant comments to reflect the new behavior. + + Adjust documentation. + + Extend unit1607 to test the new functionality. + + Closes #6294 -- CI/appveyor: disable test 571 in two cmake builds +- schannel: fix "empty expression statement has no effect" - ... they're simply too flaky there. + Bug: https://github.com/curl/curl/commit/8ab78f720ae478d533e30b202baec4b451741579#commitcomment-45445950 + Reported-by: Gisle Vanem + Closes #6381 + +- [Denis Laxalde brought this change] + + docs: remove redundant "better" in --fail help - Closes #6119 + Closes #6385 -- cmake: set the unicode feature in curl-config on Windows +- [Kevin Ushey brought this change] + + curl.1: fix typo microsft -> microsoft - ... if built that way. To make it match curl -V output. + Closes #6380 + +- [XhmikosR brought this change] + + misc: assorted typo fixes - Reviewed-by: Marcel Raad - Closes #6117 + Closes #6375 -- libssh2: require version 1.0 or later +- RELEASE-NOTES: synced + +- tool_operate: avoid NULL dereference of first_arg - ... and simplify the code accordingly. libssh2 version 1.0 was released - in April 2009. + Follow-up to 6a5e020d4d2b04a + Identified by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28999 + Closes #6377 + +- misc: fix "warning: empty expression statement has no effect" - Closes #6116 + Turned several macros into do-while(0) style to allow their use to work + find with semicolon. + + Bug: https://github.com/curl/curl/commit/08e8455dddc5e48e58a12ade3815c01ae3da3b64#commitcomment-45433279 + Follow-up to 08e8455dddc5e4 + Reported-by: Gisle Vanem + Closes #6376 -- KNOWN_BUGS: mention the individual cmake issues +- KNOWN_BUGS: 6.10 curl never completes Negotiate over HTTP - ... to make them easier to refer to and address separately and - one-by-one. + Closes #5235 + Closes #6370 -- CMake: store IDN2 information in curl_config.h +- writeout: fix NULL dereference for "this url" - This allows the build to enable IDN properly and it makes test 1014 - happier. + Detected by torture test 1029 - Ref: #6074 - Closes #6108 + Follow-up to 7a90ddf88f5a + + Closes #6374 -- CMake: call the feature unixsockets without dash +- failf: remove newline from formatting strings - ... so that curl-config gets correct and makes test 1014 happy! + ... as failf adds one itself. - Ref: #6074 - Closes #6108 + Also: add an assert() to failf() that triggers on a newline in the + format string! + + Closes #6365 -- CI/travis: add brotli and zstd to the libssh2 build +- [XhmikosR brought this change] + + CI: fix warning with the latest versions - ... to make sure such tests are run with valgrind. Suppress the zstd - valgrind warnings we get with version 1.3.3 on Ubuntu 18.04 (for debug - and non-debug builds). + `git checkout HEAD^2` is no longer needed - Closes #6105 + Closes #6369 -- runtests: revert the mistaken edit of $CURL +- INSTALL: update the list known OSes and CPU archs curl has run on - Regression from c4693adc62 + Closes #6366 -- RELEASE-NOTES: synced +- [Cherish98 brought this change] -- curl_url_set.3: fix typo in the RETURN VALUE section + curl: fix handling of -q option - Reported-by: Basuke Suzuki - Fixes #6102 - -Jay Satiro (17 Oct 2020) -- [Daniel Stenberg brought this change] + The match of the "-q" option (short for "--disable") should: + a) allow concatenation with other single-letters; and + b) be case-sensitive, lest confusing with "-Q" ("--quote") + + Closes #6364 - packages/OS400: make the source code-style compliant +- tests/badsymbols.pl: ignore stand-alone single hash lines - ... and make sure 'make checksrc' in the root dir also verifies the - packages/OS400 sources. + Bug: https://curl.se/mail/lib-2020-12/0084.html + Reported-by: Dennis Clarke + Assisted-by: Jay Satiro - Closes https://github.com/curl/curl/pull/6085 + Closes #6355 -- os400: Sync libcurl API options +- curl_easy_pause.3: add multiplexed pause effects - This fixes the OS400 build and also an incorrect entry for - CURLINFO_APPCONNECT_TIME_T where it was treated as - CURLINFO_STARTTRANSFER_TIME_T. + and generally refresh and update. Remove details for ancient versions. - Reported-by: Jon Rumsey + Reviewed-by: Jay Satiro + Closes #6360 + +Jay Satiro (22 Dec 2020) +- curl_easy_pause.3: fix man page reference - Fixes https://github.com/curl/curl/issues/6083 - Closes https://github.com/curl/curl/pull/6084 + Follow-up to ac9a724 from earlier today. + + Ref: https://github.com/curl/curl/pull/6359 -Daniel Stenberg (16 Oct 2020) -- CURLOPT_NOBODY.3: fix typo +Daniel Stenberg (22 Dec 2020) +- EXPERIMENTAL: add the Hyper backend to the list - Reported-by: Basuke Suzuki - Fixes #6097 + ... of current experimental features in curl. -Marc Hoersken (16 Oct 2020) -- CI/azure: improve on flakiness by avoiding libtool wrappers +- speedcheck: exclude paused transfers - Install curl binaries into MinGW bin folder and use that - for the tests in order to avoid libtool wrapper binaries. + Paused transfers should not be stopped due to slow speed even when + CURLOPT_LOW_SPEED_LIMIT is set. Additionally, the slow speed timer is + now reset when the transfer is unpaused - as otherwise it would easily + just trigger immediately after unpausing. - The libtool wrapper binaries (not scripts) on Windows seem - to be one of the possible causes for the following issues: + Reported-by: Harry Sintonen + Fixes #6358 + Closes #6359 + +- h2: do not wait for RECV on paused transfers - 1. Process output can be lost in the wrapper process chain. - 2. Killing the wrapper process does not kill the actual one. + ... as the socket might be readable all the time when paused and thus + causing a busy-loop. - Derived from #5904 - Closes #6049 + Reported-by: Harry Sintonen + Reviewed-by: Jay Satiro + Fixes #6356 + Closes #6357 -Daniel Stenberg (16 Oct 2020) -- CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well +- RELEASE-NOTES: synced -- [Zenju brought this change] +- cmdline-opts/gen.pl: return hard on errors + + ... as the warnings tend to go unnoticed otherwise! + + Closes #6354 - CURLOPT_TCP_NODELAY.3: fix comment in example code +- examples/libtest: add .checksrc to dist - Closes #6096 + ... so that (auto)builds from tarballs also get the correct instructions. + + Fixes #6176 + Closes #6353 -- openssl: acknowledge SRP disabling in configure properly +- test: verify new --write-out variables - Follow-up to 68a513247409 + Extended test 1029 and added 1188 + +- test970: adapted to the new internal order of variables + +- curl: add variables to --write-out - Use a new separate define that is the combination of both - HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP + In particular, these ones can help a user to create its own error + message when one or transfers fail. - Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html + writeout: add 'onerror', 'url', 'urlnum', 'exitcode', 'errormsg' - Closes #6094 - -Viktor Szakats (16 Oct 2020) -- http3: fix two build errors, silence warnings + onerror - lets a user only show the rest on non-zero exit codes - * fix two build errors due to mismatch between function - declarations and their definitions - * silence two mismatched signs warnings via casts + url - the input URL used for this transfer - Approved-by: Daniel Stenberg - Closes #6093 - -- Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 + urlnum - the numerical URL counter (0 indexed) for this transfer - Approved-by: Daniel Stenberg - Closes #6092 - -Daniel Stenberg (16 Oct 2020) -- tool_operate: fix compiler warning when --libcurl is disabled + exitcode - the numerical exit code for the transfer - Closes #6095 + errormsg - obvious + + Reported-by: Earnestly on github + Fixes #6199 + Closes #6207 -- checksrc: warn on empty line before open brace +- [Matthias Gatto brought this change] + + tests: add very simple AWS HTTP v4 Signature test - ... and fix a few occurances + Signed-off-by: Matthias Gatto + +- [Matthias Gatto brought this change] + + docs: add AWS HTTP v4 Signature + +- [Matthias Gatto brought this change] + + tool: add AWS HTTP v4 Signature support - Closes #6088 + Signed-off-by: Matthias Gatto -- urlapi: URL encode a '+' in the query part +- [Matthias Gatto brought this change] + + http: Make the call to v4 signature - ... when asked to with CURLU_URLENCODE. + This patch allow to call the v4 signature introduce in previous commit - Extended test 1560 to verify. - Reported-by: Dietmar Hauser - Fixes #6086 - Closes #6087 + Signed-off-by: Matthias Gatto -- [Cristian Morales Vega brought this change] +- [Matthias Gatto brought this change] - libcurl.pc: make it relocatable + http: introduce AWS HTTP v4 Signature - It supposes when people specify the libdir/includedir they do it to - change where under prefix/exec_prefix it should be, not to make it - independent of prefix/exec_prefix. + It is a security process for HTTP. - Closes #6061 - -- runtests: return error if no tests ran + It doesn't seems to be standard, but it is used by some cloud providers. - ... and make TESTFAIL stand out a little better by adding newlines - before and after. + Aws: + https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html + Outscale: + https://wiki.outscale.net/display/EN/Creating+a+Canonical+Request + GCP (I didn't test that this code work with GCP though): + https://cloud.google.com/storage/docs/access-control/signing-urls-manually - Reported-by: Marc Hörsken - Issue: #6052 - Closes #6053 + most of the code is in lib/http_v4_signature.c + + Information require by the algorithm: + - The URL + - Current time + - some prefix that are append to some of the signature parameters. + + The data extracted from the URL are: the URI, the region, + the host and the API type + + example: + https://api.eu-west-2.outscale.com/api/latest/ReadNets + ~~~ ~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ + ^ ^ ^ + / \ URI + API type region + + Small description of the algorithm: + - make canonical header using content type, the host, and the date + - hash the post data + - make canonical_request using custom request, the URI, + the get data, the canonical header, the signed header + and post data hash + - hash canonical_request + - make str_to_sign using one of the prefix pass in parameter, + the date, the credential scope and the canonical_request hash + - compute hmac from date, using secret key as key. + - compute hmac from region, using above hmac as key + - compute hmac from api_type, using above hmac as key + - compute hmac from request_type, using above hmac as key + - compute hmac from str_to_sign using above hmac as key + - create Authorization header using above hmac, prefix pass in parameter, + the date, and above hash + + Signed-off-by: Matthias Gatto + + Closes #5703 -- docs/FEATURE: convert to markdown +- [Matthias Gatto brought this change] + + http: add hmac support for sha256 - ... and clean it up a bit. + It seems current hmac implementation use md5 for the hash, + V4 signature require sha256, so I've added the needed struct in + this commit. - Closes #6067 + I've added the functions that do the hmac in v4 signature file + as a static function ,in the next patch of the serie, + because it's used only by this file. + + Signed-off-by: Matthias Gatto -- [Philipp Klaus Krause brought this change] +- [Cristian Rodríguez brought this change] - strerror: use 'const' as the string should never be modified + connect: on linux, enable reporting of all ICMP errors on UDP sockets - Closes #6068 + The linux kernel does not report all ICMP errors back to userspace due + to historical reasons. + + IP*_RECVERR sockopt must be turned on to have the correct behaviour + which is to pass all ICMP errors to userspace. + + See https://bugzilla.kernel.org/show_bug.cgi?id=202355 + + Closes #6341 -- [Jay Satiro brought this change] +- curl: add --create-file-mode [mode] + + This option sets the (octal) mode to use for the remote file when one is + created, using the SFTP, SCP or FILE protocols. When not set, the + default is 0644. + + Closes #6244 - connect: repair build without ipv6 availability +- c-hyper: fix compiler warnings - Assisted-by: Daniel Stenberg - Reported-by: Tom G. Christensen + Identified by clang on windows. - Fixes https://github.com/curl/curl/issues/6069 - Closes https://github.com/curl/curl/pull/6071 + Reported-by: Gisle Vanem + Bug: 58974d25d8173aec154e593ed9d866da566c9811 + + Closes #6351 -- RELEASE-NOTES: synced +- KNOWN_BUGS: Remote recursive folder creation with SFTP - Started over for the journey to next release. + Closes #5204 -- src/tool_filetime: disable -Wformat on mingw for this file +Jay Satiro (20 Dec 2020) +- badsymbols.pl: Add verbose mode -v - With gcc 10 on mingw we otherwise get this warning: + Use -v as the first option to enable verbose mode which will show source + input, extracted symbol and line info. For example: - error: ISO C does not support the 'I' printf flag [-Werror=format=] + Source: ./../include/curl/typecheck-gcc.h + Symbol: curlcheck_socket_info(info) + Line #423: #define curlcheck_socket_info(info) \ - Fixes #6079 - Closes #6082 + Ref: https://curl.se/mail/lib-2020-12/0084.html + + Closes https://github.com/curl/curl/pull/6349 -- test122[12]: remove these two tests +- KNOWN_BUGS: Secure Transport disabling hostname validation also disables SNI - ... and remove the objnames scripts they tested. They're not used for - anything anymore so testing them serves no purpose! + That behavior is a limitation of Apple's Secure Transport. - Reported-by: Marc Hörsken - Fixes #6080 - Closes #6081 + Reported-by: Cory Benfield + Reported-by: Ian Spence + Confirmed-by: Nick Zitzmann + + Ref: https://github.com/curl/curl/issues/998 + + Closes https://github.com/curl/curl/issues/6347 + Closes https://github.com/curl/curl/pull/6348 -Version 7.73.0 (14 Oct 2020) +Daniel Stenberg (18 Dec 2020) +- TODO: alt-svc should fallback if alt-svc doesn't work + + Closes #4908 -Daniel Stenberg (14 Oct 2020) -- RELEASE-NOTES: synced +- travis: restrict the openssl3 job to only run https and ftps tests - for 7.73.0 + ... as it runs too long otherwise and the other tests are verified in + other builds anyway. + + Closes #6345 -- THANKS: from 7.73.0 and .mailmap fixes +- build: repair http disabled but mqtt enabled build + + ... as the mqtt code reuses the "method" originally used for HTTP. + + Closes #6344 -- mailmap: fixups of some contributors +- [Jon Wilkes brought this change] -- projects/build-wolfssl.bat: fix the copyright year range + cookie: avoid the C1001 internal compiler error with MSVC 14 + + Fixes #6112 + Closes #6135 -Marc Hoersken (14 Oct 2020) -- [Sergei Nikulov brought this change] +- RELEASE-NOTES: synced - CI/tests: fix invocation of tests for CMake builds +- mqtt: handle POST/PUBLISH without a set POSTFIELDSIZE - Update appveyor.yml to set env variable TFLAGS and run tests - Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) - Move testdeps build to build step (per review comments) + Detected by OSS-Fuzz + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28735 - Reviewed-by: Marc Hörsken + Added test 1916 and 1917 to verify. - Closes #6066 - Fixes #6052 + Closes #6338 -- tests/server/util.c: fix support for Windows Unicode builds - - Detected via #6066 - Closes #6070 +- travis: add CI job for Hyper build -Daniel Stenberg (13 Oct 2020) -- [Jay Satiro brought this change] +- tests: updated tests for Hyper - strerror: Revert to local codepage for Windows error string +- lib: introduce c-hyper for using Hyper - - Change get_winapi_error() to return the error string in the local - codepage instead of UTF-8 encoding. + ... as an alternative HTTP backend within libcurl. + +- tool_setopt: provide helper output in debug builds - Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it - also changed the error string's encoding from local codepage to UTF-8. + ... for when setopt() returns error. + +- setopt: adjust to Hyper and disabled HTTP builds + +- rtsp: disable if Hyper is used + +- getinfo: build with disabled HTTP support + +- version: include hyper version + +- docs: add HYPER.md + +- configure: add --with-hyper - We return the local codepage version of the error string because if it - is output to the user's terminal it will likely be with functions which - expect the local codepage (eg fprintf, failf, infof). + As the first (optional) HTTP backend alternative instead of native - This is essentially a partial revert of bed5f84. The support for xbox - remains but the error string is reverted back to local codepage. + Close #6110 + +- test1522: add debug tracing - Ref: https://github.com/curl/curl/pull/6005 + I used this to track down some issues and I figured I could just as well + keep this extra logging in here for future needs. - Reviewed-by: Marcel Raad - Closes #6065 + Closes #6331 -Marc Hoersken (13 Oct 2020) -- CI/tests: use verification curl for test reporting APIs +- http: show the request as headers even when split-sending - Avoid using our own, potentially installed, curl for - the test reporting APIs in case it is broken. + When the initial request isn't possible to send in its entirety, the + remainder of request would be delivered to the debug callback as data + and would wrongly be counted internally as body-bytes sent. - Reviewed-by: Daniel Stenberg + Extended test 1295 to verify. - Preparation for #6049 - Closes #6063 + Closes #6328 -Viktor Szakats (12 Oct 2020) -- windows: fix comparison of mismatched types warning +- multi: when erroring in TOOFAST state, act as for PERFORM - clang 10, mingw-w64: - ``` - vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') - [-Wsign-compare] - if(GetLastError() != CRYPT_E_NOT_FOUND) - ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ - ``` + When failing in TOOFAST, the multi_done() wasn't called so the same + cleanup and handling wasn't done like when it fails in PERFORM, which in + the case of FTP could mean that the control connection wouldn't be + marked as "dead" for the CURLE_ABORTED_BY_CALLBACK case. Which caused + ftp_disconnect() to use it to send "QUIT", which could end up waiting + for a response a long time before giving up! - Approved-by: Daniel Stenberg - Closes #6062 - -Daniel Stenberg (11 Oct 2020) -- [Viktor Szakats brought this change] + Reported-by: Tomas Berger + Fixes #6333 + Closes #6337 - src/Makefile.m32: fix undefined curlx_dyn_* errors +- cmake: enable gophers correctly in curl-config - by linking `lib/dynbuf.c` when building a static curl binary. - Previously this source file was only included when building - a dynamic curl binary. This was likely possibly because no - functions from the `src/Makefile.inc` / `CURLX_CFILES` sources - were actually required for a curl tool build. This has - recently changed with the introduction of `curlx_dyn_*()` - memory functions and their use by the tool sources. + Closes #6336 + +- test1198/9: add two mqtt publish tests without payload lengths - Closes #6060 + Closes #6335 -- HISTORY: curl verifies SSL certs by default since version 7.10 +- tests/mqttd: extract the client id from the correct offset + + Closes #6334 -Marc Hoersken (8 Oct 2020) -- runtests.pl: use $LIBDIR variable instead of hardcoded path +- TODO: Prevent terminal injection when writing to terminal - Reviewed-by: Daniel Stenberg - Closes #6051 + Closes #6150 -Daniel Stenberg (7 Oct 2020) -- checksrc: detect // comments on column 0 +- Revert "CI/github: work-around for brew breakage on macOS" - Spotted while working on #6045 + This reverts commit 4cbb17a2cbbbe6337142d39479e21c3990b9c22f. - Closes #6048 - -- [Frederik Wedel-Heinen brought this change] - - mbedtls: add missing header when defining MBEDTLS_DEBUG + ... as the work-around now causes failures. - Closes #6045 + Closes #6332 -- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long - - Previously, it would pass on a define (int) which could make libcurl - read junk as a value - which prevented the CURLOPT_IPRESOLVE option to - "take". This could then make test 2100 do two DoH requests instead of - one! +- examples: remove superfluous asterisk uses - Fixes #6042 - Closes #6043 + ... for function pointers. Breaks in ancient compilers. - RELEASE-NOTES: synced -- scripts/release-notes.pl: don't "embed" $ in format string for printf() +- test1272: fix line ending - ... since they might contain %-codes that mess up the output! + Follow-up to f24784f9143 -Jay Satiro (5 Oct 2020) -- [M.R.T brought this change] +- URL-SYNTAX: add gophers details - build-wolfssl: fix build with Visual Studio 2019 - - Closes https://github.com/curl/curl/pull/6033 +- test1272: test gophers -Daniel Stenberg (4 Oct 2020) -- runtests: add %repeat[]% for test files - - ... and use this new keywords in all the test files larger than 50K to reduce - their sizes and make them a lot easier to read and understand. - - Closes #6040 +- runtests: add support for gophers, gopher over TLS -- [Emil Engler brought this change] +- [parazyd brought this change] - --help: move two options from the misc category + gopher: Implement secure gopher protocol. - The cmdline opts delegation and suppress-connect-headers - fit better into auth and proxy rather than misc. + This commit introduces a "gophers" handler inside the gopher protocol if + USE_SSL is defined. This protocol is no different than the usual gopher + prococol, with the added TLS encapsulation upon connecting. The protocol + has been adopted in the gopher community, and many people have enabled + TLS in their gopher daemons like geomyidae(8), and clients, like clic(1) + and hurl(1). - Follow-up to aa8777f63febc - Closes #6038 - -- [Samanta Navarro brought this change] - - docs/opts: fix typos in two manual pages + I have not implemented test units for this protocol because my knowledge + of Perl is sub-par. However, for someone more knowledgeable it might be + fairly trivial, because the same test that tests the plain gopher + protocol can be used for "gophers" just by adding a TLS listener. - Closes #6039 - -- ldap: reduce the amount of #ifdefs needed + Signed-off-by: parazyd - Closes #6035 + Closes #6208 -- runtests: provide curl's version string as %VERSION for tests - - ... so that we can check HTTP requests for User-Agent: curl/%VERSION +- TODO: Package curl for Windows in a signed installer - Update 600+ test cases accordingly. - - Closes #6037 + Closes #5424 -- checksrc: warn on space after exclamation mark +- mqtt: deal with 0 byte reads correctly - Closes #6034 - -- test1465: verify --libcurl with binary POST data - -- runtests: allow generating a binary sequence from hex - -- tool_setopt: escape binary data to hex, not octal - -- curl: make --libcurl show binary posts correctly + OSS-Fuzz found it + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=28676 - Reported-by: Stephan Mühlstrasser - Fixes #6031 - Closes #6032 + Closes #6327 -Jay Satiro (1 Oct 2020) -- strerror: fix null deref on winapi out-of-memory +- BUG-BOUNTY: minor language update - Follow-up to bed5f84 from several days ago. + ... and remove the wording about entries from before 2019 as the "within + 12 months" is still there and covers that. - Ref: https://github.com/curl/curl/pull/6005 + Closes #6318 -Daniel Stenberg (1 Oct 2020) -- [Kamil Dudka brought this change] - - vtls: deduplicate some DISABLE_PROXY ifdefs +- tooĺ_writeout: fix the -w time output units - ... in the code of gtls, nss, and openssl + Fix regression from commit fc813f80e1bcac (#6248) that changed the unit + to microseconds instead of seconds with fractions - Closes #5735 - -- RELEASE-NOTES: synced + Reported-by: 不确定 + Fixes #6321 + Closes #6322 -- [Emil Engler brought this change] +- quiche: remove fprintf() leftover - TODO: Add OpenBSD libtool notice +Jay Satiro (14 Dec 2020) +- KNOWN_BUGS: SHA-256 digest not supported in Windows SSPI builds - See #5862 - Closes #6030 + Closes https://github.com/curl/curl/issues/6302 -- tests/unit/README: convert to markdown +- digest_sspi: Show InitializeSecurityContext errors in verbose mode - ... and add to dist! + The error is shown with infof rather than failf so that the user will + see the extended error message information only in verbose mode, and + will still see the standard CURLE_AUTH_ERROR message. For example: - Closes #6028 - -- tests/README: convert to markdown + --- - Closes #6028 - -- include/README: convert to markdown + * schannel: InitializeSecurityContext failed: SEC_E_QOP_NOT_SUPPORTED + (0x8009030A) - The per-message Quality of Protection is not supported by + the security package + * multi_done + * Connection #1 to host 127.0.0.1 left intact + curl: (94) An authentication function returned an error - Closes #6028 + --- + + Ref: https://github.com/curl/curl/issues/6302 + + Closes https://github.com/curl/curl/pull/6315 -- examples/README: convert to markdown +Daniel Stenberg (13 Dec 2020) +- URL-SYNTAX: add default port numbers and IDNA details - Closes #6028 + Closes #6316 -- configure: don't say HTTPS-proxy is enabled when disabled! +- URL-SYNTAX: mention how FILE:// access can access network on windows - Reported-by: Kamil Dudka - Reviewed-by: Kamil Dudka - Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 - Closes #6029 + Closes #6314 -Daniel Gustafsson (30 Sep 2020) -- src: Consistently spell whitespace without whitespace +Jay Satiro (12 Dec 2020) +- URL-SYNTAX: Document default SMTP port 25 - Whitespace is spelled without a space between white and space, so - make sure to consistently spell it that way across the codebase. + Note that ports 25 and 587 are common ports for smtp, the former being + the default. - Closes #6023 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler + Closes https://github.com/curl/curl/pull/6310 -- MANUAL: update examples to resolve without redirects +Daniel Stenberg (12 Dec 2020) +- CURLOPT_URL.3: remove scheme specific details - www.netscape.com is redirecting to a cookie consent form on Aol, and - cool.haxx.se isn't responding to FTP anymore. Replace with examples - that resolves in case users try out the commands when reading the - manual. + ... that are now found in URL-SYNTAX.md - Closes #6024 - Reviewed-by: Daniel Stenberg - Reviewed-by: Emil Engler - -Daniel Stenberg (30 Sep 2020) -- HISTORY: add some 2020 events + Closes #6307 -- sectransp: make it build with --disable-proxy +Dan Fandrich (12 Dec 2020) +- docs: Fix some typos - Follow-up from #5466 and f3d501dc678d80 - Reported-by: Javier Navarro - Fixes #6025 - Closes #6026 + [skip ci] -- ECH: renamed from ESNI in docs and configure - - Encrypted Client Hello (ECH) is the current name. +Daniel Stenberg (12 Dec 2020) +- URL-SYNTAX: mention all supported schemes - Closes #6022 + Closes #6311 -- configure: use "no" instead of "disabled" for the end summary - - ... for consistency but also to make them more distinctly stand out next - to the "enabled" lines. +- [Douglas R. Reno brought this change] -- TODO: SSH over HTTPS proxy with more backends + URL-SYNTAX.md: minor language improvements - ... as right now only the libssh2 backend supports it. + Closes #6308 -- libssh2: handle the SSH protocols done over HTTPS proxy +- docs/URL-SYNTAX: the URL syntax curl accepts and works with - Reported-by: Robin Douine - Fixes #4295 - Closes #6021 + Closes #6285 -- [Emil Engler brought this change] +- [0xflotus brought this change] - memdebug: remove 9 year old unused debug function + docs: enable syntax highlighting in several docs files - There used to be a way to have memdebug fill allocated memory. 9 years - later this has no value there (valgrind and ASAN etc are way better). If - people need to know about it they can have a look at VCS logs. + ... for better readability - Closes #5973 + Closes #6286 -- sendf: move Curl_sendf to dict.c and make it static - - ... as the only remaining user of that function. Also fix gopher.c to - instead use Curl_write() +- test1564/1565: require the 'wakeup' feature to run - Closes #6020 + Fixes #6299 + Fixes #6300 + Closes #6301 -- ROADMAP: updates and cleanups - - Fix the HSTS PR - - Remove DoT, thread-safe init and hard-coded localhost. I feel very - little interest for these with users so I downgrade them to plain "TODO" - entries again. +- runtests: add 'wakeup' as a feature -- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root - - This matches what is returned in other TLS backends in the same - situation. +- tests/server/disabled: add "wakeup" - Reviewed-by: Jay Satiro - Reviewed-by: Emil Engler - Follow-up to 5a3efb1 - Reported-by: iammrtau on github - Fixes #6003 - Closes #6018 + To allow the test suite to know if wakeup support is disabled in the + build. -- RELEASE-NOTES: synced +- lib1564/5: verify that curl_multi_wakeup returns OK -- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL +- tests: make --libcurl tests only test FTP options if ftp enabled - Added test 348 to verify. Added a 'STOR' command to the test FTP - server to enable test 348. Documented the command in FILEFORMAT.md + Adjust six --libcurl tests to only check the FTP option if FTP is + actually present in the build. - Reported-by: Duncan Wilcox - Fixes #6016 - Closes #6017 + Fixes #6303 + Closes #6305 -- pause: only trigger a reread if the unpause sticks +- runtests.pl: fix "uninitialized value" warning - As an unpause might itself get paused again and then triggering another - reread doesn't help. + follow-up to e12825c642a88774 + +- runtests: add support for %if [feature] conditions - Follow-up from e040146f22608fd9 (shipped since 7.69.1) + ... to make tests run differently or expect different results depending + on what features that are present or not in curl. - Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html - Patch-by: Kunal Chandarana - Fixes #5988 - Closes #6013 + Bonus: initial minor 'Hyper' awareness but nothing is using that yet + + Closes #6304 -- test163[12]: require http to be built-in to run +- [Jon Rumsey brought this change] + + OS400: update ccsidcurl.c - ... as speaking over an HTTPS proxy implies http! + Add 'struct' to cast and declaration of cfcdata to fix compilation + error. - Closes #6014 + Fixes #6292 + Closes #6297 -- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define +- ngtcp2: make it build it current master again - Closes #6012 + Closes #6296 -- [Javier Blazquez brought this change] +- [Cristian Rodríguez brought this change] - strerror: honor Unicode API choice on Windows + connect: defer port selection until connect() time - Closes #6005 - -- imap: make imap_send use dynbuf for the send buffer management + If supported, defer port selection until connect() time + if --interface is given and source port is 0. - Reuses the buffer and thereby reduces number of mallocs over a transfer. + Reproducer: - Closes #6010 - -- Curl_send: return error when pre_receive_plain can't malloc + * start fast webserver on port 80 + * starve system of ephemeral ports + $ sysctl net.ipv4.ip_local_port_range="60990 60999" - ... will probably trigger some false DEAD CODE positives on non-windows - code analyzers for the conditional code. + * start a curl/libcurl "crawler" + $curl --keepalive --parallel --parallel-immediate --head --interface + 127.0.0.2 "http://127.0.0.[1-254]/file[001-002].txt" - Closes #6011 - -- ftp: separate FTPS from FTP over "HTTPS proxy" + current result: + (possible some successful data) + curl: (45) bind failed with errno 98: Address already in use - When using HTTPS proxy, SSL is used but not in the view of the FTP - protocol handler itself so separate the connection's use of SSL from the - FTP control connection's sue. + result after patch: + (complete success or few connections failing, higlhy depending on load) - Reported-by: Mingtao Yang - Fixes #5523 - Closes #6006 - -Dan Fandrich (23 Sep 2020) -- tests/data: Fix some mismatched XML tags in test cases + Fail only when all the possible 4-tuple combinations are exhausted, + which is impossible to do when port is selected at bind() time becuse + the kernel does not know if socket will be listen()'ed on or connect'ed + yet. - This allows these test files to pass xmllint. + Closes #6295 -Daniel Stenberg (23 Sep 2020) -- pingpong: use a dynbuf for the *_pp_sendf() function - - ... reuses the same dynamic buffer instead of doing repeated malloc/free - cycles. +- [Hans-Christian Noren Egtvedt brought this change] + + connect: zero variable on stack to silence valgrind complaint - Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls - after this change in my test setup (132 => 125), curl 7.72.0 needed 140 - calls for this. + Valgrind will complain that ssrem buffer usage if not explicit + initialized, hence initialize it to zero. - Test case 103 makes 9 less allocations now (130). Down from 149 in - 7.72.0. + This completes the change intially started in commit 2c0d7212151 ('ftp: + retry getpeername for FTP with TCP_FASTOPEN') where the ssloc buffer has + a similar memset to zero. - Closes #6004 + Signed-off-by: Hans-Christian Noren Egtvedt + Closes #6289 -- dynbuf: add Curl_dyn_vaddf +- RELEASE-NOTES: synced - Closes #6004 + start over on the next release cycle -- dynbuf: make *addf() not require extra mallocs +Version 7.74.0 (9 Dec 2020) + +Daniel Stenberg (9 Dec 2020) +- RELEASE-NOTES: synced - ... by introducing a printf() function that appends directly into a - dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if - the buffer is already big enough it can just printf directly into it. + for 7.74.0 + +Jay Satiro (7 Dec 2020) +- [Jacob Hoffman-Andrews brought this change] + + urldata: restore comment on ssl_connect_data.use - Since this less-malloc version requires tthe use of a library internal - printf function, we only provide this version when building libcurl and - not for the dynbuf code that is used when building the curl tool. + This comment was originally on the `use` field, but was separated from + its field in 62a2534. - Closes #5998 + Closes https://github.com/curl/curl/pull/6287 -- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport +Daniel Stenberg (7 Dec 2020) +- VERSIONS: refreshed - Closes #5403 + We always use the patch number these days: all releases are + "major.minor.patch" -- pingpong: remove a malloc per Curl_pp_vsendf call +- [Jakub Zakrzewski brought this change] + + cmake: don't use reserved target name 'test' - This typically makes 7-9 fewer mallocs per FTP transfer. + CMake up to 3.10 always reserves this name - Closes #5997 + Fixes #6257 + Closes #6258 -- symbian: drop support +- openssl: make the OCSP verification verify the certificate id - The OS is deprecated. I see no traces of anyone having actually built - curl for Symbian after 2012. + CVE-2020-8286 - The public headers are unmodified. + Reported by anonymous - Closes #5989 - -- RELEASE-NOTES: synced + Bug: https://curl.se/docs/CVE-2020-8286.html -- curl_krb5.h: rename from krb5.h +- ftp: make wc_statemach loop instead of recurse - Follow-up from f4873ebd0be32cf + CVE-2020-8285 - Turns out some older openssl installations go bananas otherwise. - Reported-by: Tom van der Woerdt - Fixes #5995 - Closes #5996 - -- test1297: verify GOT_NOTHING with http proxy tunnel + Fixes #6255 + Bug: https://curl.se/docs/CVE-2020-8285.html + Reported-by: xnynx on github -- http_proxy: do not count proxy headers in the header bytecount +- ftp: CURLOPT_FTP_SKIP_PASV_IP by default - ... as that counter is subsequently used to detect if nothing was - returned from the peer. This made curl return CURLE_OK when it should - have returned CURLE_GOT_NOTHING. + The command line tool also independently sets --ftp-skip-pasv-ip by + default. - Fixes #5992 - Reported-by: Tom van der Woerdt - Closes #5994 - -- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument + Ten test cases updated to adapt the modified --libcurl output. - Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the - option is, yeah, not known. Clarified this in the setopt man page too. + Bug: https://curl.se/docs/CVE-2020-8284.html + CVE-2020-8284 - Closes #5993 + Reported-by: Varnavas Papaioannou -- krb5: merged security.c and krb specific FTP functions in here +- urlapi: don't accept blank port number field without scheme - These two files were always tightly connected and it was hard to - understand what went into which. This also allows us to make the - ftpsend() function static (moved from ftp.c). + ... as it makes the URL parser accept "very-long-hostname://" as a valid + host name and we don't want that. The parser now only accepts a blank + (no digits) after the colon if the URL starts with a scheme. - Removed security.c - Renamed curl_sec.h to krb5.h + Reported-by: d4d on hackerone - Closes #5987 + Closes #6283 -- Curl_handler: add 'family' to each protocol - - Makes get_protocol_family() faster and it moves the knowledge about the - "families" to each protocol handler, where it belongs. +- Revert "multi: implement wait using winsock events" - Closes #5986 - -- parsedate: tune the date to epoch conversion + This reverts commit d2a7d7c185f98df8f3e585e5620cbc0482e45fac. - By avoiding an unnecessary error check and the temp use of the tm - struct, the time2epoch conversion function gets a little bit faster. - When repeating test 517, the updated version is perhaps 1% faster (on - one particular build on one particular architecture). + This commit also reverts the subsequent follow-ups to that commit, which + were all done within windows #ifdefs that are removed in this + change. Marc helped me verify this. - Closes #5985 + Fixes #6146 + Closes #6281 -- cmake: remove scary warning +- [Klaus Crusius brought this change] + + ftp: retry getpeername for FTP with TCP_FASTOPEN - Remove the text saying + In the case of TFO, the remote host name is not resolved at the + connetion time. - "the curl cmake build system is poorly maintained. Be aware" + For FTP that has lead to missing hostname for the secondary connection. + Therefore the name resolution is done at the time, when FTP requires it. - ... not because anything changed just now, but to encourage users to use - it and subsequently improve it. + Fixes #6252 + Closes #6265 + Closes #6282 + +- [Thomas Danielsson brought this change] + + scripts/completion.pl: parse all opts - Closes #5984 + For tab-completion it may be preferable to include all the + available options. + + Closes #6280 -- docs/MQTT: remove outdated paaragraphs +- RELEASE-NOTES: synced -- docs/MQTT: not experimental anymore +- openssl: use OPENSSL_init_ssl() with >= 1.1.0 - Follow-up to e37e4468688d8f + Reported-by: Kovalkov Dmitrii and Per Nilsson + Fixes #6254 + Fixes #6256 + Closes #6260 -- docs/RESOURCES: remove +- SECURITY-PROCESS: disclose on hackerone - This document is not maintained and rather than trying to refresh it, - let's kill it. A more up-to-date document with relevant RFCs is this - page on the curl website: https://curl.haxx.se/rfc/ + Once a vulnerability has been published, the hackerone issue should be + disclosed. For tranparency. - Closes #5980 + Closes #6275 -- docs/TheArtOfHttpScripting: convert to markdown +Marc Hoersken (3 Dec 2020) +- tests/util.py: fix compatibility with Python 2 - Makes it easier to browse on github etc. Offers (better) links. + Backporting the Python 3 implementation of setStream + to ClosingFileHandler as a fallback within Python 2. - It should be noted that this document is already mostly outdated and - "Everything curl" at https://ec.haxx.se/ is a better resource and - tutorial. + Reported-by: Jay Satiro - Closes #5981 + Fixes #6259 + Closes #6270 -- BUGS: convert document to markdown +Daniel Gustafsson (3 Dec 2020) +- docs: fix typos and markup in ETag manpage sections - Closes #5979 + Reported-by: emanruse on github + Fixes #6273 -- --help: strdup the category - - ... since it is converted and the original pointer is freed on Windows - unicode handling. +Daniel Stenberg (2 Dec 2020) +- quiche: close the connection - Follow-up to aa8777f63febc - Fixes #5977 - Closes #5978 - Reported-by: xwxbug on github + Reported-by: Junho Choi + Fixes #6213 + Closes #6217 -- CHECKSRC: document two missing warnings +Jay Satiro (2 Dec 2020) +- ngtcp2: Fix build error due to symbol name change + + - NGTCP2_CRYPTO_LEVEL_APP -> NGTCP2_CRYPTO_LEVEL_APPLICATION + + ngtcp2/ngtcp2@76232e9 changed the name. + + ngtcp2 master is required to build curl with http3 support. + + Closes https://github.com/curl/curl/pull/6271 -- RELEASE-NOTES: synced +Daniel Stenberg (1 Dec 2020) +- [Klaus Crusius brought this change] -- ftp: avoid risk of reading uninitialized integers + cmake: check for linux/tcp.h - If the received PASV response doesn't match the expected pattern, we - could end up reading uninitialized integers for IP address and port - number. + The HAVE_LINUX_TCP_H define was not set by cmake. - Issue pointed out by muse.dev - Closes #5972 - -- [Quentin Balland brought this change] + Closes #6252 - easy_reset: clear retry counter +- NEW-PROTOCOL: document what needs to be done to add one - Closes #5975 - Fixes #5974 + Closes #6263 -- ftp: get rid of the PPSENDF macro +- splay: rename Curl_splayremovebyaddr to Curl_splayremove - The use of such a macro hides some of what's actually going on to the - reader and is generally disapproved of in the project. + ... and remove the old unused proto for the old Curl_splayremove + version. - Closes #5971 + Closes #6269 -- man pages: switch to https://example.com URLs +- openssl: free mem_buf in error path - Since HTTPS is "the new normal", this update changes a lot of man page - examples to use https://example.com instead of the previous "http://..." + To fix a memory-leak. - Closes #5969 + Closes #6267 -- github: remove the duplicate "Security vulnerability" entry +- openssl: remove #if 0 leftover - ... since github adds an entry automatically by itself. + Follow-up to 4c9768565ec3a9 (from Sep 2008) - Closes #5970 - -- [Emil Engler brought this change] + Closes #6268 - github: use new issue template feature +- ntlm: avoid malloc(0) on zero length user and domain - This helps us to avoid getting feature requests as well as security - bugs reported into the issue tracker. + ... and simplify the too-long checks somewhat. - Closes #5936 + Detected by OSS-Fuzz + + Closes #6264 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - urlapi: use more Curl_safefree +Marc Hoersken (28 Nov 2020) +- tests/server/tftpd.c: close upload file in case of abort - Closes #5968 - -Marc Hoersken (17 Sep 2020) -- multi: align WinSock mask variables in Curl_multi_wait - - Also skip pre-checking sockets to set timeout_ms to 0 - after the first socket has been detected to be ready. + Commit c353207 removed the closing right after do_tftp + which covered the case of abort. This handles that case. - Reviewed-by: rcombs on github + Reviewed-by: Jay Satiro Reviewed-by: Daniel Stenberg - Follow up to #5886 + Follow up to #6209 + Closes #6234 -- multi: reuse WinSock events variable in Curl_multi_wait - - Since the struct is quite large (1 long and 10 ints) we - declare it once at the beginning of the function instead - of multiple times inside loops to avoid stack movements. - - Reviewed-by: Viktor Szakats - Reviewed-by: Daniel Stenberg - - Closes #5886 +Daniel Stenberg (26 Nov 2020) +- [Daiki Ueno brought this change] -Daniel Stenberg (16 Sep 2020) -- TODO: dynamically decide to use socketpair + ngtcp2: use the minimal version of QUIC supported by ngtcp2 - Suggested-by: Anders Bakken + Closes #6250 + +- [Daiki Ueno brought this change] + + ngtcp2: advertise h3 ALPN unconditionally - Closes #4829 + Closes #6250 -- TODO: add PR reference for native IDN support on macOS +- [Daiki Ueno brought this change] + + vquic/ngtcp2.h: define local_addr as sockaddr_storage - As there was work started on this that never got completed. + This field needs to be wide enough to hold sockaddr_in6 when + connecting via IPv6. Otherwise, ngtcp2_conn_read_pkt will drop the + packets because of the address mismatch: + I00000022 [...] con ignore packet from unknown path - Closes #5371 - -- tool_help.h: update copyright year range + We can safely assume that struct sockaddr_storage is available, as it + is used in the public interface of ngtcp2. - Follow-up from aa8777f63febca + Closes #6250 -- CI/azure: disable test 571 in the msys2 builds +- socks: check for DNS entries with the right port number - It's just too flaky there + The resolve call is done with the right port number, but the subsequent + check used the wrong one, which then could find a previous resolve which + would return and leave the fresh resolve "incomplete" and leaking + memory. - Reviewed-by: Marc Hoersken - Closes #5954 + Fixes #6247 + Closes #6253 -- tool_writeout: protect fputs() from NULL +- curl_setup: USE_RESOLVE_ON_IPS is for Apple native resolver use - When the code was changed to do fputs() instead of fprintf() it got - sensitive for NULL pointers; add checks for that. + ... so don't define it when instructed to use c-ares! + +- test506: make it not run in c-ares builds - Follow-up from 0c1e767e83ec66 + As the asynch nature of it may trigger events in another order. A c-ares + upgrade made it break. - Closes #5963 + Reported-by: Marc Hörsken + Fixes #6247 -- test3015: verify stdout "as text" - - Follow-up from 0c1e767e83e to please win32 tests +- runtests: make 'c-ares' a "feature" to depend on - Closes #5962 + ... also added to the docs. -- travis: use libressl v3.1.4 instead of master +- tool_writeout: use off_t getinfo-types instead of doubles - ... as their git master seems too fragile to use (and 3.2.1 which is the - latest has a build failure). + Commit 3b80d3ca46b12e52342 (June 2017) introduced getinfo replacement + variables that use curl_off_t instead of doubles. Switch the --write-out + function over to use them. - Closes #5964 + Closes #6248 -- tests/FILEFORMAT: document type=shell for +- [Emil Engler brought this change] -- tests/FILEFORMAT: document nonewline support for + file: avoid duplicated code sequence - The one in , that creates files. + file_disconnect() is identical with file_do() except the function header + but as the arguments are unused anyway so why not just return file_do() + directly! - Follow-up from b83947c8df7 + Reviewed-by: Daniel Stenberg + Closes #6249 -- [anio brought this change] +- [Rikard Falkeborn brought this change] - tool_writeout: add new writeout variable, %{num_headers} + infof/failf calls: fix format specifiers - This variable gives the number of headers. + Update a few format specifiers to match what is being printed. - Closes #5947 + Closes #6241 -- tool_urlglob: fix compiler warning "unreachable code" +- docs/INTERNALS: remove reference to Curl_sendf() - (On Windows builds.) + The function has been removed from common usage. Also removed comment in + gopher.c that still referenced it. - Follow-up to 70a3b003d9 + Reported-by: Rikard Falkeborn + Fixes #6242 + Closes #6243 -- [Gergely Nagy brought this change] +- [Rikard Falkeborn brought this change] - vtls: deduplicate client certificates in ssl_config_data + examples: update .gitignore - Closes #5629 - -- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND + Add files that are generated by 'make examples' and remove some that + have been renamed. - This is primarily interesting for cases where CURLOPT_NOBODY is set as - previously curl would not return an error for this case. + The commits that renamed the programs are e9625c5bc6c046a (imap.c and + simplesmtp.c were renamed to imap-fetch.c and smtp-send.c) and + ad39e7ec01e7 (pop3slist.c and pop3s.c were renamed to pop3-list.c and + pop3-ssl.c). - MDTM getting 550 now also returns this error (it returned - CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for - missing files across protocols and specific FTP commands. + Closes #6240 + +- asyn: use 'struct thread_data *' instead of 'void *' - libcurl already returns error on a 550 as a MDTM response (when - CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would - happen subsequently anyway since the RETR command would fail. + To reduce use of types that can't be checked at compile time. Also + removes several typecasts. - Add test 1913 and 1914 to verify. Updated several tests accordingly due - to the updated SIZE behavior. + ... and rename the struct field from 'os_specific' to 'tdata'. - Reported-by: Tomas Berger - Fixes #5953 - Closes #5957 + Closes #6239 + Reviewed-by: Jay Satiro -- curl: make checkpasswd use dynbuf +Viktor Szakats (23 Nov 2020) +- Makefile.m32: add support for UNICODE builds - Closes #5952 - -- curl: make glob_match_url use dynbuf + It requires the linker to support the `-municode` option. + This is available in more recent mingw-w64 releases. - Closes #5952 - -- curl: make file2memory use dynbuf + Ref: https://gcc.gnu.org/onlinedocs/gcc/x86-Windows-Options.html + Ref: https://stackoverflow.com/questions/3571250/wwinmain-unicode-and-mingw/11706847#11706847 - Closes #5952 - -- curl: make file2string use dynbuf + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad - Closes #5952 - -- [Antarpreet Singh brought this change] + Closes #6228 - imap: set cselect_bits to CURL_CSELECT_IN initially - - ... when continuing a transfer from a FETCH response. +Daniel Stenberg (23 Nov 2020) +- urldata: remove 'void *protop' and create the union 'p' - When the size of the file was small enough that the entirety of the - transfer happens in a single go and schannel buffers holds the entire - data. However, it wasn't completely read in Curl_pp_readresp since a - line break was found before that could happen. So, by the time we are in - imap_state_fetch_resp - there's data in buffers that needs to be read - via Curl_read but nothing to read from the socket. After we setup a - transfer (Curl_setup_transfer), curl just waits on the socket state to - change - which doesn't happen since no new data ever comes. + ... to avoid the use of 'void *' for the protocol specific structs done + per transfer. - Closes #5961 - -- RELEASE-NOTES: synced + Closes #6238 -- test434: test -K use in a single line without newline +- winbuild: remove docs from Makefiles and refer to README.md - Closes #5946 - -- runtests: allow creating files without newlines + Reduce risk for conflicting docs and makes it to a single place to fix + and polish. - Closes #5946 - -- curl: use curlx_dynbuf for realloc when loading config files + add these missing options to the readme: - ... fixes an integer overflow at the same time. + ENABLE_OPENSSL_AUTO_LOAD_CONFIG and ENABLE_UNICODE - Reported-by: ihsinme on github - Assisted-by: Jay Satiro + clarify ENABLE_SCHANNEL default varies - Closes #5946 + Fixes #6216 + Closes #6227 + Co-Authored-by: Jay Satiro -- dynbuf: provide curlx_ names for reuse by the curl tool +- [Daiki Ueno brought this change] + + http3: use the master branch of GnuTLS for testing - Closes #5946 + Closes #6235 -- dynbuf: make sure Curl_dyn_tail() zero terminates +- KNOWN_BUGS: curl with wolfSSL lacks support for renegotiation - Closes #5959 + Closes #5839 -- tests: add test1912 to the dist +- KNOWN_BUGS: wakeup socket disconnect causes havoc - Follow-up to 70984ce1be4cab6c + Closes #6132 + Closes #6133 -- docs/LICENSE-MIXING: remove +- RELEASE-NOTES: synced + +- [Oliver Urbann brought this change] + + curl: add compatibility for Amiga and GCC 6.5 - This document is not maintained and I feel that it doesn't provide much - value to users anymore (if it ever did). + Changes are mainly reordering and adding of includes required + to compile with a more recent version of GCC. - Closes #5955 - -- [Laramie Leavitt brought this change] + Closes #6220 - http: consolidate nghttp2_session_mem_recv() call paths +Marc Hoersken (20 Nov 2020) +- tests/server/tftpd.c: close upload file right after transfer - Previously there were several locations that called - nghttp2_session_mem_recv and handled responses slightly differently. - Those have been converted to call the existing - h2_process_pending_input() function. + Make sure uploaded file is no longer locked after the + transfer while waiting for the final ACK to be handled. - Moved the end-of-session check to h2_process_pending_input() since the - only place the end-of-session state can change is after nghttp2 - processes additional input frames. + Assisted-by: Daniel Stenberg - This will likely fix the fuzzing error. While I don't have a root cause - the out-of-bounds read seems like a use after free, so moving the - nghttp2_session_check_request_allowed() call to a location with a - guaranteed nghttp2 session seems reasonable. - - Also updated a few nghttp2 callsites to include error messages and added - a few additional error checks. - - Closes #5648 - -- HISTORY: mention alt-svc added in 2019 - - ... and make 1996 the first year subtitle + Bug: #6058 + Closes #6209 -- base64: also build for pop3 and imap - - Follow-up to the fix in 20417a13fb8f83 +- CI/cirrus: simplify logic for disabled tests - Reported-by: Michael Olbrich - Fixes #5937 - Closes #5948 - -- base64: enable in build with SMTP + The OpenSSH server instance for the testsuite cannot + be started on FreeBSD, therefore the SFTP and SCP + tests are disabled right away from the beginning. - The oauth2 support is used with SMTP and it uses base64 functions. + The previous OS version specific logic for SKIP_TESTS + is no longer needed/used and can therefore be removed. - Reported-by: Michael Olbrich - Fixes #5937 - Closes #5938 - -- curl_mime_headers.3: fix the example's use of curl_slist_append + Reviewed-by: Daniel Stenberg - Reported-by: sofaboss on github - Fixes #5942 - Closes #5943 + Follow up to #6211 + Closes #6229 -- lib583: fix enum mixup +Daniel Gustafsson (20 Nov 2020) +- mailmap: Daniel Hwang - grrr the previous follow-up to 17fcdf6a31 was wrong - -- libtest: fix build errors + Add Daniel Hwang to the mailmap to cover the alternative spelling + Daniel Lee Hwang which was used in one commit. - Follow-up from 17fcdf6a310d4c8076 + Closes #6230 + Reviewed-by: Daniel Stenberg -- lib: fix -Wassign-enum warnings +- openssl: guard against OOM on context creation - configure --enable-debug now enables -Wassign-enum with clang, - identifying several enum "abuses" also fixed. + EVP_MD_CTX_create will allocate memory for the context and returns + NULL in case the allocation fails. Make sure to catch any allocation + failures and exit early if so. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 + In passing, also move to EVP_DigestInit rather than EVP_DigestInit_ex + as the latter is intended for ENGINE selection which we don't do. - Closes #5929 - -- RELEASE-NOTES: synced + Closes #6224 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- [Diven Qi brought this change] +Daniel Stenberg (19 Nov 2020) +- [Vincent Torri brought this change] - url: use blank credentials when using proxy w/o username and password - - Fixes proxy regression brought in commit ad829b21ae (7.71.0) + cmake: use libcurl.rc in all Windows builds - Fixed #5911 - Closes #5914 + Reviewed-by: Marcel Raad + Closes #6215 -- travis: add a build using libressl (from git master) +- [Cristian Morales Vega brought this change] + + cmake: make CURL_ZLIB a tri-state variable - The v3.2.1 tag (latest release atm) results in a broken build. + By differentiating between ON and AUTO it can make a missing zlib + library a hard error when CURL_ZLIB=ON is used. - Closes #5932 + Reviewed-by: Jakub Zakrzewski + Closes #6221 + Fixes #6173 -- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 +- quiche: remove 'static' from local buffer - Unfortunately, this option is not detecting the same issues as clang's - -Wassign-enum flag, but should still be useful to detect future - mistakes. + For thread-safety - Closes #5930 + Closes #6223 -- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - - If the error reason from the lib is - SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return - CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. +- KNOWN_BUGS: cmake: libspsl is not supported - This unifies the libcurl return code and makes libressl run test 313 - (CRL testing) fine. + Closes #6214 + +- KNOWN_BUGS: cmake autodetects cert paths when cross-compiling - Closes #5934 + Closes #6178 -- FAQ: refreshed some very old language +- KNOWN_BUGS: cmake build doesn't fail if zlib not found + + Closes #6173 -- cmake: make HTTP_ONLY also disable MQTT +- KNOWN_BUGS: cmake libcurl.pc uses absolute library paths - ... and alphasort the order of disabling protocols to make it easier to - browse. + Closes #6169 + +- KNOWN_BUGS: cmake: generated .pc file contains strange entries - Closes #5931 + Closes #6167 -- libtest: remove lib1541 leftovers +- KNOWN_BUGS: cmake uses -lpthread instead of Threads::Threads - Caused automake errors. + Closes #6166 + +- KNOWN_BUGS: cmake build in Linux links libcurl to libdl - Follow-up to 8ca54a03ea08a + Closes #6165 -- tests/libtests: remove test 1900 and 2033 +- KNOWN_BUGS: make a new section for cmake topics - We already remove the test files, now remove the libtest codes as well. + Closes #6219 + +- [Emil Engler brought this change] + + cirrus: build with FreeBSD 12.2 in CirrusCI - Follow-up to e50a877df74 + Closes #6211 -Marc Hoersken (7 Sep 2020) -- CI/azure: add test number to title for display in analytics +Marc Hoersken (14 Nov 2020) +- tests/*server.py: close log file after each log line - To ease identification of tests the test number is added to - the test case title in order to have it on the Azure DevOps - Analytics pages and reports which currently do not show it. + Make sure the log file is not locked once a test has + finished and align with the behavior of our logmsg. - Bump test case revision to make Azure DevOps update titles. + Rename curl_test_data.py to be a general util.py. + Format and sort Python imports with isort/VSCode. - Closes #5927 + Bug: #6058 + Closes #6206 -Daniel Stenberg (6 Sep 2020) -- altsvc: clone setting in curl_easy_duphandle - - The cache content is not duplicated, like other caches, but the setting - and specified file name are. - - Test 1908 is extended to verify this somewhat. Since the duplicated - handle gets the same file name, the test unfortunately overwrites the - same file twice (with different contents) which makes it hard to check - automatically. +Daniel Stenberg (13 Nov 2020) +- CURLOPT_HSTS.3: document the file format - Closes #5923 + Closes #6205 -- test1541: remove since it is a known bug +- RELEASE-NOTES: synced + +- release-notes.pl: detect #[number] better for Ref: etc + +- curl: only warn not fail, if not finding the home dir - A shared connection cache is not thread-safe is a known issue. Stop - testing this until we believe this issue is addressed. Reduces - occasional test failures we don't care about. + ... as there's no good reason to error out completely. - The test code in lib1541.c is left in git to allow us to restore it when - we get to fix this. + Reported-by: Andreas Fischer + Fixes #6200 + Closes #6201 + +- httpput-postfields.c: new example doing PUT with POSTFIELDS - Closes #5922 + Proposed-by: Jeroen Ooms + Ref: #6186 + Closes #6188 -- tests: remove pipelining tests +- [Tobias Hieta brought this change] + + cmake: correctly handle linker flags for static libs - Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were - previously disabled. + curl CMake was setting the the EXE flags for static libraries which made + the /manifest:no flag ended up when linking the static library, which is + not a valid flag for lib.exe or llvm-lib.exe and caused llvm-lib to exit + with an error. - The Pipelining code was removed from curl in commit 2f44e94efb3df8e, - April 2019. + The better way to handle this is to make sure that we pass the correct + linker flags to CMAKE_STATIC_LINKER_FLAGS instead. - Closes #5921 + Reviewed-by: Jakub Zakrzewski + Closes #6195 -- curl: retry delays in parallel mode no longer sleeps blocking - - The previous sleep for retries would block all other concurrent - transfers. Starting now, the retry will instead be properly marked to - not get restarted until after the delay time but other transfers can - still continue in the mean time. - - Closes #5917 +- [Tobias Hieta brought this change] -- curl:parallel_transfers: make sure retry readds the transfer + cmake: don't pass -fvisibility=hidden to clang-cl on Windows - Reported-by: htasta on github - Fixes #5905 - Closes #5917 - -- build: drop support for building with Watcom + When using clang-cl on windows -fvisibility=hidden is not an known + argument. Instead it behaves exactly like MSVC in this case. So let's + make sure we take that path. - These files are not maintained, they seem to have no users, Watcom - compilers look like not having users nor releases anymore. + In CMake clang-cl sets both CMAKE_C_COMPILER_ID=clang and MSVC get's + defined since clang-cl is basically a MSVC emulator. So guarding like we + do in this patch seems logical. - Closes #5918 + Reviewed-by: Jakub Zakrzewski + Closes #6194 -- winbuild/rundebug.cmd: remove +- http_proxy: use enum with state names for 'keepon' - Seems to have been added by mistake? Not included in dists. + To make the code clearer, change the 'keepon' from an int to an enum + with better state names. - Closes #5919 + Reported-by: Niranjan Hasabnis + Bug: https://curl.se/mail/lib-2020-11/0026.html + Closes #6193 -- curl: in retry output don't call all problems "transient" +- curl_easy_escape: limit output string length to 3 * max input - ... because when --retry-all-errors is used, the error isn't necessarily - transient at all. + ... instead of the limiting it to just the max input size. As every + input byte can be expanded to 3 output bytes, this could limit the input + string to 2.66 MB instead of the intended 8 MB. - Closes #5916 + Reported-by: Marc Schlatter + Closes #6192 -- easygetopt: pass a valid enum to avoid compiler warning +- docs: document the 8MB input string limit - "integer constant not in range of enumerated type 'CURLoption'" + for curl_easy_escape and curl_easy_setopt() - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 + The limit is there to catch mistakes and abuse. It is meant to be large + enough to allow virtually all "fine" use cases. - Closes #5915 - -- [Emil Engler brought this change] + Reported-by: Marc Schlatter + Fixes #6190 + Closes #6191 - tests: Add tests for new --help - - This commit is a part of "--help me if you can" +- mqttd: fclose test file when done - Closes #5680 - -- [Emil Engler brought this change] + Reported-by: Marc Hörsken + Reviewed-by: Jay Satiro + Bug: #6058 + Closes #6189 - tool: update --help with categories - - This commit is a part of "--help me if you can" - - Closes #5680 +- RELEASE-NOTES: synced -- [Emil Engler brought this change] +- THANKS-filter: ignore autobuild links - docs: add categories to all cmdline opts +- Revert "libcurl.pc: make it relocatable" - Adapted gen.pl with 'listcats' + This reverts commit 3862c37b6373a55ca704171d45ba5ee91dec2c9f. - This commit is a part of "--help me if you can" + That fix should either be done differently or with an option. - Closes #5680 + Reported-by: asavah on github + Fixes #6157 + Closes #6183 -- RELEASE-NOTES: synced +- examples/httpput: remove use of CURLOPT_PUT + + It is deprecated and unnecessary since it already sets CURLOPT_UPLOAD. + + Reported-by: Jeroen Ooms + Fixes #6186 + Closes #6187 -- [ihsinme brought this change] +- Curl_pgrsStartNow: init speed limit time stamps at start + + By setting the speed limit time stamps unconditionally at transfer + start, we can start off a transfer without speed limits and yet allow + them to get set during transfer and have an effect. + + Reported-by: Kael1117 on github + Fixes #6162 + Closes #6184 - connect.c: remove superfluous 'else' in Curl_getconnectinfo +- ngtcp2: adapt to recent nghttp3 updates - Closes #5912 + 'reset_stream' was added to the nghttp3_conn_callbacks struct + + Closes #6185 -- [Samuel Marks brought this change] +- configure: pass -pthread to Libs.private for pkg-config + + Reported-by: Cristian Morales Vega + Fixes #6168 + Closes #6181 - CMake: remove explicit `CMAKE_ANSI_CFLAGS` +- altsvc: minimize variable scope and avoid "DEAD_STORE" - This variable was removed from cmake in commit - https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later - CMake commit removes the variable from the tests, claiming that it was - removed in CMake 2.6 + Closes #6182 + +- FAQ: remove "Why is there a HTTP/1.1 in my HTTP/2 request?" - Reviewed-By: Peter Wu - Closes #5439 + This hasn't been the case for a while now, remove. -- [cbe brought this change] +- FAQ: refresh "Why do I get "certificate verify failed" + + Add more details, remove references to ancient curl version. - libssh2: pass on the error from ssh_force_knownhost_key_type +- test493: verify --hsts upgrade and that %{url_effective} reflects that - Closes #5909 + Closes #6175 -- scripts/delta: add diffstat summary +- url: make sure an HSTS upgrade updates URL and scheme correctly - ... and make output more table-like + Closes #6175 -- [Martin Bašti brought this change] +- tool_operate: set HSTS with CURLOPT_HSTS to pass on filename + + Closes #6175 - http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set +- hsts: remove debug code leftovers - ... in case NO_PROXY takes an effect + Closes #6175 + +- FAQ: refreshed - Without this patch, the following command crashes: + - remove a few ancient questions + - add configure with static libs question + - updated wording in several places + - lowercased curl - $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ - git clone https://github.com/curl/curl.git + Closes #6177 + +Daniel Gustafsson (5 Nov 2020) +- examples: fix comment syntax - Minimal libcurl-based reproducer: + Commit ac0a88fd2 accidentally added a stray character outside of the + comment which broke compilation. Fix by removing. - #include + Reported-by: autobuild https://curl.se/dev/log.cgi?id=20201105084306-12742 + +- hsts: Remove pointless call to free in errorpath - int main() { - CURL *curl = curl_easy_init(); - if(curl) { - CURLcode ret; - curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); - curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); - /* set the proxy type */ - curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); - curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); - curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); - ret = curl_easy_perform(curl); - curl_easy_cleanup(curl); - return ret; - } - return -1; - } + The line variable will always be NULL in the error path, so remove + the free call since it's pointless. - Assisted-by: Kamil Dudka - Bug: https://bugzilla.redhat.com/1873327 - Closes #5902 + Closes #6170 + Reviewed-by: Daniel Stenberg -- travis: add a CI job with openssl3 (from git master) +- docs: Fix various typos in documentation - Closes #5908 + Closes #6171 + Reviewed-by: Daniel Stenberg -- openssl: avoid error conditions when importing native CA +Daniel Stenberg (5 Nov 2020) +- copyright: fix year ranges - The code section that is OpenSSL 3+ specific now uses the same logic as - is used in the version < 3 section. It caused a compiler error without - it. + Follow-up from 4d2f8006777 + +- HISTORY: the new domain + +- curl.se: new home - Closes #5907 + Closes #6172 -- setopt: avoid curl_ on local variable +- KNOWN_BUGS: FTPS with Schannel times out file list operation - Closes #5906 + Reported-by: bobmitchell1956 on github + Closes #5284 -- mqtt.c: avoid curl_ prefix on local variable +- KNOWN_BUGS: SMB tests fail with Python 2 - Closes #5906 + Reported-by: Jay Satiro + Closes #5983 -- wildcard: strip "curl_" prefix from private symbols +- KNOWN_BUGS: LDAPS with NSS is slow - Closes #5906 + Reported-by: nosajsnikta on github + Closes #5874 -- vtls: make it 'struct Curl_ssl_session' +Sergei Nikulov (4 Nov 2020) +- travis: use ninja-build for CMake builds - Use uppercase C for internal symbols. + Added package ninja-build to environment + Use ninja to speed up CMake builds - Closes #5906 + Closes #6077 -- curl_threads: make it 'struct Curl_actual_call' - - Internal names should not be prefixed "curl_" +Daniel Stenberg (4 Nov 2020) +- [Harry Sintonen brought this change] + + rtsp: error out on empty Session ID, unified the code + +- [Harry Sintonen brought this change] + + rtsp: fixed the RTST Session ID mismatch in test 570 - Closes #5906 + Closes #6161 -- schannel: make it 'struct Curl_schannel*' +- [Harry Sintonen brought this change] + + rtsp: fixed Session ID comparison to refuse prefix - As internal global names should use captical C. + Closes #6161 + +- RELEASE-NOTES: synced - Closes #5906 + (forgot to update the list of contributors) -- hash: make it 'struct Curl_hash' +- RELEASE-NOTES: synced + +- curlver: bumped to 7.74.0 + +- hsts: add read/write callbacks - As internal global names should use captical C. + - read/write callback options + - man pages for the 4 new setopts + - test 1915 verifies the callbacks - Closes #5906 + Closes #5896 -- llist: make it "struct Curl_llist" +- hsts: add support for Strict-Transport-Security - As internal global names should use captical C. + - enable in the build (configure) + - header parsing + - host name lookup + - unit tests for the above + - CI build + - CURL_VERSION_HSTS bit + - curl_version_info support + - curl -V output + - curl-config --features + - CURLOPT_HSTS_CTRL + - man page for CURLOPT_HSTS_CTRL + - curl --hsts (sets CURLOPT_HSTS_CTRL and works with --libcurl) + - man page for --hsts + - save cache to disk + - load cache from disk + - CURLOPT_HSTS + - man page for CURLOPT_HSTS + - added docs/HSTS.md + - fixed --version docs + - adjusted curl_easy_duphandle - Closes #5906 + Closes #5896 -Marc Hoersken (2 Sep 2020) -- telnet.c: depend on static requirement of WinSock version 2 +- [Sergei Nikulov brought this change] + + CI/tests: enable test target on TravisCI for CMake builds - Drop dynamic loading of ws2_32.dll and instead rely on the - imported version which is now required to be at least 2.2. + Added test-nonflaky target to CMake builds - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Reviewed-by: Viktor Szakats + Disabled test 1139 because the cmake build doesn't create docs/curl.1 - Closes #5854 + Closes #6074 -- win32: drop support for WinSock version 1, require version 2 - - IPv6, telnet and now also the multi API require WinSock - version 2 which is available starting with Windows 95. - - Therefore we think it is time to drop support for version 1. - - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Reviewed-by: Viktor Szakats +- tool_debug_cb: do not assume zero-terminated data - Follow up to #5634 - Closes #5854 + Follow-up to d70a5b5a0f5e3 -- select: align poll emulation to return all relevant events +- sendf: move the verbose-check into Curl_debug - The poll emulation via select already consumes POLLRDNORM, - POLLWRNORM and POLLRDBAND as input events. Therefore it - should also return them as output events if signaled. + Saves us from having the same check done everywhere. - Also fix indentation in input event handling block. + Closes #6159 + +- travis: use valgrind when running tests for debug builds - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Except the non-x86 and sanitizer builds - Replaces #5852 - Closes #5883 + Closes #6154 -- CI/azure: MQTT is now enabled by default - - Reviewed-by: Daniel Stenberg +- header.d: fix syntax mistake - Follow up to #5858 - Closes #5903 + follow-up from 1144886f38fd0 -Daniel Stenberg (2 Sep 2020) -- copyright.pl: ignore buildconf +- [Harry Sintonen brought this change] -- test971: show test mismatches "inline" + gnutls: fix memory leaks (certfields memory wasn't released) + + Closes #6153 -- lib/Makefile.am: bump VERSIONINFO due to new functions +- tests: add missing global_init/cleanup calls - ... we're generally bad at this, but we are adding new functions for - this release. + Without the cleanup call in these test files, the mbedTLS backend leaks + memory. - Closes #5899 + Closes #6156 -- optiontable: use DEBUGBUILD +- tool_operate: --retry for HTTP 408 responses too - Follow-up to commit 6e18568ba38 (#5877) - -- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 + This was inadvertently dropped from the code when the parallel support + was added. - If there are more than two items in the list, use commas for all but the - last separator which is set to 'and'. Reads better. + Regression since b88940850 (7.66.0) - Closes #5898 + Reviewed-by: Jay Satiro + Closes #6155 -- curl.1: add see also no-progress-meter on two spots +- http: pass correct header size to debug callback for chunked post - Ref: #5894 + ... when the chunked framing was added, the size of the "body part" of + the data was calculated wrongly so the debug callback would get told a + header chunk a few bytes too big that would also contain the first few + bytes of the request body. - Closes #5897 + Reported-by: Dirk Wetter + Ref: #6144 + Closes #6147 -- RELEASE-NOTES: synced +- header.d: mention the "Transfer-Encoding: chunked" handling + + Ref: #6144 + Closes #6148 -- mqtt: enable by default +- acinclude: detect manually set minimum macos/ipod version - No longer considered experimental. + ... even if set in the CC or IPHONEOS/MACOSX_DEPLOYMENT_TARGET + variables. - Closes #5858 - -- [Michael Baentsch brought this change] + Reported-by: hamstergene on github + Fixes #6138 + Closes #6140 - tls: add CURLOPT_SSL_EC_CURVES and --curves +Jay Satiro (29 Oct 2020) +- tests: fix some http/2 tests for older versions of nghttpx - Closes #5892 + - Add regex that strips http/2 server header name to those http/2 tests + that don't already have it. + + - Improve that regex in all http/2 tests. + + Tests 358 and 359 were failing for me before this change on a system + that uses an older version of nghttpx which includes its version number + in the server header. + + Closes https://github.com/curl/curl/pull/6139 -- url: remove funny embedded comments in Curl_disonnect calls +Daniel Stenberg (30 Oct 2020) +- RELEASE-NOTES: synced -- [Chris Paulson-Ellis brought this change] +- [Cristian Morales Vega brought this change] - conn: check for connection being dead before reuse - - Prevents incorrect reuse of an HTTP connection that has been prematurely - shutdown() by the server. + configure: use pkgconfig to find openSSL when cross-compiling - Partial revert of 755083d00deb16 + This reverts 736a40fec (November 2004), which doesn't explain why it was + done. - Fixes #5884 - Closes #5893 + Closes #6145 -Marc Hoersken (29 Aug 2020) -- buildconf: exec autoreconf to avoid additional process - - Also make buildconf exit with the return code of autoreconf. +- tool_operate: bail out proper on errors for parallel setup - Reviewed-by: Daniel Stenberg + ... otherwise for example trying to upload a missing file just causes a + loop. - Follow up to #5853 - Closes #5890 + Reported-by: BrumBrum on hackerone + Closes #6141 -- CI/azure: no longer ignore results of test 1013 - - Follow up to #5771 - Closes #5889 +- [Sergei Nikulov brought this change] -- docs: add description about CI platforms to CONTRIBUTE.md + CMake: make BUILD_TESTING dependent option - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Reviewed-by: Jay Satiro + CMake will now handle BUILD_TESTING depending on PERL_FOUND and + CURL_DISABLE_TESTING - Closes #5882 + Ref: #6036 + Closes #6072 -Daniel Stenberg (29 Aug 2020) -- tests/getpart: use MIME::Base64 instead of home-cooked - - Since we already use the base64 package since a while back, we can just - as well switch to that here too. +- libssh2: fix transport over HTTPS proxy - It also happens to use the exact same function name, which otherwise - causes a run-time warning. + The fix in #6021 was not enough. This fix makes sure SCP/SFTP content + can also be transfered over a HTTPS proxy. - Reported-by: Marc Hörsken - Fixes #5885 - Closes #5887 + Fixes #6113 + Closes #6128 -Marcel Raad (29 Aug 2020) -- ntlm: fix condition for curl_ntlm_core usage +- curl.1: add an "OUTPUT" section at the top of the manpage - `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES - backend is fine, but was excluded before. + Explain the basic concepts behind curl output. - This also fixes test 1013 as the condition for SMB support in - configure.ac didn't match the condition in the source code. Now it - does. + Inspired by #6124 - Fixes https://github.com/curl/curl/issues/1262 - Closes https://github.com/curl/curl/pull/5771 + Closes #6134 -- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode +- mailmap: set Viktor Szakats's email + +- runtests: show keywords when no tests ran - The Schannel builds are the most useful to verify as they make the most - use of the Windows API. Classic MinGW doesn't support Unicode at all, - only MinGW-w64 and MSVC do. + To help out future debugging, runtests now outputs the list of keywords + when it fails because no tests ran. - Closes https://github.com/curl/curl/pull/5843 + Ref: #6120 + Closes #6126 -- CMake: add option to enable Unicode on Windows +Jay Satiro (26 Oct 2020) +- CURLOPT_DNS_USE_GLOBAL_CACHE.3: fix typo - As already existing for winbuild. + Reported-by: Rui LIU - Closes https://github.com/curl/curl/pull/5843 + Closes https://github.com/curl/curl/issues/6131 -Marc Hoersken (29 Aug 2020) -- select: simplify return code handling for poll and select +- range.d: fix typo - poll and select already return -1 on error according to POSIX, - so there is no need to perform a <0 to -1 conversion in code. + Follow-up to 15ae039 from earlier today. + +Daniel Stenberg (26 Oct 2020) +- CI/github: work-around for brew breakage on macOS - Also we can just use one check with <= 0 on the return code. + ... and make it use OpenSSL 1.1 properly - Assisted-by: Daniel Stenberg - Reviewed-by: Jay Satiro + Fixes #6130 + Closes #6129 + +- [José Joaquín Atria brought this change] + + range.d: clarify that curl will not parse multipart responses - Replaces #5852 - Closes #5880 + Closes #6127 + Fixes #6124 -Daniel Stenberg (28 Aug 2020) - RELEASE-NOTES: synced -- [Jeroen Ooms brought this change] +- [Baruch Siach brought this change] - tests: add test1912 with typechecks - - Validates that gcc-typecheck macros match the new option type API. + libssh2: fix build with disabled proxy support - Closes #5873 - -- easyoptions: provide debug function when DEBUGBUILD + Build breaks because the http_proxy field is missing: - ... not CURLDEBUG as they're not always set in conjunction. + vssh/libssh2.c:3119:10: error: 'struct connectdata' has no member named 'http_proxy' - Follow-up to 6ebe63fac23f38df + Regression from #6021, shipped in curl 7.73.0 - Fixes #5877 - Closes #5878 + Closes #6125 -Marc Hoersken (28 Aug 2020) -- sockfilt: handle FD_CLOSE winsock event on write socket +- alt-svc: enable by default - Learn from the way Cygwin handles and maps the WinSock events - to simulate correct and complete poll and select behaviour - according to Richard W. Stevens Network Programming book. + Remove CURLALTSVC_IMMEDIATELY, which was never implemented/supported. - Follow up to #5867 - Closes #5879 - -- multi: handle connection state winsock events + alt-svc support in curl is no longer considered experimental - Learn from the way Cygwin handles and maps the WinSock events - to simulate correct and complete poll and select behaviour - according to Richard W. Stevens Network Programming book. + Closes #5868 + +- CI/appveyor: remove (unused) runtests.pl -b option + +- [Emil Engler brought this change] + + tool_help: make "output" description less confusing - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad + Currently the description of "output" is misleading when comparing it + "verbose". - Follow up to #5634 - Closes #5867 + Closes #6118 -Daniel Stenberg (28 Aug 2020) -- Curl_pgrsTime - return new time to avoid timeout integer overflow - - Setting a timeout to INT_MAX could cause an immediate error to get - returned as timeout because of an overflow when different values of - 'now' were used. +- CI/appveyor: disable test 571 in two cmake builds - This is primarily fixed by having Curl_pgrsTime() return the "now" when - TIMER_STARTSINGLE is set so that the parent function will continue using - that time. + ... they're simply too flaky there. - Reported-by: Ionuț-Francisc Oancea - Fixes #5583 - Closes #5847 + Closes #6119 -- TLS: fix SRP detection by using the proper #ifdefs - - USE_TLS_SRP will be true if *any* selected TLS backend can use SRP +- cmake: set the unicode feature in curl-config on Windows - HAVE_OPENSSL_SRP is defined when OpenSSL can use it + ... if built that way. To make it match curl -V output. - HAVE_GNUTLS_SRP is defined when GnuTLS can use it + Reviewed-by: Marcel Raad + Closes #6117 + +- libssh2: require version 1.0 or later - Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is - set if at least one of the supported backends offers SRP. + ... and simplify the code accordingly. libssh2 version 1.0 was released + in April 2009. - Reported-by: Stefan Strogin - Fixes #5865 - Closes #5870 + Closes #6116 -- [Dan Kenigsberg brought this change] +- KNOWN_BUGS: mention the individual cmake issues + + ... to make them easier to refer to and address separately and + one-by-one. - docs: SSLCERTS: fix English syntax +- CMake: store IDN2 information in curl_config.h - Signed-off-by: Dan Kenigsberg + This allows the build to enable IDN properly and it makes test 1014 + happier. - Closes #5876 - -- [Alessandro Ghedini brought this change] + Ref: #6074 + Closes #6108 - docs: non-existing macros in man pages - - As reported by man(1) when invoked as: +- CMake: call the feature unixsockets without dash - man --warnings -E UTF-8 -l -Tutf8 -Z >/dev/null + ... so that curl-config gets correct and makes test 1014 happy! - Closes #5846 - -- [Alessandro Ghedini brought this change] + Ref: #6074 + Closes #6108 - curl.1: fix typo invokved -> invoked +- CI/travis: add brotli and zstd to the libssh2 build - Closes #5846 - -- buildconf: invoke 'autoreconf -fi' instead + ... to make sure such tests are run with valgrind. Suppress the zstd + valgrind warnings we get with version 1.3.3 on Ubuntu 18.04 (for debug + and non-debug builds). - The custom script isn't necessary anymore - but remains for simplicity - and just invokes autoreconf. + Closes #6105 + +- runtests: revert the mistaken edit of $CURL - Closes #5853 + Regression from c4693adc62 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - lib: make Curl_gethostname accept a const pointer - - The address of that variable never gets changed, only the data in it so - why not make it a "char * const"? +- curl_url_set.3: fix typo in the RETURN VALUE section - Closes #5866 + Reported-by: Basuke Suzuki + Fixes #6102 -- docs/libcurl: update "Added in" version for curl_easy_option* - - Follow-up to 6ebe63fac23f38 +Jay Satiro (17 Oct 2020) +- [Daniel Stenberg brought this change] -- scripts: improve the "get latest curl release tag" logic + packages/OS400: make the source code-style compliant - ... by insiting on it matching "^curl-". + ... and make sure 'make checksrc' in the root dir also verifies the + packages/OS400 sources. + + Closes https://github.com/curl/curl/pull/6085 -- configure: added --disable-get-easy-options +- os400: Sync libcurl API options - To allow disabling of the curl_easy_option APIs in a build. + This fixes the OS400 build and also an incorrect entry for + CURLINFO_APPCONNECT_TIME_T where it was treated as + CURLINFO_STARTTRANSFER_TIME_T. - Closes #5365 + Reported-by: Jon Rumsey + + Fixes https://github.com/curl/curl/issues/6083 + Closes https://github.com/curl/curl/pull/6084 -- options: API for meta-data about easy options +Daniel Stenberg (16 Oct 2020) +- CURLOPT_NOBODY.3: fix typo - const struct curl_easyoption *curl_easy_option_by_name(const char *name); + Reported-by: Basuke Suzuki + Fixes #6097 + +Marc Hoersken (16 Oct 2020) +- CI/azure: improve on flakiness by avoiding libtool wrappers - const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); + Install curl binaries into MinGW bin folder and use that + for the tests in order to avoid libtool wrapper binaries. - const struct curl_easyoption * - curl_easy_option_next(const struct curl_easyoption *prev); + The libtool wrapper binaries (not scripts) on Windows seem + to be one of the possible causes for the following issues: - The purpose is to provide detailed enough information to allow for - example libcurl bindings to get option information at run-time about - what easy options that exist and what arguments they expect. + 1. Process output can be lost in the wrapper process chain. + 2. Killing the wrapper process does not kill the actual one. - Assisted-by: Jeroen Ooms - Closes #5365 + Derived from #5904 + Closes #6049 -- [Eric Curtin brought this change] +Daniel Stenberg (16 Oct 2020) +- CURLOPT_URL.3: clarify SCP/SFTP URLs are for uploads as well - HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 - - Closes #5871 +- [Zenju brought this change] -- RELEASE-NOTES: synced + CURLOPT_TCP_NODELAY.3: fix comment in example code + + Closes #6096 -Jay Satiro (26 Aug 2020) -- openssl: Fix wincrypt symbols conflict with BoringSSL +- openssl: acknowledge SRP disabling in configure properly - OpenSSL undefines the conflicting symbols but BoringSSL does not so we - must do it ourselves. + Follow-up to 68a513247409 - Reported-by: Samuel Tranchet - Assisted-by: Javier Blazquez + Use a new separate define that is the combination of both + HAVE_OPENSSL_SRP and USE_TLS_SRP: USE_OPENSSL_SRP - Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 - Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 + Bug: https://curl.haxx.se/mail/lib-2020-10/0037.html - Fixes https://github.com/curl/curl/issues/5669 - Closes https://github.com/curl/curl/pull/5857 + Closes #6094 -Daniel Stenberg (26 Aug 2020) -- socketpair: allow CURL_DISABLE_SOCKETPAIR +Viktor Szakats (16 Oct 2020) +- http3: fix two build errors, silence warnings - ... to completely disable the use of socketpair + * fix two build errors due to mismatch between function + declarations and their definitions + * silence two mismatched signs warnings via casts - Closes #5850 + Approved-by: Daniel Stenberg + Closes #6093 -- curl_get_line: build only if cookies or alt-svc are enabled +- Makefile.m32: add support for HTTP/3 via ngtcp2+nghttp3 - Closes #5851 + Approved-by: Daniel Stenberg + Closes #6092 -- [fullincome brought this change] +Daniel Stenberg (16 Oct 2020) +- tool_operate: fix compiler warning when --libcurl is disabled + + Closes #6095 - schannel: fix memory leak when using get_cert_location +- checksrc: warn on empty line before open brace - The get_cert_location function allocates memory only on success. - Previously get_cert_location was able to allocate memory and return - error. It wasn't obvious and in this case the memory wasn't - released. + ... and fix a few occurances - Fixes #5855 - Closes #5860 - -- [Emil Engler brought this change] + Closes #6088 - git: ignore libtests in 3XXX area +- urlapi: URL encode a '+' in the query part - Currently the file tests/libtest/lib3010 is not getting - ignored by git. This fixes it by adding the 3XXX area to - the according .gitignore file. + ... when asked to with CURLU_URLENCODE. - Closes #5859 + Extended test 1560 to verify. + Reported-by: Dietmar Hauser + Fixes #6086 + Closes #6087 -- [Emil Engler brought this change] +- [Cristian Morales Vega brought this change] - doh: add error message for DOH_DNS_NAME_TOO_LONG + libcurl.pc: make it relocatable - When this error code was introduced in b6a53fff6c1d07e8a9, it was - forgotten to be added in the errors array and doh_strerror function. + It supposes when people specify the libdir/includedir they do it to + change where under prefix/exec_prefix it should be, not to make it + independent of prefix/exec_prefix. - Closes #5863 + Closes #6061 -- ngtcp2: adapt to the new pkt_info arguments +- runtests: return error if no tests ran - Guidance-by: Tatsuhiro Tsujikawa + ... and make TESTFAIL stand out a little better by adding newlines + before and after. - Closes #5864 + Reported-by: Marc Hörsken + Issue: #6052 + Closes #6053 -- winbuild/README.md: make visible +- docs/FEATURE: convert to markdown - Follow-up to be753add31c2d8c - -- winbuild: convert the instruction text to README.md + ... and clean it up a bit. - Closes #5861 + Closes #6067 -- lib1560: verify "redirect" to double-slash leading URL - - Closes #5849 +- [Philipp Klaus Krause brought this change] -Marc Hoersken (25 Aug 2020) -- multi: expand pre-check for socket readiness + strerror: use 'const' as the string should never be modified - Check readiness of all sockets before waiting on them - to avoid locking in case the one-time event FD_WRITE - was already consumed by a previous wait operation. + Closes #6068 + +- [Jay Satiro brought this change] + + connect: repair build without ipv6 availability - More information about WinSock network events: - https://docs.microsoft.com/en-us/windows/win32/api/ - winsock2/nf-winsock2-wsaeventselect#return-value + Assisted-by: Daniel Stenberg + Reported-by: Tom G. Christensen - Closes #5634 + Fixes https://github.com/curl/curl/issues/6069 + Closes https://github.com/curl/curl/pull/6071 -- [rcombs brought this change] +- RELEASE-NOTES: synced + + Started over for the journey to next release. - multi: implement wait using winsock events +- src/tool_filetime: disable -Wformat on mingw for this file - This avoids using a pair of TCP ports to provide wakeup functionality - for every multi instance on Windows, where socketpair() is emulated - using a TCP socket on loopback which could in turn lead to socket - resource exhaustion. + With gcc 10 on mingw we otherwise get this warning: - A previous version of this patch failed to account for how in WinSock, - FD_WRITE is set only once when writing becomes possible and not again - until after a send has failed due to the buffer filling. This contrasts - to how FD_READ and FD_OOB continue to be set until the conditions they - refer to no longer apply. This meant that if a user wrote some data to - a socket, but not enough data to completely fill its send buffer, then - waited on that socket to become writable, we'd erroneously stall until - their configured timeout rather than returning immediately. + error: ISO C does not support the 'I' printf flag [-Werror=format=] - This version of the patch addresses that issue by checking each socket - we're waiting on to become writable with select() before the wait, and - zeroing the timeout if it's already writable. + Fixes #6079 + Closes #6082 + +- test122[12]: remove these two tests - Assisted-by: Marc Hörsken - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg - Tested-by: Gergely Nagy - Tested-by: Rasmus Melchior Jacobsen - Tested-by: Tomas Berger + ... and remove the objnames scripts they tested. They're not used for + anything anymore so testing them serves no purpose! - Replaces #5397 - Reverts #5632 - Closes #5634 + Reported-by: Marc Hörsken + Fixes #6080 + Closes #6081 -- select: reduce duplication of Curl_poll in Curl_socket_check +Version 7.73.0 (14 Oct 2020) + +Daniel Stenberg (14 Oct 2020) +- RELEASE-NOTES: synced - Change Curl_socket_check to use select-fallback in Curl_poll - instead of implementing it in Curl_socket_check and Curl_poll. + for 7.73.0 + +- THANKS: from 7.73.0 and .mailmap fixes + +- mailmap: fixups of some contributors + +- projects/build-wolfssl.bat: fix the copyright year range + +Marc Hoersken (14 Oct 2020) +- [Sergei Nikulov brought this change] + + CI/tests: fix invocation of tests for CMake builds - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro + Update appveyor.yml to set env variable TFLAGS and run tests + Remove curly braces due to CMake error (${TFLAGS} -> $TFLAGS) + Move testdeps build to build step (per review comments) - Replaces #5262 and #5492 - Closes #5707 + Reviewed-by: Marc Hörsken + + Closes #6066 + Fixes #6052 -- select: fix poll-based check not detecting connect failure +- tests/server/util.c: fix support for Windows Unicode builds - This commit changes Curl_socket_check to use POLLPRI to - check for connect failure on the write socket, because - POLLPRI maps to fds_err. This is in line with select(2). + Detected via #6066 + Closes #6070 + +Daniel Stenberg (13 Oct 2020) +- [Jay Satiro brought this change] + + strerror: Revert to local codepage for Windows error string - The select-based socket check correctly checks for connect - failures by adding the write socket also to fds_err. + - Change get_winapi_error() to return the error string in the local + codepage instead of UTF-8 encoding. - The poll-based implementation (which internally can itself - fallback to select again) did not previously check for - connect failure by using POLLPRI with the write socket. + Two weeks ago bed5f84 fixed get_winapi_error() to work on xbox, but it + also changed the error string's encoding from local codepage to UTF-8. - See the follow up commit to this for more information. + We return the local codepage version of the error string because if it + is output to the user's terminal it will likely be with functions which + expect the local codepage (eg fprintf, failf, infof). - This commit makes sure connect failures can be detected - and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. + This is essentially a partial revert of bed5f84. The support for xbox + remains but the error string is reverted back to local codepage. - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro + Ref: https://github.com/curl/curl/pull/6005 - Replaces #5509 - Prepares #5707 + Reviewed-by: Marcel Raad + Closes #6065 -- select.h: make socket validation macros test for INVALID_SOCKET +Marc Hoersken (13 Oct 2020) +- CI/tests: use verification curl for test reporting APIs - With Winsock the valid range is [0..INVALID_SOCKET-1] according to - https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 + Avoid using our own, potentially installed, curl for + the test reporting APIs in case it is broken. - Reviewed-by: Jay Satiro - Reviewed-by: Marcel Raad Reviewed-by: Daniel Stenberg - Closes #5760 + Preparation for #6049 + Closes #6063 -Daniel Stenberg (24 Aug 2020) -- docs: --output-dir is added in 7.73.0, nothing else +Viktor Szakats (12 Oct 2020) +- windows: fix comparison of mismatched types warning - Follow-up to 5620d2cc78c0 + clang 10, mingw-w64: + ``` + vtls/openssl.c:2917:33: warning: comparison of integers of different signs: 'DWORD' (aka 'unsigned long') and 'HRESULT' (aka 'long') + [-Wsign-compare] + if(GetLastError() != CRYPT_E_NOT_FOUND) + ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~ + ``` + + Approved-by: Daniel Stenberg + Closes #6062 -- curl: add --output-dir +Daniel Stenberg (11 Oct 2020) +- [Viktor Szakats brought this change] + + src/Makefile.m32: fix undefined curlx_dyn_* errors - Works with --create-dirs and with -J + by linking `lib/dynbuf.c` when building a static curl binary. + Previously this source file was only included when building + a dynamic curl binary. This was likely possibly because no + functions from the `src/Makefile.inc` / `CURLX_CFILES` sources + were actually required for a curl tool build. This has + recently changed with the introduction of `curlx_dyn_*()` + memory functions and their use by the tool sources. - Add test 3008, 3009, 3011, 3012 and 3013 to verify. + Closes #6060 + +- HISTORY: curl verifies SSL certs by default since version 7.10 + +Marc Hoersken (8 Oct 2020) +- runtests.pl: use $LIBDIR variable instead of hardcoded path - Closes #5637 + Reviewed-by: Daniel Stenberg + Closes #6051 -- configure: fix pkg-config detecting wolfssl +Daniel Stenberg (7 Oct 2020) +- checksrc: detect // comments on column 0 - When amending the include path with "/wolfssl", this now properly strips - off all whitespace from the path variable! Previously this would lead to - pkg-config builds creating bad command lines. + Spotted while working on #6045 - Closes #5848 + Closes #6048 -- [Michael Musset brought this change] +- [Frederik Wedel-Heinen brought this change] - sftp: add the option CURLKHSTAT_FINE_REPLACE + mbedtls: add missing header when defining MBEDTLS_DEBUG - Replace the old fingerprint of the host with a new. + Closes #6045 + +- curl: make sure setopt CURLOPT_IPRESOLVE passes on a long - Closes #5685 + Previously, it would pass on a define (int) which could make libcurl + read junk as a value - which prevented the CURLOPT_IPRESOLVE option to + "take". This could then make test 2100 do two DoH requests instead of + one! + + Fixes #6042 + Closes #6043 - RELEASE-NOTES: synced - - The next release is now to become 7.73.0 -- checksrc: verify do-while and spaces between the braces - - Updated mprintf.c to comply +- scripts/release-notes.pl: don't "embed" $ in format string for printf() - Closes #5845 + ... since they might contain %-codes that mess up the output! -- curl: support XDG_CONFIG_HOME to find .curlrc - - Added test433 to verify. Updated documentation. - - Reviewed-by: Jay Satiro - Suggested-by: Eli Schwartz - Fixes #5829 - Closes #5837 +Jay Satiro (5 Oct 2020) +- [M.R.T brought this change] -- etag: save and use the full received contents + build-wolfssl: fix build with Visual Studio 2019 - ... which makes it support weak tags and non-standard etags too! + Closes https://github.com/curl/curl/pull/6033 + +Daniel Stenberg (4 Oct 2020) +- runtests: add %repeat[]% for test files - Added test case 347 to verify blank incoming ETag: + ... and use this new keywords in all the test files larger than 50K to reduce + their sizes and make them a lot easier to read and understand. - Fixes #5610 - Closes #5833 + Closes #6040 -- setopt: if the buffer exists, refuse the new BUFFERSIZE +- [Emil Engler brought this change] + + --help: move two options from the misc category - The buffer only exists during transfer and then we shouldn't change the - size (the setopt is not documented to work then). + The cmdline opts delegation and suppress-connect-headers + fit better into auth and proxy rather than misc. - Reported-by: Harry Sintonen - Closes #5842 + Follow-up to aa8777f63febc + Closes #6038 -- [COFFEETALES brought this change] +- [Samanta Navarro brought this change] - sftp: add new quote commands 'atime' and 'mtime' + docs/opts: fix typos in two manual pages - Closes #5810 + Closes #6039 -- CURLE_PROXY: new error code +- ldap: reduce the amount of #ifdefs needed - Failures clearly returned from a (SOCKS) proxy now causes this return - code. Previously the situation was not very clear as what would be - returned and when. + Closes #6035 + +- runtests: provide curl's version string as %VERSION for tests - In addition: when this error code is returned, an application can use - CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then - returns a value from the new 'CURLproxycode' enum. + ... so that we can check HTTP requests for User-Agent: curl/%VERSION - Closes #5770 + Update 600+ test cases accordingly. + + Closes #6037 -- runtests: make cleardir() erase dot files too +- checksrc: warn on space after exclamation mark - Because test cases might use dot files. + Closes #6034 + +- test1465: verify --libcurl with binary POST data + +- runtests: allow generating a binary sequence from hex + +- tool_setopt: escape binary data to hex, not octal + +- curl: make --libcurl show binary posts correctly - Closes #5838 + Reported-by: Stephan Mühlstrasser + Fixes #6031 + Closes #6032 -- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses +Jay Satiro (1 Oct 2020) +- strerror: fix null deref on winapi out-of-memory - Also: the current behavior is now documented in the curl.1 and - CURLOPT_NOPROXY.3 man pages. + Follow-up to bed5f84 from several days ago. - Reported-by: Andrew Barnes - Closes #5745 - Closes #5841 + Ref: https://github.com/curl/curl/pull/6005 -Viktor Szakats (22 Aug 2020) -- Makefile.m32: add ability to override zstd libs [ci skip] - - Similarly to brotli, where this was already possible. - E.g. it allows to link zstd statically to libcurl.dll. +Daniel Stenberg (1 Oct 2020) +- [Kamil Dudka brought this change] + + vtls: deduplicate some DISABLE_PROXY ifdefs - Ref: https://github.com/curl/curl-for-win/issues/12 - Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 + ... in the code of gtls, nss, and openssl - Closes https://github.com/curl/curl/pull/5840 + Closes #5735 -Daniel Stenberg (21 Aug 2020) -- runtests: avoid 'fail to start' repeated messages in attempt loops +- RELEASE-NOTES: synced + +- [Emil Engler brought this change] + + TODO: Add OpenBSD libtool notice - Closes #5834 + See #5862 + Closes #6030 -- runtests: clear pid variables when failing to start a server +- tests/unit/README: convert to markdown - ... as otherwise the parent doesn't detect the failure and believe it - actually worked to start. + ... and add to dist! - Reported-by: Christian Weisgerber - Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html - Closes #5834 + Closes #6028 -- TODO: Virtual external sockets +- tests/README: convert to markdown - Closes #5835 + Closes #6028 -- [Don J Olmstead brought this change] +- include/README: convert to markdown + + Closes #6028 - dist: add missing CMake Find modules to the distribution +- examples/README: convert to markdown - Closes #5836 + Closes #6028 -- RELEASE-NOTES: synced +- configure: don't say HTTPS-proxy is enabled when disabled! - ... and version bumped to 7.72.1 + Reported-by: Kamil Dudka + Reviewed-by: Kamil Dudka + Bug: https://github.com/curl/curl/pull/5735#issuecomment-701376388 + Closes #6029 -- tls: provide the CApath verbose log on its own line +Daniel Gustafsson (30 Sep 2020) +- src: Consistently spell whitespace without whitespace - ... not newline separated from the previous line. This makes it output - asterisk prefixed properly like other verbose putput! + Whitespace is spelled without a space between white and space, so + make sure to consistently spell it that way across the codebase. - Reported-by: jmdavitt on github - Fixes #5826 - Closes #5827 - -Version 7.72.0 (19 Aug 2020) + Closes #6023 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -Daniel Stenberg (19 Aug 2020) -- RELEASE-NOTES: synced +- MANUAL: update examples to resolve without redirects - The curl 7.72.0 release + www.netscape.com is redirecting to a cookie consent form on Aol, and + cool.haxx.se isn't responding to FTP anymore. Replace with examples + that resolves in case users try out the commands when reading the + manual. + + Closes #6024 + Reviewed-by: Daniel Stenberg + Reviewed-by: Emil Engler -- THANKS: add names from curl 7.72.0 release +Daniel Stenberg (30 Sep 2020) +- HISTORY: add some 2020 events -Jay Satiro (18 Aug 2020) -- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions - - Reported-by: plujon@users.noreply.github.com +- sectransp: make it build with --disable-proxy - Closes https://github.com/curl/curl/issues/5488 + Follow-up from #5466 and f3d501dc678d80 + Reported-by: Javier Navarro + Fixes #6025 + Closes #6026 -Daniel Stenberg (17 Aug 2020) -- Curl_easy: remember last connection by id, not by pointer +- ECH: renamed from ESNI in docs and configure - CVE-2020-8231 + Encrypted Client Hello (ECH) is the current name. - Bug: https://curl.haxx.se/docs/CVE-2020-8231.html + Closes #6022 + +- configure: use "no" instead of "disabled" for the end summary - Reported-by: Marc Aldorasi - Closes #5824 + ... for consistency but also to make them more distinctly stand out next + to the "enabled" lines. -- examples/rtsp.c: correct the copyright year +- TODO: SSH over HTTPS proxy with more backends + + ... as right now only the libssh2 backend supports it. -- RELEASE-PROCEDURE.md: add more future release dates +- libssh2: handle the SSH protocols done over HTTPS proxy + + Reported-by: Robin Douine + Fixes #4295 + Closes #6021 -- [H3RSKO brought this change] +- [Emil Engler brought this change] - docs: change "web site" to "website" - - According to wikipedia: + memdebug: remove 9 year old unused debug function - While "web site" was the original spelling, this variant has become - rarely used, and "website" has become the standard spelling + There used to be a way to have memdebug fill allocated memory. 9 years + later this has no value there (valgrind and ASAN etc are way better). If + people need to know about it they can have a look at VCS logs. - Closes #5822 + Closes #5973 -- [Bevan Weiss brought this change] +- sendf: move Curl_sendf to dict.c and make it static + + ... as the only remaining user of that function. Also fix gopher.c to + instead use Curl_write() + + Closes #6020 - CMake: don't complain about missing nroff +- ROADMAP: updates and cleanups - The curl_nroff_check() was always being called, and complaining if - *NROFF wasn't found, even when not making the manual. + Fix the HSTS PR - Only check for nroff (and complain) if actually making the manual + Remove DoT, thread-safe init and hard-coded localhost. I feel very + little interest for these with users so I downgrade them to plain "TODO" + entries again. + +- schannel: return CURLE_PEER_FAILED_VERIFICATION for untrusted root - Closes #5817 + This matches what is returned in other TLS backends in the same + situation. + + Reviewed-by: Jay Satiro + Reviewed-by: Emil Engler + Follow-up to 5a3efb1 + Reported-by: iammrtau on github + Fixes #6003 + Closes #6018 -- [Brian Inglis brought this change] +- RELEASE-NOTES: synced - libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin - - copy the LDFLAGS approach for adding same option with `libhostname` in - `libtest/Makefile.am`: +- ftp: make a 552 response return CURLE_REMOTE_DISK_FULL - - init `libstubgss_la_LDFLAGS_EXTRA` variable, - - add option to variable inside conditional, - - use variable in `libstubgss_la_LDFLAGS` + Added test 348 to verify. Added a 'STOR' command to the test FTP + server to enable test 348. Documented the command in FILEFORMAT.md - Fixes #5819 - Closes #5820 + Reported-by: Duncan Wilcox + Fixes #6016 + Closes #6017 -- docs: clarify MAX_SEND/RECV_SPEED functionality +- pause: only trigger a reread if the unpause sticks - ... in particular what happens if the maximum speed limit is set to a - value that's smaller than the transfer buffer size in use. + As an unpause might itself get paused again and then triggering another + reread doesn't help. - Reported-by: Tomas Berger - Fixes #5788 - Closes #5813 + Follow-up from e040146f22608fd9 (shipped since 7.69.1) + + Bug: https://curl.haxx.se/mail/lib-2020-09/0081.html + Patch-by: Kunal Chandarana + Fixes #5988 + Closes #6013 -- test1140: compare stdout +- test163[12]: require http to be built-in to run - To make problems more immediately obvious when tests fail. + ... as speaking over an HTTPS proxy implies http! - Closes #5814 + Closes #6014 -- asyn-ares: correct some bad comments +- ngtcp2: adapt to new NGTCP2_PROTO_VER_MAX define - Closes #5812 + Closes #6012 -- [Emil Engler brought this change] +- [Javier Blazquez brought this change] - docs: Add video link to docs/CONTRIBUTE.md + strerror: honor Unicode API choice on Windows - Closes #5811 + Closes #6005 -- curl-config: ignore REQUIRE_LIB_DEPS in --libs output +- imap: make imap_send use dynbuf for the send buffer management - Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is - not considered for the --libs output. + Reuses the buffer and thereby reduces number of mallocs over a transfer. - Reported-by: ramsay-jones on github - Assisted-by: Brian Inglis and Ken Brown - Fixes #5793 - Closes #5808 - -- copyright: update/correct the year range on a few files - -- scripts/copyright.pl: ignore .muse files - -- [Emil Engler brought this change] + Closes #6010 - multi: Remove 10-year old out-commented code +- Curl_send: return error when pre_receive_plain can't malloc - The code hasn't been touched since 2010-08-18 + ... will probably trigger some false DEAD CODE positives on non-windows + code analyzers for the conditional code. - Closes #5805 + Closes #6011 -- KNOWN_BUGS: A shared connection cache is not thread-safe +- ftp: separate FTPS from FTP over "HTTPS proxy" - Closes #4915 - Closes #5802 - -- CONTRIBUTE: extend git commit message description + When using HTTPS proxy, SSL is used but not in the view of the FTP + protocol handler itself so separate the connection's use of SSL from the + FTP control connection's sue. - In particular how the first line works. + Reported-by: Mingtao Yang + Fixes #5523 + Closes #6006 + +Dan Fandrich (23 Sep 2020) +- tests/data: Fix some mismatched XML tags in test cases - Closes #5803 + This allows these test files to pass xmllint. -- RELEASE-NOTES: synced +Daniel Stenberg (23 Sep 2020) +- pingpong: use a dynbuf for the *_pp_sendf() function + + ... reuses the same dynamic buffer instead of doing repeated malloc/free + cycles. + + Test case 100 (FTP dir list PASV) does 7 fewer memory allocation calls + after this change in my test setup (132 => 125), curl 7.72.0 needed 140 + calls for this. + + Test case 103 makes 9 less allocations now (130). Down from 149 in + 7.72.0. + + Closes #6004 -- [Stefan Yohansson brought this change] +- dynbuf: add Curl_dyn_vaddf + + Closes #6004 - transfer: move retrycount from connect struct to easy handle +- dynbuf: make *addf() not require extra mallocs - This flag was applied to the connection struct that is released on - retry. These changes move the retry counter into Curl_easy struct that - lives across retries and retains the new connection. + ... by introducing a printf() function that appends directly into a + dynbuf: Curl_dyn_vprintf(). This avoids the mandatory extra malloc so if + the buffer is already big enough it can just printf directly into it. - Reported-by: Cherish98 on github - Fixes #5794 - Closes #5800 + Since this less-malloc version requires tthe use of a library internal + printf function, we only provide this version when building libcurl and + not for the dynbuf code that is used when building the curl tool. + + Closes #5998 -- libssh2: s/ssherr/sftperr/ +- KNOWN_BUGS: Unable to use PKCS12 certificate with Secure Transport - The debug output used ssherr instead of sftperr which not only outputs - the wrong error code but also casues a warning on Windows. + Closes #5403 + +- pingpong: remove a malloc per Curl_pp_vsendf call - Follow-up to 7370b4e39f1 + This typically makes 7-9 fewer mallocs per FTP transfer. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 - Closes #5799 + Closes #5997 -- ftp: don't do ssl_shutdown instead of ssl_close - - The shutdown function is for downgrading a connection from TLS to plain, - and this is not requested here. +- symbian: drop support - Have ssl_close reset the TLS connection state. + The OS is deprecated. I see no traces of anyone having actually built + curl for Symbian after 2012. - This partially reverts commit f002c850d98d + The public headers are unmodified. - Reported-by: Rasmus Melchior Jacobsen - Reported-by: Denis Goleshchikhin - Fixes #5797 + Closes #5989 -Marc Hoersken (9 Aug 2020) -- CI/azure: fix test outcome values and use latest API version +- RELEASE-NOTES: synced + +- curl_krb5.h: rename from krb5.h - This makes sure that tests ignored or skipped are not shown - just in the category "Other", but with their correct state. + Follow-up from f4873ebd0be32cf - Closes #5796 + Turns out some older openssl installations go bananas otherwise. + Reported-by: Tom van der Woerdt + Fixes #5995 + Closes #5996 -- CI/azure: show runtime stats to investigate slowness +- test1297: verify GOT_NOTHING with http proxy tunnel + +- http_proxy: do not count proxy headers in the header bytecount - Also avoid naming conflict of TFLAGS env and tflags variables. + ... as that counter is subsequently used to detect if nothing was + returned from the peer. This made curl return CURLE_OK when it should + have returned CURLE_GOT_NOTHING. - Closes #5776 + Fixes #5992 + Reported-by: Tom van der Woerdt + Closes #5994 -Daniel Stenberg (8 Aug 2020) -- TLS naming: fix more Winssl and Darwinssl leftovers +- setopt: return CURLE_BAD_FUNCTION_ARGUMENT on bad argument - The CMake option is now called CMAKE_USE_SCHANNEL + Fixed two return code mixups. CURLE_UNKNOWN_OPTION is saved for when the + option is, yeah, not known. Clarified this in the setopt man page too. - The winbuild flag is USE_SCHANNEL + Closes #5993 + +- krb5: merged security.c and krb specific FTP functions in here - The CI jobs and build scripts only use the new names and the new name - options + These two files were always tightly connected and it was hard to + understand what went into which. This also allows us to make the + ftpsend() function static (moved from ftp.c). - Tests now require 'Schannel' (when necessary) + Removed security.c + Renamed curl_sec.h to krb5.h - Closes #5795 + Closes #5987 -- smtp_parse_address: handle blank input string properly +- Curl_handler: add 'family' to each protocol - Closes #5792 + Makes get_protocol_family() faster and it moves the knowledge about the + "families" to each protocol handler, where it belongs. + + Closes #5986 -- runtests: run the DICT server on a random port number +- parsedate: tune the date to epoch conversion - Removed support for -b (base port number) + By avoiding an unnecessary error check and the temp use of the tm + struct, the time2epoch conversion function gets a little bit faster. + When repeating test 517, the updated version is perhaps 1% faster (on + one particular build on one particular architecture). - Closes #5783 - -- RELEASE-NOTES: synced + Closes #5985 -- runtests: move the TELNET server to a dynamic port +- cmake: remove scary warning - Rename the port variable to TELNETPORT to better match the existing - pattern. + Remove the text saying - Closes #5785 - -- ngtcp2: adapt to error code rename + "the curl cmake build system is poorly maintained. Be aware" - Closes #5786 - -- runtests: move the smbserver to use a dynamic port number + ... not because anything changed just now, but to encourage users to use + it and subsequently improve it. - Closes #5782 + Closes #5984 -- runtests: run the http2 tests on a random port number - - Closes #5779 +- docs/MQTT: remove outdated paaragraphs -- gtls: survive not being able to get name/issuer +- docs/MQTT: not experimental anymore - Closes #5778 + Follow-up to e37e4468688d8f -- runtests: move the gnutls-serv tests to a dynamic port +- docs/RESOURCES: remove - Affects test 320, 321, 322 and 324. + This document is not maintained and rather than trying to refresh it, + let's kill it. A more up-to-date document with relevant RFCs is this + page on the curl website: https://curl.haxx.se/rfc/ - Closes #5778 + Closes #5980 -- runtests: support dynamicly base64 encoded sections in tests - - This allows us to make test cases to use base64 at run-time and still - use and verify information determined at run-time, such as the IMAP test - server's port number in test 842. +- docs/TheArtOfHttpScripting: convert to markdown - This change makes 12 tests run again that basically never ran since we - moved to dynamic port numbers. + Makes it easier to browse on github etc. Offers (better) links. - ftpserver.pl is adjusted to load test instructions and test number from - the preprocessed test file. + It should be noted that this document is already mostly outdated and + "Everything curl" at https://ec.haxx.se/ is a better resource and + tutorial. - FILEFORMAT.md now documents the new base64 encoding syntax. + Closes #5981 + +- BUGS: convert document to markdown - Reported-by: Marcel Raad - Fixes #5761 - Closes #5775 + Closes #5979 -- curl.1: add a few missing valid exit codes +- --help: strdup the category - 93 - 96 can be returned as well. + ... since it is converted and the original pointer is freed on Windows + unicode handling. - Closes #5777 + Follow-up to aa8777f63febc + Fixes #5977 + Closes #5978 + Reported-by: xwxbug on github -- TODO: Use multiple parallel transfers for a single download - - Closes #5774 +- CHECKSRC: document two missing warnings -- TODO: Set the modification date on an uploaded file +- RELEASE-NOTES: synced + +- ftp: avoid risk of reading uninitialized integers - Closes #5768 + If the received PASV response doesn't match the expected pattern, we + could end up reading uninitialized integers for IP address and port + number. + + Issue pointed out by muse.dev + Closes #5972 -- [Thomas M. DuBuisson brought this change] +- [Quentin Balland brought this change] - CI: Add muse CI config + easy_reset: clear retry counter - Closes #5772 - -- [Thomas M. DuBuisson brought this change] + Closes #5975 + Fixes #5974 - travis/script.sh: fix use of `-n' with unquoted envvar +- ftp: get rid of the PPSENDF macro - Shellcheck tells us "-n doesn't work with unquoted arguments. quote or - use [[ ]]." + The use of such a macro hides some of what's actually going on to the + reader and is generally disapproved of in the project. - And testing shows: + Closes #5971 + +- man pages: switch to https://example.com URLs - ``` - docker run --rm -it ubuntu bash - root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" - I ran - root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" - root@fe85ce156856:/# - ``` + Since HTTPS is "the new normal", this update changes a lot of man page + examples to use https://example.com instead of the previous "http://..." - Closes #5773 + Closes #5969 -- h2: repair trailer handling +- github: remove the duplicate "Security vulnerability" entry - The previous h2 trailer fix in 54a2b63 was wrong and caused a - regression: it cannot deal with trailers immediately when read since - they may be read off the connection by the wrong 'data' owner. + ... since github adds an entry automatically by itself. - This change reverts the logic back to gathering all trailers into a - single buffer, like before 54a2b63. + Closes #5970 + +- [Emil Engler brought this change] + + github: use new issue template feature - Reported-by: Tadej Vengust - Fixes #5663 - Closes #5769 + This helps us to avoid getting feature requests as well as security + bugs reported into the issue tracker. + + Closes #5936 -Viktor Szakats (3 Aug 2020) -- windows: disable Unix Sockets for old mingw +- [Emil Engler brought this change] + + urlapi: use more Curl_safefree - Classic mingw and 10y+ old versions of mingw-w64 don't ship with - Windows headers having the typedef necessary for Unix Sockets - support, so try detecting these environments to disable this - feature. + Closes #5968 + +Marc Hoersken (17 Sep 2020) +- multi: align WinSock mask variables in Curl_multi_wait - Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ + Also skip pre-checking sockets to set timeout_ms to 0 + after the first socket has been detected to be ready. + Reviewed-by: rcombs on github Reviewed-by: Daniel Stenberg - Fixes #5674 - Closes #5758 + Follow up to #5886 -Marcel Raad (3 Aug 2020) -- test1908: treat file as text +- multi: reuse WinSock events variable in Curl_multi_wait - Fixes the line endings on Windows. + Since the struct is quite large (1 long and 10 ints) we + declare it once at the beginning of the function instead + of multiple times inside loops to avoid stack movements. - Closes https://github.com/curl/curl/pull/5767 + Reviewed-by: Viktor Szakats + Reviewed-by: Daniel Stenberg + + Closes #5886 -- TrackMemory tests: ignore realloc and free in getenv.c +Daniel Stenberg (16 Sep 2020) +- TODO: dynamically decide to use socketpair - These are only called for WIN32. + Suggested-by: Anders Bakken - Closes https://github.com/curl/curl/pull/5767 - -Daniel Stenberg (3 Aug 2020) -- tests/FILEFORMAT.md: mention %HTTP2PORT - -- RELEASE-NOTES: synced + Closes #4829 -- tlsv1.3.d. only for TLS-using connections +- TODO: add PR reference for native IDN support on macOS - ... and rephrase that "not all" TLS backends support it. + As there was work started on this that never got completed. - Closes #5764 + Closes #5371 -- tls-max.d: this option is only for TLS-using connections +- tool_help.h: update copyright year range - Ref: #5763 - Closes #5764 - -Marcel Raad (2 Aug 2020) -- [Cameron Cawley brought this change] + Follow-up from aa8777f63febca - tool_doswin: Simplify Windows version detection +- CI/azure: disable test 571 in the msys2 builds - Closes https://github.com/curl/curl/pull/5754 - -- [Cameron Cawley brought this change] - - win32: Add Curl_verify_windows_version() to curlx + It's just too flaky there - Closes https://github.com/curl/curl/pull/5754 + Reviewed-by: Marc Hoersken + Closes #5954 -- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL +- tool_writeout: protect fputs() from NULL - This makes the tests that require the OpenSSL feature also run for - those two compatible libraries. + When the code was changed to do fputs() instead of fprintf() it got + sensitive for NULL pointers; add checks for that. - Closes https://github.com/curl/curl/pull/5762 + Follow-up from 0c1e767e83ec66 + + Closes #5963 -Daniel Stenberg (1 Aug 2020) -- multi: Condition 'extrawait' is always true +- test3015: verify stdout "as text" - Reported by Codacy. + Follow-up from 0c1e767e83e to please win32 tests - Reviewed-by: Marcel Raad - Closes #5759 + Closes #5962 -Marcel Raad (1 Aug 2020) -- openssl: fix build with LibreSSL < 2.9.1 - - `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were - introduced in LibreSSL 2.9.1 [0]. +- travis: use libressl v3.1.4 instead of master - [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef + ... as their git master seems too fragile to use (and 3.2.1 which is the + latest has a build failure). - Closes https://github.com/curl/curl/pull/5757 + Closes #5964 -Daniel Stenberg (1 Aug 2020) -- [Marc Aldorasi brought this change] +- tests/FILEFORMAT: document type=shell for - multi_remove_handle: close unused connect-only connections +- tests/FILEFORMAT: document nonewline support for - Previously any connect-only connections in a multi handle would be kept - alive until the multi handle was closed. Since these connections cannot - be re-used, they can be marked for closure when the associated easy - handle is removed from the multi handle. + The one in , that creates files. - Closes #5749 + Follow-up from b83947c8df7 -- checksrc: invoke script with -D to find .checksrc proper +- [anio brought this change] + + tool_writeout: add new writeout variable, %{num_headers} - Without the -D command line option, checksrc.pl won't know which - directory to load the ".checksrc" file from when building out of the - source tree. + This variable gives the number of headers. - Reported-by: Marcel Raad - Fixes #5715 - Closes #5755 - -- [Carlo Marcelo Arenas Belón brought this change] + Closes #5947 - buildconf: retire ares buildconf invocation +- tool_urlglob: fix compiler warning "unreachable code" - no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 - -- [Carlo Marcelo Arenas Belón brought this change] - - buildconf: excempt defunct reference to ACLOCAL_FLAGS + (On Windows builds.) - retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as - the name is generic enough that it might be in use and relied upon from - the environment. + Follow-up to 70a3b003d9 -- [Carlo Marcelo Arenas Belón brought this change] +- [Gergely Nagy brought this change] - buildconf: avoid array concatenation in die() + vtls: deduplicate client certificates in ssl_config_data - reported as error SC2145[1] by shellcheck, but not expected to cause - any behavioural differences otherwise. + Closes #5629 + +- ftp: a 550 response to SIZE returns CURLE_REMOTE_FILE_NOT_FOUND - [1] https://github.com/koalaman/shellcheck/wiki/SC2145 + This is primarily interesting for cases where CURLOPT_NOBODY is set as + previously curl would not return an error for this case. - Closes #5701 - -- travis: add ppc64le and s390x builds + MDTM getting 550 now also returns this error (it returned + CURLE_FTP_COULDNT_RETR_FILE before) in order to unify return codes for + missing files across protocols and specific FTP commands. - Closes #5752 - -Marc Hoersken (31 Jul 2020) -- connect: remove redundant message about connect failure + libcurl already returns error on a 550 as a MDTM response (when + CURLOPT_FILETIME is set). If CURLOPT_NOBODY is not set, an error would + happen subsequently anyway since the RETR command would fail. - Reviewed-by: Daniel Stenberg + Add test 1913 and 1914 to verify. Updated several tests accordingly due + to the updated SIZE behavior. - Closes #5708 + Reported-by: Tomas Berger + Fixes #5953 + Closes #5957 -- tests/sshserver.pl: fix compatibility with OpenSSH for Windows +- curl: make checkpasswd use dynbuf - Follow up to #5721 + Closes #5952 -- CI/azure: install libssh2 for use with msys2-based builds - - This enables building and running the SFTP tests. - Unfortunately OpenSSH for Windows does not support SCP (yet). - - Reviewed-by: Daniel Stenberg +- curl: make glob_match_url use dynbuf - Closes #5721 + Closes #5952 -- CI/azure: increase Windows job timeout once again - - Avoid aborted jobs due to performance issues on Azure DevOps. - - Reviewed-by: Daniel Stenberg - Reviewed-by: Jay Satiro +- curl: make file2memory use dynbuf - Closes #5738 + Closes #5952 -Jay Satiro (30 Jul 2020) -- TODO: Schannel: 'Add option to allow abrupt server closure' - - We should offer an option to allow abrupt server closures (server closes - SSL transfer without sending a known termination point such as length of - transfer or close_notify alert). Abrupt server closures are usually - because of misconfigured or very old servers. +- curl: make file2string use dynbuf - Closes https://github.com/curl/curl/issues/4427 + Closes #5952 -- url: fix CURLU and location following +- [Antarpreet Singh brought this change] + + imap: set cselect_bits to CURL_CSELECT_IN initially - Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was - incorrectly used for the location follow, resulting in infinite requests - to the original location. + ... when continuing a transfer from a FETCH response. - Reported-by: sspiri@users.noreply.github.com + When the size of the file was small enough that the entirety of the + transfer happens in a single go and schannel buffers holds the entire + data. However, it wasn't completely read in Curl_pp_readresp since a + line break was found before that could happen. So, by the time we are in + imap_state_fetch_resp - there's data in buffers that needs to be read + via Curl_read but nothing to read from the socket. After we setup a + transfer (Curl_setup_transfer), curl just waits on the socket state to + change - which doesn't happen since no new data ever comes. - Fixes https://github.com/curl/curl/issues/5709 - Closes https://github.com/curl/curl/pull/5713 + Closes #5961 -Daniel Stenberg (30 Jul 2020) - RELEASE-NOTES: synced -- [divinity76 brought this change] - - docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions +- test434: test -K use in a single line without newline - it helps make it obvious that most developers don't have to care about - the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 - years old, November 4 2009) + Closes #5946 + +- runtests: allow creating files without newlines - Closes #5744 + Closes #5946 -Jay Satiro (29 Jul 2020) -- tool_cb_wrt: fix outfile mode flags for Windows +- curl: use curlx_dynbuf for realloc when loading config files - - Use S_IREAD and S_IWRITE mode permission flags to create the file - on Windows instead of S_IRUSR, S_IWUSR, etc. + ... fixes an integer overflow at the same time. - Windows only accepts a combination of S_IREAD and S_IWRITE. It does not - acknowledge other combinations, for which it may generate an assertion. + Reported-by: ihsinme on github + Assisted-by: Jay Satiro - This is a follow-up to 81b4e99 from yesterday, which improved the - existing file check with -J. + Closes #5946 + +- dynbuf: provide curlx_ names for reuse by the curl tool - Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks - Ref: https://github.com/curl/curl/pull/5731 + Closes #5946 + +- dynbuf: make sure Curl_dyn_tail() zero terminates - Closes https://github.com/curl/curl/pull/5742 + Closes #5959 -Daniel Stenberg (28 Jul 2020) -- checksrc: ban gmtime/localtime +- tests: add test1912 to the dist - They're not thread-safe so they should not be used in libcurl code. + Follow-up to 70984ce1be4cab6c + +- docs/LICENSE-MIXING: remove - Explictly enabled when deemed necessary and in examples and tests + This document is not maintained and I feel that it doesn't provide much + value to users anymore (if it ever did). - Reviewed-by: Nicolas Sterchele - Closes #5732 + Closes #5955 -- transfer: fix data_pending for builds with both h2 and h3 enabled - - Closes #5734 +- [Laramie Leavitt brought this change] -- curl_multi_setopt: fix compiler warning "result is always false" + http: consolidate nghttp2_session_mem_recv() call paths - On systems with 32 bit long the expression is always false. Avoid - the warning. + Previously there were several locations that called + nghttp2_session_mem_recv and handled responses slightly differently. + Those have been converted to call the existing + h2_process_pending_input() function. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 - Closes #5736 - -- curl: improve the existing file check with -J + Moved the end-of-session check to h2_process_pending_input() since the + only place the end-of-session state can change is after nghttp2 + processes additional input frames. - Previously a file that isn't user-readable but is user-writable would - not be properly avoided and would get overwritten. + This will likely fix the fuzzing error. While I don't have a root cause + the out-of-bounds read seems like a use after free, so moving the + nghttp2_session_check_request_allowed() call to a location with a + guaranteed nghttp2 session seems reasonable. - Reported-by: BrumBrum on hackerone - Assisted-by: Jay Satiro - Bug: https://hackerone.com/reports/926638 - Closes #5731 + Also updated a few nghttp2 callsites to include error messages and added + a few additional error checks. + + Closes #5648 -- [Jonathan Nieder brought this change] +- HISTORY: mention alt-svc added in 2019 + + ... and make 1996 the first year subtitle - multi: update comment to say easyp list is linear +- base64: also build for pop3 and imap - Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, - 2013-08-02), the easy handle list is not circular but ends with - ->next pointing to NULL. + Follow-up to the fix in 20417a13fb8f83 - Reported-by: Masaya Suzuki - Closes #5737 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5948 -- CURLOPT_NOBODY.3: fix the syntax for referring to options +- base64: enable in build with SMTP - As test 1140 fails otherwise! + The oauth2 support is used with SMTP and it uses base64 functions. - Follow-up to e1bac81cc815 + Reported-by: Michael Olbrich + Fixes #5937 + Closes #5938 -- ngtcp2: store address in sockaddr_storage +- curl_mime_headers.3: fix the example's use of curl_slist_append - Reported-by: Tatsuhiro Tsujikawa - Closes #5733 + Reported-by: sofaboss on github + Fixes #5942 + Closes #5943 -- CURLOPT_NOBODY.3: clarify what setting to 0 means +- lib583: fix enum mixup - ... and mention that HTTP with other methods than HEAD might get a body and - there's no option available to stop that. + grrr the previous follow-up to 17fcdf6a31 was wrong + +- libtest: fix build errors - Closes #5729 + Follow-up from 17fcdf6a310d4c8076 -- setopt: unset NOBODY switches to GET if still HEAD +- lib: fix -Wassign-enum warnings - Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented - action but before 7.71.0 that used to switch back to GET and with this - change (assuming the method is still set to HEAD) this behavior is - brought back. + configure --enable-debug now enables -Wassign-enum with clang, + identifying several enum "abuses" also fixed. - Reported-by: causal-agent on github - Fixes #5725 - Closes #5728 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/879007f8118771f4896334731aaca5850a154675#commitcomment-42087553 + + Closes #5929 -- [Ehren Bendler brought this change] +- RELEASE-NOTES: synced - configure: cleanup wolfssl + pkg-config conflicts when cross compiling. +- [Diven Qi brought this change] + + url: use blank credentials when using proxy w/o username and password - Also choose a different wolfSSL function to test for NTLM support. + Fixes proxy regression brought in commit ad829b21ae (7.71.0) - Fixes #5605 - Closes #5682 + Fixed #5911 + Closes #5914 -- configure: show zstd "no" in summary when built without it +- travis: add a build using libressl (from git master) - Reported-by: Marc Hörsken - Fixes #5720 - Closes #5730 - -- quiche: handle calling disconnect twice + The v3.2.1 tag (latest release atm) results in a broken build. - Reported-by: lilongyan-huawei on github - Fixes #5726 - Closes #5727 - -- [Nicolas Sterchele brought this change] + Closes #5932 - getinfo: reset retry-after value in initinfo +- configure: let --enable-debug set -Wenum-conversion with gcc >= 10 - - Avoid re-using retry_after value from preceding request - - Add libtest 3010 to verify + Unfortunately, this option is not detecting the same issues as clang's + -Wassign-enum flag, but should still be useful to detect future + mistakes. - Reported-by: joey-l-us on github - Fixes #5661 - Closes #5672 + Closes #5930 -Marcel Raad (27 Jul 2020) -- WIN32: stop forcing narrow-character API +- openssl: consider ALERT_CERTIFICATE_EXPIRED a failed verification - Except where the results are only used for character output. - getenv is not touched because it's part of the public API, and having - it return UTF-8 instead of ANSI would be a breaking change. + If the error reason from the lib is + SSL_R_SSLV3_ALERT_CERTIFICATE_EXPIRED, libcurl will return + CURLE_PEER_FAILED_VERIFICATION and not CURLE_SSL_CONNECT_ERROR. - Fixes https://github.com/curl/curl/issues/5658 - Fixes https://github.com/curl/curl/issues/5712 - Closes https://github.com/curl/curl/pull/5718 + This unifies the libcurl return code and makes libressl run test 313 + (CRL testing) fine. + + Closes #5934 -Jay Satiro (27 Jul 2020) -- [Tobias Stoeckmann brought this change] +- FAQ: refreshed some very old language - mprintf: Fix stack overflows - - Stack overflows can occur with precisions for integers and floats. +- cmake: make HTTP_ONLY also disable MQTT - Proof of concepts: - - curl_mprintf("%d, %.*1$d", 500, 1); - - curl_mprintf("%d, %+0500.*1$f", 500, 1); + ... and alphasort the order of disabling protocols to make it easier to + browse. - Ideally, compile with -fsanitize=address which makes this undefined - behavior a bit more defined for debug purposes. + Closes #5931 + +- libtest: remove lib1541 leftovers - The format strings are valid. The overflows occur due to invalid - arguments. If these arguments are variables with contents controlled - by an attacker, the function's stack can be corrupted. + Caused automake errors. - Also see CVE-2016-9586 which partially fixed the float aspect. + Follow-up to 8ca54a03ea08a + +- tests/libtests: remove test 1900 and 2033 - Signed-off-by: Tobias Stoeckmann + We already remove the test files, now remove the libtest codes as well. - Closes https://github.com/curl/curl/pull/5722 - -- [Tobias Stoeckmann brought this change] + Follow-up to e50a877df74 - mprintf: Fix dollar string handling - - Verify that specified parameters are in range. If parameters are too - large, fail early on and avoid out of boundary accesses. +Marc Hoersken (7 Sep 2020) +- CI/azure: add test number to title for display in analytics - Also do not read behind boundaries of illegal format strings. + To ease identification of tests the test number is added to + the test case title in order to have it on the Azure DevOps + Analytics pages and reports which currently do not show it. - These are defensive measures since it is expected that format strings - are well-formed. Format strings should not be modifiable by user - input due to possible generic format string attacks. + Bump test case revision to make Azure DevOps update titles. - Closes https://github.com/curl/curl/pull/5722 + Closes #5927 -Daniel Stenberg (26 Jul 2020) -- ntlm: free target_info before (re-)malloc +Daniel Stenberg (6 Sep 2020) +- altsvc: clone setting in curl_easy_duphandle - OSS-Fuzz found a way this could get called again with the pointer still - pointing to a malloc'ed memory, leading to a leak. + The cache content is not duplicated, like other caches, but the setting + and specified file name are. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 + Test 1908 is extended to verify this somewhat. Since the duplicated + handle gets the same file name, the test unfortunately overwrites the + same file twice (with different contents) which makes it hard to check + automatically. - Closes #5724 + Closes #5923 -Marcel Raad (26 Jul 2020) -- CI/macos: set minimum macOS version +- test1541: remove since it is a known bug - This enables some deprecation warnings. - Previously, autotools defaulted to 10.8. + A shared connection cache is not thread-safe is a known issue. Stop + testing this until we believe this issue is addressed. Reduces + occasional test failures we don't care about. - Closes https://github.com/curl/curl/pull/5723 - -Daniel Stenberg (26 Jul 2020) -- RELEASE-NOTES: synced - -Marcel Raad (25 Jul 2020) -- CI/macos: enable warnings as errors for CMake builds + The test code in lib1541.c is left in git to allow us to restore it when + we get to fix this. - Closes https://github.com/curl/curl/pull/5716 + Closes #5922 -- CMake: fix test for warning suppressions +- tests: remove pipelining tests - GCC doesn't warn for unknown `-Wno-` options, except if there are other - warnings or errors [0]. This was problematic with `CURL_WERROR` as that - warning-as-error cannot be suppressed. Notably, this always happened - with `-Wno-pedantic-ms-format` when not targeting Windows. So test for - the positive form of the warning instead, which should always result in - a diagnostic if unknown. + Remove the tests 530, 584, 1900, 1901, 1902, 1903 and 2033. They were + previously disabled. - [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html + The Pipelining code was removed from curl in commit 2f44e94efb3df8e, + April 2019. - Closes https://github.com/curl/curl/pull/5714 + Closes #5921 -Jay Satiro (23 Jul 2020) -- curl.h: update CURLINFO_LASTONE +- curl: retry delays in parallel mode no longer sleeps blocking - CURLINFO_LASTONE should have been updated when - CURLINFO_EFFECTIVE_METHOD was added. + The previous sleep for retries would block all other concurrent + transfers. Starting now, the retry will instead be properly marked to + not get restarted until after the delay time but other transfers can + still continue in the mean time. - Reported-by: xwxbug@users.noreply.github.com + Closes #5917 + +- curl:parallel_transfers: make sure retry readds the transfer - Fixes https://github.com/curl/curl/issues/5711 + Reported-by: htasta on github + Fixes #5905 + Closes #5917 -Marc Hoersken (22 Jul 2020) -- CI/azure: unconditionally enable warnings-as-errors with autotools +- build: drop support for building with Watcom - Reviewed-by: Marcel Raad + These files are not maintained, they seem to have no users, Watcom + compilers look like not having users nor releases anymore. - Follow up to #5694 - Closes #5706 + Closes #5918 -Marcel Raad (21 Jul 2020) -- doh: remove redundant cast +- winbuild/rundebug.cmd: remove - Closes https://github.com/curl/curl/pull/5704 + Seems to have been added by mistake? Not included in dists. + + Closes #5919 -- CI/macos: unconditionally enable warnings-as-errors with autotools +- curl: in retry output don't call all problems "transient" - Previously, warnings were only visible in the output for most jobs. + ... because when --retry-all-errors is used, the error isn't necessarily + transient at all. - Closes https://github.com/curl/curl/pull/5694 + Closes #5916 -- util: silence conversion warnings +- easygetopt: pass a valid enum to avoid compiler warning - timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might - be a 64-bit integer. This is the case when building for recent macOS - versions, for example. Just treat tv_usec as an int, which should - hopefully always be sufficient on systems with - `HAVE_CLOCK_GETTIME_MONOTONIC`. + "integer constant not in range of enumerated type 'CURLoption'" - Closes https://github.com/curl/curl/pull/5695 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/6ebe63fac23f38df911edc348e8ccc72280f9434#commitcomment-42042843 + + Closes #5915 -- md(4|5): don't use deprecated macOS functions +- [Emil Engler brought this change] + + tests: Add tests for new --help - They are marked as deprecated for -mmacosx-version-min >= 10.15, - which might result in warnings-as-errors. + This commit is a part of "--help me if you can" - Closes https://github.com/curl/curl/pull/5695 + Closes #5680 -Daniel Stenberg (18 Jul 2020) -- strdup: remove the odd strlen check +- [Emil Engler brought this change] + + tool: update --help with categories - It confuses code analyzers with its use of -1 for unsigned value. Also, - a check that's not normally used in strdup() code - and not necessary. + This commit is a part of "--help me if you can" - Closes #5697 + Closes #5680 -- [Alessandro Ghedini brought this change] +- [Emil Engler brought this change] - travis: update quiche builds for new boringssl layout + docs: add categories to all cmdline opts - This is required after https://github.com/cloudflare/quiche/pull/593 - moved BoringSSL around slightly. + Adapted gen.pl with 'listcats' - This also means that Go is not needed to build BoringSSL anymore (the - one provided by quiche anyway). + This commit is a part of "--help me if you can" - Closes #5691 + Closes #5680 -Marcel Raad (17 Jul 2020) -- configure: allow disabling warnings - - When using `--enable-warnings`, it was not possible to disable warnings - via CFLAGS that got explicitly enabled. Now warnings are not enabled - anymore if they are explicitly disabled (or enabled) in CFLAGS. This - works for at least GCC, clang, and TCC as they have corresponding - `-Wno-` options for every warning. - - Closes https://github.com/curl/curl/pull/5689 +- RELEASE-NOTES: synced -Daniel Stenberg (16 Jul 2020) -- ngtcp2: adjust to recent sockaddr updates +- [ihsinme brought this change] + + connect.c: remove superfluous 'else' in Curl_getconnectinfo - Closes #5690 + Closes #5912 -- page-header: provide protocol details in the curl.1 man page +- [Samuel Marks brought this change] + + CMake: remove explicit `CMAKE_ANSI_CFLAGS` - Add protocol and version specific information about all protocols curl - supports. + This variable was removed from cmake in commit + https://gitlab.kitware.com/cmake/cmake/commit/5a834b0bb0bc288. A later + CMake commit removes the variable from the tests, claiming that it was + removed in CMake 2.6 - Fixes #5679 - Reported-by: tbugfinder on github - Closes #5686 + Reviewed-By: Peter Wu + Closes #5439 -Daniel Gustafsson (16 Jul 2020) -- docs: Update a few leftover mentions of DarwinSSL +- [cbe brought this change] + + libssh2: pass on the error from ssh_force_knownhost_key_type - Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the - more correct/common name Secure Transport, but a few mentions in the docs - remained. + Closes #5909 + +- scripts/delta: add diffstat summary - Closes #5688 - Reviewed-by: Daniel Stenberg + ... and make output more table-like -Daniel Stenberg (16 Jul 2020) -- file2memory: use a define instead of -1 unsigned value +- [Martin Bašti brought this change] + + http_proxy: do not crash with HTTPS_PROXY and NO_PROXY set - ... to use the maximum value for 'size_t' when detecting integer overflow. - Changed the limit to max/4 as already that seems unreasonably large. + ... in case NO_PROXY takes an effect - Codacy didn't like the previous approach. + Without this patch, the following command crashes: - Closes #5683 - -- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream + $ GIT_CURL_VERBOSE=1 NO_PROXY=github.com HTTPS_PROXY=https://example.com \ + git clone https://github.com/curl/curl.git - ... by adding support for a new dedicated return code. + Minimal libcurl-based reproducer: - Suggested-by: Jonathan Cardoso - Assisted-by: Erik Johansson - URL: https://curl.haxx.se/mail/lib-2020-06/0099.html - Closes #5636 + #include + + int main() { + CURL *curl = curl_easy_init(); + if(curl) { + CURLcode ret; + curl_easy_setopt(curl, CURLOPT_URL, "https://github.com/"); + curl_easy_setopt(curl, CURLOPT_PROXY, "example.com"); + /* set the proxy type */ + curl_easy_setopt(curl, CURLOPT_PROXYTYPE, CURLPROXY_HTTPS); + curl_easy_setopt(curl, CURLOPT_NOPROXY, "github.com"); + curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L); + ret = curl_easy_perform(curl); + curl_easy_cleanup(curl); + return ret; + } + return -1; + } + + Assisted-by: Kamil Dudka + Bug: https://bugzilla.redhat.com/1873327 + Closes #5902 -- [Baruch Siach brought this change] +- travis: add a CI job with openssl3 (from git master) + + Closes #5908 - nss: fix build with disabled proxy support +- openssl: avoid error conditions when importing native CA - Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is - defined. + The code section that is OpenSSL 3+ specific now uses the same logic as + is used in the version < 3 section. It caused a compiler error without + it. - Closes #5667 + Closes #5907 -- test1139: make it display the difference on test failures +- setopt: avoid curl_ on local variable + + Closes #5906 -- test1119: verify stdout in the test +- mqtt.c: avoid curl_ prefix on local variable - So that failures will be displayed in the terminal, as it makes test failures - visually displayed easier and faster. + Closes #5906 + +- wildcard: strip "curl_" prefix from private symbols - Closes #5644 + Closes #5906 -- curl: add %{method} to the -w variables +- vtls: make it 'struct Curl_ssl_session' - Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. + Use uppercase C for internal symbols. - Added test 1197 to verify. + Closes #5906 -- CURLINFO_EFFECTIVE_METHOD: added +- curl_threads: make it 'struct Curl_actual_call' - Provide the HTTP method that was used on the latest request, which might - be relevant for users when there was one or more redirects involved. + Internal names should not be prefixed "curl_" - Closes #5511 + Closes #5906 -Viktor Szakats (14 Jul 2020) -- windows: add unicode to feature list +- schannel: make it 'struct Curl_schannel*' - Reviewed-by: Marcel Raad - Reviewed-by: Marc Hörsken + As internal global names should use captical C. - Closes #5491 + Closes #5906 -Daniel Stenberg (14 Jul 2020) -- multi: remove two checks always true +- hash: make it 'struct Curl_hash' - Detected by Codacy - Closes #5676 - -Marc Hoersken (13 Jul 2020) -- workflows: limit what branches to run CodeQL on + As internal global names should use captical C. - Align CodeQL action with existing CI actions: - - Update branch filter to avoid duplicate CI runs. - - Shorten workflow name due to informative job name. + Closes #5906 + +- llist: make it "struct Curl_llist" - Reviewed-by: Daniel Stenberg + As internal global names should use captical C. - Closes #5660 + Closes #5906 -- appveyor: collect libcurl.dll variants with prefix or suffix +Marc Hoersken (2 Sep 2020) +- telnet.c: depend on static requirement of WinSock version 2 - On some platforms libcurl is build with a platform-specific - prefix and/or a version number suffix. + Drop dynamic loading of ws2_32.dll and instead rely on the + imported version which is now required to be at least 2.2. - Assisted-by: Jay Satiro + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats - Closes #5659 - -Daniel Stenberg (12 Jul 2020) -- [ihsinme brought this change] + Closes #5854 - socks: use size_t for size variable +- win32: drop support for WinSock version 1, require version 2 - Use the unsigned type (size_t) in the arithmetic of pointers. In this - context, the signed type (ssize_t) is used unnecessarily. + IPv6, telnet and now also the multi API require WinSock + version 2 which is available starting with Windows 95. - Authored-by: ihsinme on github - Closes #5654 - -- RELEASE-NOTES: synced + Therefore we think it is time to drop support for version 1. - ... and bumped to 7.72.0 as the next release version number - -- [Gilles Vollant brought this change] + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Reviewed-by: Viktor Szakats + + Follow up to #5634 + Closes #5854 - content_encoding: add zstd decoding support +- select: align poll emulation to return all relevant events - include zstd curl patch for Makefile.m32 from vszakats - and include Add CMake support for zstd from Peter Wu + The poll emulation via select already consumes POLLRDNORM, + POLLWRNORM and POLLRDBAND as input events. Therefore it + should also return them as output events if signaled. - Helped-by: Viktor Szakats - Helped-by: Peter Wu - Closes #5453 - -- asyn.h: remove the Curl_resolver_getsock define + Also fix indentation in input event handling block. - - not used - - used the wrong number of arguments - - confused the Codeacy code analyzer + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5647 - -- [Nicolas Sterchele brought this change] + Replaces #5852 + Closes #5883 - configure.ac: Sort features name in summary +- CI/azure: MQTT is now enabled by default - - Same as protocols + Reviewed-by: Daniel Stenberg - Closes #5656 + Follow up to #5858 + Closes #5903 -- [Matthias Naegler brought this change] +Daniel Stenberg (2 Sep 2020) +- copyright.pl: ignore buildconf - cmake: fix windows xp build +- test971: show test mismatches "inline" + +- lib/Makefile.am: bump VERSIONINFO due to new functions - Reviewed-by: Marcel Raad - Closes #5662 + ... we're generally bad at this, but we are adding new functions for + this release. + + Closes #5899 -- ngtcp2: update to modified qlog callback prototype +- optiontable: use DEBUGBUILD - Closes #5675 + Follow-up to commit 6e18568ba38 (#5877) -- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle +- cmdline-opts/gen.pl: generate nicer "See Also" in curl.1 - Added test case 674 to reproduce and verify the bug report. + If there are more than two items in the list, use commas for all but the + last separator which is set to 'and'. Reads better. - Fixes #5665 - Reported-by: NobodyXu on github - Closes #5673 - -- [Baruch Siach brought this change] + Closes #5898 - bearssl: fix build with disabled proxy support +- curl.1: add see also no-progress-meter on two spots - Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is - defined. + Ref: #5894 - Reviewed-by: Nicolas Sterchele - Closes #5666 + Closes #5897 - RELEASE-NOTES: synced -Jay Satiro (11 Jul 2020) -- [Carlo Marcelo Arenas Belón brought this change] - - cirrus-ci: upgrade 11-STABLE to 11.4 +- mqtt: enable by default - Meant to be the last of the 11 series and so make sure that all - other references reflect all 11 versions so they can be retired - together later. + No longer considered experimental. - Closes https://github.com/curl/curl/pull/5668 + Closes #5858 -- [Filip Salomonsson brought this change] +- [Michael Baentsch brought this change] - CURLINFO_CERTINFO.3: fix typo + tls: add CURLOPT_SSL_EC_CURVES and --curves - Closes https://github.com/curl/curl/pull/5655 + Closes #5892 -Daniel Stenberg (4 Jul 2020) -- http2: only do the *done() cleanups for HTTP - - Follow-up to ef86daf4d3 - - Closes #5650 - Fixes #5646 +- url: remove funny embedded comments in Curl_disonnect calls -- [Alex Kiernan brought this change] +- [Chris Paulson-Ellis brought this change] - gnutls: repair the build with `CURL_DISABLE_PROXY` + conn: check for connection being dead before reuse - `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` - if `CURL_DISABLE_PROXY` is enabled. Repair the build with that - configuration. + Prevents incorrect reuse of an HTTP connection that has been prematurely + shutdown() by the server. - Signed-off-by: Alex Kiernan - Closes #5645 - -Alex Kiernan (3 Jul 2020) -- gnutls: Fetch backend when using proxy + Partial revert of 755083d00deb16 - Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") - Signed-off-by: Alex Kiernan - -Daniel Stenberg (3 Jul 2020) -- [Laramie Leavitt brought this change] + Fixes #5884 + Closes #5893 - http2: close the http2 connection when no more requests may be sent - - Well-behaving HTTP2 servers send two GOAWAY messages. The first - message is a warning that indicates that the server is going to - stop accepting streams. The second one actually closes the stream. +Marc Hoersken (29 Aug 2020) +- buildconf: exec autoreconf to avoid additional process - nghttp2 reports this state (and the other state of no more stream - identifiers) via the call nghttp2_session_check_request_allowed(). - In this state the client should not create more streams on the - session (tcp connection), and in curl this means that the server - has requested that the connection is closed. + Also make buildconf exit with the return code of autoreconf. - It would be also be possible to put the connclose() call into the - on_http2_frame_recv() function that triggers on the GOAWAY message. + Reviewed-by: Daniel Stenberg - This fixes a bug seen when the client sees the following sequence of - frames: + Follow up to #5853 + Closes #5890 + +- CI/azure: no longer ignore results of test 1013 - // advisory GOAWAY - HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] - ... some additional frames + Follow up to #5771 + Closes #5889 + +- docs: add description about CI platforms to CONTRIBUTE.md - // final GOAWAY - HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + Reviewed-by: Jay Satiro - Before this change, curl will attempt to reuse the connection even - after the last stream, will encounter this error: + Closes #5882 + +Daniel Stenberg (29 Aug 2020) +- tests/getpart: use MIME::Base64 instead of home-cooked - * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] - * Re-using existing connection! (#0) with host localhost - * Connected to localhost (::1) port 10443 (#0) - * Using Stream ID: 9 (easy handle 0x5595f0a72e30) - > GET /index.html?5 HTTP/2 - > Host: localhost:10443 - > user-agent: curl/7.68.0 - > accept: */* - > - * stopped the pause stream! - * Connection #0 to host localhost left intact - curl: (16) Error in the HTTP2 framing layer + Since we already use the base64 package since a while back, we can just + as well switch to that here too. - This error may posion the connection cache, causing future requests - which resolve to the same curl connection to go through the same error - path. + It also happens to use the exact same function name, which otherwise + causes a run-time warning. - Closes #5643 + Reported-by: Marc Hörsken + Fixes #5885 + Closes #5887 -- ftpserver: don't verify SMTP MAIL FROM names +Marcel Raad (29 Aug 2020) +- ntlm: fix condition for curl_ntlm_core usage - Rely on tests asking the names to get refused instead - test servers - should be as dumb as possible. Edited test 914, 955 and 959 accordingly. + `USE_WINDOWS_SSPI` without `USE_WIN32_CRYPTO` but with any other DES + backend is fine, but was excluded before. - Closes #5639 + This also fixes test 1013 as the condition for SMB support in + configure.ac didn't match the condition in the source code. Now it + does. + + Fixes https://github.com/curl/curl/issues/1262 + Closes https://github.com/curl/curl/pull/5771 -- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated +- AppVeyor: switch 64-bit Schannel Debug CMake builds to Unicode - This came up in #5640. It make sense to clarify this in the docs! + The Schannel builds are the most useful to verify as they make the most + use of the Windows API. Classic MinGW doesn't support Unicode at all, + only MinGW-w64 and MSVC do. - Reminded-by: Kamil Dudka - Closes #5642 + Closes https://github.com/curl/curl/pull/5843 -Kamil Dudka (3 Jul 2020) -- tool_getparam: make --krb option work again +- CMake: add option to enable Unicode on Windows - It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. + As already existing for winbuild. - Bug: https://bugzilla.redhat.com/1833193 - Closes #5640 - -Daniel Stenberg (2 Jul 2020) -- [Jeremy Maitin-Shepard brought this change] + Closes https://github.com/curl/curl/pull/5843 - http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages +Marc Hoersken (29 Aug 2020) +- select: simplify return code handling for poll and select - Confusingly, nghttp2 has two different error code enums: + poll and select already return -1 on error according to POSIX, + so there is no need to perform a <0 to -1 conversion in code. - - nghttp2_error, to be used with nghttp2_strerror - - nghttp2_error_code, to be used with nghttp2_http2_strerror + Also we can just use one check with <= 0 on the return code. - Closes #5641 - -Marcel Raad (2 Jul 2020) -- url: silence MSVC warning - - Since commit f3d501dc678, if proxy support is disabled, MSVC warns: - url.c : warning C4701: potentially uninitialized local variable - 'hostaddr' used - url.c : error C4703: potentially uninitialized local pointer variable - 'hostaddr' used - - That could actually only happen if both `conn->bits.proxy` and - `CURL_DISABLE_PROXY` were enabled. - Initialize it to NULL to silence the warning. + Assisted-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Closes https://github.com/curl/curl/pull/5638 + Replaces #5852 + Closes #5880 -Daniel Stenberg (1 Jul 2020) +Daniel Stenberg (28 Aug 2020) - RELEASE-NOTES: synced -Version 7.71.1 (30 Jun 2020) - -Daniel Stenberg (30 Jun 2020) -- RELEASE-NOTES: curl 7.71.1 - -- THANKS: add contributors to 7.71.1 - -- scripts/copyright.pl: skip .dcignore +- [Jeroen Ooms brought this change] -- Revert "multi: implement wait using winsock events" - - This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. + tests: add test1912 with typechecks - That commit (from #5397) introduced a regression in 7.71.0. + Validates that gcc-typecheck macros match the new option type API. - Reported-by: tmkk on github - Fixes #5631 - Closes #5632 - -- TODO: Add flag to specify download directory - -- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection + Closes #5873 -- cirrus-ci: disable FreeBSD 13 (again) - - It has been failing for a good while again. This time we better leave it - disabled until we have more reason to believe it behaves. +- easyoptions: provide debug function when DEBUGBUILD - Closes #5628 - -- ngtcp2: sync with current master + ... not CURLDEBUG as they're not always set in conjunction. - ngtcp2 added two new callbacks + Follow-up to 6ebe63fac23f38df - Reported-by: Lucien Zürcher - Fixes #5624 - Closes #5627 + Fixes #5877 + Closes #5878 -- examples/multithread.c: call curl_global_cleanup() +Marc Hoersken (28 Aug 2020) +- sockfilt: handle FD_CLOSE winsock event on write socket - Reported-by: qiandu2006 on github - Fixes #5622 - Closes #5623 - -- vtls: compare cert blob when finding a connection to reuse + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Reported-by: Gergely Nagy - Fixes #5617 - Closes #5619 - -- RELEASE-NOTES: synced + Follow up to #5867 + Closes #5879 -- terminology: call them null-terminated strings +- multi: handle connection state winsock events - Updated terminology in docs, comments and phrases to refer to C strings - as "null-terminated". Done to unify with how most other C oriented docs - refer of them and what users in general seem to prefer (based on a - single highly unscientific poll on twitter). + Learn from the way Cygwin handles and maps the WinSock events + to simulate correct and complete poll and select behaviour + according to Richard W. Stevens Network Programming book. - Reported-by: coinhubs on github - Fixes #5598 - Closes #5608 + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + + Follow up to #5634 + Closes #5867 -- http: fix proxy auth with blank password +Daniel Stenberg (28 Aug 2020) +- Curl_pgrsTime - return new time to avoid timeout integer overflow - Regression in 7.71.0 + Setting a timeout to INT_MAX could cause an immediate error to get + returned as timeout because of an overflow when different values of + 'now' were used. - Added test case 346 to verify. + This is primarily fixed by having Curl_pgrsTime() return the "now" when + TIMER_STARTSINGLE is set so that the parent function will continue using + that time. - Reported-by: Kristoffer Gleditsch - Fixes #5613 - Closes #5616 + Reported-by: Ionuț-Francisc Oancea + Fixes #5583 + Closes #5847 -- .dcignore: ignore tests and docs directories +- TLS: fix SRP detection by using the proper #ifdefs - This is a config file for deepcode.ai, a static code analyzer. - -Jay Satiro (26 Jun 2020) -- tool_cb_hdr: Fix etag warning output and return code + USE_TLS_SRP will be true if *any* selected TLS backend can use SRP - - Return 'failure' on failure, to follow the existing style. + HAVE_OPENSSL_SRP is defined when OpenSSL can use it - - Put Warning: and the warning message on the same line. + HAVE_GNUTLS_SRP is defined when GnuTLS can use it - Ref: https://github.com/curl/curl/issues/5610 + Clarify in the curl_verison_info docs that CURL_VERSION_TLSAUTH_SRP is + set if at least one of the supported backends offers SRP. - Closes https://github.com/curl/curl/pull/5612 + Reported-by: Stefan Strogin + Fixes #5865 + Closes #5870 -Daniel Stenberg (26 Jun 2020) -- CURLOPT_READFUNCTION.3: provide the upload data size up front - - Assisted-by: Jay Satiro - Closes #5607 +- [Dan Kenigsberg brought this change] -- test1539: do a HTTP 1.0 POST without a set size (fails) + docs: SSLCERTS: fix English syntax - Attempt to reproduce #5593. Test case 1514 is very similar but uses - HTTP/1.1 and thus switches to chunked. + Signed-off-by: Dan Kenigsberg - Closes #5595 + Closes #5876 -- [Baruch Siach brought this change] +- [Alessandro Ghedini brought this change] - mbedtls: fix build with disabled proxy support - - Don't reference fields that do not exist. Fixes build failure: + docs: non-existing macros in man pages - vtls/mbedtls.c: In function 'mbed_connect_step1': - vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' + As reported by man(1) when invoked as: - Closes #5615 - -- codeql-analysis.yml: fix the 'languages' setting + man --warnings -E UTF-8 -l -Tutf8 -Z >/dev/null - It needs a 'with:' in front of it. + Closes #5846 -GitHub (26 Jun 2020) -- [Daniel Stenberg brought this change] +- [Alessandro Ghedini brought this change] - gtihub: codeql-analysis.yml + curl.1: fix typo invokved -> invoked - enables code security scanning with github actions + Closes #5846 -Daniel Stenberg (25 Jun 2020) -- tests: verify newline in username and password for HTTP +- buildconf: invoke 'autoreconf -fi' instead - test 1296 is a simply command line test + The custom script isn't necessary anymore - but remains for simplicity + and just invokes autoreconf. - test 1910 is a libcurl test including a redirect + Closes #5853 -- url: allow user + password to contain "control codes" for HTTP(S) - - Reported-by: Jon Johnson Jr - Fixes #5582 - Closes #5592 +- [Emil Engler brought this change] -- escape: make the URL decode able to reject only %00 bytes + lib: make Curl_gethostname accept a const pointer - ... or all "control codes" or nothing. + The address of that variable never gets changed, only the data in it so + why not make it a "char * const"? - Assisted-by: Nicolas Sterchele + Closes #5866 -- http2: set the correct URL in pushed transfers - - ...previously CURLINFO_EFFECTIVE_URL would report the URL of the - original "mother transfer", not the actually pushed resource. +- docs/libcurl: update "Added in" version for curl_easy_option* - Reported-by: Jonathan Cardoso Machado - Fixes #5589 - Closes #5591 - -Jay Satiro (25 Jun 2020) -- [Javier Blazquez brought this change] + Follow-up to 6ebe63fac23f38 - openssl: Fix compilation on Windows when ngtcp2 is enabled - - - Include wincrypt before OpenSSL includes so that the latter can - properly handle any conflicts between the two. +- scripts: improve the "get latest curl release tag" logic - Closes https://github.com/curl/curl/pull/5606 + ... by insiting on it matching "^curl-". -Daniel Stenberg (25 Jun 2020) -- test543: extended to verify zero length input +- configure: added --disable-get-easy-options - As was reported in #5601 + To allow disabling of the curl_easy_option APIs in a build. + + Closes #5365 -- escape: zero length input should return a zero length output +- options: API for meta-data about easy options - Regression added in 7.71.0. + const struct curl_easyoption *curl_easy_option_by_name(const char *name); - Fixes #5601 - Reported-by: Kristoffer Gleditsch - Closes #5602 - -- Curl_inet_ntop: always check the return code + const struct curl_easyoption *curl_easy_option_by_id (CURLoption id); - Reported-by: Siva Sivaraman - Fixes #5412 - Closes #5597 - -- sendf: improve the message on client write errors + const struct curl_easyoption * + curl_easy_option_next(const struct curl_easyoption *prev); - Replace "Failed writing body (X != Y)" with - "Failure writing output to destination". Possibly slightly less cryptic. + The purpose is to provide detailed enough information to allow for + example libcurl bindings to get option information at run-time about + what easy options that exist and what arguments they expect. - Reported-by: coinhubs on github - Fixes #5594 - Closes #5596 - -- RELEASE-NOTES: synced - -- curlver: start working on 7.71.1 + Assisted-by: Jeroen Ooms + Closes #5365 -- [Denis Baručić brought this change] +- [Eric Curtin brought this change] - DYNBUF.md: fix a typo: trail => tail + HTTP/3: update to OpenSSL_1_1_1g-quic-draft-29 - Closes #5599 - -Version 7.71.0 (23 Jun 2020) - -Daniel Stenberg (23 Jun 2020) -- RELEASE-NOTES: curl 7.71.0 release + Closes #5871 -- THANKS: curl 7.71.0 additions +- RELEASE-NOTES: synced -- url: make sure pushed streams get an allocated download buffer - - Follow-up to c4e6968127e876b0 - - When a new transfer is created, as a resuly of an acknowledged push, - that transfer needs a download buffer allocated. +Jay Satiro (26 Aug 2020) +- openssl: Fix wincrypt symbols conflict with BoringSSL - Closes #5590 - -Jay Satiro (22 Jun 2020) -- openssl: Don't ignore CA paths when using Windows CA store + OpenSSL undefines the conflicting symbols but BoringSSL does not so we + must do it ourselves. - This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does - not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default - locations. Instead the CA store can now be used at the same time. + Reported-by: Samuel Tranchet + Assisted-by: Javier Blazquez - The change is due to the impending release. The issue is still being - discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and - is now documented as experimental. + Ref: https://bugs.chromium.org/p/boringssl/issues/detail?id=371 + Ref: https://github.com/openssl/openssl/blob/OpenSSL_1_1_1g/include/openssl/ossl_typ.h#L66-L73 - Ref: bc052cc (parent commit) - Ref: https://github.com/curl/curl/issues/5585 + Fixes https://github.com/curl/curl/issues/5669 + Closes https://github.com/curl/curl/pull/5857 -- tool_operate: Don't use Windows CA store as a fallback - - Background: - - 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate - store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides - CURLOPT_CAINFO if both are set. The curl tool will fall back to - CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set - via CURLOPT_CAINFO. - - Problem: - - libcurl may be built with hardcoded paths to a certificate bundle or - directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are - ignored. - - Solution: +Daniel Stenberg (26 Aug 2020) +- socketpair: allow CURL_DISABLE_SOCKETPAIR - A solution is still being discussed but since there's an impending - release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + ... to completely disable the use of socketpair - Ref: https://github.com/curl/curl/issues/5585 + Closes #5850 -- openssl: Fix CA fallback logic for OpenSSL 3.0 build - - Prior to this change I assume a build error would occur when - CURL_CA_FALLBACK was used. +- curl_get_line: build only if cookies or alt-svc are enabled - Closes https://github.com/curl/curl/pull/5587 - -Daniel Stenberg (22 Jun 2020) -- copyright: update mismatched copyright years + Closes #5851 -- test1460: verify that -Ji is not ok +- [fullincome brought this change] -- tool_getparam: -i is not OK if -J is used + schannel: fix memory leak when using get_cert_location - Reported-by: sn on hackerone - Bug: https://curl.haxx.se/docs/CVE-2020-8177.html - -- [Peter Wu brought this change] - - CMake: ignore INTERFACE_LIBRARY targets for pkg-config file + The get_cert_location function allocates memory only on success. + Previously get_cert_location was able to allocate memory and return + error. It wasn't obvious and in this case the memory wasn't + released. - Reviewed-by: Marcel Raad - Fixes #5512 - Closes #5517 + Fixes #5855 + Closes #5860 -- [Valentyn Korniienko brought this change] +- [Emil Engler brought this change] - multibyte: Fixed access-> waccess to file for Windows Plarform + git: ignore libtests in 3XXX area - Reviewed-by: Marcel Raad - Closes #5580 - -- altsvc: bump to h3-29 + Currently the file tests/libtest/lib3010 is not getting + ignored by git. This fixes it by adding the 3XXX area to + the according .gitignore file. - Closes #5584 + Closes #5859 -- urlglob: treat literal IPv6 addresses with zone IDs as a host name +- [Emil Engler brought this change] + + doh: add error message for DOH_DNS_NAME_TOO_LONG - ... and not as a "glob". Now done by passing the supposed host to the - URL parser which supposedly will do a better job at identifying "real" - numerical IPv6 addresses. + When this error code was introduced in b6a53fff6c1d07e8a9, it was + forgotten to be added in the errors array and doh_strerror function. - Reported-by: puckipedia on github - Fixes #5576 - Closes #5579 - -- test1179: verify error message for non-existing cmdline option + Closes #5863 -- tool_getparam: repair the error message for unknown flag +- ngtcp2: adapt to the new pkt_info arguments - Follow-up to 9e5669f3880674 - Detected by Coverity CID 1464582 ("Logically dead code") + Guidance-by: Tatsuhiro Tsujikawa - Closes #5577 - -- FILEFORMAT: describe verify/stderr + Closes #5864 -- connect: improve happy eyeballs handling - - For QUIC but also for regular TCP when the second family runs out of IPs - with a failure while the first family is still trying to connect. +- winbuild/README.md: make visible - Separated the timeout handling for IPv4 and IPv6 connections when they - both have a number of addresses to iterate over. + Follow-up to be753add31c2d8c -- ngtcp2: never call fprintf() in lib code in release version +- winbuild: convert the instruction text to README.md + + Closes #5861 -- ngtcp2: fix happy eyeballs quic connect crash +- lib1560: verify "redirect" to double-slash leading URL - Reported-by: Peter Wu - Fixes #5565 - Closes #5568 + Closes #5849 -- select: remove the unused ELAPSED_MS() macro +Marc Hoersken (25 Aug 2020) +- multi: expand pre-check for socket readiness - Closes #5573 + Check readiness of all sockets before waiting on them + to avoid locking in case the one-time event FD_WRITE + was already consumed by a previous wait operation. + + More information about WinSock network events: + https://docs.microsoft.com/en-us/windows/win32/api/ + winsock2/nf-winsock2-wsaeventselect#return-value + + Closes #5634 -Marc Hoersken (17 Jun 2020) - [rcombs brought this change] multi: implement wait using winsock events @@ -4471,2970 +4662,2786 @@ Marc Hoersken (17 Jun 2020) using a TCP socket on loopback which could in turn lead to socket resource exhaustion. - Reviewed-by: Gergely Nagy - Reviewed-by: Marc Hörsken + A previous version of this patch failed to account for how in WinSock, + FD_WRITE is set only once when writing becomes possible and not again + until after a send has failed due to the buffer filling. This contrasts + to how FD_READ and FD_OOB continue to be set until the conditions they + refer to no longer apply. This meant that if a user wrote some data to + a socket, but not enough data to completely fill its send buffer, then + waited on that socket to become writable, we'd erroneously stall until + their configured timeout rather than returning immediately. - Closes #5397 - -Daniel Stenberg (17 Jun 2020) -- manpage: add three missing environment variables + This version of the patch addresses that issue by checking each socket + we're waiting on to become writable with select() before the wait, and + zeroing the timeout if it's already writable. - CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE + Assisted-by: Marc Hörsken + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + Tested-by: Gergely Nagy + Tested-by: Rasmus Melchior Jacobsen + Tested-by: Tomas Berger - Closes #5571 - -- RELEASE-NOTES: synced + Replaces #5397 + Reverts #5632 + Closes #5634 -- configure: for wolfSSL, check for the DES func needed for NTLM +- select: reduce duplication of Curl_poll in Curl_socket_check - Also adds pkg-config support for the wolfSSL detection. - -- [Ruurd Beerstra brought this change] - - ntlm: enable NTLM support with wolfSSL + Change Curl_socket_check to use select-fallback in Curl_poll + instead of implementing it in Curl_socket_check and Curl_poll. - When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* - functions that OpenSSL has. This change take advantage of that. + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro - Co-authored-by: Daniel Stenberg - Closes #5556 - Fixes #5548 + Replaces #5262 and #5492 + Closes #5707 -- http: move header storage to Curl_easy from connectdata +- select: fix poll-based check not detecting connect failure - Since the connection can be used by many independent requests (using - HTTP/2 or HTTP/3), things like user-agent and other transfer-specific - data MUST NOT be kept connection oriented as it could lead to requests - getting the wrong string for their requests. This struct data was - lingering like this due to old HTTP1 legacy thinking where it didn't - mattered.. + This commit changes Curl_socket_check to use POLLPRI to + check for connect failure on the write socket, because + POLLPRI maps to fds_err. This is in line with select(2). - Fixes #5566 - Closes #5567 - -- CODE_REVIEW.md: how to do code reviews in curl + The select-based socket check correctly checks for connect + failures by adding the write socket also to fds_err. - Assisted-by: Daniel Gustafsson - Assisted-by: Rich Salz - Assisted-by: Hugo van Kemenade - Assisted-by: James Fuller - Assisted-by: Marc Hörsken - Assisted-by: Jay Satiro + The poll-based implementation (which internally can itself + fallback to select again) did not previously check for + connect failure by using POLLPRI with the write socket. - Closes #5555 - -- altsvc: remove the num field from the altsvc struct + See the follow up commit to this for more information. - It was superfluous since we have the list.size alredy + This commit makes sure connect failures can be detected + and handled if HAVE_POLL_FINE is defined, eg. on msys2-devel. - Reported-by: Jay Satiro - Fixes #5553 - Closes #5563 + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + + Replaces #5509 + Prepares #5707 -- version.d: expanded and alpha-sorted +- select.h: make socket validation macros test for INVALID_SOCKET - Added a few missing features not previously mentioned. Ordered them - alphabetically. + With Winsock the valid range is [0..INVALID_SOCKET-1] according to + https://docs.microsoft.com/en-us/windows/win32/winsock/socket-data-type-2 - Closes #5558 + Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg + + Closes #5760 -- ABI.md: rename to .md and polish the markdown +Daniel Stenberg (24 Aug 2020) +- docs: --output-dir is added in 7.73.0, nothing else - Closes #5562 + Follow-up to 5620d2cc78c0 -- HELP-US: add a section for "smaller tasks" +- curl: add --output-dir - The point of this section is to meet the CII Best Practices gold level - critera: + Works with --create-dirs and with -J - "The project MUST clearly identify small tasks that can be performed by - new or casual contributors" + Add test 3008, 3009, 3011, 3012 and 3013 to verify. - Closes #5560 + Closes #5637 -- TODO: retry on the redirected-to URL +- configure: fix pkg-config detecting wolfssl - Closes #5462 - -- mailmap: Nicolas Sterchele + When amending the include path with "/wolfssl", this now properly strips + off all whitespace from the path variable! Previously this would lead to + pkg-config builds creating bad command lines. + + Closes #5848 -- [Nicolas Sterchele brought this change] +- [Michael Musset brought this change] - TODO: remove 19.3 section title + sftp: add the option CURLKHSTAT_FINE_REPLACE - Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on - curl documentation website + Replace the old fingerprint of the host with a new. - Closes #5561 - -- [Martin V brought this change] + Closes #5685 - test1560: avoid possibly negative association in wording +- RELEASE-NOTES: synced - Closes #5549 + The next release is now to become 7.73.0 -- share: don't set the share flag it something fails - - When asking for a specific feature to be shared in the share object, - that bit was previously set unconditionally even if the shared feature - failed or otherwise wouldn't work. +- checksrc: verify do-while and spaces between the braces - Closes #5554 - -- buildconf: remove -print from the find command that removes files + Updated mprintf.c to comply - It's just too annoying and unnecessary to get a long list of files shown - -- RELEASE-NOTES: synced + Closes #5845 -- wording: avoid blacklist/whitelist stereotypes +- curl: support XDG_CONFIG_HOME to find .curlrc - Instead of discussing if there's value or meaning (implied or not) in - the colors, let's use words without the same possibly negative - associations. + Added test433 to verify. Updated documentation. - Closes #5546 + Reviewed-by: Jay Satiro + Suggested-by: Eli Schwartz + Fixes #5829 + Closes #5837 -Jay Satiro (9 Jun 2020) -- tool_getparam: fix memory leak in parse_args +- etag: save and use the full received contents - Prior to this change in Windows Unicode builds most parsed options would - not be freed. + ... which makes it support weak tags and non-standard etags too! - Found using _CrtDumpMemoryLeaks(). + Added test case 347 to verify blank incoming ETag: - Ref: https://github.com/curl/curl/issues/5545 + Fixes #5610 + Closes #5833 -Daniel Stenberg (8 Jun 2020) -- socks: detect connection close during handshake +- setopt: if the buffer exists, refuse the new BUFFERSIZE - The SOCKS4/5 state machines weren't properly terminated when the proxy - connection got closed, leading to a busy-loop. + The buffer only exists during transfer and then we shouldn't change the + size (the setopt is not documented to work then). - Reported-By: zloi-user on github - Fixes #5532 - Closes #5542 + Reported-by: Harry Sintonen + Closes #5842 -- [James Fuller brought this change] +- [COFFEETALES brought this change] - multi: add defensive check on data->multi->num_alive + sftp: add new quote commands 'atime' and 'mtime' - Closes #5540 + Closes #5810 -- Curl_addrinfo: use one malloc instead of three +- CURLE_PROXY: new error code - To reduce the amount of allocations needed for creating a Curl_addrinfo - struct, make a single larger malloc instead of three separate smaller - ones. + Failures clearly returned from a (SOCKS) proxy now causes this return + code. Previously the situation was not very clear as what would be + returned and when. - Closes #5533 - -- [Alessandro Ghedini brought this change] + In addition: when this error code is returned, an application can use + CURLINFO_PROXY_ERROR to query libcurl for the detailed error, which then + returns a value from the new 'CURLproxycode' enum. + + Closes #5770 - quiche: update SSLKEYLOGFILE support +- runtests: make cleardir() erase dot files too - quiche now requires the application to explicitly set the keylog path - for each connection, rather than reading the environment variable - itself. + Because test cases might use dot files. - Closes #5541 + Closes #5838 -- tests: add two simple tests for --login-options +- KNOWN_BUGS: 'no_proxy' string-matches IPv6 numerical addreses - Test 895 and 896 - as a follow-up to a3e972313b + Also: the current behavior is now documented in the curl.1 and + CURLOPT_NOPROXY.3 man pages. - Closes #5539 + Reported-by: Andrew Barnes + Closes #5745 + Closes #5841 -- ngtcp2: update with recent API changes +Viktor Szakats (22 Aug 2020) +- Makefile.m32: add ability to override zstd libs [ci skip] - Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. + Similarly to brotli, where this was already possible. + E.g. it allows to link zstd statically to libcurl.dll. - Assisted-by: Tatsuhiro Tsujikawa - Closes #5538 - -- [James Fuller brought this change] + Ref: https://github.com/curl/curl-for-win/issues/12 + Ref: https://github.com/curl/curl-for-win/commit/d9b266afd2e5d3f5604483010ef62340b5918c89 + + Closes https://github.com/curl/curl/pull/5840 - socks: remove unreachable breaks in socks.c and mime.c +Daniel Stenberg (21 Aug 2020) +- runtests: avoid 'fail to start' repeated messages in attempt loops - Closes #5537 + Closes #5834 -- tool_cfgable: free login_options at exit +- runtests: clear pid variables when failing to start a server - Memory leak - Reported-by: Geeknik Labs - Fixes #5535 - Closes #5536 + ... as otherwise the parent doesn't detect the failure and believe it + actually worked to start. + + Reported-by: Christian Weisgerber + Bug: https://curl.haxx.se/mail/lib-2020-08/0018.html + Closes #5834 -- libssh2: keep sftp errors as 'unsigned long' +- TODO: Virtual external sockets - Remove weird work-around for storing the SFTP errors as int instead of - the "unsigned long" that libssh2 actually returns for SFTP errors. + Closes #5835 + +- [Don J Olmstead brought this change] + + dist: add missing CMake Find modules to the distribution - Closes #5534 + Closes #5836 -Marc Hoersken (6 Jun 2020) -- timeouts: move ms timeouts to timediff_t from int and long +- RELEASE-NOTES: synced - Now that all functions in select.[ch] take timediff_t instead - of the limited int or long, we can remove type conversions - and related preprocessor checks to silence compiler warnings. + ... and version bumped to 7.72.1 + +- tls: provide the CApath verbose log on its own line - Avoiding conversions from time_t was already done in 842f73de. + ... not newline separated from the previous line. This makes it output + asterisk prefixed properly like other verbose putput! - Based upon #5262 - Supersedes #5214, #5220 and #5221 - Follow up to #5343 and #5479 - Closes #5490 + Reported-by: jmdavitt on github + Fixes #5826 + Closes #5827 -Daniel Stenberg (6 Jun 2020) -- [François Rigault brought this change] +Version 7.72.0 (19 Aug 2020) - openssl: set FLAG_TRUSTED_FIRST unconditionally +Daniel Stenberg (19 Aug 2020) +- RELEASE-NOTES: synced - On some systems, openssl 1.0.x is still the default, but it has been - patched to contain all the recent security fixes. As a result of this - patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be - defined, while the previous behavior of openssl to not look at trusted - chains first, remains. + The curl 7.72.0 release + +- THANKS: add names from curl 7.72.0 release + +Jay Satiro (18 Aug 2020) +- KNOWN_BUGS: Schannel TLS 1.2 handshake bug in old Windows versions - Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to - probe for the behavior of openssl based on the existence ofmacros. + Reported-by: plujon@users.noreply.github.com - Closes #5530 + Closes https://github.com/curl/curl/issues/5488 -- server/util: fix logmsg format using curl_off_t argument +Daniel Stenberg (17 Aug 2020) +- Curl_easy: remember last connection by id, not by pointer - ... this caused segfaults on armv7. + CVE-2020-8231 - Regression added in dd0365d560aea5a (7.70.0) + Bug: https://curl.haxx.se/docs/CVE-2020-8231.html - Reviewed-by: Jay Satiro - Closes #5529 + Reported-by: Marc Aldorasi + Closes #5824 -- RELEASE-NOTES: synced +- examples/rtsp.c: correct the copyright year -- [Cherish98 brought this change] +- RELEASE-PROCEDURE.md: add more future release dates - socks: fix expected length of SOCKS5 reply +- [H3RSKO brought this change] + + docs: change "web site" to "website" - Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the - reply ATYP is X'01'. This resulted in erroneously expecting more bytes - when the request length is greater than the reply length (e.g., when - remotely resolving the hostname). + According to wikipedia: - Closes #5527 - -Marc Hoersken (5 Jun 2020) -- .gitignore: add directory containing the stats repo + While "web site" was the original spelling, this variant has become + rarely used, and "website" has become the standard spelling - Since the new curl/stats repository is designed to be - checked out into the curl repository working tree as stats/ - it should be on the ignore list to aid in commit staging. + Closes #5822 -Daniel Stenberg (5 Jun 2020) -- [Adnan Khan brought this change] +- [Bevan Weiss brought this change] - HTTP3.md: clarify cargo build directory + CMake: don't complain about missing nroff - Cargo needs to be called from within the 'quiche' directory. + The curl_nroff_check() was always being called, and complaining if + *NROFF wasn't found, even when not making the manual. - Closes #5522 - -- user-agent.d: spell out what happens given a blank argument + Only check for nroff (and complain) if actually making the manual - Closes #5525 + Closes #5817 -- trailers: switch h1-trailer logic to use dynbuf +- [Brian Inglis brought this change] + + libtest/Makefile.am: add -no-undefined for libstubgss for Cygwin - In the continued effort to remove "manual" realloc schemes. + copy the LDFLAGS approach for adding same option with `libhostname` in + `libtest/Makefile.am`: - Closes #5524 - -- CURLINFO_ACTIVESOCKET.3: clarify the description + - init `libstubgss_la_LDFLAGS_EXTRA` variable, + - add option to variable inside conditional, + - use variable in `libstubgss_la_LDFLAGS` - Reported-by: Jay Satiro - Fixes #5299 - Closes #5520 - -- mailmap: Don J Olmstead + Fixes #5819 + Closes #5820 -- configure: only strip first -L from LDFLAGS - - In the logic that works out if a given OpenSSL path works, it stripped - off a possibly leading -L flag using an incorrect sed pattern which - would remove all instances of -L in the string, including if the path - itself contained that two-letter sequence! +- docs: clarify MAX_SEND/RECV_SPEED functionality - The same pattern was used and is now updated in multiple places. Now it - only removes -L if it starts the strings. + ... in particular what happens if the maximum speed limit is set to a + value that's smaller than the transfer buffer size in use. - Reported-by: Mohamed Osama - Fixes #5519 - Closes #5521 + Reported-by: Tomas Berger + Fixes #5788 + Closes #5813 -Peter Wu (4 Jun 2020) -- quiche: advertise draft 28 support +- test1140: compare stdout - Fix the verbose message while at it, quiche currently supports draft - 27 and draft 28 simultaneously. + To make problems more immediately obvious when tests fail. - Closes #5518 + Closes #5814 -Daniel Stenberg (4 Jun 2020) -- KNOWN_BUGS: RTSP authentication breaks without redirect support +- asyn-ares: correct some bad comments - Closes #4750 + Closes #5812 -Jay Satiro (4 Jun 2020) -- projects: Add crypt32.lib to dependencies for all OpenSSL configs - - Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO - need crypt32. - - Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. - - The changes that are in this commit were made by script. - - Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 +- [Emil Engler brought this change] + + docs: Add video link to docs/CONTRIBUTE.md - Closes https://github.com/curl/curl/pull/5516 + Closes #5811 -Marc Hoersken (3 Jun 2020) -- CI/macos: fix 'is already installed' errors by using bundle +- curl-config: ignore REQUIRE_LIB_DEPS in --libs output - Avoid failing CI builds due to nghttp2 being already installed. + Fixes a curl-config issue on cygwin by making sure REQUIRE_LIB_DEPS is + not considered for the --libs output. - Closes #5513 + Reported-by: ramsay-jones on github + Assisted-by: Brian Inglis and Ken Brown + Fixes #5793 + Closes #5808 -Daniel Stenberg (3 Jun 2020) -- altsvc: fix 'dsthost' may be used uninitialized in this function +- copyright: update/correct the year range on a few files -- RELEASE-NOTES: synced +- scripts/copyright.pl: ignore .muse files -- urldata: let the HTTP method be in the set.* struct - - When the method is updated inside libcurl we must still not change the - method as set by the user as then repeated transfers with that same - handle might not execute the same operation anymore! - - This fixes the libcurl part of #5462 +- [Emil Engler brought this change] + + multi: Remove 10-year old out-commented code - Test 1633 added to verify. + The code hasn't been touched since 2010-08-18 - Closes #5499 + Closes #5805 -- hostip: fix the memory-leak introduced in 67d2802 +- KNOWN_BUGS: A shared connection cache is not thread-safe - Fixes #5503 - Closes #5504 + Closes #4915 + Closes #5802 -- test970: make it require proxy support +- CONTRIBUTE: extend git commit message description - This test verifies the -w %json output and the test case includes a full - generated "blob". If there's no proxy support built into libcurl, it - will return an error for proxy related info variables and they will not - be included in the json, thus causing a mismatch and this test fails. + In particular how the first line works. - Reported-by: Marc Hörsken - Fixes #5501 - Closes #5502 - -- [Radoslav Georgiev brought this change] + Closes #5803 - examples/http2-down/upload: add error checks - - If `index.html` does not exist in the directory from which the example - is invoked, the fopen(upload, "rb") invocation in `setup` would fail, - returning NULL. This value is subsequently passed as the FILE* argument - of the `fread` invocation in the `read_callback` function, which is the - actual cause of the crash (apparently `fread` assumes that argument to - be non-null). - - In addition, mitigate some possible crashes of similar origin. - - Closes #5463 +- RELEASE-NOTES: synced -- [kotoriのねこ brought this change] +- [Stefan Yohansson brought this change] - examples/ephiperfifo: turn off interval when setting timerfd + transfer: move retrycount from connect struct to easy handle - Reported-by: therealhirudo on github - Fixes #5485 - Closes #5497 - -- [Saleem Abdulrasool brought this change] + This flag was applied to the connection struct that is released on + retry. These changes move the retry counter into Curl_easy struct that + lives across retries and retains the new connection. + + Reported-by: Cherish98 on github + Fixes #5794 + Closes #5800 - vtls: repair the build with `CURL_DISABLE_PROXY` +- libssh2: s/ssherr/sftperr/ - `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` - is enabled. Repair the build with that configuration. + The debug output used ssherr instead of sftperr which not only outputs + the wrong error code but also casues a warning on Windows. - Follow-up to f3d501dc67 + Follow-up to 7370b4e39f1 - Closes #5498 + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/7370b4e39f1390e701f5b68d910c619151daf72b#r41334700 + Closes #5799 -- transfer: remove k->str NULL check +- ftp: don't do ssl_shutdown instead of ssl_close - "Null-checking k->str suggests that it may be null, but it has already - been dereferenced on all paths leading to the check" - and it can't - legally be NULL at this point. Remove check. + The shutdown function is for downgrading a connection from TLS to plain, + and this is not requested here. - Detected by Coverity CID 1463884 + Have ssl_close reset the TLS connection state. - Closes #5495 - -Marc Hoersken (1 Jun 2020) -- select: always use Sleep in Curl_wait_ms on Win32 + This partially reverts commit f002c850d98d - Since Win32 almost always will also have USE_WINSOCK, - we can reduce complexity and always use Sleep there. + Reported-by: Rasmus Melchior Jacobsen + Reported-by: Denis Goleshchikhin + Fixes #5797 + +Marc Hoersken (9 Aug 2020) +- CI/azure: fix test outcome values and use latest API version - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + This makes sure that tests ignored or skipped are not shown + just in the category "Other", but with their correct state. - Follow up to #5343 - Closes #5489 + Closes #5796 -Daniel Stenberg (31 May 2020) -- conncache: download buffer needs +1 size for trailing zero +- CI/azure: show runtime stats to investigate slowness - Follow-up to c4e6968127e - Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 + Also avoid naming conflict of TFLAGS env and tflags variables. + + Closes #5776 -Marc Hoersken (31 May 2020) -- azure: use matrix strategy to avoid configuration redundancy +Daniel Stenberg (8 Aug 2020) +- TLS naming: fix more Winssl and Darwinssl leftovers - This also includes the following changes: + The CMake option is now called CMAKE_USE_SCHANNEL - - Use the same timeout for all jobs on Linux (60 minutes) - and Windows (90 minutes) - - Use CLI stable apt-get install -y instead of apt install - which warns about that and run apt-get update first - - Enable MQTT for Windows msys2 builds instead of - legacy msys1 builds - - Add ./configure --prefix parameter to the msys2 builds - - The MSYSTEM environment variable is now preset inside - the container images for the msys2 builds + The winbuild flag is USE_SCHANNEL - Note: on Azure Pipelines the matrix strategy is basically - just a simple list of job copies and not really a matrix. + The CI jobs and build scripts only use the new names and the new name + options - Closes #5468 + Tests now require 'Schannel' (when necessary) + + Closes #5795 -Daniel Stenberg (30 May 2020) -- build: disable more code/data when built without proxy support +- smtp_parse_address: handle blank input string properly - Added build to travis to verify + Closes #5792 + +- runtests: run the DICT server on a random port number - Closes #5466 + Removed support for -b (base port number) + + Closes #5783 -- url: alloc the download buffer at transfer start +- RELEASE-NOTES: synced + +- runtests: move the TELNET server to a dynamic port - ... and free it as soon as the transfer is done. It removes the extra - alloc when a new size is set with setopt() and reduces memory for unused - easy handles. + Rename the port variable to TELNETPORT to better match the existing + pattern. - In addition: the closure_handle now doesn't use an allocated buffer at - all but the smallest supported size as a stack based one. + Closes #5785 + +- ngtcp2: adapt to error code rename - Closes #5472 + Closes #5786 -- timeouts: change millisecond timeouts to timediff_t from time_t +- runtests: move the smbserver to use a dynamic port number - For millisecond timers we like timediff_t better. Also, time_t can be - unsigned so returning a negative value doesn't work then. + Closes #5782 + +- runtests: run the http2 tests on a random port number - Closes #5479 + Closes #5779 -Marc Hoersken (30 May 2020) -- select: add overflow checks for timeval conversions +- gtls: survive not being able to get name/issuer - Using time_t and suseconds_t if suseconds_t is available, - long on Windows (maybe others in the future) and int elsewhere. + Closes #5778 + +- runtests: move the gnutls-serv tests to a dynamic port - Also handle case of ULONG_MAX being greater or equal to INFINITE. + Affects test 320, 321, 322 and 324. - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Closes #5778 + +- runtests: support dynamicly base64 encoded sections in tests - Part of #5343 + This allows us to make test cases to use base64 at run-time and still + use and verify information determined at run-time, such as the IMAP test + server's port number in test 842. + + This change makes 12 tests run again that basically never ran since we + moved to dynamic port numbers. + + ftpserver.pl is adjusted to load test instructions and test number from + the preprocessed test file. + + FILEFORMAT.md now documents the new base64 encoding syntax. + + Reported-by: Marcel Raad + Fixes #5761 + Closes #5775 -- select: use timediff_t instead of time_t and int for timeout_ms +- curl.1: add a few missing valid exit codes - Make all functions in select.[ch] take timeout_ms as timediff_t - which should always be large enough and signed on all platforms - to take all possible timeout values and avoid type conversions. + 93 - 96 can be returned as well. - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Closes #5777 + +- TODO: Use multiple parallel transfers for a single download - Replaces #5107 and partially #5262 - Related to #5240 and #5286 - Closes #5343 + Closes #5774 -- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' +- TODO: Set the modification date on an uploaded file - GCC 10 warns about this with warning: implicit conversion - from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] + Closes #5768 + +- [Thomas M. DuBuisson brought this change] + + CI: Add muse CI config - Since 'expected_result' is not really of type 'CURLcode' and - it is not exposed in any way, we can just use 'SANITIZEcode'. + Closes #5772 + +- [Thomas M. DuBuisson brought this change] + + travis/script.sh: fix use of `-n' with unquoted envvar - Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad + Shellcheck tells us "-n doesn't work with unquoted arguments. quote or + use [[ ]]." - Closes #5476 + And testing shows: + + ``` + docker run --rm -it ubuntu bash + root@fe85ce156856:/# [ -n $DOES_NOT_EXIST ] && echo "I ran" + I ran + root@fe85ce156856:/# [ -n "$DOES_NOT_EXIST" ] && echo "I ran" + root@fe85ce156856:/# + ``` + + Closes #5773 -- tests/libtest: fix undefined reference to 'curlx_win32_fopen' +- h2: repair trailer handling - Since curl_setup.h now makes use of curlx_win32_fopen for Win32 - builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, - we need to include the relevant files for tests using fopen, - because the libtest sources are also including curl_setup.h + The previous h2 trailer fix in 54a2b63 was wrong and caused a + regression: it cannot deal with trailers immediately when read since + they may be read off the connection by the wrong 'data' owner. - Reviewed-by: Marcel Raad - Reviewed-by: Daniel Stenberg + This change reverts the logic back to gathering all trailers into a + single buffer, like before 54a2b63. - Follow up to #3784 (ffdddb45d9) - Closes #5475 + Reported-by: Tadej Vengust + Fixes #5663 + Closes #5769 -- appveyor: add non-debug plain autotools-based build +Viktor Szakats (3 Aug 2020) +- windows: disable Unix Sockets for old mingw - This should enable us to catch linking issues with the - testsuite early, like the one described/fixed in #5475. + Classic mingw and 10y+ old versions of mingw-w64 don't ship with + Windows headers having the typedef necessary for Unix Sockets + support, so try detecting these environments to disable this + feature. + + Ref: https://sourceforge.net/p/mingw-w64/mingw-w64/ci/cf6afc57179a5910621215f8f4037d406892072c/ Reviewed-by: Daniel Stenberg - Reviewed-by: Marcel Raad - Closes #5477 + Fixes #5674 + Closes #5758 + +Marcel Raad (3 Aug 2020) +- test1908: treat file as text + + Fixes the line endings on Windows. + + Closes https://github.com/curl/curl/pull/5767 + +- TrackMemory tests: ignore realloc and free in getenv.c + + These are only called for WIN32. + + Closes https://github.com/curl/curl/pull/5767 + +Daniel Stenberg (3 Aug 2020) +- tests/FILEFORMAT.md: mention %HTTP2PORT -Daniel Stenberg (29 May 2020) - RELEASE-NOTES: synced -- Revert "buildconf: use find -execdir" +- tlsv1.3.d. only for TLS-using connections - This partially reverts commit c712009838f44211958854de431315586995bc61. + ... and rephrase that "not all" TLS backends support it. - Keep the ares_ files removed but bring back the older way to run find, - to make it work with busybox's find, as apparently that's being used. + Closes #5764 + +- tls-max.d: this option is only for TLS-using connections - Reported-by: Max Peal - Fixes #5483 - Closes #5484 + Ref: #5763 + Closes #5764 -- server/sws: fix asan warning on use of uninitialized variable +Marcel Raad (2 Aug 2020) +- [Cameron Cawley brought this change] -- libssh2: improved error output for wrong quote syntax + tool_doswin: Simplify Windows version detection - Reported-by: Werner Stolz + Closes https://github.com/curl/curl/pull/5754 + +- [Cameron Cawley brought this change] + + win32: Add Curl_verify_windows_version() to curlx - Closes #5474 + Closes https://github.com/curl/curl/pull/5754 -- mk-lib1521: generate code for testing BLOB options as well +- runtests.pl: treat LibreSSL and BoringSSL as OpenSSL - Follow-up to cac5374298b3 + This makes the tests that require the OpenSSL feature also run for + those two compatible libraries. - Closes #5478 + Closes https://github.com/curl/curl/pull/5762 -- configure: repair the check if argv can be written to +Daniel Stenberg (1 Aug 2020) +- multi: Condition 'extrawait' is always true - Due to bad escaping of the test code, the test wouldn't build and thus - result in a negative test result, which would lead to the unconditional - assumption that overwriting the arguments doesn't work and thus curl - would never hide credentials given in the command line, even when it - would otherwise be possible. + Reported by Codacy. - Regression from commit 2d4c2152c (7.60.0) + Reviewed-by: Marcel Raad + Closes #5759 + +Marcel Raad (1 Aug 2020) +- openssl: fix build with LibreSSL < 2.9.1 - Reported-by: huzunhao on github - Fixes #5470 - Closes #5471 + `SSL_CTX_add0_chain_cert` and `SSL_CTX_clear_chain_certs` were + introduced in LibreSSL 2.9.1 [0]. + + [0] https://github.com/libressl-portable/openbsd/commit/0db809ee178457c8170abfae3931d7bd13abf3ef + + Closes https://github.com/curl/curl/pull/5757 -Peter Wu (28 May 2020) -- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes +Daniel Stenberg (1 Aug 2020) +- [Marc Aldorasi brought this change] + + multi_remove_handle: close unused connect-only connections - Otherwise the build might fail due to missing source files, as - demonstrated by the recent keylog.c addition on an existing build dir. + Previously any connect-only connections in a multi handle would be kept + alive until the multi handle was closed. Since these connections cannot + be re-used, they can be marked for closure when the associated easy + handle is removed from the multi handle. - Closes #5469 + Closes #5749 -Daniel Stenberg (28 May 2020) -- urldata: fix comments: Curl_done() is called multi_done() now +- checksrc: invoke script with -D to find .checksrc proper - ... since 575e885db + Without the -D command line option, checksrc.pl won't know which + directory to load the ".checksrc" file from when building out of the + source tree. + + Reported-by: Marcel Raad + Fixes #5715 + Closes #5755 -Peter Wu (27 May 2020) -- ngtcp2: use common key log routine for better thread-safety +- [Carlo Marcelo Arenas Belón brought this change] + + buildconf: retire ares buildconf invocation - Tested with ngtcp2 built against the OpenSSL library. Additionally - tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + no longer needed after 4259d2df7dd95637a4b1e3fb174fe5e5aef81069 + +- [Carlo Marcelo Arenas Belón brought this change] + + buildconf: excempt defunct reference to ACLOCAL_FLAGS - The TLS backend (independent of QUIC) may or may not already have opened - the keylog file before. Therefore Curl_tls_keylog_open is always called - to ensure the file is open. + retired with 09f278121e815028adb24d228d8092fc6cb022aa but kept around as + the name is generic enough that it might be in use and relied upon from + the environment. -- wolfssl: add SSLKEYLOGFILE support +- [Carlo Marcelo Arenas Belón brought this change] + + buildconf: avoid array concatenation in die() - Tested following the same curl and tshark commands as in commit - "vtls: Extract and simplify key log file handling from OpenSSL" using - WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with - `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. + reported as error SC2145[1] by shellcheck, but not expected to cause + any behavioural differences otherwise. - Full support for this feature requires certain wolfSSL build options, - see "Availability note" in lib/vtls/wolfssl.c for details. + [1] https://github.com/koalaman/shellcheck/wiki/SC2145 - Closes #5327 + Closes #5701 -- vtls: Extract and simplify key log file handling from OpenSSL +- travis: add ppc64le and s390x builds - Create a set of routines for TLS key log file handling to enable reuse - with other TLS backends. Simplify the OpenSSL backend as follows: + Closes #5752 + +Marc Hoersken (31 Jul 2020) +- connect: remove redundant message about connect failure - - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. - - Do not perform dynamic memory allocation when preparing a log entry. - Unless the TLS specifications change we can suffice with a reasonable - fixed-size buffer. - - Simplify state tracking when SSL_CTX_set_keylog_callback is - unavailable. My original sslkeylog.c code included this tracking in - order to handle multiple calls to SSL_connect and detect new keys - after renegotiation (via SSL_read/SSL_write). For curl however we can - be sure that a single master secret eventually becomes available - after SSL_connect, so a simple flag is sufficient. An alternative to - the flag is examining SSL_state(), but this seems more complex and is - not pursued. Capturing keys after server renegotiation was already - unsupported in curl and remains unsupported. + Reviewed-by: Daniel Stenberg - Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f - (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) - against an OpenSSL 1.1.1f server configured with: + Closes #5708 + +- tests/sshserver.pl: fix compatibility with OpenSSH for Windows - # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 - openssl s_server -www -tls1 - # Likewise, but fail the server handshake. - openssl s_server -www -tls1 -Verify 2 - # TLS 1.3 test. No need to test the failing server handshake. - openssl s_server -www -tls1_3 + Follow up to #5721 + +- CI/azure: install libssh2 for use with msys2-based builds - Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly - written using Wireshark. For the first and third case, expect four - matches per connection (decrypted Server Finished, Client Finished, HTTP - Request, HTTP Response). For the second case where the handshake fails, - expect a decrypted Server Finished only. + This enables building and running the SFTP tests. + Unfortunately OpenSSH for Windows does not support SCP (yet). - tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ - -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ - -dtls.port==4433,http -ohttp.desegment_body:FALSE \ - -Y 'tls.handshake.verify_data or http' + Reviewed-by: Daniel Stenberg - A single connection can easily be identified via the `tcp.stream` field. + Closes #5721 -Daniel Stenberg (27 May 2020) -- FILEFORMAT: add more features that tests can depend on +- CI/azure: increase Windows job timeout once again + + Avoid aborted jobs due to performance issues on Azure DevOps. + + Reviewed-by: Daniel Stenberg + Reviewed-by: Jay Satiro + + Closes #5738 -- [Michael Kaufmann brought this change] +Jay Satiro (30 Jul 2020) +- TODO: Schannel: 'Add option to allow abrupt server closure' + + We should offer an option to allow abrupt server closures (server closes + SSL transfer without sending a known termination point such as length of + transfer or close_notify alert). Abrupt server closures are usually + because of misconfigured or very old servers. + + Closes https://github.com/curl/curl/issues/4427 - transfer: close connection after excess data has been read +- url: fix CURLU and location following - For HTTP 1.x, it's a protocol error when the server sends more bytes - than announced. If this happens, don't reuse the connection, because the - start position of the next response is undefined. + Prior to this change if the user set a URL handle (CURLOPT_CURLU) it was + incorrectly used for the location follow, resulting in infinite requests + to the original location. - Closes #5440 + Reported-by: sspiri@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/5709 + Closes https://github.com/curl/curl/pull/5713 -- [Estanislau Augé-Pujadas brought this change] +Daniel Stenberg (30 Jul 2020) +- RELEASE-NOTES: synced - Revert "ssh: ignore timeouts during disconnect" +- [divinity76 brought this change] + + docs: add date of 7.20 to CURLM_CALL_MULTI_PERFORM mentions - This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in - curl 7.54.1. + it helps make it obvious that most developers don't have to care about + the CURLM_CALL_MULTI_PERFORM value (last release using it is nearly 11 + years old, November 4 2009) - Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html - Closes #5465 + Closes #5744 -- urldata: connect related booleans live in struct ConnectBits +Jay Satiro (29 Jul 2020) +- tool_cb_wrt: fix outfile mode flags for Windows - And remove a few unused booleans! + - Use S_IREAD and S_IWRITE mode permission flags to create the file + on Windows instead of S_IRUSR, S_IWUSR, etc. - Closes #5461 + Windows only accepts a combination of S_IREAD and S_IWRITE. It does not + acknowledge other combinations, for which it may generate an assertion. + + This is a follow-up to 81b4e99 from yesterday, which improved the + existing file check with -J. + + Ref: https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/open-wopen#remarks + Ref: https://github.com/curl/curl/pull/5731 + + Closes https://github.com/curl/curl/pull/5742 -- hostip: on macOS avoid DoH when given a numerical IP address +Daniel Stenberg (28 Jul 2020) +- checksrc: ban gmtime/localtime - When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that - numerical IP addresses still need to get "resolved" - but not with DoH. + They're not thread-safe so they should not be used in libcurl code. - Reported-by: Viktor Szakats - Fixes #5454 - Closes #5459 + Explictly enabled when deemed necessary and in examples and tests + + Reviewed-by: Nicolas Sterchele + Closes #5732 -- ngtcp2: cleanup memory when failing to connect +- transfer: fix data_pending for builds with both h2 and h3 enabled - Reported-by: Peter Wu - Fixes #5447 (the ngtcp2 side of it) - Closes #5451 + Closes #5734 -- quiche: clean up memory properly when failing to connect +- curl_multi_setopt: fix compiler warning "result is always false" - Addresses the quiche side of #5447 - Reported-by: Peter Wu - Closes #5450 + On systems with 32 bit long the expression is always false. Avoid + the warning. + + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/61a08508f6a458fe21bbb18cd2a9bac2f039452b#commitcomment-40941232 + Closes #5736 -- cleanup: use a single space after equals sign in assignments +- curl: improve the existing file check with -J + + Previously a file that isn't user-readable but is user-writable would + not be properly avoided and would get overwritten. + + Reported-by: BrumBrum on hackerone + Assisted-by: Jay Satiro + Bug: https://hackerone.com/reports/926638 + Closes #5731 -- url: accept "any length" credentials for proxy auth +- [Jonathan Nieder brought this change] + + multi: update comment to say easyp list is linear - They're only limited to the maximum string input restrictions, not to - 256 bytes. + Since 09b9fc900 (multi: remove 'Curl_one_easy' struct, phase 1, + 2013-08-02), the easy handle list is not circular but ends with + ->next pointing to NULL. - Added test 1178 to verify + Reported-by: Masaya Suzuki + Closes #5737 + +- CURLOPT_NOBODY.3: fix the syntax for referring to options - Reported-by: Will Roberts - Fixes #5448 - Closes #5449 + As test 1140 fails otherwise! + + Follow-up to e1bac81cc815 -- [Maksim Stsepanenka brought this change] +- ngtcp2: store address in sockaddr_storage + + Reported-by: Tatsuhiro Tsujikawa + Closes #5733 - test1167: fixes in badsymbols.pl +- CURLOPT_NOBODY.3: clarify what setting to 0 means - Closes #5442 + ... and mention that HTTP with other methods than HEAD might get a body and + there's no option available to stop that. + + Closes #5729 -- altsvc: fix parser for lines ending with CRLF +- setopt: unset NOBODY switches to GET if still HEAD + + Unsetting CURLOPT_NOBODY with 0L when doing HTTP has no documented + action but before 7.71.0 that used to switch back to GET and with this + change (assuming the method is still set to HEAD) this behavior is + brought back. + + Reported-by: causal-agent on github + Fixes #5725 + Closes #5728 + +- [Ehren Bendler brought this change] + + configure: cleanup wolfssl + pkg-config conflicts when cross compiling. + + Also choose a different wolfSSL function to test for NTLM support. + + Fixes #5605 + Closes #5682 + +- configure: show zstd "no" in summary when built without it + + Reported-by: Marc Hörsken + Fixes #5720 + Closes #5730 + +- quiche: handle calling disconnect twice + + Reported-by: lilongyan-huawei on github + Fixes #5726 + Closes #5727 + +- [Nicolas Sterchele brought this change] + + getinfo: reset retry-after value in initinfo + + - Avoid re-using retry_after value from preceding request + - Add libtest 3010 to verify + + Reported-by: joey-l-us on github + Fixes #5661 + Closes #5672 + +Marcel Raad (27 Jul 2020) +- WIN32: stop forcing narrow-character API + + Except where the results are only used for character output. + getenv is not touched because it's part of the public API, and having + it return UTF-8 instead of ANSI would be a breaking change. + + Fixes https://github.com/curl/curl/issues/5658 + Fixes https://github.com/curl/curl/issues/5712 + Closes https://github.com/curl/curl/pull/5718 + +Jay Satiro (27 Jul 2020) +- [Tobias Stoeckmann brought this change] + + mprintf: Fix stack overflows + + Stack overflows can occur with precisions for integers and floats. + + Proof of concepts: + - curl_mprintf("%d, %.*1$d", 500, 1); + - curl_mprintf("%d, %+0500.*1$f", 500, 1); + + Ideally, compile with -fsanitize=address which makes this undefined + behavior a bit more defined for debug purposes. - Fixed the alt-svc parser to treat a newline as end of line. + The format strings are valid. The overflows occur due to invalid + arguments. If these arguments are variables with contents controlled + by an attacker, the function's stack can be corrupted. - The unit tests in test 1654 were done without CRLF and thus didn't quite - match the real world. Now they use CRLF as well. + Also see CVE-2016-9586 which partially fixed the float aspect. - Reported-by: Peter Wu - Assisted-by: Peter Wu - Assisted-by: Jay Satiro - Fixes #5445 - Closes #5446 - -Viktor Szakats (25 May 2020) -- all: fix codespell errors + Signed-off-by: Tobias Stoeckmann - Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5452 + Closes https://github.com/curl/curl/pull/5722 -Peter Wu (25 May 2020) -- ngtcp2: fix build with current ngtcp2 master implementing draft 28 +- [Tobias Stoeckmann brought this change] + + mprintf: Fix dollar string handling - Based on client.cc changes from ngtcp2. Tested with current git master, - ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. + Verify that specified parameters are in range. If parameters are too + large, fail early on and avoid out of boundary accesses. - Fixes #5444 - Closes #5443 - -Daniel Stenberg (25 May 2020) -- RELEASE-NOTES: synced + Also do not read behind boundaries of illegal format strings. - moved the new setopts up to a "change" - -- RELEASE-NOTES: synced - -- copyright: updated year ranges out of sync + These are defensive measures since it is expected that format strings + are well-formed. Format strings should not be modifiable by user + input due to possible generic format string attacks. - ... and whitelisted a few more files in the the copyright.pl script. - -- [Gilles Vollant brought this change] + Closes https://github.com/curl/curl/pull/5722 - setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency +Daniel Stenberg (26 Jul 2020) +- ntlm: free target_info before (re-)malloc - Closes #5431 - -- curl: remove -J "informational" written on stdout + OSS-Fuzz found a way this could get called again with the pointer still + pointing to a malloc'ed memory, leading to a leak. - curl would previously show "curl: Saved to filename 'name from header'" - if -J was used and a name was picked from the Content-Disposition - header. That output could interfer with other stdout output, such as -w. + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24379 - This commit removes that output line. - Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html - Reported-by: Коваленко Анатолий Викторович - Closes #5435 + Closes #5724 -Peter Wu (22 May 2020) -- travis: simplify quiche build instructions wrt boringssl +Marcel Raad (26 Jul 2020) +- CI/macos: set minimum macOS version - quiche builds boringssl as static library, reuse that instead of - building another shared library. + This enables some deprecation warnings. + Previously, autotools defaulted to 10.8. - Closes #5438 + Closes https://github.com/curl/curl/pull/5723 -- configure: fix pthread check with static boringssl +Daniel Stenberg (26 Jul 2020) +- RELEASE-NOTES: synced + +Marcel Raad (25 Jul 2020) +- CI/macos: enable warnings as errors for CMake builds - A shared boringssl/OpenSSL library requires -lcrypto only for linking. - A static build additionally requires `-ldl -lpthread`. In the latter - case `-lpthread` is added to LIBS which prevented `-pthread` from being - added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. + Closes https://github.com/curl/curl/pull/5716 -Daniel Stenberg (22 May 2020) -- Revert "sendf: make failf() use the mvsnprintf() return code" +- CMake: fix test for warning suppressions - This reverts commit 74623551f306990e70c7c5515b88972005604a74. + GCC doesn't warn for unknown `-Wno-` options, except if there are other + warnings or errors [0]. This was problematic with `CURL_WERROR` as that + warning-as-error cannot be suppressed. Notably, this always happened + with `-Wno-pedantic-ms-format` when not targeting Windows. So test for + the positive form of the warning instead, which should always result in + a diagnostic if unknown. - Instead mark the function call with (void). Getting the return code and - using it instead triggered Coverity warning CID 1463596 because - snprintf() can return a negative value... + [0] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html - Closes #5441 + Closes https://github.com/curl/curl/pull/5714 -- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' +Jay Satiro (23 Jul 2020) +- curl.h: update CURLINFO_LASTONE - Reported-by: Billyzou0741326 on github - Fixes #5432 - Closes #5436 - -- tests/server/util.h: add extern to silence compiler warning + CURLINFO_LASTONE should have been updated when + CURLINFO_EFFECTIVE_METHOD was added. - Follow-up from a3b0699d5c1 + Reported-by: xwxbug@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/5711 -- typecheck-gcc.h: fix the OFF_T check +Marc Hoersken (22 Jul 2020) +- CI/azure: unconditionally enable warnings-as-errors with autotools - The option number also needs to be less than CURLOPTTYPE_BLOB. + Reviewed-by: Marcel Raad - Follow-up to cac5374298 - Reported-by: Jeroen Ooms - Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 + Follow up to #5694 + Closes #5706 -- TODO: --dry-run +Marcel Raad (21 Jul 2020) +- doh: remove redundant cast - Closes #5426 + Closes https://github.com/curl/curl/pull/5704 -- TODO: Ratelimit or wait between serial requests +- CI/macos: unconditionally enable warnings-as-errors with autotools - Closes #5406 - -- tool_paramhlp: fixup C89 mistake + Previously, warnings were only visible in the output for most jobs. - Follow-up to c5f0a9db22. - -- [Siva Sivaraman brought this change] + Closes https://github.com/curl/curl/pull/5694 - tool_paramhlp: fixed potentially uninitialized strtol() variable +- util: silence conversion warnings - Seems highly unlikely to actually be possible, but better safe than - sorry. + timeval::tv_usec might be a 32-bit integer and timespec::tv_nsec might + be a 64-bit integer. This is the case when building for recent macOS + versions, for example. Just treat tv_usec as an int, which should + hopefully always be sufficient on systems with + `HAVE_CLOCK_GETTIME_MONOTONIC`. - Closes #5417 - -- [Siva Sivaraman brought this change] + Closes https://github.com/curl/curl/pull/5695 - tool_operate: fixed potentially uninitialized variables +- md(4|5): don't use deprecated macOS functions - ... in curl_easy_getinfo() calls. They're harmless but clearing the - variables makes the code safer and comforts the reader. + They are marked as deprecated for -mmacosx-version-min >= 10.15, + which might result in warnings-as-errors. - Closes #5416 + Closes https://github.com/curl/curl/pull/5695 -- sha256: move assign to the declaration line +Daniel Stenberg (18 Jul 2020) +- strdup: remove the odd strlen check - Follow-up to fae30656. Should've been squashed with that commit... + It confuses code analyzers with its use of -1 for unsigned value. Also, + a check that's not normally used in strdup() code - and not necessary. + + Closes #5697 -- [Siva Sivaraman brought this change] +- [Alessandro Ghedini brought this change] - sha256: fixed potentially uninitialized variable + travis: update quiche builds for new boringssl layout - Closes #5414 - -- sendf: make failf() use the mvsnprintf() return code + This is required after https://github.com/cloudflare/quiche/pull/593 + moved BoringSSL around slightly. - ... and avoid a strlen() call. Fixes a MonocleAI warning. + This also means that Go is not needed to build BoringSSL anymore (the + one provided by quiche anyway). - Reported-by: MonocleAI - Fixes #5413 - Closes #5420 + Closes #5691 -- hostip: make Curl_printable_address not return anything +Marcel Raad (17 Jul 2020) +- configure: allow disabling warnings - It was not used much anyway and instead we let it store a blank buffer - in case of failure. + When using `--enable-warnings`, it was not possible to disable warnings + via CFLAGS that got explicitly enabled. Now warnings are not enabled + anymore if they are explicitly disabled (or enabled) in CFLAGS. This + works for at least GCC, clang, and TCC as they have corresponding + `-Wno-` options for every warning. - Reported-by: MonocleAI - Fixes #5411 - Closes #5418 + Closes https://github.com/curl/curl/pull/5689 -- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) +Daniel Stenberg (16 Jul 2020) +- ngtcp2: adjust to recent sockaddr updates - They're done on purpose, make that visible in the code. - Reported-by: MonocleAI - Fixes #5412 - Closes #549 + Closes #5690 -- TODO: forbid TLS post-handshake auth and do TLS record padding +- page-header: provide protocol details in the curl.1 man page - Closes #5396 - Closes #5398 - -- RELEASE-NOTES: synced + Add protocol and version specific information about all protocols curl + supports. + + Fixes #5679 + Reported-by: tbugfinder on github + Closes #5686 -- dynbuf: return NULL when there's no buffer length +Daniel Gustafsson (16 Jul 2020) +- docs: Update a few leftover mentions of DarwinSSL - ... as returning a "" is not a good idea as the string is supposed to be - allocated and returning a const string will cause issues. + Commit 76a9c3c4be10b3d4d379d5b23ca76806bbae536a renamed DarwinSSL to the + more correct/common name Secure Transport, but a few mentions in the docs + remained. - Reported-by: Brian Carpenter - Follow-up to ed35d6590e72c - Closes #5405 + Closes #5688 + Reviewed-by: Daniel Stenberg -Peter Wu (16 May 2020) -- travis: upgrade to bionic, clang-9, improve readability +Daniel Stenberg (16 Jul 2020) +- file2memory: use a define instead of -1 unsigned value - Changes, partially to reduce build failures from external dependencies: - - Upgrade Ubuntu and drop unnecessary third-party repos. - - Properly clone apt config to ensure retries. - - Upgrade to clang-9 from the standard repos. - - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey - fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use - focal everywhere yet since Travis CI has not documented this option. - In focal, python-impacket (Py2.7) has been removed, leaving only - python3-impacket. Since it is only needed for SMB tests and not SSH, - skip it for the libssh job since it might need more work. - - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. + ... to use the maximum value for 'size_t' when detecting integer overflow. + Changed the limit to max/4 as already that seems unreasonably large. - Non-functional cleanups: - - Simplify test matrix, drop redundant os and compiler keys. - - Deprecation fixes: remove sudo, rename matrix -> jobs. - - Every job has an 'env' key, put this key first in a list item. + Codacy didn't like the previous approach. - Closes #5370 + Closes #5683 -- travis: whitespace-only changes for consistency +- CURL_PUSH_ERROROUT: allow the push callback to fail the parent stream - Automatically apply a consistent indentation with: + ... by adding support for a new dedicated return code. - python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' + Suggested-by: Jonathan Cardoso + Assisted-by: Erik Johansson + URL: https://curl.haxx.se/mail/lib-2020-06/0099.html + Closes #5636 + +- [Baruch Siach brought this change] + + nss: fix build with disabled proxy support - followed by manually re-indenting three comments. + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - Closes #5370 + Closes #5667 -- CMake: add libssh build support - - Closes #5372 +- test1139: make it display the difference on test failures -Daniel Stenberg (15 May 2020) -- KNOWN_BUGS: wolfssh: publickey auth doesn't work +- test1119: verify stdout in the test - Closes #4820 - -- KNOWN_BUGS: OS400 port requires deprecated IBM library + So that failures will be displayed in the terminal, as it makes test failures + visually displayed easier and faster. - Closes #5176 - -- [Vyron Tsingaras brought this change] + Closes #5644 - http2: keep trying to send pending frames after req.upload_done +- curl: add %{method} to the -w variables - Fixes #1410 - Closes #5401 + Gets the CURLINFO_EFFECTIVE_METHOD from libcurl. + + Added test 1197 to verify. -- [Gilles Vollant brought this change] +- CURLINFO_EFFECTIVE_METHOD: added + + Provide the HTTP method that was used on the latest request, which might + be relevant for users when there was one or more redirects involved. + + Closes #5511 - setopt: support certificate options in memory with struct curl_blob +Viktor Szakats (14 Jul 2020) +- windows: add unicode to feature list - This change introduces a generic way to provide binary data in setopt - options, called BLOBs. + Reviewed-by: Marcel Raad + Reviewed-by: Marc Hörsken - This change introduces these new setopts: + Closes #5491 + +Daniel Stenberg (14 Jul 2020) +- multi: remove two checks always true - CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, - CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. + Detected by Codacy + Closes #5676 + +Marc Hoersken (13 Jul 2020) +- workflows: limit what branches to run CodeQL on + + Align CodeQL action with existing CI actions: + - Update branch filter to avoid duplicate CI runs. + - Shorten workflow name due to informative job name. Reviewed-by: Daniel Stenberg - Closes #5357 + + Closes #5660 -- source cleanup: remove all custom typedef structs +- appveyor: collect libcurl.dll variants with prefix or suffix - - Stick to a single unified way to use structs - - Make checksrc complain on 'typedef struct {' - - Allow them in tests, public headers and examples + On some platforms libcurl is build with a platform-specific + prefix and/or a version number suffix. - - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually - typedef different types/structs depending on build conditions. + Assisted-by: Jay Satiro - Closes #5338 + Closes #5659 -- travis: remove the .checksrc fiddling +Daniel Stenberg (12 Jul 2020) +- [ihsinme brought this change] -- ftp: make domore_getsock() return the secondary socket properly + socks: use size_t for size variable - Previously, after PASV and immediately after the data connection has - connected, the function would only return the control socket to wait for - which then made the data connection simply timeout and not get polled - correctly. This become obvious when running test 1631 and 1632 event- - based. + Use the unsigned type (size_t) in the arithmetic of pointers. In this + context, the signed type (ssize_t) is used unnecessarily. + + Authored-by: ihsinme on github + Closes #5654 -- test1632: verify FTP through HTTPS-proxy with connection re-use +- RELEASE-NOTES: synced + + ... and bumped to 7.72.0 as the next release version number -- test1631: verify FTP download through HTTPS-proxy +- [Gilles Vollant brought this change] -- sws: as last resort, get test number from server cmd file + content_encoding: add zstd decoding support - If it can't be found in the request. Also support --cmdfile to set it to - a custom file name. + include zstd curl patch for Makefile.m32 from vszakats + and include Add CMake support for zstd from Peter Wu - runtests.pl always writes this file with the test number in it since a - while back. + Helped-by: Viktor Szakats + Helped-by: Peter Wu + Closes #5453 -- ftp: shut down the secondary connection properly when SSL is used +- asyn.h: remove the Curl_resolver_getsock define - Reported-by: Neal Poole - Fixes #5340 - Closes #5385 + - not used + - used the wrong number of arguments + - confused the Codeacy code analyzer + + Closes #5647 -Marcel Raad (14 May 2020) -- KNOWN_BUGS: adapt 5.5 to recent changes +- [Nicolas Sterchele brought this change] + + configure.ac: Sort features name in summary - It only applies to non-Unicode builds now. - Also merge 5.10 into it as it's effectively a duplicate. + - Same as protocols - Closes https://github.com/curl/curl/pull/3784 + Closes #5656 -- curl_setup: support Unicode functions to open files on Windows +- [Matthias Naegler brought this change] + + cmake: fix windows xp build - Use them only if `_UNICODE` is defined, in which case command-line - arguments have been converted to UTF-8. + Reviewed-by: Marcel Raad + Closes #5662 + +- ngtcp2: update to modified qlog callback prototype - Closes https://github.com/curl/curl/pull/3784 + Closes #5675 -- tool: support UTF-16 command line on Windows +- transfer: fix memory-leak with CURLOPT_CURLU in a duped handle - - use `wmain` instead of `main` when `_UNICODE` is defined [0] - - define `argv_item_t` as `wchar_t *` in this case - - use the curl_multibyte gear to convert the command-line arguments to - UTF-8 + Added test case 674 to reproduce and verify the bug report. - This makes it possible to pass parameters with characters outside of - the current locale on Windows, which is required for some tests, e.g. - the IDN tests. Out of the box, this currently only works with the - Visual Studio project files, which default to Unicode, and winbuild - with the `ENABLE_UNICODE` option. + Fixes #5665 + Reported-by: NobodyXu on github + Closes #5673 + +- [Baruch Siach brought this change] + + bearssl: fix build with disabled proxy support - [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + Avoid reference to fields that do not exist when CURL_DISABLE_PROXY is + defined. - Ref: https://github.com/curl/curl/issues/3747 - Closes https://github.com/curl/curl/pull/3784 + Reviewed-by: Nicolas Sterchele + Closes #5666 -- curl_multibyte: add to curlx +- RELEASE-NOTES: synced + +Jay Satiro (11 Jul 2020) +- [Carlo Marcelo Arenas Belón brought this change] + + cirrus-ci: upgrade 11-STABLE to 11.4 - This will also be needed in the tool and tests. + Meant to be the last of the 11 series and so make sure that all + other references reflect all 11 versions so they can be retired + together later. - Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 - Closes https://github.com/curl/curl/pull/3784 + Closes https://github.com/curl/curl/pull/5668 -Daniel Stenberg (14 May 2020) -- url: make the updated credentials URL-encoded in the URL +- [Filip Salomonsson brought this change] + + CURLINFO_CERTINFO.3: fix typo - Found-by: Gregory Jefferis - Reported-by: Jeroen Ooms - Added test 1168 to verify. Bug spotted when doing a redirect. - Bug: https://github.com/jeroen/curl/issues/224 - Closes #5400 + Closes https://github.com/curl/curl/pull/5655 -- tests: add https-proxy support to the test suite +Daniel Stenberg (4 Jul 2020) +- http2: only do the *done() cleanups for HTTP - Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like - HTTP proxy but with a full TLS connection to the proxy. + Follow-up to ef86daf4d3 - Closes #5399 - -- mailmap: James Fuller + Closes #5650 + Fixes #5646 -- [Major_Tom brought this change] +- [Alex Kiernan brought this change] - vauth/cleartext: fix theoretical integer overflow + gnutls: repair the build with `CURL_DISABLE_PROXY` - Fix theoretical integer overflow in Curl_auth_create_plain_message. + `http_proxy`/`proxy_ssl`/`tunnel_proxy` will not be available in `conn` + if `CURL_DISABLE_PROXY` is enabled. Repair the build with that + configuration. - The security impact of the overflow was discussed on hackerone. We - agreed this is more of a theoretical vulnerability, as the integer - overflow would only be triggerable on systems using 32-bits size_t with - over 4GB of available memory space for the process. + Signed-off-by: Alex Kiernan + Closes #5645 + +Alex Kiernan (3 Jul 2020) +- gnutls: Fetch backend when using proxy - Closes #5391 + Fixes: 89865c149 ("gnutls: remove the BACKEND define kludge") + Signed-off-by: Alex Kiernan -Jay Satiro (13 May 2020) -- curl.1: Quote globbed URLs +Daniel Stenberg (3 Jul 2020) +- [Laramie Leavitt brought this change] + + http2: close the http2 connection when no more requests may be sent - - Quote the globbing example URLs that contain characters [] {} since - otherwise they may be interpreted as shell metacharacters. + Well-behaving HTTP2 servers send two GOAWAY messages. The first + message is a warning that indicates that the server is going to + stop accepting streams. The second one actually closes the stream. + + nghttp2 reports this state (and the other state of no more stream + identifiers) via the call nghttp2_session_check_request_allowed(). + In this state the client should not create more streams on the + session (tcp connection), and in curl this means that the server + has requested that the connection is closed. + + It would be also be possible to put the connclose() call into the + on_http2_frame_recv() function that triggers on the GOAWAY message. + + This fixes a bug seen when the client sees the following sequence of + frames: + + // advisory GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = -1] + ... some additional frames - Bug: https://github.com/curl/curl/issues/5388 - Reported-by: John Simpson + // final GOAWAY + HTTP2 GOAWAY [stream-id = 0, promised-stream-id = N ] - Closes https://github.com/curl/curl/pull/5394 - -Daniel Stenberg (14 May 2020) -- checksrc: enhance the ASTERISKSPACE and update code accordingly + Before this change, curl will attempt to reuse the connection even + after the last stream, will encounter this error: - Fine: "struct hello *world" + * Found bundle for host localhost: 0x5595f0a694e0 [can multiplex] + * Re-using existing connection! (#0) with host localhost + * Connected to localhost (::1) port 10443 (#0) + * Using Stream ID: 9 (easy handle 0x5595f0a72e30) + > GET /index.html?5 HTTP/2 + > Host: localhost:10443 + > user-agent: curl/7.68.0 + > accept: */* + > + * stopped the pause stream! + * Connection #0 to host localhost left intact + curl: (16) Error in the HTTP2 framing layer - Not fine: "struct hello* world" (and variations) + This error may posion the connection cache, causing future requests + which resolve to the same curl connection to go through the same error + path. - Closes #5386 + Closes #5643 -- docs/options-in-versions: which version added each cmdline option +- ftpserver: don't verify SMTP MAIL FROM names - Added test 971 to verify that the list is in sync with the files in - cmdline-opts. The check also verifies that .d-files that uses Added: - specify the same version number as the options-in-versions file does. + Rely on tests asking the names to get refused instead - test servers + should be as dumb as possible. Edited test 914, 955 and 959 accordingly. - Closes #5381 + Closes #5639 -- docs: unify protocol lists +- curl_version_info.3: CURL_VERSION_KERBEROS4 is deprecated - We boast support for 25 transfer protocols. Make sure the lists are - consistent + This came up in #5640. It make sense to clarify this in the docs! - Closes #5384 + Reminded-by: Kamil Dudka + Closes #5642 -- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN +Kamil Dudka (3 Jul 2020) +- tool_getparam: make --krb option work again - ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. + It was disabled by mistake in commit curl-7_37_1-23-ge38ba4301. - Reported-by: Michael Kaufmann - Fixes #5374 - Closes #5376 - -- tls13-ciphers.d: shorten the Arg - -- sasl-authzid.d: add Arg: and shorten the desc + Bug: https://bugzilla.redhat.com/1833193 + Closes #5640 -- cert-type.d: mention the available types in the desc +Daniel Stenberg (2 Jul 2020) +- [Jeremy Maitin-Shepard brought this change] -- tool: shorten 3 --help descriptions + http2: fix nghttp2_strerror -> nghttp2_http2_strerror in debug messages - --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort + Confusingly, nghttp2 has two different error code enums: - gen.pl already warned about these lines but we didn't listen + - nghttp2_error, to be used with nghttp2_strerror + - nghttp2_error_code, to be used with nghttp2_http2_strerror - Closes #5379 + Closes #5641 -- configure: the wolfssh backend does not provide SCP +Marcel Raad (2 Jul 2020) +- url: silence MSVC warning - Closes #5387 - -- RELEASE-NOTES: synced - -- url: reject too long input when parsing credentials + Since commit f3d501dc678, if proxy support is disabled, MSVC warns: + url.c : warning C4701: potentially uninitialized local variable + 'hostaddr' used + url.c : error C4703: potentially uninitialized local pointer variable + 'hostaddr' used - Since input passed to libcurl with CURLOPT_USERPWD and - CURLOPT_PROXYUSERPWD circumvents the regular string length check we have - in Curl_setstropt(), the input length limit is enforced in - Curl_parse_login_details too, separately. + That could actually only happen if both `conn->bits.proxy` and + `CURL_DISABLE_PROXY` were enabled. + Initialize it to NULL to silence the warning. - Reported-by: Thomas Bouzerar - Closes #5383 - -- list-only.d: this option existed already in 4.0 + Closes https://github.com/curl/curl/pull/5638 -Jay Satiro (12 May 2020) -- retry-all-errors.d: Shorten the summary line - - Follow-up to b995bb5 from a few moments ago. - - Reported-by: Daniel Stenberg - - Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 +Daniel Stenberg (1 Jul 2020) +- RELEASE-NOTES: synced -- [denzor brought this change] +Version 7.71.1 (30 Jun 2020) - easy: fix dangling pointer on easy_perform fail - - Closes https://github.com/curl/curl/pull/5363 +Daniel Stenberg (30 Jun 2020) +- RELEASE-NOTES: curl 7.71.1 -- tool: Add option --retry-all-errors to retry on any error - - The "sledgehammer" of retrying. - - Closes https://github.com/curl/curl/pull/5185 +- THANKS: add contributors to 7.71.1 -Daniel Stenberg (12 May 2020) -- [James Le Cuirot brought this change] +- scripts/copyright.pl: skip .dcignore - libcurl.pc: Merge Libs.private into Libs for static-only builds - - A project being built entirely statically will call pkg-config with - --static, which utilises the Libs.private field. Conversely it will - not use --static when not being built entirely statically, even if - there is only a static build of libcurl available. This will most - likely cause the build to fail due to underlinking unless we merge the - Libs fields. +- Revert "multi: implement wait using winsock events" - Consider that this is what the Meson build system does when it - generates pkg-config files. + This reverts commit 8bc25c590e530de87595d1bb3577f699eb1309b9. - I have also reflected this in the --libs argument of curl-config even - though REQUIRE_LIB_DEPS always seems to be "yes" anyway. + That commit (from #5397) introduced a regression in 7.71.0. - Closes #5373 - -- [Peter Wu brought this change] + Reported-by: tmkk on github + Fixes #5631 + Closes #5632 - CMake: fix runtests.pl with CMake, add new test targets - - * runtests.pl: - - Fix out-of-tree build under CMake when srcdir is not set. Default - srcdir to the location of runtests.pl. - - Add a hack to allow CMake to use the TFLAGS option as documented - in tests/README and used in scripts/travis/script.sh. - * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie - support (no one should care, it is already EOL.). - * Remove CTest since it defines its own 'test' target with no tests - since all unittests are already broken and not built by default. - * Add new test targets based on the options from Makefile.am. Since - new test targets are rarely added, I opted for duplicating the - runtests.pl options as opposed to creating a new Makefile.inc file. - Use top-level target names (test-x) instead of x-test since that is - used by CI and others. - - Closes #5358 +- TODO: Add flag to specify download directory -- [Peter Wu brought this change] +- TODO: return code to CURLMOPT_PUSHFUNCTION to fail connection - CMake: do not build test programs by default +- cirrus-ci: disable FreeBSD 13 (again) - The default target should only build libcurl and curl. Add a dedicated - 'testdeps' target which will be used later when running tests. Note that - unittests are currently broken in CMake and already excluded. + It has been failing for a good while again. This time we better leave it + disabled until we have more reason to believe it behaves. - Closes #5368 - -- FILEFORMAT: moved up the variables section and further polished + Closes #5628 -- runtests: remove ftp2 support, not used +- ngtcp2: sync with current master - We once supported two separate ftp instances in the test suite. Has not - been used the last decade. + ngtcp2 added two new callbacks - Closes #5375 + Reported-by: Lucien Zürcher + Fixes #5624 + Closes #5627 -- url: sort the protocol schemes in rough popularity order - - When looking for a protocol match among supported schemes, check the - most "popular" schemes first. It has zero functionality difference and - for all practical purposes a speed difference will not be measureable - but it still think it makes sense to put the least likely matches last. - - "Popularity" based on the 2019 user survey. +- examples/multithread.c: call curl_global_cleanup() - Closes #5377 + Reported-by: qiandu2006 on github + Fixes #5622 + Closes #5623 -Marc Hoersken (11 May 2020) -- test1238: avoid tftpd being busy for tests shortly following - - The tftpd server may still be busy if the total timeout of - 25 seconds has not been reached or no sread error was received - during or after the execution of the timeout test 1238. - - Once the next TFTP test comes around (eg. 1242 or 1243), - those will fail because the tftpd server is still waiting - on data from curl due to the UDP protocol being stateless - and having no connection close. On Linux this error may not - happen, because ICMP errors generated due to a swrite error - can also be returned async on the next sread call instead. - - Therefore we will now just kill the tftpd server after test - 1238 to make sure that the following tests are not affected. - - This enables us to no longer ignore tests 1242, 1243, 2002 - and 2003 on the CI platforms CirrusCI and AppVeyor. +- vtls: compare cert blob when finding a connection to reuse - Assisted-by: Peter Wu - Closes #5364 + Reported-by: Gergely Nagy + Fixes #5617 + Closes #5619 -Daniel Stenberg (11 May 2020) -- write-out.d: added "response_code" +- RELEASE-NOTES: synced -- KNOWN_BUGS: Build with staticly built dependency +- terminology: call them null-terminated strings - I rewrote the item 5.4 to be more generic about static dependencies. - -- ROADMAP: remove old entries + Updated terminology in docs, comments and phrases to refer to C strings + as "null-terminated". Done to unify with how most other C oriented docs + refer of them and what users in general seem to prefer (based on a + single highly unscientific poll on twitter). - MQTT - the start has already landed + Reported-by: coinhubs on github + Fixes #5598 + Closes #5608 + +- http: fix proxy auth with blank password - tiny-curl - also mostly landed and is a continuous work + Regression in 7.71.0 - make menuconfig - basically no interest from users, not pushing there - -- [Peter Wu brought this change] - - travis: Add ngtcp2 and quiche tests for CMake + Added test case 346 to verify. - To avoid an explosion of jobs, extend the existing CMake tests with - ngtcp2 and quiche support. macOS was previously moved to GitHub actions, - so the non-Linux case can be dropped. - -- [Peter Wu brought this change] + Reported-by: Kristoffer Gleditsch + Fixes #5613 + Closes #5616 - CMake: add ENABLE_ALT_SVC option +- .dcignore: ignore tests and docs directories - Tested alt-svc with quiche. While at it, add missing MultiSSL reporting - (not tested). - -- [Peter Wu brought this change] + This is a config file for deepcode.ai, a static code analyzer. - CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) - - Add three new CMake Find modules (using the curl license, but I grant - others the right to apply the CMake BSD license instead). +Jay Satiro (26 Jun 2020) +- tool_cb_hdr: Fix etag warning output and return code - This CMake config is simpler than the autotools one because it assumes - ngtcp2 and nghttp3 to be used together. Another difference is that this - CMake config checks whether QUIC is actually supported by the TLS - library (patched OpenSSL or boringssl) since this can be a common - configuration mistake that could result in build errors later. + - Return 'failure' on failure, to follow the existing style. - Unlike autotools, CMake does not warn you that the features are - experimental. The user is supposed to already know that and read the - documentation. It requires a very special build environment anyway. + - Put Warning: and the warning message on the same line. - Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from - current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure - out which features (libldap-2.4, libssh2) to disable due to conflicts - with boringssl. + Ref: https://github.com/curl/curl/issues/5610 - Closes #5359 + Closes https://github.com/curl/curl/pull/5612 -Marc Hoersken (10 May 2020) -- tests/server/tftpd.c: fix include and enhance debug logging +Daniel Stenberg (26 Jun 2020) +- CURLOPT_READFUNCTION.3: provide the upload data size up front - setjmp.h should only be included if HAVE_SETJMP_H is defined. + Assisted-by: Jay Satiro + Closes #5607 + +- test1539: do a HTTP 1.0 POST without a set size (fails) - Add additional log statements to see wether reads and writes - are blocking or finishing before an alarm signal is received. + Attempt to reproduce #5593. Test case 1514 is very similar but uses + HTTP/1.1 and thus switches to chunked. - Assisted-by: Peter Wu - Part of #5364 + Closes #5595 -Daniel Stenberg (10 May 2020) -- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present - - Reported-by: Marcel Raad - Follow-up to 148534db5 - Fixes #5367 - Closes #5369 +- [Baruch Siach brought this change] -Marc Hoersken (9 May 2020) -- appveyor: update comments to be clear about toolchain + mbedtls: fix build with disabled proxy support - - CMake-based MSYS builds use mingw-w64 to cross-compile. - - autotools-based builds are compiled using msys2-devel. + Don't reference fields that do not exist. Fixes build failure: - The difference is that the later ones are not cross-compiled - to Windows and instead require the msys2 runtime to be present. + vtls/mbedtls.c: In function 'mbed_connect_step1': + vtls/mbedtls.c:249:54: error: 'struct connectdata' has no member named 'http_proxy' - At the moment only the Azure Pipelines CI builds actually - run autotools-based cross-compilation builds for Windows. + Closes #5615 -- TODO: update regarding missing Schannel features - - Some aspects have already been implemented over the years. - - 15.1 Client certificates are now supported: - - - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 - - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 - - 15.2 Ciphers can now be specified through: - - - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 +- codeql-analysis.yml: fix the 'languages' setting - Reviewed-by: Daniel Stenberg and Marcel Raad - Closes #5358 + It needs a 'with:' in front of it. -Daniel Stenberg (8 May 2020) -- checksrc: close the .checksrc file handle when done reading +GitHub (26 Jun 2020) +- [Daniel Stenberg brought this change] -- RELEASE-NOTES: synced + gtihub: codeql-analysis.yml - And bumped next version to 7.71.0 - -- [Gilles Vollant brought this change] + enables code security scanning with github actions - CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) +Daniel Stenberg (25 Jun 2020) +- tests: verify newline in username and password for HTTP + + test 1296 is a simply command line test - Closes #4346 + test 1910 is a libcurl test including a redirect -- TODO: native IDN support on macOS +- url: allow user + password to contain "control codes" for HTTP(S) + + Reported-by: Jon Johnson Jr + Fixes #5582 + Closes #5592 -- urlapi: accept :: as a valid IPv6 address +- escape: make the URL decode able to reject only %00 bytes - Text 1560 is extended to verify. + ... or all "control codes" or nothing. - Reported-by: Pavel Volgarev - Fixes #5344 - Closes #5351 - -- THANKS-filter: Peter Wang - -- [Peter Wang brought this change] + Assisted-by: Nicolas Sterchele - *_sspi: fix bad uses of CURLE_NOT_BUILT_IN +- http2: set the correct URL in pushed transfers - Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other - instances of QuerySecurityPackageInfo failing, as in - commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. + ...previously CURLINFO_EFFECTIVE_URL would report the URL of the + original "mother transfer", not the actually pushed resource. - Closes #5355 + Reported-by: Jonathan Cardoso Machado + Fixes #5589 + Closes #5591 -- docs/HTTP3: add qlog to the quiche build instruction +Jay Satiro (25 Jun 2020) +- [Javier Blazquez brought this change] -- ngtcp2: introduce qlog support - - If the QLOGDIR environment variable is set, enable qlogging. + openssl: Fix compilation on Windows when ngtcp2 is enabled - ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for - QUIC functions that are backend independent. + - Include wincrypt before OpenSSL includes so that the latter can + properly handle any conflicts between the two. - Closes #5353 + Closes https://github.com/curl/curl/pull/5606 -- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN - - That return code is reserved for build-time conditional code not being - present while this was a regular run-time error from a Windows API. +Daniel Stenberg (25 Jun 2020) +- test543: extended to verify zero length input - Reported-by: wangp on github - Fixes #5349 - Closes #5350 - -- runtests: show elapsed test time with higher precision (ms) - -- RELEASE-NOTES: synced + As was reported in #5601 -- http2: simplify and clean up trailer handling - - Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in - ed35d6590e72. This should make the trailer handling more straight forward and - hopefully less error-prone. +- escape: zero length input should return a zero length output - Deliver the trailer header to the callback already at receive-time. No - longer caches the trailers to get delivered at end of stream. + Regression added in 7.71.0. - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 - Closes #5348 + Fixes #5601 + Reported-by: Kristoffer Gleditsch + Closes #5602 -Marc Hoersken (7 May 2020) -- appveyor: disable test 1139 instead of ignoring it +- Curl_inet_ntop: always check the return code - Spending time on manpage checking makes no sense - for these builds due to lacking manpage support. + Reported-by: Siva Sivaraman + Fixes #5412 + Closes #5597 -- appveyor: disable flaky test 1501 and ignore broken 1056 +- sendf: improve the message on client write errors - Test 1501 is flaky on Windows CI due to being time sensitive - and the testsuite relying on taskkill.exe to check for the - existance of processes which can take to much time itself. + Replace "Failed writing body (X != Y)" with + "Failure writing output to destination". Possibly slightly less cryptic. - Test 1056 is broken in autotools-based Windows builds due - to scope ID support missing in these builds at the moment. + Reported-by: coinhubs on github + Fixes #5594 + Closes #5596 -- test613.pl: make tests 613 and 614 work with OpenSSH for Windows - - OpenSSH for Windows shows group and other/world permissions as *, - because those concepts do not exist on Windows. It also does not - show the current or parent directory, so we just ignore those. - - Reviewed-by: Daniel Stenberg - Closes #5328 +- RELEASE-NOTES: synced -Daniel Stenberg (6 May 2020) -- runtests: set +x mode again +- curlver: start working on 7.71.1 -- libssh2: convert over to use dynbuf - - In my very basic test that lists sftp://127.0.0.1/tmp/, this patched - code makes 161 allocations compared to 194 in git master. A 17% - reduction. +- [Denis Baručić brought this change] + + DYNBUF.md: fix a typo: trail => tail - Closes #5336 + Closes #5599 + +Version 7.71.0 (23 Jun 2020) -- travis: add "qlog" as feature in the quiche build +Daniel Stenberg (23 Jun 2020) +- RELEASE-NOTES: curl 7.71.0 release -- quiche: enable qlog output - - quiche has the potential to log qlog files. To enable this, you must - build quiche with the qlog feature enabled `cargo build --features - qlog`. curl then passes a file descriptor to quiche, which takes - ownership of the file. The FD transfer only works on UNIX. +- THANKS: curl 7.71.0 additions + +- url: make sure pushed streams get an allocated download buffer - The convention is to enable logging when the QLOGDIR environment is - set. This should be a path to a folder where files are written with the - naming template .qlog. + Follow-up to c4e6968127e876b0 - Co-authored-by: Lucas Pardue - Replaces #5337 - Closes #5341 - -- urldata.h: remove #define HEADERSIZE, not used anymore + When a new transfer is created, as a resuly of an acknowledged push, + that transfer needs a download buffer allocated. - Follow-up to ed35d6590e72c + Closes #5590 -- ngtcp2: convert to dynbuf +Jay Satiro (22 Jun 2020) +- openssl: Don't ignore CA paths when using Windows CA store - Closes #5335 - -- connect: make happy eyeballs work for QUIC (again) + This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does + not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default + locations. Instead the CA store can now be used at the same time. - Follow-up from dbd16c3e256c6c (regression in 7.70.0) + The change is due to the impending release. The issue is still being + discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and + is now documented as experimental. - Closes #5334 - -- connect: add two asserts to clue code analyzers in a little + Ref: bc052cc (parent commit) + Ref: https://github.com/curl/curl/issues/5585 -- http_proxy: ported to use dynbuf instead of a static size buffer +- tool_operate: Don't use Windows CA store as a fallback - Removes a 16K static buffer from the easy handle. Simplifies the code. - -- dynbuf: introduce internal generic dynamic buffer functions + Background: - A common set of functions instead of many separate implementations for - creating buffers that can grow when appending data to them. Existing - functionality has been ported over. + 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate + store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides + CURLOPT_CAINFO if both are set. The curl tool will fall back to + CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set + via CURLOPT_CAINFO. - In my early basic testing, the total number of allocations seem at - roughly the same amount as before, possibly a few less. + Problem: - See docs/DYNBUF.md for a description of the API. + libcurl may be built with hardcoded paths to a certificate bundle or + directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are + ignored. - Closes #5300 - -- runtests: remove sleep calls + Solution: - Remove many one second sleeps that were done *after* each newly started - test server already has been verified. They should not have any purpose - there. + A solution is still being discussed but since there's an impending + release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. - Closes #5323 + Ref: https://github.com/curl/curl/issues/5585 -- asyn-*: remove support for never-used NULL entry pointers +- openssl: Fix CA fallback logic for OpenSSL 3.0 build - ... and instead convert those to asserts to make sure they are truly - never NULL. + Prior to this change I assume a build error would occur when + CURL_CA_FALLBACK was used. - Closes #5324 + Closes https://github.com/curl/curl/pull/5587 -- [Emil Engler brought this change] +Daniel Stenberg (22 Jun 2020) +- copyright: update mismatched copyright years - doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax - - Closes #5325 +- test1460: verify that -Ji is not ok -Jay Satiro (2 May 2020) -- asyn-thread: fix cppcheck warning - - - Check for NULL entry parameter before attempting to deref entry in - Curl_resolver_is_resolved, like is already done in asyn-ares. - - This is to silence cppcheck which does not seem to understand that - asyn-ares and asyn-thread have separate Curl_resolver_is_resolved - and those units are mutually exclusive. Prior to this change it warned - of a scenario where asyn-thread's Curl_resolver_is_resolved is called - with a NULL entry from asyn-ares, but that couldn't happen. - - Reported-by: rl1987@users.noreply.github.com +- tool_getparam: -i is not OK if -J is used - Fixes https://github.com/curl/curl/issues/5326 + Reported-by: sn on hackerone + Bug: https://curl.haxx.se/docs/CVE-2020-8177.html -- select: fix overflow protection in Curl_socket_check - - Follow-up to a96c752 which changed the timeout_ms type from time_t to - timediff_t. - - Ref: https://github.com/curl/curl/pull/5240 - - Closes https://github.com/curl/curl/pull/5286 +- [Peter Wu brought this change] -Marc Hoersken (2 May 2020) -- sockfilt: make select_ws stop waiting on exit signal event - - This makes sure that select_ws behaves similar to real select - which stops waiting on a signal handler being triggered. - - This makes it possible to gracefully stop sockfilt.exe on - Windows with taskkill /IM sockfilt.exe (without /F force flag). + CMake: ignore INTERFACE_LIBRARY targets for pkg-config file - Reviewed-by: Jay Satiro - Part of #5260 + Reviewed-by: Marcel Raad + Fixes #5512 + Closes #5517 -- tests/server/util.[ch]: add exit event to stop waiting on Windows - - This commit adds a global exit event to the test servers that - Windows-specific wait routines can use to get triggered if the - program was signaled to be terminated, eg. select_ws in sockfilt.c - - The exit event will be managed by the signal handling code and is - set to not reset automatically to support multiple wait routines. - - Reviewed-by: Jay Satiro - Closes #5260 +- [Valentyn Korniienko brought this change] -- tests/server/util.c: fix thread handle not being closed + multibyte: Fixed access-> waccess to file for Windows Plarform - Reviewed-by: Jay Satiro - Part of #5260 + Reviewed-by: Marcel Raad + Closes #5580 -- tests/server/util.c: use raise instead of calling signal handler - - Use raise to trigger signal handler instead of calling it - directly and causing potential unexpected control flow. +- altsvc: bump to h3-29 - Reviewed-by: Jay Satiro - Part of #5260 + Closes #5584 -- tests: add support for SSH server variant specific transfer paths +- urlglob: treat literal IPv6 addresses with zone IDs as a host name - OpenSSH for Windows requires paths in the format of /C:/ - instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ + ... and not as a "glob". Now done by passing the supposed host to the + URL parser which supposedly will do a better job at identifying "real" + numerical IPv6 addresses. - Reviewed-by: Daniel Stenberg - Closes #5298 + Reported-by: puckipedia on github + Fixes #5576 + Closes #5579 -Daniel Stenberg (2 May 2020) -- RELEASE-NOTES: synced +- test1179: verify error message for non-existing cmdline option -- libssh2: set the expected total size in SCP upload init +- tool_getparam: repair the error message for unknown flag - ... as otherwise the progress callback gets called without that - information, making the progress meter have less info. + Follow-up to 9e5669f3880674 + Detected by Coverity CID 1464582 ("Logically dead code") - Reported-by: Murugan Balraj - Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html - Closes #5317 + Closes #5577 -- runtests: make the logmsg from the ssh server only show in verbose +- FILEFORMAT: describe verify/stderr -- tests: make test 1248 + 1249 use %NOLISTENPORT +- connect: improve happy eyeballs handling - ... instead of a port of a non-running server so that it works - stand-alone. + For QUIC but also for regular TCP when the second family runs out of IPs + with a failure while the first family is still trying to connect. - Closes #5318 + Separated the timeout handling for IPv4 and IPv6 connections when they + both have a number of addresses to iterate over. -- examples: remove asiohiper.cpp - - This example has repeatedly been reported to contain bugs, and as users - copy and paste code from this into production, I now deem it better to - not provide the example at all. - - Closes #5090 - Closes #5322 +- ngtcp2: never call fprintf() in lib code in release version -- [Emil Engler brought this change] +- ngtcp2: fix happy eyeballs quic connect crash + + Reported-by: Peter Wu + Fixes #5565 + Closes #5568 - doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 +- select: remove the unused ELAPSED_MS() macro - Closes #5320 + Closes #5573 -- [Emil Engler brought this change] +Marc Hoersken (17 Jun 2020) +- [rcombs brought this change] - KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" + multi: implement wait using winsock events + + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. - It was fixed in 9a2cbf3 + Reviewed-by: Gergely Nagy + Reviewed-by: Marc Hörsken - Closes #5319 + Closes #5397 -- cirrus: disable SFTP and SCP tests +Daniel Stenberg (17 Jun 2020) +- manpage: add three missing environment variables - ... as we can't seem to start the sshd server on it. Those problems - existed before d1239b50bececd (running the SSH server on a random port), - but they're more noticable now since there are more failed attempts in - the logs. + CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE - Closes #5315 + Closes #5571 -- [Emil Engler brought this change] +- RELEASE-NOTES: synced - runtests: fix typo in the existence of disabled tests checker +- configure: for wolfSSL, check for the DES func needed for NTLM - Closes #5316 + Also adds pkg-config support for the wolfSSL detection. -Dan Fandrich (30 Apr 2020) -- test75: Remove precheck test - - This has not been needed since commit 9fa42bed and often prevents it - from running at all with dynamic test ports. +- [Ruurd Beerstra brought this change] -- tests: Stop referring to server ports when they're not used + ntlm: enable NTLM support with wolfSSL - Several tests referred to specific server ports even when the test - didn't actually use that server or specify that it's needed. In such - cases, the test harness substitutes the text "[not running]" as the port - number which causes many such tests to fail due to the inability to - parse the URL. These tests are changed to use %NOLISTENPORT which will - always be substituted correctly. - -Daniel Stenberg (30 Apr 2020) -- [Emil Engler brought this change] - - GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT + When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* + functions that OpenSSL has. This change take advantage of that. - Closes #5287 + Co-authored-by: Daniel Stenberg + Closes #5556 + Fixes #5548 -- conncache: various concept cleanups - - More connection cache accesses are protected by locks. - - CONNCACHE_* is a beter prefix for the connection cache lock macros. - - Curl_attach_connnection: now called as soon as there's a connection - struct available and before the connection is added to the connection - cache. +- http: move header storage to Curl_easy from connectdata - Curl_disconnect: now assumes that the connection is already removed from - the connection cache. + Since the connection can be used by many independent requests (using + HTTP/2 or HTTP/3), things like user-agent and other transfer-specific + data MUST NOT be kept connection oriented as it could lead to requests + getting the wrong string for their requests. This struct data was + lingering like this due to old HTTP1 legacy thinking where it didn't + mattered.. - Ref: #4915 - Closes #5009 + Fixes #5566 + Closes #5567 -- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports +- CODE_REVIEW.md: how to do code reviews in curl - As stunnel is an external tool and it has no specific option to export - the actually used port number when asked to listen to 0, runtests - instead iterates over ten randomly picked high number ports and sticks - to the first one stunnel can listen to. + Assisted-by: Daniel Gustafsson + Assisted-by: Rich Salz + Assisted-by: Hugo van Kemenade + Assisted-by: James Fuller + Assisted-by: Marc Hörsken + Assisted-by: Jay Satiro - Closes #5267 + Closes #5555 -- tests: pick a random port number for SSH +- altsvc: remove the num field from the altsvc struct - Since sshd doesn't have such an option by itself, we iterate over a - series of random ports until one works. + It was superfluous since we have the list.size alredy - Closes #5273 - -- [Rikard Falkeborn brought this change] + Reported-by: Jay Satiro + Fixes #5553 + Closes #5563 - libtest/cmake: Remove commented code +- version.d: expanded and alpha-sorted - These were commented out in e9dd0998706a when Makefile.inc was included - instead. 11 years have passed since then and the commented code is of - course very outdated. Remove it to avoid confusion. + Added a few missing features not previously mentioned. Ordered them + alphabetically. - Closes #5311 + Closes #5558 -- schannel: source code reindent - - White space edits only. Conform better to standard curl source code - indenting style. +- ABI.md: rename to .md and polish the markdown - Closes #5305 + Closes #5562 -Kamil Dudka (29 Apr 2020) -- test1177: look for curl.h in source directory - - If we use a separate build directory, there is no copy of the header. +- HELP-US: add a section for "smaller tasks" - Closes #5310 - -- tests: look for preprocessed tests in build directory + The point of this section is to meet the CII Best Practices gold level + critera: - ... which is not always the same directory as source directory + "The project MUST clearly identify small tasks that can be performed by + new or casual contributors" - Closes #5310 + Closes #5560 -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: synced +- TODO: retry on the redirected-to URL - ... and bumped curlver.h to 7.70.1 - -Version 7.70.0 (29 Apr 2020) + Closes #5462 -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: 7.70.0 +- mailmap: Nicolas Sterchele -- THANKS: synced with the 7.70.0 release +- [Nicolas Sterchele brought this change] -- headers: copyright range fix + TODO: remove 19.3 section title + + Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on + curl documentation website + + Closes #5561 -- [Rikard Falkeborn brought this change] +- [Martin V brought this change] - doh: Constify some input pointers + test1560: avoid possibly negative association in wording - Closes #5306 + Closes #5549 -- nss: check for PK11_CreateDigestContext() returning NULL +- share: don't set the share flag it something fails - ... to avoid crashes! + When asking for a specific feature to be shared in the share object, + that bit was previously set unconditionally even if the shared feature + failed or otherwise wouldn't work. - Reported-by: Hao Wu - Fixes #5302 - Closes #5303 + Closes #5554 -- travis: bump the wolfssl CI build to use 4.4.0 +- buildconf: remove -print from the find command that removes files - Closes #5301 + It's just too annoying and unnecessary to get a long list of files shown -- copyright updates: adjust year ranges +- RELEASE-NOTES: synced -Marc Hoersken (26 Apr 2020) -- CI: do not include */ci branches in PR builds +- wording: avoid blacklist/whitelist stereotypes - Align Azure Pipelines with GitHub Actions. - -Daniel Stenberg (25 Apr 2020) -- runtests: check for the disabled tests relative srcdir + Instead of discussing if there's value or meaning (implied or not) in + the colors, let's use words without the same possibly negative + associations. - To make it work correctly for out-of-tree builds. + Closes #5546 + +Jay Satiro (9 Jun 2020) +- tool_getparam: fix memory leak in parse_args - Follow-up to 75e8feb6fb08b + Prior to this change in Windows Unicode builds most parsed options would + not be freed. - Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 - Reported-by: Marcel Raad - Closes #5297 - -- runtests: revert commenting out a line I did for debugging + Found using _CrtDumpMemoryLeaks(). - Follow-up to 11091cd4d. It was not meant to be pushed! + Ref: https://github.com/curl/curl/issues/5545 -- smtp: set auth correctly +Daniel Stenberg (8 Jun 2020) +- socks: detect connection close during handshake - Regression since 7.69.0 and 68fb25fa3fcff. + The SOCKS4/5 state machines weren't properly terminated when the proxy + connection got closed, leading to a busy-loop. - The code wrongly assigned 'from' instead of 'auth' which probably was a - copy and paste mistake from other code, leading to that auth could - remain NULL and later cause an error to be returned. + Reported-By: zloi-user on github + Fixes #5532 + Closes #5542 + +- [James Fuller brought this change] + + multi: add defensive check on data->multi->num_alive - Assisted-by: Eric Sauvageau - Fixes #5294 - Closes #5295 + Closes #5540 -Marcel Raad (25 Apr 2020) -- lib: clean up whitespace +- Curl_addrinfo: use one malloc instead of three + + To reduce the amount of allocations needed for creating a Curl_addrinfo + struct, make a single larger malloc instead of three separate smaller + ones. - This fixes CodeFactor warnings. + Closes #5533 -Daniel Stenberg (25 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] +- [Alessandro Ghedini brought this change] - libssh: avoid options override by configuration files - - Previously, options set explicitly through command line options could be - overridden by the configuration files parsed automatically when - ssh_connect() was called. + quiche: update SSLKEYLOGFILE support - By calling ssh_options_parse_config() explicitly, the configuration - files are parsed before setting the options, avoiding the options - override. Once the configuration files are parsed, the automatic - configuration parsing is not executed. + quiche now requires the application to explicitly set the keylog path + for each connection, rather than reading the environment variable + itself. - Fixes #4972 - Closes #5283 - Signed-off-by: Anderson Toshiyuki Sasaki + Closes #5541 -- runtests: when mentions http, kill http/2 too +- tests: add two simple tests for --login-options + + Test 895 and 896 - as a follow-up to a3e972313b - Since the http2 test server is a mere proxy that needs to know about the - dynamic port the HTTP server is using, it too needs to get restarted - when the http server is killed. + Closes #5539 + +- ngtcp2: update with recent API changes - A regression caused by 80d6515. + Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. - Fixes #5289 - Closes #5291 + Assisted-by: Tatsuhiro Tsujikawa + Closes #5538 -- [Yuri Slobodyanyuk brought this change] +- [James Fuller brought this change] - docs: fix two typos + socks: remove unreachable breaks in socks.c and mime.c - Closes #5292 - -- [Emil Engler brought this change] + Closes #5537 - tests/git: ignore mqttd and port files +- tool_cfgable: free login_options at exit - Closes #5290 + Memory leak + Reported-by: Geeknik Labs + Fixes #5535 + Closes #5536 -- tests: make runtests check that disabled tests exists +- libssh2: keep sftp errors as 'unsigned long' - ... and error out if so. Removed '536' from DISABLED as there is no such - test file. + Remove weird work-around for storing the SFTP errors as int instead of + the "unsigned long" that libssh2 actually returns for SFTP errors. - Closes #5288 - -- test1154: set a proper name + Closes #5534 -- select: make Curl_socket_check take timediff_t timeout +Marc Hoersken (6 Jun 2020) +- timeouts: move ms timeouts to timediff_t from int and long - Coverity found CID 1461718: + Now that all functions in select.[ch] take timediff_t instead + of the limited int or long, we can remove type conversions + and related preprocessor checks to silence compiler warnings. - Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > - 9223372036854775807L" is always false regardless of the values of its - operands. This occurs as the logical second operand of "||". + Avoiding conversions from time_t was already done in 842f73de. - Closes #5240 - -- [i-ky brought this change] + Based upon #5262 + Supersedes #5214, #5220 and #5221 + Follow up to #5343 and #5479 + Closes #5490 - libcurl-multi.3: added missing full stop - - Closes #5285 +Daniel Stenberg (6 Jun 2020) +- [François Rigault brought this change] -Jay Satiro (22 Apr 2020) -- transfer: Switch PUT to GET/HEAD on 303 redirect + openssl: set FLAG_TRUSTED_FIRST unconditionally - Prior to this change if there was a 303 reply to a PUT request then - the subsequent request to respond to that redirect would also be a PUT. - It was determined that was most likely incorrect based on the language - of the RFCs. Basically 303 means "see other" resource, which implies it - is most likely not the same resource, therefore we should not try to PUT - to that different resource. + On some systems, openssl 1.0.x is still the default, but it has been + patched to contain all the recent security fixes. As a result of this + patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be + defined, while the previous behavior of openssl to not look at trusted + chains first, remains. - Refer to the discussions in #5237 and #5248 for more information. + Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to + probe for the behavior of openssl based on the existence ofmacros. - Fixes https://github.com/curl/curl/issues/5237 - Closes https://github.com/curl/curl/pull/5248 + Closes #5530 -Daniel Stenberg (22 Apr 2020) -- lib/mk-ca-bundle: skip empty certs +- server/util: fix logmsg format using curl_off_t argument - Reviewed-by: Emil Engler - Reported-by: Ashwin Metpalli - Fixes #5278 - Closes #5280 - -- version: skip idn2_check_version() check and add precaution + ... this caused segfaults on armv7. - A gcc-10's -fanalyze complaint made me spot and do these improvements. + Regression added in dd0365d560aea5a (7.70.0) - Closes #5281 + Reviewed-by: Jay Satiro + Closes #5529 - RELEASE-NOTES: synced -- [Brian Bergeron brought this change] +- [Cherish98 brought this change] - curl.h: update comment typo - - "routines with be invoked" -> "routines will be invoked" + socks: fix expected length of SOCKS5 reply - Closes #5279 - -- [Emil Engler brought this change] - - GnuTLS: Don't skip really long certificate fields + Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the + reply ATYP is X'01'. This resulted in erroneously expecting more bytes + when the request length is greater than the reply length (e.g., when + remotely resolving the hostname). - Closes #5271 + Closes #5527 -- gnutls: bump lowest supported version to 3.1.10 - - GnuTLS 3.1.10 added new functions we want to use. That version was - released on Mar 22, 2013. Removing support for older versions also - greatly simplifies the code. +Marc Hoersken (5 Jun 2020) +- .gitignore: add directory containing the stats repo - Ref: #5271 - Closes #5276 + Since the new curl/stats repository is designed to be + checked out into the curl repository working tree as stats/ + it should be on the ignore list to aid in commit staging. -- mqtt: make NOSTATE get within the debug name array +Daniel Stenberg (5 Jun 2020) +- [Adnan Khan brought this change] -- tests: run the RTSP test server on a dynamic port number + HTTP3.md: clarify cargo build directory - To avoid port collisions. + Cargo needs to be called from within the 'quiche' directory. - Closes #5272 + Closes #5522 -- tests: add %NOLISTENPORT and use it - - The purpose with this variable is to provide a port number that is - reasonably likely to not have a listener on the local host so that tests - can try connect failures against it. It uses port 47 - "reserved" - according to IANA. - - Updated six tests to use it instead of the previous different ports. +- user-agent.d: spell out what happens given a blank argument - Assisted-by: Emil Engler - Closes #5270 + Closes #5525 -- mqtt: remove code with no purpose +- trailers: switch h1-trailer logic to use dynbuf - Detected by Coverity. CID 1462319. + In the continued effort to remove "manual" realloc schemes. - "The same code is executed when the condition result is true or false, - because the code in the if-then branch and after the if statement is - identical." + Closes #5524 + +- CURLINFO_ACTIVESOCKET.3: clarify the description - Closes #5275 + Reported-by: Jay Satiro + Fixes #5299 + Closes #5520 -- mqtt: fix Curl_read() error handling while reading remaining length +- mailmap: Don J Olmstead + +- configure: only strip first -L from LDFLAGS - Detected by Coverity. CID 1462320. + In the logic that works out if a given OpenSSL path works, it stripped + off a possibly leading -L flag using an incorrect sed pattern which + would remove all instances of -L in the string, including if the path + itself contained that two-letter sequence! - Closes #5274 - -- server/tftpd: fix compiler warning + The same pattern was used and is now updated in multiple places. Now it + only removes -L if it starts the strings. - Follow-up from 369ce38ac1d - Reported-by: Marc Hörsken + Reported-by: Mohamed Osama + Fixes #5519 + Closes #5521 -- http: free memory when Alt-Used header creation fails due to OOM +Peter Wu (4 Jun 2020) +- quiche: advertise draft 28 support - Reported-by: James Fuller - Fixes #5268 - Closes #5269 - -Daniel Gustafsson (20 Apr 2020) -- lib: fix typos in comments and errormessages + Fix the verbose message while at it, quiche currently supports draft + 27 and draft 28 simultaneously. - This fixes a few randomly spotted typos in recently merged code, most - notably one in a userfacing errormessage the schannel code. + Closes #5518 -Daniel Stenberg (20 Apr 2020) -- tests: run the SOCKS test server on a dynamic port number +Daniel Stenberg (4 Jun 2020) +- KNOWN_BUGS: RTSP authentication breaks without redirect support - Closes #5266 - -- [Johannes Schindelin brought this change] + Closes #4750 - multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - - When cURL is compiled with support for multiple SSL backends, it is - possible to configure an SSL backend via `curl_global_sslset()`, but - only *before* `curl_global_init()` was called. +Jay Satiro (4 Jun 2020) +- projects: Add crypt32.lib to dependencies for all OpenSSL configs - If another SSL backend should be used after that, a user might be - tempted to call `curl_global_cleanup()` to start over. However, we did - not foresee that use case and forgot to reset the SSL backend in that - cleanup. + Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO + need crypt32. - Let's allow that use case. + Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. - Fixes #5255 - Closes #5257 - Reported-by: davidedec on github - Signed-off-by: Johannes Schindelin - -- tests: run the TFTP test server on a dynamic port number + The changes that are in this commit were made by script. - Picking a dynamic unused port is better than a fixed to avoid the - collision risk. + Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 - Closes #5265 + Closes https://github.com/curl/curl/pull/5516 -- mqtt: improve the state machine - - To handle PUBLISH before SUBACK and more. +Marc Hoersken (3 Jun 2020) +- CI/macos: fix 'is already installed' errors by using bundle - Updated the existing tests and added three new ones. + Avoid failing CI builds due to nghttp2 being already installed. - Reported-by: Christoph Krey - Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html - Closes #5246 + Closes #5513 -- runtests: always put test number in servercmd file +Daniel Stenberg (3 Jun 2020) +- altsvc: fix 'dsthost' may be used uninitialized in this function - RELEASE-NOTES: synced -- release-notes.pl: fix parsing typo - -James Fuller (20 Apr 2020) -- ensure all references to ports are replaced by vars - -- add more alt-svc test coverage - -Daniel Stenberg (20 Apr 2020) -- test1247: use http server to get the port number set +- urldata: let the HTTP method be in the set.* struct - Follow-up to 0f5db7b263f - -- runtests: use a unix domain socket path with the pid in the name + When the method is updated inside libcurl we must still not change the + method as set by the user as then repeated transfers with that same + handle might not execute the same operation anymore! - To make it impossible for test cases to access the file name without - using the proper variable for the purpose. + This fixes the libcurl part of #5462 - Closes #5264 - -Daniel Gustafsson (19 Apr 2020) -- [Mipsters on github brought this change] - - src: Remove C99 constructs to ensure C89 compliance + Test 1633 added to verify. - This fixes the error: 'for' loop initial declaration used outside C99 - mode by declaring the loop increment variable in the beginning of the - block instead of inside the for loop. + Closes #5499 + +- hostip: fix the memory-leak introduced in 67d2802 - Fixes #5254 - Reviewed-by: Daniel Gustafsson + Fixes #5503 + Closes #5504 -Daniel Stenberg (19 Apr 2020) -- runtests: dummy init the ports variables to avoid warnings +- test970: make it require proxy support + + This test verifies the -w %json output and the test case includes a full + generated "blob". If there's no proxy support built into libcurl, it + will return an error for proxy related info variables and they will not + be included in the json, thus causing a mismatch and this test fails. - ... and generate something that can help debug test cases. + Reported-by: Marc Hörsken + Fixes #5501 + Closes #5502 -- [Patrick Monnerat brought this change] +- [Radoslav Georgiev brought this change] - mime: properly check Content-Type even if it has parameters + examples/http2-down/upload: add error checks - New test 669 checks this fix is effective. + If `index.html` does not exist in the directory from which the example + is invoked, the fopen(upload, "rb") invocation in `setup` would fail, + returning NULL. This value is subsequently passed as the FILE* argument + of the `fread` invocation in the `read_callback` function, which is the + actual cause of the crash (apparently `fread` assumes that argument to + be non-null). - Fixes #5256 - Closes #5258 - Reported-by: thanhchungbtc on github - -- tests/FILEFORMAT: converted to markdown and extended + In addition, mitigate some possible crashes of similar origin. - Closes #5261 + Closes #5463 -- test1245: make it work with dynamic FTP server port +- [kotoriのねこ brought this change] -- test1055: make it work with dynamic FTP port + examples/ephiperfifo: turn off interval when setting timerfd + + Reported-by: therealhirudo on github + Fixes #5485 + Closes #5497 -- test1028: make it run on dynamic FTP server port +- [Saleem Abdulrasool brought this change] -- tests: move pingpong server to dynamic listening port + vtls: repair the build with `CURL_DISABLE_PROXY` + + `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` + is enabled. Repair the build with that configuration. - FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic - ports + Follow-up to f3d501dc67 - Test 842-845 are unfortunately a bit hard to move over to this concept - right now and require "default port" still... - -- test1056: work with dynamic HTTP ipv6 port + Closes #5498 -- test1448: work with dynamic HTTP server port +- transfer: remove k->str NULL check + + "Null-checking k->str suggests that it may be null, but it has already + been dereferenced on all paths leading to the check" - and it can't + legally be NULL at this point. Remove check. + + Detected by Coverity CID 1463884 + + Closes #5495 -- tests: introduce preprocessed test cases +Marc Hoersken (1 Jun 2020) +- select: always use Sleep in Curl_wait_ms on Win32 - The runtests script now always performs variable replacement on the - entire test source file before the test gets executed, and saves the - updated version in a temporary file (log/test[num]) so that all test - case readers/servers can use that version (if present) and thus enjoy - the powers of test case variable substitution. + Since Win32 almost always will also have USE_WINSOCK, + we can reduce complexity and always use Sleep there. - This is necessary to allow complete port number freedom. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Test 309 is updated to work with a non-fixed port number thanks to this. + Follow up to #5343 + Closes #5489 -- tests: make 2006-2010 handle different port number lengths +Daniel Stenberg (31 May 2020) +- conncache: download buffer needs +1 size for trailing zero + + Follow-up to c4e6968127e + Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 -- tests: run the sws server on "any port" +Marc Hoersken (31 May 2020) +- azure: use matrix strategy to avoid configuration redundancy - Makes the test servers for HTTP and Gopher pop up on a currently unused - port and runtests adapts to that! + This also includes the following changes: - Closes #5247 - -Marc Hoersken (18 Apr 2020) -- sockfilt: tidy variable naming and data structure in select_ws + - Use the same timeout for all jobs on Linux (60 minutes) + and Windows (90 minutes) + - Use CLI stable apt-get install -y instead of apt install + which warns about that and run apt-get update first + - Enable MQTT for Windows msys2 builds instead of + legacy msys1 builds + - Add ./configure --prefix parameter to the msys2 builds + - The MSYSTEM environment variable is now preset inside + the container images for the msys2 builds - This commit does not introduce any logical changes to the code. + Note: on Azure Pipelines the matrix strategy is basically + just a simple list of job copies and not really a matrix. - Reviewed-by: Jay Satiro and Marcel Raad - Closes #5238 - -Daniel Stenberg (17 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] + Closes #5468 - libssh: Use new ECDSA key types to check known hosts +Daniel Stenberg (30 May 2020) +- build: disable more code/data when built without proxy support - From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA - keys depending on the curve. + Added build to travis to verify - Signed-off-by: Anderson Toshiyuki Sasaki - Fixes #5252 - Closes #5253 + Closes #5466 -Marcel Raad (17 Apr 2020) -- appveyor: add Unicode winbuild jobs +- url: alloc the download buffer at transfer start - These are cheap as they don't build tests. + ... and free it as soon as the transfer is done. It removes the extra + alloc when a new size is set with setopt() and reduces memory for unused + easy handles. - Closes https://github.com/curl/curl/pull/5063 - -Daniel Stenberg (16 Apr 2020) -- mqttd: s/errno/SOCKERRNO + In addition: the closure_handle now doesn't use an allocated buffer at + all but the smallest supported size as a stack based one. - To behave proper on Windows - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 - Closes #5241 + Closes #5472 -- buildconf: use find -execdir instead, remove -print and the ares files +- timeouts: change millisecond timeouts to timediff_t from time_t - Follow-up to 1e41bec96a6e + For millisecond timers we like timediff_t better. Also, time_t can be + unsigned so returning a negative value doesn't work then. - Suggested-by: Marc Hörsken - -- [Alexander V. Tikhonov brought this change] + Closes #5479 - buildconf: avoid using tempfile when removing files +Marc Hoersken (30 May 2020) +- select: add overflow checks for timeval conversions - Closes #5213 - -- copyright: bump the copyright year range - -- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords - -- [JP Mens brought this change] - - docs/MQTT: replace confusing 80 by 75 + Using time_t and suseconds_t if suseconds_t is available, + long on Windows (maybe others in the future) and int elsewhere. + + Also handle case of ULONG_MAX being greater or equal to INFINITE. - I was a bit surprised by the `80`: first thought: what's HTTP doing - here? ;) + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Closes #5236 + Part of #5343 -- [Brad King brought this change] +- select: use timediff_t instead of time_t and int for timeout_ms + + Make all functions in select.[ch] take timeout_ms as timediff_t + which should always be large enough and signed on all platforms + to take all possible timeout values and avoid type conversions. + + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Replaces #5107 and partially #5262 + Related to #5240 and #5286 + Closes #5343 - cmake: Avoid MSVC C4273 warnings in send/recv checks +- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' - We use `check_c_source_compiles` to check possible send/recv signatures - by reproducing the forward declarations from system headers. On Windows - the `winsock2.h` header adds dll linkage settings to its forward - declaration. If ours does not match the compiler warns: + GCC 10 warns about this with warning: implicit conversion + from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] - warning C4273: 'recv': inconsistent dll linkage + Since 'expected_result' is not really of type 'CURLcode' and + it is not exposed in any way, we can just use 'SANITIZEcode'. - Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so - that our linkage is consistent with that from `winsock2.h`. + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad - Fixes #4764 - Closes #5232 + Closes #5476 -Jay Satiro (14 Apr 2020) -- KNOWN_BUGS: Add entry 'Blocking socket operations' - - - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of - known blocking operations. - - - New known bugs entry 'Blocking socket operations in non-blocking API' - that directs to the TODO's list of known blocking operations. +- tests/libtest: fix undefined reference to 'curlx_win32_fopen' - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 + Since curl_setup.h now makes use of curlx_win32_fopen for Win32 + builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, + we need to include the relevant files for tests using fopen, + because the libtest sources are also including curl_setup.h - Reported-by: Marc Hoersken + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - Closes https://github.com/curl/curl/pull/5216 + Follow up to #3784 (ffdddb45d9) + Closes #5475 -Marc Hoersken (14 Apr 2020) -- test2043: use revoked.badssl.com instead of revoked.grc.com - - The certificate of revoked.grc.com has expired on 2020-04-13. +- appveyor: add non-debug plain autotools-based build - Reviewed-by: Jay Satiro + This should enable us to catch linking issues with the + testsuite early, like the one described/fixed in #5475. - Closes #5233 - -- sockfilt: fix broken pipe on Windows to be ready in select_ws + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad - Closes #5228 + Closes #5477 -Daniel Stenberg (14 Apr 2020) +Daniel Stenberg (29 May 2020) - RELEASE-NOTES: synced -- scripts/release-notes: fix duplicate output header - -- github/workflow: enable MQTT in the macOS debug build - -- azure: add mqtt support to one of the Windows builds - -- travis: add mqtt job on Linux - -- tests: add four MQTT tests 1190 - 1193 - -- tests: add the mqtt test server mqttd - -- tests: support hex encoded data and mqtt server +- Revert "buildconf: use find -execdir" + + This partially reverts commit c712009838f44211958854de431315586995bc61. + + Keep the ares_ files removed but bring back the older way to run find, + to make it work with busybox's find, as apparently that's being used. - The mqtt server is started using a "random" port. + Reported-by: Max Peal + Fixes #5483 + Closes #5484 -- [Björn Stenberg brought this change] +- server/sws: fix asan warning on use of uninitialized variable - mqtt: add new experimental protocol +- libssh2: improved error output for wrong quote syntax - Closes #5173 - -- TODO: Consider convenience options for JSON and XML? + Reported-by: Werner Stolz - Closes #5203 + Closes #5474 -- tool: do not declare functions with Curl_ prefix +- mk-lib1521: generate code for testing BLOB options as well - To avoid collision risks with private libcurl symbols when linked with - static versions (or just versions not hiding internal symbols). + Follow-up to cac5374298b3 - Reported-by: hydra3333 on github - Fixes #5219 - Closes #5234 - -- [Nathaniel R. Lewis brought this change] + Closes #5478 - cmake: add aliases so exported target names are available in tree +- configure: repair the check if argv can be written to - Reviewed-by: Brad King - Closes #5206 - -- version: increase buffer space for ssl version output + Due to bad escaping of the test code, the test wouldn't build and thus + result in a negative test result, which would lead to the unconditional + assumption that overwriting the arguments doesn't work and thus curl + would never hide credentials given in the command line, even when it + would otherwise be possible. - To avoid it getting truncated, especially when several SSL backends are - built-in. + Regression from commit 2d4c2152c (7.60.0) - Reported-by: Gisle Vanem - Fixes #5222 - Closes #5226 + Reported-by: huzunhao on github + Fixes #5470 + Closes #5471 -Marc Hoersken (13 Apr 2020) -- cirrus: no longer ignore test 504 which is working again +Peter Wu (28 May 2020) +- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - The test is working again, because TCP blackholing is disabled. - -- appveyor: completely disable tests that fail to timeout early + Otherwise the build might fail due to missing source files, as + demonstrated by the recent keylog.c addition on an existing build dir. - The tests changed from ignored to disabled are tests that are - about connecting to non-listening socket. On AppVeyor these - tests are not reliable, because for some unknown reason the - connect is not timing out before the test time limit is reached. + Closes #5469 -Daniel Stenberg (13 Apr 2020) -- test1908: avoid using fixed port number in test data +Daniel Stenberg (28 May 2020) +- urldata: fix comments: Curl_done() is called multi_done() now - Closes #5225 - -Jay Satiro (12 Apr 2020) -- [Andrew Kurushin brought this change] + ... since 575e885db - schannel: Fix blocking timeout logic +Peter Wu (27 May 2020) +- ngtcp2: use common key log routine for better thread-safety - - Fix schannel_send for the case when no timeout was set. + Tested with ngtcp2 built against the OpenSSL library. Additionally + tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + + The TLS backend (independent of QUIC) may or may not already have opened + the keylog file before. Therefore Curl_tls_keylog_open is always called + to ensure the file is open. + +- wolfssl: add SSLKEYLOGFILE support - Prior to this change schannel would error if the socket was not ready - to send data and no timeout was set. + Tested following the same curl and tshark commands as in commit + "vtls: Extract and simplify key log file handling from OpenSSL" using + WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with + `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. - This commit is similar to parent commit 89dc6e0 which recently made the - same change for SOCKS, for the same reason. Basically it was not well - understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms - but actually means no timeout. + Full support for this feature requires certain wolfSSL build options, + see "Availability note" in lib/vtls/wolfssl.c for details. - Fixes https://github.com/curl/curl/issues/5177 - Closes https://github.com/curl/curl/pull/5221 + Closes #5327 -- socks: Fix blocking timeout logic - - - Document in Curl_timeleft's comment block that returning 0 signals no - timeout (ie there's infinite time left). +- vtls: Extract and simplify key log file handling from OpenSSL - - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. + Create a set of routines for TLS key log file handling to enable reuse + with other TLS backends. Simplify the OpenSSL backend as follows: - Prior to this change if the timeout had a value of 0 and that was passed - to SOCKET_READABLE it would return right away instead of blocking. That - was likely because it was not well understood that when Curl_timeleft - returns 0 it is not a timeout of 0 ms but actually means no timeout. + - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. + - Do not perform dynamic memory allocation when preparing a log entry. + Unless the TLS specifications change we can suffice with a reasonable + fixed-size buffer. + - Simplify state tracking when SSL_CTX_set_keylog_callback is + unavailable. My original sslkeylog.c code included this tracking in + order to handle multiple calls to SSL_connect and detect new keys + after renegotiation (via SSL_read/SSL_write). For curl however we can + be sure that a single master secret eventually becomes available + after SSL_connect, so a simple flag is sufficient. An alternative to + the flag is examining SSL_state(), but this seems more complex and is + not pursued. Capturing keys after server renegotiation was already + unsupported in curl and remains unsupported. - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f + (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) + against an OpenSSL 1.1.1f server configured with: - Closes https://github.com/curl/curl/pull/5220 - -- [Marc Hoersken brought this change] - - gopher: check remaining time left during write busy loop + # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 + openssl s_server -www -tls1 + # Likewise, but fail the server handshake. + openssl s_server -www -tls1 -Verify 2 + # TLS 1.3 test. No need to test the failing server handshake. + openssl s_server -www -tls1_3 - Prior to this change gopher's blocking code would block forever, - ignoring any set timeout value. + Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly + written using Wireshark. For the first and third case, expect four + matches per connection (decrypted Server Finished, Client Finished, HTTP + Request, HTTP Response). For the second case where the handshake fails, + expect a decrypted Server Finished only. - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ + -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ + -dtls.port==4433,http -ohttp.desegment_body:FALSE \ + -Y 'tls.handshake.verify_data or http' - Similar to #5220 and #5221 - Closes #5214 + A single connection can easily be identified via the `tcp.stream` field. -Daniel Stenberg (13 Apr 2020) -- [Dirkjan Bussink brought this change] +Daniel Stenberg (27 May 2020) +- FILEFORMAT: add more features that tests can depend on - gnutls: ensure TLS 1.3 when SRP isn't requested - - When SRP is requested in the priority string, GnuTLS will disable - support for TLS 1.3. Before this change, curl would always add +SRP to - the priority list, effectively always disabling TLS 1.3 support. - - With this change, +SRP is only added to the priority list when SRP - authentication is also requested. This also allows updating the error - handling here to not have to retry without SRP. This is because SRP is - only added when requested and in that case a retry is not needed. - - Closes #5223 +- [Michael Kaufmann brought this change] -Marc Hoersken (12 Apr 2020) -- tests/server: add hidden window to gracefully handle WM_CLOSE + transfer: close connection after excess data has been read - Forward Window events as signals to existing signal event handler. - -- tests/server: add CTRL event handler for Win32 consoles + For HTTP 1.x, it's a protocol error when the server sends more bytes + than announced. If this happens, don't reuse the connection, because the + start position of the next response is undefined. - Forward CTRL events as signals to existing signal event handler. + Closes #5440 -- tests/server: move all signal handling routines to util.[ch] - - Avoid code duplication to prepare for portability enhancements. +- [Estanislau Augé-Pujadas brought this change] -Daniel Stenberg (12 Apr 2020) -- compressed.d: stress that the headers are not modified + Revert "ssh: ignore timeouts during disconnect" - Suggested-by: Michael Osipov - Assisted-by: Jay Satiro - Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 - Closes #5217 + This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in + curl 7.54.1. + + Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html + Closes #5465 -Marc Hoersken (11 Apr 2020) -- tests/server/util.c: use curl_off_t instead of long for pid +- urldata: connect related booleans live in struct ConnectBits - Avoid potential overflow of huge PIDs on Windows. + And remove a few unused booleans! - Related to #5188 - Assisted-by: Marcel Raad + Closes #5461 -- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows - - Since the Windows versions of both programs would write Windows - PIDs to their pidfiles which we cannot handle, we need to use - our known perl.exe Cygwin/msys PID together with exec() in order - to tie the spawned processes to the existance of our perl.exe +- hostip: on macOS avoid DoH when given a numerical IP address - The perl.exe that is executing secureserver.pl and sshserver.pl - has a Cygwin/msys PID, because it is started inside Cygwin/msys. + When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that + numerical IP addresses still need to get "resolved" - but not with DoH. - Related to #5188 + Reported-by: Viktor Szakats + Fixes #5454 + Closes #5459 -- tests: add Windows compatible pidwait like pidkill and pidterm +- ngtcp2: cleanup memory when failing to connect - Related to #5188 + Reported-by: Peter Wu + Fixes #5447 (the ngtcp2 side of it) + Closes #5451 -- tests: fix conflict between Cygwin/msys and Windows PIDs - - Add 65536 to Windows PIDs to allow Windows specific treatment - by having disjunct ranges for Cygwin/msys and Windows PIDs. - - See also: - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=b5e1003722cb14235c4f166be72c09acdffc62ea - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe +- quiche: clean up memory properly when failing to connect - Replaces #5178 - Closes #5188 - -Daniel Stenberg (11 Apr 2020) -- RELEASE-NOTES: synced + Addresses the quiche side of #5447 + Reported-by: Peter Wu + Closes #5450 -- release-notes.pl: detect the start of the references in cleanup mode +- cleanup: use a single space after equals sign in assignments -- Revert "file: on Windows, refuse paths that start with \\" +- url: accept "any length" credentials for proxy auth - This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. + They're only limited to the maximum string input restrictions, not to + 256 bytes. - Reminded-by: Chris Roberts - Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html + Added test 1178 to verify - Closes #5215 + Reported-by: Will Roberts + Fixes #5448 + Closes #5449 -Jay Satiro (11 Apr 2020) -- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE - - - If loss of data may occur converting a timediff_t to time_t and - the time value is > TIME_T_MAX then treat it as TIME_T_MAX. - - This is a follow-up to 8843678 which removed the (time_t) typecast - from the macros so that conversion warnings could be identified. - - Closes https://github.com/curl/curl/pull/5199 +- [Maksim Stsepanenka brought this change] -- test1148: tolerate progress updates better (again) - - - Ignore intermediate progress updates. - - - Support locales that use a character other than period as decimal - separator (eg 100,0%). - - test1148 checks that the progress finishes at 100% and has the right - bar width. Prior to this change the test assumed that the only progress - reported for such a quick transfer was 100%, however in rare instances - (like in the CI where transfer time can slow considerably) there may be - intermediate updates. For example, below is stderrlog1148 from a failed - CI run with explicit \r and \n added (it is one line; broken up so that - it's easier to understand). - - \r - \r################################## 48.3% - \r######################################################################## 100.0% - \n + test1167: fixes in badsymbols.pl - Closes https://github.com/curl/curl/pull/5194 - -Marc Hoersken (10 Apr 2020) -- sshserver.pl: use cached Win32 environment check variable + Closes #5442 -- appveyor: partially revert 3413a110 to keep build without proxy +- altsvc: fix parser for lines ending with CRLF - Ref: #5211 and #4526 - Reported-by: Marcel Raad - -- appveyor: ignore failing 'connect to non-listening proxy' tests + Fixed the alt-svc parser to treat a newline as end of line. - Closes #5211 - -- CI/macos: convert CRLF to LF and align indentation - -Daniel Stenberg (9 Apr 2020) -- url: allow non-HTTPS altsvc-matching for debug builds + The unit tests in test 1654 were done without CRLF and thus didn't quite + match the real world. Now they use CRLF as well. - This is already partly supported but this part was missing. - Reported-by: James Fuller + Reported-by: Peter Wu + Assisted-by: Peter Wu + Assisted-by: Jay Satiro + Fixes #5445 + Closes #5446 + +Viktor Szakats (25 May 2020) +- all: fix codespell errors - Closes #5205 + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg + Closes https://github.com/curl/curl/pull/5452 -- server/resolve: remove AI_CANONNAME to make macos tell the truth +Peter Wu (25 May 2020) +- ngtcp2: fix build with current ngtcp2 master implementing draft 28 - With this bit set, my mac successfully resolves "ip6-localhost" when in - fact there is no such host known to my machine! That in turn made test - 241 wrongly execute and fail. + Based on client.cc changes from ngtcp2. Tested with current git master, + ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. - Closes #5202 + Fixes #5444 + Closes #5443 -- runtests: fix warning about using an undefined variable +Daniel Stenberg (25 May 2020) +- RELEASE-NOTES: synced - Follow-up from 4d939ef6ceb2db1 + moved the new setopts up to a "change" -- release-notes: fix the initial reference list output +- RELEASE-NOTES: synced -- github actions: run when pushed to master or */ci + PRs - - Avoid double-builds when using "local" branches for PRs. For both macos - and fuzz jobs. +- copyright: updated year ranges out of sync - Closes #5201 - -- runtests: provide nicer errormsg when protocol "dump" file is empty + ... and whitelisted a few more files in the the copyright.pl script. - [Gilles Vollant brought this change] - schannel: support .P12 or .PFX client certificates - - Used with curl command line option like this: --cert - : --cert-type p12 + setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency - Closes #5193 + Closes #5431 -- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - - test1294: "split request" being when the entire request isn't sent in - the first go, and the remainder is sent in the PERFORM state. A GET - request is otherwise not sending anything during PERFORM. +- curl: remove -J "informational" written on stdout - test1295: same kind of split but with POST + curl would previously show "curl: Saved to filename 'name from header'" + if -J was used and a name was picked from the Content-Disposition + header. That output could interfer with other stdout output, such as -w. - Closes #5197 + This commit removes that output line. + Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html + Reported-by: Коваленко Анатолий Викторович + Closes #5435 -- http: don't consider upload done if the request isn't completely sent off +Peter Wu (22 May 2020) +- travis: simplify quiche build instructions wrt boringssl - Fixes #4919 - Closes #5197 + quiche builds boringssl as static library, reuse that instead of + building another shared library. + + Closes #5438 -- http: allow Curl_add_buffer_send() to do a short first send by force +- configure: fix pthread check with static boringssl - In a debug build, settting the environment variable "CURL_SMALLREQSEND" - will make the first HTTP request send not send more bytes than the set - amount, thus ending up verifying that the logic for handling a split - HTTP request send works correctly. + A shared boringssl/OpenSSL library requires -lcrypto only for linking. + A static build additionally requires `-ldl -lpthread`. In the latter + case `-lpthread` is added to LIBS which prevented `-pthread` from being + added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. -- connect: store connection info for QUIC connections +Daniel Stenberg (22 May 2020) +- Revert "sendf: make failf() use the mvsnprintf() return code" + + This reverts commit 74623551f306990e70c7c5515b88972005604a74. - Restores the --head functionality to the curl utility which extracts - 'protocol' that is stored that way. + Instead mark the function call with (void). Getting the return code and + using it instead triggered Coverity warning CID 1463596 because + snprintf() can return a negative value... - Reported-by: James Fuller - Fixes #5196 - Closes #5198 + Closes #5441 -- tests/README: update the port numbers list +- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' - Since the pipelining server is long gone. - Reported-by: James Fuller + Reported-by: Billyzou0741326 on github + Fixes #5432 + Closes #5436 -- select: remove typecast from SOCKET_WRITABLE/READABLE macros - - So that they don't hide conversions-by-mistake +- tests/server/util.h: add extern to silence compiler warning - Reviewed-by: Jay Satiro - Closes #5190 + Follow-up from a3b0699d5c1 -- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also +- typecheck-gcc.h: fix the OFF_T check - Closes #5192 - -- release-notes: output trailing references sorted numerically - -- cleanup: correct copyright year range on a few files - -- configure: remove use of -vec-report0 from CFLAGS with icc + The option number also needs to be less than CURLOPTTYPE_BLOB. - ... as it apparently isn't (always) supported. - Reported-by: Alain Miniussi - Fixes #5096 - Closes #5191 + Follow-up to cac5374298 + Reported-by: Jeroen Ooms + Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 -- warnless: remove code block for icc that didn't work +- TODO: --dry-run - Reported-by: Alain Miniussi - Fixes #5096 + Closes #5426 -Marc Hoersken (6 Apr 2020) -- dist: add missing setup-win32.h +- TODO: Ratelimit or wait between serial requests - Follow up to d820224b8b - -Daniel Stenberg (6 Apr 2020) -- RELEASE-NOTES: synced + Closes #5406 -- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance +- tool_paramhlp: fixup C89 mistake - This script helps putting entries in the RELEASE-NOTES using a coherent - style and sorting with a minimal human editing effort - as long as the - first line in the commit message is good enough! There's a short howto - at the top of the file. + Follow-up to c5f0a9db22. -- [Dennis Felsing brought this change] +- [Siva Sivaraman brought this change] - configure: don't check for Security.framework when cross-compiling + tool_paramhlp: fixed potentially uninitialized strtol() variable - Since it checks for the local file, not the cross-compiled one. + Seems highly unlikely to actually be possible, but better safe than + sorry. - Closes #5189 + Closes #5417 -- TODO: Option to make -Z merge lined based outputs on stdout - - Closes #5175 +- [Siva Sivaraman brought this change] -- lib: never define CURL_CA_BUNDLE with a getenv - - - it breaks the build (since 6de756c9b1de34b7a1) - - it's not documented and not consistent across platforms - - the curl tool does that getenv magic + tool_operate: fixed potentially uninitialized variables - Bug: https://github.com/curl/curl/commit/6de756c#r38127030 - Reported-by: Gisle Vanem + ... in curl_easy_getinfo() calls. They're harmless but clearing the + variables makes the code safer and comforts the reader. - Closes #5187 - -Marc Hoersken (5 Apr 2020) -- lib670: use the same Win32 API check as all other lib tests + Closes #5416 -- appveyor: use random test server ports based upon APPVEYOR_API_URL - - Avoid conflicts of test server ports with AppVeyor API on localhost. +- sha256: move assign to the declaration line - Closes #5034 + Follow-up to fae30656. Should've been squashed with that commit... -- appveyor: sort builds by type and add two new variants - - Related to #5034 and #5063 +- [Siva Sivaraman brought this change] -- appveyor: show failed tests in log even if test is ignored + sha256: fixed potentially uninitialized variable - And print API response with newline only if there is one - -- appveyor: turn disabled tests into ignored result tests + Closes #5414 -Daniel Stenberg (5 Apr 2020) -- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" +- sendf: make failf() use the mvsnprintf() return code - Fixed with #5170 (commit 23a870f2fd041278) + ... and avoid a strlen() call. Fixes a MonocleAI warning. + + Reported-by: MonocleAI + Fixes #5413 + Closes #5420 -- test1566: verify --etag-compare that gets a 304 back +- hostip: make Curl_printable_address not return anything - Verifies the fix in #5183 + It was not used much anyway and instead we let it store a blank buffer + in case of failure. - Closes #5186 - -- [Kwon-Young Choi brought this change] + Reported-by: MonocleAI + Fixes #5411 + Closes #5418 - CURLINFO_CONDITION_UNMET: return true for 304 http status code +- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) - In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the - output file if the server did not transfered a file based on time - condition. In the same manner, getting a 304 HTTP response back from the - server, for example after passing a custom If-Match-* header, also - fulfill this condition. + They're done on purpose, make that visible in the code. + Reported-by: MonocleAI + Fixes #5412 + Closes #549 + +- TODO: forbid TLS post-handshake auth and do TLS record padding - Fixes #5181 - Closes #5183 + Closes #5396 + Closes #5398 -- [Kwon-Young Choi brought this change] +- RELEASE-NOTES: synced - curl: allow both --etag-compare and --etag-save with same file name - - This change inverse the order of processing for the --etag-compare and - --etag-save option to process first --etag-compare. This in turn allows - to use the same file name to compare and save an etag. +- dynbuf: return NULL when there's no buffer length - The original behavior of not failing if the etag file does not exists is - conserved. + ... as returning a "" is not a good idea as the string is supposed to be + allocated and returning a const string will cause issues. - Fixes #5179 - Closes #5180 + Reported-by: Brian Carpenter + Follow-up to ed35d6590e72c + Closes #5405 -Viktor Szakats (4 Apr 2020) -- windows: enable UnixSockets with all build toolchains - - Extend existing unix socket support in Windows builds to be - enabled for all toolchain vendors or versions. (Previously - it was only supported with certain MSVC versions + more recent - Windows 10 SDKs) +Peter Wu (16 May 2020) +- travis: upgrade to bionic, clang-9, improve readability - Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ - Ref: https://github.com/curl/curl/issues/5162 - Closes: https://github.com/curl/curl/pull/5170 - -Daniel Stenberg (4 Apr 2020) -- KNOWN_BUGS: Store TLS context per transfer instead of per connection + Changes, partially to reduce build failures from external dependencies: + - Upgrade Ubuntu and drop unnecessary third-party repos. + - Properly clone apt config to ensure retries. + - Upgrade to clang-9 from the standard repos. + - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey + fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use + focal everywhere yet since Travis CI has not documented this option. + In focal, python-impacket (Py2.7) has been removed, leaving only + python3-impacket. Since it is only needed for SMB tests and not SSH, + skip it for the libssh job since it might need more work. + - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. - Closes #5102 - -Marc Hoersken (3 Apr 2020) -- sockfilt: remove redundancy in timeout handling + Non-functional cleanups: + - Simplify test matrix, drop redundant os and compiler keys. + - Deprecation fixes: remove sudo, rename matrix -> jobs. + - Every job has an 'env' key, put this key first in a list item. - And update other logmsg output in select_ws on Windows. + Closes #5370 -- sockfilt: fix handling of ready closed sockets on Windows - - Replace the incomplete workaround regarding FD_CLOSE - only signalling once by instead doing a pre-check with - standard select and storing the result for later use. +- travis: whitespace-only changes for consistency - select keeps triggering on closed sockets on Windows while - WSAEventSelect fires only once with data still available. - By doing the pre-check we do not run in a deadlock - due to waiting forever for another FD_CLOSE event. - -- sockfilt: fix race-condition of waiting threads and event handling + Automatically apply a consistent indentation with: - Fix race-condition of waiting threads finishing while events are - already being processed which lead to invalid or skipped events. + python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' - Use mutex to check for one event at a time or do post-processing. - In addition to mutex-based locking use specific event as signal. + followed by manually re-indenting three comments. - Closes #5156 - -Daniel Stenberg (2 Apr 2020) -- [Leo Neat brought this change] + Closes #5370 - CI-fuzz: increase fuzz time to 40 minutes +- CMake: add libssh build support - Closes #5174 + Closes #5372 -Marc Hoersken (2 Apr 2020) -- CI: increase Azure Pipelines timeouts due to performance issues +Daniel Stenberg (15 May 2020) +- KNOWN_BUGS: wolfssh: publickey auth doesn't work - The current demand on Azure negatively impacts the CI performance. - -- runtests.pl: log host OS as detected by Perl environment - -- ftpserver.pl: log before and after data connection is closed - -Daniel Stenberg (1 Apr 2020) -- RELEASE-NOTES: synced - -- RELEASE-PROCEDURE.md: run the copyright.pl script! + Closes #4820 -- vquic/ngtcp2.h: update copyright year range +- KNOWN_BUGS: OS400 port requires deprecated IBM library - Follow-up to 0736ee73d346a52 - -- [Daiki Ueno brought this change] - - CI: add build with ngtcp2 + gnutls on Travis CI + Closes #5176 -- [Daiki Ueno brought this change] +- [Vyron Tsingaras brought this change] - vquic: add support for GnuTLS backend of ngtcp2 - - Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile - time. Therefore OpenSSL support needs to be explicitly disabled. + http2: keep trying to send pending frames after req.upload_done - Signed-off-by: Daiki Ueno - Closes #5148 + Fixes #1410 + Closes #5401 -- [Gisle Vanem brought this change] +- [Gilles Vollant brought this change] - examples/sessioninfo.c: add include to fix compiler warning + setopt: support certificate options in memory with struct curl_blob - Fixes #5171 - -- misc: copyright year updates + This change introduces a generic way to provide binary data in setopt + options, called BLOBs. - Follow-up to 7a71965e9 - -- [Harry Sintonen brought this change] - - build: fixed build for systems with select() in unistd.h + This change introduces these new setopts: - Closes #5169 - -- memdebug: don't log free(NULL) + CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, + CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. - ... it serves no purpose and fills up the log. + Reviewed-by: Daniel Stenberg + Closes #5357 -- cleanup: insert newline after if() conditions +- source cleanup: remove all custom typedef structs - Our code style mandates we put the conditional block on a separate - line. These mistakes are now detected by the updated checksrc. - -- checksrc: warn on obvious conditional blocks on the same line as if() + - Stick to a single unified way to use structs + - Make checksrc complain on 'typedef struct {' + - Allow them in tests, public headers and examples - Closes #5164 - -- [Roger Orr brought this change] - - cmake: add CMAKE_MSVC_RUNTIME_LIBRARY + - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually + typedef different types/structs depending on build conditions. - Fixes #5165 - Closes #5167 + Closes #5338 -- [Daiki Ueno brought this change] +- travis: remove the .checksrc fiddling - ngtcp2: update to git master for the key installation API change - - This updates the ngtcp2 OpenSSL backend to follow the API change in - commit 32e703164 of ngtcp2. - - Notable changes are: - - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace - ngtcp2_crypto_derive_and_install_key - - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key - has been removed +- ftp: make domore_getsock() return the secondary socket properly - Fixes #5166 - Closes #5168 + Previously, after PASV and immediately after the data connection has + connected, the function would only return the control socket to wait for + which then made the data connection simply timeout and not get polled + correctly. This become obvious when running test 1631 and 1632 event- + based. -- [Cyrus brought this change] +- test1632: verify FTP through HTTPS-proxy with connection re-use - SECURITY.md: minor rephrase - - Closes #5158 +- test1631: verify FTP download through HTTPS-proxy -- output.d: quote the URL when globbing +- sws: as last resort, get test number from server cmd file - Some shells do globbing of their own unless the URL is quoted, so maybe - encourage this. + If it can't be found in the request. Also support --cmdfile to set it to + a custom file name. - Co-authored-by: Jay Satiro - Closes #5160 + runtests.pl always writes this file with the test number in it since a + while back. -- dist: add tests/version-scan.pl to tarball - - ... used in test 1177. +- ftp: shut down the secondary connection properly when SSL is used - Follow-up to a97d826f6de3 - -- test1177: verify that all the CURL_VERSION_ bits are documented + Reported-by: Neal Poole + Fixes #5340 + Closes #5385 -- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented +Marcel Raad (14 May 2020) +- KNOWN_BUGS: adapt 5.5 to recent changes - Considered experimental and therefore we can do this. + It only applies to non-Unicode builds now. + Also merge 5.10 into it as it's effectively a duplicate. - Closes #5157 + Closes https://github.com/curl/curl/pull/3784 -- KNOWN_BUGS: DoH doesn't inherit all transfer options +- curl_setup: support Unicode functions to open files on Windows - Closes #4578 - Closes #4579 - -- KNOWN_BUGS: DoH leaks memory after followlocation + Use them only if `_UNICODE` is defined, in which case command-line + arguments have been converted to UTF-8. - Closes #4592 + Closes https://github.com/curl/curl/pull/3784 -- KNOWN_BUGS: "FTPS needs session reuse" +- tool: support UTF-16 command line on Windows - Closes #4654 - -- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed - -- TODO: Set custom client ip when using haproxy protocol + - use `wmain` instead of `main` when `_UNICODE` is defined [0] + - define `argv_item_t` as `wchar_t *` in this case + - use the curl_multibyte gear to convert the command-line arguments to + UTF-8 - Closes #5125 - -Michael Kaufmann (27 Mar 2020) -- writeout_json: Fix data type issues + This makes it possible to pass parameters with characters outside of + the current locale on Windows, which is required for some tests, e.g. + the IDN tests. Out of the box, this currently only works with the + Visual Studio project files, which default to Unicode, and winbuild + with the `ENABLE_UNICODE` option. - Load long values correctly (e.g. for http_code). + [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 - Use curl_off_t (not long) for: - - size_download (CURLINFO_SIZE_DOWNLOAD_T) - - size_upload (CURLINFO_SIZE_UPLOAD_T) + Ref: https://github.com/curl/curl/issues/3747 + Closes https://github.com/curl/curl/pull/3784 + +- curl_multibyte: add to curlx - The unit for these values is bytes/second, not microseconds: - - speed_download (CURLINFO_SPEED_DOWNLOAD_T) - - speed_upload (CURLINFO_SPEED_UPLOAD_T) + This will also be needed in the tool and tests. - Fixes #5131 - Closes #5152 + Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 + Closes https://github.com/curl/curl/pull/3784 -Daniel Stenberg (27 Mar 2020) -- mailmap: fixup a few author names/fields +Daniel Stenberg (14 May 2020) +- url: make the updated credentials URL-encoded in the URL - Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and - Svyatoslav Mishyn + Found-by: Gregory Jefferis + Reported-by: Jeroen Ooms + Added test 1168 to verify. Bug spotted when doing a redirect. + Bug: https://github.com/jeroen/curl/issues/224 + Closes #5400 -- version: add 'cainfo' and 'capath' to version info struct +- tests: add https-proxy support to the test suite - Suggested-by: Timothe Litt - URL: https://curl.haxx.se/mail/lib-2020-03/0090.html - Reviewed-by: Jay Satiro + Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like + HTTP proxy but with a full TLS connection to the proxy. - Closes #5150 + Closes #5399 -- RELEASE-NOTES: synced +- mailmap: James Fuller + +- [Major_Tom brought this change] -Jay Satiro (26 Mar 2020) -- SSLCERTS.md: Fix example code for setting CA cert file + vauth/cleartext: fix theoretical integer overflow - Prior to this change the documentation erroneously said use - CURLOPT_CAPATH to set a CA cert file. + Fix theoretical integer overflow in Curl_auth_create_plain_message. - Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html - Reported-by: Timothe Litt + The security impact of the overflow was discussed on hackerone. We + agreed this is more of a theoretical vulnerability, as the integer + overflow would only be triggerable on systems using 32-bits size_t with + over 4GB of available memory space for the process. - Closes https://github.com/curl/curl/pull/5151 + Closes #5391 -Marc Hoersken (26 Mar 2020) -- sockfilt: add logmsg output to select_ws_wait_thread on Windows +Jay Satiro (13 May 2020) +- curl.1: Quote globbed URLs - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + - Quote the globbing example URLs that contain characters [] {} since + otherwise they may be interpreted as shell metacharacters. - Closes #5086 + Bug: https://github.com/curl/curl/issues/5388 + Reported-by: John Simpson + + Closes https://github.com/curl/curl/pull/5394 -Daniel Stenberg (26 Mar 2020) -- docs/make: generate curl.1 from listed files only +Daniel Stenberg (14 May 2020) +- checksrc: enhance the ASTERISKSPACE and update code accordingly - Previously it rendered the page from files matching "*.d" in the correct - directory, which worked fine in git builds when the files were added but - made it easy to forget adding the files to the dist. + Fine: "struct hello *world" - Now, only man page sections listed in DPAGES in Makefile.inc will be - used, thus "forcing" us to update this to get the man page right and get - it included in the dist at the same time. + Not fine: "struct hello* world" (and variations) - Ref: #5146 - Closes #5149 + Closes #5386 -- openssl: adapt to functions marked as deprecated since version 3 - - OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES - functions we use. - - Fix the MD4 and SSL_CTX_load_verify_locations warnings. +- docs/options-in-versions: which version added each cmdline option - In configure, detect OpenSSL v3 and if so, inhibit the deprecation - warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and - until we rewrite the code to use non-deprecated functions we better - ignore these warnings as they don't help us. + Added test 971 to verify that the list is in sync with the files in + cmdline-opts. The check also verifies that .d-files that uses Added: + specify the same version number as the options-in-versions file does. - Closes #5139 + Closes #5381 -- dist: add mail-rcpt-allowfails.d to the tarball +- docs: unify protocol lists - Reported-by: Maksim Stsepanenka - Reviewed-by: Jat Satiro + We boast support for 25 transfer protocols. Make sure the lists are + consistent - Closes #5146 + Closes #5384 diff --git a/libs/libcurl/docs/COPYING b/libs/libcurl/docs/COPYING index 9d9e4af8d8..48f144758e 100644 --- a/libs/libcurl/docs/COPYING +++ b/libs/libcurl/docs/COPYING @@ -1,6 +1,6 @@ COPYRIGHT AND PERMISSION NOTICE -Copyright (c) 1996 - 2020, Daniel Stenberg, , and many +Copyright (c) 1996 - 2021, Daniel Stenberg, , and many contributors, see the THANKS file. All rights reserved. diff --git a/libs/libcurl/docs/THANKS b/libs/libcurl/docs/THANKS index fcac1f5f30..69c3c11dca 100644 --- a/libs/libcurl/docs/THANKS +++ b/libs/libcurl/docs/THANKS @@ -4,6 +4,7 @@ If you have contributed but are missing here, please let us know! +0xflotus on github 1ocalhost on github 3dyd on github Aaro Koskinen @@ -151,6 +152,7 @@ Andrew Moise Andrew Potter Andrew Robbins Andrew Wansink +Andrey Gursky Andrey Labunets Andrii Moiseiev Andrius Merkys @@ -305,6 +307,7 @@ Bruno Thomsen Bryan Henderson Bryan Kemp bsammon on github +Bubu on github buzo-ffm on github bxac on github Bylon2 on github @@ -489,6 +492,7 @@ David Blaikie David Byron David Cohen David E. Narváez +David Earl David Eriksson David Garske David Houlder @@ -525,6 +529,7 @@ Denis Baručić Denis Chaplygin Denis Feklushkin Denis Goleshchikhin +Denis Laxalde Denis Ollier Dennis Clarke Dennis Felsing @@ -569,6 +574,7 @@ Dmitry Mikhirev Dmitry Popov Dmitry Rechkin Dmitry S. Baikov +Dmitry Wagin dnivras on github Dolbneff A.V Domenico Andreoli @@ -585,6 +591,7 @@ Douglas E. Wegscheid Douglas Kilpatrick Douglas Mencken Douglas R. Horner +Douglas R. Reno Douglas Steinwand Dov Murik dpull on github @@ -661,6 +668,7 @@ Erik Jacobsen Erik Janssen Erik Johansson Erik Minekus +Erik Olsson Ernest Beinrohr Ernst Sjöstrand Erwan Legrand @@ -698,6 +706,7 @@ Felix Yan Feng Tu Fernando Muñoz Filip Salomonsson +Flameborn on github Flavio Medeiros Florian Pritz Florian Schoppmann @@ -731,6 +740,7 @@ FuccDucc on github fullincome on github Gabriel Kuri Gabriel Sjoberg +Ganesh Kamath Garrett Holmstrom Gary Maxwell Gaurav Malhotra @@ -811,6 +821,7 @@ Hannes Magnusson Hanno Böck Hanno Kranzhoff Hans Steegers +Hans-Christian Noren Egtvedt Hans-Jurgen May Hao Wu Hardeep Singh @@ -834,9 +845,11 @@ Henry Ludemann Henry Roeland Herve Amblard Hidemoto Nakada +Himanshu Gupta Ho-chi Chen Hoi-Ho Chan Hongli Lai +Hongyi Zhao Howard Blaise Howard Chu hsiao yi @@ -853,6 +866,7 @@ Ian Fette Ian Ford Ian Gulliver Ian Lynagh +Ian Spence Ian Turner Ian Wilkes Ignacio Vazquez-Abrams @@ -864,6 +878,7 @@ Igor Polyakov Ihor Karpenko ihsinme on github Iida Yosiaki +Ikko Ashimine Ilguiz Latypov Ilja van Sprundel Ilya Kosarev @@ -978,6 +993,7 @@ Jeremy Lin Jeremy Maitin-Shepard Jeremy Pearson Jeremy Tan +Jeremy Thibault Jeroen Koekkoek Jeroen Ooms Jerome Muffat-Meridol @@ -1065,6 +1081,7 @@ Jon Spencer Jon Torrey Jon Travis Jon Turner +Jon Wilkes Jonas Forsman Jonas Minnberg Jonas Schnelli @@ -1100,6 +1117,7 @@ Judson Bishop Juergen Hoetzel Juergen Wilke Jukka Pihl +Julian Montes Julian Noble Julian Ospald Julian Romero Nieto @@ -1163,6 +1181,7 @@ Kevin R. Bulgrien Kevin Reed Kevin Roth Kevin Smith +Kevin Ushey Kim Minjoong Kim Rinnewitz Kim Vandry @@ -1378,6 +1397,7 @@ Mathieu Legare Mats Lidell Matt Arsenault Matt Ford +Matt Holt Matt Kraai Matt McClure Matt Veenstra @@ -1392,6 +1412,7 @@ Matthew Hall Matthew Kerwin Matthew Whitehead Matthias Bolte +Matthias Gatto Matthias Naegler Mattias Fornander Matus Uzak @@ -1473,6 +1494,7 @@ Mike Crowe Mike Dobbs Mike Dowell Mike Frysinger +Mike Gelfand Mike Giancola Mike Hasselberg Mike Henshaw @@ -1482,6 +1504,7 @@ Mike Norton Mike Power Mike Protts Mike Revi +Mike Tzou Miklos Nemeth Miloš Ljumović Mingliang Zhu @@ -1576,6 +1599,7 @@ Ofer Okhin Vasilij Ola Mork Olaf Flebbe +Olaf Hering Olaf Stüben Oleg Pudeyev Olen Andoni @@ -1606,6 +1630,7 @@ Palo Markovic Paolo Mossino Paolo Piacentini Paras Sethia +parazyd on github Pascal Gaudette Pascal Terjan Pasha Kuznetsov @@ -1886,6 +1911,7 @@ Ryan Winograd Ryuichi KAWAMATA Rémy Léone S. Moonesamy +Sai Ram Kunala Salah-Eddin Shaban Saleem Abdulrasool Salvador Dávila @@ -1922,6 +1948,7 @@ Scott McCreary Sean Boudreau Sean Burford Sean MacLennan +Sean McArthur Sean Miller Sebastiaan van Erk Sebastian Haglund @@ -2180,6 +2207,7 @@ Ulf Samuelsson Ulrich Doehner Ulrich Telle Ulrich Zadow +UrsusArctos on github Valentin David Valentyn Korniienko Valerii Zapodovnikov @@ -2247,6 +2275,7 @@ Wouter Van Rooy Wu Yongzheng Wyatt O'Day Xavier Bouchoux +XhmikosR on github XhstormR on github Xiang Xiao Xiangbin Li @@ -2295,4 +2324,5 @@ zzq1015 on github Štefan Kremeň Коваленко Анатолий Викторович Никита Дорохин +不确定 加藤郁之 -- cgit v1.2.3