From 133594758488f47f6d1d6d01fcb95483526df40b Mon Sep 17 00:00:00 2001 From: dartraiden Date: Thu, 25 Jun 2020 10:07:52 +0300 Subject: libcurl: update to 7.71.0 --- libs/libcurl/docs/CHANGES | 10096 +++++++++++++++++++++++--------------------- 1 file changed, 5191 insertions(+), 4905 deletions(-) (limited to 'libs/libcurl/docs/CHANGES') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index de44c16641..5061b40139 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,7214 +6,7500 @@ Changelog -Version 7.70.0 (29 Apr 2020) +Version 7.71.0 (23 Jun 2020) -Daniel Stenberg (29 Apr 2020) -- RELEASE-NOTES: 7.70.0 +Daniel Stenberg (23 Jun 2020) +- RELEASE-NOTES: curl 7.71.0 release -- THANKS: synced with the 7.70.0 release +- THANKS: curl 7.71.0 additions -- headers: copyright range fix +- url: make sure pushed streams get an allocated download buffer + + Follow-up to c4e6968127e876b0 + + When a new transfer is created, as a resuly of an acknowledged push, + that transfer needs a download buffer allocated. + + Closes #5590 -- [Rikard Falkeborn brought this change] +Jay Satiro (22 Jun 2020) +- openssl: Don't ignore CA paths when using Windows CA store + + This commit changes the behavior of CURLSSLOPT_NATIVE_CA so that it does + not override CURLOPT_CAINFO / CURLOPT_CAPATH, or the hardcoded default + locations. Instead the CA store can now be used at the same time. + + The change is due to the impending release. The issue is still being + discussed. The behavior of CURLSSLOPT_NATIVE_CA is subject to change and + is now documented as experimental. + + Ref: bc052cc (parent commit) + Ref: https://github.com/curl/curl/issues/5585 - doh: Constify some input pointers +- tool_operate: Don't use Windows CA store as a fallback - Closes #5306 + Background: + + 148534d added CURLSSLOPT_NATIVE_CA to use the Windows OS certificate + store in libcurl w/ OpenSSL on Windows. CURLSSLOPT_NATIVE_CA overrides + CURLOPT_CAINFO if both are set. The curl tool will fall back to + CURLSSLOPT_NATIVE_CA if it could not find a certificate bundle to set + via CURLOPT_CAINFO. + + Problem: + + libcurl may be built with hardcoded paths to a certificate bundle or + directory, and if CURLSSLOPT_NATIVE_CA is used then those paths are + ignored. + + Solution: + + A solution is still being discussed but since there's an impending + release this commit removes using CURLSSLOPT_NATIVE_CA in the curl tool. + + Ref: https://github.com/curl/curl/issues/5585 -- nss: check for PK11_CreateDigestContext() returning NULL +- openssl: Fix CA fallback logic for OpenSSL 3.0 build - ... to avoid crashes! + Prior to this change I assume a build error would occur when + CURL_CA_FALLBACK was used. - Reported-by: Hao Wu - Fixes #5302 - Closes #5303 + Closes https://github.com/curl/curl/pull/5587 -- travis: bump the wolfssl CI build to use 4.4.0 +Daniel Stenberg (22 Jun 2020) +- copyright: update mismatched copyright years + +- test1460: verify that -Ji is not ok + +- tool_getparam: -i is not OK if -J is used - Closes #5301 + Reported-by: sn on hackerone + Bug: https://curl.haxx.se/docs/CVE-2020-8177.html -- copyright updates: adjust year ranges +- [Peter Wu brought this change] -Marc Hoersken (26 Apr 2020) -- CI: do not include */ci branches in PR builds + CMake: ignore INTERFACE_LIBRARY targets for pkg-config file - Align Azure Pipelines with GitHub Actions. + Reviewed-by: Marcel Raad + Fixes #5512 + Closes #5517 -Daniel Stenberg (25 Apr 2020) -- runtests: check for the disabled tests relative srcdir +- [Valentyn Korniienko brought this change] + + multibyte: Fixed access-> waccess to file for Windows Plarform - To make it work correctly for out-of-tree builds. + Reviewed-by: Marcel Raad + Closes #5580 + +- altsvc: bump to h3-29 - Follow-up to 75e8feb6fb08b + Closes #5584 + +- urlglob: treat literal IPv6 addresses with zone IDs as a host name - Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 - Reported-by: Marcel Raad - Closes #5297 + ... and not as a "glob". Now done by passing the supposed host to the + URL parser which supposedly will do a better job at identifying "real" + numerical IPv6 addresses. + + Reported-by: puckipedia on github + Fixes #5576 + Closes #5579 -- runtests: revert commenting out a line I did for debugging +- test1179: verify error message for non-existing cmdline option + +- tool_getparam: repair the error message for unknown flag - Follow-up to 11091cd4d. It was not meant to be pushed! + Follow-up to 9e5669f3880674 + Detected by Coverity CID 1464582 ("Logically dead code") + + Closes #5577 -- smtp: set auth correctly +- FILEFORMAT: describe verify/stderr + +- connect: improve happy eyeballs handling - Regression since 7.69.0 and 68fb25fa3fcff. + For QUIC but also for regular TCP when the second family runs out of IPs + with a failure while the first family is still trying to connect. - The code wrongly assigned 'from' instead of 'auth' which probably was a - copy and paste mistake from other code, leading to that auth could - remain NULL and later cause an error to be returned. + Separated the timeout handling for IPv4 and IPv6 connections when they + both have a number of addresses to iterate over. + +- ngtcp2: never call fprintf() in lib code in release version + +- ngtcp2: fix happy eyeballs quic connect crash - Assisted-by: Eric Sauvageau - Fixes #5294 - Closes #5295 + Reported-by: Peter Wu + Fixes #5565 + Closes #5568 -Marcel Raad (25 Apr 2020) -- lib: clean up whitespace +- select: remove the unused ELAPSED_MS() macro - This fixes CodeFactor warnings. + Closes #5573 -Daniel Stenberg (25 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] +Marc Hoersken (17 Jun 2020) +- [rcombs brought this change] - libssh: avoid options override by configuration files + multi: implement wait using winsock events - Previously, options set explicitly through command line options could be - overridden by the configuration files parsed automatically when - ssh_connect() was called. + This avoids using a pair of TCP ports to provide wakeup functionality + for every multi instance on Windows, where socketpair() is emulated + using a TCP socket on loopback which could in turn lead to socket + resource exhaustion. - By calling ssh_options_parse_config() explicitly, the configuration - files are parsed before setting the options, avoiding the options - override. Once the configuration files are parsed, the automatic - configuration parsing is not executed. + Reviewed-by: Gergely Nagy + Reviewed-by: Marc Hörsken - Fixes #4972 - Closes #5283 - Signed-off-by: Anderson Toshiyuki Sasaki + Closes #5397 -- runtests: when mentions http, kill http/2 too - - Since the http2 test server is a mere proxy that needs to know about the - dynamic port the HTTP server is using, it too needs to get restarted - when the http server is killed. +Daniel Stenberg (17 Jun 2020) +- manpage: add three missing environment variables - A regression caused by 80d6515. + CURL_SSL_BACKEND, QLOGDIR and SSLKEYLOGFILE - Fixes #5289 - Closes #5291 + Closes #5571 -- [Yuri Slobodyanyuk brought this change] +- RELEASE-NOTES: synced - docs: fix two typos +- configure: for wolfSSL, check for the DES func needed for NTLM - Closes #5292 + Also adds pkg-config support for the wolfSSL detection. -- [Emil Engler brought this change] +- [Ruurd Beerstra brought this change] - tests/git: ignore mqttd and port files + ntlm: enable NTLM support with wolfSSL - Closes #5290 + When wolfSSL is built with its OpenSSL API layer, it fetures the same DES* + functions that OpenSSL has. This change take advantage of that. + + Co-authored-by: Daniel Stenberg + Closes #5556 + Fixes #5548 -- tests: make runtests check that disabled tests exists +- http: move header storage to Curl_easy from connectdata - ... and error out if so. Removed '536' from DISABLED as there is no such - test file. + Since the connection can be used by many independent requests (using + HTTP/2 or HTTP/3), things like user-agent and other transfer-specific + data MUST NOT be kept connection oriented as it could lead to requests + getting the wrong string for their requests. This struct data was + lingering like this due to old HTTP1 legacy thinking where it didn't + mattered.. - Closes #5288 - -- test1154: set a proper name + Fixes #5566 + Closes #5567 -- select: make Curl_socket_check take timediff_t timeout +- CODE_REVIEW.md: how to do code reviews in curl - Coverity found CID 1461718: - - Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > - 9223372036854775807L" is always false regardless of the values of its - operands. This occurs as the logical second operand of "||". + Assisted-by: Daniel Gustafsson + Assisted-by: Rich Salz + Assisted-by: Hugo van Kemenade + Assisted-by: James Fuller + Assisted-by: Marc Hörsken + Assisted-by: Jay Satiro - Closes #5240 - -- [i-ky brought this change] + Closes #5555 - libcurl-multi.3: added missing full stop +- altsvc: remove the num field from the altsvc struct - Closes #5285 - -Jay Satiro (22 Apr 2020) -- transfer: Switch PUT to GET/HEAD on 303 redirect + It was superfluous since we have the list.size alredy - Prior to this change if there was a 303 reply to a PUT request then - the subsequent request to respond to that redirect would also be a PUT. - It was determined that was most likely incorrect based on the language - of the RFCs. Basically 303 means "see other" resource, which implies it - is most likely not the same resource, therefore we should not try to PUT - to that different resource. + Reported-by: Jay Satiro + Fixes #5553 + Closes #5563 + +- version.d: expanded and alpha-sorted - Refer to the discussions in #5237 and #5248 for more information. + Added a few missing features not previously mentioned. Ordered them + alphabetically. - Fixes https://github.com/curl/curl/issues/5237 - Closes https://github.com/curl/curl/pull/5248 + Closes #5558 -Daniel Stenberg (22 Apr 2020) -- lib/mk-ca-bundle: skip empty certs +- ABI.md: rename to .md and polish the markdown - Reviewed-by: Emil Engler - Reported-by: Ashwin Metpalli - Fixes #5278 - Closes #5280 + Closes #5562 -- version: skip idn2_check_version() check and add precaution +- HELP-US: add a section for "smaller tasks" - A gcc-10's -fanalyze complaint made me spot and do these improvements. + The point of this section is to meet the CII Best Practices gold level + critera: - Closes #5281 + "The project MUST clearly identify small tasks that can be performed by + new or casual contributors" + + Closes #5560 -- RELEASE-NOTES: synced +- TODO: retry on the redirected-to URL + + Closes #5462 -- [Brian Bergeron brought this change] +- mailmap: Nicolas Sterchele - curl.h: update comment typo +- [Nicolas Sterchele brought this change] + + TODO: remove 19.3 section title - "routines with be invoked" -> "routines will be invoked" + Follow-up to ad6416986755e417c66e2c6, which caused wrong formatting on + curl documentation website - Closes #5279 + Closes #5561 -- [Emil Engler brought this change] +- [Martin V brought this change] - GnuTLS: Don't skip really long certificate fields + test1560: avoid possibly negative association in wording - Closes #5271 + Closes #5549 -- gnutls: bump lowest supported version to 3.1.10 +- share: don't set the share flag it something fails - GnuTLS 3.1.10 added new functions we want to use. That version was - released on Mar 22, 2013. Removing support for older versions also - greatly simplifies the code. + When asking for a specific feature to be shared in the share object, + that bit was previously set unconditionally even if the shared feature + failed or otherwise wouldn't work. - Ref: #5271 - Closes #5276 + Closes #5554 -- mqtt: make NOSTATE get within the debug name array +- buildconf: remove -print from the find command that removes files + + It's just too annoying and unnecessary to get a long list of files shown -- tests: run the RTSP test server on a dynamic port number +- RELEASE-NOTES: synced + +- wording: avoid blacklist/whitelist stereotypes - To avoid port collisions. + Instead of discussing if there's value or meaning (implied or not) in + the colors, let's use words without the same possibly negative + associations. - Closes #5272 + Closes #5546 -- tests: add %NOLISTENPORT and use it +Jay Satiro (9 Jun 2020) +- tool_getparam: fix memory leak in parse_args - The purpose with this variable is to provide a port number that is - reasonably likely to not have a listener on the local host so that tests - can try connect failures against it. It uses port 47 - "reserved" - according to IANA. + Prior to this change in Windows Unicode builds most parsed options would + not be freed. - Updated six tests to use it instead of the previous different ports. + Found using _CrtDumpMemoryLeaks(). - Assisted-by: Emil Engler - Closes #5270 + Ref: https://github.com/curl/curl/issues/5545 -- mqtt: remove code with no purpose +Daniel Stenberg (8 Jun 2020) +- socks: detect connection close during handshake - Detected by Coverity. CID 1462319. + The SOCKS4/5 state machines weren't properly terminated when the proxy + connection got closed, leading to a busy-loop. - "The same code is executed when the condition result is true or false, - because the code in the if-then branch and after the if statement is - identical." + Reported-By: zloi-user on github + Fixes #5532 + Closes #5542 + +- [James Fuller brought this change] + + multi: add defensive check on data->multi->num_alive - Closes #5275 + Closes #5540 -- mqtt: fix Curl_read() error handling while reading remaining length +- Curl_addrinfo: use one malloc instead of three - Detected by Coverity. CID 1462320. + To reduce the amount of allocations needed for creating a Curl_addrinfo + struct, make a single larger malloc instead of three separate smaller + ones. - Closes #5274 + Closes #5533 -- server/tftpd: fix compiler warning - - Follow-up from 369ce38ac1d - Reported-by: Marc Hörsken +- [Alessandro Ghedini brought this change] -- http: free memory when Alt-Used header creation fails due to OOM + quiche: update SSLKEYLOGFILE support - Reported-by: James Fuller - Fixes #5268 - Closes #5269 + quiche now requires the application to explicitly set the keylog path + for each connection, rather than reading the environment variable + itself. + + Closes #5541 -Daniel Gustafsson (20 Apr 2020) -- lib: fix typos in comments and errormessages +- tests: add two simple tests for --login-options - This fixes a few randomly spotted typos in recently merged code, most - notably one in a userfacing errormessage the schannel code. + Test 895 and 896 - as a follow-up to a3e972313b + + Closes #5539 -Daniel Stenberg (20 Apr 2020) -- tests: run the SOCKS test server on a dynamic port number +- ngtcp2: update with recent API changes - Closes #5266 + Syncs with ngtcp2 commit 7e9a917d386d98 merged June 7 2020. + + Assisted-by: Tatsuhiro Tsujikawa + Closes #5538 -- [Johannes Schindelin brought this change] +- [James Fuller brought this change] - multi-ssl: reset the SSL backend on `Curl_global_cleanup()` + socks: remove unreachable breaks in socks.c and mime.c - When cURL is compiled with support for multiple SSL backends, it is - possible to configure an SSL backend via `curl_global_sslset()`, but - only *before* `curl_global_init()` was called. + Closes #5537 + +- tool_cfgable: free login_options at exit - If another SSL backend should be used after that, a user might be - tempted to call `curl_global_cleanup()` to start over. However, we did - not foresee that use case and forgot to reset the SSL backend in that - cleanup. + Memory leak + Reported-by: Geeknik Labs + Fixes #5535 + Closes #5536 + +- libssh2: keep sftp errors as 'unsigned long' - Let's allow that use case. + Remove weird work-around for storing the SFTP errors as int instead of + the "unsigned long" that libssh2 actually returns for SFTP errors. - Fixes #5255 - Closes #5257 - Reported-by: davidedec on github - Signed-off-by: Johannes Schindelin + Closes #5534 -- tests: run the TFTP test server on a dynamic port number +Marc Hoersken (6 Jun 2020) +- timeouts: move ms timeouts to timediff_t from int and long - Picking a dynamic unused port is better than a fixed to avoid the - collision risk. + Now that all functions in select.[ch] take timediff_t instead + of the limited int or long, we can remove type conversions + and related preprocessor checks to silence compiler warnings. - Closes #5265 + Avoiding conversions from time_t was already done in 842f73de. + + Based upon #5262 + Supersedes #5214, #5220 and #5221 + Follow up to #5343 and #5479 + Closes #5490 -- mqtt: improve the state machine +Daniel Stenberg (6 Jun 2020) +- [François Rigault brought this change] + + openssl: set FLAG_TRUSTED_FIRST unconditionally - To handle PUBLISH before SUBACK and more. + On some systems, openssl 1.0.x is still the default, but it has been + patched to contain all the recent security fixes. As a result of this + patching, it is possible for macro X509_V_FLAG_NO_ALT_CHAINS to be + defined, while the previous behavior of openssl to not look at trusted + chains first, remains. - Updated the existing tests and added three new ones. + Fix it: ensure X509_V_FLAG_TRUSTED_FIRST is always set, do not try to + probe for the behavior of openssl based on the existence ofmacros. - Reported-by: Christoph Krey - Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html - Closes #5246 + Closes #5530 -- runtests: always put test number in servercmd file +- server/util: fix logmsg format using curl_off_t argument + + ... this caused segfaults on armv7. + + Regression added in dd0365d560aea5a (7.70.0) + + Reviewed-by: Jay Satiro + Closes #5529 - RELEASE-NOTES: synced -- release-notes.pl: fix parsing typo +- [Cherish98 brought this change] -James Fuller (20 Apr 2020) -- [xquery brought this change] + socks: fix expected length of SOCKS5 reply + + Commit 4a4b63d forgot to set the expected SOCKS5 reply length when the + reply ATYP is X'01'. This resulted in erroneously expecting more bytes + when the request length is greater than the reply length (e.g., when + remotely resolving the hostname). + + Closes #5527 - ensure all references to ports are replaced by vars +Marc Hoersken (5 Jun 2020) +- .gitignore: add directory containing the stats repo + + Since the new curl/stats repository is designed to be + checked out into the curl repository working tree as stats/ + it should be on the ignore list to aid in commit staging. -- [xquery brought this change] +Daniel Stenberg (5 Jun 2020) +- [Adnan Khan brought this change] - add more alt-svc test coverage + HTTP3.md: clarify cargo build directory + + Cargo needs to be called from within the 'quiche' directory. + + Closes #5522 -Daniel Stenberg (20 Apr 2020) -- test1247: use http server to get the port number set +- user-agent.d: spell out what happens given a blank argument - Follow-up to 0f5db7b263f + Closes #5525 -- runtests: use a unix domain socket path with the pid in the name +- trailers: switch h1-trailer logic to use dynbuf - To make it impossible for test cases to access the file name without - using the proper variable for the purpose. + In the continued effort to remove "manual" realloc schemes. - Closes #5264 + Closes #5524 -Daniel Gustafsson (19 Apr 2020) -- [Tom brought this change] +- CURLINFO_ACTIVESOCKET.3: clarify the description + + Reported-by: Jay Satiro + Fixes #5299 + Closes #5520 - src: Remove C99 constructs to ensure C89 compliance +- mailmap: Don J Olmstead + +- configure: only strip first -L from LDFLAGS - This fixes the error: 'for' loop initial declaration used outside C99 - mode by declaring the loop increment variable in the beginning of the - block instead of inside the for loop. + In the logic that works out if a given OpenSSL path works, it stripped + off a possibly leading -L flag using an incorrect sed pattern which + would remove all instances of -L in the string, including if the path + itself contained that two-letter sequence! - Fixes #5254 - Reviewed-by: Daniel Gustafsson - -Daniel Stenberg (19 Apr 2020) -- runtests: dummy init the ports variables to avoid warnings + The same pattern was used and is now updated in multiple places. Now it + only removes -L if it starts the strings. - ... and generate something that can help debug test cases. + Reported-by: Mohamed Osama + Fixes #5519 + Closes #5521 -- [Patrick Monnerat brought this change] - - mime: properly check Content-Type even if it has parameters +Peter Wu (4 Jun 2020) +- quiche: advertise draft 28 support - New test 669 checks this fix is effective. + Fix the verbose message while at it, quiche currently supports draft + 27 and draft 28 simultaneously. - Fixes #5256 - Closes #5258 - Reported-by: thanhchungbtc on github + Closes #5518 -- tests/FILEFORMAT: converted to markdown and extended +Daniel Stenberg (4 Jun 2020) +- KNOWN_BUGS: RTSP authentication breaks without redirect support - Closes #5261 - -- test1245: make it work with dynamic FTP server port - -- test1055: make it work with dynamic FTP port + Closes #4750 -- test1028: make it run on dynamic FTP server port +Jay Satiro (4 Jun 2020) +- projects: Add crypt32.lib to dependencies for all OpenSSL configs + + Windows project configurations that use OpenSSL with USE_WIN32_CRYPTO + need crypt32. + + Follow-up to 148534d which added CURLSSLOPT_NATIVE_CA for 7.71.0. + + The changes that are in this commit were made by script. + + Ref: https://gist.github.com/jay/a1861b50ecce2b32931237180f856e28 + + Closes https://github.com/curl/curl/pull/5516 -- tests: move pingpong server to dynamic listening port +Marc Hoersken (3 Jun 2020) +- CI/macos: fix 'is already installed' errors by using bundle - FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic - ports + Avoid failing CI builds due to nghttp2 being already installed. - Test 842-845 are unfortunately a bit hard to move over to this concept - right now and require "default port" still... + Closes #5513 -- test1056: work with dynamic HTTP ipv6 port +Daniel Stenberg (3 Jun 2020) +- altsvc: fix 'dsthost' may be used uninitialized in this function -- test1448: work with dynamic HTTP server port +- RELEASE-NOTES: synced -- tests: introduce preprocessed test cases +- urldata: let the HTTP method be in the set.* struct - The runtests script now always performs variable replacement on the - entire test source file before the test gets executed, and saves the - updated version in a temporary file (log/test[num]) so that all test - case readers/servers can use that version (if present) and thus enjoy - the powers of test case variable substitution. + When the method is updated inside libcurl we must still not change the + method as set by the user as then repeated transfers with that same + handle might not execute the same operation anymore! - This is necessary to allow complete port number freedom. + This fixes the libcurl part of #5462 - Test 309 is updated to work with a non-fixed port number thanks to this. + Test 1633 added to verify. + + Closes #5499 -- tests: make 2006-2010 handle different port number lengths +- hostip: fix the memory-leak introduced in 67d2802 + + Fixes #5503 + Closes #5504 -- tests: run the sws server on "any port" +- test970: make it require proxy support - Makes the test servers for HTTP and Gopher pop up on a currently unused - port and runtests adapts to that! + This test verifies the -w %json output and the test case includes a full + generated "blob". If there's no proxy support built into libcurl, it + will return an error for proxy related info variables and they will not + be included in the json, thus causing a mismatch and this test fails. - Closes #5247 + Reported-by: Marc Hörsken + Fixes #5501 + Closes #5502 -Marc Hoersken (18 Apr 2020) -- sockfilt: tidy variable naming and data structure in select_ws +- [Radoslav Georgiev brought this change] + + examples/http2-down/upload: add error checks - This commit does not introduce any logical changes to the code. + If `index.html` does not exist in the directory from which the example + is invoked, the fopen(upload, "rb") invocation in `setup` would fail, + returning NULL. This value is subsequently passed as the FILE* argument + of the `fread` invocation in the `read_callback` function, which is the + actual cause of the crash (apparently `fread` assumes that argument to + be non-null). - Reviewed-by: Jay Satiro and Marcel Raad - Closes #5238 + In addition, mitigate some possible crashes of similar origin. + + Closes #5463 -Daniel Stenberg (17 Apr 2020) -- [Anderson Toshiyuki Sasaki brought this change] +- [kotoriのねこ brought this change] - libssh: Use new ECDSA key types to check known hosts - - From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA - keys depending on the curve. + examples/ephiperfifo: turn off interval when setting timerfd - Signed-off-by: Anderson Toshiyuki Sasaki - Fixes #5252 - Closes #5253 + Reported-by: therealhirudo on github + Fixes #5485 + Closes #5497 -Marcel Raad (17 Apr 2020) -- appveyor: add Unicode winbuild jobs +- [Saleem Abdulrasool brought this change] + + vtls: repair the build with `CURL_DISABLE_PROXY` - These are cheap as they don't build tests. + `http_proxy` will not be available in `conndata` if `CURL_DISABLE_PROXY` + is enabled. Repair the build with that configuration. - Closes https://github.com/curl/curl/pull/5063 - -Daniel Stenberg (16 Apr 2020) -- mqttd: s/errno/SOCKERRNO + Follow-up to f3d501dc67 - To behave proper on Windows - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 - Closes #5241 + Closes #5498 -- buildconf: use find -execdir instead, remove -print and the ares files +- transfer: remove k->str NULL check - Follow-up to 1e41bec96a6e + "Null-checking k->str suggests that it may be null, but it has already + been dereferenced on all paths leading to the check" - and it can't + legally be NULL at this point. Remove check. - Suggested-by: Marc Hörsken - -- [Alexander V. Tikhonov brought this change] + Detected by Coverity CID 1463884 + + Closes #5495 - buildconf: avoid using tempfile when removing files +Marc Hoersken (1 Jun 2020) +- select: always use Sleep in Curl_wait_ms on Win32 - Closes #5213 + Since Win32 almost always will also have USE_WINSOCK, + we can reduce complexity and always use Sleep there. + + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Follow up to #5343 + Closes #5489 -- copyright: bump the copyright year range +Daniel Stenberg (31 May 2020) +- conncache: download buffer needs +1 size for trailing zero + + Follow-up to c4e6968127e + Detected by OSS-Fuzz: https://oss-fuzz.com/testcase-detail/5727799779524608 -- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords +Marc Hoersken (31 May 2020) +- azure: use matrix strategy to avoid configuration redundancy + + This also includes the following changes: + + - Use the same timeout for all jobs on Linux (60 minutes) + and Windows (90 minutes) + - Use CLI stable apt-get install -y instead of apt install + which warns about that and run apt-get update first + - Enable MQTT for Windows msys2 builds instead of + legacy msys1 builds + - Add ./configure --prefix parameter to the msys2 builds + - The MSYSTEM environment variable is now preset inside + the container images for the msys2 builds + + Note: on Azure Pipelines the matrix strategy is basically + just a simple list of job copies and not really a matrix. + + Closes #5468 -- [JP Mens brought this change] +Daniel Stenberg (30 May 2020) +- build: disable more code/data when built without proxy support + + Added build to travis to verify + + Closes #5466 - docs/MQTT: replace confusing 80 by 75 +- url: alloc the download buffer at transfer start - I was a bit surprised by the `80`: first thought: what's HTTP doing - here? ;) + ... and free it as soon as the transfer is done. It removes the extra + alloc when a new size is set with setopt() and reduces memory for unused + easy handles. - Closes #5236 + In addition: the closure_handle now doesn't use an allocated buffer at + all but the smallest supported size as a stack based one. + + Closes #5472 -- [Brad King brought this change] +- timeouts: change millisecond timeouts to timediff_t from time_t + + For millisecond timers we like timediff_t better. Also, time_t can be + unsigned so returning a negative value doesn't work then. + + Closes #5479 - cmake: Avoid MSVC C4273 warnings in send/recv checks +Marc Hoersken (30 May 2020) +- select: add overflow checks for timeval conversions - We use `check_c_source_compiles` to check possible send/recv signatures - by reproducing the forward declarations from system headers. On Windows - the `winsock2.h` header adds dll linkage settings to its forward - declaration. If ours does not match the compiler warns: + Using time_t and suseconds_t if suseconds_t is available, + long on Windows (maybe others in the future) and int elsewhere. - warning C4273: 'recv': inconsistent dll linkage + Also handle case of ULONG_MAX being greater or equal to INFINITE. - Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so - that our linkage is consistent with that from `winsock2.h`. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Fixes #4764 - Closes #5232 + Part of #5343 -Jay Satiro (14 Apr 2020) -- KNOWN_BUGS: Add entry 'Blocking socket operations' +- select: use timediff_t instead of time_t and int for timeout_ms - - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of - known blocking operations. + Make all functions in select.[ch] take timeout_ms as timediff_t + which should always be large enough and signed on all platforms + to take all possible timeout values and avoid type conversions. - - New known bugs entry 'Blocking socket operations in non-blocking API' - that directs to the TODO's list of known blocking operations. + Reviewed-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 + Replaces #5107 and partially #5262 + Related to #5240 and #5286 + Closes #5343 + +- unit1604.c: fix implicit conv from 'SANITIZEcode' to 'CURLcode' - Reported-by: Marc Hoersken + GCC 10 warns about this with warning: implicit conversion + from 'SANITIZEcode' to 'CURLcode' [-Wenum-conversion] - Closes https://github.com/curl/curl/pull/5216 + Since 'expected_result' is not really of type 'CURLcode' and + it is not exposed in any way, we can just use 'SANITIZEcode'. + + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + + Closes #5476 -Marc Hoersken (14 Apr 2020) -- test2043: use revoked.badssl.com instead of revoked.grc.com +- tests/libtest: fix undefined reference to 'curlx_win32_fopen' - The certificate of revoked.grc.com has expired on 2020-04-13. + Since curl_setup.h now makes use of curlx_win32_fopen for Win32 + builds with USE_WIN32_LARGE_FILES or USE_WIN32_SMALL_FILES defined, + we need to include the relevant files for tests using fopen, + because the libtest sources are also including curl_setup.h - Reviewed-by: Jay Satiro + Reviewed-by: Marcel Raad + Reviewed-by: Daniel Stenberg - Closes #5233 + Follow up to #3784 (ffdddb45d9) + Closes #5475 -- sockfilt: fix broken pipe on Windows to be ready in select_ws +- appveyor: add non-debug plain autotools-based build - Closes #5228 + This should enable us to catch linking issues with the + testsuite early, like the one described/fixed in #5475. + + Reviewed-by: Daniel Stenberg + Reviewed-by: Marcel Raad + + Closes #5477 -Daniel Stenberg (14 Apr 2020) +Daniel Stenberg (29 May 2020) - RELEASE-NOTES: synced -- scripts/release-notes: fix duplicate output header - -- github/workflow: enable MQTT in the macOS debug build +- Revert "buildconf: use find -execdir" + + This partially reverts commit c712009838f44211958854de431315586995bc61. + + Keep the ares_ files removed but bring back the older way to run find, + to make it work with busybox's find, as apparently that's being used. + + Reported-by: Max Peal + Fixes #5483 + Closes #5484 -- azure: add mqtt support to one of the Windows builds +- server/sws: fix asan warning on use of uninitialized variable -- travis: add mqtt job on Linux +- libssh2: improved error output for wrong quote syntax + + Reported-by: Werner Stolz + + Closes #5474 -- tests: add four MQTT tests 1190 - 1193 +- mk-lib1521: generate code for testing BLOB options as well + + Follow-up to cac5374298b3 + + Closes #5478 -- tests: add the mqtt test server mqttd +- configure: repair the check if argv can be written to + + Due to bad escaping of the test code, the test wouldn't build and thus + result in a negative test result, which would lead to the unconditional + assumption that overwriting the arguments doesn't work and thus curl + would never hide credentials given in the command line, even when it + would otherwise be possible. + + Regression from commit 2d4c2152c (7.60.0) + + Reported-by: huzunhao on github + Fixes #5470 + Closes #5471 -- tests: support hex encoded data and mqtt server +Peter Wu (28 May 2020) +- CMake: rebuild Makefile.inc.cmake when Makefile.inc changes - The mqtt server is started using a "random" port. + Otherwise the build might fail due to missing source files, as + demonstrated by the recent keylog.c addition on an existing build dir. + + Closes #5469 -- [Björn Stenberg brought this change] +Daniel Stenberg (28 May 2020) +- urldata: fix comments: Curl_done() is called multi_done() now + + ... since 575e885db - mqtt: add new experimental protocol +Peter Wu (27 May 2020) +- ngtcp2: use common key log routine for better thread-safety - Closes #5173 + Tested with ngtcp2 built against the OpenSSL library. Additionally + tested with MultiSSL (NSS for TLS and ngtcp2+OpenSSL for QUIC). + + The TLS backend (independent of QUIC) may or may not already have opened + the keylog file before. Therefore Curl_tls_keylog_open is always called + to ensure the file is open. -- TODO: Consider convenience options for JSON and XML? +- wolfssl: add SSLKEYLOGFILE support - Closes #5203 + Tested following the same curl and tshark commands as in commit + "vtls: Extract and simplify key log file handling from OpenSSL" using + WolfSSL v4.4.0-stable-128-g5179503e8 from git master built with + `./configure --enable-all --enable-debug CFLAGS=-DHAVE_SECRET_CALLBACK`. + + Full support for this feature requires certain wolfSSL build options, + see "Availability note" in lib/vtls/wolfssl.c for details. + + Closes #5327 -- tool: do not declare functions with Curl_ prefix +- vtls: Extract and simplify key log file handling from OpenSSL - To avoid collision risks with private libcurl symbols when linked with - static versions (or just versions not hiding internal symbols). + Create a set of routines for TLS key log file handling to enable reuse + with other TLS backends. Simplify the OpenSSL backend as follows: - Reported-by: hydra3333 on github - Fixes #5219 - Closes #5234 + - Drop the ENABLE_SSLKEYLOGFILE macro as it is unconditionally enabled. + - Do not perform dynamic memory allocation when preparing a log entry. + Unless the TLS specifications change we can suffice with a reasonable + fixed-size buffer. + - Simplify state tracking when SSL_CTX_set_keylog_callback is + unavailable. My original sslkeylog.c code included this tracking in + order to handle multiple calls to SSL_connect and detect new keys + after renegotiation (via SSL_read/SSL_write). For curl however we can + be sure that a single master secret eventually becomes available + after SSL_connect, so a simple flag is sufficient. An alternative to + the flag is examining SSL_state(), but this seems more complex and is + not pursued. Capturing keys after server renegotiation was already + unsupported in curl and remains unsupported. + + Tested with curl built against OpenSSL 0.9.8zh, 1.0.2u, and 1.1.1f + (`SSLKEYLOGFILE=keys.txt curl -vkso /dev/null https://localhost:4433`) + against an OpenSSL 1.1.1f server configured with: + + # Force non-TLSv1.3, use TLSv1.0 since 0.9.8 fails with 1.1 or 1.2 + openssl s_server -www -tls1 + # Likewise, but fail the server handshake. + openssl s_server -www -tls1 -Verify 2 + # TLS 1.3 test. No need to test the failing server handshake. + openssl s_server -www -tls1_3 + + Verify that all secrets (1 for TLS 1.0, 4 for TLS 1.3) are correctly + written using Wireshark. For the first and third case, expect four + matches per connection (decrypted Server Finished, Client Finished, HTTP + Request, HTTP Response). For the second case where the handshake fails, + expect a decrypted Server Finished only. + + tshark -i lo -pf tcp -otls.keylog_file:keys.txt -Tfields \ + -eframe.number -eframe.time -etcp.stream -e_ws.col.Info \ + -dtls.port==4433,http -ohttp.desegment_body:FALSE \ + -Y 'tls.handshake.verify_data or http' + + A single connection can easily be identified via the `tcp.stream` field. -- [Nathaniel R. Lewis brought this change] +Daniel Stenberg (27 May 2020) +- FILEFORMAT: add more features that tests can depend on - cmake: add aliases so exported target names are available in tree - - Reviewed-by: Brad King - Closes #5206 +- [Michael Kaufmann brought this change] -- version: increase buffer space for ssl version output + transfer: close connection after excess data has been read - To avoid it getting truncated, especially when several SSL backends are - built-in. + For HTTP 1.x, it's a protocol error when the server sends more bytes + than announced. If this happens, don't reuse the connection, because the + start position of the next response is undefined. - Reported-by: Gisle Vanem - Fixes #5222 - Closes #5226 + Closes #5440 -Marc Hoersken (13 Apr 2020) -- cirrus: no longer ignore test 504 which is working again - - The test is working again, because TCP blackholing is disabled. +- [Estanislau Augé-Pujadas brought this change] -- appveyor: completely disable tests that fail to timeout early + Revert "ssh: ignore timeouts during disconnect" - The tests changed from ignored to disabled are tests that are - about connecting to non-listening socket. On AppVeyor these - tests are not reliable, because for some unknown reason the - connect is not timing out before the test time limit is reached. - -Daniel Stenberg (13 Apr 2020) -- test1908: avoid using fixed port number in test data + This reverts commit f31760e63b4e9ef1eb25f8f211390f8239388515. Shipped in + curl 7.54.1. - Closes #5225 - -Jay Satiro (12 Apr 2020) -- [Andrew Kurushin brought this change] + Bug: https://curl.haxx.se/mail/lib-2020-05/0068.html + Closes #5465 - schannel: Fix blocking timeout logic +- urldata: connect related booleans live in struct ConnectBits - - Fix schannel_send for the case when no timeout was set. + And remove a few unused booleans! - Prior to this change schannel would error if the socket was not ready - to send data and no timeout was set. + Closes #5461 + +- hostip: on macOS avoid DoH when given a numerical IP address - This commit is similar to parent commit 89dc6e0 which recently made the - same change for SOCKS, for the same reason. Basically it was not well - understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms - but actually means no timeout. + When USE_RESOLVE_ON_IPS is set (defined on macOS), it means that + numerical IP addresses still need to get "resolved" - but not with DoH. - Fixes https://github.com/curl/curl/issues/5177 - Closes https://github.com/curl/curl/pull/5221 + Reported-by: Viktor Szakats + Fixes #5454 + Closes #5459 -- socks: Fix blocking timeout logic +- ngtcp2: cleanup memory when failing to connect - - Document in Curl_timeleft's comment block that returning 0 signals no - timeout (ie there's infinite time left). + Reported-by: Peter Wu + Fixes #5447 (the ngtcp2 side of it) + Closes #5451 + +- quiche: clean up memory properly when failing to connect - - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. + Addresses the quiche side of #5447 + Reported-by: Peter Wu + Closes #5450 + +- cleanup: use a single space after equals sign in assignments + +- url: accept "any length" credentials for proxy auth - Prior to this change if the timeout had a value of 0 and that was passed - to SOCKET_READABLE it would return right away instead of blocking. That - was likely because it was not well understood that when Curl_timeleft - returns 0 it is not a timeout of 0 ms but actually means no timeout. + They're only limited to the maximum string input restrictions, not to + 256 bytes. - Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + Added test 1178 to verify - Closes https://github.com/curl/curl/pull/5220 + Reported-by: Will Roberts + Fixes #5448 + Closes #5449 -- [Marc Hoersken brought this change] +- [Maksim Stsepanenka brought this change] - gopher: check remaining time left during write busy loop + test1167: fixes in badsymbols.pl - Prior to this change gopher's blocking code would block forever, - ignoring any set timeout value. + Closes #5442 + +- altsvc: fix parser for lines ending with CRLF + + Fixed the alt-svc parser to treat a newline as end of line. + The unit tests in test 1654 were done without CRLF and thus didn't quite + match the real world. Now they use CRLF as well. + + Reported-by: Peter Wu + Assisted-by: Peter Wu Assisted-by: Jay Satiro + Fixes #5445 + Closes #5446 + +Viktor Szakats (25 May 2020) +- all: fix codespell errors + + Reviewed-by: Jay Satiro Reviewed-by: Daniel Stenberg + Closes https://github.com/curl/curl/pull/5452 + +Peter Wu (25 May 2020) +- ngtcp2: fix build with current ngtcp2 master implementing draft 28 - Similar to #5220 and #5221 - Closes #5214 + Based on client.cc changes from ngtcp2. Tested with current git master, + ngtcp2 commit c77d5731ce92, nghttp3 commit 65ff479d4380. + + Fixes #5444 + Closes #5443 -Daniel Stenberg (13 Apr 2020) -- [Dirkjan Bussink brought this change] - - gnutls: ensure TLS 1.3 when SRP isn't requested - - When SRP is requested in the priority string, GnuTLS will disable - support for TLS 1.3. Before this change, curl would always add +SRP to - the priority list, effectively always disabling TLS 1.3 support. - - With this change, +SRP is only added to the priority list when SRP - authentication is also requested. This also allows updating the error - handling here to not have to retry without SRP. This is because SRP is - only added when requested and in that case a retry is not needed. +Daniel Stenberg (25 May 2020) +- RELEASE-NOTES: synced - Closes #5223 + moved the new setopts up to a "change" -Marc Hoersken (12 Apr 2020) -- tests/server: add hidden window to gracefully handle WM_CLOSE - - Forward Window events as signals to existing signal event handler. +- RELEASE-NOTES: synced -- tests/server: add CTRL event handler for Win32 consoles +- copyright: updated year ranges out of sync - Forward CTRL events as signals to existing signal event handler. + ... and whitelisted a few more files in the the copyright.pl script. -- tests/server: move all signal handling routines to util.[ch] +- [Gilles Vollant brought this change] + + setopt: add CURLOPT_PROXY_ISSUERCERT(_BLOB) for coherency - Avoid code duplication to prepare for portability enhancements. + Closes #5431 -Daniel Stenberg (12 Apr 2020) -- compressed.d: stress that the headers are not modified +- curl: remove -J "informational" written on stdout - Suggested-by: Michael Osipov - Assisted-by: Jay Satiro - Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 - Closes #5217 + curl would previously show "curl: Saved to filename 'name from header'" + if -J was used and a name was picked from the Content-Disposition + header. That output could interfer with other stdout output, such as -w. + + This commit removes that output line. + Bug: https://curl.haxx.se/mail/archive-2020-05/0044.html + Reported-by: Коваленко Анатолий Викторович + Closes #5435 -Marc Hoersken (11 Apr 2020) -- tests/server/util.c: use curl_off_t instead of long for pid +Peter Wu (22 May 2020) +- travis: simplify quiche build instructions wrt boringssl - Avoid potential overflow of huge PIDs on Windows. + quiche builds boringssl as static library, reuse that instead of + building another shared library. - Related to #5188 - Assisted-by: Marcel Raad + Closes #5438 -- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows +- configure: fix pthread check with static boringssl - Since the Windows versions of both programs would write Windows - PIDs to their pidfiles which we cannot handle, we need to use - our known perl.exe Cygwin/msys PID together with exec() in order - to tie the spawned processes to the existance of our perl.exe + A shared boringssl/OpenSSL library requires -lcrypto only for linking. + A static build additionally requires `-ldl -lpthread`. In the latter + case `-lpthread` is added to LIBS which prevented `-pthread` from being + added to CFLAGS. Clear LIBS to fix linking failures for libtest tests. + +Daniel Stenberg (22 May 2020) +- Revert "sendf: make failf() use the mvsnprintf() return code" - The perl.exe that is executing secureserver.pl and sshserver.pl - has a Cygwin/msys PID, because it is started inside Cygwin/msys. + This reverts commit 74623551f306990e70c7c5515b88972005604a74. - Related to #5188 + Instead mark the function call with (void). Getting the return code and + using it instead triggered Coverity warning CID 1463596 because + snprintf() can return a negative value... + + Closes #5441 -- tests: add Windows compatible pidwait like pidkill and pidterm +- typecheck-gcc.h: CURLINFO_PRIVATE does not need a 'char *' - Related to #5188 + Reported-by: Billyzou0741326 on github + Fixes #5432 + Closes #5436 -- tests: fix conflict between Cygwin/msys and Windows PIDs +- tests/server/util.h: add extern to silence compiler warning - Add 65536 to Windows PIDs to allow Windows specific treatment - by having disjunct ranges for Cygwin/msys and Windows PIDs. + Follow-up from a3b0699d5c1 + +- typecheck-gcc.h: fix the OFF_T check - See also: - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=b5e1003722cb14235c4f166be72c09acdffc62ea - - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ - h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe + The option number also needs to be less than CURLOPTTYPE_BLOB. - Replaces #5178 - Closes #5188 + Follow-up to cac5374298 + Reported-by: Jeroen Ooms + Bug: https://github.com/curl/curl/pull/5365#issuecomment-631084114 -Daniel Stenberg (11 Apr 2020) -- RELEASE-NOTES: synced +- TODO: --dry-run + + Closes #5426 -- release-notes.pl: detect the start of the references in cleanup mode +- TODO: Ratelimit or wait between serial requests + + Closes #5406 -- Revert "file: on Windows, refuse paths that start with \\" +- tool_paramhlp: fixup C89 mistake - This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. + Follow-up to c5f0a9db22. + +- [Siva Sivaraman brought this change] + + tool_paramhlp: fixed potentially uninitialized strtol() variable - Reminded-by: Chris Roberts - Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html + Seems highly unlikely to actually be possible, but better safe than + sorry. - Closes #5215 + Closes #5417 -Jay Satiro (11 Apr 2020) -- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE +- [Siva Sivaraman brought this change] + + tool_operate: fixed potentially uninitialized variables - - If loss of data may occur converting a timediff_t to time_t and - the time value is > TIME_T_MAX then treat it as TIME_T_MAX. + ... in curl_easy_getinfo() calls. They're harmless but clearing the + variables makes the code safer and comforts the reader. - This is a follow-up to 8843678 which removed the (time_t) typecast - from the macros so that conversion warnings could be identified. + Closes #5416 + +- sha256: move assign to the declaration line - Closes https://github.com/curl/curl/pull/5199 + Follow-up to fae30656. Should've been squashed with that commit... -- test1148: tolerate progress updates better (again) +- [Siva Sivaraman brought this change] + + sha256: fixed potentially uninitialized variable - - Ignore intermediate progress updates. + Closes #5414 + +- sendf: make failf() use the mvsnprintf() return code - - Support locales that use a character other than period as decimal - separator (eg 100,0%). + ... and avoid a strlen() call. Fixes a MonocleAI warning. - test1148 checks that the progress finishes at 100% and has the right - bar width. Prior to this change the test assumed that the only progress - reported for such a quick transfer was 100%, however in rare instances - (like in the CI where transfer time can slow considerably) there may be - intermediate updates. For example, below is stderrlog1148 from a failed - CI run with explicit \r and \n added (it is one line; broken up so that - it's easier to understand). + Reported-by: MonocleAI + Fixes #5413 + Closes #5420 + +- hostip: make Curl_printable_address not return anything - \r - \r################################## 48.3% - \r######################################################################## 100.0% - \n + It was not used much anyway and instead we let it store a blank buffer + in case of failure. - Closes https://github.com/curl/curl/pull/5194 - -Marc Hoersken (10 Apr 2020) -- sshserver.pl: use cached Win32 environment check variable + Reported-by: MonocleAI + Fixes #5411 + Closes #5418 -- appveyor: partially revert 3413a110 to keep build without proxy +- ftp: mark return-ignoring calls to Curl_GetFTPResponse with (void) - Ref: #5211 and #4526 - Reported-by: Marcel Raad + They're done on purpose, make that visible in the code. + Reported-by: MonocleAI + Fixes #5412 + Closes #549 -- appveyor: ignore failing 'connect to non-listening proxy' tests +- TODO: forbid TLS post-handshake auth and do TLS record padding - Closes #5211 + Closes #5396 + Closes #5398 -- CI/macos: convert CRLF to LF and align indentation +- RELEASE-NOTES: synced -Daniel Stenberg (9 Apr 2020) -- url: allow non-HTTPS altsvc-matching for debug builds +- dynbuf: return NULL when there's no buffer length - This is already partly supported but this part was missing. - Reported-by: James Fuller + ... as returning a "" is not a good idea as the string is supposed to be + allocated and returning a const string will cause issues. - Closes #5205 + Reported-by: Brian Carpenter + Follow-up to ed35d6590e72c + Closes #5405 -- server/resolve: remove AI_CANONNAME to make macos tell the truth +Peter Wu (16 May 2020) +- travis: upgrade to bionic, clang-9, improve readability - With this bit set, my mac successfully resolves "ip6-localhost" when in - fact there is no such host known to my machine! That in turn made test - 241 wrongly execute and fail. + Changes, partially to reduce build failures from external dependencies: + - Upgrade Ubuntu and drop unnecessary third-party repos. + - Properly clone apt config to ensure retries. + - Upgrade to clang-9 from the standard repos. + - Use Ubuntu 20.04 focal for the libssh build, use of ssh_get_publickey + fails on -Werror=deprecated-declarations in Ubuntu 18.04. Do not use + focal everywhere yet since Travis CI has not documented this option. + In focal, python-impacket (Py2.7) has been removed, leaving only + python3-impacket. Since it is only needed for SMB tests and not SSH, + skip it for the libssh job since it might need more work. + - apt: Remove gcc-8 and libstdc++-8-dev, already installed via g++-8. - Closes #5202 + Non-functional cleanups: + - Simplify test matrix, drop redundant os and compiler keys. + - Deprecation fixes: remove sudo, rename matrix -> jobs. + - Every job has an 'env' key, put this key first in a list item. + + Closes #5370 -- runtests: fix warning about using an undefined variable +- travis: whitespace-only changes for consistency - Follow-up from 4d939ef6ceb2db1 + Automatically apply a consistent indentation with: + + python3 -c 'from ruamel.yaml import YAML;y=YAML();d=y.load(open(".travis.yml"));y.width=500;y.dump(d,open(".travis.yml.new","w"))' + + followed by manually re-indenting three comments. + + Closes #5370 -- release-notes: fix the initial reference list output +- CMake: add libssh build support + + Closes #5372 -- github actions: run when pushed to master or */ci + PRs +Daniel Stenberg (15 May 2020) +- KNOWN_BUGS: wolfssh: publickey auth doesn't work - Avoid double-builds when using "local" branches for PRs. For both macos - and fuzz jobs. + Closes #4820 + +- KNOWN_BUGS: OS400 port requires deprecated IBM library - Closes #5201 + Closes #5176 -- runtests: provide nicer errormsg when protocol "dump" file is empty +- [Vyron Tsingaras brought this change] + + http2: keep trying to send pending frames after req.upload_done + + Fixes #1410 + Closes #5401 - [Gilles Vollant brought this change] - schannel: support .P12 or .PFX client certificates + setopt: support certificate options in memory with struct curl_blob - Used with curl command line option like this: --cert - : --cert-type p12 + This change introduces a generic way to provide binary data in setopt + options, called BLOBs. - Closes #5193 + This change introduces these new setopts: + + CURLOPT_ISSUERCERT_BLOB, CURLOPT_PROXY_SSLCERT_BLOB, + CURLOPT_PROXY_SSLKEY_BLOB, CURLOPT_SSLCERT_BLOB and CURLOPT_SSLKEY_BLOB. + + Reviewed-by: Daniel Stenberg + Closes #5357 -- tests: verify split initial HTTP requests with CURL_SMALLREQSEND +- source cleanup: remove all custom typedef structs - test1294: "split request" being when the entire request isn't sent in - the first go, and the remainder is sent in the PERFORM state. A GET - request is otherwise not sending anything during PERFORM. + - Stick to a single unified way to use structs + - Make checksrc complain on 'typedef struct {' + - Allow them in tests, public headers and examples - test1295: same kind of split but with POST + - Let MD4_CTX, MD5_CTX, and SHA256_CTX typedefs remain as they actually + typedef different types/structs depending on build conditions. - Closes #5197 + Closes #5338 -- http: don't consider upload done if the request isn't completely sent off - - Fixes #4919 - Closes #5197 +- travis: remove the .checksrc fiddling -- http: allow Curl_add_buffer_send() to do a short first send by force +- ftp: make domore_getsock() return the secondary socket properly - In a debug build, settting the environment variable "CURL_SMALLREQSEND" - will make the first HTTP request send not send more bytes than the set - amount, thus ending up verifying that the logic for handling a split - HTTP request send works correctly. + Previously, after PASV and immediately after the data connection has + connected, the function would only return the control socket to wait for + which then made the data connection simply timeout and not get polled + correctly. This become obvious when running test 1631 and 1632 event- + based. -- connect: store connection info for QUIC connections +- test1632: verify FTP through HTTPS-proxy with connection re-use + +- test1631: verify FTP download through HTTPS-proxy + +- sws: as last resort, get test number from server cmd file - Restores the --head functionality to the curl utility which extracts - 'protocol' that is stored that way. + If it can't be found in the request. Also support --cmdfile to set it to + a custom file name. - Reported-by: James Fuller - Fixes #5196 - Closes #5198 + runtests.pl always writes this file with the test number in it since a + while back. -- tests/README: update the port numbers list +- ftp: shut down the secondary connection properly when SSL is used - Since the pipelining server is long gone. - Reported-by: James Fuller + Reported-by: Neal Poole + Fixes #5340 + Closes #5385 -- select: remove typecast from SOCKET_WRITABLE/READABLE macros +Marcel Raad (14 May 2020) +- KNOWN_BUGS: adapt 5.5 to recent changes - So that they don't hide conversions-by-mistake + It only applies to non-Unicode builds now. + Also merge 5.10 into it as it's effectively a duplicate. - Reviewed-by: Jay Satiro - Closes #5190 + Closes https://github.com/curl/curl/pull/3784 -- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also +- curl_setup: support Unicode functions to open files on Windows - Closes #5192 - -- release-notes: output trailing references sorted numerically + Use them only if `_UNICODE` is defined, in which case command-line + arguments have been converted to UTF-8. + + Closes https://github.com/curl/curl/pull/3784 -- cleanup: correct copyright year range on a few files +- tool: support UTF-16 command line on Windows + + - use `wmain` instead of `main` when `_UNICODE` is defined [0] + - define `argv_item_t` as `wchar_t *` in this case + - use the curl_multibyte gear to convert the command-line arguments to + UTF-8 + + This makes it possible to pass parameters with characters outside of + the current locale on Windows, which is required for some tests, e.g. + the IDN tests. Out of the box, this currently only works with the + Visual Studio project files, which default to Unicode, and winbuild + with the `ENABLE_UNICODE` option. + + [0] https://devblogs.microsoft.com/oldnewthing/?p=40643 + + Ref: https://github.com/curl/curl/issues/3747 + Closes https://github.com/curl/curl/pull/3784 -- configure: remove use of -vec-report0 from CFLAGS with icc +- curl_multibyte: add to curlx - ... as it apparently isn't (always) supported. - Reported-by: Alain Miniussi - Fixes #5096 - Closes #5191 + This will also be needed in the tool and tests. + + Ref: https://github.com/curl/curl/pull/3758#issuecomment-482197512 + Closes https://github.com/curl/curl/pull/3784 -- warnless: remove code block for icc that didn't work +Daniel Stenberg (14 May 2020) +- url: make the updated credentials URL-encoded in the URL - Reported-by: Alain Miniussi - Fixes #5096 + Found-by: Gregory Jefferis + Reported-by: Jeroen Ooms + Added test 1168 to verify. Bug spotted when doing a redirect. + Bug: https://github.com/jeroen/curl/issues/224 + Closes #5400 -Marc Hoersken (6 Apr 2020) -- dist: add missing setup-win32.h +- tests: add https-proxy support to the test suite - Follow up to d820224b8b + Initial test 1630 added with basic HTTPS-proxy use. HTTPS-proxy is like + HTTP proxy but with a full TLS connection to the proxy. + + Closes #5399 -Daniel Stenberg (6 Apr 2020) -- RELEASE-NOTES: synced +- mailmap: James Fuller -- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance +- [Major_Tom brought this change] + + vauth/cleartext: fix theoretical integer overflow - This script helps putting entries in the RELEASE-NOTES using a coherent - style and sorting with a minimal human editing effort - as long as the - first line in the commit message is good enough! There's a short howto - at the top of the file. + Fix theoretical integer overflow in Curl_auth_create_plain_message. + + The security impact of the overflow was discussed on hackerone. We + agreed this is more of a theoretical vulnerability, as the integer + overflow would only be triggerable on systems using 32-bits size_t with + over 4GB of available memory space for the process. + + Closes #5391 -- [Dennis Felsing brought this change] +Jay Satiro (13 May 2020) +- curl.1: Quote globbed URLs + + - Quote the globbing example URLs that contain characters [] {} since + otherwise they may be interpreted as shell metacharacters. + + Bug: https://github.com/curl/curl/issues/5388 + Reported-by: John Simpson + + Closes https://github.com/curl/curl/pull/5394 - configure: don't check for Security.framework when cross-compiling +Daniel Stenberg (14 May 2020) +- checksrc: enhance the ASTERISKSPACE and update code accordingly - Since it checks for the local file, not the cross-compiled one. + Fine: "struct hello *world" - Closes #5189 + Not fine: "struct hello* world" (and variations) + + Closes #5386 -- TODO: Option to make -Z merge lined based outputs on stdout +- docs/options-in-versions: which version added each cmdline option - Closes #5175 + Added test 971 to verify that the list is in sync with the files in + cmdline-opts. The check also verifies that .d-files that uses Added: + specify the same version number as the options-in-versions file does. + + Closes #5381 -- lib: never define CURL_CA_BUNDLE with a getenv +- docs: unify protocol lists - - it breaks the build (since 6de756c9b1de34b7a1) - - it's not documented and not consistent across platforms - - the curl tool does that getenv magic + We boast support for 25 transfer protocols. Make sure the lists are + consistent - Bug: https://github.com/curl/curl/commit/6de756c#r38127030 - Reported-by: Gisle Vanem + Closes #5384 + +- OpenSSL: have CURLOPT_CRLFILE imply CURLSSLOPT_NO_PARTIALCHAIN - Closes #5187 + ... to avoid an OpenSSL bug that otherwise makes the CRL check to fail. + + Reported-by: Michael Kaufmann + Fixes #5374 + Closes #5376 -Marc Hoersken (5 Apr 2020) -- lib670: use the same Win32 API check as all other lib tests +- tls13-ciphers.d: shorten the Arg -- appveyor: use random test server ports based upon APPVEYOR_API_URL +- sasl-authzid.d: add Arg: and shorten the desc + +- cert-type.d: mention the available types in the desc + +- tool: shorten 3 --help descriptions - Avoid conflicts of test server ports with AppVeyor API on localhost. + --happy-eyeballs-timeout-ms, --resolve and --ssl-revoke-best-effort - Closes #5034 - -- appveyor: sort builds by type and add two new variants + gen.pl already warned about these lines but we didn't listen - Related to #5034 and #5063 + Closes #5379 -- appveyor: show failed tests in log even if test is ignored +- configure: the wolfssh backend does not provide SCP - And print API response with newline only if there is one + Closes #5387 -- appveyor: turn disabled tests into ignored result tests +- RELEASE-NOTES: synced -Daniel Stenberg (5 Apr 2020) -- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" +- url: reject too long input when parsing credentials - Fixed with #5170 (commit 23a870f2fd041278) + Since input passed to libcurl with CURLOPT_USERPWD and + CURLOPT_PROXYUSERPWD circumvents the regular string length check we have + in Curl_setstropt(), the input length limit is enforced in + Curl_parse_login_details too, separately. + + Reported-by: Thomas Bouzerar + Closes #5383 -- test1566: verify --etag-compare that gets a 304 back +- list-only.d: this option existed already in 4.0 + +Jay Satiro (12 May 2020) +- retry-all-errors.d: Shorten the summary line - Verifies the fix in #5183 + Follow-up to b995bb5 from a few moments ago. - Closes #5186 + Reported-by: Daniel Stenberg + + Ref: https://github.com/curl/curl/commit/b995bb5#r39108929 -- [Kwon-Young Choi brought this change] +- [denzor brought this change] - CURLINFO_CONDITION_UNMET: return true for 304 http status code + easy: fix dangling pointer on easy_perform fail - In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the - output file if the server did not transfered a file based on time - condition. In the same manner, getting a 304 HTTP response back from the - server, for example after passing a custom If-Match-* header, also - fulfill this condition. + Closes https://github.com/curl/curl/pull/5363 + +- tool: Add option --retry-all-errors to retry on any error - Fixes #5181 - Closes #5183 + The "sledgehammer" of retrying. + + Closes https://github.com/curl/curl/pull/5185 -- [Kwon-Young Choi brought this change] +Daniel Stenberg (12 May 2020) +- [James Le Cuirot brought this change] - curl: allow both --etag-compare and --etag-save with same file name + libcurl.pc: Merge Libs.private into Libs for static-only builds - This change inverse the order of processing for the --etag-compare and - --etag-save option to process first --etag-compare. This in turn allows - to use the same file name to compare and save an etag. + A project being built entirely statically will call pkg-config with + --static, which utilises the Libs.private field. Conversely it will + not use --static when not being built entirely statically, even if + there is only a static build of libcurl available. This will most + likely cause the build to fail due to underlinking unless we merge the + Libs fields. - The original behavior of not failing if the etag file does not exists is - conserved. + Consider that this is what the Meson build system does when it + generates pkg-config files. - Fixes #5179 - Closes #5180 + I have also reflected this in the --libs argument of curl-config even + though REQUIRE_LIB_DEPS always seems to be "yes" anyway. + + Closes #5373 -Viktor Szakats (4 Apr 2020) -- windows: enable UnixSockets with all build toolchains +- [Peter Wu brought this change] + + CMake: fix runtests.pl with CMake, add new test targets + + * runtests.pl: + - Fix out-of-tree build under CMake when srcdir is not set. Default + srcdir to the location of runtests.pl. + - Add a hack to allow CMake to use the TFLAGS option as documented + in tests/README and used in scripts/travis/script.sh. + * Bump CMake version to 3.2 for USES_TERMINAL, dropping Debian Jessie + support (no one should care, it is already EOL.). + * Remove CTest since it defines its own 'test' target with no tests + since all unittests are already broken and not built by default. + * Add new test targets based on the options from Makefile.am. Since + new test targets are rarely added, I opted for duplicating the + runtests.pl options as opposed to creating a new Makefile.inc file. + Use top-level target names (test-x) instead of x-test since that is + used by CI and others. + + Closes #5358 + +- [Peter Wu brought this change] + + CMake: do not build test programs by default - Extend existing unix socket support in Windows builds to be - enabled for all toolchain vendors or versions. (Previously - it was only supported with certain MSVC versions + more recent - Windows 10 SDKs) + The default target should only build libcurl and curl. Add a dedicated + 'testdeps' target which will be used later when running tests. Note that + unittests are currently broken in CMake and already excluded. - Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ - Ref: https://github.com/curl/curl/issues/5162 - Closes: https://github.com/curl/curl/pull/5170 + Closes #5368 -Daniel Stenberg (4 Apr 2020) -- KNOWN_BUGS: Store TLS context per transfer instead of per connection - - Closes #5102 +- FILEFORMAT: moved up the variables section and further polished -Marc Hoersken (3 Apr 2020) -- sockfilt: remove redundancy in timeout handling +- runtests: remove ftp2 support, not used - And update other logmsg output in select_ws on Windows. + We once supported two separate ftp instances in the test suite. Has not + been used the last decade. + + Closes #5375 -- sockfilt: fix handling of ready closed sockets on Windows +- url: sort the protocol schemes in rough popularity order - Replace the incomplete workaround regarding FD_CLOSE - only signalling once by instead doing a pre-check with - standard select and storing the result for later use. + When looking for a protocol match among supported schemes, check the + most "popular" schemes first. It has zero functionality difference and + for all practical purposes a speed difference will not be measureable + but it still think it makes sense to put the least likely matches last. - select keeps triggering on closed sockets on Windows while - WSAEventSelect fires only once with data still available. - By doing the pre-check we do not run in a deadlock - due to waiting forever for another FD_CLOSE event. + "Popularity" based on the 2019 user survey. + + Closes #5377 -- sockfilt: fix race-condition of waiting threads and event handling +Marc Hoersken (11 May 2020) +- test1238: avoid tftpd being busy for tests shortly following - Fix race-condition of waiting threads finishing while events are - already being processed which lead to invalid or skipped events. + The tftpd server may still be busy if the total timeout of + 25 seconds has not been reached or no sread error was received + during or after the execution of the timeout test 1238. - Use mutex to check for one event at a time or do post-processing. - In addition to mutex-based locking use specific event as signal. + Once the next TFTP test comes around (eg. 1242 or 1243), + those will fail because the tftpd server is still waiting + on data from curl due to the UDP protocol being stateless + and having no connection close. On Linux this error may not + happen, because ICMP errors generated due to a swrite error + can also be returned async on the next sread call instead. - Closes #5156 + Therefore we will now just kill the tftpd server after test + 1238 to make sure that the following tests are not affected. + + This enables us to no longer ignore tests 1242, 1243, 2002 + and 2003 on the CI platforms CirrusCI and AppVeyor. + + Assisted-by: Peter Wu + Closes #5364 -Daniel Stenberg (2 Apr 2020) -- [Leo Neat brought this change] +Daniel Stenberg (11 May 2020) +- write-out.d: added "response_code" - CI-fuzz: increase fuzz time to 40 minutes +- KNOWN_BUGS: Build with staticly built dependency - Closes #5174 + I rewrote the item 5.4 to be more generic about static dependencies. -Marc Hoersken (2 Apr 2020) -- CI: increase Azure Pipelines timeouts due to performance issues +- ROADMAP: remove old entries - The current demand on Azure negatively impacts the CI performance. - -- runtests.pl: log host OS as detected by Perl environment + MQTT - the start has already landed + + tiny-curl - also mostly landed and is a continuous work + + make menuconfig - basically no interest from users, not pushing there -- ftpserver.pl: log before and after data connection is closed +- [Peter Wu brought this change] -Daniel Stenberg (1 Apr 2020) -- RELEASE-NOTES: synced + travis: Add ngtcp2 and quiche tests for CMake + + To avoid an explosion of jobs, extend the existing CMake tests with + ngtcp2 and quiche support. macOS was previously moved to GitHub actions, + so the non-Linux case can be dropped. -- RELEASE-PROCEDURE.md: run the copyright.pl script! +- [Peter Wu brought this change] -- vquic/ngtcp2.h: update copyright year range + CMake: add ENABLE_ALT_SVC option - Follow-up to 0736ee73d346a52 - -- [Daiki Ueno brought this change] + Tested alt-svc with quiche. While at it, add missing MultiSSL reporting + (not tested). - CI: add build with ngtcp2 + gnutls on Travis CI +- [Peter Wu brought this change] -- [Daiki Ueno brought this change] + CMake: add HTTP/3 support (ngtcp2+nghttp3, quiche) + + Add three new CMake Find modules (using the curl license, but I grant + others the right to apply the CMake BSD license instead). + + This CMake config is simpler than the autotools one because it assumes + ngtcp2 and nghttp3 to be used together. Another difference is that this + CMake config checks whether QUIC is actually supported by the TLS + library (patched OpenSSL or boringssl) since this can be a common + configuration mistake that could result in build errors later. + + Unlike autotools, CMake does not warn you that the features are + experimental. The user is supposed to already know that and read the + documentation. It requires a very special build environment anyway. + + Tested with ngtcp2+OpenSSL+nghttp3 and quiche+boringssl, both built from + current git master. Use `LD_DEBUG=files src/curl |& grep need` to figure + out which features (libldap-2.4, libssh2) to disable due to conflicts + with boringssl. + + Closes #5359 - vquic: add support for GnuTLS backend of ngtcp2 +Marc Hoersken (10 May 2020) +- tests/server/tftpd.c: fix include and enhance debug logging - Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile - time. Therefore OpenSSL support needs to be explicitly disabled. + setjmp.h should only be included if HAVE_SETJMP_H is defined. - Signed-off-by: Daiki Ueno - Closes #5148 + Add additional log statements to see wether reads and writes + are blocking or finishing before an alarm signal is received. + + Assisted-by: Peter Wu + Part of #5364 -- [Gisle Vanem brought this change] +Daniel Stenberg (10 May 2020) +- tool_operate: only set CURLOPT_SSL_OPTIONS if SSL support is present + + Reported-by: Marcel Raad + Follow-up to 148534db5 + Fixes #5367 + Closes #5369 - examples/sessioninfo.c: add include to fix compiler warning +Marc Hoersken (9 May 2020) +- appveyor: update comments to be clear about toolchain - Fixes #5171 + - CMake-based MSYS builds use mingw-w64 to cross-compile. + - autotools-based builds are compiled using msys2-devel. + + The difference is that the later ones are not cross-compiled + to Windows and instead require the msys2 runtime to be present. + + At the moment only the Azure Pipelines CI builds actually + run autotools-based cross-compilation builds for Windows. -- misc: copyright year updates +- TODO: update regarding missing Schannel features - Follow-up to 7a71965e9 + Some aspects have already been implemented over the years. + + 15.1 Client certificates are now supported: + + - System stores via e35b0256eb34f1fe562e3e2a2615beb50a391c52 + - PKCS#12 files via 0fdf96512613574591f501d63fe49495ba40e1d5 + + 15.2 Ciphers can now be specified through: + + - Algorithms via 9aefbff30d280c60fc9d8cc3e0b2f19fc70a2f28 + + Reviewed-by: Daniel Stenberg and Marcel Raad + Closes #5358 -- [Harry Sintonen brought this change] +Daniel Stenberg (8 May 2020) +- checksrc: close the .checksrc file handle when done reading - build: fixed build for systems with select() in unistd.h +- RELEASE-NOTES: synced - Closes #5169 + And bumped next version to 7.71.0 -- memdebug: don't log free(NULL) +- [Gilles Vollant brought this change] + + CURLOPT_SSL_OPTIONS: add *_NATIVE_CA to use Windows CA store (with openssl) - ... it serves no purpose and fills up the log. + Closes #4346 -- cleanup: insert newline after if() conditions +- TODO: native IDN support on macOS + +- urlapi: accept :: as a valid IPv6 address - Our code style mandates we put the conditional block on a separate - line. These mistakes are now detected by the updated checksrc. + Text 1560 is extended to verify. + + Reported-by: Pavel Volgarev + Fixes #5344 + Closes #5351 -- checksrc: warn on obvious conditional blocks on the same line as if() +- THANKS-filter: Peter Wang + +- [Peter Wang brought this change] + + *_sspi: fix bad uses of CURLE_NOT_BUILT_IN - Closes #5164 + Return CURLE_AUTH_ERROR instead of CURLE_NOT_BUILT_IN for other + instances of QuerySecurityPackageInfo failing, as in + commit 2a81439553286f12cd04a4bdcdf66d8e026d8201. + + Closes #5355 -- [Roger Orr brought this change] +- docs/HTTP3: add qlog to the quiche build instruction - cmake: add CMAKE_MSVC_RUNTIME_LIBRARY +- ngtcp2: introduce qlog support - Fixes #5165 - Closes #5167 + If the QLOGDIR environment variable is set, enable qlogging. + + ... and create Curl_qlogdir() in the new generic vquic/vquic.c file for + QUIC functions that are backend independent. + + Closes #5353 -- [Daiki Ueno brought this change] +- ntlm_sspi: fix bad use of CURLE_NOT_BUILT_IN + + That return code is reserved for build-time conditional code not being + present while this was a regular run-time error from a Windows API. + + Reported-by: wangp on github + Fixes #5349 + Closes #5350 - ngtcp2: update to git master for the key installation API change +- runtests: show elapsed test time with higher precision (ms) + +- RELEASE-NOTES: synced + +- http2: simplify and clean up trailer handling - This updates the ngtcp2 OpenSSL backend to follow the API change in - commit 32e703164 of ngtcp2. + Triggered by a crash detected by OSS-Fuzz after the dynbuf introduction in + ed35d6590e72. This should make the trailer handling more straight forward and + hopefully less error-prone. - Notable changes are: - - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace - ngtcp2_crypto_derive_and_install_key - - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key - has been removed + Deliver the trailer header to the callback already at receive-time. No + longer caches the trailers to get delivered at end of stream. - Fixes #5166 - Closes #5168 + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22030 + Closes #5348 -- [Cyrus brought this change] +Marc Hoersken (7 May 2020) +- appveyor: disable test 1139 instead of ignoring it + + Spending time on manpage checking makes no sense + for these builds due to lacking manpage support. - SECURITY.md: minor rephrase +- appveyor: disable flaky test 1501 and ignore broken 1056 - Closes #5158 + Test 1501 is flaky on Windows CI due to being time sensitive + and the testsuite relying on taskkill.exe to check for the + existance of processes which can take to much time itself. + + Test 1056 is broken in autotools-based Windows builds due + to scope ID support missing in these builds at the moment. -- output.d: quote the URL when globbing +- test613.pl: make tests 613 and 614 work with OpenSSH for Windows - Some shells do globbing of their own unless the URL is quoted, so maybe - encourage this. + OpenSSH for Windows shows group and other/world permissions as *, + because those concepts do not exist on Windows. It also does not + show the current or parent directory, so we just ignore those. - Co-authored-by: Jay Satiro - Closes #5160 + Reviewed-by: Daniel Stenberg + Closes #5328 -- dist: add tests/version-scan.pl to tarball +Daniel Stenberg (6 May 2020) +- runtests: set +x mode again + +- libssh2: convert over to use dynbuf - ... used in test 1177. + In my very basic test that lists sftp://127.0.0.1/tmp/, this patched + code makes 161 allocations compared to 194 in git master. A 17% + reduction. - Follow-up to a97d826f6de3 + Closes #5336 -- test1177: verify that all the CURL_VERSION_ bits are documented +- travis: add "qlog" as feature in the quiche build -- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented +- quiche: enable qlog output - Considered experimental and therefore we can do this. + quiche has the potential to log qlog files. To enable this, you must + build quiche with the qlog feature enabled `cargo build --features + qlog`. curl then passes a file descriptor to quiche, which takes + ownership of the file. The FD transfer only works on UNIX. - Closes #5157 + The convention is to enable logging when the QLOGDIR environment is + set. This should be a path to a folder where files are written with the + naming template .qlog. + + Co-authored-by: Lucas Pardue + Replaces #5337 + Closes #5341 -- KNOWN_BUGS: DoH doesn't inherit all transfer options +- urldata.h: remove #define HEADERSIZE, not used anymore - Closes #4578 - Closes #4579 + Follow-up to ed35d6590e72c -- KNOWN_BUGS: DoH leaks memory after followlocation +- ngtcp2: convert to dynbuf - Closes #4592 + Closes #5335 -- KNOWN_BUGS: "FTPS needs session reuse" +- connect: make happy eyeballs work for QUIC (again) - Closes #4654 + Follow-up from dbd16c3e256c6c (regression in 7.70.0) + + Closes #5334 -- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed +- connect: add two asserts to clue code analyzers in a little -- TODO: Set custom client ip when using haproxy protocol +- http_proxy: ported to use dynbuf instead of a static size buffer - Closes #5125 + Removes a 16K static buffer from the easy handle. Simplifies the code. -Michael Kaufmann (27 Mar 2020) -- writeout_json: Fix data type issues +- dynbuf: introduce internal generic dynamic buffer functions - Load long values correctly (e.g. for http_code). + A common set of functions instead of many separate implementations for + creating buffers that can grow when appending data to them. Existing + functionality has been ported over. - Use curl_off_t (not long) for: - - size_download (CURLINFO_SIZE_DOWNLOAD_T) - - size_upload (CURLINFO_SIZE_UPLOAD_T) + In my early basic testing, the total number of allocations seem at + roughly the same amount as before, possibly a few less. - The unit for these values is bytes/second, not microseconds: - - speed_download (CURLINFO_SPEED_DOWNLOAD_T) - - speed_upload (CURLINFO_SPEED_UPLOAD_T) + See docs/DYNBUF.md for a description of the API. - Fixes #5131 - Closes #5152 + Closes #5300 -Daniel Stenberg (27 Mar 2020) -- mailmap: fixup a few author names/fields +- runtests: remove sleep calls - Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and - Svyatoslav Mishyn + Remove many one second sleeps that were done *after* each newly started + test server already has been verified. They should not have any purpose + there. + + Closes #5323 -- version: add 'cainfo' and 'capath' to version info struct +- asyn-*: remove support for never-used NULL entry pointers - Suggested-by: Timothe Litt - URL: https://curl.haxx.se/mail/lib-2020-03/0090.html - Reviewed-by: Jay Satiro + ... and instead convert those to asserts to make sure they are truly + never NULL. - Closes #5150 + Closes #5324 -- RELEASE-NOTES: synced +- [Emil Engler brought this change] -Jay Satiro (26 Mar 2020) -- SSLCERTS.md: Fix example code for setting CA cert file + doc: Rename VERSIONS to VERSIONS.md as it already has Markdown syntax - Prior to this change the documentation erroneously said use - CURLOPT_CAPATH to set a CA cert file. + Closes #5325 + +Jay Satiro (2 May 2020) +- asyn-thread: fix cppcheck warning - Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html - Reported-by: Timothe Litt + - Check for NULL entry parameter before attempting to deref entry in + Curl_resolver_is_resolved, like is already done in asyn-ares. - Closes https://github.com/curl/curl/pull/5151 - -Marc Hoersken (26 Mar 2020) -- sockfilt: add logmsg output to select_ws_wait_thread on Windows + This is to silence cppcheck which does not seem to understand that + asyn-ares and asyn-thread have separate Curl_resolver_is_resolved + and those units are mutually exclusive. Prior to this change it warned + of a scenario where asyn-thread's Curl_resolver_is_resolved is called + with a NULL entry from asyn-ares, but that couldn't happen. - Assisted-by: Jay Satiro - Reviewed-by: Daniel Stenberg + Reported-by: rl1987@users.noreply.github.com - Closes #5086 + Fixes https://github.com/curl/curl/issues/5326 -Daniel Stenberg (26 Mar 2020) -- docs/make: generate curl.1 from listed files only +- select: fix overflow protection in Curl_socket_check - Previously it rendered the page from files matching "*.d" in the correct - directory, which worked fine in git builds when the files were added but - made it easy to forget adding the files to the dist. + Follow-up to a96c752 which changed the timeout_ms type from time_t to + timediff_t. - Now, only man page sections listed in DPAGES in Makefile.inc will be - used, thus "forcing" us to update this to get the man page right and get - it included in the dist at the same time. + Ref: https://github.com/curl/curl/pull/5240 - Ref: #5146 - Closes #5149 + Closes https://github.com/curl/curl/pull/5286 -- openssl: adapt to functions marked as deprecated since version 3 - - OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES - functions we use. +Marc Hoersken (2 May 2020) +- sockfilt: make select_ws stop waiting on exit signal event - Fix the MD4 and SSL_CTX_load_verify_locations warnings. + This makes sure that select_ws behaves similar to real select + which stops waiting on a signal handler being triggered. - In configure, detect OpenSSL v3 and if so, inhibit the deprecation - warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and - until we rewrite the code to use non-deprecated functions we better - ignore these warnings as they don't help us. + This makes it possible to gracefully stop sockfilt.exe on + Windows with taskkill /IM sockfilt.exe (without /F force flag). - Closes #5139 + Reviewed-by: Jay Satiro + Part of #5260 -- dist: add mail-rcpt-allowfails.d to the tarball +- tests/server/util.[ch]: add exit event to stop waiting on Windows - Reported-by: Maksim Stsepanenka - Reviewed-by: Jat Satiro + This commit adds a global exit event to the test servers that + Windows-specific wait routines can use to get triggered if the + program was signaled to be terminated, eg. select_ws in sockfilt.c - Closes #5146 - -- travis: update the ngtcp2 build to use the latest OpenSSL patch + The exit event will be managed by the signal handling code and is + set to not reset automatically to support multiple wait routines. - ... which also makes it OpenSSL 1.1.1d based and not v3. + Reviewed-by: Jay Satiro + Closes #5260 -Marc Hoersken (24 Mar 2020) -- CI: remove default Ubuntu build from GitHub Actions +- tests/server/util.c: fix thread handle not being closed - We are already running a very similar Ubuntu build on Travis CI. - The macOS variant of this default build is kept on Github Actions. + Reviewed-by: Jay Satiro + Part of #5260 -- CI: bring GitHub Actions fuzzing job in line with macOS jobs +- tests/server/util.c: use raise instead of calling signal handler - Update YAML formatting, job naming and triggers. + Use raise to trigger signal handler instead of calling it + directly and causing potential unexpected control flow. + + Reviewed-by: Jay Satiro + Part of #5260 -- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions +- tests: add support for SSH server variant specific transfer paths - Reduce workload on Azure Pipelines and Travis CI while - consolidating macOS jobs onto less utilized GitHub Actions. + OpenSSH for Windows requires paths in the format of /C:/ + instead of the pseudo-POSIX paths /cygdrive/c/ or just /c/ Reviewed-by: Daniel Stenberg - - Closes #5124 + Closes #5298 -Daniel Stenberg (24 Mar 2020) -- config: remove all defines of HAVE_DES_H +Daniel Stenberg (2 May 2020) +- RELEASE-NOTES: synced + +- libssh2: set the expected total size in SCP upload init - As there's no code using it. + ... as otherwise the progress callback gets called without that + information, making the progress meter have less info. - Closes #5144 + Reported-by: Murugan Balraj + Bug: https://curl.haxx.se/mail/archive-2020-05/0000.html + Closes #5317 -- copyright: fix out-of-date copyright ranges and missing headers - - Reported by the new script 'scripts/copyright.pl'. The script has a - regex whitelist for the files that don't need copyright headers. +- runtests: make the logmsg from the ssh server only show in verbose + +- tests: make test 1248 + 1249 use %NOLISTENPORT - Removed three (mostly usesless) README files from docs/ + ... instead of a port of a non-running server so that it works + stand-alone. - Closes #5141 + Closes #5318 -- packages: add OS400/chkstrings.c to the dist +- examples: remove asiohiper.cpp - Reported-by: Jon Rumsey - Fixes #5142 - Closes #5143 + This example has repeatedly been reported to contain bugs, and as users + copy and paste code from this into production, I now deem it better to + not provide the example at all. + + Closes #5090 + Closes #5322 -- [Clément Notin brought this change] +- [Emil Engler brought this change] - nghttp2: 1.12.0 required + doc: add missing closing parenthesis in CURLINFO_SSL_VERIFYRESULT.3 - since nghttp2_session_set_local_window_size is needed - - Closes #5140 - -- RELEASE-NOTES: synced + Closes #5320 -- [Calvin Buckley brought this change] +- [Emil Engler brought this change] - OS400: Update strings for ccsid-ifier + KNOWN_BUGS: Remove "curl --upload-file . hang if delay in STDIN" - Fixes build. + It was fixed in 9a2cbf3 - Closes #5132 + Closes #5319 -- cirrus: make freebsd ignore the tests instead of skipping +- cirrus: disable SFTP and SCP tests - To allow us to see in the CI logs how they actually behave + ... as we can't seem to start the sshd server on it. Those problems + existed before d1239b50bececd (running the SSH server on a random port), + but they're more noticable now since there are more failed attempts in + the logs. - Closes #5091 + Closes #5315 -- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 +- [Emil Engler brought this change] -- Revert "cirrus-ci: disable the FreeBSD 13 builds" + runtests: fix typo in the existence of disabled tests checker - This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. + Closes #5316 -- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override - - To let debug-builds return fake values, like in test 970. +Dan Fandrich (30 Apr 2020) +- test75: Remove precheck test - Ref: #5131 - Closes #5136 + This has not been needed since commit 9fa42bed and often prevents it + from running at all with dynamic test ports. -- test970: improve the test +- tests: Stop referring to server ports when they're not used - - send more data to make problems more obvious - - don't start the data with minus, it makes diffs harder to read - - skip the headers in the stdout comparison - - save to a file name to also verify 'filename_effective' + Several tests referred to specific server ports even when the test + didn't actually use that server or specify that it's needed. In such + cases, the test harness substitutes the text "[not running]" as the port + number which causes many such tests to fail due to the inability to + parse the URL. These tests are changed to use %NOLISTENPORT which will + always be substituted correctly. + +Daniel Stenberg (30 Apr 2020) +- [Emil Engler brought this change] + + GnuTLS: Backend support for CURLINFO_SSL_VERIFYRESULT - Ref: #5131 + Closes #5287 -- CURLINFO_NUM_CONNECTS: improve accuracy +- conncache: various concept cleanups - The counter was not bumped in all cases correctly. + More connection cache accesses are protected by locks. - Reported-by: Marcel Raad - Ref: #5131 - Closes #5135 - -- TODO: Use "random" ports for the test servers - -- lib/curl_setup: adjust the copyright year range + CONNCACHE_* is a beter prefix for the connection cache lock macros. - Follow-up from d820224b8 + Curl_attach_connnection: now called as soon as there's a connection + struct available and before the connection is added to the connection + cache. + + Curl_disconnect: now assumes that the connection is already removed from + the connection cache. + + Ref: #4915 + Closes #5009 -Jay Satiro (21 Mar 2020) -- curl_setup: define _WIN32_WINNT_[OS] symbols +- tests: tests: run stunnel for HTTPS and FTPS on dynamic ports - .. because not all Windows build systems have those symbols, and even - those that do may be missing newer symbols (eg the Windows 7 SDK does - not define _WIN32_WINNT_WIN10). + As stunnel is an external tool and it has no specific option to export + the actually used port number when asked to listen to 0, runtests + instead iterates over ten randomly picked high number ports and sticks + to the first one stunnel can listen to. - Those symbols are used in build-time logic to decide which API to use - and prior to this change if the symbols were missing it would have - resulted in deprecated API being used when more recent functions were - available (eg GetVersionEx used instead of VerifyVersionInfo). + Closes #5267 + +- tests: pick a random port number for SSH - Reported-by: FuccDucc@users.noreply.github.com + Since sshd doesn't have such an option by itself, we iterate over a + series of random ports until one works. - Probably fixes https://github.com/curl/curl/issues/4995 - Closes https://github.com/curl/curl/pull/5057 + Closes #5273 -- [Ross Burton brought this change] +- [Rikard Falkeborn brought this change] - curl-functions.m4: remove inappropriate AC_REQUIRE - - AC_REQUIRE means "if this macro hasn't been executed already, execute - it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) - isn't correct at that will execute AC_RUN_IFELSE without any arguments. + libtest/cmake: Remove commented code - With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, - AC_RUN_IFELSE without a default value when cross-compiling is fatal. - The result is that curl with autoconf 2.70 cannot cross-compile. + These were commented out in e9dd0998706a when Makefile.inc was included + instead. 11 years have passed since then and the commented code is of + course very outdated. Remove it to avoid confusion. - Fixes https://github.com/curl/curl/issues/5126 - Closes https://github.com/curl/curl/pull/5130 + Closes #5311 -Marc Hoersken (20 Mar 2020) -- ci/tests: fix Azure Pipelines not running Windows containers - - Workaround posted here: microsoft/azure-pipelines-agent#2864 +- schannel: source code reindent - Assisted-by: Simon Chalifoux - Assisted-by: Tommy Petty + White space edits only. Conform better to standard curl source code + indenting style. - Fixes #5117 - Closes #5129 + Closes #5305 -Daniel Stenberg (20 Mar 2020) -- tests: add test 430, 431 and 432 to verify the --config fix +Kamil Dudka (29 Apr 2020) +- test1177: look for curl.h in source directory - Verify the fixes in 4e0b4fee4 - -- [Rici Lake brought this change] + If we use a separate build directory, there is no copy of the header. + + Closes #5310 - cmdline: fix handling of OperationConfig linked list (--next) +- tests: look for preprocessed tests in build directory - Ensures that -K/--config inserts new items at the end of the list - instead of overwriting the second item, and that after a -K/--config - option has been parsed, the option parser's view of the current config - is update. + ... which is not always the same directory as source directory - Fixes #5120 - Closes #5123 - -Marc Hoersken (20 Mar 2020) -- test2100: fix static port instead of dynamic value being used + Closes #5310 -- test970: fix static ip:port instead of dynamic values being used - -Daniel Stenberg (19 Mar 2020) -- secure transport: remove the BACKEND define kludge +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: synced - Closes #5122 + ... and bumped curlver.h to 7.70.1 -- mbedtls: remove the BACKEND define kludge +Version 7.70.0 (29 Apr 2020) -- bearssl: remove the BACKEND define kludge +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: 7.70.0 -- wolfssl: remove the BACKEND define kludge +- THANKS: synced with the 7.70.0 release -- nss: remove the BACKEND define kludge +- headers: copyright range fix -- gnutls: remove the BACKEND define kludge +- [Rikard Falkeborn brought this change] -- openssl: remove the BACKEND define kludge + doh: Constify some input pointers - Use a proper variable instead to make it easier to use a debugger and - read the code. + Closes #5306 -Marc Hoersken (19 Mar 2020) -- tests: make Python-based servers compatible with Python 2 and 3 - - Update smbserver.py and negtelnetserver.py to be compatible with - Python 3 while staying backwards-compatible to support Python 2. +- nss: check for PK11_CreateDigestContext() returning NULL - Fix string encoding and handling of echoed and transferred data. + ... to avoid crashes! - Tested with both Python 2.7.17 and Python 3.7.7 + Reported-by: Hao Wu + Fixes #5302 + Closes #5303 + +- travis: bump the wolfssl CI build to use 4.4.0 - Reported-by: Daniel Stenberg - Assisted-by: Kamil Dudka - Reviewed-by: Marcel Raad + Closes #5301 + +- copyright updates: adjust year ranges + +Marc Hoersken (26 Apr 2020) +- CI: do not include */ci branches in PR builds - Fixes #5104 - Closes #5110 + Align Azure Pipelines with GitHub Actions. -Daniel Stenberg (18 Mar 2020) -- writeout_json: use curl_off_t printf() option for the time output +Daniel Stenberg (25 Apr 2020) +- runtests: check for the disabled tests relative srcdir - Follow-up to: 04c03416e68fd635a15 + To make it work correctly for out-of-tree builds. - Closes #5115 - -- RELEASE-NOTES: synced + Follow-up to 75e8feb6fb08b - Uh, I missed this in 1a46b218db + Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 + Reported-by: Marcel Raad + Closes #5297 -- RELEASE-NOTES: synced +- runtests: revert commenting out a line I did for debugging - ... and bumped curlver.h to 7.70.0 + Follow-up to 11091cd4d. It was not meant to be pushed! -Jay Satiro (18 Mar 2020) -- http2: Fix erroneous debug message that h2 connection closed - - Prior to this change in libcurl debug builds http2 stream closure was - erroneously referred to as connection closure. +- smtp: set auth correctly - Before: - * nread <= 0, server closed connection, bailing + Regression since 7.69.0 and 68fb25fa3fcff. - After: - * nread == 0, stream closed, bailing + The code wrongly assigned 'from' instead of 'auth' which probably was a + copy and paste mistake from other code, leading to that auth could + remain NULL and later cause an error to be returned. - Closes https://github.com/curl/curl/pull/5118 + Assisted-by: Eric Sauvageau + Fixes #5294 + Closes #5295 -Daniel Stenberg (18 Mar 2020) -- tool_setopt: correct the copyright year range +Marcel Raad (25 Apr 2020) +- lib: clean up whitespace - Follow-up to 5450428491 + This fixes CodeFactor warnings. -Jay Satiro (18 Mar 2020) -- [Johannes Schindelin brought this change] +Daniel Stenberg (25 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] - schannel: add "best effort" revocation check option + libssh: avoid options override by configuration files - - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and - --ssl-revoke-best-effort to allow a "best effort" revocation check. + Previously, options set explicitly through command line options could be + overridden by the configuration files parsed automatically when + ssh_connect() was called. - A best effort revocation check ignores errors that the revocation check - was unable to take place. The reasoning is described in detail below and - discussed further in the PR. + By calling ssh_options_parse_config() explicitly, the configuration + files are parsed before setting the options, avoiding the options + override. Once the configuration files are parsed, the automatic + configuration parsing is not executed. - --- + Fixes #4972 + Closes #5283 + Signed-off-by: Anderson Toshiyuki Sasaki + +- runtests: when mentions http, kill http/2 too - When running e.g. with Fiddler, the schannel backend fails with an - unhelpful error message: + Since the http2 test server is a mere proxy that needs to know about the + dynamic port the HTTP server is using, it too needs to get restarted + when the http server is killed. - Unknown error (0x80092012) - The revocation function was unable - to check revocation for the certificate. + A regression caused by 80d6515. - Sadly, many enterprise users who are stuck behind MITM proxies suffer - the very same problem. + Fixes #5289 + Closes #5291 + +- [Yuri Slobodyanyuk brought this change] + + docs: fix two typos - This has been discussed in plenty of issues: - https://github.com/curl/curl/issues/3727, - https://github.com/curl/curl/issues/264, for example. + Closes #5292 + +- [Emil Engler brought this change] + + tests/git: ignore mqttd and port files - In the latter, a Microsoft Edge developer even made the case that the - common behavior is to ignore issues when a certificate has no recorded - distribution point for revocation lists, or when the server is offline. - This is also known as "best effort" strategy and addresses the Fiddler - issue. - - Unfortunately, this strategy was not chosen as the default for schannel - (and is therefore a backend-specific behavior: OpenSSL seems to happily - ignore the offline servers and missing distribution points). + Closes #5290 + +- tests: make runtests check that disabled tests exists - To maintain backward-compatibility, we therefore add a new flag - (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option - (`--ssl-revoke-best-effort`) to select the new behavior. + ... and error out if so. Removed '536' from DISABLED as there is no such + test file. - Due to the many related issues Git for Windows and GitHub Desktop, the - plan is to make this behavior the default in these software packages. + Closes #5288 + +- test1154: set a proper name + +- select: make Curl_socket_check take timediff_t timeout - The test 2070 was added to verify this behavior, adapted from 310. + Coverity found CID 1461718: - Based-on-work-by: georgeok - Co-authored-by: Markus Olsson - Signed-off-by: Johannes Schindelin + Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > + 9223372036854775807L" is always false regardless of the values of its + operands. This occurs as the logical second operand of "||". - Closes https://github.com/curl/curl/pull/4981 + Closes #5240 -- multi: Improve parameter check for curl_multi_remove_handle +- [i-ky brought this change] + + libcurl-multi.3: added missing full stop - - If an easy handle is owned by a multi different from the one specified - then return CURLM_BAD_EASY_HANDLE. + Closes #5285 + +Jay Satiro (22 Apr 2020) +- transfer: Switch PUT to GET/HEAD on 303 redirect - Prior to this change I assume user error could cause corruption. + Prior to this change if there was a 303 reply to a PUT request then + the subsequent request to respond to that redirect would also be a PUT. + It was determined that was most likely incorrect based on the language + of the RFCs. Basically 303 means "see other" resource, which implies it + is most likely not the same resource, therefore we should not try to PUT + to that different resource. - Closes https://github.com/curl/curl/pull/5116 - -Viktor Szakats (17 Mar 2020) -- windows: suppress UI in all CryptAcquireContext() calls + Refer to the discussions in #5237 and #5248 for more information. - Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters - Reviewed-by: Marc Hörsken - Closes https://github.com/curl/curl/pull/5088 + Fixes https://github.com/curl/curl/issues/5237 + Closes https://github.com/curl/curl/pull/5248 -Daniel Stenberg (17 Mar 2020) -- writeout_json: add missing comma to fix the HTTP version +Daniel Stenberg (22 Apr 2020) +- lib/mk-ca-bundle: skip empty certs - Follow-up to 04c03416e68fd635a15 + Reviewed-by: Emil Engler + Reported-by: Ashwin Metpalli + Fixes #5278 + Closes #5280 -- test 970: verify --write-out '%{json}' - - Makes curl_easy_getinfo() of "variable" numerical content instead return - the number set in the env variable `CURL_TIME`. +- version: skip idn2_check_version() check and add precaution - Makes curl_version() of "variable" textual content. This guarantees a - stable version string which can be tested against. Environment variable - `CURL_VERSION` defines the content. + A gcc-10's -fanalyze complaint made me spot and do these improvements. - Assisted-by: Mathias Gumz + Closes #5281 -- [Mathias Gumz brought this change] +- RELEASE-NOTES: synced - writeout: support to generate JSON output - - This commit adds support to generate JSON via the writeout feature: +- [Brian Bergeron brought this change] + + curl.h: update comment typo - -w "%{json}" + "routines with be invoked" -> "routines will be invoked" - It leverages the existing infrastructure as much as possible. Thus, - generating the JSON on STDERR is possible by: + Closes #5279 + +- [Emil Engler brought this change] + + GnuTLS: Don't skip really long certificate fields - -w "%{stderr}%{json}" + Closes #5271 + +- gnutls: bump lowest supported version to 3.1.10 - This implements a variant of - https://github.com/curl/curl/wiki/JSON#--write-out-json. + GnuTLS 3.1.10 added new functions we want to use. That version was + released on Mar 22, 2013. Removing support for older versions also + greatly simplifies the code. - Closes #4870 + Ref: #5271 + Closes #5276 -- CI: stop ignoring 323, it is disabled +- mqtt: make NOSTATE get within the debug name array -- DISABLED: disable test 323 - - The test uses SRP to "a server not supporting it" but modern stunnel - versions will silently accept it and remain happy. The test is therefore - faulty. +- tests: run the RTSP test server on a dynamic port number - I haven't figured out how to make stunnel explicitly reject SRP-using - connects. + To avoid port collisions. - Reported-by: Marc Hörsken - Fixes #5105 - Closes #5113 + Closes #5272 -Marc Hoersken (17 Mar 2020) -- ci/tests: increase timeouts for torture builds on Azure Pipelines +- tests: add %NOLISTENPORT and use it - For some reason the torture builds have slowed down recently. + The purpose with this variable is to provide a port number that is + reasonably likely to not have a listener on the local host so that tests + can try connect failures against it. It uses port 47 - "reserved" + according to IANA. - Reported-by: Daniel Stenberg + Updated six tests to use it instead of the previous different ports. + + Assisted-by: Emil Engler + Closes #5270 -Daniel Stenberg (16 Mar 2020) -- cmake: add support for building with wolfSSL +- mqtt: remove code with no purpose - My working build cmdline: + Detected by Coverity. CID 1462319. - $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . + "The same code is executed when the condition result is true or false, + because the code in the if-then branch and after the if statement is + identical." - Assisted-by: Brad King - Closes #5095 + Closes #5275 -- tool_operate: fix add_parallel_transfers when more are in queue +- mqtt: fix Curl_read() error handling while reading remaining length - Trying to return early from the function if no new transfers were added - would break the "morep" argument and cause issues. This could lead to - zero content "transfers" (within quotes since they would never be - started) when parallel-max was reduced. + Detected by Coverity. CID 1462320. - Reported-by: Gavin Wong - Analyzed-by: Jay Satiro - Fixes #4937 - Closes #5112 + Closes #5274 -- vtls: free ssl_config leftovers on out-of-memory - - Torture testing 2034 and 2037 found this. +- server/tftpd: fix compiler warning + Follow-up from 369ce38ac1d Reported-by: Marc Hörsken - Fixes #5108 - Closes #5109 -Marc Hoersken (16 Mar 2020) -- ci/tests: fix Azure Pipelines not running for pull requests +- http: free memory when Alt-Used header creation fails due to OOM - Closes #5111 + Reported-by: James Fuller + Fixes #5268 + Closes #5269 -Daniel Stenberg (15 Mar 2020) -- gskit: update the copyright year range +Daniel Gustafsson (20 Apr 2020) +- lib: fix typos in comments and errormessages - Follow-up from 083603c63a3 + This fixes a few randomly spotted typos in recently merged code, most + notably one in a userfacing errormessage the schannel code. -Marc Hoersken (15 Mar 2020) -- gskit: use our internal select wrapper for portability +Daniel Stenberg (20 Apr 2020) +- tests: run the SOCKS test server on a dynamic port number - Follow up to c52b342 - Closes #5106 + Closes #5266 -- tests: fix verification of stdout in test 1452 due to newline - - Fixes test1452:41:1: error: missing tag before +- [Johannes Schindelin brought this change] -- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI + multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - Also force the package index/cache to be updated before installing. + When cURL is compiled with support for multiple SSL backends, it is + possible to configure an SSL backend via `curl_global_sslset()`, but + only *before* `curl_global_init()` was called. - Closes #5103 - -- tests/README: add note about manually installing python-impacket + If another SSL backend should be used after that, a user might be + tempted to call `curl_global_cleanup()` to start over. However, we did + not foresee that use case and forgot to reset the SSL backend in that + cleanup. - Follow up to 4be2560 + Let's allow that use case. + + Fixes #5255 + Closes #5257 + Reported-by: davidedec on github + Signed-off-by: Johannes Schindelin -Daniel Stenberg (15 Mar 2020) -- transfer: cap retries of "dead connections" to 5 +- tests: run the TFTP test server on a dynamic port number - When libcurl retries a connection due to it being "seemingly dead" or by - REFUSED_STREAM, it will now only do it up five times before giving up, - to avoid never-ending loops. + Picking a dynamic unused port is better than a fixed to avoid the + collision risk. - Reported-by: Dima Tisnek - Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html - Closes #5074 + Closes #5265 -- TODO: TLS-PSK with OpenSSL +- mqtt: improve the state machine - Closes #5081 + To handle PUBLISH before SUBACK and more. + + Updated the existing tests and added three new ones. + + Reported-by: Christoph Krey + Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html + Closes #5246 -Marc Hoersken (15 Mar 2020) -- select: add 'timeout_ms' wrap-around precaution to Curl_select +- runtests: always put test number in servercmd file -- select: fix 'pending_ms' is assigned a value that is never used - - Detected by Codacy +- RELEASE-NOTES: synced -- select: move duplicate select preparation code into Curl_select - - Reviewed by Daniel Stenberg - Reviewed by Marcel Raad - Closes #5078 +- release-notes.pl: fix parsing typo -Daniel Stenberg (15 Mar 2020) -- connect: happy eyeballs cleanup +James Fuller (20 Apr 2020) +- ensure all references to ports are replaced by vars + +- add more alt-svc test coverage + +Daniel Stenberg (20 Apr 2020) +- test1247: use http server to get the port number set - Make sure each separate index in connn->tempaddr[] is used for a fixed - family (and only that family) during the connection process. + Follow-up to 0f5db7b263f + +- runtests: use a unix domain socket path with the pid in the name - If family one takes a long time and family two fails immediately, the - previous logic could misbehave and retry the same family two address - repeatedly. + To make it impossible for test cases to access the file name without + using the proper variable for the purpose. - Reported-by: Paul Vixie - Reported-by: Jay Satiro - Fixes #5083 - Fixes #4954 - Closes #5089 - -Marc Hoersken (15 Mar 2020) -- ci/tests: fix and align setting TFLAGS for make test-nonflaky + Closes #5264 -- ci/tests: install test suite dependencies stunnel and impacket +Daniel Gustafsson (19 Apr 2020) +- [Tom brought this change] -- tests: remove python_dependencies for smbserver from our tree - - Users of the SMB tests will have to install impacket manually. + src: Remove C99 constructs to ensure C89 compliance - Reasoning: our in-tree version of impacket was quite outdated - and only compatible with Python 2 which is already end-of-life. - Upgrading to Python 3 and a compatible impacket version would - require to import additional Python-only and CPython-extension - dependencies. This would have hindered portability enormously. + This fixes the error: 'for' loop initial declaration used outside C99 + mode by declaring the loop increment variable in the beginning of the + block instead of inside the for loop. - Closes #5094 + Fixes #5254 + Reviewed-by: Daniel Gustafsson -Jay Satiro (14 Mar 2020) -- Makefile.m32: Improve windres parameter compatibility - - - s/COFF/coff/ - - Some versions of windres do not recognize uppercase COFF as a valid - way to specify the COFF output format. - - Reported-by: Steven Penny +Daniel Stenberg (19 Apr 2020) +- runtests: dummy init the ports variables to avoid warnings - Fixes https://github.com/curl/curl/issues/5099 - Closes https://github.com/curl/curl/pull/5101 + ... and generate something that can help debug test cases. -- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares - - - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in - curl_easy_duphandle. - - Prior to this change if c-ares was used as the resolver backend and - either it was too old or libcurl was built without IPv6 support then - some of our resolver functions could return CURLE_NOT_BUILT_IN to - curl_easy_duphandle causing it to fail. - - Caused by c8f086b which shipped in 7.69.1. - - Reported-by: Karl Chen - - Fixes https://github.com/curl/curl/issues/5097 - Closes https://github.com/curl/curl/pull/5100 +- [Patrick Monnerat brought this change] -Daniel Stenberg (13 Mar 2020) -- docs: add warnings about FILE: URLs on Windows + mime: properly check Content-Type even if it has parameters - - --url man page section - - libcurl-security.3 gets the full text - - CURLOPT_URL.3 + New test 669 checks this fix is effective. - Reported-by: Tim Sedlmeyer + Fixes #5256 + Closes #5258 + Reported-by: thanhchungbtc on github -- server/getpart: make the "XML-parser" stricter - - When extracting a
and there's no before -
, this now outputs an error and returns a wrong string to - make users spot the mistake. +- tests/FILEFORMAT: converted to markdown and extended - Ref: #5070 - Closes #5071 + Closes #5261 -Marc Hoersken (13 Mar 2020) -- impacket: some more Python 3 code compatibility updates - - This makes smbserver load on Python 3, but still not work completely. +- test1245: make it work with dynamic FTP server port -- smbserver: pin Python version to 2 since we are not yet 3 compatible +- test1055: make it work with dynamic FTP port + +- test1028: make it run on dynamic FTP server port + +- tests: move pingpong server to dynamic listening port - Even though the existing code can be fixed to run on Python 3, the - tests will fail due to the Unicode transition the protocol is invalid. + FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic + ports - Follow up to ee63837 - Closes #5085 + Test 842-845 are unfortunately a bit hard to move over to this concept + right now and require "default port" still... -Daniel Stenberg (12 Mar 2020) -- [Viktor Szakats brought this change] +- test1056: work with dynamic HTTP ipv6 port - cleanup: fix some text/comment typos - - Closes #5087 +- test1448: work with dynamic HTTP server port -Marc Hoersken (12 Mar 2020) -- smbserver: fix Python version specific ConfigParser import +- tests: introduce preprocessed test cases - Follow up to ee63837 and 8c7c4a6 - Fixes #5077 - -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: synced + The runtests script now always performs variable replacement on the + entire test source file before the test gets executed, and saves the + updated version in a temporary file (log/test[num]) so that all test + case readers/servers can use that version (if present) and thus enjoy + the powers of test case variable substitution. - bumped to 7.69.2 - -Dan Fandrich (11 Mar 2020) -- tests/data: Fix some XML formatting issues in test cases + This is necessary to allow complete port number freedom. - This allows these test files to pass xmllint. + Test 309 is updated to work with a non-fixed port number thanks to this. -Daniel Stenberg (11 Mar 2020) -- [Muhammad Herdiansyah brought this change] +- tests: make 2006-2010 handle different port number lengths - Makefile: run the cd commands in a subshell +- tests: run the sws server on "any port" - In bmake, if the directory is changed (with cd or anything else), bmake - won't return to the "root directory" on the next command (in the same - Makefile rule). This commit runs the cd command in a subshell so it - would work in bmake. + Makes the test servers for HTTP and Gopher pop up on a currently unused + port and runtests adapts to that! - Closes #5073 + Closes #5247 -- configure: convert -I to -isystem as a last step +Marc Hoersken (18 Apr 2020) +- sockfilt: tidy variable naming and data structure in select_ws - As all the -I uses in CFLAGS at that point are for system headers and - third party libraries this helps us remove/ignore warnings on those! + This commit does not introduce any logical changes to the code. - Closes #5060 + Reviewed-by: Jay Satiro and Marcel Raad + Closes #5238 -- configure: fix -pedantic-errors for GCC 5 and later - - If --enable-werror is used. +Daniel Stenberg (17 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] + + libssh: Use new ECDSA key types to check known hosts - Follow-up to d5c0351055d5709da which added it too early in the configure - script before $compiler_num was set correctly and thus this option was - never used. + From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA + keys depending on the curve. - Reported-by: Stepan Efremov - Fixes #5067 - Closes #5068 + Signed-off-by: Anderson Toshiyuki Sasaki + Fixes #5252 + Closes #5253 -- configure: document 'compiler_num' for gcc +Marcel Raad (17 Apr 2020) +- appveyor: add Unicode winbuild jobs - The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + - MINOR and ignores the patch version, and since gcc version 7 it only - sets it to MAJOR*100. + These are cheap as they don't build tests. - Reported-by: Stepan Efremov - Ref: #5067 - Closes #5069 - -Version 7.69.1 (11 Mar 2020) - -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: 7.69.1 - -- THANKS: from the 7.69.1 release - -- [Marc Hoersken brought this change] + Closes https://github.com/curl/curl/pull/5063 - test1129: fix invalid case of closing XML-tag and Content-Length +Daniel Stenberg (16 Apr 2020) +- mqttd: s/errno/SOCKERRNO - Fixes #5070 - Closes #5072 + To behave proper on Windows + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 + Closes #5241 -Marc Hoersken (10 Mar 2020) -- tests/data: fix static ip instead of dynamic value being used +- buildconf: use find -execdir instead, remove -print and the ares files - Follow up to 94ced8e - -- tests/data: fix static ip:port instead of dynamic values being used + Follow-up to 1e41bec96a6e - Closes #5065 + Suggested-by: Marc Hörsken -- tests/server: fix missing use of exe_ext helper function +- [Alexander V. Tikhonov brought this change] + + buildconf: avoid using tempfile when removing files - Follow up to 9819984 and 3dce984 - Reviewed-By: Daniel Stenberg - Closes #5064 + Closes #5213 -- runtests: log minimal and maximal used port numbers +- copyright: bump the copyright year range -Daniel Stenberg (9 Mar 2020) -- [Jim Fuller brought this change] +- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords - sftp: fix segfault regression introduced by #4747 +- [JP Mens brought this change] + + docs/MQTT: replace confusing 80 by 75 - This fix adds a defensive check for the case where the char *name in - struct libssh2_knownhost is NULL + I was a bit surprised by the `80`: first thought: what's HTTP doing + here? ;) - Fixes #5041 - Closes #5062 + Closes #5236 -- RELEASE-NOTES: synced +- [Brad King brought this change] -- socks4: fix host resolve regression + cmake: Avoid MSVC C4273 warnings in send/recv checks - 1. The socks4 state machine was broken in the host resolving phase + We use `check_c_source_compiles` to check possible send/recv signatures + by reproducing the forward declarations from system headers. On Windows + the `winsock2.h` header adds dll linkage settings to its forward + declaration. If ours does not match the compiler warns: - 2. The code now insists on IPv4-only when using SOCKS4 as the protocol - only supports that. + warning C4273: 'recv': inconsistent dll linkage - Regression from #4907 and 4a4b63d, shipped in 7.69.0 + Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so + that our linkage is consistent with that from `winsock2.h`. - Reported-by: amishmm on github - Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 - Closes #5061 - -- [Patrick Monnerat brought this change] + Fixes #4764 + Closes #5232 - silly web server: silent a compilation warning - - Recent gcc warns when byte count of strncpy() equals the destination - buffer size. Since the destination buffer is previously cleared and - the source string is always shorter, reducing the byte count by one - silents the warning without affecting the result. +Jay Satiro (14 Apr 2020) +- KNOWN_BUGS: Add entry 'Blocking socket operations' - Closes #5059 - -- [Patrick Monnerat brought this change] - - cookie: get_top_domain() sets zero length for null domains + - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of + known blocking operations. - This silents a compilation warning with gcc -O3. - -- [Patrick Monnerat brought this change] - - test 1560: avoid valgrind false positives + - New known bugs entry 'Blocking socket operations in non-blocking API' + that directs to the TODO's list of known blocking operations. - When using maximum code optimization level (-O3), valgrind wrongly - detects uses of uninitialized values in strcmp(). + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 - Preset buffers with all zeroes to avoid that. - -Steve Holme (8 Mar 2020) -- sha256: Added WinCrypt implementation + Reported-by: Marc Hoersken - Closed #5030 - -- sha256: Added SecureTransport implementation + Closes https://github.com/curl/curl/pull/5216 -Daniel Stenberg (7 Mar 2020) -- lib1564: reduce number of mid-wait wakeup calls - - This test does A LOT of *wakeup() calls and then calls curl_multi_poll() - twice. The first *poll() is then expected to return early and the second - not - as the first is supposed to drain the socketpair pipe. +Marc Hoersken (14 Apr 2020) +- test2043: use revoked.badssl.com instead of revoked.grc.com - It turns out however that when given "excessive" amounts of writes to - the pipe, some operating systems (the Solaris based are known) will - return EAGAIN before the pipe is drained, which in our test case causes - the second *poll() call to also abort early. + The certificate of revoked.grc.com has expired on 2020-04-13. - This change attempts to avoid the OS-specific behaviors in the test by - reducing the amount of wakeup calls from 1234567 to 10. + Reviewed-by: Jay Satiro - Reported-by: Andy Fiddaman - Fixes #5037 - Closes #5058 - -- [Patrick Monnerat brought this change] + Closes #5233 - mime: fix the binary encoder to handle large data properly - - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. +- sockfilt: fix broken pipe on Windows to be ready in select_ws - Fixes #4860 - Closes #4833 - Reported-by: RuurdBeerstra on github + Closes #5228 -- [Patrick Monnerat brought this change] +Daniel Stenberg (14 Apr 2020) +- RELEASE-NOTES: synced - mime: do not perform more than one read in a row - - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. - - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 667 checks mimepost using single-byte read callback with encoder. - New test 668 checks the end of part data early detection. - - Fixes #4826 - Reported-by: MrdUkk on github +- scripts/release-notes: fix duplicate output header -- [Patrick Monnerat brought this change] +- github/workflow: enable MQTT in the macOS debug build - mime: latch last read callback status. - - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. - - Fixes #4813 - Reported-by: MrdUkk on github +- azure: add mqtt support to one of the Windows builds -Marc Hoersken (7 Mar 2020) -- runtests: fix missing use of exe_ext helper function +- travis: add mqtt job on Linux -Daniel Stenberg (7 Mar 2020) -- [Ernst Sjöstrand brought this change] +- tests: add four MQTT tests 1190 - 1193 - ares: store dns parameters for duphandle - - With c-ares the dns parameters lives in ares_channel. Store them in the - curl handle and set them again in easy_duphandle. - - Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. - - Fixes #4893 - Closes #5020 - Signed-off-by: Ernst Sjöstrand +- tests: add the mqtt test server mqttd -- version: make curl_version* thread-safe without using global context +- tests: support hex encoded data and mqtt server - Closes #5010 + The mqtt server is started using a "random" port. -- RELEASE-NOTES: synced +- [Björn Stenberg brought this change] -Marc Hoersken (7 Mar 2020) -- tests: use native Sleep function as fallback on Windows + mqtt: add new experimental protocol - Reviewed-By: Daniel Stenberg - Closes #5054 - -- perl: align order and completeness of Windows OS checks + Closes #5173 -Daniel Stenberg (7 Mar 2020) -- tool_cb_see: set correct copyright year range +- TODO: Consider convenience options for JSON and XML? - Follow-up to a39e5bfb9 + Closes #5203 -Marc Hoersken (7 Mar 2020) -- seek: fix fallback for missing ftruncate on Windows +- tool: do not declare functions with Curl_ prefix - This fixes test 198 on versions of MinGW-w64 without ftruncate + To avoid collision risks with private libcurl symbols when linked with + static versions (or just versions not hiding internal symbols). - Reviewed-By: Daniel Stenberg - Reviewed-By: Marcel Raad - Closes #5055 - -- config-win32: Windows does not have ftruncate + Reported-by: hydra3333 on github + Fixes #5219 + Closes #5234 -Daniel Stenberg (7 Mar 2020) -- pause: force a connection (re-)check after unpausing - - There might be data available that was already read off the socket, for - example in the TLS layer. - - Reported-by: Anders Berg - Fixes #4966 - Closes #5049 +- [Nathaniel R. Lewis brought this change] -- socks5: switch state properly when the resolve is done + cmake: add aliases so exported target names are available in tree - Regression from 4a4b63d (and #4907) - Reported-by: vitaha85 on github - Fixes #5053 - Closes #5056 + Reviewed-by: Brad King + Closes #5206 -Jay Satiro (7 Mar 2020) -- libssh: Fix matching user-specified MD5 hex key - - Prior to this change a match would never be successful because it - was mistakenly coded to compare binary data from libssh to a - user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). +- version: increase buffer space for ssl version output - Reported-by: fds242@users.noreply.github.com + To avoid it getting truncated, especially when several SSL backends are + built-in. - Fixes https://github.com/curl/curl/issues/4971 - Closes https://github.com/curl/curl/pull/4974 + Reported-by: Gisle Vanem + Fixes #5222 + Closes #5226 -Daniel Stenberg (6 Mar 2020) -- pause: bail out on bad input - - A NULL easy handle or an easy handle without an associated connection - cannot be paused or unpaused. +Marc Hoersken (13 Apr 2020) +- cirrus: no longer ignore test 504 which is working again - Closes #5050 + The test is working again, because TCP blackholing is disabled. -Steve Holme (6 Mar 2020) -- unit1612: fixed the inclusion and compilation of the HMAC unit test - - Follow up to 3f74e5e6 to fix: - - - A typo in Makefile.inc where unit1611 was used instead - - Some compilation issues in unit1612.c +- appveyor: completely disable tests that fail to timeout early - Closes #5024 + The tests changed from ignored to disabled are tests that are + about connecting to non-listening socket. On AppVeyor these + tests are not reliable, because for some unknown reason the + connect is not timing out before the test time limit is reached. -Daniel Stenberg (6 Mar 2020) -- pause: return early for calls that don't change pause state +Daniel Stenberg (13 Apr 2020) +- test1908: avoid using fixed port number in test data - Reviewed-by: Patrick Monnerat - Ref: #4833 - Closes #5026 + Closes #5225 -Jay Satiro (6 Mar 2020) -- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - - Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused - CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. - - Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html - Reported-by: Felipe Gasper - - Closes https://github.com/curl/curl/pull/5048 +Jay Satiro (12 Apr 2020) +- [Andrew Kurushin brought this change] -- multi: skip EINTR check on wakeup socket if it was closed - - - Don't check errno on wakeup socket if sread returned 0 since sread - doesn't set errno in that case. + schannel: Fix blocking timeout logic - This is a follow-up to cf7760a from several days ago which fixed - Curl_multi_wait to stop busy looping sread on the non-blocking wakeup - socket if it was closed (ie sread returns 0). Due to a logic error it - was still possible to busy loop in that case if errno == EINTR. + - Fix schannel_send for the case when no timeout was set. - Closes https://github.com/curl/curl/pull/5047 - -Daniel Stenberg (6 Mar 2020) -- transfer: set correct copyright year range - -- urldata: remove the 'stream_was_rewound' connectdata struct member + Prior to this change schannel would error if the socket was not ready + to send data and no timeout was set. - ... as it is never set anywhere. + This commit is similar to parent commit 89dc6e0 which recently made the + same change for SOCKS, for the same reason. Basically it was not well + understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms + but actually means no timeout. - Follow-up to 2f44e94ef - Closes #5046 + Fixes https://github.com/curl/curl/issues/5177 + Closes https://github.com/curl/curl/pull/5221 -- Revert "pause: force-drain the transfer on unpause" +- socks: Fix blocking timeout logic - This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) + - Document in Curl_timeleft's comment block that returning 0 signals no + timeout (ie there's infinite time left). - Clearly that didn't solve the problem correctly. + - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. - Reported-by: Christopher Reid - Reopens #4966 - Fixes #5044 - -- RELEASE-NOTES: synced + Prior to this change if the timeout had a value of 0 and that was passed + to SOCKET_READABLE it would return right away instead of blocking. That + was likely because it was not well understood that when Curl_timeleft + returns 0 it is not a timeout of 0 ms but actually means no timeout. - and bumped curlver.h - -- MANUAL: update a dict-using command line + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 - The 'web1913' database is now invalid, use 'gcide' instead. + Closes https://github.com/curl/curl/pull/5220 -- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - - Closes #3841 +- [Marc Hoersken brought this change] -- polarssl: remove more references and mentions + gopher: check remaining time left during write busy loop + + Prior to this change gopher's blocking code would block forever, + ignoring any set timeout value. Assisted-by: Jay Satiro - Follow-up to 6357a19ff29dac04 - Closes #5036 - -Marc Hoersken (4 Mar 2020) -- tests: wrap ignored test failures in braces + Reviewed-by: Daniel Stenberg + + Similar to #5220 and #5221 + Closes #5214 -- tests: align some Windows sleep defines with each other +Daniel Stenberg (13 Apr 2020) +- [Dirkjan Bussink brought this change] -- tests: try to make sleeping portable by avoiding select - - select does not support just waiting on Windows: - https://perldoc.perl.org/perlport.html#select + gnutls: ensure TLS 1.3 when SRP isn't requested - Reviewed-By: Daniel Stenberg - Closes #5035 - -Daniel Stenberg (4 Mar 2020) -- runtests.1: rephrase how to specify what tests to run + When SRP is requested in the priority string, GnuTLS will disable + support for TLS 1.3. Before this change, curl would always add +SRP to + the priority list, effectively always disabling TLS 1.3 support. - Also mention the new tilde-prefixed way to ignore test results. + With this change, +SRP is only added to the priority list when SRP + authentication is also requested. This also allows updating the error + handling here to not have to retry without SRP. This is because SRP is + only added when requested and in that case a retry is not needed. - Reviewed-By: Marc Hoersken - Closes #5033 + Closes #5223 -- cirrus-ci: disable the FreeBSD 13 builds +Marc Hoersken (12 Apr 2020) +- tests/server: add hidden window to gracefully handle WM_CLOSE - FreeBSD 13.0 is apparently close to a year away from a stable release - and has proven to cause intermittent builds failures recently. + Forward Window events as signals to existing signal event handler. + +- tests/server: add CTRL event handler for Win32 consoles - Assisted-by: Dan Fandrich - Assisted-by: Fedor Korotkov - Fixes #5028 - Closes #5029 + Forward CTRL events as signals to existing signal event handler. -Version 7.69.0 (4 Mar 2020) +- tests/server: move all signal handling routines to util.[ch] + + Avoid code duplication to prepare for portability enhancements. -Daniel Stenberg (4 Mar 2020) -- RELEASE-NOTES: 7.69.0 +Daniel Stenberg (12 Apr 2020) +- compressed.d: stress that the headers are not modified + + Suggested-by: Michael Osipov + Assisted-by: Jay Satiro + Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 + Closes #5217 -- THANKS: from 7.69.0 +Marc Hoersken (11 Apr 2020) +- tests/server/util.c: use curl_off_t instead of long for pid - Now sorted case insensitive - -Marc Hoersken (3 Mar 2020) -- ci/tests: fix escaping of testnames and disable proxy for CI APIs + Avoid potential overflow of huge PIDs on Windows. - Follow up to ada581f and c0d8b96 - Closes #5031 + Related to #5188 + Assisted-by: Marcel Raad -Jay Satiro (3 Mar 2020) -- cmake: Show HTTPS-proxy in the features output +- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows - - Show HTTPS-proxy in the features output for those backends that - support it: OpenSSL, GnuTLS and NSS. + Since the Windows versions of both programs would write Windows + PIDs to their pidfiles which we cannot handle, we need to use + our known perl.exe Cygwin/msys PID together with exec() in order + to tie the spawned processes to the existance of our perl.exe - Prior to this change HTTPS-proxy was missing from the cmake features - output even if curl was built with it. Only cmake output was affected. - Both the library and tool correctly reported the feature. + The perl.exe that is executing secureserver.pl and sshserver.pl + has a Cygwin/msys PID, because it is started inside Cygwin/msys. - Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html - Reported-by: David Lopes + Related to #5188 + +- tests: add Windows compatible pidwait like pidkill and pidterm - Closes https://github.com/curl/curl/pull/5025 + Related to #5188 -Marc Hoersken (3 Mar 2020) -- ci/tests: Make it possible to still run but ignore failing tests +- tests: fix conflict between Cygwin/msys and Windows PIDs - This enables the development of a solution for the failing tests by - running them on CI while ignoring their result for the overall status. + Add 65536 to Windows PIDs to allow Windows specific treatment + by having disjunct ranges for Cygwin/msys and Windows PIDs. - Closes #4994 + See also: + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=b5e1003722cb14235c4f166be72c09acdffc62ea + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe + + Replaces #5178 + Closes #5188 -- README.md: add Azure DevOps Pipelines build status badge +Daniel Stenberg (11 Apr 2020) +- RELEASE-NOTES: synced -- ci/tests: Move CI test result creation above environment setup +- release-notes.pl: detect the start of the references in cleanup mode + +- Revert "file: on Windows, refuse paths that start with \\" - This avoids using our test servers as proxy to the AppVeyor API. + This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. - Closes #5022 - -- ci/tests: Send test results to AppVeyor for status overview + Reminded-by: Chris Roberts + Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html - Closes #5021 + Closes #5215 -Daniel Stenberg (3 Mar 2020) -- Revert "sha256: Added SecureTransport implementation" +Jay Satiro (11 Apr 2020) +- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE - This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) + - If loss of data may occur converting a timediff_t to time_t and + the time value is > TIME_T_MAX then treat it as TIME_T_MAX. - That commit broke test 1610 on macos builds without TLS. + This is a follow-up to 8843678 which removed the (time_t) typecast + from the macros so that conversion warnings could be identified. - Closes #5027 + Closes https://github.com/curl/curl/pull/5199 -- dist: include tests/azure.pm in the tarball +- test1148: tolerate progress updates better (again) - Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 - Reported-by: Marcel Raad - -Steve Holme (3 Mar 2020) -- configure.ac: Disable metalink if mbedTLS is specified + - Ignore intermediate progress updates. - Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as - being one of the backends that metalink needs to be disabled for, I - seem to have included it in the list of allowed SSL/TLS backends in - comnfigure.ac :( + - Support locales that use a character other than period as decimal + separator (eg 100,0%). - Closes #5013 - -- sha256: Tidy up following recent changes + test1148 checks that the progress finishes at 100% and has the right + bar width. Prior to this change the test assumed that the only progress + reported for such a quick transfer was 100%, however in rare instances + (like in the CI where transfer time can slow considerably) there may be + intermediate updates. For example, below is stderrlog1148 from a failed + CI run with explicit \r and \n added (it is one line; broken up so that + it's easier to understand). - Reviewed-by: Daniel Stenberg - Closes #4956 - -- sha256: Added WinCrypt implementation + \r + \r################################## 48.3% + \r######################################################################## 100.0% + \n + + Closes https://github.com/curl/curl/pull/5194 -- sha256: Added SecureTransport implementation +Marc Hoersken (10 Apr 2020) +- sshserver.pl: use cached Win32 environment check variable -- sha256: Added mbedtls implementation +- appveyor: partially revert 3413a110 to keep build without proxy + + Ref: #5211 and #4526 + Reported-by: Marcel Raad -- sha256: Added GNU TLS gcrypt implementation +- appveyor: ignore failing 'connect to non-listening proxy' tests + + Closes #5211 -- sha256: Added GNU TLS Nettle implementation +- CI/macos: convert CRLF to LF and align indentation -Jay Satiro (2 Mar 2020) -- curl_escape.3: Add a link to curl_free +Daniel Stenberg (9 Apr 2020) +- url: allow non-HTTPS altsvc-matching for debug builds - Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 + This is already partly supported but this part was missing. + Reported-by: James Fuller + + Closes #5205 -- curl_getenv.3: Fix the memory handling description +- server/resolve: remove AI_CANONNAME to make macos tell the truth - - Tell the user to call curl_free() to free the pointer returned by - curl_getenv(). + With this bit set, my mac successfully resolves "ip6-localhost" when in + fact there is no such host known to my machine! That in turn made test + 241 wrongly execute and fail. - Prior to this change the user was directed to call free(), but that - would not work in cases where the library and application use separate C - runtimes and therefore have separate heap memory management. + Closes #5202 + +- runtests: fix warning about using an undefined variable - Closes https://github.com/curl/curl/pull/5016 + Follow-up from 4d939ef6ceb2db1 -Daniel Stenberg (2 Mar 2020) -- [Nick Zitzmann brought this change] +- release-notes: fix the initial reference list output - md4: use init/update/final functions in Secure Transport +- github actions: run when pushed to master or */ci + PRs - We can use CC_MD4_Init/Update/Final without having to allocate memory - directly. + Avoid double-builds when using "local" branches for PRs. For both macos + and fuzz jobs. - Closes #4979 + Closes #5201 -Marc Hoersken (2 Mar 2020) -- ci/tests: some MacOS builds randomly take longer than 20min +- runtests: provide nicer errormsg when protocol "dump" file is empty -Daniel Stenberg (2 Mar 2020) -- multi_wait: stop loop when sread() returns zero - - It's unclear why it would ever return zero here, but this change fixes - Robert's problem and it shouldn't loop forever... - - Reported-by: Robert Dunaj - Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html - Closes #5019 +- [Gilles Vollant brought this change] -- http: mark POSTs with no body as "upload done" from the start + schannel: support .P12 or .PFX client certificates - As we have logic that checks if we get a >= 400 reponse code back before - the upload is done, which then got confused since it wasn't "done" but - yet there was no data to send! + Used with curl command line option like this: --cert + : --cert-type p12 - Reported-by: IvanoG on github - Fixes #4996 - Closes #5002 + Closes #5193 -- tests: disable 962, 963 and 964 on Windows +- tests: verify split initial HTTP requests with CURL_SMALLREQSEND - These tests are also doing UTF-8 SMTP. + test1294: "split request" being when the entire request isn't sent in + the first go, and the remainder is sent in the PERFORM state. A GET + request is otherwise not sending anything during PERFORM. - Follow-up to df207d2dd93b9e73 - -Marc Hoersken (2 Mar 2020) -- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer + test1295: same kind of split but with POST + + Closes #5197 -Daniel Stenberg (2 Mar 2020) -- configure: bump the AC_COPYRIGHT year range +- http: don't consider upload done if the request isn't completely sent off + + Fixes #4919 + Closes #5197 -- [Steve Holme brought this change] +- http: allow Curl_add_buffer_send() to do a short first send by force + + In a debug build, settting the environment variable "CURL_SMALLREQSEND" + will make the first HTTP request send not send more bytes than the set + amount, thus ending up verifying that the logic for handling a split + HTTP request send works correctly. - tests: disable SMTP UTF-8 tests on Windows +- connect: store connection info for QUIC connections - Fixes #4988 - Closes #4992 + Restores the --head functionality to the curl utility which extracts + 'protocol' that is stored that way. + + Reported-by: James Fuller + Fixes #5196 + Closes #5198 -- formdata/mime: copyright year range update +- tests/README: update the port numbers list - Due to the merge/revert cycle + Since the pipelining server is long gone. + Reported-by: James Fuller -- Revert "mime: latch last read callback status." +- select: remove typecast from SOCKET_WRITABLE/READABLE macros - This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. + So that they don't hide conversions-by-mistake - Fixes #5014 - Closes #5015 - Reopens #4833 + Reviewed-by: Jay Satiro + Closes #5190 -- Revert "mime: do not perform more than one read in a row" +- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also - This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. + Closes #5192 -- Revert "mime: fix the binary encoder to handle large data properly" +- release-notes: output trailing references sorted numerically + +- cleanup: correct copyright year range on a few files + +- configure: remove use of -vec-report0 from CFLAGS with icc - This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. + ... as it apparently isn't (always) supported. + Reported-by: Alain Miniussi + Fixes #5096 + Closes #5191 -- altsvc: both h3 backends now speak h3-27 +- warnless: remove code block for icc that didn't work + + Reported-by: Alain Miniussi + Fixes #5096 + +Marc Hoersken (6 Apr 2020) +- dist: add missing setup-win32.h - ... also updated the HTTP3 build description for ngtcp2 accordingly. + Follow up to d820224b8b -- [Patrick Monnerat brought this change] +Daniel Stenberg (6 Apr 2020) +- RELEASE-NOTES: synced - mime: fix the binary encoder to handle large data properly - - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. +- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance - Fixes #4860 - Reported-by: RuurdBeerstra on github + This script helps putting entries in the RELEASE-NOTES using a coherent + style and sorting with a minimal human editing effort - as long as the + first line in the commit message is good enough! There's a short howto + at the top of the file. -- [Patrick Monnerat brought this change] +- [Dennis Felsing brought this change] - mime: do not perform more than one read in a row - - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. + configure: don't check for Security.framework when cross-compiling - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 664 checks mimepost using single-byte read callback with encoder. - New test 665 checks the end of part data early detection. + Since it checks for the local file, not the cross-compiled one. - Fixes #4826 - Reported-by: MrdUkk on github - -- [Patrick Monnerat brought this change] + Closes #5189 - mime: latch last read callback status. - - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. +- TODO: Option to make -Z merge lined based outputs on stdout - Fixes #4813 - Reported-by: MrdUkk on github - Closes #4833 + Closes #5175 -Steve Holme (1 Mar 2020) -- unit1651: Fixed conversion compilation warning +- lib: never define CURL_CA_BUNDLE with a getenv - 371:17: warning: conversion to 'unsigned char' from 'int' may alter its - value [-Wconversion] + - it breaks the build (since 6de756c9b1de34b7a1) + - it's not documented and not consistent across platforms + - the curl tool does that getenv magic - Closes #5008 - -- configure.ac: Disable metalink support if an incompatible SSL/TLS specified + Bug: https://github.com/curl/curl/commit/6de756c#r38127030 + Reported-by: Gisle Vanem - tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, - The Win32 Crypto library and Apple's Common Crypto library. + Closes #5187 + +Marc Hoersken (5 Apr 2020) +- lib670: use the same Win32 API check as all other lib tests + +- appveyor: use random test server ports based upon APPVEYOR_API_URL - If an TLS backend such as mbedTLS or WolfSSL is specified then the - following error is given during compilation along, with a load of - unresolved extern errors: + Avoid conflicts of test server ports with AppVeyor API on localhost. - Can't compile METALINK support without a crypto library. + Closes #5034 + +- appveyor: sort builds by type and add two new variants - Reviewed-by: Daniel Stenberg - Closes #5006 + Related to #5034 and #5063 -Marc Hoersken (1 Mar 2020) -- ci/tests: Update Azure DevOps pipeline job display names +- appveyor: show failed tests in log even if test is ignored - Make the configure step more descriptive and align others. + And print API response with newline only if there is one -- ci/tests: Fix typo in previous commit 597cf2 +- appveyor: turn disabled tests into ignored result tests -- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available +Daniel Stenberg (5 Apr 2020) +- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" - For security reasons the access token is not available to PR builds. - Therefore we should not try to use the DevOps API with an empty token. + Fixed with #5170 (commit 23a870f2fd041278) -Daniel Stenberg (1 Mar 2020) -- build: remove all HAVE_OPENSSL_ENGINE_H defines +- test1566: verify --etag-compare that gets a 304 back - ... as there's nothing in the code that actually uses the define! The - last reference was removed in 38203f158. + Verifies the fix in #5183 - Closes #5007 + Closes #5186 -Jay Satiro (29 Feb 2020) -- [Rolf Eike Beer brought this change] +- [Kwon-Young Choi brought this change] - CMake: clean up and improve build procedures - - - remove check for unsupported old CMake versions - - - do not link to c-ares library twice - - - modernize custom Find modules - - - FindLibSSH2: - - pass version to FPHSA to show it in the output - - use LIBSSH2_VERSION define to extract the version number in - one shot. This variable exists in the header for 10 years. - - remove unneeded code + CURLINFO_CONDITION_UNMET: return true for 304 http status code - - FindNGHTTP2.cmake: - - drop needless FPHSA argument - - mark found variables as advanced + In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the + output file if the server did not transfered a file based on time + condition. In the same manner, getting a 304 HTTP response back from the + server, for example after passing a custom If-Match-* header, also + fulfill this condition. - - FindNSS.cmake: - - show version number + Fixes #5181 + Closes #5183 + +- [Kwon-Young Choi brought this change] + + curl: allow both --etag-compare and --etag-save with same file name - - FindCARES.cmake: - - drop default paths - - use FPHSA instead of checking things by hand + This change inverse the order of processing for the --etag-compare and + --etag-save option to process first --etag-compare. This in turn allows + to use the same file name to compare and save an etag. - - remove needless explict variable dereference + The original behavior of not failing if the etag file does not exists is + conserved. - - simplify count_true() + Fixes #5179 + Closes #5180 + +Viktor Szakats (4 Apr 2020) +- windows: enable UnixSockets with all build toolchains - - allow all policies up to version 3.16 to be set to NEW + Extend existing unix socket support in Windows builds to be + enabled for all toolchain vendors or versions. (Previously + it was only supported with certain MSVC versions + more recent + Windows 10 SDKs) - - do not rerun check for -Wstrict-aliasing=3 every time + Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ + Ref: https://github.com/curl/curl/issues/5162 + Closes: https://github.com/curl/curl/pull/5170 + +Daniel Stenberg (4 Apr 2020) +- KNOWN_BUGS: Store TLS context per transfer instead of per connection - In contrast to every other compiler flag this has a = in it, which CMake - can't have in a variable name. + Closes #5102 + +Marc Hoersken (3 Apr 2020) +- sockfilt: remove redundancy in timeout handling - - only read the interesting strings from curlver.h + And update other logmsg output in select_ws on Windows. + +- sockfilt: fix handling of ready closed sockets on Windows - Reviewed-by: Peter Wu + Replace the incomplete workaround regarding FD_CLOSE + only signalling once by instead doing a pre-check with + standard select and storing the result for later use. - Closes https://github.com/curl/curl/pull/4975 + select keeps triggering on closed sockets on Windows while + WSAEventSelect fires only once with data still available. + By doing the pre-check we do not run in a deadlock + due to waiting forever for another FD_CLOSE event. -- runtests: fix output to command log +- sockfilt: fix race-condition of waiting threads and event handling - - Record only the command of the most recently ran test in the command - log. + Fix race-condition of waiting threads finishing while events are + already being processed which lead to invalid or skipped events. - This is a follow-up to 02988b7 from several weeks ago which fixed - writing to the command log, however it saved all commands for all tests - instead of just the most recently ran test as we would now expect. + Use mutex to check for one event at a time or do post-processing. + In addition to mutex-based locking use specific event as signal. - Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 - Closes https://github.com/curl/curl/pull/5001 + Closes #5156 -Steve Holme (1 Mar 2020) -- polarssl: Additional removal +Daniel Stenberg (2 Apr 2020) +- [Leo Neat brought this change] + + CI-fuzz: increase fuzz time to 40 minutes - Follow up to 6357a19f. + Closes #5174 + +Marc Hoersken (2 Apr 2020) +- CI: increase Azure Pipelines timeouts due to performance issues - Reviewed-by: Daniel Stenberg - Closes #5004 + The current demand on Azure negatively impacts the CI performance. -- [Jonathan Cardoso Machado brought this change] +- runtests.pl: log host OS as detected by Perl environment - docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always - - Reviewed-by: Steve Holme - Closes #5005 +- ftpserver.pl: log before and after data connection is closed -- md5: Added implementation for mbedTLS - - Reviewed-by: Jay Satiro - Closes #4980 +Daniel Stenberg (1 Apr 2020) +- RELEASE-NOTES: synced -- md5: Use pointer notation for array parameters in GnuTLS implementation +- RELEASE-PROCEDURE.md: run the copyright.pl script! -- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 +- vquic/ngtcp2.h: update copyright year range - Closes #4983 + Follow-up to 0736ee73d346a52 -Marc Hoersken (29 Feb 2020) -- ci/tests: Send test results to Azure DevOps for reporting +- [Daiki Ueno brought this change] -Daniel Stenberg (29 Feb 2020) -- pause: force-drain the transfer on unpause + CI: add build with ngtcp2 + gnutls on Travis CI + +- [Daiki Ueno brought this change] + + vquic: add support for GnuTLS backend of ngtcp2 - ... since the socket might not actually be readable anymore when for - example the data is already buffered in the TLS layer. + Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile + time. Therefore OpenSSL support needs to be explicitly disabled. - Fixes #4966 - Reported-by: Anders Berg - Closes #5000 + Signed-off-by: Daiki Ueno + Closes #5148 + +- [Gisle Vanem brought this change] -- TODO: curl --proxycommand + examples/sessioninfo.c: add include to fix compiler warning - Suggested-by: Kristian Mide - Closes #4941 + Fixes #5171 -- smtp: overwriting 'from' leaks memory - - Detected by Coverity. CID 1418139. +- misc: copyright year updates - Also, make sure to return error if the new 'from' allocation fails. + Follow-up to 7a71965e9 + +- [Harry Sintonen brought this change] + + build: fixed build for systems with select() in unistd.h - Closes #4997 + Closes #5169 -- CIfuzz: switch off 'dry_run' mode +- memdebug: don't log free(NULL) - Follow-up from #4960: now make it fail if it detects problems. + ... it serves no purpose and fills up the log. + +- cleanup: insert newline after if() conditions - Closes #4998 + Our code style mandates we put the conditional block on a separate + line. These mistakes are now detected by the updated checksrc. -Marc Hoersken (28 Feb 2020) -- ci/tests: Increase timeouts of Windows builds due to new tests +- checksrc: warn on obvious conditional blocks on the same line as if() - Recently added tests increased their runtime above the limit of 60min. + Closes #5164 -- ci/tests: align Azure Pipeline job names with each other +- [Roger Orr brought this change] -- ci/tests: Add Windows builds via Azure Pipelines using Docker + cmake: add CMAKE_MSVC_RUNTIME_LIBRARY + + Fixes #5165 + Closes #5167 -- tests: fix Python 3 compatibility of smbserver.py +- [Daiki Ueno brought this change] -Daniel Stenberg (27 Feb 2020) -- runtests: restore the command log + ngtcp2: update to git master for the key installation API change - The log file with all command lines for the invoked command lines is now - called logs/commands.log + This updates the ngtcp2 OpenSSL backend to follow the API change in + commit 32e703164 of ngtcp2. - Fixes #4911 - Closes #4989 + Notable changes are: + - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace + ngtcp2_crypto_derive_and_install_key + - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key + has been removed + + Fixes #5166 + Closes #5168 -- smtp: fix memory leak on exit path +- [Cyrus brought this change] + + SECURITY.md: minor rephrase - Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' - going out of scope leaks the storage it points to" + Closes #5158 + +- output.d: quote the URL when globbing - Closes #4990 + Some shells do globbing of their own unless the URL is quoted, so maybe + encourage this. + + Co-authored-by: Jay Satiro + Closes #5160 -Steve Holme (27 Feb 2020) -- gtls: Fixed compilation when using GnuTLS < 3.5.0 +- dist: add tests/version-scan.pl to tarball - Reverts the functionality from 41fcb4f when compiling with GnuTLS older - than 3.5.0. + ... used in test 1177. - Reviewed-by: Daniel Stenberg - Closes #4984 + Follow-up to a97d826f6de3 -- RELEASE-NOTES: Corrected the link to issue #4892 +- test1177: verify that all the CURL_VERSION_ bits are documented -Daniel Stenberg (27 Feb 2020) -- Curl_is_ASCII_name: handle a NULL argument +- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented - Make the function tolerate a NULL pointer input to avoid dereferencing - that pointer. + Considered experimental and therefore we can do this. - Follow-up to efce3ea5a85126d - Detected by OSS-Fuzz - Reviewed-By: Steve Holme - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 - Fixes #4985 - Closes #4986 - -- RELEASE-NOTES: synced + Closes #5157 -- http2: make pausing/unpausing set/clear local stream window +- KNOWN_BUGS: DoH doesn't inherit all transfer options - This reduces the HTTP/2 window size to 32 MB since libcurl might have to - buffer up to this amount of data in memory and yet we don't want it set - lower to potentially impact tranfer performance on high speed networks. + Closes #4578 + Closes #4579 + +- KNOWN_BUGS: DoH leaks memory after followlocation - Requires nghttp2 commit b3f85e2daa629 - (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end - up in the next release after 1.40.0. + Closes #4592 + +- KNOWN_BUGS: "FTPS needs session reuse" - Fixes #4939 - Closes #4940 + Closes #4654 -- [Anderson Toshiyuki Sasaki brought this change] +- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed - libssh: improve known hosts handling +- TODO: Set custom client ip when using haproxy protocol - Previously, it was not possible to get a known hosts file entry due to - the lack of an API. ssh_session_get_known_hosts_entry(), introduced in - libssh-0.9.0, allows libcurl to obtain such information and behave the - same as when compiled with libssh2. + Closes #5125 + +Michael Kaufmann (27 Mar 2020) +- writeout_json: Fix data type issues - This also tries to avoid the usage of deprecated functions when the - replacements are available. The behaviour will not change if versions - older than libssh-0.8.0 are used. + Load long values correctly (e.g. for http_code). - Signed-off-by: Anderson Toshiyuki Sasaki + Use curl_off_t (not long) for: + - size_download (CURLINFO_SIZE_DOWNLOAD_T) + - size_upload (CURLINFO_SIZE_UPLOAD_T) - Fixes #4953 - Closes #4962 + The unit for these values is bytes/second, not microseconds: + - speed_download (CURLINFO_SPEED_DOWNLOAD_T) + - speed_upload (CURLINFO_SPEED_UPLOAD_T) + + Fixes #5131 + Closes #5152 -Steve Holme (27 Feb 2020) -- tests: Automatically deduce the tool name from the test case for unit tests +Daniel Stenberg (27 Mar 2020) +- mailmap: fixup a few author names/fields - It is still possible to override the executable to run during the test, - using the tag, but this patch removes the requirement that the - tag must be present for unit tests. + Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and + Svyatoslav Mishyn + +- version: add 'cainfo' and 'capath' to version info struct - It also removes the possibility of human error when existing test cases - are used as the basis for new tests, as recently witnessed in 81c37124. + Suggested-by: Timothe Litt + URL: https://curl.haxx.se/mail/lib-2020-03/0090.html + Reviewed-by: Jay Satiro - Reviewed-by: Daniel Stenberg - Closes #4976 + Closes #5150 -- test1323: Added the missing 'unit test' feature requirement in the test case +- RELEASE-NOTES: synced -Daniel Stenberg (26 Feb 2020) -- cookie: remove unnecessary check for 'out != 0' +Jay Satiro (26 Mar 2020) +- SSLCERTS.md: Fix example code for setting CA cert file - ... as it will always be non-NULL at this point. + Prior to this change the documentation erroneously said use + CURLOPT_CAPATH to set a CA cert file. - Detected by Coverity: CID 1459009 + Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html + Reported-by: Timothe Litt + + Closes https://github.com/curl/curl/pull/5151 -- http: added 417 response treatment +Marc Hoersken (26 Mar 2020) +- sockfilt: add logmsg output to select_ws_wait_thread on Windows - When doing a request with a body + Expect: 100-continue and the server - responds with a 417, the same request will be retried immediately - without the Expect: header. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Added test 357 to verify. + Closes #5086 + +Daniel Stenberg (26 Mar 2020) +- docs/make: generate curl.1 from listed files only - Also added a control instruction to tell the sws test server to not read - the request body if Expect: is present, which the new test 357 uses. + Previously it rendered the page from files matching "*.d" in the correct + directory, which worked fine in git builds when the files were added but + made it easy to forget adding the files to the dist. - Reported-by: bramus on github - Fixes #4949 - Closes #4964 - -Steve Holme (26 Feb 2020) -- smtp: Tidy up, following recent changes, to maintain the coding style + Now, only man page sections listed in DPAGES in Makefile.inc will be + used, thus "forcing" us to update this to get the man page right and get + it included in the dist at the same time. - Closes #4892 + Ref: #5146 + Closes #5149 -- smtp: Support the SMTPUTF8 extension for the EXPN command +- openssl: adapt to functions marked as deprecated since version 3 - Simply notify the server we support the SMTPUTF8 extension if it does. - -- smtp: Support the SMTPUTF8 extension in the VRFY command - -- smtp: Support the SMTPUTF8 extension in the RCPT TO command + OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES + functions we use. - Note: The RCPT TO command isn't required to advertise to the server that - it contains UTF-8 characters, instead the server is told that a mail may - contain UTF-8 in any envelope command via the MAIL command. - -- smtp: Support the SMTPUTF8 extension in the MAIL command + Fix the MD4 and SSL_CTX_load_verify_locations warnings. - Support the SMTPUTF8 extension when sending mailbox information in the - MAIL command (FROM and AUTH parameters). Non-ASCII domain names will - be ACE encoded, if IDN is supported, whilst non-ASCII characters in - the local address part are passed to the server. + In configure, detect OpenSSL v3 and if so, inhibit the deprecation + warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and + until we rewrite the code to use non-deprecated functions we better + ignore these warnings as they don't help us. - Reported-by: ygthien on github - Fixes #4828 - -- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 - -- smtp: Support UTF-8 based host names in the VRFY command - -- smtp: Support UTF-8 based host names in the RCPT TO command + Closes #5139 -- smtp: Support UTF-8 based host names in the MAIL command +- dist: add mail-rcpt-allowfails.d to the tarball - Non-ASCII host names will be ACE encoded if IDN is supported. - -- url: Make the IDN conversion functions available to others + Reported-by: Maksim Stsepanenka + Reviewed-by: Jat Satiro + + Closes #5146 -- smtp: Added UTF-8 mailbox tests to verify existing behaviour +- travis: update the ngtcp2 build to use the latest OpenSSL patch + + ... which also makes it OpenSSL 1.1.1d based and not v3. -- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case +Marc Hoersken (24 Mar 2020) +- CI: remove default Ubuntu build from GitHub Actions + + We are already running a very similar Ubuntu build on Travis CI. + The macOS variant of this default build is kept on Github Actions. -- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() +- CI: bring GitHub Actions fuzzing job in line with macOS jobs - The dot character between the host and the tld was not being escaped, - which meant it specified a match of 'any' character rather than an - explicit dot separator. + Update YAML formatting, job naming and triggers. + +- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions - Additionally removed the dot character from the host name as it allowed - the following to be specified as a valid address in our test cases: + Reduce workload on Azure Pipelines and Travis CI while + consolidating macOS jobs onto less utilized GitHub Actions. - + Reviewed-by: Daniel Stenberg - Both are typos from 98f7ca7 and 8880f84 :( + Closes #5124 + +Daniel Stenberg (24 Mar 2020) +- config: remove all defines of HAVE_DES_H - I can't remember whether my intention was to allow sub-domains to be - specified in the host or not with these additional dots, but by placing - it outside of the host means it can only be specified once per domain - and by placing a + after the new grouping support for sub-domains is - kept. + As there's no code using it. - Closes #4912 + Closes #5144 -- hmac: Added a unit test for the HMAC hash generation +- copyright: fix out-of-date copyright ranges and missing headers - Closes #4973 - -- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function - -- tests: Added a unit test for MD4 digest generation + Reported by the new script 'scripts/copyright.pl'. The script has a + regex whitelist for the files that don't need copyright headers. - Closes #4970 - -- md4: Use const for the length input parameter + Removed three (mostly usesless) README files from docs/ - This keeps the interface the same as md5 and sha256. + Closes #5141 -- test1610: Fixed the link to the unit test +- packages: add OS400/chkstrings.c to the dist - Typo from 81c37124. + Reported-by: Jon Rumsey + Fixes #5142 + Closes #5143 -- ntlm: Removed the dependency on the TLS libaries when using MD5 +- [Clément Notin brought this change] + + nghttp2: 1.12.0 required - As we have our own MD5 implementation use the MD5 wrapper to remove the - TLS dependency. + since nghttp2_session_set_local_window_size is needed - Closes #4967 + Closes #5140 -- md5/sha256: Updated the functions to allow non-string data to be hashed +- RELEASE-NOTES: synced -- digest: Corrected the name of the local HTTP digest function - - Follow up to 2b5b37cb. Local static functions do not require the Curl - prefix. +- [Calvin Buckley brought this change] -- tests: Added a unit test for SHA256 digest generation + OS400: Update strings for ccsid-ifier - Follow up to 2b5b37c. + Fixes build. - Closes #4968 + Closes #5132 -- md4: Fixed compilation issues when using GNU TLS gcrypt - - * Don't include 'struct' in the gcrypt MD4_CTX typedef - * The call to gcry_md_read() should use a dereferenced ctx - * The call to gcry_md_close() should use a dereferenced ctx +- cirrus: make freebsd ignore the tests instead of skipping - Additional minor whitespace issue in the USE_WIN32_CRYPTO code. + To allow us to see in the CI logs how they actually behave - Closes #4959 + Closes #5091 -Daniel Stenberg (21 Feb 2020) -- RELEASE-NOTES: synced +- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 -- http2: now require nghttp2 >= 1.12.0 +- Revert "cirrus-ci: disable the FreeBSD 13 builds" - To simplify our code and since earlier versions lack important function - calls libcurl needs to function correctly. + This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. + +- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override - nghttp2 1.12.0 was relased on June 26, 2016. + To let debug-builds return fake values, like in test 970. - Closes #4961 + Ref: #5131 + Closes #5136 -- gtls: fix the copyright year +- test970: improve the test - Follow-up from 41fcb4f609 - -- [jethrogb brought this change] + - send more data to make problems more obvious + - don't start the data with minus, it makes diffs harder to read + - skip the headers in the stdout comparison + - save to a file name to also verify 'filename_effective' + + Ref: #5131 - GnuTLS: Always send client cert +- CURLINFO_NUM_CONNECTS: improve accuracy - TLS servers may request a certificate from the client. This request - includes a list of 0 or more acceptable issuer DNs. The client may use - this list to determine which certificate to send. GnuTLS's default - behavior is to not send a client certificate if there is no - match. However, OpenSSL's default behavior is to send the configured - certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL - behavior. + The counter was not bumped in all cases correctly. - Authored-by: jethrogb on github - Fixes #1411 - Closes #4958 + Reported-by: Marcel Raad + Ref: #5131 + Closes #5135 -- [Leo Neat brought this change] +- TODO: Use "random" ports for the test servers - github action: add CIFuzz +- lib/curl_setup: adjust the copyright year range - Closes #4960 + Follow-up from d820224b8 -- cleanup: comment typos +Jay Satiro (21 Mar 2020) +- curl_setup: define _WIN32_WINNT_[OS] symbols - Spotted by 'codespell' + .. because not all Windows build systems have those symbols, and even + those that do may be missing newer symbols (eg the Windows 7 SDK does + not define _WIN32_WINNT_WIN10). - Closes #4957 + Those symbols are used in build-time logic to decide which API to use + and prior to this change if the symbols were missing it would have + resulted in deprecated API being used when more recent functions were + available (eg GetVersionEx used instead of VerifyVersionInfo). + + Reported-by: FuccDucc@users.noreply.github.com + + Probably fixes https://github.com/curl/curl/issues/4995 + Closes https://github.com/curl/curl/pull/5057 -Steve Holme (20 Feb 2020) -- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions +- [Ross Burton brought this change] + + curl-functions.m4: remove inappropriate AC_REQUIRE - Whilst lib\md4.c used this pre-processor, lib\md5.c and - src\tool_metalink.c did not and simply relied on the WIN32 - pre-processor directive. + AC_REQUIRE means "if this macro hasn't been executed already, execute + it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) + isn't correct at that will execute AC_RUN_IFELSE without any arguments. - Reviewed-by: Marcel Raad - Closes #4955 + With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, + AC_RUN_IFELSE without a default value when cross-compiling is fatal. + The result is that curl with autoconf 2.70 cannot cross-compile. + + Fixes https://github.com/curl/curl/issues/5126 + Closes https://github.com/curl/curl/pull/5130 -Daniel Stenberg (19 Feb 2020) -- connect: remove some spurious infof() calls +Marc Hoersken (20 Mar 2020) +- ci/tests: fix Azure Pipelines not running Windows containers - As they were added primarily for debugging, they provide little use for - users. + Workaround posted here: microsoft/azure-pipelines-agent#2864 - Closes #4951 + Assisted-by: Simon Chalifoux + Assisted-by: Tommy Petty + + Fixes #5117 + Closes #5129 -- HTTP-COOKIES: mention that a trailing newline is required +Daniel Stenberg (20 Mar 2020) +- tests: add test 430, 431 and 432 to verify the --config fix - ... so that we know we got the whole and not a partial line. + Verify the fixes in 4e0b4fee4 + +- [Rici Lake brought this change] + + cmdline: fix handling of OperationConfig linked list (--next) - Also, changed the formatting of the fields away from a table again since - the table format requires a github-markdown tool version that we don't - run on the web server atm. + Ensures that -K/--config inserts new items at the end of the list + instead of overwriting the second item, and that after a -K/--config + option has been parsed, the option parser's view of the current config + is update. - Reported-by: Sunny Bean - Fixes #4946 - Closes #4947 + Fixes #5120 + Closes #5123 -- nit: Copyright year out of date +Marc Hoersken (20 Mar 2020) +- test2100: fix static port instead of dynamic value being used + +- test970: fix static ip:port instead of dynamic values being used + +Daniel Stenberg (19 Mar 2020) +- secure transport: remove the BACKEND define kludge - Follow-up to 1fc0617dcc + Closes #5122 -Jay Satiro (18 Feb 2020) -- tool_util: Improve Windows version of tvnow() +- mbedtls: remove the BACKEND define kludge + +- bearssl: remove the BACKEND define kludge + +- wolfssl: remove the BACKEND define kludge + +- nss: remove the BACKEND define kludge + +- gnutls: remove the BACKEND define kludge + +- openssl: remove the BACKEND define kludge - - Change tool_util.c tvnow() for Windows to match more closely to - timeval.c Curl_now(). + Use a proper variable instead to make it easier to use a debugger and + read the code. + +Marc Hoersken (19 Mar 2020) +- tests: make Python-based servers compatible with Python 2 and 3 - - Create a win32 init function for the tool, since some initialization - is required for the tvnow() changes. + Update smbserver.py and negtelnetserver.py to be compatible with + Python 3 while staying backwards-compatible to support Python 2. - Prior to this change the monotonic time function used by curl in Windows - was determined at build-time and not runtime. That was a problem because - when curl was built targeted for compatibility with old versions of - Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps - every 49.7 days that Windows has been running. + Fix string encoding and handling of echoed and transferred data. - This change makes curl behave similar to libcurl's tvnow function, which - determines at runtime whether the OS is Vista+ and if so calls - QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used - because it has higher resolution than the more obvious candidate - GetTickCount64). The changes to tvnow are basically a copy and paste but - the types in some cases are different. + Tested with both Python 2.7.17 and Python 3.7.7 + + Reported-by: Daniel Stenberg + Assisted-by: Kamil Dudka + Reviewed-by: Marcel Raad + + Fixes #5104 + Closes #5110 + +Daniel Stenberg (18 Mar 2020) +- writeout_json: use curl_off_t printf() option for the time output - Ref: https://github.com/curl/curl/issues/3309 + Follow-up to: 04c03416e68fd635a15 - Closes https://github.com/curl/curl/pull/4847 + Closes #5115 -Daniel Stenberg (18 Feb 2020) -- SOCKS: fix typo in printf formatting +- RELEASE-NOTES: synced - Follow-up to 4a4b63daa + Uh, I missed this in 1a46b218db + +- RELEASE-NOTES: synced - Reported-by: Peter Piekarski - Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 + ... and bumped curlver.h to 7.70.0 -- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section +Jay Satiro (18 Mar 2020) +- http2: Fix erroneous debug message that h2 connection closed - to be in sync with the description above + Prior to this change in libcurl debug builds http2 stream closure was + erroneously referred to as connection closure. - Reported-by: Joonas Kuorilehto - Fixes #4943 - Closes #4945 - -- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" - -- altsvc: make saving the cache an atomic operation + Before: + * nread <= 0, server closed connection, bailing - ... by writing the file to temp name then rename to the final when done. + After: + * nread == 0, stream closed, bailing - Assisted-by: Jay Satiro - Fixes #4936 - Closes #4942 + Closes https://github.com/curl/curl/pull/5118 -- rename: a new file for Curl_rename() +Daniel Stenberg (18 Mar 2020) +- tool_setopt: correct the copyright year range - And make the cookie save function use it. + Follow-up to 5450428491 -- cookies: make saving atomic with a rename +Jay Satiro (18 Mar 2020) +- [Johannes Schindelin brought this change] + + schannel: add "best effort" revocation check option - Saves the file as "[filename].[8 random hex digits].tmp" and renames - away the extension when done. + - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and + --ssl-revoke-best-effort to allow a "best effort" revocation check. - Co-authored-by: Jay Satiro - Reported-by: Mike Frysinger - Fixes #4914 - Closes #4926 - -- RELEASE-NOTES: synced - -- socks: make the connect phase non-blocking + A best effort revocation check ignores errors that the revocation check + was unable to take place. The reasoning is described in detail below and + discussed further in the PR. - Removes two entries from KNOWN_BUGS. + --- - Closes #4907 - -- multi: if Curl_readwrite sets 'comeback' use expire, not loop + When running e.g. with Fiddler, the schannel backend fails with an + unhelpful error message: - Otherwise, a very fast single transfer ricks starving out other - concurrent transfers. + Unknown error (0x80092012) - The revocation function was unable + to check revocation for the certificate. - Closes #4927 - -- ftp: convert 'sock_accepted' to a plain boolean + Sadly, many enterprise users who are stuck behind MITM proxies suffer + the very same problem. - This was an array indexed with sockindex but it was only ever used for - the secondary socket. + This has been discussed in plenty of issues: + https://github.com/curl/curl/issues/3727, + https://github.com/curl/curl/issues/264, for example. - Closes #4929 - -Jay Satiro (15 Feb 2020) -- CURLINFO_COOKIELIST.3: Fix example + In the latter, a Microsoft Edge developer even made the case that the + common behavior is to ignore issues when a certificate has no recorded + distribution point for revocation lists, or when the server is offline. + This is also known as "best effort" strategy and addresses the Fiddler + issue. - Prior to this change the example would try to import cookies from stdin, - which wasn't what was intended. + Unfortunately, this strategy was not chosen as the default for schannel + (and is therefore a backend-specific behavior: OpenSSL seems to happily + ignore the offline servers and missing distribution points). - Reported-by: 3dyd@users.noreply.github.com + To maintain backward-compatibility, we therefore add a new flag + (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option + (`--ssl-revoke-best-effort`) to select the new behavior. - Fixes https://github.com/curl/curl/issues/4930 - -Daniel Stenberg (14 Feb 2020) -- TODO: Paged searches on LDAP server + Due to the many related issues Git for Windows and GitHub Desktop, the + plan is to make this behavior the default in these software packages. - Closes #4452 - -- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS + The test 2070 was added to verify this behavior, adapted from 310. - Closes #4108 + Based-on-work-by: georgeok + Co-authored-by: Markus Olsson + Signed-off-by: Johannes Schindelin + + Closes https://github.com/curl/curl/pull/4981 -- azure: disable brotli on the macos debug-builds +- multi: Improve parameter check for curl_multi_remove_handle - Because of: + - If an easy handle is owned by a multi different from the one specified + then return CURLM_BAD_EASY_HANDLE. - brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] - const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], + Prior to this change I assume user error could cause corruption. - Closes #4925 + Closes https://github.com/curl/curl/pull/5116 -Steve Holme (13 Feb 2020) -- tool_home: Fix the copyright year being out of date +Viktor Szakats (17 Mar 2020) +- windows: suppress UI in all CryptAcquireContext() calls - Follow up to 9dc350b6. + Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters + Reviewed-by: Marc Hörsken + Closes https://github.com/curl/curl/pull/5088 -Jay Satiro (12 Feb 2020) -- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() - - - Deduplicate GetEnv() code. - - - On Windows change ultimate call to use Windows API - GetEnvironmentVariable() instead of C runtime getenv(). +Daniel Stenberg (17 Mar 2020) +- writeout_json: add missing comma to fix the HTTP version - Prior to this change both libcurl and the tool had their own GetEnv - which over time diverged. Now the tool's GetEnv is a wrapper around - curl_getenv (libcurl API function which is itself a wrapper around - libcurl's GetEnv). + Follow-up to 04c03416e68fd635a15 + +- test 970: verify --write-out '%{json}' - Furthermore this change fixes a bug in that Windows API - GetEnvironmentVariable() is called instead of C runtime getenv() to get - the environment variable since some changes aren't always visible to the - latter. + Makes curl_easy_getinfo() of "variable" numerical content instead return + the number set in the env variable `CURL_TIME`. - Reported-by: Christoph M. Becker + Makes curl_version() of "variable" textual content. This guarantees a + stable version string which can be tested against. Environment variable + `CURL_VERSION` defines the content. - Fixes https://github.com/curl/curl/issues/4774 - Closes https://github.com/curl/curl/pull/4863 + Assisted-by: Mathias Gumz -Daniel Stenberg (12 Feb 2020) -- strerror.h: Copyright year out of date - - Follow-up to 1c4fa67e8a8fcf6 +- [Mathias Gumz brought this change] -Jay Satiro (12 Feb 2020) -- strerror: Increase STRERROR_LEN 128 -> 256 + writeout: support to generate JSON output - STRERROR_LEN is the constant used throughout the library to set the size - of the buffer on the stack that the curl strerror functions write to. + This commit adds support to generate JSON via the writeout feature: - Prior to this change some extended length Windows error messages could - be truncated. + -w "%{json}" - Closes https://github.com/curl/curl/pull/4920 - -- multi: fix outdated comment + It leverages the existing infrastructure as much as possible. Thus, + generating the JSON on STDERR is possible by: - - Do not say that conn->data is "cleared" by multi_done(). + -w "%{stderr}%{json}" - If the connection is in use then multi_done assigns another easy handle - still using the connection to conn->data, therefore in that case it is - not cleared. + This implements a variant of + https://github.com/curl/curl/wiki/JSON#--write-out-json. - Closes https://github.com/curl/curl/pull/4901 + Closes #4870 -- easy: remove dead code +- CI: stop ignoring 323, it is disabled + +- DISABLED: disable test 323 - multi is already assigned to data->multi by curl_multi_add_handle. + The test uses SRP to "a server not supporting it" but modern stunnel + versions will silently accept it and remain happy. The test is therefore + faulty. - Closes https://github.com/curl/curl/pull/4900 - -Daniel Stenberg (12 Feb 2020) -- create-dirs.d: mention the mode + I haven't figured out how to make stunnel explicitly reject SRP-using + connects. - Reported-by: Dan Jacobson - Fixes #4766 - Closes #4916 + Reported-by: Marc Hörsken + Fixes #5105 + Closes #5113 -- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording +Marc Hoersken (17 Mar 2020) +- ci/tests: increase timeouts for torture builds on Azure Pipelines - Assisted-by: Jay Satiro - Reported-by: Craig Andrews - Fixes #4909 - Closes #4910 - -- RELEASE-NOTES: synced + For some reason the torture builds have slowed down recently. + + Reported-by: Daniel Stenberg -Steve Holme (9 Feb 2020) -- smtp: Simplify the MAIL command and avoid a duplication of send strings +Daniel Stenberg (16 Mar 2020) +- cmake: add support for building with wolfSSL - This avoids the duplication of strings when the optional AUTH and SIZE - parameters are required. It also assists with the modifications that - are part of #4892. + My working build cmdline: - Closes #4903 - -Daniel Stenberg (9 Feb 2020) -- altsvc: keep a copy of the file name to survive handle reset + $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . - The alt-svc cache survives a call to curl_easy_reset fine, but the file - name to use for saving the cache was cleared. Now the alt-svc cache has - a copy of the file name to survive handle resets. + Assisted-by: Brad King + Closes #5095 + +- tool_operate: fix add_parallel_transfers when more are in queue - Added test 1908 to verify. + Trying to return early from the function if no new transfers were added + would break the "morep" argument and cause issues. This could lead to + zero content "transfers" (within quotes since they would never be + started) when parallel-max was reduced. - Reported-by: Craig Andrews - Fixes #4898 - Closes #4902 + Reported-by: Gavin Wong + Analyzed-by: Jay Satiro + Fixes #4937 + Closes #5112 -Steve Holme (9 Feb 2020) -- url: Include the failure reason when curl_win32_idn_to_ascii() fails +- vtls: free ssl_config leftovers on out-of-memory - Provide the failure reason in the failf() info just as we do for the - libidn2 version of code. + Torture testing 2034 and 2037 found this. - Closes #4899 - -Jay Satiro (9 Feb 2020) -- asyn-thread: remove dead code - -Daniel Stenberg (8 Feb 2020) -- [Emil Engler brought this change] + Reported-by: Marc Hörsken + Fixes #5108 + Closes #5109 - github: Instructions to post "uname -a" on Unix systems in issues +Marc Hoersken (16 Mar 2020) +- ci/tests: fix Azure Pipelines not running for pull requests - Closes #4896 - -- [Cristian Greco brought this change] + Closes #5111 - configure.ac: fix comments about --with-quiche +Daniel Stenberg (15 Mar 2020) +- gskit: update the copyright year range - A simple s/nghttp3/quiche in some comments of --with-quiche. - Looks like a copy-paste error from --with-nghttp3. + Follow-up from 083603c63a3 + +Marc Hoersken (15 Mar 2020) +- gskit: use our internal select wrapper for portability - Closes #4897 + Follow up to c52b342 + Closes #5106 -Steve Holme (7 Feb 2020) -- checksrc.bat: Fix not being able to run script from the main curl directory +- tests: fix verification of stdout in test 1452 due to newline - If the script was ran from the main curl directory rather then the - projects directory then the script would simply exit without error: + Fixes test1452:41:1: error: missing tag before + +- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI - C:\url> projects\checksrc.bat + Also force the package index/cache to be updated before installing. - The user would either need to change to the projects directory, - explicitly specify the current working directory, or perform a - oneline hacky workaround: + Closes #5103 + +- tests/README: add note about manually installing python-impacket - C:\url> cd projects - C:\url\projects> checksrc.bat + Follow up to 4be2560 + +Daniel Stenberg (15 Mar 2020) +- transfer: cap retries of "dead connections" to 5 - C:\url> checksrc.bat %cd% + When libcurl retries a connection due to it being "seemingly dead" or by + REFUSED_STREAM, it will now only do it up five times before giving up, + to avoid never-ending loops. - C:\url> pushd projects & checksrc.bat & popd + Reported-by: Dima Tisnek + Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html + Closes #5074 + +- TODO: TLS-PSK with OpenSSL - Closes #4894 + Closes #5081 -Daniel Stenberg (7 Feb 2020) -- [Pierre-Yves Bigourdan brought this change] +Marc Hoersken (15 Mar 2020) +- select: add 'timeout_ms' wrap-around precaution to Curl_select - digest: Do not quote algorithm in HTTP authorisation - - RFC 7616 section 3.4 (The Authorization Header Field) states that "For - historical reasons, a sender MUST NOT generate the quoted string syntax - for the following parameters: algorithm, qop, and nc". This removes the - quoting for the algorithm parameter. +- select: fix 'pending_ms' is assigned a value that is never used - Reviewed-by: Steve Holme - Closes #4890 + Detected by Codacy -- ftp: remove the duplicated user/password struct fields +- select: move duplicate select preparation code into Curl_select - Closes #4887 + Reviewed by Daniel Stenberg + Reviewed by Marcel Raad + Closes #5078 -- ftp: remove superfluous checking for crlf in user or pwd +Daniel Stenberg (15 Mar 2020) +- connect: happy eyeballs cleanup - ... as this is already done much earlier in the URL parser. + Make sure each separate index in connn->tempaddr[] is used for a fixed + family (and only that family) during the connection process. - Also add test case 894 that verifies that pop3 with an encodedd CR in - the user name is rejected. + If family one takes a long time and family two fails immediately, the + previous logic could misbehave and retry the same family two address + repeatedly. - Closes #4887 + Reported-by: Paul Vixie + Reported-by: Jay Satiro + Fixes #5083 + Fixes #4954 + Closes #5089 -Steve Holme (6 Feb 2020) -- ntlm_wb: Use Curl_socketpair() for greater portability - - Reported-by: Daniel Stenberg - Closes #4886 +Marc Hoersken (15 Mar 2020) +- ci/tests: fix and align setting TFLAGS for make test-nonflaky -Daniel Stenberg (5 Feb 2020) -- [Frank Gevaerts brought this change] +- ci/tests: install test suite dependencies stunnel and impacket - contributors: Also include people who contributed to curl-www +- tests: remove python_dependencies for smbserver from our tree - Closes #4884 - -- [Frank Gevaerts brought this change] - - contrithanks: Use the most recent tag by default + Users of the SMB tests will have to install impacket manually. - (similar to 5296abe) + Reasoning: our in-tree version of impacket was quite outdated + and only compatible with Python 2 which is already end-of-life. + Upgrading to Python 3 and a compatible impacket version would + require to import additional Python-only and CPython-extension + dependencies. This would have hindered portability enormously. - Closes #4883 + Closes #5094 -- scripts: use last set tag if none given +Jay Satiro (14 Mar 2020) +- Makefile.m32: Improve windres parameter compatibility - Makes 'delta' and 'contributors.sh' easier to use. + - s/COFF/coff/ - Make the delta script invoke contrithanks to get current number of - contributors instead of counting THANKS, for accuracy. + Some versions of windres do not recognize uppercase COFF as a valid + way to specify the COFF output format. - Closes #4881 + Reported-by: Steven Penny + + Fixes https://github.com/curl/curl/issues/5099 + Closes https://github.com/curl/curl/pull/5101 -- ftp: shrink temp buffers used for PORT +- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares - These two stack based buffers only need to be 46 + 66 bytes instead of - 256 + 1024. + - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in + curl_easy_duphandle. - Closes #4880 - -- curl: error on --alt-svc use w/o support + Prior to this change if c-ares was used as the resolver backend and + either it was too old or libcurl was built without IPv6 support then + some of our resolver functions could return CURLE_NOT_BUILT_IN to + curl_easy_duphandle causing it to fail. - Make the tool check for alt-svc support at run-time and return error - accordingly if not present when the option is used. + Caused by c8f086b which shipped in 7.69.1. - Reported-by: Harry Sintonen - Closes #4878 - -- docs/HTTP3: add --enable-alt-svc to curl's configure + Reported-by: Karl Chen + + Fixes https://github.com/curl/curl/issues/5097 + Closes https://github.com/curl/curl/pull/5100 -- RELEASE-PROCEDURE: feature win is closed post-release a few days +Daniel Stenberg (13 Mar 2020) +- docs: add warnings about FILE: URLs on Windows - We've tried to uphold this already but let's make it official by - publicly stating this is the way we do it. + - --url man page section + - libcurl-security.3 gets the full text + - CURLOPT_URL.3 - Closes #4877 + Reported-by: Tim Sedlmeyer -- altsvc: set h3 version at a common single spot +- server/getpart: make the "XML-parser" stricter - ... and move the #ifdefs out of the functions. Addresses the fact they - were different before this change. + When extracting a
and there's no before +
, this now outputs an error and returns a wrong string to + make users spot the mistake. - Reported-by: Harry Sintonen - Closes #4876 - -- [Harry Sintonen brought this change] + Ref: #5070 + Closes #5071 - altsvc: improved header parser - - - Fixed the flag parsing to apply to specific alternative entry only, as - per RFC. The earlier code would also get totally confused by - multiprotocol header, parsing flags from the wrong part of the header. +Marc Hoersken (13 Mar 2020) +- impacket: some more Python 3 code compatibility updates - - Fixed the parser terminating on unknown protocols, instead of skipping - them. + This makes smbserver load on Python 3, but still not work completely. + +- smbserver: pin Python version to 2 since we are not yet 3 compatible - - Fixed a busyloop when protocol-id was present without an equal sign. + Even though the existing code can be fixed to run on Python 3, the + tests will fail due to the Unicode transition the protocol is invalid. - Closes #4875 - -- [Harry Sintonen brought this change] + Follow up to ee63837 + Closes #5085 - ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 +Daniel Stenberg (12 Mar 2020) +- [Viktor Szakats brought this change] -- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 + cleanup: fix some text/comment typos - Reported-by: James Fuller + Closes #5087 -Steve Holme (4 Feb 2020) -- ntlm: Pass the Curl_easy structure to the private winbind functions +Marc Hoersken (12 Mar 2020) +- smbserver: fix Python version specific ConfigParser import - ...rather than the full conndata structure. + Follow up to ee63837 and 8c7c4a6 + Fixes #5077 -Daniel Stenberg (4 Feb 2020) +Daniel Stenberg (11 Mar 2020) - RELEASE-NOTES: synced + + bumped to 7.69.2 -- tool_operhlp: Copyright year out of date, should be 2020 +Dan Fandrich (11 Mar 2020) +- tests/data: Fix some XML formatting issues in test cases - Follow-up from 2bc373740a3 + This allows these test files to pass xmllint. -- [Orgad Shaneh brought this change] +Daniel Stenberg (11 Mar 2020) +- [Muhammad Herdiansyah brought this change] - curl: avoid using strlen for testing if a string is empty + Makefile: run the cd commands in a subshell - Closes #4873 + In bmake, if the directory is changed (with cd or anything else), bmake + won't return to the "root directory" on the next command (in the same + Makefile rule). This commit runs the cd command in a subshell so it + would work in bmake. + + Closes #5073 -Steve Holme (3 Feb 2020) -- ntlm: Ensure the HTTP header data is not stored in the challenge/response +- configure: convert -I to -isystem as a last step + + As all the -I uses in CFLAGS at that point are for system headers and + third party libraries this helps us remove/ignore warnings on those! + + Closes #5060 -Marcel Raad (3 Feb 2020) -- openssl: remove redundant assignment +- configure: fix -pedantic-errors for GCC 5 and later - Fixes a scan-build failure on Bionic. + If --enable-werror is used. - Closes https://github.com/curl/curl/pull/4872 + Follow-up to d5c0351055d5709da which added it too early in the configure + script before $compiler_num was set correctly and thus this option was + never used. + + Reported-by: Stepan Efremov + Fixes #5067 + Closes #5068 -- travis: update non-OpenSSL Linux jobs to Bionic +- configure: document 'compiler_num' for gcc - For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is - failing with "curl returned 52, when expecting 35". + The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + + MINOR and ignores the patch version, and since gcc version 7 it only + sets it to MAJOR*100. - Closes https://github.com/curl/curl/pull/4872 + Reported-by: Stepan Efremov + Ref: #5067 + Closes #5069 -Dan Fandrich (3 Feb 2020) -- cirrus: Add some missing semicolons - - Newlines aren't preserved in this section so they're needed to separate - commands. The exports luckily worked anyway as a single long line, but - erroneously exported a variable called "export" - [skip ci] +Version 7.69.1 (11 Mar 2020) -Daniel Gustafsson (2 Feb 2020) -- [Pedro Monreal brought this change] +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: 7.69.1 - cleanup: fix typos and wording in docs and comments +- THANKS: from the 7.69.1 release + +- [Marc Hoersken brought this change] + + test1129: fix invalid case of closing XML-tag and Content-Length - Closes #4869 - Reviewed-by: Emil Engler and Daniel Gustafsson + Fixes #5070 + Closes #5072 -Steve Holme (2 Feb 2020) -- ntlm: Move the winbind data into the NTLM data structure +Marc Hoersken (10 Mar 2020) +- tests/data: fix static ip instead of dynamic value being used - To assist with adding winbind support to the SASL NTLM authentication, - move the winbind specific data out of conndata into ntlmdata. + Follow up to 94ced8e -Daniel Stenberg (30 Jan 2020) -- quiche: Copyright year out of date +- tests/data: fix static ip:port instead of dynamic values being used - Follow-up to 7fc63d72333a + Closes #5065 -- altsvc: use h3-25 +- tests/server: fix missing use of exe_ext helper function - Closes #4868 + Follow up to 9819984 and 3dce984 + Reviewed-By: Daniel Stenberg + Closes #5064 -- [Alessandro Ghedini brought this change] +- runtests: log minimal and maximal used port numbers - quiche: update to draft-25 - - Closes #4867 +Daniel Stenberg (9 Mar 2020) +- [James Fuller brought this change] -- ngtcp2: update to git master and its draft-25 support + sftp: fix segfault regression introduced by #4747 - Closes #4865 + This fix adds a defensive check for the case where the char *name in + struct libssh2_knownhost is NULL + + Fixes #5041 + Closes #5062 -- cookie: check __Secure- and __Host- case sensitively +- RELEASE-NOTES: synced + +- socks4: fix host resolve regression - While most keywords in cookies are case insensitive, these prefixes are - specified explicitly to get checked "with a case-sensitive match". + 1. The socks4 state machine was broken in the host resolving phase - (From the 6265bis document in progress) + 2. The code now insists on IPv4-only when using SOCKS4 as the protocol + only supports that. - Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 - Closes #4864 - -- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header - -- oauth2-bearer.d: works for HTTP too + Regression from #4907 and 4a4b63d, shipped in 7.69.0 - Reported-by: Mischa Salle - Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html - Closes #4862 + Reported-by: amishmm on github + Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 + Closes #5061 -- multi_done: if multiplexed, make conn->data point to another transfer +- [Patrick Monnerat brought this change] + + silly web server: silent a compilation warning - ... since the current transfer is being killed. Setting to NULL is - wrong, leaving it pointing to 'data' is wrong since that handle might be - about to get freed. + Recent gcc warns when byte count of strncpy() equals the destination + buffer size. Since the destination buffer is previously cleared and + the source string is always shorter, reducing the byte count by one + silents the warning without affecting the result. - Fixes #4845 - Closes #4858 - Reported-by: dmitrmax on github + Closes #5059 -- location.d: the method change is from POST to GET only - - Not from generic non-GET to GET. +- [Patrick Monnerat brought this change] + + cookie: get_top_domain() sets zero length for null domains - Reported-by: Andrius Merkys - Ref: #4859 - Closes #4861 + This silents a compilation warning with gcc -O3. -- urlapi: guess scheme correct even with credentials given +- [Patrick Monnerat brought this change] + + test 1560: avoid valgrind false positives - In the "scheme-less" parsing case, we need to strip off credentials - first before we guess scheme based on the host name! + When using maximum code optimization level (-O3), valgrind wrongly + detects uses of uninitialized values in strcmp(). - Assisted-by: Jay Satiro - Fixes #4856 - Closes #4857 + Preset buffers with all zeroes to avoid that. -- global_init: move the IPv6 works status bool to multi handle - - Previously it was stored in a global state which contributed to - curl_global_init's thread unsafety. This boolean is now instead figured - out in curl_multi_init() and stored in the multi handle. Less effective, - but thread safe. +Steve Holme (8 Mar 2020) +- sha256: Added WinCrypt implementation - Closes #4851 + Closed #5030 -- [Jay Satiro brought this change] +- sha256: Added SecureTransport implementation - README: mention that the docs is in docs/ +Daniel Stenberg (7 Mar 2020) +- lib1564: reduce number of mid-wait wakeup calls - Reported-by: Austin Green - Fixes #4830 - Closes #4853 - -- curl.h: define CURL_WIN32 on windows + This test does A LOT of *wakeup() calls and then calls curl_multi_poll() + twice. The first *poll() is then expected to return early and the second + not - as the first is supposed to drain the socketpair pipe. - ... so that the subsequent logic below can use a single known define to know - when built on Windows (as we don't define WIN32 anymore). + It turns out however that when given "excessive" amounts of writes to + the pipe, some operating systems (the Solaris based are known) will + return EAGAIN before the pipe is drained, which in our test case causes + the second *poll() call to also abort early. - Follow-up to 1adebe7886ddf20b + This change attempts to avoid the OS-specific behaviors in the test by + reducing the amount of wakeup calls from 1234567 to 10. - Reported-by: crazydef on github - Assisted-by: Marcel Raad - Fixes #4854 - Closes #4855 - -- RELEASE-NOTES: synced + Reported-by: Andy Fiddaman + Fixes #5037 + Closes #5058 -- [Jon Rumsey brought this change] +- [Patrick Monnerat brought this change] - urldata: do string enums without #ifdefs for build scripts + mime: fix the binary encoder to handle large data properly - ... and check for inconsistencies for OS400 at build time with the new - chkstrings tool. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes #4822 + Fixes #4860 + Closes #4833 + Reported-by: RuurdBeerstra on github -- curl: make the -# spaceship bar not wrap the line +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - The fixed-point math made us lose precision and thus a too high index - value could be used for outputting the hashtags which could overwrite - the newline. + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - The fix increases the precision in the sine table (*100) and the - associated position math. + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 667 checks mimepost using single-byte read callback with encoder. + New test 668 checks the end of part data early detection. - Reported-by: Andrew Potter - Fixes #4849 - Closes #4850 + Fixes #4826 + Reported-by: MrdUkk on github -- global_init: assume the EINTR bit by default - - - Removed from global_init since it isn't thread-safe. The symbol will - still remain to not break compiles, it just won't have any effect going - forward. +- [Patrick Monnerat brought this change] + + mime: latch last read callback status. - - make the internals NOT loop on EINTR (the opposite from previously). - It only risks returning from the select/poll/wait functions early, and that - should be risk-free. + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Closes #4840 + Fixes #4813 + Reported-by: MrdUkk on github -- [Peter Piekarski brought this change] +Marc Hoersken (7 Mar 2020) +- runtests: fix missing use of exe_ext helper function - conn: do not reuse connection if SOCKS proxy credentials differ - - Closes #4835 +Daniel Stenberg (7 Mar 2020) +- [Ernst Sjöstrand brought this change] -- llist: removed unused Curl_llist_move() + ares: store dns parameters for duphandle - (and the corresponding unit test) + With c-ares the dns parameters lives in ares_channel. Store them in the + curl handle and set them again in easy_duphandle. + + Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. - Closes #4842 - -- conncache: removed unused Curl_conncache_bundle_size() + Fixes #4893 + Closes #5020 + Signed-off-by: Ernst Sjöstrand -- strcase: turn Curl_raw_tolower into static +- version: make curl_version* thread-safe without using global context - Only ever used from within this file. + Closes #5010 -- singleuse.pl: support new API functions, fix curl_dbg_ handling +- RELEASE-NOTES: synced -- wolfssh: make it init properly via Curl_ssh_init() +Marc Hoersken (7 Mar 2020) +- tests: use native Sleep function as fallback on Windows - Closes #4846 + Reviewed-By: Daniel Stenberg + Closes #5054 -- [Aron Rotteveel brought this change] +- perl: align order and completeness of Windows OS checks - form.d: fix two minor typos +Daniel Stenberg (7 Mar 2020) +- tool_cb_see: set correct copyright year range - Closes #4843 + Follow-up to a39e5bfb9 -- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields +Marc Hoersken (7 Mar 2020) +- seek: fix fallback for missing ftruncate on Windows - Avoid "reparsing" the content and instead deliver more exactly what is - provided in the certificate and avoid truncating the data after 512 - bytes as done previously. This no longer removes embedded newlines. + This fixes test 198 on versions of MinGW-w64 without ftruncate - Fixes #4837 - Reported-by: bnfp on github - Closes #4841 + Reviewed-By: Daniel Stenberg + Reviewed-By: Marcel Raad + Closes #5055 -Jay Satiro (23 Jan 2020) -- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 - - - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. - - Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the - CURLOPT_PROXY_SSL_OPTIONS description. +- config-win32: Windows does not have ftruncate -Daniel Stenberg (22 Jan 2020) -- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER +Daniel Stenberg (7 Mar 2020) +- pause: force a connection (re-)check after unpausing - For now, no cert in the bundle actually sets a date there... + There might be data available that was already read off the socket, for + example in the TLS layer. - Co-Authored-by: Jay Satiro - Reported-by: Christian Heimes - Fixes #4834 - Closes #4836 - -- RELEASE-NOTES: synced + Reported-by: Anders Berg + Fixes #4966 + Closes #5049 -- [Pavel Volgarev brought this change] +- socks5: switch state properly when the resolve is done + + Regression from 4a4b63d (and #4907) + Reported-by: vitaha85 on github + Fixes #5053 + Closes #5056 - smtp: Allow RCPT TO command to fail for some recipients +Jay Satiro (7 Mar 2020) +- libssh: Fix matching user-specified MD5 hex key - Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. + Prior to this change a match would never be successful because it + was mistakenly coded to compare binary data from libssh to a + user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). - Verified with the new tests 3002-3007 + Reported-by: fds242@users.noreply.github.com - Closes #4816 + Fixes https://github.com/curl/curl/issues/4971 + Closes https://github.com/curl/curl/pull/4974 -- copyright: fix year ranges +Daniel Stenberg (6 Mar 2020) +- pause: bail out on bad input - follow-up from dea17b519d (one of these days I'll learn to check before - I push) - -- [nao brought this change] + A NULL easy handle or an easy handle without an associated connection + cannot be paused or unpaused. + + Closes #5050 - http: move "oauth_bearer" from connectdata to Curl_easy +Steve Holme (6 Mar 2020) +- unit1612: fixed the inclusion and compilation of the HMAC unit test - Fixes the bug where oauth_bearer gets deallocated when we re-use a - connection. + Follow up to 3f74e5e6 to fix: - Closes #4824 - -- [Emil Engler brought this change] + - A typo in Makefile.inc where unit1611 was used instead + - Some compilation issues in unit1612.c + + Closes #5024 - curl: Let -D merge headers in one file again +Daniel Stenberg (6 Mar 2020) +- pause: return early for calls that don't change pause state - Closes #4762 - Fixes #4753 + Reviewed-by: Patrick Monnerat + Ref: #4833 + Closes #5026 -- data.d: remove "Multiple files can also be specified" +Jay Satiro (6 Mar 2020) +- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - It is superfluous and could even be misleading. + Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused + CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. - Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html - Reported-by: Mike Norton - Closes #4832 + Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html + Reported-by: Felipe Gasper + + Closes https://github.com/curl/curl/pull/5048 -Marcel Raad (20 Jan 2020) -- CMake: support specifying the target Windows version +- multi: skip EINTR check on wakeup socket if it was closed - Previously, it was only possible to set it to Windows Vista or XP by - setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. - Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to - explicitly set the target Windows version. `ENABLE_INET_PTON` is - ignored in this case. + - Don't check errno on wakeup socket if sread returned 0 since sread + doesn't set errno in that case. - Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 - Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 - Closes https://github.com/curl/curl/pull/4815 - -Daniel Stenberg (20 Jan 2020) -- http.h: Copyright year out of date, should be 2020 + This is a follow-up to cf7760a from several days ago which fixed + Curl_multi_wait to stop busy looping sread on the non-blocking wakeup + socket if it was closed (ie sread returns 0). Due to a logic error it + was still possible to busy loop in that case if errno == EINTR. - Follow-up to 7ff9222ced8c + Closes https://github.com/curl/curl/pull/5047 -- [加藤郁之 brought this change] +Daniel Stenberg (6 Mar 2020) +- transfer: set correct copyright year range - HTTP: increase EXPECT_100_THRESHOLD to 1Mb +- urldata: remove the 'stream_was_rewound' connectdata struct member - Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html + ... as it is never set anywhere. - Closes #4814 + Follow-up to 2f44e94ef + Closes #5046 -- ROADMAP: thread-safe `curl_global_init()` +- Revert "pause: force-drain the transfer on unpause" - I'd like to see this happen. + This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) + + Clearly that didn't solve the problem correctly. + + Reported-by: Christopher Reid + Reopens #4966 + Fixes #5044 - RELEASE-NOTES: synced - -- wolfssl: use the wc-prefixed symbol alternatives - The symbols without wc_ prefix are not always provided. + and bumped curlver.h + +- MANUAL: update a dict-using command line - Ref: https://github.com/wolfSSL/wolfssl/issues/2744 + The 'web1913' database is now invalid, use 'gcide' instead. + +- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - Closes #4827 + Closes #3841 -- polarssl: removed +- polarssl: remove more references and mentions - As detailed in DEPRECATE.md, the polarssl support is now removed after - having been disabled for 6 months and nobody has missed it. + Assisted-by: Jay Satiro + Follow-up to 6357a19ff29dac04 + Closes #5036 + +Marc Hoersken (4 Mar 2020) +- tests: wrap ignored test failures in braces + +- tests: align some Windows sleep defines with each other + +- tests: try to make sleeping portable by avoiding select - The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix - instead of the former 'polarssl' and the common functions that - previously were shared between mbedtls and polarssl and contained the - name 'polarssl' have now all been renamed to instead say 'mbedtls'. + select does not support just waiting on Windows: + https://perldoc.perl.org/perlport.html#select - Closes #4825 + Reviewed-By: Daniel Stenberg + Closes #5035 -Marcel Raad (16 Jan 2020) -- libssh2: fix variable type +Daniel Stenberg (4 Mar 2020) +- runtests.1: rephrase how to specify what tests to run - This led to a conversion warning on 64-bit MinGW, which has 32-bit - `long` but 64-bit `size_t`. + Also mention the new tilde-prefixed way to ignore test results. - Closes https://github.com/curl/curl/pull/4823 + Reviewed-By: Marc Hoersken + Closes #5033 -Daniel Stenberg (16 Jan 2020) -- curl:progressbarinit: ignore column width from terminals < 20 +- cirrus-ci: disable the FreeBSD 13 builds - To avoid division by zero - or other issues. + FreeBSD 13.0 is apparently close to a year away from a stable release + and has proven to cause intermittent builds failures recently. - Reported-by: Daniel Marjamäki - Closes #4818 + Assisted-by: Dan Fandrich + Assisted-by: Fedor Korotkov + Fixes #5028 + Closes #5029 -- wolfssh: set the password correctly for PASSWORD auth +Version 7.69.0 (4 Mar 2020) -- wolfssh: remove fprintf() calls (and uses of __func__) +Daniel Stenberg (4 Mar 2020) +- RELEASE-NOTES: 7.69.0 -Marcel Raad (14 Jan 2020) -- CMake: use check_symbol_exists also for inet_pton - - It doesn't make much sense to only check if the function can be linked - when it's not declared in any header and that is treated as an error. - With the correct target Windows version set, the function is declared - in ws2tcpip.h and the comment above the modified block is invalid. - - Also, move the definition of `_WIN32_WINNT` up to before all symbol - availability checks so that we don't have to care which ones must be - done after it. +- THANKS: from 7.69.0 - Tested with Visual Studio 2019 and current MinGW-w64. + Now sorted case insensitive + +Marc Hoersken (3 Mar 2020) +- ci/tests: fix escaping of testnames and disable proxy for CI APIs - Closes https://github.com/curl/curl/pull/4808 + Follow up to ada581f and c0d8b96 + Closes #5031 -Jay Satiro (13 Jan 2020) -- schannel_verify: Fix alt names manual verify for UNICODE builds +Jay Satiro (3 Mar 2020) +- cmake: Show HTTPS-proxy in the features output + + - Show HTTPS-proxy in the features output for those backends that + support it: OpenSSL, GnuTLS and NSS. - Follow-up to 29e40a6 from two days ago, which added that feature for - Windows 7 and earlier. The bug only occurred in same. + Prior to this change HTTPS-proxy was missing from the cmake features + output even if curl was built with it. Only cmake output was affected. + Both the library and tool correctly reported the feature. - Ref: https://github.com/curl/curl/pull/4761 + Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html + Reported-by: David Lopes + + Closes https://github.com/curl/curl/pull/5025 -Daniel Stenberg (13 Jan 2020) -- HTTP-COOKIES.md: describe the cookie file format +Marc Hoersken (3 Mar 2020) +- ci/tests: Make it possible to still run but ignore failing tests - ... and refer to that file from from CURLOPT_COOKIEFILE.3 and - CURLOPT_COOKIELIST.3 + This enables the development of a solution for the failing tests by + running them on CI while ignoring their result for the overall status. - Assisted-by: Jay Satiro - Reported-by: bsammon on github - Fixes #4805 - Closes #4806 + Closes #4994 -- [Tobias Hieta brought this change] +- README.md: add Azure DevOps Pipelines build status badge - CMake: Add support for CMAKE_LTO option. +- ci/tests: Move CI test result creation above environment setup - This enables Link Time Optimization. LTO is a proven technique for - optimizing across compilation units. + This avoids using our test servers as proxy to the AppVeyor API. - Closes #4799 - -- RELEASE-NOTES: synced + Closes #5022 -- ConnectionExists: respect the max_concurrent_streams limits +- ci/tests: Send test results to AppVeyor for status overview - A regression made the code use 'multiplexed' as a boolean instead of the - counter it is intended to be. This made curl try to "over-populate" - connections with new streams. + Closes #5021 + +Daniel Stenberg (3 Mar 2020) +- Revert "sha256: Added SecureTransport implementation" - This regression came with 41fcdf71a1, shipped in curl 7.65.0. + This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) - Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same - check. + That commit broke test 1610 on macos builds without TLS. - Reported-by: Kunal Ekawde - Fixes #4779 - Closes #4784 + Closes #5027 -- curl: make #0 not output the full URL - - It was not intended nor documented! +- dist: include tests/azure.pm in the tarball - Added test 1176 to verify. + Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 + Reported-by: Marcel Raad + +Steve Holme (3 Mar 2020) +- configure.ac: Disable metalink if mbedTLS is specified - Reported-by: vshmuk on hackerone + Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as + being one of the backends that metalink needs to be disabled for, I + seem to have included it in the list of allowed SSL/TLS backends in + comnfigure.ac :( - Closes #4812 + Closes #5013 -- wolfSSH: new SSH backend +- sha256: Tidy up following recent changes - Adds support for SFTP (not SCP) using WolfSSH. + Reviewed-by: Daniel Stenberg + Closes #4956 + +- sha256: Added WinCrypt implementation + +- sha256: Added SecureTransport implementation + +- sha256: Added mbedtls implementation + +- sha256: Added GNU TLS gcrypt implementation + +- sha256: Added GNU TLS Nettle implementation + +Jay Satiro (2 Mar 2020) +- curl_escape.3: Add a link to curl_free - Closes #4231 + Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 -- curl: remove 'config' field from OutStruct +- curl_getenv.3: Fix the memory handling description - As it was just unnecessary duplicated information already stored in the - 'per_transfer' struct and that's around mostly anyway. + - Tell the user to call curl_free() to free the pointer returned by + curl_getenv(). - The duplicated pointer caused problems when the code flow was aborted - before the dupe was filled in and could cause a NULL pointer access. + Prior to this change the user was directed to call free(), but that + would not work in cases where the library and application use separate C + runtimes and therefore have separate heap memory management. - Reported-by: Brian Carpenter - Fixes #4807 - Closes #4810 + Closes https://github.com/curl/curl/pull/5016 -- misc: Copyright year out of date, should be 2020 +Daniel Stenberg (2 Mar 2020) +- [Nick Zitzmann brought this change] + + md4: use init/update/final functions in Secure Transport - Follow-up to recent commits + We can use CC_MD4_Init/Update/Final without having to allocate memory + directly. - [skip ci] + Closes #4979 -Jay Satiro (11 Jan 2020) -- [Santino Keupp brought this change] +Marc Hoersken (2 Mar 2020) +- ci/tests: some MacOS builds randomly take longer than 20min - libssh2: add support for forcing a hostkey type - - - Allow forcing the host's key type found in the known_hosts file. +Daniel Stenberg (2 Mar 2020) +- multi_wait: stop loop when sread() returns zero - Currently, curl (with libssh2) does not take keys from your known_hosts - file into account when talking to a server. With this patch the - known_hosts file will be searched for an entry matching the hostname - and, if found, libssh2 will be told to claim this key type from the - server. + It's unclear why it would ever return zero here, but this change fixes + Robert's problem and it shouldn't loop forever... - Closes https://github.com/curl/curl/pull/4747 - -- [Nicolas Guillier brought this change] + Reported-by: Robert Dunaj + Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html + Closes #5019 - cmake: Improve libssh2 check on Windows +- http: mark POSTs with no body as "upload done" from the start - - Add "libssh2" name to FindLibSSH2 library search. + As we have logic that checks if we get a >= 400 reponse code back before + the upload is done, which then got confused since it wasn't "done" but + yet there was no data to send! - On Windows systems, libSSH2 CMake installation may name the library - "LibSSH2". + Reported-by: IvanoG on github + Fixes #4996 + Closes #5002 + +- tests: disable 962, 963 and 964 on Windows - Prior to this change cmake only checked for name "ssh2". On Linux that - works fine because it will prepend the "lib", but it doesn't do that on - Windows. + These tests are also doing UTF-8 SMTP. - Closes https://github.com/curl/curl/pull/4804 + Follow-up to df207d2dd93b9e73 -- [Faizur Rahman brought this change] +Marc Hoersken (2 Mar 2020) +- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer - schannel: Make CURLOPT_CAINFO work better on Windows 7 - - - Support hostname verification via alternative names (SAN) in the - peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. - - CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a - result CertGetNameString doesn't quite work on those versions of - Windows. This change provides an alternative solution for - CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier - versions of Windows. - - Prior to this change many certificates failed the hostname validation - when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates - now represent multiple hostnames and rely on the alternative names field - exclusively to represent their hostnames. - - Reported-by: Jeroen Ooms - - Fixes https://github.com/curl/curl/issues/3711 - Closes https://github.com/curl/curl/pull/4761 +Daniel Stenberg (2 Mar 2020) +- configure: bump the AC_COPYRIGHT year range -- [Emil Engler brought this change] +- [Steve Holme brought this change] - ngtcp2: Add an error code for QUIC connection errors - - - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection - errors. + tests: disable SMTP UTF-8 tests on Windows - Prior to this change CURLE_FAILED_INIT was used, but that was not - correct. + Fixes #4988 + Closes #4992 + +- formdata/mime: copyright year range update - Closes https://github.com/curl/curl/pull/4754 + Due to the merge/revert cycle -- multi: Change curl_multi_wait/poll to error on negative timeout +- Revert "mime: latch last read callback status." - - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when - curl_multi_wait/poll is passed timeout param < 0. + This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. - Prior to this change passing a negative value to curl_multi_wait/poll - such as -1 could cause the function to wait forever. + Fixes #5014 + Closes #5015 + Reopens #4833 + +- Revert "mime: do not perform more than one read in a row" - Reported-by: hamstergene@users.noreply.github.com + This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. + +- Revert "mime: fix the binary encoder to handle large data properly" - Fixes https://github.com/curl/curl/issues/4763 + This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. + +- altsvc: both h3 backends now speak h3-27 - Closes https://github.com/curl/curl/pull/4765 + ... also updated the HTTP3 build description for ngtcp2 accordingly. -- [Marc Aldorasi brought this change] +- [Patrick Monnerat brought this change] - cmake: Enable SMB for Windows builds + mime: fix the binary encoder to handle large data properly - - Define USE_WIN32_CRYPTO by default. This enables SMB. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - - Show whether SMB is enabled in the "Enabled features" output. + Fixes #4860 + Reported-by: RuurdBeerstra on github + +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - - Fix mingw compiler warning for call to CryptHashData by casting away - const param. mingw CryptHashData prototype is wrong. + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Closes https://github.com/curl/curl/pull/4717 + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 664 checks mimepost using single-byte read callback with encoder. + New test 665 checks the end of part data early detection. + + Fixes #4826 + Reported-by: MrdUkk on github + +- [Patrick Monnerat brought this change] -- vtls: Refactor Curl_multissl_version to make the code clearer - - Reported-by: Johannes Schindelin + mime: latch last read callback status. - Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Closes https://github.com/curl/curl/pull/4803 + Fixes #4813 + Reported-by: MrdUkk on github + Closes #4833 -Daniel Stenberg (10 Jan 2020) -- fix: Copyright year out of date, should be 2020 +Steve Holme (1 Mar 2020) +- unit1651: Fixed conversion compilation warning - Follow-up to 875314ed0bf3b - -Marcel Raad (10 Jan 2020) -- hostip: move code to resolve IP address literals to `Curl_resolv` + 371:17: warning: conversion to 'unsigned char' from 'int' may alter its + value [-Wconversion] - The code was duplicated in the various resolver backends. + Closes #5008 + +- configure.ac: Disable metalink support if an incompatible SSL/TLS specified - Also, it was called after the call to `Curl_ipvalid`, which matters in - case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This - caused test 1048 to fail on classic MinGW. + tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, + The Win32 Crypto library and Apple's Common Crypto library. - The code ignores `conn->ip_version` as done previously in the - individual resolver backends. + If an TLS backend such as mbedTLS or WolfSSL is specified then the + following error is given during compilation along, with a load of + unresolved extern errors: - Move the call to the `resolver_start` callback up to appease test 655, - which wants it to be called also for literal addresses. + Can't compile METALINK support without a crypto library. - Closes https://github.com/curl/curl/pull/4798 - -Daniel Stenberg (9 Jan 2020) -- scripts/delta: adapt to new public header layout + Reviewed-by: Daniel Stenberg + Closes #5006 -- test1167: verify global symbols in public headers are curl prefixed - - ... using the new badsymbols.pl perl script +Marc Hoersken (1 Mar 2020) +- ci/tests: Update Azure DevOps pipeline job display names - Fixes #4793 - Closes #4794 + Make the configure step more descriptive and align others. -- libtest/mk-lib1521: adapt to new public header layout +- ci/tests: Fix typo in previous commit 597cf2 -- include: remove non-curl prefixed defines +- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available - ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ - enums. + For security reasons the access token is not available to PR builds. + Therefore we should not try to use the DevOps API with an empty token. -- curl.h: remove WIN32 define +Daniel Stenberg (1 Mar 2020) +- build: remove all HAVE_OPENSSL_ENGINE_H defines - It isn't our job to define this in a public header - and it defines a - name outside of our naming scope. - -- tool_dirhie.c: fix the copyright year range + ... as there's nothing in the code that actually uses the define! The + last reference was removed in 38203f158. - Follow-up to: 4027bd72d9 + Closes #5007 -- bump: work towards 7.69.0 is started +Jay Satiro (29 Feb 2020) +- [Rolf Eike Beer brought this change] -Jay Satiro (9 Jan 2020) -- tool_dirhie: Allow directory traversal during creation - - - When creating a directory hierarchy do not error when mkdir fails due - to error EACCESS (13) "access denied". + CMake: clean up and improve build procedures - Some file systems allow for directory traversal; in this case that it - should be possible to create child directories when permission to the - parent directory is restricted. + - remove check for unsupported old CMake versions - This is a regression caused by me in f16bed0 (precedes curl-7_61_1). - Basically I had assumed that if a directory already existed it would - fail only with error EEXIST, and not error EACCES. The latter may - happen if the directory exists but has certain restricted permissions. + - do not link to c-ares library twice - Reported-by: mbeifuss@users.noreply.github.com + - modernize custom Find modules - Fixes https://github.com/curl/curl/issues/4796 - Closes https://github.com/curl/curl/pull/4797 - -Daniel Stenberg (9 Jan 2020) -- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers + - FindLibSSH2: + - pass version to FPHSA to show it in the output + - use LIBSSH2_VERSION define to extract the version number in + one shot. This variable exists in the header for 10 years. + - remove unneeded code - Closes #4080 - -- docs/RELEASE-PROCEDURE.md: pushed some release dates + - FindNGHTTP2.cmake: + - drop needless FPHSA argument + - mark found variables as advanced - Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html - -- runtests: make random seed fixed for a month + - FindNSS.cmake: + - show version number - When using randomized features of runtests (-R and --shallow) it is - useful to have a fixed random seed to make sure for example extra - commits in a branch or a rebase won't change the seed that would make - repeated runs work differently. + - FindCARES.cmake: + - drop default paths + - use FPHSA instead of checking things by hand - As it is also useful to change seed sometimes, the default seed is now - determined based on the current month (and first line curl -V - output). When the month changes, so will the random seed. + - remove needless explict variable dereference - The specific seed is also shown in the standard test suite top header - and it can be set explictly with the new --seed=[num] option so that the - exact order of a previous run can be achieved. + - simplify count_true() - Closes #4734 - -- RELEASE-PROCEDURE.md: fix next release date (Feb 26) + - allow all policies up to version 3.16 to be set to NEW - [skip ci] - -Version 7.68.0 (8 Jan 2020) - -Daniel Stenberg (8 Jan 2020) -- RELEASE-NOTES: 7.68.0 - -- THANKS: updated with names from the 7.68.0 release - -- RELEASE-PROCEDURE: add four future release dates + - do not rerun check for -Wstrict-aliasing=3 every time - and remove four past release dates + In contrast to every other compiler flag this has a = in it, which CMake + can't have in a variable name. - [skip ci] - -Marcel Raad (6 Jan 2020) -- TrackMemory tests: always remove CR before LF + - only read the interesting strings from curlver.h - It was removed for output containing ' =' via `s/ =.*//`. With classic - MinGW, this made lines with `free()` end with CRLF, but lines with e.g. - `malloc()` end with only LF. The tests expect LF only. + Reviewed-by: Peter Wu - Closes https://github.com/curl/curl/pull/4788 + Closes https://github.com/curl/curl/pull/4975 -Daniel Stenberg (6 Jan 2020) -- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header +- runtests: fix output to command log - ... to the private multihhandle.h. It is not for public use and it - wasn't prefixed correctly anyway! + - Record only the command of the most recently ran test in the command + log. - Closes #4790 - -- file: fix copyright year range + This is a follow-up to 02988b7 from several weeks ago which fixed + writing to the command log, however it saved all commands for all tests + instead of just the most recently ran test as we would now expect. - Follow-up to 1b71bc532bd + Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 + Closes https://github.com/curl/curl/pull/5001 -- curl -w: handle a blank input file correctly - - Previously it would end up with an uninitialized memory buffer that - would lead to a crash or junk getting output. +Steve Holme (1 Mar 2020) +- polarssl: Additional removal - Added test 1271 to verify. + Follow up to 6357a19f. - Reported-by: Brian Carpenter - Closes #4786 + Reviewed-by: Daniel Stenberg + Closes #5004 -- file: on Windows, refuse paths that start with \\ +- [Jonathan Cardoso Machado brought this change] + + docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always - ... as that might cause an unexpected SMB connection to a given host - name. + Reviewed-by: Steve Holme + Closes #5005 + +- md5: Added implementation for mbedTLS - Reported-by: Fernando Muñoz - CVE-2019-15601 - Bug: https://curl.haxx.se/docs/CVE-2019-15601.html + Reviewed-by: Jay Satiro + Closes #4980 -Jay Satiro (6 Jan 2020) -- CURLOPT_READFUNCTION.3: fix fopen params in example +- md5: Use pointer notation for array parameters in GnuTLS implementation -- CURLOPT_READFUNCTION.3: fix variable name in example - - Reported-by: Paul Joyce +- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 - Fixes https://github.com/curl/curl/issues/4787 + Closes #4983 -Daniel Stenberg (5 Jan 2020) -- curl:getparameter return error for --http3 if libcurl doesn't support - - Closes #4785 +Marc Hoersken (29 Feb 2020) +- ci/tests: Send test results to Azure DevOps for reporting -- docs: mention CURL_MAX_INPUT_LENGTH restrictions +Daniel Stenberg (29 Feb 2020) +- pause: force-drain the transfer on unpause - ... for curl_easy_setopt() and curl_url_set(). + ... since the socket might not actually be readable anymore when for + example the data is already buffered in the TLS layer. - [skip ci] + Fixes #4966 + Reported-by: Anders Berg + Closes #5000 + +- TODO: curl --proxycommand - Closes #4783 + Suggested-by: Kristian Mide + Closes #4941 -- curl: properly free mimepost data +- smtp: overwriting 'from' leaks memory - ... as it could otherwise leak memory when a transfer failed. + Detected by Coverity. CID 1418139. - Added test 1293 to verify. + Also, make sure to return error if the new 'from' allocation fails. - Reported-by: Brian Carpenter - Fixes #4781 - Closes #4782 + Closes #4997 -- curl: cleanup multi handle on failure - - ... to fix memory leak in error path. +- CIfuzz: switch off 'dry_run' mode - Fixes #4772 - Closes #4780 - Reported-by: Brian Carpenter - -Marcel Raad (3 Jan 2020) -- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` + Follow-up from #4960: now make it fail if it detects problems. - Closes https://github.com/curl/curl/pull/4775 + Closes #4998 -Daniel Stenberg (3 Jan 2020) -- COPYING: it's 2020! +Marc Hoersken (28 Feb 2020) +- ci/tests: Increase timeouts of Windows builds due to new tests - [skip ci] + Recently added tests increased their runtime above the limit of 60min. -Jay Satiro (3 Jan 2020) -- [Marc Aldorasi brought this change] +- ci/tests: align Azure Pipeline job names with each other - tests: Fix bounce requests with truncated writes +- ci/tests: Add Windows builds via Azure Pipelines using Docker + +- tests: fix Python 3 compatibility of smbserver.py + +Daniel Stenberg (27 Feb 2020) +- runtests: restore the command log - Prior to this change the swsbounce check in service_connection could - fail because prevtestno and prevpartno were not set, which would cause - the wrong response data to be sent to some tests and cause them to fail. + The log file with all command lines for the invoked command lines is now + called logs/commands.log - Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 + Fixes #4911 + Closes #4989 -Marcel Raad (31 Dec 2019) -- tool: make a few char pointers point to const char instead +- smtp: fix memory leak on exit path - These are read-only. + Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' + going out of scope leaks the storage it points to" - Closes https://github.com/curl/curl/pull/4771 + Closes #4990 -Jay Satiro (31 Dec 2019) -- tests: Change NTLM tests to require SSL +Steve Holme (27 Feb 2020) +- gtls: Fixed compilation when using GnuTLS < 3.5.0 - Prior to this change tests that required NTLM feature did not require - SSL feature. + Reverts the functionality from 41fcb4f when compiling with GnuTLS older + than 3.5.0. - There are pending changes to cmake builds that will allow enabling NTLM - in non-SSL builds in Windows. In that case the NTLM auth strings created - are different from what is expected by the NTLM tests and they fail: + Reviewed-by: Daniel Stenberg + Closes #4984 + +- RELEASE-NOTES: Corrected the link to issue #4892 + +Daniel Stenberg (27 Feb 2020) +- Curl_is_ASCII_name: handle a NULL argument - "The issue with NTLM is that previous non-SSL builds would not enable - NTLM and so the NTLM tests would be skipped." + Make the function tolerate a NULL pointer input to avoid dereferencing + that pointer. - Assisted-by: marc-groundctl@users.noreply.github.com + Follow-up to efce3ea5a85126d + Detected by OSS-Fuzz + Reviewed-By: Steve Holme + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 + Fixes #4985 + Closes #4986 + +- RELEASE-NOTES: synced + +- http2: make pausing/unpausing set/clear local stream window - Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 + This reduces the HTTP/2 window size to 32 MB since libcurl might have to + buffer up to this amount of data in memory and yet we don't want it set + lower to potentially impact tranfer performance on high speed networks. - Closes https://github.com/curl/curl/pull/4768 + Requires nghttp2 commit b3f85e2daa629 + (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end + up in the next release after 1.40.0. + + Fixes #4939 + Closes #4940 -- [Michael Forney brought this change] +- [Anderson Toshiyuki Sasaki brought this change] - bearssl: Improve I/O handling + libssh: improve known hosts handling - Factor out common I/O loop as bearssl_run_until, which reads/writes TLS - records until the desired engine state is reached. This is now used for - the handshake, read, write, and close. + Previously, it was not possible to get a known hosts file entry due to + the lack of an API. ssh_session_get_known_hosts_entry(), introduced in + libssh-0.9.0, allows libcurl to obtain such information and behave the + same as when compiled with libssh2. - Match OpenSSL SSL_write behavior, and don't return the number of bytes - written until the corresponding records have been completely flushed - across the socket. This involves keeping track of the length of data - buffered into the TLS engine, and assumes that when CURLE_AGAIN is - returned, the write function will be called again with the same data - and length arguments. This is the same requirement of SSL_write. + This also tries to avoid the usage of deprecated functions when the + replacements are available. The behaviour will not change if versions + older than libssh-0.8.0 are used. - Handle TLS close notify as EOF when reading by returning 0. + Signed-off-by: Anderson Toshiyuki Sasaki - Closes https://github.com/curl/curl/pull/4748 + Fixes #4953 + Closes #4962 -- travis: Fix error detection +Steve Holme (27 Feb 2020) +- tests: Automatically deduce the tool name from the test case for unit tests - - Stop using inline shell scripts for before_script and script sections. + It is still possible to override the executable to run during the test, + using the tag, but this patch removes the requirement that the + tag must be present for unit tests. - Prior to this change Travis could ignore errors from commands in inline - scripts. I don't understand how or why it happens. This is a workaround. + It also removes the possibility of human error when existing test cases + are used as the basis for new tests, as recently witnessed in 81c37124. - Assisted-by: Simon Warta + Reviewed-by: Daniel Stenberg + Closes #4976 + +- test1323: Added the missing 'unit test' feature requirement in the test case + +Daniel Stenberg (26 Feb 2020) +- cookie: remove unnecessary check for 'out != 0' - Ref: https://github.com/travis-ci/travis-ci/issues/1066 + ... as it will always be non-NULL at this point. - Fixes https://github.com/curl/curl/issues/3730 - Closes https://github.com/curl/curl/pull/3755 + Detected by Coverity: CID 1459009 -- tool_operate: fix mem leak when failed config parse +- http: added 417 response treatment - Found by fuzzing the config file. + When doing a request with a body + Expect: 100-continue and the server + responds with a 417, the same request will be retried immediately + without the Expect: header. - Reported-by: Geeknik Labs + Added test 357 to verify. - Fixes https://github.com/curl/curl/issues/4767 - -- [Xiang Xiao brought this change] + Also added a control instruction to tell the sws test server to not read + the request body if Expect: is present, which the new test 357 uses. + + Reported-by: bramus on github + Fixes #4949 + Closes #4964 - lib: remove erroneous +x file permission on some c files +Steve Holme (26 Feb 2020) +- smtp: Tidy up, following recent changes, to maintain the coding style - Modified by commit eb9a604 accidentally. + Closes #4892 + +- smtp: Support the SMTPUTF8 extension for the EXPN command - Closes https://github.com/curl/curl/pull/4756 + Simply notify the server we support the SMTPUTF8 extension if it does. -- [Xiang Xiao brought this change] +- smtp: Support the SMTPUTF8 extension in the VRFY command - lib: fix warnings found when porting to NuttX - - - Undefine DEBUGASSERT in curl_setup_once.h in case it was already - defined as a system macro. +- smtp: Support the SMTPUTF8 extension in the RCPT TO command - - Don't compile write32_le in curl_endian unless - CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. + Note: The RCPT TO command isn't required to advertise to the server that + it contains UTF-8 characters, instead the server is told that a mail may + contain UTF-8 in any envelope command via the MAIL command. + +- smtp: Support the SMTPUTF8 extension in the MAIL command - - Include in socketpair.c. + Support the SMTPUTF8 extension when sending mailbox information in the + MAIL command (FROM and AUTH parameters). Non-ASCII domain names will + be ACE encoded, if IDN is supported, whilst non-ASCII characters in + the local address part are passed to the server. - Closes https://github.com/curl/curl/pull/4756 + Reported-by: ygthien on github + Fixes #4828 -- os400: Add missing CURLE error constants +- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 + +- smtp: Support UTF-8 based host names in the VRFY command + +- smtp: Support UTF-8 based host names in the RCPT TO command + +- smtp: Support UTF-8 based host names in the MAIL command - Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 - Reported-by: Emil Engler + Non-ASCII host names will be ACE encoded if IDN is supported. -- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 +- url: Make the IDN conversion functions available to others + +- smtp: Added UTF-8 mailbox tests to verify existing behaviour + +- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case + +- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() - For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback - is passed two `size_t` parameters which, when multiplied, designate the - number of bytes of data passed in. In practice, CURL always sets the - first parameter (`size`) to 1. + The dot character between the host and the tld was not being escaped, + which meant it specified a match of 'any' character rather than an + explicit dot separator. - This practice is also enshrined in documentation and cannot be changed - in future. The documentation states that the default callback is - `fwrite`, which means `fwrite` must be a suitable function for this - purpose. However, the documentation also states that the callback must - return the number of *bytes* it successfully handled, whereas ISO C - `fwrite` returns the number of items (each of size `size`) which it - wrote. The only way these numbers can be equal is if `size` is 1. + Additionally removed the dot character from the host name as it allowed + the following to be specified as a valid address in our test cases: - Since `size` is 1 and can never be changed in future anyway, document - that fact explicitly and let users rely on it. + - Reported-by: Frank Gevaerts - Commit-message-by: Christopher Head + Both are typos from 98f7ca7 and 8880f84 :( - Ref: https://github.com/curl/curl/pull/2787 + I can't remember whether my intention was to allow sub-domains to be + specified in the host or not with these additional dots, but by placing + it outside of the host means it can only be specified once per domain + and by placing a + after the new grouping support for sub-domains is + kept. - Fixes https://github.com/curl/curl/issues/4758 + Closes #4912 -- examples/postinmemory.c: Call curl_global_cleanup always - - Prior to this change curl_global_cleanup was not called if - curl_easy_init failed. +- hmac: Added a unit test for the HMAC hash generation - Reported-by: kouzhudong@users.noreply.github.com + Closes #4973 + +- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function + +- tests: Added a unit test for MD4 digest generation - Fixes https://github.com/curl/curl/issues/4751 + Closes #4970 -Daniel Stenberg (21 Dec 2019) -- url2file.c: fix copyright year +- md4: Use const for the length input parameter - Follow-up to 525787269599b5 + This keeps the interface the same as md5 and sha256. -- [Rickard Hallerbäck brought this change] +- test1610: Fixed the link to the unit test + + Typo from 81c37124. - examples/url2file.c: corrected a comment +- ntlm: Removed the dependency on the TLS libaries when using MD5 - The comment was confusing and suggested that setting CURLOPT_NOPROGRESS - to 0L would both enable and disable debug output at the same time, like - a Schrödinger's cat of CURLOPTs. + As we have our own MD5 implementation use the MD5 wrapper to remove the + TLS dependency. - Closes #4745 + Closes #4967 -- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 +- md5/sha256: Updated the functions to allow non-string data to be hashed -- RELEASE-NOTES: synced +- digest: Corrected the name of the local HTTP digest function + + Follow up to 2b5b37cb. Local static functions do not require the Curl + prefix. -Jay Satiro (20 Dec 2019) -- ngtcp2: Support the latest update key callback type +- tests: Added a unit test for SHA256 digest generation - - Remove our cb_update_key in favor of ngtcp2's new - ngtcp2_crypto_update_key_cb which does the same thing. + Follow up to 2b5b37c. - Several days ago the ngtcp2_update_key callback function prototype was - changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to - fix up our cb_update_key for that change they also added - ngtcp2_crypto_update_key_cb which does the same thing so we'll use that - instead. + Closes #4968 + +- md4: Fixed compilation issues when using GNU TLS gcrypt - Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c + * Don't include 'struct' in the gcrypt MD4_CTX typedef + * The call to gcry_md_read() should use a dereferenced ctx + * The call to gcry_md_close() should use a dereferenced ctx - Closes https://github.com/curl/curl/pull/4735 + Additional minor whitespace issue in the USE_WIN32_CRYPTO code. + + Closes #4959 -Daniel Stenberg (19 Dec 2019) -- sws: search for "Testno:" header uncondtionally if no testno +Daniel Stenberg (21 Feb 2020) +- RELEASE-NOTES: synced + +- http2: now require nghttp2 >= 1.12.0 - Even if the initial request line wasn't found. With the fix to 1455, the - test number is now detected correctly. + To simplify our code and since earlier versions lack important function + calls libcurl needs to function correctly. - (Problem found when running tests in random order.) + nghttp2 1.12.0 was relased on June 26, 2016. - Closes #4744 + Closes #4961 -- tests: set LC_ALL in more tests - - Follow-up to 23208e330ac0c21 +- gtls: fix the copyright year - Closes #4743 + Follow-up from 41fcb4f609 -- test165: set LC_ALL=en_US.UTF-8 too +- [jethrogb brought this change] + + GnuTLS: Always send client cert - On my current Debian Unstable with libidn2 2.2.0, I get an error if - LC_ALL is set to blank. Then curl errors out with: + TLS servers may request a certificate from the client. This request + includes a list of 0 or more acceptable issuer DNs. The client may use + this list to determine which certificate to send. GnuTLS's default + behavior is to not send a client certificate if there is no + match. However, OpenSSL's default behavior is to send the configured + certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL + behavior. - curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 + Authored-by: jethrogb on github + Fixes #1411 + Closes #4958 + +- [Leo Neat brought this change] + + github action: add CIFuzz - Closes #4738 + Closes #4960 -- curl.h: add two defines for the "pre ISO C" case +- cleanup: comment typos - Without this fix, this caused a compilation failure on AIX with IBM xlc - 13.1.3 compiler. + Spotted by 'codespell' - Reported-by: Ram Krushna Mishra - Fixes #4739 - Closes #4740 + Closes #4957 -- create_conn: prefer multiplexing to using new connections +Steve Holme (20 Feb 2020) +- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions - ... as it would previously prefer new connections rather than - multiplexing in most conditions! The (now removed) code was a leftover - from the Pipelining code that was translated wrongly into a - multiplex-only world. + Whilst lib\md4.c used this pre-processor, lib\md5.c and + src\tool_metalink.c did not and simply relied on the WIN32 + pre-processor directive. - Reported-by: Kunal Ekawde - Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html - Closes #4732 + Reviewed-by: Marcel Raad + Closes #4955 -- test1456: remove the use of a fixed local port - - Fixup the test to instead not compare the port number. It sometimes - caused problems like this: +Daniel Stenberg (19 Feb 2020) +- connect: remove some spurious infof() calls - "curl: (45) bind failed with errno 98: Address already in use" + As they were added primarily for debugging, they provide little use for + users. - Closes #4733 + Closes #4951 -Jay Satiro (18 Dec 2019) -- CURLOPT_QUOTE.3: fix typos +- HTTP-COOKIES: mention that a trailing newline is required - Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man - pages would not compile because a variable name was incorrect. + ... so that we know we got the whole and not a partial line. - Reported-by: Bylon2@users.noreply.github.com + Also, changed the formatting of the fields away from a table again since + the table format requires a github-markdown tool version that we don't + run on the web server atm. - Fixes https://github.com/curl/curl/issues/4736 + Reported-by: Sunny Bean + Fixes #4946 + Closes #4947 -- [Gisle Vanem brought this change] +- nit: Copyright year out of date + + Follow-up to 1fc0617dcc - strerror: Fix compiler warning "empty expression" +Jay Satiro (18 Feb 2020) +- tool_util: Improve Windows version of tvnow() - - Remove the final semi-colon in the SEC2TXT() macro definition. + - Change tool_util.c tvnow() for Windows to match more closely to + timeval.c Curl_now(). - Before: #define SEC2TXT(sec) case sec: txt = #sec; break; + - Create a win32 init function for the tool, since some initialization + is required for the tvnow() changes. - After: #define SEC2TXT(sec) case sec: txt = #sec; break + Prior to this change the monotonic time function used by curl in Windows + was determined at build-time and not runtime. That was a problem because + when curl was built targeted for compatibility with old versions of + Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps + every 49.7 days that Windows has been running. - Prior to this change SEC2TXT(foo); would generate break;; which caused - the empty expression warning. + This change makes curl behave similar to libcurl's tvnow function, which + determines at runtime whether the OS is Vista+ and if so calls + QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used + because it has higher resolution than the more obvious candidate + GetTickCount64). The changes to tvnow are basically a copy and paste but + the types in some cases are different. - Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 - -Daniel Stenberg (18 Dec 2019) -- curl/parseconfig: use curl_free() to free memory allocated by libcurl + Ref: https://github.com/curl/curl/issues/3309 - Reported-by: bxac on github - Fixes #4730 - Closes #4731 + Closes https://github.com/curl/curl/pull/4847 -- curl/parseconfig: fix mem-leak +Daniel Stenberg (18 Feb 2020) +- SOCKS: fix typo in printf formatting - When looping, first trying '.curlrc' and then '_curlrc', the function - would not free the first string. + Follow-up to 4a4b63daa - Closes #4731 + Reported-by: Peter Piekarski + Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 -- CURLOPT_URL.3: "curl supports SMB version 1 (only)" +- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section - [skip ci] - -- test1270: a basic -w redirect_url test + to be in sync with the description above - Closes #4728 + Reported-by: Joonas Kuorilehto + Fixes #4943 + Closes #4945 -- HISTORY: the SMB(S) support landed in 2014 +- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" -- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore +- altsvc: make saving the cache an atomic operation - It is covered by USE_OPENSSL_ENGINE now. + ... by writing the file to temp name then rename to the final when done. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 + Assisted-by: Jay Satiro + Fixes #4936 + Closes #4942 + +- rename: a new file for Curl_rename() - Closes #4725 + And make the cookie save function use it. -- lib: remove ASSIGNWITHINCONDITION exceptions, use our code style +- cookies: make saving atomic with a rename - ... even for macros + Saves the file as "[filename].[8 random hex digits].tmp" and renames + away the extension when done. - Reviewed-by: Daniel Gustafsson - Reviewed-by: Jay Satiro - Reported-by: Jay Satiro - Fixes #4683 - Closes #4722 + Co-authored-by: Jay Satiro + Reported-by: Mike Frysinger + Fixes #4914 + Closes #4926 -- tests: make sure checksrc runs on header files too +- RELEASE-NOTES: synced -- Revert "checksrc: fix regexp for ASSIGNWITHINCONDITION" +- socks: make the connect phase non-blocking - This reverts commit ba82673dac3e8d00a76aa5e3779a0cb80e7442af. + Removes two entries from KNOWN_BUGS. - Bug: #4683 + Closes #4907 -- KNOWN_BUGS: TLS session cache doesn't work with TFO +- multi: if Curl_readwrite sets 'comeback' use expire, not loop - [skip ci] - Closes #4301 - -- KNOWN_BUGS: Connection information when using TCP Fast Open + Otherwise, a very fast single transfer ricks starving out other + concurrent transfers. - Also point to #4296 for more details - Closes #4296 + Closes #4927 -- KNOWN_BUGS: LDAP on Windows doesn't work +- ftp: convert 'sock_accepted' to a plain boolean - Closes #4261 - -- docs: TLS SRP doesn't work with TLS 1.3 + This was an array indexed with sockindex but it was only ever used for + the secondary socket. - Reported-by: sayrer on github - Closes #4262 - [skip ci] + Closes #4929 -Dan Fandrich (16 Dec 2019) -- cirrus: Switch to the FreeBSD 12.1 point release & enable more tests. +Jay Satiro (15 Feb 2020) +- CURLINFO_COOKIELIST.3: Fix example - A few tests are now passing on FreeBSD, so no longer skip them. - [skip ci] + Prior to this change the example would try to import cookies from stdin, + which wasn't what was intended. + + Reported-by: 3dyd@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/4930 -Daniel Stenberg (16 Dec 2019) -- azure: the macos cmake doesn't need to install cmake +Daniel Stenberg (14 Feb 2020) +- TODO: Paged searches on LDAP server - Error: cmake 3.15.5 is already installed - To upgrade to 3.16.1, run `brew upgrade cmake`. + Closes #4452 + +- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS - Closes #4723 + Closes #4108 -Jay Satiro (15 Dec 2019) -- winbuild: Document CURL_STATICLIB requirement for static libcurl +- azure: disable brotli on the macos debug-builds - A static libcurl (ie winbuild mode=static) requires that the user define - CURL_STATICLIB when using it in their application. This is already - covered in the FAQ and INSTALL.md, but is a pretty important point so - now it's noted in the BUILD.WINDOWS.txt as well. + Because of: - Assisted-by: Michael Vittiglio + brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] + const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], - Closes https://github.com/curl/curl/pull/4721 + Closes #4925 -Daniel Stenberg (15 Dec 2019) -- [Santino Keupp brought this change] +Steve Holme (13 Feb 2020) +- tool_home: Fix the copyright year being out of date + + Follow up to 9dc350b6. - libssh2: add support for ECDSA and ed25519 knownhost keys +Jay Satiro (12 Feb 2020) +- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() - ... if a new enough libssh2 version is present. + - Deduplicate GetEnv() code. - Source: https://curl.haxx.se/mail/archive-2019-12/0023.html - Co-Authored-by: Daniel Stenberg - Closes #4714 - -- lib1591: free memory properly on OOM, in the trailers callback + - On Windows change ultimate call to use Windows API + GetEnvironmentVariable() instead of C runtime getenv(). - Detected by torture tests. + Prior to this change both libcurl and the tool had their own GetEnv + which over time diverged. Now the tool's GetEnv is a wrapper around + curl_getenv (libcurl API function which is itself a wrapper around + libcurl's GetEnv). - Closes #4720 - -- runtests: --repeat=[num] to repeat tests + Furthermore this change fixes a bug in that Windows API + GetEnvironmentVariable() is called instead of C runtime getenv() to get + the environment variable since some changes aren't always visible to the + latter. - Closes #4715 - -- RELEASE-NOTES: synced - -- azure: add a torture test on mac + Reported-by: Christoph M. Becker - Uses --shallow=25 to keep it small enough to get through in time. + Fixes https://github.com/curl/curl/issues/4774 + Closes https://github.com/curl/curl/pull/4863 + +Daniel Stenberg (12 Feb 2020) +- strerror.h: Copyright year out of date - Closes #4712 + Follow-up to 1c4fa67e8a8fcf6 -- multi: free sockhash on OOM +Jay Satiro (12 Feb 2020) +- strerror: Increase STRERROR_LEN 128 -> 256 - This would otherwise leak memory in the error path. + STRERROR_LEN is the constant used throughout the library to set the size + of the buffer on the stack that the curl strerror functions write to. - Detected by torture test 1540. + Prior to this change some extended length Windows error messages could + be truncated. - Closes #4713 + Closes https://github.com/curl/curl/pull/4920 -Marcel Raad (13 Dec 2019) -- tests: use DoH feature for DoH tests +- multi: fix outdated comment - Previously, http/2 was used instead. + - Do not say that conn->data is "cleared" by multi_done(). - Assisted-by: Jay Satiro - Closes https://github.com/curl/curl/pull/4692 + If the connection is in use then multi_done assigns another easy handle + still using the connection to conn->data, therefore in that case it is + not cleared. + + Closes https://github.com/curl/curl/pull/4901 -- hostip: suppress compiler warning +- easy: remove dead code - With `--disable-doh --disable-threaded-resolver`, the `dns` parameter - is not used. + multi is already assigned to data->multi by curl_multi_add_handle. - Closes https://github.com/curl/curl/pull/4692 + Closes https://github.com/curl/curl/pull/4900 -- tests: fix build with `CURL_DISABLE_DOH` +Daniel Stenberg (12 Feb 2020) +- create-dirs.d: mention the mode - Closes https://github.com/curl/curl/pull/4692 + Reported-by: Dan Jacobson + Fixes #4766 + Closes #4916 -Daniel Stenberg (13 Dec 2019) -- azure: add a torture test - - Skipping all FTP tests for speed reasons. +- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording - Closes #4697 + Assisted-by: Jay Satiro + Reported-by: Craig Andrews + Fixes #4909 + Closes #4910 -- azure: make the default build use --enable-debug --enable-werror +- RELEASE-NOTES: synced -- ntlm_wb: fix double-free in OOM +Steve Holme (9 Feb 2020) +- smtp: Simplify the MAIL command and avoid a duplication of send strings - Detected by torture testing test 1310 + This avoids the duplication of strings when the optional AUTH and SIZE + parameters are required. It also assists with the modifications that + are part of #4892. - Closes #4710 + Closes #4903 -Dan Fandrich (13 Dec 2019) -- cirrus: Drop the FreeBSD 10.4 build +Daniel Stenberg (9 Feb 2020) +- altsvc: keep a copy of the file name to survive handle reset - Upstream support for 10.4 ended a year ago, and it looks like the image - is now gone, too. - [skip ci] - -Daniel Stenberg (13 Dec 2019) -- unit1620: fix bad free in OOM + The alt-svc cache survives a call to curl_easy_reset fine, but the file + name to use for saving the cache was cleared. Now the alt-svc cache has + a copy of the file name to survive handle resets. - Closes #4709 - -- unit1609: fix mem-leak in OOM + Added test 1908 to verify. - Closes #4709 + Reported-by: Craig Andrews + Fixes #4898 + Closes #4902 -- unit1607: fix mem-leak in OOM +Steve Holme (9 Feb 2020) +- url: Include the failure reason when curl_win32_idn_to_ascii() fails - Closes #4709 - -- lib1559: fix mem-leak in OOM + Provide the failure reason in the failf() info just as we do for the + libidn2 version of code. - Closes #4709 + Closes #4899 -- lib1557: fix mem-leak in OOM +Jay Satiro (9 Feb 2020) +- asyn-thread: remove dead code + +Daniel Stenberg (8 Feb 2020) +- [Emil Engler brought this change] + + github: Instructions to post "uname -a" on Unix systems in issues - Closes #4709 + Closes #4896 -- altsvc: make the save function ignore NULL filenames +- [Cristian Greco brought this change] + + configure.ac: fix comments about --with-quiche - It might happen in OOM situations. Detected bv torture tests. + A simple s/nghttp3/quiche in some comments of --with-quiche. + Looks like a copy-paste error from --with-nghttp3. - Closes #4707 + Closes #4897 -- curl: fix memory leak in OOM in etags logic +Steve Holme (7 Feb 2020) +- checksrc.bat: Fix not being able to run script from the main curl directory - Detected by torture tests + If the script was ran from the main curl directory rather then the + projects directory then the script would simply exit without error: - Closes #4706 - -- doh: make it behave when built without proxy support + C:\url> projects\checksrc.bat - Reported-by: Marcel Raad - Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734 + The user would either need to change to the projects directory, + explicitly specify the current working directory, or perform a + oneline hacky workaround: - Closes #4704 - -- curl: improved cleanup in upload error path + C:\url> cd projects + C:\url\projects> checksrc.bat - Memory leak found by torture test 58 + C:\url> checksrc.bat %cd% - Closes #4705 - -- mailmap: fix Andrew Ishchuk - -- travis: make torture use --shallow=40 + C:\url> pushd projects & checksrc.bat & popd - As a first step to enable it to run over a more diverse set of tests in - a reasonable time. + Closes #4894 -- runtests: introduce --shallow to reduce huge torture tests +Daniel Stenberg (7 Feb 2020) +- [Pierre-Yves Bigourdan brought this change] + + digest: Do not quote algorithm in HTTP authorisation - When set, shallow mode limits runtests -t to make no more than NUM fails - per test case. If more are found, it will randomly discard entries until - the number is right. The random seed can also be set. + RFC 7616 section 3.4 (The Authorization Header Field) states that "For + historical reasons, a sender MUST NOT generate the quoted string syntax + for the following parameters: algorithm, qop, and nc". This removes the + quoting for the algorithm parameter. - This is particularly useful when running MANY tests as then most torture - failures will already fail the same functions over and over and make the - total operation painfully tedious. + Reviewed-by: Steve Holme + Closes #4890 + +- ftp: remove the duplicated user/password struct fields - Closes #4699 + Closes #4887 -- conncache: CONNECT_ONLY connections assumed always in-use +- ftp: remove superfluous checking for crlf in user or pwd - This makes them never to be considered "the oldest" to be discarded when - reaching the connection cache limit. The reasoning here is that - CONNECT_ONLY is primarily used in combination with using the - connection's socket post connect and since that is used outside of - curl's knowledge we must assume that it is in use until explicitly - closed. + ... as this is already done much earlier in the URL parser. - Reported-by: Pavel Pavlov - Reported-by: Pavel Löbl - Fixes #4426 - Fixes #4369 - Closes #4696 - -- [Gisle Vanem brought this change] - - vtls: make BearSSL possible to set with CURL_SSL_BACKEND + Also add test case 894 that verifies that pop3 with an encodedd CR in + the user name is rejected. - Ref: https://github.com/curl/curl/commit/9b879160df01e7ddbb4770904391d3b74114302b#commitcomment-36355622 + Closes #4887 + +Steve Holme (6 Feb 2020) +- ntlm_wb: Use Curl_socketpair() for greater portability - Closes #4698 + Reported-by: Daniel Stenberg + Closes #4886 -- RELEASE-NOTES: synced +Daniel Stenberg (5 Feb 2020) +- [Frank Gevaerts brought this change] -- travis: remove "coverage", make it "torture" - - The coveralls service and test coverage numbers are just too unreliable. - Removed badge from README.md as well. + contributors: Also include people who contributed to curl-www - Fixes #4694 - Closes #4695 + Closes #4884 -- azure: add libssh2 and cmake macos builds +- [Frank Gevaerts brought this change] + + contrithanks: Use the most recent tag by default - Removed the macos libssh2 build from travis + (similar to 5296abe) - Closes #4686 + Closes #4883 -- curl: use errorf() better +- scripts: use last set tag if none given - Change series of error outputs to use errorf(). + Makes 'delta' and 'contributors.sh' easier to use. - Only errors that are due to mistakes in command line option usage should - use helpf(), other types of errors in the tool should rather use - errorf(). + Make the delta script invoke contrithanks to get current number of + contributors instead of counting THANKS, for accuracy. - Closes #4691 - -Jay Satiro (9 Dec 2019) -- [Marc Hoersken brought this change] + Closes #4881 - tests: make it possible to set executable extensions - - This enables the use of Windows Subsystem for Linux (WSL) to run the - testsuite against Windows binaries while using Linux servers. +- ftp: shrink temp buffers used for PORT - This commit introduces the following environment variables: - - CURL_TEST_EXE_EXT: set the executable extension for all components - - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only - - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only + These two stack based buffers only need to be 46 + 66 bytes instead of + 256 + 1024. - Later testcurl.pl could be adjusted to make use of those variables. - - CURL_TEST_EXE_EXT_SRV: set it for the test servers only + Closes #4880 + +- curl: error on --alt-svc use w/o support - (This is one of several commits to support use of WSL for the tests.) + Make the tool check for alt-svc support at run-time and return error + accordingly if not present when the option is used. - Closes https://github.com/curl/curl/pull/3899 + Reported-by: Harry Sintonen + Closes #4878 -- [Marc Hoersken brought this change] +- docs/HTTP3: add --enable-alt-svc to curl's configure - tests: fix permissions of ssh keys in WSL +- RELEASE-PROCEDURE: feature win is closed post-release a few days - Keys created on Windows Subsystem for Linux (WSL) require it for some - reason. + We've tried to uphold this already but let's make it official by + publicly stating this is the way we do it. - (This is one of several commits to support use of WSL for the tests.) + Closes #4877 + +- altsvc: set h3 version at a common single spot - Ref: https://github.com/curl/curl/pull/3899 + ... and move the #ifdefs out of the functions. Addresses the fact they + were different before this change. + + Reported-by: Harry Sintonen + Closes #4876 -- [Marc Hoersken brought this change] +- [Harry Sintonen brought this change] - tests: use \r\n for log messages in WSL + altsvc: improved header parser - Bash in Windows Subsystem for Linux (WSL) requires it for some reason. + - Fixed the flag parsing to apply to specific alternative entry only, as + per RFC. The earlier code would also get totally confused by + multiprotocol header, parsing flags from the wrong part of the header. - (This is one of several commits to support use of WSL for the tests.) + - Fixed the parser terminating on unknown protocols, instead of skipping + them. - Ref: https://github.com/curl/curl/pull/3899 + - Fixed a busyloop when protocol-id was present without an equal sign. + + Closes #4875 -- [Andrew Ishchuk brought this change] +- [Harry Sintonen brought this change] - winbuild: Define CARES_STATICLIB when WITH_CARES=static - - When libcurl is built with MODE=static, c-ares is forced into static - linkage too. That doesn't happen when MODE=dll so linker would break - over undefined symbols. + ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 + +- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 - closes https://github.com/curl/curl/pull/4688 + Reported-by: James Fuller -Daniel Stenberg (9 Dec 2019) -- conn: always set bits.close with connclose() +Steve Holme (4 Feb 2020) +- ntlm: Pass the Curl_easy structure to the private winbind functions - Closes #4690 + ...rather than the full conndata structure. -- cirrus: enable clang sanitizers on freebsd 13 +Daniel Stenberg (4 Feb 2020) +- RELEASE-NOTES: synced -- conncache: fix multi-thread use of shared connection cache +- tool_operhlp: Copyright year out of date, should be 2020 - It could accidentally let the connection get used by more than one - thread, leading to double-free and more. + Follow-up from 2bc373740a3 + +- [Orgad Shaneh brought this change] + + curl: avoid using strlen for testing if a string is empty - Reported-by: Christopher Reid - Fixes #4544 - Closes #4557 + Closes #4873 -- azure: add a vanilla macos build +Steve Holme (3 Feb 2020) +- ntlm: Ensure the HTTP header data is not stored in the challenge/response + +Marcel Raad (3 Feb 2020) +- openssl: remove redundant assignment - Closes #4685 + Fixes a scan-build failure on Bionic. + + Closes https://github.com/curl/curl/pull/4872 -- curl: make the etag load logic work without fseek +- travis: update non-OpenSSL Linux jobs to Bionic - The fseek()s were unnecessary and caused Coverity warning CID 1456554 + For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is + failing with "curl returned 52, when expecting 35". - Closes #4681 + Closes https://github.com/curl/curl/pull/4872 -- mailmap: Mohammad Hasbini +Dan Fandrich (3 Feb 2020) +- cirrus: Add some missing semicolons + + Newlines aren't preserved in this section so they're needed to separate + commands. The exports luckily worked anyway as a single long line, but + erroneously exported a variable called "export" + [skip ci] -- [Mohammad Hasbini brought this change] +Daniel Gustafsson (2 Feb 2020) +- [Pedro Monreal brought this change] - docs: fix some typos + cleanup: fix typos and wording in docs and comments - Closes #4680 - -- RELEASE-NOTES: synced + Closes #4869 + Reviewed-by: Emil Engler and Daniel Gustafsson -Jay Satiro (5 Dec 2019) -- lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN +Steve Holme (2 Feb 2020) +- ntlm: Move the winbind data into the NTLM data structure - Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS - and OS400 package spec. + To assist with adding winbind support to the SASL NTLM authentication, + move the winbind specific data out of conndata into ntlmdata. + +Daniel Stenberg (30 Jan 2020) +- quiche: Copyright year out of date - Also I added the option to the NameValue list in the tool even though it - isn't exposed as a command-line option (...yet?). (NameValue stringizes - the option name for the curl cmd -> libcurl source generator) + Follow-up to 7fc63d72333a + +- altsvc: use h3-25 - Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN. + Closes #4868 + +- [Alessandro Ghedini brought this change] + + quiche: update to draft-25 - Ref: https://github.com/curl/curl/pull/4655 + Closes #4867 -- setopt: Fix ALPN / NPN user option when built without HTTP2 +- ngtcp2: update to git master and its draft-25 support - - Stop treating lack of HTTP2 as an unknown option error result for - CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN. + Closes #4865 + +- cookie: check __Secure- and __Host- case sensitively - Prior to this change it was impossible to disable ALPN / NPN if libcurl - was built without HTTP2. Setting either option would result in - CURLE_UNKNOWN_OPTION and the respective internal option would not be - set. That was incorrect since ALPN and NPN are used independent of - HTTP2. + While most keywords in cookies are case insensitive, these prefixes are + specified explicitly to get checked "with a case-sensitive match". - Reported-by: Shailesh Kapse + (From the 6265bis document in progress) - Fixes https://github.com/curl/curl/issues/4668 - Closes https://github.com/curl/curl/pull/4672 + Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 + Closes #4864 -Daniel Stenberg (5 Dec 2019) -- etag: allow both --etag-compare and --etag-save in same cmdline +- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header + +- oauth2-bearer.d: works for HTTP too - Fixes #4669 - Closes #4678 + Reported-by: Mischa Salle + Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html + Closes #4862 -Marcel Raad (5 Dec 2019) -- curl_setup: fix `CURLRES_IPV6` condition +- multi_done: if multiplexed, make conn->data point to another transfer - Move the definition of `CURLRES_IPV6` to before undefining - `HAVE_GETADDRINFO`. Regression from commit 67a08dca27a which caused - some tests to fail and others to be skipped with c-ares. + ... since the current transfer is being killed. Setting to NULL is + wrong, leaving it pointing to 'data' is wrong since that handle might be + about to get freed. - Fixes https://github.com/curl/curl/issues/4673 - Closes https://github.com/curl/curl/pull/4677 + Fixes #4845 + Closes #4858 + Reported-by: dmitrmax on github -Daniel Stenberg (5 Dec 2019) -- test342: make it return a 304 as the tag matches +- location.d: the method change is from POST to GET only + + Not from generic non-GET to GET. + + Reported-by: Andrius Merkys + Ref: #4859 + Closes #4861 -Peter Wu (4 Dec 2019) -- CMake: add support for building with the NSS vtls backend +- urlapi: guess scheme correct even with credentials given - Options are cross-checked with configure.ac and acinclude.m4. - Tested on Arch Linux, untested on other platforms like Windows or macOS. + In the "scheme-less" parsing case, we need to strip off credentials + first before we guess scheme based on the host name! - Closes #4663 - Reviewed-by: Kamil Dudka + Assisted-by: Jay Satiro + Fixes #4856 + Closes #4857 -Daniel Stenberg (4 Dec 2019) -- azure: add more builds +- global_init: move the IPv6 works status bool to multi handle - ... removed two from travis (that now runs on azure instead) + Previously it was stored in a global state which contributed to + curl_global_init's thread unsafety. This boolean is now instead figured + out in curl_multi_init() and stored in the multi handle. Less effective, + but thread safe. - Closes #4671 + Closes #4851 -- CURLOPT_VERBOSE.3: see also ERRORBUFFER +- [Jay Satiro brought this change] -- hostip4.c: bump copyright year range + README: mention that the docs is in docs/ + + Reported-by: Austin Green + Fixes #4830 + Closes #4853 -Marcel Raad (3 Dec 2019) -- configure: enable IPv6 support without `getaddrinfo` +- curl.h: define CURL_WIN32 on windows - This makes it possible to recognize and connect to literal IPv6 - addresses when `getaddrinfo` is not available, which is already the - case for the CMake build. This affects e.g. classic MinGW because it - still targets Windows 2000 by default, where `getaddrinfo` is not - available, but general IPv6 support is. + ... so that the subsequent logic below can use a single known define to know + when built on Windows (as we don't define WIN32 anymore). - Instead of checking for `getaddrinfo`, check for `sockaddr_in6` as the - CMake build does. + Follow-up to 1adebe7886ddf20b - Closes https://github.com/curl/curl/pull/4662 + Reported-by: crazydef on github + Assisted-by: Marcel Raad + Fixes #4854 + Closes #4855 -- curl_setup: disable IPv6 resolver without `getaddrinfo` +- RELEASE-NOTES: synced + +- [Jon Rumsey brought this change] + + urldata: do string enums without #ifdefs for build scripts - Also, use `CURLRES_IPV6` only for actual DNS resolution, not for IPv6 - address support. This makes it possible to connect to IPv6 literals by - setting `ENABLE_IPV6` even without `getaddrinfo` support. It also fixes - the CMake build when using the synchronous resolver without - `getaddrinfo` support. + ... and check for inconsistencies for OS400 at build time with the new + chkstrings tool. - Closes https://github.com/curl/curl/pull/4662 + Closes #4822 -Daniel Stenberg (3 Dec 2019) -- github action/azure pipeline: run 'make test-nonflaky' for tests +- curl: make the -# spaceship bar not wrap the line - To match travis and give more info on failures. - -- openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains + The fixed-point math made us lose precision and thus a too high index + value could be used for outputting the hashtags which could overwrite + the newline. - Closes #4655 + The fix increases the precision in the sine table (*100) and the + associated position math. + + Reported-by: Andrew Potter + Fixes #4849 + Closes #4850 -- openssl: set X509_V_FLAG_PARTIAL_CHAIN +- global_init: assume the EINTR bit by default - Have intermediate certificates in the trust store be treated as - trust-anchors, in the same way as self-signed root CA certificates - are. This allows users to verify servers using the intermediate cert - only, instead of needing the whole chain. + - Removed from global_init since it isn't thread-safe. The symbol will + still remain to not break compiles, it just won't have any effect going + forward. - Other TLS backends already accept partial chains. + - make the internals NOT loop on EINTR (the opposite from previously). + It only risks returning from the select/poll/wait functions early, and that + should be risk-free. - Reported-by: Jeffrey Walton - Bug: https://curl.haxx.se/mail/lib-2019-11/0094.html + Closes #4840 -- curl: show better error message when no homedir is found +- [Peter Piekarski brought this change] + + conn: do not reuse connection if SOCKS proxy credentials differ - Reported-by: Vlastimil Ovčáčík - Fixes #4644 - Closes #4665 + Closes #4835 -- OPENSOCKETFUNCTION.3: correct the purpose description +- llist: removed unused Curl_llist_move() - Reported-by: Jeff Mears - Bug: https://curl.haxx.se/mail/lib-2019-12/0007.html + (and the corresponding unit test) - Closes #4667 + Closes #4842 -- [Peter Wu brought this change] +- conncache: removed unused Curl_conncache_bundle_size() - travis: do not use OVERRIDE_CC or OVERRIDE_CXX if empty - - Fixes the macOS builds where OVERRIDE_CC and OVERRIDE_CXX are not set. +- strcase: turn Curl_raw_tolower into static - Reported-by: Jay Satiro - Fixes #4659 - Closes #4661 - Closes #4664 + Only ever used from within this file. -- azure-pipelines: fix the test script +- singleuse.pl: support new API functions, fix curl_dbg_ handling -- Azure Pipelines: initial CI setup +- wolfssh: make it init properly via Curl_ssh_init() - [skip ci] + Closes #4846 -- docs: add "added: 7.68.0" to the --etag-* docs +- [Aron Rotteveel brought this change] -- copyright: fix the year ranges for two files + form.d: fix two minor typos - Follow-up to 9c1806ae + Closes #4843 -Jay Satiro (1 Dec 2019) -- build: Disable Visual Studio warning "conditional expression is constant" - - - Disable warning C4127 "conditional expression is constant" globally - in curl_setup.h for when building with Microsoft's compiler. - - This mainly affects building with the Visual Studio project files found - in the projects dir. - - Prior to this change the cmake and winbuild build systems already - disabled 4127 globally for when building with Microsoft's compiler. - Also, 4127 was already disabled for all build systems in the limited - circumstance of the WHILE_FALSE macro which disabled the warning - specifically for while(0). This commit removes the WHILE_FALSE macro and - all other cruft in favor of disabling globally in curl_setup. - - Background: - - We have various macros that cause 0 or 1 to be evaluated, which would - cause warning C4127 in Visual Studio. For example this causes it: - - #define Curl_resolver_asynch() 1 - - Full behavior is not clearly defined and inconsistent across versions. - However it is documented that since VS 2015 Update 3 Microsoft has - addressed this somewhat but not entirely, not warning on while(true) for - example. +- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields - Prior to this change some C4127 warnings occurred when I built with - Visual Studio using the generated projects in the projects dir. + Avoid "reparsing" the content and instead deliver more exactly what is + provided in the certificate and avoid truncating the data after 512 + bytes as done previously. This no longer removes embedded newlines. - Closes https://github.com/curl/curl/pull/4658 + Fixes #4837 + Reported-by: bnfp on github + Closes #4841 -- openssl: retrieve reported LibreSSL version at runtime +Jay Satiro (23 Jan 2020) +- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 - - Retrieve LibreSSL runtime version when supported (>= 2.7.1). + - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. - For earlier versions we continue to use the compile-time version. + Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the + CURLOPT_PROXY_SSL_OPTIONS description. + +Daniel Stenberg (22 Jan 2020) +- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER - Ref: https://man.openbsd.org/OPENSSL_VERSION_NUMBER.3 + For now, no cert in the bundle actually sets a date there... - Closes https://github.com/curl/curl/pull/2425 + Co-Authored-by: Jay Satiro + Reported-by: Christian Heimes + Fixes #4834 + Closes #4836 -- strerror: Add Curl_winapi_strerror for Win API specific errors - - - In all code call Curl_winapi_strerror instead of Curl_strerror when - the error code is known to be from Windows GetLastError. +- RELEASE-NOTES: synced + +- [Pavel Volgarev brought this change] + + smtp: Allow RCPT TO command to fail for some recipients - Curl_strerror prefers CRT error codes (errno) over Windows API error - codes (GetLastError) when the two overlap. When we know the error code - is from GetLastError it is more accurate to prefer the Windows API error - messages. + Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. - Reported-by: Richard Alcock + Verified with the new tests 3002-3007 - Fixes https://github.com/curl/curl/issues/4550 - Closes https://github.com/curl/curl/pull/4581 + Closes #4816 -Daniel Stenberg (2 Dec 2019) -- global_init: undo the "intialized" bump in case of failure - - ... so that failures in the global init function don't count as a - working init and it can then be called again. +- copyright: fix year ranges - Reported-by: Paul Groke - Fixes #4636 - Closes #4653 + follow-up from dea17b519d (one of these days I'll learn to check before + I push) -- parsedate: offer a getdate_capped() alternative +- [nao brought this change] + + http: move "oauth_bearer" from connectdata to Curl_easy - ... and use internally. This function will return TIME_T_MAX instead of - failure if the parsed data is found to be larger than what can be - represented. TIME_T_MAX being the largest value curl can represent. + Fixes the bug where oauth_bearer gets deallocated when we re-use a + connection. - Reviewed-by: Daniel Gustafsson - Reported-by: JanB on github - Fixes #4152 - Closes #4651 + Closes #4824 -- docs: add more references to curl_multi_poll - - Fixes #4643 - Closes #4652 +- [Emil Engler brought this change] -- sha256: bump the copyright year range + curl: Let -D merge headers in one file again - Follow-up from 66e21520f + Closes #4762 + Fixes #4753 -Daniel Gustafsson (28 Nov 2019) -- curl_setup_once: consistently use WHILE_FALSE in macros +- data.d: remove "Multiple files can also be specified" - The WHILE_FALSE construction is used to avoid compiler warnings in - macro constructions. This fixes a few instances where it was not - used in order to keep the code consistent. + It is superfluous and could even be misleading. - Closes #4649 - Reviewed-by: Daniel Stenberg - -Daniel Stenberg (28 Nov 2019) -- [Steve Holme brought this change] + Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html + Reported-by: Mike Norton + Closes #4832 - http_ntlm: Remove duplicate NSS initialisation +Marcel Raad (20 Jan 2020) +- CMake: support specifying the target Windows version - Given that this is performed by the NTLM code there is no need to - perform the initialisation in the HTTP layer. This also keeps the - initialisation the same as the SASL based protocols and also fixes a - possible compilation issue if both NSS and SSPI were to be used as - multiple SSL backends. + Previously, it was only possible to set it to Windows Vista or XP by + setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. + Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to + explicitly set the target Windows version. `ENABLE_INET_PTON` is + ignored in this case. - Reviewed-by: Kamil Dudka - Closes #3935 + Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 + Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 + Closes https://github.com/curl/curl/pull/4815 -Daniel Gustafsson (28 Nov 2019) -- checksrc: fix regexp for ASSIGNWITHINCONDITION - - The regexp looking for assignments within conditions was too greedy - and matched a too long string in the case of multiple conditionals - on the same line. This is basically only a problem in single line - macros, and the code which exemplified this was essentially: +Daniel Stenberg (20 Jan 2020) +- http.h: Copyright year out of date, should be 2020 - do { if((x) != NULL) { x = NULL; } } while(0) + Follow-up to 7ff9222ced8c + +- [加藤郁之 brought this change] + + HTTP: increase EXPECT_100_THRESHOLD to 1Mb - ..where the final parenthesis of while(0) matched the regexp, and - the legal assignment in the block triggered the warning. Fix by - making the regexp less greedy by matching for the tell-tale signs - of the if statement ending. + Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html - Also remove the one occurrence where the warning was disabled due - to a construction like the above, where the warning didn't apply - when fixed. + Closes #4814 + +- ROADMAP: thread-safe `curl_global_init()` - Closes #4647 - Reviewed-by: Daniel Stenberg + I'd like to see this happen. -Daniel Stenberg (28 Nov 2019) - RELEASE-NOTES: synced -- [Maros Priputen brought this change] +- wolfssl: use the wc-prefixed symbol alternatives + + The symbols without wc_ prefix are not always provided. + + Ref: https://github.com/wolfSSL/wolfssl/issues/2744 + + Closes #4827 - curl: two new command line options for etags +- polarssl: removed - --etag-compare and --etag-save + As detailed in DEPRECATE.md, the polarssl support is now removed after + having been disabled for 6 months and nobody has missed it. - Suggested-by: Paul Hoffman - Fixes #4277 - Closes #4543 + The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix + instead of the former 'polarssl' and the common functions that + previously were shared between mbedtls and polarssl and contained the + name 'polarssl' have now all been renamed to instead say 'mbedtls'. + + Closes #4825 -Daniel Gustafsson (28 Nov 2019) -- docs: fix typos +Marcel Raad (16 Jan 2020) +- libssh2: fix variable type + + This led to a conversion warning on 64-bit MinGW, which has 32-bit + `long` but 64-bit `size_t`. + + Closes https://github.com/curl/curl/pull/4823 -Daniel Stenberg (28 Nov 2019) -- mailmap: Niall O'Reilly's name +Daniel Stenberg (16 Jan 2020) +- curl:progressbarinit: ignore column width from terminals < 20 + + To avoid division by zero - or other issues. + + Reported-by: Daniel Marjamäki + Closes #4818 -- [Niall O'Reilly brought this change] +- wolfssh: set the password correctly for PASSWORD auth - doh: use dedicated probe slots +- wolfssh: remove fprintf() calls (and uses of __func__) + +Marcel Raad (14 Jan 2020) +- CMake: use check_symbol_exists also for inet_pton - ... to easier allow additional DNS transactions. + It doesn't make much sense to only check if the function can be linked + when it's not declared in any header and that is treated as an error. + With the correct target Windows version set, the function is declared + in ws2tcpip.h and the comment above the modified block is invalid. - Closes #4629 - -- travis: build ngtcp2 with --enable-lib-only + Also, move the definition of `_WIN32_WINNT` up to before all symbol + availability checks so that we don't have to care which ones must be + done after it. - ... makes it skip the examples and other stuff we don't neeed. + Tested with Visual Studio 2019 and current MinGW-w64. - Closes #4646 - -- [David Benjamin brought this change] + Closes https://github.com/curl/curl/pull/4808 - ngtcp2: fix thread-safety bug in error-handling +Jay Satiro (13 Jan 2020) +- schannel_verify: Fix alt names manual verify for UNICODE builds - ERR_error_string(NULL) should never be called. It places the error in a - global buffer, which is not thread-safe. Use ERR_error_string_n with a - local buffer instead. + Follow-up to 29e40a6 from two days ago, which added that feature for + Windows 7 and earlier. The bug only occurred in same. - Closes #4645 + Ref: https://github.com/curl/curl/pull/4761 -- travis: export the CC/CXX variables when set +Daniel Stenberg (13 Jan 2020) +- HTTP-COOKIES.md: describe the cookie file format - Suggested-by: Peter Wu - Fixes #4637 - Closes #4640 + ... and refer to that file from from CURLOPT_COOKIEFILE.3 and + CURLOPT_COOKIELIST.3 + + Assisted-by: Jay Satiro + Reported-by: bsammon on github + Fixes #4805 + Closes #4806 -Marcel Raad (26 Nov 2019) -- dist: add error-codes.pl +- [Tobias Hieta brought this change] + + CMake: Add support for CMAKE_LTO option. - Follow-up to commit 74f441c6d31. - This should fix test 1175 when run via the daily source tarballs. + This enables Link Time Optimization. LTO is a proven technique for + optimizing across compilation units. - Closes https://github.com/curl/curl/pull/4638 + Closes #4799 -Daniel Stenberg (26 Nov 2019) -- [John Schroeder brought this change] +- RELEASE-NOTES: synced - curl: fix --upload-file . hangs if delay in STDIN +- ConnectionExists: respect the max_concurrent_streams limits - Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION. + A regression made the code use 'multiplexed' as a boolean instead of the + counter it is intended to be. This made curl try to "over-populate" + connections with new streams. - When uploading from stdin in non-blocking mode, a delay in reading - the stream (EAGAIN) causes curl to pause sending data - (CURL_READFUNC_PAUSE). Prior to this change, a busy read was - detected and unpaused only in the CURLOPT_WRITEFUNCTION handler. - This change performs the same busy read handling in a - CURLOPT_XFERINFOFUNCTION handler. + This regression came with 41fcdf71a1, shipped in curl 7.65.0. - Fixes #2051 - Closes #4599 - Reported-by: bdry on github - -- [John Schroeder brought this change] + Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same + check. + + Reported-by: Kunal Ekawde + Fixes #4779 + Closes #4784 - XFERINFOFUNCTION: support CURL_PROGRESSFUNC_CONTINUE +- curl: make #0 not output the full URL - (also for PROGRESSFUNCTION) + It was not intended nor documented! - By returning this value from the callback, the internal progress - function call is still called afterward. + Added test 1176 to verify. - Closes #4599 + Reported-by: vshmuk on hackerone + + Closes #4812 -- [Michael Forney brought this change] +- wolfSSH: new SSH backend + + Adds support for SFTP (not SCP) using WolfSSH. + + Closes #4231 - TLS: add BearSSL vtls implementation +- curl: remove 'config' field from OutStruct - Closes #4597 + As it was just unnecessary duplicated information already stored in the + 'per_transfer' struct and that's around mostly anyway. + + The duplicated pointer caused problems when the code flow was aborted + before the dupe was filled in and could cause a NULL pointer access. + + Reported-by: Brian Carpenter + Fixes #4807 + Closes #4810 -- curl_multi_wakeup.3: add example and AVAILABILITY +- misc: Copyright year out of date, should be 2020 - Reviewed-by: Gergely Nagy - Closes #4635 + Follow-up to recent commits + + [skip ci] -- [Gergely Nagy brought this change] +Jay Satiro (11 Jan 2020) +- [Santino Keupp brought this change] - multi: add curl_multi_wakeup() + libssh2: add support for forcing a hostkey type - This commit adds curl_multi_wakeup() which was previously in the TODO - list under the curl_multi_unblock name. + - Allow forcing the host's key type found in the known_hosts file. - On some platforms and with some configurations this feature might not be - available or can fail, in these cases a new error code - (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). + Currently, curl (with libssh2) does not take keys from your known_hosts + file into account when talking to a server. With this patch the + known_hosts file will be searched for an entry matching the hostname + and, if found, libssh2 will be told to claim this key type from the + server. - Fixes #4418 - Closes #4608 + Closes https://github.com/curl/curl/pull/4747 -Jay Satiro (24 Nov 2019) -- [Xiaoyin Liu brought this change] +- [Nicolas Guillier brought this change] - schannel: fix --tls-max for when min is --tlsv1 or default + cmake: Improve libssh2 check on Windows - Prior to this change schannel ignored --tls-max (CURL_SSLVERSION_MAX_ - macros) when --tlsv1 (CURL_SSLVERSION_TLSv1) or default TLS - (CURL_SSLVERSION_DEFAULT), using a max of TLS 1.2 always. + - Add "libssh2" name to FindLibSSH2 library search. - Closes https://github.com/curl/curl/pull/4633 - -- checksrc.bat: Add a check for vquic and vssh directories + On Windows systems, libSSH2 CMake installation may name the library + "LibSSH2". - Ref: https://github.com/curl/curl/pull/4607 + Prior to this change cmake only checked for name "ssh2". On Linux that + works fine because it will prepend the "lib", but it doesn't do that on + Windows. + + Closes https://github.com/curl/curl/pull/4804 -- projects: Fix Visual Studio projects SSH builds +- [Faizur Rahman brought this change] + + schannel: Make CURLOPT_CAINFO work better on Windows 7 - - Generate VQUIC and VSSH filenames in Visual Studio project files. + - Support hostname verification via alternative names (SAN) in the + peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. - Prior to this change generated Visual Studio project configurations that - enabled SSH did not build properly. Broken since SSH files were moved to - lib/vssh 3 months ago in 5b2d703. + CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a + result CertGetNameString doesn't quite work on those versions of + Windows. This change provides an alternative solution for + CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier + versions of Windows. - Fixes https://github.com/curl/curl/issues/4492 - Fixes https://github.com/curl/curl/issues/4630 - Closes https://github.com/curl/curl/pull/4607 + Prior to this change many certificates failed the hostname validation + when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates + now represent multiple hostnames and rely on the alternative names field + exclusively to represent their hostnames. + + Reported-by: Jeroen Ooms + + Fixes https://github.com/curl/curl/issues/3711 + Closes https://github.com/curl/curl/pull/4761 -Daniel Stenberg (23 Nov 2019) -- RELEASE-NOTES: synced +- [Emil Engler brought this change] -Jay Satiro (22 Nov 2019) -- openssl: Revert to less sensitivity for SYSCALL errors + ngtcp2: Add an error code for QUIC connection errors - - Disable the extra sensitivity except in debug builds (--enable-debug). + - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection + errors. - - Improve SYSCALL error message logic in ossl_send and ossl_recv so that - "No error" / "Success" socket error text isn't shown on SYSCALL error. + Prior to this change CURLE_FAILED_INIT was used, but that was not + correct. - Prior to this change 0ab38f5 (precedes 7.67.0) increased the sensitivity - of OpenSSL's SSL_ERROR_SYSCALL error so that abrupt server closures were - also considered errors. For example, a server that does not send a known - protocol termination point (eg HTTP content length or chunked encoding) - _and_ does not send a TLS termination point (close_notify alert) would - cause an error if it closed the connection. + Closes https://github.com/curl/curl/pull/4754 + +- multi: Change curl_multi_wait/poll to error on negative timeout - To be clear that behavior made it into release build 7.67.0 - unintentionally. Several users have reported it as an issue. + - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when + curl_multi_wait/poll is passed timeout param < 0. - Ultimately the idea is a good one, since it can help prevent against a - truncation attack. Other SSL backends may already behave similarly (such - as Windows native OS SSL Schannel). However much more of our user base - is using OpenSSL and there is a mass of legacy users in that space, so I - think that behavior should be partially reverted and then rolled out - slowly. + Prior to this change passing a negative value to curl_multi_wait/poll + such as -1 could cause the function to wait forever. - This commit changes the behavior so that the increased sensitivity is - disabled in all curl builds except curl debug builds (DEBUGBUILD). If - after a period of time there are no major issues then it can be enabled - in dev and release builds with the newest OpenSSL (1.1.1+), since users - using the newest OpenSSL are the least likely to have legacy problems. + Reported-by: hamstergene@users.noreply.github.com - Bug: https://github.com/curl/curl/issues/4409#issuecomment-555955794 - Reported-by: Bjoern Franke + Fixes https://github.com/curl/curl/issues/4763 - Fixes https://github.com/curl/curl/issues/4624 - Closes https://github.com/curl/curl/pull/4623 + Closes https://github.com/curl/curl/pull/4765 -- [Daniel Stenberg brought this change] +- [Marc Aldorasi brought this change] - openssl: improve error message for SYSCALL during connect + cmake: Enable SMB for Windows builds - Reported-by: Paulo Roberto Tomasi - Bug: https://curl.haxx.se/mail/archive-2019-11/0005.html + - Define USE_WIN32_CRYPTO by default. This enables SMB. - Closes https://github.com/curl/curl/pull/4593 - -Daniel Stenberg (22 Nov 2019) -- test1175: verify symbols-in-versions and libcurl-errors.3 in sync + - Show whether SMB is enabled in the "Enabled features" output. - Closes #4628 + - Fix mingw compiler warning for call to CryptHashData by casting away + const param. mingw CryptHashData prototype is wrong. + + Closes https://github.com/curl/curl/pull/4717 -- include: make CURLE_HTTP3 use a new error code +- vtls: Refactor Curl_multissl_version to make the code clearer - To avoid potential issues with error code reuse. + Reported-by: Johannes Schindelin - Reported-by: Christoph M. Becker - Assisted-by: Dan Fandrich - Fixes #4601 - Closes #4627 + Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 + + Closes https://github.com/curl/curl/pull/4803 -- bump: next release will be 7.68.0 +Daniel Stenberg (10 Jan 2020) +- fix: Copyright year out of date, should be 2020 + + Follow-up to 875314ed0bf3b -- curl: add --parallel-immediate +Marcel Raad (10 Jan 2020) +- hostip: move code to resolve IP address literals to `Curl_resolv` - Starting with this change when doing parallel transfers, without this - option set, curl will prefer to create new transfers multiplexed on an - existing connection rather than creating a brand new one. + The code was duplicated in the various resolver backends. - --parallel-immediate can be set to tell curl to prefer to use new - connections rather than to wait and try to multiplex. + Also, it was called after the call to `Curl_ipvalid`, which matters in + case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This + caused test 1048 to fail on classic MinGW. - libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default - on parallel transfers. + The code ignores `conn->ip_version` as done previously in the + individual resolver backends. - Suggested-by: Tom van der Woerdt - Closes #4500 + Move the call to the `resolver_start` callback up to appease test 655, + which wants it to be called also for literal addresses. + + Closes https://github.com/curl/curl/pull/4798 -Daniel Gustafsson (20 Nov 2019) -- [Victor Magierski brought this change] +Daniel Stenberg (9 Jan 2020) +- scripts/delta: adapt to new public header layout - docs: fix typos +- test1167: verify global symbols in public headers are curl prefixed - Change 'experiemental' to 'experimental'. + ... using the new badsymbols.pl perl script - Closes #4618 - Reviewed-by: Daniel Gustafsson + Fixes #4793 + Closes #4794 -Jay Satiro (18 Nov 2019) -- projects: Fix Visual Studio wolfSSL configurations - - - s/USE_CYASSL/USE_WOLFSSL/ +- libtest/mk-lib1521: adapt to new public header layout + +- include: remove non-curl prefixed defines - - Remove old compatibility macros. + ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ + enums. + +- curl.h: remove WIN32 define - Follow-up to 1c6c59a from several months ago when CyaSSL named symbols - were renamed to wolfSSL. The wolfSSL library was formerly named CyaSSL - and we kept using their old name for compatibility reasons, until - earlier this year. + It isn't our job to define this in a public header - and it defines a + name outside of our naming scope. -Daniel Stenberg (18 Nov 2019) -- RELEASE-NOTES: synced +- tool_dirhie.c: fix the copyright year range + + Follow-up to: 4027bd72d9 -- [Javier Blazquez brought this change] +- bump: work towards 7.69.0 is started - ngtcp2: use overflow buffer for extra HTTP/3 data +Jay Satiro (9 Jan 2020) +- tool_dirhie: Allow directory traversal during creation - Fixes #4525 - Closes #4603 - -- altsvc: bump to h3-24 + - When creating a directory hierarchy do not error when mkdir fails due + to error EACCESS (13) "access denied". - ... as both ngtcp2 and quiche now support that in their master branches + Some file systems allow for directory traversal; in this case that it + should be possible to create child directories when permission to the + parent directory is restricted. - Closes #4604 - -- ngtcp2: free used resources on disconnect + This is a regression caused by me in f16bed0 (precedes curl-7_61_1). + Basically I had assumed that if a directory already existed it would + fail only with error EEXIST, and not error EACCES. The latter may + happen if the directory exists but has certain restricted permissions. - Fixes #4614 - Closes #4615 - -- ngtcp2: handle key updates as ngtcp2 master branch tells us + Reported-by: mbeifuss@users.noreply.github.com - Reviewed-by: Tatsuhiro Tsujikawa + Fixes https://github.com/curl/curl/issues/4796 + Closes https://github.com/curl/curl/pull/4797 + +Daniel Stenberg (9 Jan 2020) +- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers - Fixes #4612 - Closes #4613 + Closes #4080 -Jay Satiro (17 Nov 2019) -- [Gergely Nagy brought this change] +- docs/RELEASE-PROCEDURE.md: pushed some release dates + + Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html - multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds +- runtests: make random seed fixed for a month - Prior to this change: + When using randomized features of runtests (-R and --shallow) it is + useful to have a fixed random seed to make sure for example extra + commits in a branch or a rebase won't change the seed that would make + repeated runs work differently. - The check if an extra wait is necessary was based not on the - number of extra fds but on the pointer. + As it is also useful to change seed sometimes, the default seed is now + determined based on the current month (and first line curl -V + output). When the month changes, so will the random seed. - If a non-null pointer was given in extra_fds, but extra_nfds - was zero, then the wait was skipped even though poll was not - called. + The specific seed is also shown in the standard test suite top header + and it can be set explictly with the new --seed=[num] option so that the + exact order of a previous run can be achieved. - Closes https://github.com/curl/curl/pull/4610 + Closes #4734 -- lib: Move lib/ssh.h -> lib/vssh/ssh.h - - Follow-up to 5b2d703 which moved ssh source files to vssh. +- RELEASE-PROCEDURE.md: fix next release date (Feb 26) - Closes https://github.com/curl/curl/pull/4609 + [skip ci] -Daniel Stenberg (16 Nov 2019) -- [Andreas Falkenhahn brought this change] +Version 7.68.0 (8 Jan 2020) - INSTALL.md: provide Android build instructions - - Closes #4606 +Daniel Stenberg (8 Jan 2020) +- RELEASE-NOTES: 7.68.0 -- [Niall O'Reilly brought this change] +- THANKS: updated with names from the 7.68.0 release - doh: improced both encoding and decoding - - Improved estimation of expected_len and updated related comments; - increased strictness of QNAME-encoding, adding error detection for empty - labels and names longer than the overall limit; avoided treating DNAME - as unexpected; - - updated unit test 1655 with more thorough set of proofs and tests +- RELEASE-PROCEDURE: add four future release dates - Closes #4598 - -- ngtcp2: increase QUIC window size when data is consumed + and remove four past release dates - Assisted-by: Javier Blazquez - Ref #4525 (partial fix) - Closes #4600 - -- [Melissa Mears brought this change] + [skip ci] - config-win32: cpu-machine-OS for Windows on ARM +Marcel Raad (6 Jan 2020) +- TrackMemory tests: always remove CR before LF - Define the OS macro properly for Windows on ARM builds. Also, we might - as well add the GCC-style IA-64 macro. + It was removed for output containing ' =' via `s/ =.*//`. With classic + MinGW, this made lines with `free()` end with CRLF, but lines with e.g. + `malloc()` end with only LF. The tests expect LF only. - Closes #4590 + Closes https://github.com/curl/curl/pull/4788 -- examples: add multi-poll.c +Daniel Stenberg (6 Jan 2020) +- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header - Show how curl_multi_poll() makes it even easier to use the multi - interface. + ... to the private multihhandle.h. It is not for public use and it + wasn't prefixed correctly anyway! - Closes #4596 + Closes #4790 -- multi_poll: avoid busy-loop when called without easy handles attached +- file: fix copyright year range - Fixes #4594 - Closes #4595 - Reported-by: 3dyd on github + Follow-up to 1b71bc532bd -- curl: fix -T globbing +- curl -w: handle a blank input file correctly - Regression from e59371a4936f8 (7.67.0) + Previously it would end up with an uninitialized memory buffer that + would lead to a crash or junk getting output. - Added test 490, 491 and 492 to verify the functionality. + Added test 1271 to verify. + + Reported-by: Brian Carpenter + Closes #4786 + +- file: on Windows, refuse paths that start with \\ - Reported-by: Kamil Dudka - Reported-by: Anderson Sasaki + ... as that might cause an unexpected SMB connection to a given host + name. - Fixes #4588 - Closes #4591 + Reported-by: Fernando Muñoz + CVE-2019-15601 + Bug: https://curl.haxx.se/docs/CVE-2019-15601.html -- HISTORY: added cmake, HTTP/3 and parallel downloads with curl +Jay Satiro (6 Jan 2020) +- CURLOPT_READFUNCTION.3: fix fopen params in example -- quiche: reject headers in the wrong order +- CURLOPT_READFUNCTION.3: fix variable name in example - Pseudo header MUST come before regular headers or cause an error. + Reported-by: Paul Joyce - Reported-by: Cynthia Coan - Fixes #4571 - Closes #4584 + Fixes https://github.com/curl/curl/issues/4787 -- openssl: prevent recursive function calls from ctx callbacks - - Follow the pattern of many other callbacks. +Daniel Stenberg (5 Jan 2020) +- curl:getparameter return error for --http3 if libcurl doesn't support - Ref: #4546 - Closes #4585 + Closes #4785 -- CURL-DISABLE: initial docs for the CURL_DISABLE_* defines +- docs: mention CURL_MAX_INPUT_LENGTH restrictions - The disable-scan script used in test 1165 is extended to also verify - that the docs cover all used defines and all defines offered by - configure. + ... for curl_easy_setopt() and curl_url_set(). - Reported-by: SLDiggie on github - Fixes #4545 - Closes #4587 + [skip ci] + + Closes #4783 -- remove_handle: clear expire timers after multi_done() +- curl: properly free mimepost data - Since 59041f0, a new timer might be set in multi_done() so the clearing - of the timers need to happen afterwards! + ... as it could otherwise leak memory when a transfer failed. - Reported-by: Max Kellermann - Fixes #4575 - Closes #4583 + Added test 1293 to verify. + + Reported-by: Brian Carpenter + Fixes #4781 + Closes #4782 -Marcel Raad (10 Nov 2019) -- test1558: use double slash after file: +- curl: cleanup multi handle on failure - Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this - test unnecessarily failed when using `file:/` instead of `file:///`. + ... to fix memory leak in error path. - Closes https://github.com/curl/curl/pull/4554 + Fixes #4772 + Closes #4780 + Reported-by: Brian Carpenter -Daniel Stenberg (10 Nov 2019) -- pause: avoid updating socket if done was already called +Marcel Raad (3 Jan 2020) +- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` - ... avoids unnecesary recursive risk when the transfer is already done. + Closes https://github.com/curl/curl/pull/4775 + +Daniel Stenberg (3 Jan 2020) +- COPYING: it's 2020! - Reported-by: Richard Bowker - Fixes #4563 - Closes #4574 + [skip ci] -Jay Satiro (9 Nov 2019) -- strerror: Fix an error looking up some Windows error strings +Jay Satiro (3 Jan 2020) +- [Marc Aldorasi brought this change] + + tests: Fix bounce requests with truncated writes - - Use FORMAT_MESSAGE_IGNORE_INSERTS to ignore format specifiers in - Windows error strings. + Prior to this change the swsbounce check in service_connection could + fail because prevtestno and prevpartno were not set, which would cause + the wrong response data to be sent to some tests and cause them to fail. - Since we are not in control of the error code we don't know what - information may be needed by the error string's format specifiers. + Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 + +Marcel Raad (31 Dec 2019) +- tool: make a few char pointers point to const char instead - Prior to this change Windows API error strings which contain specifiers - (think specifiers like similar to printf specifiers) would not be shown. - The FormatMessage Windows API call which turns a Windows error code into - a string could fail and set error ERROR_INVALID_PARAMETER if that error - string contained a format specifier. FormatMessage expects a va_list for - the specifiers, unless inserts are ignored in which case no substitution - is attempted. + These are read-only. - Ref: https://devblogs.microsoft.com/oldnewthing/20071128-00/?p=24353 - -- [r-a-sattarov brought this change] + Closes https://github.com/curl/curl/pull/4771 - system.h: fix for MCST lcc compiler +Jay Satiro (31 Dec 2019) +- tests: Change NTLM tests to require SSL - Fixed build by MCST lcc compiler on MCST Elbrus 2000 architecture and do - some code cleanup. + Prior to this change tests that required NTLM feature did not require + SSL feature. - e2k (Elbrus 2000) - this is VLIW/EPIC architecture, like Intel Itanium - architecture. + There are pending changes to cmake builds that will allow enabling NTLM + in non-SSL builds in Windows. In that case the NTLM auth strings created + are different from what is expected by the NTLM tests and they fail: - Ref: https://en.wikipedia.org/wiki/Elbrus_2000 + "The issue with NTLM is that previous non-SSL builds would not enable + NTLM and so the NTLM tests would be skipped." - Closes https://github.com/curl/curl/pull/4576 - -Daniel Stenberg (8 Nov 2019) -- TODO: curl_multi_unblock + Assisted-by: marc-groundctl@users.noreply.github.com - Closes #4418 - -- TODO: Run web-platform-tests url tests + Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 - Closes #4477 + Closes https://github.com/curl/curl/pull/4768 -- TODO: 1.4 alt-svc sharing - - Closes #4476 +- [Michael Forney brought this change] -- test1560: require IPv6 for IPv6 aware URL parsing + bearssl: Improve I/O handling - The URL parser function can't reject a bad IPv6 address properly when - curl was built without IPv6 support. + Factor out common I/O loop as bearssl_run_until, which reads/writes TLS + records until the desired engine state is reached. This is now used for + the handshake, read, write, and close. - Reported-by: Marcel Raad - Fixes #4556 - Closes #4572 - -- checksrc: repair the copyrightyear check + Match OpenSSL SSL_write behavior, and don't return the number of bytes + written until the corresponding records have been completely flushed + across the socket. This involves keeping track of the length of data + buffered into the TLS engine, and assumes that when CURLE_AGAIN is + returned, the write function will be called again with the same data + and length arguments. This is the same requirement of SSL_write. - - Consider a modified file to be committed this year. + Handle TLS close notify as EOF when reading by returning 0. - - Make the travis CHECKSRC also do COPYRIGHTYEAR scan in examples and - includes + Closes https://github.com/curl/curl/pull/4748 + +- travis: Fix error detection - - Ignore 0 parents when getting latest commit date of file. + - Stop using inline shell scripts for before_script and script sections. - since in the CI we're dealing with a truncated repo of last 50 commits, - the file's most recent commit may not be available. when this happens - git log and rev-list show the initial commit (ie first commit not to be - truncated) but that's incorrect so ignore it. + Prior to this change Travis could ignore errors from commands in inline + scripts. I don't understand how or why it happens. This is a workaround. - Ref: https://github.com/curl/curl/pull/4547 + Assisted-by: Simon Warta - Closes https://github.com/curl/curl/pull/4549 + Ref: https://github.com/travis-ci/travis-ci/issues/1066 - Co-authored-by: Jay Satiro + Fixes https://github.com/curl/curl/issues/3730 + Closes https://github.com/curl/curl/pull/3755 -- copyrights: fix copyright year range +- tool_operate: fix mem leak when failed config parse - .. because checksrc's copyright year check stopped working. + Found by fuzzing the config file. - Ref: https://github.com/curl/curl/pull/4547 + Reported-by: Geeknik Labs - Closes https://github.com/curl/curl/pull/4549 - -- RELEASE-NOTES: synced - -- curlver: bump to 7.67.1 + Fixes https://github.com/curl/curl/issues/4767 -- mailmap: fixup Massimiliano Fantuzzi +- [Xiang Xiao brought this change] -- scripts/contributors: make committers get included too + lib: remove erroneous +x file permission on some c files - in addition to authors - -Jay Satiro (8 Nov 2019) -- [Massimiliano Fantuzzi brought this change] - - configure: fix typo in help text + Modified by commit eb9a604 accidentally. - Closes https://github.com/curl/curl/pull/4570 + Closes https://github.com/curl/curl/pull/4756 -Daniel Stenberg (7 Nov 2019) -- [Christian Schmitz brought this change] +- [Xiang Xiao brought this change] - ntlm: USE_WIN32_CRYPTO check removed to get USE_NTLM2SESSION set + lib: fix warnings found when porting to NuttX - Closes #3704 - -Jay Satiro (6 Nov 2019) -- [Wyatt O'Day brought this change] - - build: fix for CURL_DISABLE_DOH + - Undefine DEBUGASSERT in curl_setup_once.h in case it was already + defined as a system macro. - Fixes https://github.com/curl/curl/issues/4565 - Closes https://github.com/curl/curl/pull/4566 - -- [Leonardo Taccari brought this change] - - configure: avoid unportable `==' test(1) operator + - Don't compile write32_le in curl_endian unless + CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. - Closes https://github.com/curl/curl/pull/4567 - -Version 7.67.0 (5 Nov 2019) - -Daniel Stenberg (5 Nov 2019) -- RELEASE-NOTES: synced + - Include in socketpair.c. - The 7.67.0 release + Closes https://github.com/curl/curl/pull/4756 -- THANKS: add new names from 7.67.0 +- os400: Add missing CURLE error constants + + Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 + Reported-by: Emil Engler -- configure: only say ipv6 enabled when the variable is set +- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 + + For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback + is passed two `size_t` parameters which, when multiplied, designate the + number of bytes of data passed in. In practice, CURL always sets the + first parameter (`size`) to 1. + + This practice is also enshrined in documentation and cannot be changed + in future. The documentation states that the default callback is + `fwrite`, which means `fwrite` must be a suitable function for this + purpose. However, the documentation also states that the callback must + return the number of *bytes* it successfully handled, whereas ISO C + `fwrite` returns the number of items (each of size `size`) which it + wrote. The only way these numbers can be equal is if `size` is 1. + + Since `size` is 1 and can never be changed in future anyway, document + that fact explicitly and let users rely on it. + + Reported-by: Frank Gevaerts + Commit-message-by: Christopher Head - Previously it could say "IPv6: enabled" at the end of the configure run - but the define wasn't set because of a missing getaddrinfo(). + Ref: https://github.com/curl/curl/pull/2787 - Reported-by: Marcel Raad - Fixes #4555 - Closes #4560 + Fixes https://github.com/curl/curl/issues/4758 -Marcel Raad (2 Nov 2019) -- certs/Server-localhost-lastSAN-sv: regenerate with sha256 +- examples/postinmemory.c: Call curl_global_cleanup always - All other certificates were regenerated in commit ba782baac30, but - this one was missed. - Fixes test3001 on modern systems. + Prior to this change curl_global_cleanup was not called if + curl_easy_init failed. - Closes https://github.com/curl/curl/pull/4551 - -Daniel Stenberg (2 Nov 2019) -- [Vilhelm Prytz brought this change] + Reported-by: kouzhudong@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/4751 - copyrights: update all copyright notices to 2019 on files changed this year +Daniel Stenberg (21 Dec 2019) +- url2file.c: fix copyright year - Closes #4547 + Follow-up to 525787269599b5 -- [Bastien Bouclet brought this change] +- [Rickard Hallerbäck brought this change] - mbedtls: add error message for cert validity starting in the future + examples/url2file.c: corrected a comment - Closes #4552 + The comment was confusing and suggested that setting CURLOPT_NOPROGRESS + to 0L would both enable and disable debug output at the same time, like + a Schrödinger's cat of CURLOPTs. + + Closes #4745 -Jay Satiro (1 Nov 2019) -- schannel_verify: Fix concurrent openings of CA file +- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 + +- RELEASE-NOTES: synced + +Jay Satiro (20 Dec 2019) +- ngtcp2: Support the latest update key callback type - - Open the CA file using FILE_SHARE_READ mode so that others can read - from it as well. + - Remove our cb_update_key in favor of ngtcp2's new + ngtcp2_crypto_update_key_cb which does the same thing. - Prior to this change our schannel code opened the CA file without - sharing which meant concurrent openings (eg an attempt from another - thread or process) would fail during the time it was open without - sharing, which in curl's case would cause error: - "schannel: failed to open CA file". + Several days ago the ngtcp2_update_key callback function prototype was + changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to + fix up our cb_update_key for that change they also added + ngtcp2_crypto_update_key_cb which does the same thing so we'll use that + instead. - Bug: https://curl.haxx.se/mail/lib-2019-10/0104.html - Reported-by: Richard Alcock + Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c + + Closes https://github.com/curl/curl/pull/4735 -Daniel Stenberg (31 Oct 2019) -- gtls: make gnutls_bye() not wait for response on shutdown +Daniel Stenberg (19 Dec 2019) +- sws: search for "Testno:" header uncondtionally if no testno - ... as it can make it wait there for a long time for no good purpose. + Even if the initial request line wasn't found. With the fix to 1455, the + test number is now detected correctly. - Patched-by: Jay Satiro - Reported-by: Bylon2 on github - Adviced-by: Nikos Mavrogiannopoulos + (Problem found when running tests in random order.) - Fixes #4487 - Closes #4541 - -- [Michał Janiszewski brought this change] + Closes #4744 - appveyor: publish artifacts on appveyor +- tests: set LC_ALL in more tests - This allows obtaining upstream builds of curl directly from appveyor for - all the available configurations + Follow-up to 23208e330ac0c21 - Closes #4509 + Closes #4743 -- url: make Curl_close() NULLify the pointer too +- test165: set LC_ALL=en_US.UTF-8 too - This is the common pattern used in the code and by a unified approach we - avoid mistakes. + On my current Debian Unstable with libidn2 2.2.0, I get an error if + LC_ALL is set to blank. Then curl errors out with: - Closes #4534 - -- [Trivikram Kamat brought this change] - - INSTALL: add missing space for configure commands + curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 - Closes #4539 + Closes #4738 -- url: Curl_free_request_state() should also free doh handles +- curl.h: add two defines for the "pre ISO C" case - ... or risk DoH memory leaks. + Without this fix, this caused a compilation failure on AIX with IBM xlc + 13.1.3 compiler. - Reported-by: Paul Dreik - Fixes #4463 - Closes #4527 + Reported-by: Ram Krushna Mishra + Fixes #4739 + Closes #4740 -- examples: remove the "this exact code has not been verified" +- create_conn: prefer multiplexing to using new connections - ... as really confuses the reader to not know what to believe! - -- [Trivikram Kamat brought this change] - - HTTP3: fix typo somehere1 > somewhere1 + ... as it would previously prefer new connections rather than + multiplexing in most conditions! The (now removed) code was a leftover + from the Pipelining code that was translated wrongly into a + multiplex-only world. - Closes #4535 - -Jay Satiro (28 Oct 2019) -- [Javier Blazquez brought this change] + Reported-by: Kunal Ekawde + Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html + Closes #4732 - HTTP3: fix invalid use of sendto for connected UDP socket +- test1456: remove the use of a fixed local port - On macOS/BSD, trying to call sendto on a connected UDP socket fails - with a EISCONN error. Because the singleipconnect has already called - connect on the socket when we're trying to use it for QUIC transfers - we need to use plain send instead. + Fixup the test to instead not compare the port number. It sometimes + caused problems like this: - Fixes #4529 - Closes https://github.com/curl/curl/pull/4533 - -Daniel Stenberg (28 Oct 2019) -- RELEASE-NOTES: synced - -- [Javier Blazquez brought this change] + "curl: (45) bind failed with errno 98: Address already in use" + + Closes #4733 - HTTP3: fix Windows build +Jay Satiro (18 Dec 2019) +- CURLOPT_QUOTE.3: fix typos - The ngtcp2 QUIC backend was using the MSG_DONTWAIT flag for send/recv - in order to perform nonblocking operations. On Windows this flag does - not exist. Instead, the socket must be set to nonblocking mode via - ioctlsocket. + Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man + pages would not compile because a variable name was incorrect. - This change sets the nonblocking flag on UDP sockets used for QUIC on - all platforms so the use of MSG_DONTWAIT is not needed. + Reported-by: Bylon2@users.noreply.github.com - Fixes #4531 - Closes #4532 + Fixes https://github.com/curl/curl/issues/4736 -Marcel Raad (27 Oct 2019) -- appveyor: add --disable-proxy autotools build - - This would have caught issue #3926. +- [Gisle Vanem brought this change] + + strerror: Fix compiler warning "empty expression" - Also make formatting more consistent. + - Remove the final semi-colon in the SEC2TXT() macro definition. - Closes https://github.com/curl/curl/pull/4526 - -Daniel Stenberg (25 Oct 2019) -- appveyor: make winbuilds with DEBUG=no/yes and VS 2015/2017 + Before: #define SEC2TXT(sec) case sec: txt = #sec; break; - ... and invoke "curl -V" once done + After: #define SEC2TXT(sec) case sec: txt = #sec; break - Co-Authored-By: Jay Satiro + Prior to this change SEC2TXT(foo); would generate break;; which caused + the empty expression warning. - Closes #4523 - -- [Francois Rivard brought this change] + Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 - schannel: reverse the order of certinfo insertions +Daniel Stenberg (18 Dec 2019) +- curl/parseconfig: use curl_free() to free memory allocated by libcurl - Fixes #4518 - Closes #4519 + Reported-by: bxac on github + Fixes #4730 + Closes #4731 -Marcel Raad (24 Oct 2019) -- test1591: fix spelling of http feature +- curl/parseconfig: fix mem-leak - The test never got run because the feature name is `http` in lowercase. + When looping, first trying '.curlrc' and then '_curlrc', the function + would not free the first string. - Closes https://github.com/curl/curl/pull/4520 + Closes #4731 -Daniel Stenberg (23 Oct 2019) -- [Michał Janiszewski brought this change] +- CURLOPT_URL.3: "curl supports SMB version 1 (only)" + + [skip ci] - appveyor: Use two parallel compilation on appveyor with CMake +- test1270: a basic -w redirect_url test - Appveyor provides 2 CPUs for each builder[1], make sure to use parallel - compilation, when running with CMake. CMake learned this new option in - version 3.12[2] and the version provided by appveyor is fresh enough. + Closes #4728 + +- HISTORY: the SMB(S) support landed in 2014 + +- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore - Curl doesn't really take that long to build and it is using the slowest - builder available, msbuild, so expect only a moderate improvement in - build times. + It is covered by USE_OPENSSL_ENGINE now. - [1] https://www.appveyor.com/docs/build-environment/ - [2] https://cmake.org/cmake/help/v3.12/release/3.12.html + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 - Closes #4508 + Closes #4725 -- conn-reuse: requests wanting NTLM can reuse non-NTLM connections +- lib: remove ASSIGNWITHINCONDITION exceptions, use our code style - Added test case 338 to verify. + ... even for macros - Reported-by: Daniel Silverstone - Fixes #4499 - Closes #4514 - -Marcel Raad (23 Oct 2019) -- tests: add missing proxy features + Reviewed-by: Daniel Gustafsson + Reviewed-by: Jay Satiro + Reported-by: Jay Satiro + Fixes #4683 + Closes #4722 -Daniel Stenberg (22 Oct 2019) -- RELEASE-NOTES: synced +- tests: make sure checksrc runs on header files too -Marcel Raad (21 Oct 2019) -- tests: use %FILE_PWD for file:// URLs +- Revert "checksrc: fix regexp for ASSIGNWITHINCONDITION" - This way, we always have exactly one slash after the host name, making - the tests pass when curl is compiled with the MSYS GCC. + This reverts commit ba82673dac3e8d00a76aa5e3779a0cb80e7442af. - Closes https://github.com/curl/curl/pull/4512 + Bug: #4683 -- tests: add `connect to non-listen` keywords - - These tests try to connect to ports nothing is listening on. +- KNOWN_BUGS: TLS session cache doesn't work with TFO - Closes https://github.com/curl/curl/pull/4511 + [skip ci] + Closes #4301 -- runtests: get textaware info from curl instead of perl +- KNOWN_BUGS: Connection information when using TCP Fast Open - The MSYS system on Windows can run the test suite for curl built with - any toolset. When built with the MSYS GCC, curl uses Unix line endings, - while it uses Windows line endings when built with the MinGW GCC, and - `^O` reports 'msys' in both cases. Use the curl executable itself to - determine the line endings instead, which reports 'x86_64-pc-msys' when - built with the MSYS GCC. + Also point to #4296 for more details + Closes #4296 + +- KNOWN_BUGS: LDAP on Windows doesn't work - Closes https://github.com/curl/curl/pull/4506 + Closes #4261 -Daniel Stenberg (20 Oct 2019) -- [Michał Janiszewski brought this change] +- docs: TLS SRP doesn't work with TLS 1.3 + + Reported-by: sayrer on github + Closes #4262 + [skip ci] - appveyor: Add MSVC ARM64 build +Dan Fandrich (16 Dec 2019) +- cirrus: Switch to the FreeBSD 12.1 point release & enable more tests. - Closes #4507 + A few tests are now passing on FreeBSD, so no longer skip them. + [skip ci] -- http2_recv: a closed stream trumps pause state +Daniel Stenberg (16 Dec 2019) +- azure: the macos cmake doesn't need to install cmake - ... and thus should return 0, not EAGAIN. + Error: cmake 3.15.5 is already installed + To upgrade to 3.16.1, run `brew upgrade cmake`. - Reported-by: Tom van der Woerdt - Fixes #4496 - Closes #4505 + Closes #4723 -- http2: expire a timeout at end of stream +Jay Satiro (15 Dec 2019) +- winbuild: Document CURL_STATICLIB requirement for static libcurl + + A static libcurl (ie winbuild mode=static) requires that the user define + CURL_STATICLIB when using it in their application. This is already + covered in the FAQ and INSTALL.md, but is a pretty important point so + now it's noted in the BUILD.WINDOWS.txt as well. - To make sure that transfer is being dealt with. Streams without - Content-Length need a final read to notice the end-of-stream state. + Assisted-by: Michael Vittiglio - Reported-by: Tom van der Woerdt - Fixes #4496 + Closes https://github.com/curl/curl/pull/4721 -Dan Fandrich (18 Oct 2019) -- travis: Add an ARM64 build - - Test 323 is failing for some reason, so disable it there for now. +Daniel Stenberg (15 Dec 2019) +- [Santino Keupp brought this change] -Marcel Raad (18 Oct 2019) -- examples/sslbackend: fix -Wchar-subscripts warning - - With the `isdigit` implementation that comes with MSYS2, the argument - is used as an array subscript, resulting in a -Wchar-subscripts - warning. `isdigit`'s behavior is undefined if the argument is negative - and not EOF [0]. As done in lib/curl_ctype.h, cast the `char` variable - to `unsigned char` to avoid that. + libssh2: add support for ECDSA and ed25519 knownhost keys - [0] https://en.cppreference.com/w/c/string/byte/isdigit + ... if a new enough libssh2 version is present. - Closes https://github.com/curl/curl/pull/4503 + Source: https://curl.haxx.se/mail/archive-2019-12/0023.html + Co-Authored-by: Daniel Stenberg + Closes #4714 -Daniel Stenberg (18 Oct 2019) -- configure: remove all cyassl references +- lib1591: free memory properly on OOM, in the trailers callback - In particular, this removes the case where configure would find an old - cyall installation rather than a wolfssl one if present. The library is - named wolfssl in modern days so there's no real need to keep support for - the former. + Detected by torture tests. - Reported-by: Jacob Barthelmeh - Closes #4502 + Closes #4720 -Marcel Raad (17 Oct 2019) -- test1162: disable MSYS2's POSIX path conversion +- runtests: --repeat=[num] to repeat tests - This avoids MSYS2 converting the backslasb in the URL to a slash, - causing the test to fail. + Closes #4715 -Daniel Stenberg (17 Oct 2019) - RELEASE-NOTES: synced -Jay Satiro (16 Oct 2019) -- CURLOPT_TIMEOUT.3: Clarify transfer timeout time includes queue time - - Prior to this change some users did not understand that the "request" - starts when the handle is added to the multi handle, or probably they - did not understand that some of those transfers may be queued and that - time is included in timeout. +- azure: add a torture test on mac - Reported-by: Jeroen Ooms + Uses --shallow=25 to keep it small enough to get through in time. - Fixes https://github.com/curl/curl/issues/4486 - Closes https://github.com/curl/curl/pull/4489 - -- [Stian Soiland-Reyes brought this change] + Closes #4712 - tool_operate: Fix retry sleep time shown to user when Retry-After +- multi: free sockhash on OOM - - If server header Retry-After is being used for retry sleep time then - show that value to the user instead of the normal retry sleep time. + This would otherwise leak memory in the error path. - This is a follow-up to 640b973 (7.66.0) which changed curl tool so that - the value from Retry-After header overrides other retry timing options. + Detected by torture test 1540. - Closes https://github.com/curl/curl/pull/4498 + Closes #4713 -Daniel Stenberg (16 Oct 2019) -- url: normalize CURLINFO_EFFECTIVE_URL - - The URL extracted with CURLINFO_EFFECTIVE_URL was returned as given as - input in most cases, which made it not get a scheme prefixed like before - if the URL was given without one, and it didn't remove dotdot sequences - etc. - - Added test case 1907 to verify that this now works as intended and as - before 7.62.0. +Marcel Raad (13 Dec 2019) +- tests: use DoH feature for DoH tests - Regression introduced in 7.62.0 + Previously, http/2 was used instead. - Reported-by: Christophe Dervieux - Fixes #4491 - Closes #4493 + Assisted-by: Jay Satiro + Closes https://github.com/curl/curl/pull/4692 -Marcel Raad (16 Oct 2019) -- tests: line ending fixes for Windows +- hostip: suppress compiler warning - Mark some files as text. + With `--disable-doh --disable-threaded-resolver`, the `dns` parameter + is not used. - Closes https://github.com/curl/curl/pull/4490 + Closes https://github.com/curl/curl/pull/4692 -- tests: use proxy feature - - This makes the tests succeed when using --disable-proxy. +- tests: fix build with `CURL_DISABLE_DOH` - Closes https://github.com/curl/curl/pull/4488 + Closes https://github.com/curl/curl/pull/4692 -- smbserver: fix Python 3 compatibility +Daniel Stenberg (13 Dec 2019) +- azure: add a torture test - Python 2's `ConfigParser` module is spelled `configparser` in Python 3. + Skipping all FTP tests for speed reasons. - Closes https://github.com/curl/curl/pull/4484 + Closes #4697 -- security: silence conversion warning - - With MinGW-w64, `curl_socket_t` is is a 32 or 64 bit unsigned integer, - while `read` expects a 32 bit signed integer. - Use `sread` instead of `read` to use the correct parameter type. - - Closes https://github.com/curl/curl/pull/4483 +- azure: make the default build use --enable-debug --enable-werror -- connect: silence sign-compare warning +- ntlm_wb: fix double-free in OOM - With MinGW-w64 using WinSock, `curl_socklen_t` is signed, while the - result of `sizeof` is unsigned. + Detected by torture testing test 1310 - Closes https://github.com/curl/curl/pull/4483 + Closes #4710 -Daniel Stenberg (13 Oct 2019) -- TODO: Handle growing SFTP files +Dan Fandrich (13 Dec 2019) +- cirrus: Drop the FreeBSD 10.4 build - Closes #4344 + Upstream support for 10.4 ended a year ago, and it looks like the image + is now gone, too. + [skip ci] -- KNOWN_BUGS: remove "CURLFORM_CONTENTLEN in an array" +Daniel Stenberg (13 Dec 2019) +- unit1620: fix bad free in OOM - The curl_formadd() function is deprecated and shouldn't be used so the - real fix for applications is to switch to the curl_mime_* API. + Closes #4709 -- KNOWN_BUGS: "LDAP on Windows does authentication wrong" +- unit1609: fix mem-leak in OOM - Closes #3116 + Closes #4709 -- appveyor: add a winbuild that uses VS2017 +- unit1607: fix mem-leak in OOM - Closes #4482 - -- [Harry Sintonen brought this change] + Closes #4709 - socketpair: fix include and define for older TCP header systems - - fixed build for systems that need netinet/in.h for IPPROTO_TCP and are - missing INADDR_LOOPBACK +- lib1559: fix mem-leak in OOM - Closes #4480 + Closes #4709 -- socketpair: fix double-close in error case +- lib1557: fix mem-leak in OOM - Follow-up to bc2dbef0afc08 - -- gskit: use the generic Curl_socketpair - -- asyn-thread: make use of Curl_socketpair() where available + Closes #4709 -- socketpair: an implemention for Windows and more +- altsvc: make the save function ignore NULL filenames - Curl_socketpair() is designed to be used and work everywhere if there's - no native version or the native version isn't good enough. + It might happen in OOM situations. Detected bv torture tests. - Closes #4466 - -- RELEASE-NOTES: synced + Closes #4707 -- connect: return CURLE_OPERATION_TIMEDOUT for errno == ETIMEDOUT - - Previosly all connect() failures would return CURLE_COULDNT_CONNECT, no - matter what errno said. - - This makes for example --retry work on these transfer failures. +- curl: fix memory leak in OOM in etags logic - Reported-by: Nathaniel J. Smith - Fixes #4461 - Clsoes #4462 - -- cirrus: switch off blackhole status on the freebsd CI machines - -- tests: use port 2 instead of 60000 for a safer non-listening port + Detected by torture tests - ... when the tests want "connection refused". + Closes #4706 -- KNOWN_BUGS: IDN tests failing on Windows +- doh: make it behave when built without proxy support - Closes #3747 - -Dan Fandrich (9 Oct 2019) -- cirrus: Increase the git clone depth. + Reported-by: Marcel Raad + Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734 - If more commits are submitted to master between the time of triggering - the first Cirrus build and the time the final build gets started, the - desired commit is no longer at HEAD and the build will error out. - [skip ci] - -Daniel Stenberg (9 Oct 2019) -- docs: make sure the --no-progress-meter docs file is in dist too + Closes #4704 -- docs: document it as --no-progress-meter instead of the reverse +- curl: improved cleanup in upload error path - Follow-up to 93373a960c3bb4 + Memory leak found by torture test 58 - Reported-by: infinnovation-dev on github - Fixes #4474 - Closes #4475 + Closes #4705 + +- mailmap: fix Andrew Ishchuk -Dan Fandrich (9 Oct 2019) -- cirrus: Switch the FreeBSD 11.x build to 11.3 and add a 13.0 build. +- travis: make torture use --shallow=40 - Also, select the images using image_family to get the latest snapshots - automatically. - [skip ci] + As a first step to enable it to run over a more diverse set of tests in + a reasonable time. -Daniel Stenberg (8 Oct 2019) -- curl: --no-progress-meter +- runtests: introduce --shallow to reduce huge torture tests - New option that allows a user to ONLY switch off curl's progress meter - and leave everything else in "talkative" mode. + When set, shallow mode limits runtests -t to make no more than NUM fails + per test case. If more are found, it will randomly discard entries until + the number is right. The random seed can also be set. - Reported-by: Piotr Komborski - Fixes #4422 - Closes #4470 - -- TODO: Consult %APPDATA% also for .netrc + This is particularly useful when running MANY tests as then most torture + failures will already fail the same functions over and over and make the + total operation painfully tedious. - Closes #4016 + Closes #4699 -- CURLOPT_TIMEOUT.3: remove the mention of "minutes" +- conncache: CONNECT_ONLY connections assumed always in-use - ... just say that limiting operations risk aborting otherwise fine - working transfers. If that means seconds, minutes or hours, we leave to - the user. + This makes them never to be considered "the oldest" to be discarded when + reaching the connection cache limit. The reasoning here is that + CONNECT_ONLY is primarily used in combination with using the + connection's socket post connect and since that is used outside of + curl's knowledge we must assume that it is in use until explicitly + closed. - Reported-by: Martin Gartner - Closes #4469 + Reported-by: Pavel Pavlov + Reported-by: Pavel Löbl + Fixes #4426 + Fixes #4369 + Closes #4696 -- [Andrei Valeriu BICA brought this change] +- [Gisle Vanem brought this change] - docs: added multi-event.c example + vtls: make BearSSL possible to set with CURL_SSL_BACKEND - Similar to multi-uv.c but using libevent 2. This is a simpler libevent - integration example then hiperfifo.c. + Ref: https://github.com/curl/curl/commit/9b879160df01e7ddbb4770904391d3b74114302b#commitcomment-36355622 - Closes #4471 + Closes #4698 -Jay Satiro (5 Oct 2019) -- [Nicolas brought this change] +- RELEASE-NOTES: synced - ldap: fix OOM error on missing query string +- travis: remove "coverage", make it "torture" - - Allow missing queries, don't return NO_MEMORY error in such a case. + The coveralls service and test coverage numbers are just too unreliable. + Removed badge from README.md as well. - It is acceptable for there to be no specified query string, for example: + Fixes #4694 + Closes #4695 + +- azure: add libssh2 and cmake macos builds - curl ldap://ldap.forumsys.com + Removed the macos libssh2 build from travis - A regression bug in 1b443a7 caused this issue. + Closes #4686 + +- curl: use errorf() better - This is a partial fix for #4261. + Change series of error outputs to use errorf(). - Bug: https://github.com/curl/curl/issues/4261#issuecomment-525543077 - Reported-by: Jojojov@users.noreply.github.com - Analyzed-by: Samuel Surtees + Only errors that are due to mistakes in command line option usage should + use helpf(), other types of errors in the tool should rather use + errorf(). - Closes https://github.com/curl/curl/pull/4467 + Closes #4691 -- [Paul B. Omta brought this change] +Jay Satiro (9 Dec 2019) +- [Marc Hoersken brought this change] - build: Remove unused HAVE_LIBSSL and HAVE_LIBCRYPTO defines + tests: make it possible to set executable extensions - Closes https://github.com/curl/curl/pull/4460 - -Daniel Stenberg (5 Oct 2019) -- RELEASE-NOTES: synced - -- [Stian Soiland-Reyes brought this change] - - curl: ensure HTTP 429 triggers --retry + This enables the use of Windows Subsystem for Linux (WSL) to run the + testsuite against Windows binaries while using Linux servers. - This completes #3794. + This commit introduces the following environment variables: + - CURL_TEST_EXE_EXT: set the executable extension for all components + - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only + - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only - Also make sure the new tests from #4195 are enabled + Later testcurl.pl could be adjusted to make use of those variables. + - CURL_TEST_EXE_EXT_SRV: set it for the test servers only - Closes #4465 - -Marcel Raad (4 Oct 2019) -- [apique brought this change] - - winbuild: add ENABLE_UNICODE option + (This is one of several commits to support use of WSL for the tests.) - Fixes https://github.com/curl/curl/issues/4308 - Closes https://github.com/curl/curl/pull/4309 + Closes https://github.com/curl/curl/pull/3899 -Daniel Stenberg (4 Oct 2019) -- ngtcp2: adapt to API change - - Closes #4457 +- [Marc Hoersken brought this change] -- cookies: change argument type for Curl_flush_cookies - - The second argument is really a 'bool' so use that and pass in TRUE/FALSE - to make it clear. + tests: fix permissions of ssh keys in WSL - Closes #4455 - -- http2: move state-init from creation to pre-transfer + Keys created on Windows Subsystem for Linux (WSL) require it for some + reason. - To make sure that the HTTP/2 state is initialized correctly for - duplicated handles. It would otherwise easily generate "spurious" - PRIORITY frames to get sent over HTTP/2 connections when duplicated easy - handles were used. + (This is one of several commits to support use of WSL for the tests.) - Reported-by: Daniel Silverstone - Fixes #4303 - Closes #4442 + Ref: https://github.com/curl/curl/pull/3899 -- urlapi: fix use-after-free bug - - Follow-up from 2c20109a9b5d04 +- [Marc Hoersken brought this change] + + tests: use \r\n for log messages in WSL - Added test 663 to verify. + Bash in Windows Subsystem for Linux (WSL) requires it for some reason. - Reported by OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/17954 + (This is one of several commits to support use of WSL for the tests.) - Closes #4453 + Ref: https://github.com/curl/curl/pull/3899 -- [Paul Dreik brought this change] +- [Andrew Ishchuk brought this change] - cookie: avoid harmless use after free + winbuild: Define CARES_STATICLIB when WITH_CARES=static - This fix removes a use after free which can be triggered by - the internal cookie fuzzer, but otherwise is probably - impossible to trigger from an ordinary application. + When libcurl is built with MODE=static, c-ares is forced into static + linkage too. That doesn't happen when MODE=dll so linker would break + over undefined symbols. - The following program reproduces it: + closes https://github.com/curl/curl/pull/4688 + +Daniel Stenberg (9 Dec 2019) +- conn: always set bits.close with connclose() - curl_global_init(CURL_GLOBAL_DEFAULT); - CURL* handle=curl_easy_init(); - CookieInfo* info=Curl_cookie_init(handle,NULL,NULL,false); - curl_easy_setopt(handle, CURLOPT_COOKIEJAR, "/dev/null"); - Curl_flush_cookies(handle, true); - Curl_cookie_cleanup(info); - curl_easy_cleanup(handle); - curl_global_cleanup(); + Closes #4690 + +- cirrus: enable clang sanitizers on freebsd 13 + +- conncache: fix multi-thread use of shared connection cache - This was found through fuzzing. + It could accidentally let the connection get used by more than one + thread, leading to double-free and more. - Closes #4454 + Reported-by: Christopher Reid + Fixes #4544 + Closes #4557 -- [Denis Chaplygin brought this change] +- azure: add a vanilla macos build + + Closes #4685 - docs: add note on failed handles not being counted by curl_multi_perform +- curl: make the etag load logic work without fseek + + The fseek()s were unnecessary and caused Coverity warning CID 1456554 - Closes #4446 + Closes #4681 -- CURLMOPT_MAX_CONCURRENT_STREAMS.3: fix SEE ALSO typo +- mailmap: Mohammad Hasbini -- [Niall O'Reilly brought this change] +- [Mohammad Hasbini brought this change] - ESNI: initial build/setup + docs: fix some typos - Closes #4011 + Closes #4680 - RELEASE-NOTES: synced -- redirect: when following redirects to an absolute URL, URL encode it +Jay Satiro (5 Dec 2019) +- lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN - ... to make it handle for example (RFC violating) embeded spaces. + Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS + and OS400 package spec. - Reported-by: momala454 on github - Fixes #4445 - Closes #4447 - -- urlapi: fix URL encoding when setting a full URL - -- tool_operate: rename functions to make more sense - -- curl: create easy handles on-demand and not ahead of time + Also I added the option to the NameValue list in the tool even though it + isn't exposed as a command-line option (...yet?). (NameValue stringizes + the option name for the curl cmd -> libcurl source generator) - This should again enable crazy-large download ranges of the style - [1-10000000] that otherwise easily ran out of memory starting in 7.66.0 - when this new handle allocating scheme was introduced. + Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN. - Reported-by: Peter Sumatra - Fixes #4393 - Closes #4438 + Ref: https://github.com/curl/curl/pull/4655 -- [Kunal Ekawde brought this change] +- setopt: Fix ALPN / NPN user option when built without HTTP2 + + - Stop treating lack of HTTP2 as an unknown option error result for + CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN. + + Prior to this change it was impossible to disable ALPN / NPN if libcurl + was built without HTTP2. Setting either option would result in + CURLE_UNKNOWN_OPTION and the respective internal option would not be + set. That was incorrect since ALPN and NPN are used independent of + HTTP2. + + Reported-by: Shailesh Kapse + + Fixes https://github.com/curl/curl/issues/4668 + Closes https://github.com/curl/curl/pull/4672 - CURLMOPT_MAX_CONCURRENT_STREAMS: new setopt +Daniel Stenberg (5 Dec 2019) +- etag: allow both --etag-compare and --etag-save in same cmdline - Closes #4410 + Fixes #4669 + Closes #4678 -- chunked-encoding: stop hiding the CURLE_BAD_CONTENT_ENCODING error +Marcel Raad (5 Dec 2019) +- curl_setup: fix `CURLRES_IPV6` condition - Unknown content-encoding would get returned as CURLE_WRITE_ERROR if the - response is chunked-encoded. + Move the definition of `CURLRES_IPV6` to before undefining + `HAVE_GETADDRINFO`. Regression from commit 67a08dca27a which caused + some tests to fail and others to be skipped with c-ares. - Reported-by: Ilya Kosarev - Fixes #4310 - Closes #4449 + Fixes https://github.com/curl/curl/issues/4673 + Closes https://github.com/curl/curl/pull/4677 + +Daniel Stenberg (5 Dec 2019) +- test342: make it return a 304 as the tag matches -Marcel Raad (1 Oct 2019) -- checksrc: fix uninitialized variable warning +Peter Wu (4 Dec 2019) +- CMake: add support for building with the NSS vtls backend - The loop doesn't need to be executed without a file argument. + Options are cross-checked with configure.ac and acinclude.m4. + Tested on Arch Linux, untested on other platforms like Windows or macOS. - Closes https://github.com/curl/curl/pull/4444 + Closes #4663 + Reviewed-by: Kamil Dudka -- urlapi: fix unused variable warning +Daniel Stenberg (4 Dec 2019) +- azure: add more builds - `dest` is only used with `ENABLE_IPV6`. + ... removed two from travis (that now runs on azure instead) - Closes https://github.com/curl/curl/pull/4444 + Closes #4671 -- lib: silence conversion warnings - - Closes https://github.com/curl/curl/pull/4444 +- CURLOPT_VERBOSE.3: see also ERRORBUFFER + +- hostip4.c: bump copyright year range -- AppVeyor: add 32-bit MinGW-w64 build +Marcel Raad (3 Dec 2019) +- configure: enable IPv6 support without `getaddrinfo` - With WinSSL and testing enabled so that it would have detected most of - the warnings fixed in [0] and [1]. + This makes it possible to recognize and connect to literal IPv6 + addresses when `getaddrinfo` is not available, which is already the + case for the CMake build. This affects e.g. classic MinGW because it + still targets Windows 2000 by default, where `getaddrinfo` is not + available, but general IPv6 support is. - [0] https://github.com/curl/curl/pull/4398 - [1] https://github.com/curl/curl/pull/4415 + Instead of checking for `getaddrinfo`, check for `sockaddr_in6` as the + CMake build does. - Closes https://github.com/curl/curl/pull/4433 + Closes https://github.com/curl/curl/pull/4662 -- AppVeyor: remove MSYS2_ARG_CONV_EXCL for winbuild +- curl_setup: disable IPv6 resolver without `getaddrinfo` - It's only used for MSYS2 with MinGW. + Also, use `CURLRES_IPV6` only for actual DNS resolution, not for IPv6 + address support. This makes it possible to connect to IPv6 literals by + setting `ENABLE_IPV6` even without `getaddrinfo` support. It also fixes + the CMake build when using the synchronous resolver without + `getaddrinfo` support. - Closes + Closes https://github.com/curl/curl/pull/4662 -Daniel Stenberg (30 Sep 2019) -- [Emil Engler brought this change] +Daniel Stenberg (3 Dec 2019) +- github action/azure pipeline: run 'make test-nonflaky' for tests + + To match travis and give more info on failures. - git: add tests/server/disabled to .gitignore +- openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains - Closes #4441 + Closes #4655 -- altsvc: accept quoted ma and persist values +- openssl: set X509_V_FLAG_PARTIAL_CHAIN - As mandated by the spec. Test 1654 is extended to verify. + Have intermediate certificates in the trust store be treated as + trust-anchors, in the same way as self-signed root CA certificates + are. This allows users to verify servers using the intermediate cert + only, instead of needing the whole chain. - Closes #4443 - -- mailmap: a Lucas fix - -Alessandro Ghedini (29 Sep 2019) -- [Lucas Pardue brought this change] - - quiche: update HTTP/3 config creation to new API - -Daniel Stenberg (29 Sep 2019) -- BINDINGS: PureBasic, Net::Curl for perl and Nim - -- BINDINGS: Kapito is an Erlang library, basically a binding - -- BINDINGS: added clj-curl + Other TLS backends already accept partial chains. - Reported-by: Lucas Severo - -- [Jay Satiro brought this change] + Reported-by: Jeffrey Walton + Bug: https://curl.haxx.se/mail/lib-2019-11/0094.html - docs: disambiguate CURLUPART_HOST is for host name (ie no port) +- curl: show better error message when no homedir is found - Closes #4424 + Reported-by: Vlastimil Ovčáčík + Fixes #4644 + Closes #4665 -- cookies: using a share with cookies shouldn't enable the cookie engine - - The 'share object' only sets the storage area for cookies. The "cookie - engine" still needs to be enabled or activated using the normal cookie - options. +- OPENSOCKETFUNCTION.3: correct the purpose description - This caused the curl command line tool to accidentally use cookies - without having been told to, since curl switched to using shared cookies - in 7.66.0. + Reported-by: Jeff Mears + Bug: https://curl.haxx.se/mail/lib-2019-12/0007.html - Test 1166 verifies + Closes #4667 + +- [Peter Wu brought this change] + + travis: do not use OVERRIDE_CC or OVERRIDE_CXX if empty - Updated test 506 + Fixes the macOS builds where OVERRIDE_CC and OVERRIDE_CXX are not set. - Fixes #4429 - Closes #4434 + Reported-by: Jay Satiro + Fixes #4659 + Closes #4661 + Closes #4664 -- setopt: handle ALTSVC set to NULL +- azure-pipelines: fix the test script -- RELEASE-NOTES: synced +- Azure Pipelines: initial CI setup + + [skip ci] -- [grdowns brought this change] +- docs: add "added: 7.68.0" to the --etag-* docs - INSTALL: add vcpkg installation instructions +- copyright: fix the year ranges for two files - Closes #4435 - -- [Zenju brought this change] + Follow-up to 9c1806ae - FTP: add test for FTPFILE_NOCWD: Avoid redundant CWDs +Jay Satiro (1 Dec 2019) +- build: Disable Visual Studio warning "conditional expression is constant" - Add libtest 661 + - Disable warning C4127 "conditional expression is constant" globally + in curl_setup.h for when building with Microsoft's compiler. - Closes #4417 - -- [Zenju brought this change] - - FTP: url-decode path before evaluation + This mainly affects building with the Visual Studio project files found + in the projects dir. - Closes #4428 - -Marcel Raad (27 Sep 2019) -- tests: fix narrowing conversion warnings + Prior to this change the cmake and winbuild build systems already + disabled 4127 globally for when building with Microsoft's compiler. + Also, 4127 was already disabled for all build systems in the limited + circumstance of the WHILE_FALSE macro which disabled the warning + specifically for while(0). This commit removes the WHILE_FALSE macro and + all other cruft in favor of disabling globally in curl_setup. - `timediff_t` is 64 bits wide also on 32-bit systems since - commit b1616dad8f0. + Background: - Closes https://github.com/curl/curl/pull/4415 - -Jay Satiro (27 Sep 2019) -- [Julian Z brought this change] - - vtls: Fix comment typo about macosx-version-min compiler flag + We have various macros that cause 0 or 1 to be evaluated, which would + cause warning C4127 in Visual Studio. For example this causes it: - Closes https://github.com/curl/curl/pull/4425 - -Daniel Stenberg (26 Sep 2019) -- [Yechiel Kalmenson brought this change] - - README: minor grammar fix + #define Curl_resolver_asynch() 1 - Closes #4431 - -- [Spezifant brought this change] - - HTTP3: fix prefix parameter for ngtcp2 build + Full behavior is not clearly defined and inconsistent across versions. + However it is documented that since VS 2015 Update 3 Microsoft has + addressed this somewhat but not entirely, not warning on while(true) for + example. - Closes #4430 - -- quiche: don't close connection at end of stream! - -- quiche: set 'drain' when returning without having drained the queues - -- Revert "FTP: url-decode path before evaluation" + Prior to this change some C4127 warnings occurred when I built with + Visual Studio using the generated projects in the projects dir. - This reverts commit 2f036a72d543e96128bd75cb0fedd88815fd42e2. - -- HTTP3: merged and simplified the two 'running' sections - -- HTTP3: show an --alt-svc using example too - -- [Zenju brought this change] + Closes https://github.com/curl/curl/pull/4658 - FTP: url-decode path before evaluation +- openssl: retrieve reported LibreSSL version at runtime - Closes #4423 - -- openssl: use strerror on SSL_ERROR_SYSCALL + - Retrieve LibreSSL runtime version when supported (>= 2.7.1). - Instead of showing the somewhat nonsensical errno number, use strerror() - to provide a more relatable error message. + For earlier versions we continue to use the compile-time version. - Closes #4411 - -- HTTP3: update quic.aiortc.org + add link to server list + Ref: https://man.openbsd.org/OPENSSL_VERSION_NUMBER.3 - Reported-by: Jeremy Lainé + Closes https://github.com/curl/curl/pull/2425 -Jay Satiro (26 Sep 2019) -- url: don't set appconnect time for non-ssl/non-ssh connections - - Prior to this change non-ssl/non-ssh connections that were reused set - TIMER_APPCONNECT [1]. Arguably that was incorrect since no SSL/SSH - handshake took place. +- strerror: Add Curl_winapi_strerror for Win API specific errors - [1]: TIMER_APPCONNECT is publicly known as CURLINFO_APPCONNECT_TIME in - libcurl and %{time_appconnect} in the curl tool. It is documented as - "the time until the SSL/SSH handshake is completed". + - In all code call Curl_winapi_strerror instead of Curl_strerror when + the error code is known to be from Windows GetLastError. - Reported-by: Marcel Hernandez + Curl_strerror prefers CRT error codes (errno) over Windows API error + codes (GetLastError) when the two overlap. When we know the error code + is from GetLastError it is more accurate to prefer the Windows API error + messages. - Ref: https://github.com/curl/curl/issues/3760 + Reported-by: Richard Alcock - Closes https://github.com/curl/curl/pull/3773 + Fixes https://github.com/curl/curl/issues/4550 + Closes https://github.com/curl/curl/pull/4581 -Daniel Stenberg (25 Sep 2019) -- ngtcp2: remove fprintf() calls +Daniel Stenberg (2 Dec 2019) +- global_init: undo the "intialized" bump in case of failure - - convert some of them to H3BUF() calls to infof() - - remove some of them completely - - made DEBUG_HTTP3 defined only if CURLDEBUG is set for now + ... so that failures in the global init function don't count as a + working init and it can then be called again. - Closes #4421 - -- [Jay Satiro brought this change] + Reported-by: Paul Groke + Fixes #4636 + Closes #4653 - url: fix the NULL hostname compiler warning case +- parsedate: offer a getdate_capped() alternative - Closes #4403 - -- [Jay Satiro brought this change] - - travis: move the go install to linux-only + ... and use internally. This function will return TIME_T_MAX instead of + failure if the parsed data is found to be larger than what can be + represented. TIME_T_MAX being the largest value curl can represent. - ... to repair the build again - Closes #4403 - -- altsvc: correct the #ifdef for the ngtcp2 backend + Reviewed-by: Daniel Gustafsson + Reported-by: JanB on github + Fixes #4152 + Closes #4651 -- altsvc: save h3 as h3-23 +- docs: add more references to curl_multi_poll - Follow-up to d176a2c7e5 + Fixes #4643 + Closes #4652 -- urlapi: question mark within fragment is still fragment - - The parser would check for a query part before fragment, which caused it - to do wrong when the fragment contains a question mark. - - Extended test 1560 to verify. +- sha256: bump the copyright year range - Reported-by: Alex Konev - Fixes #4412 - Closes #4413 - -- [Alex Samorukov brought this change] + Follow-up from 66e21520f - HTTP3.md: move -p for mkdir, remove -j for make - - - mkdir on OSX/Darwin requires `-p` argument before dir +Daniel Gustafsson (28 Nov 2019) +- curl_setup_once: consistently use WHILE_FALSE in macros - - portabbly figuring out number of cores is an exercise for somewhere - else + The WHILE_FALSE construction is used to avoid compiler warnings in + macro constructions. This fixes a few instances where it was not + used in order to keep the code consistent. - Closes #4407 + Closes #4649 + Reviewed-by: Daniel Stenberg -Patrick Monnerat (24 Sep 2019) -- os400: getpeername() and getsockname() return ebcdic AF_UNIX sockaddr, +Daniel Stenberg (28 Nov 2019) +- [Steve Holme brought this change] + + http_ntlm: Remove duplicate NSS initialisation - As libcurl now uses these 2 system functions, wrappers are needed on os400 - to convert returned AF_UNIX sockaddrs to ascii. + Given that this is performed by the NTLM code there is no need to + perform the initialisation in the HTTP layer. This also keeps the + initialisation the same as the SASL based protocols and also fixes a + possible compilation issue if both NSS and SSPI were to be used as + multiple SSL backends. - This is a follow-up to commit 7fb54ef. - See also #4037. - Closes #4214 - -Jay Satiro (24 Sep 2019) -- [Lucas Pardue brought this change] + Reviewed-by: Kamil Dudka + Closes #3935 - strcase: fix raw lowercasing the letter X +Daniel Gustafsson (28 Nov 2019) +- checksrc: fix regexp for ASSIGNWITHINCONDITION - Casing mistake in Curl_raw_tolower 'X' wasn't lowercased as 'x' prior to - this change. + The regexp looking for assignments within conditions was too greedy + and matched a too long string in the case of multiple conditionals + on the same line. This is basically only a problem in single line + macros, and the code which exemplified this was essentially: - Follow-up to 0023fce which added the function several days ago. + do { if((x) != NULL) { x = NULL; } } while(0) - Ref: https://github.com/curl/curl/pull/4401#discussion_r327396546 + ..where the final parenthesis of while(0) matched the regexp, and + the legal assignment in the block triggered the warning. Fix by + making the regexp less greedy by matching for the tell-tale signs + of the if statement ending. - Closes https://github.com/curl/curl/pull/4408 - -Daniel Stenberg (23 Sep 2019) -- http2: Expression 'stream->stream_id != - 1' is always true + Also remove the one occurrence where the warning was disabled due + to a construction like the above, where the warning didn't apply + when fixed. - PVS-Studio warning - Fixes #4402 + Closes #4647 + Reviewed-by: Daniel Stenberg -- http2: A value is being subtracted from the unsigned variable - - PVS-Studio warning - Fixes #4402 +Daniel Stenberg (28 Nov 2019) +- RELEASE-NOTES: synced -- libssh: part of conditional expression is always true: !result - - PVS-Studio warning - Fixed #4402 +- [Maros Priputen brought this change] -- libssh: part of conditional expression is always true + curl: two new command line options for etags - PVS-Studio warning - Fixes #4402 - -- libssh: The expression is excessive or contains a misprint + --etag-compare and --etag-save - PVS-Studio warning - Fixes #4402 + Suggested-by: Paul Hoffman + Fixes #4277 + Closes #4543 -- quiche: The expression must be surrounded by parentheses - - PVS-Studio warning - Fixes #4402 +Daniel Gustafsson (28 Nov 2019) +- docs: fix typos -- vauth: The parameter 'status' must be surrounded by parentheses - - PVS-Studio warning - Fixes #4402 +Daniel Stenberg (28 Nov 2019) +- mailmap: Niall O'Reilly's name -- [Paul Dreik brought this change] +- [Niall O'Reilly brought this change] - doh: allow only http and https in debug mode + doh: use dedicated probe slots - Otherwise curl may be told to use for instance pop3 to - communicate with the doh server, which most likely - is not what you want. + ... to easier allow additional DNS transactions. + + Closes #4629 + +- travis: build ngtcp2 with --enable-lib-only - Found through fuzzing. + ... makes it skip the examples and other stuff we don't neeed. - Closes #4406 + Closes #4646 -- [Paul Dreik brought this change] +- [David Benjamin brought this change] - doh: return early if there is no time left + ngtcp2: fix thread-safety bug in error-handling - Closes #4406 - -- [Barry Pollard brought this change] + ERR_error_string(NULL) should never be called. It places the error in a + global buffer, which is not thread-safe. Use ERR_error_string_n with a + local buffer instead. + + Closes #4645 - http: lowercase headernames for HTTP/2 and HTTP/3 +- travis: export the CC/CXX variables when set - Closes #4401 - Fixes #4400 + Suggested-by: Peter Wu + Fixes #4637 + Closes #4640 -Marcel Raad (23 Sep 2019) -- vtls: fix narrowing conversion warnings +Marcel Raad (26 Nov 2019) +- dist: add error-codes.pl - Curl_timeleft returns `timediff_t`, which is 64 bits wide also on - 32-bit systems since commit b1616dad8f0. + Follow-up to commit 74f441c6d31. + This should fix test 1175 when run via the daily source tarballs. - Closes https://github.com/curl/curl/pull/4398 + Closes https://github.com/curl/curl/pull/4638 -Daniel Stenberg (23 Sep 2019) -- [Joel Depooter brought this change] +Daniel Stenberg (26 Nov 2019) +- [John Schroeder brought this change] - winbuild: Add manifest to curl.exe for proper OS version detection + curl: fix --upload-file . hangs if delay in STDIN + + Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION. - This is a small fix to commit ebd213270a017a6830928ee2e1f4a9cabc799898 - in pull request #1221. That commit added the CURL_EMBED_MANIFEST flag to - CURL_RC_FLAGS. However, later in the file CURL_RC_FLAGS is - overwritten. The fix is to append values to CURL_RC_FLAGS instead of - overwriting + When uploading from stdin in non-blocking mode, a delay in reading + the stream (EAGAIN) causes curl to pause sending data + (CURL_READFUNC_PAUSE). Prior to this change, a busy read was + detected and unpaused only in the CURLOPT_WRITEFUNCTION handler. + This change performs the same busy read handling in a + CURLOPT_XFERINFOFUNCTION handler. - Closes #4399 + Fixes #2051 + Closes #4599 + Reported-by: bdry on github -- RELEASE-NOTES: synced +- [John Schroeder brought this change] -Marcel Raad (22 Sep 2019) -- openssl: fix compiler warning with LibreSSL + XFERINFOFUNCTION: support CURL_PROGRESSFUNC_CONTINUE - It was already fixed for BoringSSL in commit a0f8fccb1e0. - LibreSSL has had the second argument to SSL_CTX_set_min_proto_version - as uint16_t ever since the function was added in [0]. + (also for PROGRESSFUNCTION) - [0] https://github.com/libressl-portable/openbsd/commit/56f107201baefb5533486d665a58d8f57fd3aeda + By returning this value from the callback, the internal progress + function call is still called afterward. - Closes https://github.com/curl/curl/pull/4397 + Closes #4599 -Daniel Stenberg (22 Sep 2019) -- curl: exit the create_transfers loop on errors - - When looping around the ranges and given URLs to create transfers, all - errors should exit the loop and return. Previously it would keep - looping. - - Reported-by: SumatraPeter on github - Bug: #4393 - Closes #4396 +- [Michael Forney brought this change] -Jay Satiro (21 Sep 2019) -- socks: Fix destination host shown on SOCKS5 error - - Prior to this change when a server returned a socks5 connect error then - curl would parse the destination address:port from that data and show it - to the user as the destination: - - curld -v --socks5 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to IPv4 172.217.12.206 (locally resolved) - * Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) - curl: (7) Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) + TLS: add BearSSL vtls implementation - That's incorrect because the address:port included in the connect error - is actually a bind address:port (typically unused) and not the - destination address:port. This fix changes curl to show the destination - information that curl sent to the server instead: + Closes #4597 + +- curl_multi_wakeup.3: add example and AVAILABILITY - curld -v --socks5 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to IPv4 172.217.7.14:99 (locally resolved) - * Can't complete SOCKS5 connection to 172.217.7.14:99. (1) - curl: (7) Can't complete SOCKS5 connection to 172.217.7.14:99. (1) + Reviewed-by: Gergely Nagy + Closes #4635 + +- [Gergely Nagy brought this change] + + multi: add curl_multi_wakeup() - curld -v --socks5-hostname 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to google.com:99 (remotely resolved) - * Can't complete SOCKS5 connection to google.com:99. (1) - curl: (7) Can't complete SOCKS5 connection to google.com:99. (1) + This commit adds curl_multi_wakeup() which was previously in the TODO + list under the curl_multi_unblock name. - Ref: https://tools.ietf.org/html/rfc1928#section-6 + On some platforms and with some configurations this feature might not be + available or can fail, in these cases a new error code + (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). - Closes https://github.com/curl/curl/pull/4394 - -Daniel Stenberg (21 Sep 2019) -- travis: enable ngtcp2 h3-23 builds + Fixes #4418 + Closes #4608 -- altsvc: both backends run h3-23 now - - Closes #4395 +Jay Satiro (24 Nov 2019) +- [Xiaoyin Liu brought this change] -- http: fix warning on conversion from int to bit + schannel: fix --tls-max for when min is --tlsv1 or default - Follow-up from 03ebe66d70 - -- urldata: use 'bool' for the bit type on MSVC compilers + Prior to this change schannel ignored --tls-max (CURL_SSLVERSION_MAX_ + macros) when --tlsv1 (CURL_SSLVERSION_TLSv1) or default TLS + (CURL_SSLVERSION_DEFAULT), using a max of TLS 1.2 always. - Closes #4387 - Fixes #4379 + Closes https://github.com/curl/curl/pull/4633 -- appveyor: upgrade VS2017 to VS2019 +- checksrc.bat: Add a check for vquic and vssh directories - Closes #4383 - -- [Zenju brought this change] + Ref: https://github.com/curl/curl/pull/4607 - FTP: FTPFILE_NOCWD: avoid redundant CWDs +- projects: Fix Visual Studio projects SSH builds - Closes #4382 - -- cookie: pass in the correct cookie amount to qsort() + - Generate VQUIC and VSSH filenames in Visual Studio project files. - As the loop discards cookies without domain set. This bug would lead to - qsort() trying to sort uninitialized pointers. We have however not found - it a security problem. + Prior to this change generated Visual Studio project configurations that + enabled SSH did not build properly. Broken since SSH files were moved to + lib/vssh 3 months ago in 5b2d703. - Reported-by: Paul Dreik - Closes #4386 + Fixes https://github.com/curl/curl/issues/4492 + Fixes https://github.com/curl/curl/issues/4630 + Closes https://github.com/curl/curl/pull/4607 -- [Paul Dreik brought this change] +Daniel Stenberg (23 Nov 2019) +- RELEASE-NOTES: synced - urlapi: avoid index underflow for short ipv6 hostnames - - If the input hostname is "[", hlen will underflow to max of size_t when - it is subtracted with 2. - - hostname[hlen] will then cause a warning by ubsanitizer: +Jay Satiro (22 Nov 2019) +- openssl: Revert to less sensitivity for SYSCALL errors - runtime error: addition of unsigned offset to 0x overflowed to - 0x + - Disable the extra sensitivity except in debug builds (--enable-debug). - I think that in practice, the generated code will work, and the output - of hostname[hlen] will be the first character "[". + - Improve SYSCALL error message logic in ossl_send and ossl_recv so that + "No error" / "Success" socket error text isn't shown on SYSCALL error. - This can be demonstrated by the following program (tested in both clang - and gcc, with -O3) + Prior to this change 0ab38f5 (precedes 7.67.0) increased the sensitivity + of OpenSSL's SSL_ERROR_SYSCALL error so that abrupt server closures were + also considered errors. For example, a server that does not send a known + protocol termination point (eg HTTP content length or chunked encoding) + _and_ does not send a TLS termination point (close_notify alert) would + cause an error if it closed the connection. - int main() { - char* hostname=strdup("["); - size_t hlen = strlen(hostname); + To be clear that behavior made it into release build 7.67.0 + unintentionally. Several users have reported it as an issue. - hlen-=2; - hostname++; - printf("character is %d\n",+hostname[hlen]); - free(hostname-1); - } + Ultimately the idea is a good one, since it can help prevent against a + truncation attack. Other SSL backends may already behave similarly (such + as Windows native OS SSL Schannel). However much more of our user base + is using OpenSSL and there is a mass of legacy users in that space, so I + think that behavior should be partially reverted and then rolled out + slowly. - I found this through fuzzing, and even if it seems harmless, the proper - thing is to return early with an error. + This commit changes the behavior so that the increased sensitivity is + disabled in all curl builds except curl debug builds (DEBUGBUILD). If + after a period of time there are no major issues then it can be enabled + in dev and release builds with the newest OpenSSL (1.1.1+), since users + using the newest OpenSSL are the least likely to have legacy problems. - Closes #4389 - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: compile with latest ngtcp2 + nghttp3 draft-23 + Bug: https://github.com/curl/curl/issues/4409#issuecomment-555955794 + Reported-by: Bjoern Franke - Closes #4392 + Fixes https://github.com/curl/curl/issues/4624 + Closes https://github.com/curl/curl/pull/4623 -- THANKS-filter: deal with my typos 'Jat' => 'Jay' +- [Daniel Stenberg brought this change] -- travis: use go master - - ... as the boringssl builds needs a very recent version + openssl: improve error message for SYSCALL during connect - Co-authored-by: Jat Satiro - Closes #4361 - -- tool_operate: removed unused variable 'done' + Reported-by: Paulo Roberto Tomasi + Bug: https://curl.haxx.se/mail/archive-2019-11/0005.html - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/4593 -- tool_operate: Expression 'config->resume_from' is always true +Daniel Stenberg (22 Nov 2019) +- test1175: verify symbols-in-versions and libcurl-errors.3 in sync - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4628 -- tool_getparam: remove duplicate switch case +- include: make CURLE_HTTP3 use a new error code - Fixes warning detected by PVS-Studio - Fixes #4374 - -- libssh2: part of conditional expression is always true: !result + To avoid potential issues with error code reuse. - Fixes warning detected by PVS-Studio - Fixes #4374 + Reported-by: Christoph M. Becker + Assisted-by: Dan Fandrich + Fixes #4601 + Closes #4627 -- urlapi: Expression 'storep' is always true - - Fixes warning detected by PVS-Studio - Fixes #4374 +- bump: next release will be 7.68.0 -- urlapi: 'scheme' is always true +- curl: add --parallel-immediate - Fixes warning detected by PVS-Studio - Fixes #4374 - -- urlapi: part of conditional expression is always true: (relurl[0] == '/') + Starting with this change when doing parallel transfers, without this + option set, curl will prefer to create new transfers multiplexed on an + existing connection rather than creating a brand new one. - Fixes warning detected by PVS-Studio - Fixes #4374 - -- setopt: store CURLOPT_RTSP_SERVER_CSEQ correctly + --parallel-immediate can be set to tell curl to prefer to use new + connections rather than to wait and try to multiplex. - Fixes bug detected by PVS-Studio - Fixes #4374 - -- mime: make Curl_mime_duppart() assert if called without valid dst + libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default + on parallel transfers. - Fixes warning detected by PVS-Studio - Fixes #4374 + Suggested-by: Tom van der Woerdt + Closes #4500 -- http_proxy: part of conditional expression is always true: !error - - Fixes warning detected by PVS-Studio - Fixes #4374 +Daniel Gustafsson (20 Nov 2019) +- [Victor Magierski brought this change] -- imap: merged two case-branches performing the same action + docs: fix typos - Fixes warning detected by PVS-Studio - Fixes #4374 - -- multi: value '2L' is assigned to a boolean + Change 'experiemental' to 'experimental'. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4618 + Reviewed-by: Daniel Gustafsson -- easy: part of conditional expression is always true: !result +Jay Satiro (18 Nov 2019) +- projects: Fix Visual Studio wolfSSL configurations - Fixes warning detected by PVS-Studio - Fixes #4374 - -- netrc: part of conditional expression is always true: !done + - s/USE_CYASSL/USE_WOLFSSL/ - Fixes warning detected by PVS-Studio - Fixes #4374 - -- version: Expression 'left > 1' is always true + - Remove old compatibility macros. - Fixes warning detected by PVS-Studio - Fixes #4374 + Follow-up to 1c6c59a from several months ago when CyaSSL named symbols + were renamed to wolfSSL. The wolfSSL library was formerly named CyaSSL + and we kept using their old name for compatibility reasons, until + earlier this year. -- url: remove dead code - - Fixes warning detected by PVS-Studio - Fixes #4374 +Daniel Stenberg (18 Nov 2019) +- RELEASE-NOTES: synced -- url: part of expression is always true: (bundle->multiuse == 0) - - Fixes warning detected by PVS-Studio - Fixes #4374 +- [Javier Blazquez brought this change] -- ftp: the conditional expression is always true - - ... both !result and (ftp->transfer != FTPTRANSFER_BODY)! + ngtcp2: use overflow buffer for extra HTTP/3 data - Fixes warning detected by PVS-Studio - Fixes #4374 + Fixes #4525 + Closes #4603 -- ftp: Expression 'ftpc->wait_data_conn' is always false +- altsvc: bump to h3-24 - Fixes warning detected by PVS-Studio - Fixes #4374 - -- ftp: Expression 'ftpc->wait_data_conn' is always true + ... as both ngtcp2 and quiche now support that in their master branches - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4604 -- ftp: part of conditional expression is always true: !result +- ngtcp2: free used resources on disconnect - Fixes warning detected by PVS-Studio - Fixes #4374 + Fixes #4614 + Closes #4615 -- http: fix Expression 'http->postdata' is always false +- ngtcp2: handle key updates as ngtcp2 master branch tells us - Fixes warning detected by PVS-Studio - Fixes #4374 - Reported-by: Valerii Zapodovnikov - -- [Niall O'Reilly brought this change] - - doh: avoid truncating DNS QTYPE to lower octet + Reviewed-by: Tatsuhiro Tsujikawa - Closes #4381 + Fixes #4612 + Closes #4613 -- [Jens Finkhaeuser brought this change] +Jay Satiro (17 Nov 2019) +- [Gergely Nagy brought this change] - urlapi: CURLU_NO_AUTHORITY allows empty authority/host part - - CURLU_NO_AUTHORITY is intended for use with unknown schemes (i.e. not - "file:///") to override cURL's default demand that an authority exists. + multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds - Closes #4349 - -- version: next release will be 7.67.0 - -- RELEASE-NOTES: synced - -- url: only reuse TLS connections with matching pinning + Prior to this change: - If the requests have different CURLOPT_PINNEDPUBLICKEY strings set, the - connection should not be reused. + The check if an extra wait is necessary was based not on the + number of extra fds but on the pointer. - Bug: https://curl.haxx.se/mail/lib-2019-09/0061.html - Reported-by: Sebastian Haglund + If a non-null pointer was given in extra_fds, but extra_nfds + was zero, then the wait was skipped even though poll was not + called. - Closes #4347 + Closes https://github.com/curl/curl/pull/4610 -- README: add OSS-Fuzz badge [skip ci] +- lib: Move lib/ssh.h -> lib/vssh/ssh.h - Closes #4380 - -Michael Kaufmann (18 Sep 2019) -- http: merge two "case" statements - -Daniel Stenberg (18 Sep 2019) -- [Zenju brought this change] - - FTP: remove trailing slash from path for LIST/MLSD + Follow-up to 5b2d703 which moved ssh source files to vssh. - Closes #4348 + Closes https://github.com/curl/curl/pull/4609 -- mime: when disabled, avoid C99 macro - - Closes #4368 +Daniel Stenberg (16 Nov 2019) +- [Andreas Falkenhahn brought this change] -- url: cleanup dangling DOH request headers too - - Follow-up to 9bc44ff64d9081 - - Credit to OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/17269 + INSTALL.md: provide Android build instructions - Closes #4372 + Closes #4606 -- [Christoph M. Becker brought this change] +- [Niall O'Reilly brought this change] - http2: relax verification of :authority in push promise requests + doh: improced both encoding and decoding - If the :authority pseudo header field doesn't contain an explicit port, - we assume it is valid for the default port, instead of rejecting the - request for all ports. + Improved estimation of expected_len and updated related comments; + increased strictness of QNAME-encoding, adding error detection for empty + labels and names longer than the overall limit; avoided treating DNAME + as unexpected; - Ref: https://curl.haxx.se/mail/lib-2019-09/0041.html + updated unit test 1655 with more thorough set of proofs and tests - Closes #4365 + Closes #4598 -- doh: clean up dangling DOH handles and memory on easy close - - If you set the same URL for target as for DoH (and it isn't a DoH - server), like "https://example.com" in both, the easy handles used for - the DoH requests could be left "dangling" and end up not getting freed. +- ngtcp2: increase QUIC window size when data is consumed - Reported-by: Paul Dreik - Closes #4366 + Assisted-by: Javier Blazquez + Ref #4525 (partial fix) + Closes #4600 -- unit1655: make it C90 compliant - - Unclear why this was not detected in the CI. - - Follow-up to b7666027296a +- [Melissa Mears brought this change] -- smb: check for full size message before reading message details + config-win32: cpu-machine-OS for Windows on ARM - To avoid reading of uninitialized data. + Define the OS macro properly for Windows on ARM builds. Also, we might + as well add the GCC-style IA-64 macro. - Assisted-by: Max Dymond - Bug: https://crbug.com/oss-fuzz/16907 - Closes #4363 + Closes #4590 -- quiche: persist connection details +- examples: add multi-poll.c - ... like we do for other protocols at connect time. This makes "curl -I" - and other things work. + Show how curl_multi_poll() makes it even easier to use the multi + interface. - Reported-by: George Liu - Fixes #4358 - Closes #4360 + Closes #4596 -- openssl: fix warning with boringssl and SSL_CTX_set_min_proto_version +- multi_poll: avoid busy-loop when called without easy handles attached - Follow-up to ffe34b7b59 - Closes #4359 - -- [Paul Dreik brought this change] + Fixes #4594 + Closes #4595 + Reported-by: 3dyd on github - doh: fix undefined behaviour and open up for gcc and clang optimization +- curl: fix -T globbing - The undefined behaviour is annoying when running fuzzing with - sanitizers. The codegen is the same, but the meaning is now not up for - dispute. See https://cppinsights.io/s/516a2ff4 + Regression from e59371a4936f8 (7.67.0) + + Added test 490, 491 and 492 to verify the functionality. - By incrementing the pointer first, both gcc and clang recognize this as - a bswap and optimizes it to a single instruction. See - https://godbolt.org/z/994Zpx + Reported-by: Kamil Dudka + Reported-by: Anderson Sasaki - Closes #4350 + Fixes #4588 + Closes #4591 -- [Paul Dreik brought this change] +- HISTORY: added cmake, HTTP/3 and parallel downloads with curl - doh: fix (harmless) buffer overrun +- quiche: reject headers in the wrong order - Added unit test case 1655 to verify. - Close #4352 + Pseudo header MUST come before regular headers or cause an error. - the code correctly finds the flaws in the old code, - if one temporarily restores doh.c to the old version. - -Alessandro Ghedini (15 Sep 2019) -- docs: remove trailing ':' from section names in CURLOPT_TRAILER* man - -- docs: fix typo in CURLOPT_HTTP_VERSION man - -GitHub (14 Sep 2019) -- [Daniel Stenberg brought this change] + Reported-by: Cynthia Coan + Fixes #4571 + Closes #4584 - CI: inintial github action job +- openssl: prevent recursive function calls from ctx callbacks + + Follow the pattern of many other callbacks. - First shot at a CI build on github actions + Ref: #4546 + Closes #4585 -Daniel Stenberg (13 Sep 2019) -- appveyor: add a winbuild +- CURL-DISABLE: initial docs for the CURL_DISABLE_* defines - Assisted-by: Marcel Raad - Assisted-by: Jay Satiro + The disable-scan script used in test 1165 is extended to also verify + that the docs cover all used defines and all defines offered by + configure. - Closes #4324 + Reported-by: SLDiggie on github + Fixes #4545 + Closes #4587 -- FTP: allow "rubbish" prepended to the SIZE response +- remove_handle: clear expire timers after multi_done() - This is a protocol violation but apparently there are legacy proprietary - servers doing this. + Since 59041f0, a new timer might be set in multi_done() so the clearing + of the timers need to happen afterwards! + + Reported-by: Max Kellermann + Fixes #4575 + Closes #4583 + +Marcel Raad (10 Nov 2019) +- test1558: use double slash after file: - Added test 336 and 337 to verify. + Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this + test unnecessarily failed when using `file:/` instead of `file:///`. - Reported-by: Philippe Marguinaud - Closes #4339 - -- [Zenju brought this change] + Closes https://github.com/curl/curl/pull/4554 - FTP: skip CWD to entry dir when target is absolute +Daniel Stenberg (10 Nov 2019) +- pause: avoid updating socket if done was already called + + ... avoids unnecesary recursive risk when the transfer is already done. - Closes #4332 + Reported-by: Richard Bowker + Fixes #4563 + Closes #4574 -Kamil Dudka (13 Sep 2019) -- curl: fix memory leaked by parse_metalink() +Jay Satiro (9 Nov 2019) +- strerror: Fix an error looking up some Windows error strings - This commit fixes a regression introduced by curl-7_65_3-5-gb88940850. - Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind - and libmetalink enabled. + - Use FORMAT_MESSAGE_IGNORE_INSERTS to ignore format specifiers in + Windows error strings. - Closes #4326 - -Daniel Stenberg (13 Sep 2019) -- parsedate: still provide the name arrays when disabled + Since we are not in control of the error code we don't know what + information may be needed by the error string's format specifiers. - If FILE or FTP are enabled, since they also use them! + Prior to this change Windows API error strings which contain specifiers + (think specifiers like similar to printf specifiers) would not be shown. + The FormatMessage Windows API call which turns a Windows error code into + a string could fail and set error ERROR_INVALID_PARAMETER if that error + string contained a format specifier. FormatMessage expects a va_list for + the specifiers, unless inserts are ignored in which case no substitution + is attempted. - Reported-by: Roland Hieber - Fixes #4325 - Closes #4343 + Ref: https://devblogs.microsoft.com/oldnewthing/20071128-00/?p=24353 -- [Gilles Vollant brought this change] +- [r-a-sattarov brought this change] - curl:file2string: load large files much faster - - ... by using a more efficient realloc scheme. + system.h: fix for MCST lcc compiler - Bug: https://curl.haxx.se/mail/lib-2019-09/0045.html - Closes #4336 - -- openssl: close_notify on the FTP data connection doesn't mean closure + Fixed build by MCST lcc compiler on MCST Elbrus 2000 architecture and do + some code cleanup. - For FTPS transfers, curl gets close_notify on the data connection - without that being a signal to close the control connection! + e2k (Elbrus 2000) - this is VLIW/EPIC architecture, like Intel Itanium + architecture. - Regression since 3f5da4e59a556fc (7.65.0) + Ref: https://en.wikipedia.org/wiki/Elbrus_2000 - Reported-by: Zenju on github - Reviewed-by: Jay Satiro - Fixes #4329 - Closes #4340 - -- [Jimmy Gaussen brought this change] + Closes https://github.com/curl/curl/pull/4576 - docs/HTTP3: fix `--with-ssl` ngtcp2 configure flag +Daniel Stenberg (8 Nov 2019) +- TODO: curl_multi_unblock - Closes #4338 - -- RELEASE-NOTES: synced - -- curlver: bump to 7.66.1 - -- [Zenju brought this change] + Closes #4418 - setopt: make it easier to add new enum values - - ... by using the *_LAST define names better. +- TODO: Run web-platform-tests url tests - Closes #4321 + Closes #4477 -- asyn-thread: s/AF_LOCAL/AF_UNIX for Solaris +- TODO: 1.4 alt-svc sharing - Reported-by: Dagobert Michelsen - Fixes #4328 - Closes #4333 - -- [Bernhard Walle brought this change] + Closes #4476 - winbuild/MakefileBuild.vc: Add vssh - - Without that modification, the Windows build using the makefiles doesn't - work. +- test1560: require IPv6 for IPv6 aware URL parsing - Signed-off-by: Bernhard Walle + The URL parser function can't reject a bad IPv6 address properly when + curl was built without IPv6 support. - Fixes #4322 - Closes #4323 + Reported-by: Marcel Raad + Fixes #4556 + Closes #4572 -Bernhard Walle (11 Sep 2019) -- winbuild/MakefileBuild.vc: Fix line endings - - The file had mixed line endings. +- checksrc: repair the copyrightyear check - Signed-off-by: Bernhard Walle - -Jay Satiro (11 Sep 2019) -- ldap: Stop using wide char version of ldapp_err2string + - Consider a modified file to be committed this year. - Despite ldapp_err2string being documented by MS as returning a - PCHAR (char *), when UNICODE it is mapped to ldap_err2stringW and - returns PWCHAR (wchar_t *). + - Make the travis CHECKSRC also do COPYRIGHTYEAR scan in examples and + includes - We have lots of code that expects ldap_err2string to return char *, - most of it failf used like this: + - Ignore 0 parents when getting latest commit date of file. - failf(data, "LDAP local: Some error: %s", ldap_err2string(rc)); + since in the CI we're dealing with a truncated repo of last 50 commits, + the file's most recent commit may not be available. when this happens + git log and rev-list show the initial commit (ie first commit not to be + truncated) but that's incorrect so ignore it. - Closes https://github.com/curl/curl/pull/4272 - -Version 7.66.0 (10 Sep 2019) - -Daniel Stenberg (10 Sep 2019) -- RELEASE-NOTES: curl 7.66.0 - -- THANKS: from the 7.66.0 release - -- curl: make sure the parallel transfers do them all + Ref: https://github.com/curl/curl/pull/4547 - The logic could erroneously break the loop too early before all - transfers had been transferred. + Closes https://github.com/curl/curl/pull/4549 - Reported-by: Tom van der Woerdt - Fixes #4316 - Closes #4317 - -- urlapi: one colon is enough for the strspn() input (typo) + Co-authored-by: Jay Satiro -- urlapi: verify the IPv6 numerical address - - It needs to parse correctly. Otherwise it could be tricked into letting - through a-f using host names that libcurl would then resolve. Like - '[ab.be]'. +- copyrights: fix copyright year range - Reported-by: Thomas Vegas - Closes #4315 - -- [Clément Notin brought this change] - - openssl: use SSL_CTX_set__proto_version() when available + .. because checksrc's copyright year check stopped working. - OpenSSL 1.1.0 adds SSL_CTX_set__proto_version() that we now use - when available. Existing code is preserved for older versions of - OpenSSL. + Ref: https://github.com/curl/curl/pull/4547 - Closes #4304 + Closes https://github.com/curl/curl/pull/4549 -- [Clément Notin brought this change] +- RELEASE-NOTES: synced - openssl: indent, re-organize and add comments +- curlver: bump to 7.67.1 -- [migueljcrum brought this change] +- mailmap: fixup Massimiliano Fantuzzi - sspi: fix memory leaks +- scripts/contributors: make committers get included too - Closes #4299 + in addition to authors -- travis: disable ngtcp2 builds (again) +Jay Satiro (8 Nov 2019) +- [Massimiliano Fantuzzi brought this change] -- Curl_fillreadbuffer: avoid double-free trailer buf on error - - Reviewed-by: Jay Satiro - Reported-by: Thomas Vegas + configure: fix typo in help text - Closes #4307 + Closes https://github.com/curl/curl/pull/4570 -- tool_setopt: handle a libcurl build without netrc support - - Reported-by: codesniffer13 on github - Fixes #4302 - Closes #4305 +Daniel Stenberg (7 Nov 2019) +- [Christian Schmitz brought this change] -- security:read_data fix bad realloc() - - ... that could end up a double-free + ntlm: USE_WIN32_CRYPTO check removed to get USE_NTLM2SESSION set - CVE-2019-5481 - Bug: https://curl.haxx.se/docs/CVE-2019-5481.html + Closes #3704 -- [Thomas Vegas brought this change] +Jay Satiro (6 Nov 2019) +- [Wyatt O'Day brought this change] - tftp: Alloc maximum blksize, and use default unless OACK is received - - Fixes potential buffer overflow from 'recvfrom()', should the server - return an OACK without blksize. + build: fix for CURL_DISABLE_DOH - Bug: https://curl.haxx.se/docs/CVE-2019-5482.html - CVE-2019-5482 + Fixes https://github.com/curl/curl/issues/4565 + Closes https://github.com/curl/curl/pull/4566 -- [Thomas Vegas brought this change] +- [Leonardo Taccari brought this change] - tftp: return error when packet is too small for options + configure: avoid unportable `==' test(1) operator + + Closes https://github.com/curl/curl/pull/4567 -- KNOWN_BUGS/TODO: cleanup and remove outdated issues +Version 7.67.0 (5 Nov 2019) +Daniel Stenberg (5 Nov 2019) - RELEASE-NOTES: synced - -- netrc: free 'home' on error - - Follow-up to f9c7ba9096ec2 - - Coverity CID 1453474 - Closes #4291 - -- urldata: avoid 'generic', use dedicated pointers - - For the 'proto' union within the connectdata struct. - - Closes #4290 + The 7.67.0 release -- cleanup: move functions out of url.c and make them static - - Closes #4289 +- THANKS: add new names from 7.67.0 -- smtp: check for and bail out on too short EHLO response - - Otherwise, a three byte response would make the smtp_state_ehlo_resp() - function misbehave. - - Credit to OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/16918 +- configure: only say ipv6 enabled when the variable is set - Assisted-by: Max Dymond + Previously it could say "IPv6: enabled" at the end of the configure run + but the define wasn't set because of a missing getaddrinfo(). - Closes #4287 + Reported-by: Marcel Raad + Fixes #4555 + Closes #4560 -- smb: init *msg to NULL in smb_send_and_recv() - - ... it might otherwise return OK from this function leaving that pointer - uninitialized. - - Bug: https://crbug.com/oss-fuzz/16907 +Marcel Raad (2 Nov 2019) +- certs/Server-localhost-lastSAN-sv: regenerate with sha256 - Closes #4286 - -- ROADMAP: updated after recent user poll + All other certificates were regenerated in commit ba782baac30, but + this one was missed. + Fixes test3001 on modern systems. - In rough prio order + Closes https://github.com/curl/curl/pull/4551 -- THANKS: remove duplicate +Daniel Stenberg (2 Nov 2019) +- [Vilhelm Prytz brought this change] -- Curl_addr2string: take an addrlen argument too - - This allows the function to figure out if a unix domain socket has a - file name or not associated with it! When a socket is created with - socketpair(), as done in the fuzzer testing, the path struct member is - uninitialized and must not be accessed. - - Bug: https://crbug.com/oss-fuzz/16699 + copyrights: update all copyright notices to 2019 on files changed this year - Closes #4283 - -- [Rolf Eike Beer brought this change] - - CMake: remove needless newlines at end of gss variables - -- [Rolf Eike Beer brought this change] - - CI: remove duplicate configure flag for LGTM.com + Closes #4547 -- [Rolf Eike Beer brought this change] +- [Bastien Bouclet brought this change] - CMake: use platform dependent name for dlopen() library + mbedtls: add error message for cert validity starting in the future - Closes #4279 + Closes #4552 -- quiche: expire when poll returned data +Jay Satiro (1 Nov 2019) +- schannel_verify: Fix concurrent openings of CA file - ... to make sure we continue draining the queue until empty + - Open the CA file using FILE_SHARE_READ mode so that others can read + from it as well. - Closes #4281 - -- quiche: decrease available buffer size, don't assign it! + Prior to this change our schannel code opened the CA file without + sharing which meant concurrent openings (eg an attempt from another + thread or process) would fail during the time it was open without + sharing, which in curl's case would cause error: + "schannel: failed to open CA file". - Found-by: Jeremy Lainé - -- RELEASE-NOTES: synced - -- [Kyohei Kadota brought this change] - - curl: fix include conditions - -- [Kyohei Kadota brought this change] + Bug: https://curl.haxx.se/mail/lib-2019-10/0104.html + Reported-by: Richard Alcock - plan9: fix installation instructions +Daniel Stenberg (31 Oct 2019) +- gtls: make gnutls_bye() not wait for response on shutdown - Closes #4276 - -- ngtcp2: on h3 stream close, call expire + ... as it can make it wait there for a long time for no good purpose. - ... to trigger a new read to detect the stream close! + Patched-by: Jay Satiro + Reported-by: Bylon2 on github + Adviced-by: Nikos Mavrogiannopoulos - Closes #4275 + Fixes #4487 + Closes #4541 -- [Tatsuhiro Tsujikawa brought this change] +- [Michał Janiszewski brought this change] - ngtcp2: build latest ngtcp2 and ngtcp2_crypto_openssl + appveyor: publish artifacts on appveyor - Closes #4278 - -- ngtcp2: set flow control window to stream buffer size + This allows obtaining upstream builds of curl directly from appveyor for + all the available configurations - Closes #4274 - -- [Christopher Head brought this change] + Closes #4509 - CURLOPT_HEADERFUNCTION.3: clarify +- url: make Curl_close() NULLify the pointer too + + This is the common pattern used in the code and by a unified approach we + avoid mistakes. - Closes #4273 + Closes #4534 + +- [Trivikram Kamat brought this change] -- CURLINFO docs: mention that in redirects times are added + INSTALL: add missing space for configure commands - Suggested-by: Brandon Dong - Fixes #4250 - Closes #4269 + Closes #4539 -- travis: enable ngtcp2 builds again +- url: Curl_free_request_state() should also free doh handles - Switched to the openssl-quic-draft-22 openssl branch. + ... or risk DoH memory leaks. - Closes #4271 + Reported-by: Paul Dreik + Fixes #4463 + Closes #4527 -- HTTP3: switched openssl branch to use +- examples: remove the "this exact code has not been verified" + + ... as really confuses the reader to not know what to believe! -- [Tatsuhiro Tsujikawa brought this change] +- [Trivikram Kamat brought this change] - ngtcp2: Build with latest ngtcp2 and ngtcp2_crypto_openssl + HTTP3: fix typo somehere1 > somewhere1 - Closes #4270 + Closes #4535 -- http2: when marked for closure and wanted to close == OK +Jay Satiro (28 Oct 2019) +- [Javier Blazquez brought this change] + + HTTP3: fix invalid use of sendto for connected UDP socket - It could otherwise return an error even when closed correctly if GOAWAY - had been received previously. + On macOS/BSD, trying to call sendto on a connected UDP socket fails + with a EISCONN error. Because the singleipconnect has already called + connect on the socket when we're trying to use it for QUIC transfers + we need to use plain send instead. - Reported-by: Tom van der Woerdt - Fixes #4267 - Closes #4268 + Fixes #4529 + Closes https://github.com/curl/curl/pull/4533 +Daniel Stenberg (28 Oct 2019) - RELEASE-NOTES: synced -- build-openssl: fix build with Visual Studio 2019 - - Reviewed-by: Marcel Raad - Contributed-by: osabc on github - Fixes #4188 - Closes #4266 +- [Javier Blazquez brought this change] -Kamil Dudka (26 Aug 2019) -- vauth: return CURLE_AUTH_ERROR on gss_init_sec_context() failure + HTTP3: fix Windows build - This is a follow-up to https://github.com/curl/curl/pull/3864 . + The ngtcp2 QUIC backend was using the MSG_DONTWAIT flag for send/recv + in order to perform nonblocking operations. On Windows this flag does + not exist. Instead, the socket must be set to nonblocking mode via + ioctlsocket. - Closes #4224 - -Daniel Stenberg (26 Aug 2019) -- KNOWN_BUGS: USE_UNIX_SOCKETS on Windows + This change sets the nonblocking flag on UDP sockets used for QUIC on + all platforms so the use of MSG_DONTWAIT is not needed. - Closes #4040 + Fixes #4531 + Closes #4532 -- quiche: send the HTTP body correctly on callback uploads +Marcel Raad (27 Oct 2019) +- appveyor: add --disable-proxy autotools build - Closes #4265 - -- travis: disable ngtcp2 builds (temporarily) + This would have caught issue #3926. - Just too many API changes right now + Also make formatting more consistent. - Closes #4264 + Closes https://github.com/curl/curl/pull/4526 -- ngtcp2: add support for SSLKEYLOGFILE +Daniel Stenberg (25 Oct 2019) +- appveyor: make winbuilds with DEBUG=no/yes and VS 2015/2017 - Closes #4260 - -- ngtcp2: improve h3 response receiving + ... and invoke "curl -V" once done - Closes #4259 - -- ngtcp2: use nghttp3_version() - -- ngtcp2: sync with upstream API changes + Co-Authored-By: Jay Satiro - Assisted-by: Tatsuhiro Tsujikawa + Closes #4523 -- [Kyle Abramowitz brought this change] +- [Francois Rivard brought this change] - scp: fix directory name length used in memcpy - - Fix read off end of array due to bad pointer math in getworkingpath for - SCP home directory case. + schannel: reverse the order of certinfo insertions - Closes #4258 + Fixes #4518 + Closes #4519 -- cgit v1.2.3