From e963209266bbf3809cb8b44740de1b61e58f9ace Mon Sep 17 00:00:00 2001 From: dartraiden Date: Sat, 2 May 2020 22:10:12 +0300 Subject: libcurl: update to 7.70.0 --- libs/libcurl/docs/CHANGES | 9940 +++++++++++++++++++++++---------------------- 1 file changed, 4978 insertions(+), 4962 deletions(-) (limited to 'libs/libcurl/docs/CHANGES') diff --git a/libs/libcurl/docs/CHANGES b/libs/libcurl/docs/CHANGES index 2862b0eeb9..de44c16641 100644 --- a/libs/libcurl/docs/CHANGES +++ b/libs/libcurl/docs/CHANGES @@ -6,7198 +6,7214 @@ Changelog -Version 7.69.1 (11 Mar 2020) +Version 7.70.0 (29 Apr 2020) -Daniel Stenberg (11 Mar 2020) -- RELEASE-NOTES: 7.69.1 +Daniel Stenberg (29 Apr 2020) +- RELEASE-NOTES: 7.70.0 -- THANKS: from the 7.69.1 release +- THANKS: synced with the 7.70.0 release -- [Marc Hoersken brought this change] +- headers: copyright range fix - test1129: fix invalid case of closing XML-tag and Content-Length - - Fixes #5070 - Closes #5072 +- [Rikard Falkeborn brought this change] -Marc Hoersken (10 Mar 2020) -- tests/data: fix static ip instead of dynamic value being used + doh: Constify some input pointers - Follow up to 94ced8e + Closes #5306 -- tests/data: fix static ip:port instead of dynamic values being used +- nss: check for PK11_CreateDigestContext() returning NULL - Closes #5065 - -- tests/server: fix missing use of exe_ext helper function + ... to avoid crashes! - Follow up to 9819984 and 3dce984 - Reviewed-By: Daniel Stenberg - Closes #5064 - -- runtests: log minimal and maximal used port numbers - -Daniel Stenberg (9 Mar 2020) -- [Jim Fuller brought this change] + Reported-by: Hao Wu + Fixes #5302 + Closes #5303 - sftp: fix segfault regression introduced by #4747 - - This fix adds a defensive check for the case where the char *name in - struct libssh2_knownhost is NULL +- travis: bump the wolfssl CI build to use 4.4.0 - Fixes #5041 - Closes #5062 + Closes #5301 -- RELEASE-NOTES: synced +- copyright updates: adjust year ranges -- socks4: fix host resolve regression +Marc Hoersken (26 Apr 2020) +- CI: do not include */ci branches in PR builds - 1. The socks4 state machine was broken in the host resolving phase + Align Azure Pipelines with GitHub Actions. + +Daniel Stenberg (25 Apr 2020) +- runtests: check for the disabled tests relative srcdir - 2. The code now insists on IPv4-only when using SOCKS4 as the protocol - only supports that. + To make it work correctly for out-of-tree builds. - Regression from #4907 and 4a4b63d, shipped in 7.69.0 + Follow-up to 75e8feb6fb08b - Reported-by: amishmm on github - Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 - Closes #5061 + Bug: https://github.com/curl/curl/pull/5288#issuecomment-619346389 + Reported-by: Marcel Raad + Closes #5297 -- [Patrick Monnerat brought this change] +- runtests: revert commenting out a line I did for debugging + + Follow-up to 11091cd4d. It was not meant to be pushed! - silly web server: silent a compilation warning +- smtp: set auth correctly - Recent gcc warns when byte count of strncpy() equals the destination - buffer size. Since the destination buffer is previously cleared and - the source string is always shorter, reducing the byte count by one - silents the warning without affecting the result. + Regression since 7.69.0 and 68fb25fa3fcff. - Closes #5059 - -- [Patrick Monnerat brought this change] + The code wrongly assigned 'from' instead of 'auth' which probably was a + copy and paste mistake from other code, leading to that auth could + remain NULL and later cause an error to be returned. + + Assisted-by: Eric Sauvageau + Fixes #5294 + Closes #5295 - cookie: get_top_domain() sets zero length for null domains +Marcel Raad (25 Apr 2020) +- lib: clean up whitespace - This silents a compilation warning with gcc -O3. + This fixes CodeFactor warnings. -- [Patrick Monnerat brought this change] +Daniel Stenberg (25 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] - test 1560: avoid valgrind false positives + libssh: avoid options override by configuration files - When using maximum code optimization level (-O3), valgrind wrongly - detects uses of uninitialized values in strcmp(). + Previously, options set explicitly through command line options could be + overridden by the configuration files parsed automatically when + ssh_connect() was called. - Preset buffers with all zeroes to avoid that. - -Steve Holme (8 Mar 2020) -- sha256: Added WinCrypt implementation + By calling ssh_options_parse_config() explicitly, the configuration + files are parsed before setting the options, avoiding the options + override. Once the configuration files are parsed, the automatic + configuration parsing is not executed. - Closed #5030 - -- sha256: Added SecureTransport implementation + Fixes #4972 + Closes #5283 + Signed-off-by: Anderson Toshiyuki Sasaki -Daniel Stenberg (7 Mar 2020) -- lib1564: reduce number of mid-wait wakeup calls - - This test does A LOT of *wakeup() calls and then calls curl_multi_poll() - twice. The first *poll() is then expected to return early and the second - not - as the first is supposed to drain the socketpair pipe. +- runtests: when mentions http, kill http/2 too - It turns out however that when given "excessive" amounts of writes to - the pipe, some operating systems (the Solaris based are known) will - return EAGAIN before the pipe is drained, which in our test case causes - the second *poll() call to also abort early. + Since the http2 test server is a mere proxy that needs to know about the + dynamic port the HTTP server is using, it too needs to get restarted + when the http server is killed. - This change attempts to avoid the OS-specific behaviors in the test by - reducing the amount of wakeup calls from 1234567 to 10. + A regression caused by 80d6515. - Reported-by: Andy Fiddaman - Fixes #5037 - Closes #5058 + Fixes #5289 + Closes #5291 -- [Patrick Monnerat brought this change] +- [Yuri Slobodyanyuk brought this change] - mime: fix the binary encoder to handle large data properly - - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. + docs: fix two typos - Fixes #4860 - Closes #4833 - Reported-by: RuurdBeerstra on github + Closes #5292 -- [Patrick Monnerat brought this change] +- [Emil Engler brought this change] - mime: do not perform more than one read in a row + tests/git: ignore mqttd and port files - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. + Closes #5290 + +- tests: make runtests check that disabled tests exists - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 667 checks mimepost using single-byte read callback with encoder. - New test 668 checks the end of part data early detection. + ... and error out if so. Removed '536' from DISABLED as there is no such + test file. - Fixes #4826 - Reported-by: MrdUkk on github + Closes #5288 -- [Patrick Monnerat brought this change] +- test1154: set a proper name - mime: latch last read callback status. +- select: make Curl_socket_check take timediff_t timeout - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. + Coverity found CID 1461718: - Fixes #4813 - Reported-by: MrdUkk on github + Integer handling issues (CONSTANT_EXPRESSION_RESULT) "timeout_ms > + 9223372036854775807L" is always false regardless of the values of its + operands. This occurs as the logical second operand of "||". + + Closes #5240 -Marc Hoersken (7 Mar 2020) -- runtests: fix missing use of exe_ext helper function +- [i-ky brought this change] -Daniel Stenberg (7 Mar 2020) -- [Ernst Sjöstrand brought this change] + libcurl-multi.3: added missing full stop + + Closes #5285 - ares: store dns parameters for duphandle +Jay Satiro (22 Apr 2020) +- transfer: Switch PUT to GET/HEAD on 303 redirect - With c-ares the dns parameters lives in ares_channel. Store them in the - curl handle and set them again in easy_duphandle. + Prior to this change if there was a 303 reply to a PUT request then + the subsequent request to respond to that redirect would also be a PUT. + It was determined that was most likely incorrect based on the language + of the RFCs. Basically 303 means "see other" resource, which implies it + is most likely not the same resource, therefore we should not try to PUT + to that different resource. - Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. + Refer to the discussions in #5237 and #5248 for more information. - Fixes #4893 - Closes #5020 - Signed-off-by: Ernst Sjöstrand + Fixes https://github.com/curl/curl/issues/5237 + Closes https://github.com/curl/curl/pull/5248 -- version: make curl_version* thread-safe without using global context +Daniel Stenberg (22 Apr 2020) +- lib/mk-ca-bundle: skip empty certs - Closes #5010 + Reviewed-by: Emil Engler + Reported-by: Ashwin Metpalli + Fixes #5278 + Closes #5280 + +- version: skip idn2_check_version() check and add precaution + + A gcc-10's -fanalyze complaint made me spot and do these improvements. + + Closes #5281 - RELEASE-NOTES: synced -Marc Hoersken (7 Mar 2020) -- tests: use native Sleep function as fallback on Windows +- [Brian Bergeron brought this change] + + curl.h: update comment typo - Reviewed-By: Daniel Stenberg - Closes #5054 + "routines with be invoked" -> "routines will be invoked" + + Closes #5279 -- perl: align order and completeness of Windows OS checks +- [Emil Engler brought this change] -Daniel Stenberg (7 Mar 2020) -- tool_cb_see: set correct copyright year range + GnuTLS: Don't skip really long certificate fields - Follow-up to a39e5bfb9 + Closes #5271 -Marc Hoersken (7 Mar 2020) -- seek: fix fallback for missing ftruncate on Windows +- gnutls: bump lowest supported version to 3.1.10 - This fixes test 198 on versions of MinGW-w64 without ftruncate + GnuTLS 3.1.10 added new functions we want to use. That version was + released on Mar 22, 2013. Removing support for older versions also + greatly simplifies the code. - Reviewed-By: Daniel Stenberg - Reviewed-By: Marcel Raad - Closes #5055 + Ref: #5271 + Closes #5276 -- config-win32: Windows does not have ftruncate +- mqtt: make NOSTATE get within the debug name array -Daniel Stenberg (7 Mar 2020) -- pause: force a connection (re-)check after unpausing +- tests: run the RTSP test server on a dynamic port number - There might be data available that was already read off the socket, for - example in the TLS layer. - - Reported-by: Anders Berg - Fixes #4966 - Closes #5049 - -- socks5: switch state properly when the resolve is done + To avoid port collisions. - Regression from 4a4b63d (and #4907) - Reported-by: vitaha85 on github - Fixes #5053 - Closes #5056 + Closes #5272 -Jay Satiro (7 Mar 2020) -- libssh: Fix matching user-specified MD5 hex key +- tests: add %NOLISTENPORT and use it - Prior to this change a match would never be successful because it - was mistakenly coded to compare binary data from libssh to a - user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). + The purpose with this variable is to provide a port number that is + reasonably likely to not have a listener on the local host so that tests + can try connect failures against it. It uses port 47 - "reserved" + according to IANA. - Reported-by: fds242@users.noreply.github.com + Updated six tests to use it instead of the previous different ports. - Fixes https://github.com/curl/curl/issues/4971 - Closes https://github.com/curl/curl/pull/4974 + Assisted-by: Emil Engler + Closes #5270 -Daniel Stenberg (6 Mar 2020) -- pause: bail out on bad input +- mqtt: remove code with no purpose - A NULL easy handle or an easy handle without an associated connection - cannot be paused or unpaused. + Detected by Coverity. CID 1462319. - Closes #5050 - -Steve Holme (6 Mar 2020) -- unit1612: fixed the inclusion and compilation of the HMAC unit test + "The same code is executed when the condition result is true or false, + because the code in the if-then branch and after the if statement is + identical." - Follow up to 3f74e5e6 to fix: + Closes #5275 + +- mqtt: fix Curl_read() error handling while reading remaining length - - A typo in Makefile.inc where unit1611 was used instead - - Some compilation issues in unit1612.c + Detected by Coverity. CID 1462320. - Closes #5024 + Closes #5274 -Daniel Stenberg (6 Mar 2020) -- pause: return early for calls that don't change pause state +- server/tftpd: fix compiler warning - Reviewed-by: Patrick Monnerat - Ref: #4833 - Closes #5026 + Follow-up from 369ce38ac1d + Reported-by: Marc Hörsken -Jay Satiro (6 Mar 2020) -- curl_share_setopt.3: Note sharing cookies doesn't enable the engine +- http: free memory when Alt-Used header creation fails due to OOM - Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused - CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. + Reported-by: James Fuller + Fixes #5268 + Closes #5269 + +Daniel Gustafsson (20 Apr 2020) +- lib: fix typos in comments and errormessages - Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html - Reported-by: Felipe Gasper + This fixes a few randomly spotted typos in recently merged code, most + notably one in a userfacing errormessage the schannel code. + +Daniel Stenberg (20 Apr 2020) +- tests: run the SOCKS test server on a dynamic port number - Closes https://github.com/curl/curl/pull/5048 + Closes #5266 -- multi: skip EINTR check on wakeup socket if it was closed +- [Johannes Schindelin brought this change] + + multi-ssl: reset the SSL backend on `Curl_global_cleanup()` - - Don't check errno on wakeup socket if sread returned 0 since sread - doesn't set errno in that case. + When cURL is compiled with support for multiple SSL backends, it is + possible to configure an SSL backend via `curl_global_sslset()`, but + only *before* `curl_global_init()` was called. - This is a follow-up to cf7760a from several days ago which fixed - Curl_multi_wait to stop busy looping sread on the non-blocking wakeup - socket if it was closed (ie sread returns 0). Due to a logic error it - was still possible to busy loop in that case if errno == EINTR. + If another SSL backend should be used after that, a user might be + tempted to call `curl_global_cleanup()` to start over. However, we did + not foresee that use case and forgot to reset the SSL backend in that + cleanup. - Closes https://github.com/curl/curl/pull/5047 - -Daniel Stenberg (6 Mar 2020) -- transfer: set correct copyright year range + Let's allow that use case. + + Fixes #5255 + Closes #5257 + Reported-by: davidedec on github + Signed-off-by: Johannes Schindelin -- urldata: remove the 'stream_was_rewound' connectdata struct member +- tests: run the TFTP test server on a dynamic port number - ... as it is never set anywhere. + Picking a dynamic unused port is better than a fixed to avoid the + collision risk. - Follow-up to 2f44e94ef - Closes #5046 + Closes #5265 -- Revert "pause: force-drain the transfer on unpause" +- mqtt: improve the state machine - This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) + To handle PUBLISH before SUBACK and more. - Clearly that didn't solve the problem correctly. + Updated the existing tests and added three new ones. - Reported-by: Christopher Reid - Reopens #4966 - Fixes #5044 + Reported-by: Christoph Krey + Bug: https://curl.haxx.se/mail/lib-2020-04/0021.html + Closes #5246 + +- runtests: always put test number in servercmd file - RELEASE-NOTES: synced - - and bumped curlver.h -- MANUAL: update a dict-using command line - - The 'web1913' database is now invalid, use 'gcide' instead. +- release-notes.pl: fix parsing typo -- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - - Closes #3841 +James Fuller (20 Apr 2020) +- [xquery brought this change] -- polarssl: remove more references and mentions - - Assisted-by: Jay Satiro - Follow-up to 6357a19ff29dac04 - Closes #5036 + ensure all references to ports are replaced by vars -Marc Hoersken (4 Mar 2020) -- tests: wrap ignored test failures in braces +- [xquery brought this change] -- tests: align some Windows sleep defines with each other + add more alt-svc test coverage -- tests: try to make sleeping portable by avoiding select - - select does not support just waiting on Windows: - https://perldoc.perl.org/perlport.html#select +Daniel Stenberg (20 Apr 2020) +- test1247: use http server to get the port number set - Reviewed-By: Daniel Stenberg - Closes #5035 + Follow-up to 0f5db7b263f -Daniel Stenberg (4 Mar 2020) -- runtests.1: rephrase how to specify what tests to run +- runtests: use a unix domain socket path with the pid in the name - Also mention the new tilde-prefixed way to ignore test results. + To make it impossible for test cases to access the file name without + using the proper variable for the purpose. - Reviewed-By: Marc Hoersken - Closes #5033 + Closes #5264 -- cirrus-ci: disable the FreeBSD 13 builds +Daniel Gustafsson (19 Apr 2020) +- [Tom brought this change] + + src: Remove C99 constructs to ensure C89 compliance - FreeBSD 13.0 is apparently close to a year away from a stable release - and has proven to cause intermittent builds failures recently. + This fixes the error: 'for' loop initial declaration used outside C99 + mode by declaring the loop increment variable in the beginning of the + block instead of inside the for loop. - Assisted-by: Dan Fandrich - Assisted-by: Fedor Korotkov - Fixes #5028 - Closes #5029 + Fixes #5254 + Reviewed-by: Daniel Gustafsson -Version 7.69.0 (4 Mar 2020) +Daniel Stenberg (19 Apr 2020) +- runtests: dummy init the ports variables to avoid warnings + + ... and generate something that can help debug test cases. -Daniel Stenberg (4 Mar 2020) -- RELEASE-NOTES: 7.69.0 +- [Patrick Monnerat brought this change] -- THANKS: from 7.69.0 + mime: properly check Content-Type even if it has parameters - Now sorted case insensitive - -Marc Hoersken (3 Mar 2020) -- ci/tests: fix escaping of testnames and disable proxy for CI APIs + New test 669 checks this fix is effective. - Follow up to ada581f and c0d8b96 - Closes #5031 + Fixes #5256 + Closes #5258 + Reported-by: thanhchungbtc on github -Jay Satiro (3 Mar 2020) -- cmake: Show HTTPS-proxy in the features output - - - Show HTTPS-proxy in the features output for those backends that - support it: OpenSSL, GnuTLS and NSS. +- tests/FILEFORMAT: converted to markdown and extended - Prior to this change HTTPS-proxy was missing from the cmake features - output even if curl was built with it. Only cmake output was affected. - Both the library and tool correctly reported the feature. + Closes #5261 + +- test1245: make it work with dynamic FTP server port + +- test1055: make it work with dynamic FTP port + +- test1028: make it run on dynamic FTP server port + +- tests: move pingpong server to dynamic listening port - Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html - Reported-by: David Lopes + FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic + ports - Closes https://github.com/curl/curl/pull/5025 + Test 842-845 are unfortunately a bit hard to move over to this concept + right now and require "default port" still... -Marc Hoersken (3 Mar 2020) -- ci/tests: Make it possible to still run but ignore failing tests +- test1056: work with dynamic HTTP ipv6 port + +- test1448: work with dynamic HTTP server port + +- tests: introduce preprocessed test cases - This enables the development of a solution for the failing tests by - running them on CI while ignoring their result for the overall status. + The runtests script now always performs variable replacement on the + entire test source file before the test gets executed, and saves the + updated version in a temporary file (log/test[num]) so that all test + case readers/servers can use that version (if present) and thus enjoy + the powers of test case variable substitution. - Closes #4994 + This is necessary to allow complete port number freedom. + + Test 309 is updated to work with a non-fixed port number thanks to this. -- README.md: add Azure DevOps Pipelines build status badge +- tests: make 2006-2010 handle different port number lengths -- ci/tests: Move CI test result creation above environment setup +- tests: run the sws server on "any port" - This avoids using our test servers as proxy to the AppVeyor API. + Makes the test servers for HTTP and Gopher pop up on a currently unused + port and runtests adapts to that! - Closes #5022 + Closes #5247 -- ci/tests: Send test results to AppVeyor for status overview +Marc Hoersken (18 Apr 2020) +- sockfilt: tidy variable naming and data structure in select_ws - Closes #5021 + This commit does not introduce any logical changes to the code. + + Reviewed-by: Jay Satiro and Marcel Raad + Closes #5238 -Daniel Stenberg (3 Mar 2020) -- Revert "sha256: Added SecureTransport implementation" +Daniel Stenberg (17 Apr 2020) +- [Anderson Toshiyuki Sasaki brought this change] + + libssh: Use new ECDSA key types to check known hosts - This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) + From libssh 0.9.0, ssh_key_type() returns different key types for ECDSA + keys depending on the curve. - That commit broke test 1610 on macos builds without TLS. + Signed-off-by: Anderson Toshiyuki Sasaki + Fixes #5252 + Closes #5253 + +Marcel Raad (17 Apr 2020) +- appveyor: add Unicode winbuild jobs - Closes #5027 + These are cheap as they don't build tests. + + Closes https://github.com/curl/curl/pull/5063 -- dist: include tests/azure.pm in the tarball +Daniel Stenberg (16 Apr 2020) +- mqttd: s/errno/SOCKERRNO - Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 - Reported-by: Marcel Raad + To behave proper on Windows + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/5e855bbd18f84a02c951be7cac6188276818cdac#r38507132 + Closes #5241 -Steve Holme (3 Mar 2020) -- configure.ac: Disable metalink if mbedTLS is specified +- buildconf: use find -execdir instead, remove -print and the ares files - Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as - being one of the backends that metalink needs to be disabled for, I - seem to have included it in the list of allowed SSL/TLS backends in - comnfigure.ac :( + Follow-up to 1e41bec96a6e - Closes #5013 + Suggested-by: Marc Hörsken -- sha256: Tidy up following recent changes +- [Alexander V. Tikhonov brought this change] + + buildconf: avoid using tempfile when removing files - Reviewed-by: Daniel Stenberg - Closes #4956 + Closes #5213 -- sha256: Added WinCrypt implementation +- copyright: bump the copyright year range -- sha256: Added SecureTransport implementation +- scripts/release-notes.pl: accept colon after the Fixes/Closes keywords -- sha256: Added mbedtls implementation +- [JP Mens brought this change] -- sha256: Added GNU TLS gcrypt implementation + docs/MQTT: replace confusing 80 by 75 + + I was a bit surprised by the `80`: first thought: what's HTTP doing + here? ;) + + Closes #5236 -- sha256: Added GNU TLS Nettle implementation +- [Brad King brought this change] -Jay Satiro (2 Mar 2020) -- curl_escape.3: Add a link to curl_free + cmake: Avoid MSVC C4273 warnings in send/recv checks - Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 - -- curl_getenv.3: Fix the memory handling description + We use `check_c_source_compiles` to check possible send/recv signatures + by reproducing the forward declarations from system headers. On Windows + the `winsock2.h` header adds dll linkage settings to its forward + declaration. If ours does not match the compiler warns: - - Tell the user to call curl_free() to free the pointer returned by - curl_getenv(). + warning C4273: 'recv': inconsistent dll linkage - Prior to this change the user was directed to call free(), but that - would not work in cases where the library and application use separate C - runtimes and therefore have separate heap memory management. + Add `WINSOCK_API_LINKAGE` to our test signatures when it is defined so + that our linkage is consistent with that from `winsock2.h`. - Closes https://github.com/curl/curl/pull/5016 - -Daniel Stenberg (2 Mar 2020) -- [Nick Zitzmann brought this change] + Fixes #4764 + Closes #5232 - md4: use init/update/final functions in Secure Transport +Jay Satiro (14 Apr 2020) +- KNOWN_BUGS: Add entry 'Blocking socket operations' - We can use CC_MD4_Init/Update/Final without having to allocate memory - directly. + - Add threaded resolver cleanup and GSSAPI for FTP to the TODO list of + known blocking operations. - Closes #4979 - -Marc Hoersken (2 Mar 2020) -- ci/tests: some MacOS builds randomly take longer than 20min - -Daniel Stenberg (2 Mar 2020) -- multi_wait: stop loop when sread() returns zero + - New known bugs entry 'Blocking socket operations in non-blocking API' + that directs to the TODO's list of known blocking operations. - It's unclear why it would ever return zero here, but this change fixes - Robert's problem and it shouldn't loop forever... + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612488021 - Reported-by: Robert Dunaj - Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html - Closes #5019 + Reported-by: Marc Hoersken + + Closes https://github.com/curl/curl/pull/5216 -- http: mark POSTs with no body as "upload done" from the start +Marc Hoersken (14 Apr 2020) +- test2043: use revoked.badssl.com instead of revoked.grc.com - As we have logic that checks if we get a >= 400 reponse code back before - the upload is done, which then got confused since it wasn't "done" but - yet there was no data to send! + The certificate of revoked.grc.com has expired on 2020-04-13. - Reported-by: IvanoG on github - Fixes #4996 - Closes #5002 + Reviewed-by: Jay Satiro + + Closes #5233 -- tests: disable 962, 963 and 964 on Windows +- sockfilt: fix broken pipe on Windows to be ready in select_ws - These tests are also doing UTF-8 SMTP. + Closes #5228 + +Daniel Stenberg (14 Apr 2020) +- RELEASE-NOTES: synced + +- scripts/release-notes: fix duplicate output header + +- github/workflow: enable MQTT in the macOS debug build + +- azure: add mqtt support to one of the Windows builds + +- travis: add mqtt job on Linux + +- tests: add four MQTT tests 1190 - 1193 + +- tests: add the mqtt test server mqttd + +- tests: support hex encoded data and mqtt server - Follow-up to df207d2dd93b9e73 + The mqtt server is started using a "random" port. -Marc Hoersken (2 Mar 2020) -- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer +- [Björn Stenberg brought this change] -Daniel Stenberg (2 Mar 2020) -- configure: bump the AC_COPYRIGHT year range + mqtt: add new experimental protocol + + Closes #5173 -- [Steve Holme brought this change] +- TODO: Consider convenience options for JSON and XML? + + Closes #5203 - tests: disable SMTP UTF-8 tests on Windows +- tool: do not declare functions with Curl_ prefix - Fixes #4988 - Closes #4992 + To avoid collision risks with private libcurl symbols when linked with + static versions (or just versions not hiding internal symbols). + + Reported-by: hydra3333 on github + Fixes #5219 + Closes #5234 -- formdata/mime: copyright year range update +- [Nathaniel R. Lewis brought this change] + + cmake: add aliases so exported target names are available in tree - Due to the merge/revert cycle + Reviewed-by: Brad King + Closes #5206 -- Revert "mime: latch last read callback status." +- version: increase buffer space for ssl version output - This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. + To avoid it getting truncated, especially when several SSL backends are + built-in. - Fixes #5014 - Closes #5015 - Reopens #4833 + Reported-by: Gisle Vanem + Fixes #5222 + Closes #5226 -- Revert "mime: do not perform more than one read in a row" +Marc Hoersken (13 Apr 2020) +- cirrus: no longer ignore test 504 which is working again - This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. + The test is working again, because TCP blackholing is disabled. -- Revert "mime: fix the binary encoder to handle large data properly" +- appveyor: completely disable tests that fail to timeout early - This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. + The tests changed from ignored to disabled are tests that are + about connecting to non-listening socket. On AppVeyor these + tests are not reliable, because for some unknown reason the + connect is not timing out before the test time limit is reached. -- altsvc: both h3 backends now speak h3-27 +Daniel Stenberg (13 Apr 2020) +- test1908: avoid using fixed port number in test data - ... also updated the HTTP3 build description for ngtcp2 accordingly. + Closes #5225 -- [Patrick Monnerat brought this change] +Jay Satiro (12 Apr 2020) +- [Andrew Kurushin brought this change] - mime: fix the binary encoder to handle large data properly + schannel: Fix blocking timeout logic - New test 666 checks this is effective. - As upload buffer size is significant in this kind of tests, shorten it - in similar test 652. + - Fix schannel_send for the case when no timeout was set. - Fixes #4860 - Reported-by: RuurdBeerstra on github - -- [Patrick Monnerat brought this change] + Prior to this change schannel would error if the socket was not ready + to send data and no timeout was set. + + This commit is similar to parent commit 89dc6e0 which recently made the + same change for SOCKS, for the same reason. Basically it was not well + understood that when Curl_timeleft returns 0 it is not a timeout of 0 ms + but actually means no timeout. + + Fixes https://github.com/curl/curl/issues/5177 + Closes https://github.com/curl/curl/pull/5221 - mime: do not perform more than one read in a row +- socks: Fix blocking timeout logic - Input buffer filling may delay the data sending if data reads are slow. - To overcome this problem, file and callback data reads do not accumulate - in buffer anymore. All other data (memory data and mime framing) are - considered as fast and still concatenated in buffer. - As this may highly impact performance in terms of data overhead, an early - end of part data check is added to spare a read call. - When encoding a part's data, an encoder may require more bytes than made - available by a single read. In this case, the above rule does not apply - and reads are performed until the encoder is able to deliver some data. + - Document in Curl_timeleft's comment block that returning 0 signals no + timeout (ie there's infinite time left). - Tests 643, 644, 645, 650 and 654 have been adapted to the output data - changes, with test data size reduced to avoid the boredom of long lists of - 1-byte chunks in verification data. - New test 664 checks mimepost using single-byte read callback with encoder. - New test 665 checks the end of part data early detection. + - Fix SOCKS' Curl_blockread_all for the case when no timeout was set. - Fixes #4826 - Reported-by: MrdUkk on github + Prior to this change if the timeout had a value of 0 and that was passed + to SOCKET_READABLE it would return right away instead of blocking. That + was likely because it was not well understood that when Curl_timeleft + returns 0 it is not a timeout of 0 ms but actually means no timeout. + + Ref: https://github.com/curl/curl/pull/5214#issuecomment-612512360 + + Closes https://github.com/curl/curl/pull/5220 -- [Patrick Monnerat brought this change] +- [Marc Hoersken brought this change] - mime: latch last read callback status. + gopher: check remaining time left during write busy loop - In case a read callback returns a status (pause, abort, eof, - error) instead of a byte count, drain the bytes read so far but - remember this status for further processing. - Takes care of not losing data when pausing, and properly resume a - paused mime structure when requested. - New tests 670-673 check unpausing cases, with easy or multi - interface and mime or form api. + Prior to this change gopher's blocking code would block forever, + ignoring any set timeout value. - Fixes #4813 - Reported-by: MrdUkk on github - Closes #4833 + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg + + Similar to #5220 and #5221 + Closes #5214 -Steve Holme (1 Mar 2020) -- unit1651: Fixed conversion compilation warning +Daniel Stenberg (13 Apr 2020) +- [Dirkjan Bussink brought this change] + + gnutls: ensure TLS 1.3 when SRP isn't requested - 371:17: warning: conversion to 'unsigned char' from 'int' may alter its - value [-Wconversion] + When SRP is requested in the priority string, GnuTLS will disable + support for TLS 1.3. Before this change, curl would always add +SRP to + the priority list, effectively always disabling TLS 1.3 support. - Closes #5008 + With this change, +SRP is only added to the priority list when SRP + authentication is also requested. This also allows updating the error + handling here to not have to retry without SRP. This is because SRP is + only added when requested and in that case a retry is not needed. + + Closes #5223 -- configure.ac: Disable metalink support if an incompatible SSL/TLS specified +Marc Hoersken (12 Apr 2020) +- tests/server: add hidden window to gracefully handle WM_CLOSE - tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, - The Win32 Crypto library and Apple's Common Crypto library. + Forward Window events as signals to existing signal event handler. + +- tests/server: add CTRL event handler for Win32 consoles - If an TLS backend such as mbedTLS or WolfSSL is specified then the - following error is given during compilation along, with a load of - unresolved extern errors: + Forward CTRL events as signals to existing signal event handler. + +- tests/server: move all signal handling routines to util.[ch] - Can't compile METALINK support without a crypto library. + Avoid code duplication to prepare for portability enhancements. + +Daniel Stenberg (12 Apr 2020) +- compressed.d: stress that the headers are not modified - Reviewed-by: Daniel Stenberg - Closes #5006 + Suggested-by: Michael Osipov + Assisted-by: Jay Satiro + Bug: https://github.com/curl/curl/issues/5182#issuecomment-611638008 + Closes #5217 -Marc Hoersken (1 Mar 2020) -- ci/tests: Update Azure DevOps pipeline job display names +Marc Hoersken (11 Apr 2020) +- tests/server/util.c: use curl_off_t instead of long for pid - Make the configure step more descriptive and align others. + Avoid potential overflow of huge PIDs on Windows. + + Related to #5188 + Assisted-by: Marcel Raad -- ci/tests: Fix typo in previous commit 597cf2 +- tests: use Cygwin/msys PIDs for stunnel and sshd on Windows + + Since the Windows versions of both programs would write Windows + PIDs to their pidfiles which we cannot handle, we need to use + our known perl.exe Cygwin/msys PID together with exec() in order + to tie the spawned processes to the existance of our perl.exe + + The perl.exe that is executing secureserver.pl and sshserver.pl + has a Cygwin/msys PID, because it is started inside Cygwin/msys. + + Related to #5188 -- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available +- tests: add Windows compatible pidwait like pidkill and pidterm - For security reasons the access token is not available to PR builds. - Therefore we should not try to use the DevOps API with an empty token. + Related to #5188 -Daniel Stenberg (1 Mar 2020) -- build: remove all HAVE_OPENSSL_ENGINE_H defines +- tests: fix conflict between Cygwin/msys and Windows PIDs - ... as there's nothing in the code that actually uses the define! The - last reference was removed in 38203f158. + Add 65536 to Windows PIDs to allow Windows specific treatment + by having disjunct ranges for Cygwin/msys and Windows PIDs. - Closes #5007 + See also: + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=b5e1003722cb14235c4f166be72c09acdffc62ea + - https://cygwin.com/git/?p=newlib-cygwin.git;a=commit; ↵ + h=448cf5aa4b429d5a9cebf92a0da4ab4b5b6d23fe + + Replaces #5178 + Closes #5188 -Jay Satiro (29 Feb 2020) -- [Rolf Eike Beer brought this change] +Daniel Stenberg (11 Apr 2020) +- RELEASE-NOTES: synced - CMake: clean up and improve build procedures +- release-notes.pl: detect the start of the references in cleanup mode + +- Revert "file: on Windows, refuse paths that start with \\" - - remove check for unsupported old CMake versions + This reverts commit 1b71bc532bde8621fd3260843f8197182a467ff2. - - do not link to c-ares library twice + Reminded-by: Chris Roberts + Bug: https://curl.haxx.se/mail/archive-2020-04/0013.html - - modernize custom Find modules + Closes #5215 + +Jay Satiro (11 Apr 2020) +- lib: fix conversion warnings for SOCKET_WRITABLE/READABLE - - FindLibSSH2: - - pass version to FPHSA to show it in the output - - use LIBSSH2_VERSION define to extract the version number in - one shot. This variable exists in the header for 10 years. - - remove unneeded code + - If loss of data may occur converting a timediff_t to time_t and + the time value is > TIME_T_MAX then treat it as TIME_T_MAX. - - FindNGHTTP2.cmake: - - drop needless FPHSA argument - - mark found variables as advanced + This is a follow-up to 8843678 which removed the (time_t) typecast + from the macros so that conversion warnings could be identified. - - FindNSS.cmake: - - show version number + Closes https://github.com/curl/curl/pull/5199 + +- test1148: tolerate progress updates better (again) - - FindCARES.cmake: - - drop default paths - - use FPHSA instead of checking things by hand + - Ignore intermediate progress updates. - - remove needless explict variable dereference + - Support locales that use a character other than period as decimal + separator (eg 100,0%). - - simplify count_true() + test1148 checks that the progress finishes at 100% and has the right + bar width. Prior to this change the test assumed that the only progress + reported for such a quick transfer was 100%, however in rare instances + (like in the CI where transfer time can slow considerably) there may be + intermediate updates. For example, below is stderrlog1148 from a failed + CI run with explicit \r and \n added (it is one line; broken up so that + it's easier to understand). - - allow all policies up to version 3.16 to be set to NEW + \r + \r################################## 48.3% + \r######################################################################## 100.0% + \n - - do not rerun check for -Wstrict-aliasing=3 every time + Closes https://github.com/curl/curl/pull/5194 + +Marc Hoersken (10 Apr 2020) +- sshserver.pl: use cached Win32 environment check variable + +- appveyor: partially revert 3413a110 to keep build without proxy - In contrast to every other compiler flag this has a = in it, which CMake - can't have in a variable name. + Ref: #5211 and #4526 + Reported-by: Marcel Raad + +- appveyor: ignore failing 'connect to non-listening proxy' tests - - only read the interesting strings from curlver.h + Closes #5211 + +- CI/macos: convert CRLF to LF and align indentation + +Daniel Stenberg (9 Apr 2020) +- url: allow non-HTTPS altsvc-matching for debug builds - Reviewed-by: Peter Wu + This is already partly supported but this part was missing. + Reported-by: James Fuller - Closes https://github.com/curl/curl/pull/4975 + Closes #5205 -- runtests: fix output to command log +- server/resolve: remove AI_CANONNAME to make macos tell the truth - - Record only the command of the most recently ran test in the command - log. + With this bit set, my mac successfully resolves "ip6-localhost" when in + fact there is no such host known to my machine! That in turn made test + 241 wrongly execute and fail. - This is a follow-up to 02988b7 from several weeks ago which fixed - writing to the command log, however it saved all commands for all tests - instead of just the most recently ran test as we would now expect. + Closes #5202 + +- runtests: fix warning about using an undefined variable - Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 - Closes https://github.com/curl/curl/pull/5001 + Follow-up from 4d939ef6ceb2db1 -Steve Holme (1 Mar 2020) -- polarssl: Additional removal +- release-notes: fix the initial reference list output + +- github actions: run when pushed to master or */ci + PRs - Follow up to 6357a19f. + Avoid double-builds when using "local" branches for PRs. For both macos + and fuzz jobs. - Reviewed-by: Daniel Stenberg - Closes #5004 + Closes #5201 -- [Jonathan Cardoso Machado brought this change] +- runtests: provide nicer errormsg when protocol "dump" file is empty - docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always - - Reviewed-by: Steve Holme - Closes #5005 +- [Gilles Vollant brought this change] -- md5: Added implementation for mbedTLS + schannel: support .P12 or .PFX client certificates - Reviewed-by: Jay Satiro - Closes #4980 + Used with curl command line option like this: --cert + : --cert-type p12 + + Closes #5193 -- md5: Use pointer notation for array parameters in GnuTLS implementation +- tests: verify split initial HTTP requests with CURL_SMALLREQSEND + + test1294: "split request" being when the entire request isn't sent in + the first go, and the remainder is sent in the PERFORM state. A GET + request is otherwise not sending anything during PERFORM. + + test1295: same kind of split but with POST + + Closes #5197 -- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 +- http: don't consider upload done if the request isn't completely sent off - Closes #4983 + Fixes #4919 + Closes #5197 -Marc Hoersken (29 Feb 2020) -- ci/tests: Send test results to Azure DevOps for reporting +- http: allow Curl_add_buffer_send() to do a short first send by force + + In a debug build, settting the environment variable "CURL_SMALLREQSEND" + will make the first HTTP request send not send more bytes than the set + amount, thus ending up verifying that the logic for handling a split + HTTP request send works correctly. -Daniel Stenberg (29 Feb 2020) -- pause: force-drain the transfer on unpause +- connect: store connection info for QUIC connections - ... since the socket might not actually be readable anymore when for - example the data is already buffered in the TLS layer. + Restores the --head functionality to the curl utility which extracts + 'protocol' that is stored that way. - Fixes #4966 - Reported-by: Anders Berg - Closes #5000 + Reported-by: James Fuller + Fixes #5196 + Closes #5198 -- TODO: curl --proxycommand +- tests/README: update the port numbers list - Suggested-by: Kristian Mide - Closes #4941 + Since the pipelining server is long gone. + Reported-by: James Fuller -- smtp: overwriting 'from' leaks memory +- select: remove typecast from SOCKET_WRITABLE/READABLE macros - Detected by Coverity. CID 1418139. + So that they don't hide conversions-by-mistake - Also, make sure to return error if the new 'from' allocation fails. + Reviewed-by: Jay Satiro + Closes #5190 + +- CURLOPT_WRITEFUNCTION.3: add inline example and new see-also - Closes #4997 + Closes #5192 -- CIfuzz: switch off 'dry_run' mode +- release-notes: output trailing references sorted numerically + +- cleanup: correct copyright year range on a few files + +- configure: remove use of -vec-report0 from CFLAGS with icc - Follow-up from #4960: now make it fail if it detects problems. + ... as it apparently isn't (always) supported. + Reported-by: Alain Miniussi + Fixes #5096 + Closes #5191 + +- warnless: remove code block for icc that didn't work - Closes #4998 + Reported-by: Alain Miniussi + Fixes #5096 -Marc Hoersken (28 Feb 2020) -- ci/tests: Increase timeouts of Windows builds due to new tests +Marc Hoersken (6 Apr 2020) +- dist: add missing setup-win32.h - Recently added tests increased their runtime above the limit of 60min. + Follow up to d820224b8b -- ci/tests: align Azure Pipeline job names with each other +Daniel Stenberg (6 Apr 2020) +- RELEASE-NOTES: synced -- ci/tests: Add Windows builds via Azure Pipelines using Docker +- scripts/release-notes.pl: add helper script for RELEASE-NOTES maintenance + + This script helps putting entries in the RELEASE-NOTES using a coherent + style and sorting with a minimal human editing effort - as long as the + first line in the commit message is good enough! There's a short howto + at the top of the file. -- tests: fix Python 3 compatibility of smbserver.py +- [Dennis Felsing brought this change] -Daniel Stenberg (27 Feb 2020) -- runtests: restore the command log + configure: don't check for Security.framework when cross-compiling - The log file with all command lines for the invoked command lines is now - called logs/commands.log + Since it checks for the local file, not the cross-compiled one. - Fixes #4911 - Closes #4989 + Closes #5189 -- smtp: fix memory leak on exit path - - Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' - going out of scope leaks the storage it points to" +- TODO: Option to make -Z merge lined based outputs on stdout - Closes #4990 + Closes #5175 -Steve Holme (27 Feb 2020) -- gtls: Fixed compilation when using GnuTLS < 3.5.0 +- lib: never define CURL_CA_BUNDLE with a getenv - Reverts the functionality from 41fcb4f when compiling with GnuTLS older - than 3.5.0. + - it breaks the build (since 6de756c9b1de34b7a1) + - it's not documented and not consistent across platforms + - the curl tool does that getenv magic - Reviewed-by: Daniel Stenberg - Closes #4984 + Bug: https://github.com/curl/curl/commit/6de756c#r38127030 + Reported-by: Gisle Vanem + + Closes #5187 -- RELEASE-NOTES: Corrected the link to issue #4892 +Marc Hoersken (5 Apr 2020) +- lib670: use the same Win32 API check as all other lib tests -Daniel Stenberg (27 Feb 2020) -- Curl_is_ASCII_name: handle a NULL argument +- appveyor: use random test server ports based upon APPVEYOR_API_URL - Make the function tolerate a NULL pointer input to avoid dereferencing - that pointer. + Avoid conflicts of test server ports with AppVeyor API on localhost. - Follow-up to efce3ea5a85126d - Detected by OSS-Fuzz - Reviewed-By: Steve Holme - Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 - Fixes #4985 - Closes #4986 + Closes #5034 -- RELEASE-NOTES: synced +- appveyor: sort builds by type and add two new variants + + Related to #5034 and #5063 -- http2: make pausing/unpausing set/clear local stream window +- appveyor: show failed tests in log even if test is ignored - This reduces the HTTP/2 window size to 32 MB since libcurl might have to - buffer up to this amount of data in memory and yet we don't want it set - lower to potentially impact tranfer performance on high speed networks. + And print API response with newline only if there is one + +- appveyor: turn disabled tests into ignored result tests + +Daniel Stenberg (5 Apr 2020) +- KNOWN_BUGS: fixed "USE_UNIX_SOCKETS on Windows" - Requires nghttp2 commit b3f85e2daa629 - (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end - up in the next release after 1.40.0. + Fixed with #5170 (commit 23a870f2fd041278) + +- test1566: verify --etag-compare that gets a 304 back - Fixes #4939 - Closes #4940 + Verifies the fix in #5183 + + Closes #5186 -- [Anderson Toshiyuki Sasaki brought this change] +- [Kwon-Young Choi brought this change] - libssh: improve known hosts handling + CURLINFO_CONDITION_UNMET: return true for 304 http status code - Previously, it was not possible to get a known hosts file entry due to - the lack of an API. ssh_session_get_known_hosts_entry(), introduced in - libssh-0.9.0, allows libcurl to obtain such information and behave the - same as when compiled with libssh2. + In libcurl, CURLINFO_CONDITION_UNMET is used to avoid writing to the + output file if the server did not transfered a file based on time + condition. In the same manner, getting a 304 HTTP response back from the + server, for example after passing a custom If-Match-* header, also + fulfill this condition. - This also tries to avoid the usage of deprecated functions when the - replacements are available. The behaviour will not change if versions - older than libssh-0.8.0 are used. + Fixes #5181 + Closes #5183 + +- [Kwon-Young Choi brought this change] + + curl: allow both --etag-compare and --etag-save with same file name - Signed-off-by: Anderson Toshiyuki Sasaki + This change inverse the order of processing for the --etag-compare and + --etag-save option to process first --etag-compare. This in turn allows + to use the same file name to compare and save an etag. - Fixes #4953 - Closes #4962 + The original behavior of not failing if the etag file does not exists is + conserved. + + Fixes #5179 + Closes #5180 -Steve Holme (27 Feb 2020) -- tests: Automatically deduce the tool name from the test case for unit tests +Viktor Szakats (4 Apr 2020) +- windows: enable UnixSockets with all build toolchains - It is still possible to override the executable to run during the test, - using the tag, but this patch removes the requirement that the - tag must be present for unit tests. + Extend existing unix socket support in Windows builds to be + enabled for all toolchain vendors or versions. (Previously + it was only supported with certain MSVC versions + more recent + Windows 10 SDKs) - It also removes the possibility of human error when existing test cases - are used as the basis for new tests, as recently witnessed in 81c37124. + Ref: https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ + Ref: https://github.com/curl/curl/issues/5162 + Closes: https://github.com/curl/curl/pull/5170 + +Daniel Stenberg (4 Apr 2020) +- KNOWN_BUGS: Store TLS context per transfer instead of per connection - Reviewed-by: Daniel Stenberg - Closes #4976 + Closes #5102 -- test1323: Added the missing 'unit test' feature requirement in the test case +Marc Hoersken (3 Apr 2020) +- sockfilt: remove redundancy in timeout handling + + And update other logmsg output in select_ws on Windows. -Daniel Stenberg (26 Feb 2020) -- cookie: remove unnecessary check for 'out != 0' +- sockfilt: fix handling of ready closed sockets on Windows - ... as it will always be non-NULL at this point. + Replace the incomplete workaround regarding FD_CLOSE + only signalling once by instead doing a pre-check with + standard select and storing the result for later use. - Detected by Coverity: CID 1459009 + select keeps triggering on closed sockets on Windows while + WSAEventSelect fires only once with data still available. + By doing the pre-check we do not run in a deadlock + due to waiting forever for another FD_CLOSE event. -- http: added 417 response treatment +- sockfilt: fix race-condition of waiting threads and event handling - When doing a request with a body + Expect: 100-continue and the server - responds with a 417, the same request will be retried immediately - without the Expect: header. + Fix race-condition of waiting threads finishing while events are + already being processed which lead to invalid or skipped events. - Added test 357 to verify. + Use mutex to check for one event at a time or do post-processing. + In addition to mutex-based locking use specific event as signal. - Also added a control instruction to tell the sws test server to not read - the request body if Expect: is present, which the new test 357 uses. + Closes #5156 + +Daniel Stenberg (2 Apr 2020) +- [Leo Neat brought this change] + + CI-fuzz: increase fuzz time to 40 minutes - Reported-by: bramus on github - Fixes #4949 - Closes #4964 + Closes #5174 -Steve Holme (26 Feb 2020) -- smtp: Tidy up, following recent changes, to maintain the coding style +Marc Hoersken (2 Apr 2020) +- CI: increase Azure Pipelines timeouts due to performance issues - Closes #4892 + The current demand on Azure negatively impacts the CI performance. -- smtp: Support the SMTPUTF8 extension for the EXPN command +- runtests.pl: log host OS as detected by Perl environment + +- ftpserver.pl: log before and after data connection is closed + +Daniel Stenberg (1 Apr 2020) +- RELEASE-NOTES: synced + +- RELEASE-PROCEDURE.md: run the copyright.pl script! + +- vquic/ngtcp2.h: update copyright year range - Simply notify the server we support the SMTPUTF8 extension if it does. + Follow-up to 0736ee73d346a52 -- smtp: Support the SMTPUTF8 extension in the VRFY command +- [Daiki Ueno brought this change] -- smtp: Support the SMTPUTF8 extension in the RCPT TO command + CI: add build with ngtcp2 + gnutls on Travis CI + +- [Daiki Ueno brought this change] + + vquic: add support for GnuTLS backend of ngtcp2 - Note: The RCPT TO command isn't required to advertise to the server that - it contains UTF-8 characters, instead the server is told that a mail may - contain UTF-8 in any envelope command via the MAIL command. + Currently, the TLS backend used by vquic/ngtcp2.c is selected at compile + time. Therefore OpenSSL support needs to be explicitly disabled. + + Signed-off-by: Daiki Ueno + Closes #5148 -- smtp: Support the SMTPUTF8 extension in the MAIL command +- [Gisle Vanem brought this change] + + examples/sessioninfo.c: add include to fix compiler warning - Support the SMTPUTF8 extension when sending mailbox information in the - MAIL command (FROM and AUTH parameters). Non-ASCII domain names will - be ACE encoded, if IDN is supported, whilst non-ASCII characters in - the local address part are passed to the server. + Fixes #5171 + +- misc: copyright year updates - Reported-by: ygthien on github - Fixes #4828 + Follow-up to 7a71965e9 -- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 +- [Harry Sintonen brought this change] -- smtp: Support UTF-8 based host names in the VRFY command + build: fixed build for systems with select() in unistd.h + + Closes #5169 -- smtp: Support UTF-8 based host names in the RCPT TO command +- memdebug: don't log free(NULL) + + ... it serves no purpose and fills up the log. -- smtp: Support UTF-8 based host names in the MAIL command +- cleanup: insert newline after if() conditions - Non-ASCII host names will be ACE encoded if IDN is supported. + Our code style mandates we put the conditional block on a separate + line. These mistakes are now detected by the updated checksrc. -- url: Make the IDN conversion functions available to others +- checksrc: warn on obvious conditional blocks on the same line as if() + + Closes #5164 -- smtp: Added UTF-8 mailbox tests to verify existing behaviour +- [Roger Orr brought this change] -- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case + cmake: add CMAKE_MSVC_RUNTIME_LIBRARY + + Fixes #5165 + Closes #5167 -- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() +- [Daiki Ueno brought this change] + + ngtcp2: update to git master for the key installation API change - The dot character between the host and the tld was not being escaped, - which meant it specified a match of 'any' character rather than an - explicit dot separator. + This updates the ngtcp2 OpenSSL backend to follow the API change in + commit 32e703164 of ngtcp2. - Additionally removed the dot character from the host name as it allowed - the following to be specified as a valid address in our test cases: + Notable changes are: + - ngtcp2_crypto_derive_and_install_{rx,tx}_key have been added to replace + ngtcp2_crypto_derive_and_install_key + - the 'side' argument of ngtcp2_crypto_derive_and_install_initial_key + has been removed - + Fixes #5166 + Closes #5168 + +- [Cyrus brought this change] + + SECURITY.md: minor rephrase - Both are typos from 98f7ca7 and 8880f84 :( + Closes #5158 + +- output.d: quote the URL when globbing - I can't remember whether my intention was to allow sub-domains to be - specified in the host or not with these additional dots, but by placing - it outside of the host means it can only be specified once per domain - and by placing a + after the new grouping support for sub-domains is - kept. + Some shells do globbing of their own unless the URL is quoted, so maybe + encourage this. - Closes #4912 + Co-authored-by: Jay Satiro + Closes #5160 -- hmac: Added a unit test for the HMAC hash generation +- dist: add tests/version-scan.pl to tarball - Closes #4973 + ... used in test 1177. + + Follow-up to a97d826f6de3 -- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function +- test1177: verify that all the CURL_VERSION_ bits are documented -- tests: Added a unit test for MD4 digest generation +- curl.h: remnove CURL_VERSION_ESNI. Never supported nor documented - Closes #4970 + Considered experimental and therefore we can do this. + + Closes #5157 -- md4: Use const for the length input parameter +- KNOWN_BUGS: DoH doesn't inherit all transfer options - This keeps the interface the same as md5 and sha256. + Closes #4578 + Closes #4579 -- test1610: Fixed the link to the unit test +- KNOWN_BUGS: DoH leaks memory after followlocation - Typo from 81c37124. + Closes #4592 -- ntlm: Removed the dependency on the TLS libaries when using MD5 - - As we have our own MD5 implementation use the MD5 wrapper to remove the - TLS dependency. +- KNOWN_BUGS: "FTPS needs session reuse" - Closes #4967 + Closes #4654 -- md5/sha256: Updated the functions to allow non-string data to be hashed +- KNOWN_BUGS: "stick to same family over SOCKS pro" is presumed fixed -- digest: Corrected the name of the local HTTP digest function +- TODO: Set custom client ip when using haproxy protocol - Follow up to 2b5b37cb. Local static functions do not require the Curl - prefix. + Closes #5125 -- tests: Added a unit test for SHA256 digest generation +Michael Kaufmann (27 Mar 2020) +- writeout_json: Fix data type issues - Follow up to 2b5b37c. + Load long values correctly (e.g. for http_code). - Closes #4968 + Use curl_off_t (not long) for: + - size_download (CURLINFO_SIZE_DOWNLOAD_T) + - size_upload (CURLINFO_SIZE_UPLOAD_T) + + The unit for these values is bytes/second, not microseconds: + - speed_download (CURLINFO_SPEED_DOWNLOAD_T) + - speed_upload (CURLINFO_SPEED_UPLOAD_T) + + Fixes #5131 + Closes #5152 -- md4: Fixed compilation issues when using GNU TLS gcrypt +Daniel Stenberg (27 Mar 2020) +- mailmap: fixup a few author names/fields - * Don't include 'struct' in the gcrypt MD4_CTX typedef - * The call to gcry_md_read() should use a dereferenced ctx - * The call to gcry_md_close() should use a dereferenced ctx + Douglas Steinwand, Gökhan Şengün, Jessa Chandler, Julian Z and + Svyatoslav Mishyn + +- version: add 'cainfo' and 'capath' to version info struct - Additional minor whitespace issue in the USE_WIN32_CRYPTO code. + Suggested-by: Timothe Litt + URL: https://curl.haxx.se/mail/lib-2020-03/0090.html + Reviewed-by: Jay Satiro - Closes #4959 + Closes #5150 -Daniel Stenberg (21 Feb 2020) - RELEASE-NOTES: synced -- http2: now require nghttp2 >= 1.12.0 +Jay Satiro (26 Mar 2020) +- SSLCERTS.md: Fix example code for setting CA cert file - To simplify our code and since earlier versions lack important function - calls libcurl needs to function correctly. - - nghttp2 1.12.0 was relased on June 26, 2016. + Prior to this change the documentation erroneously said use + CURLOPT_CAPATH to set a CA cert file. - Closes #4961 - -- gtls: fix the copyright year + Bug: https://curl.haxx.se/mail/lib-2020-03/0121.html + Reported-by: Timothe Litt - Follow-up from 41fcb4f609 + Closes https://github.com/curl/curl/pull/5151 -- [jethrogb brought this change] - - GnuTLS: Always send client cert +Marc Hoersken (26 Mar 2020) +- sockfilt: add logmsg output to select_ws_wait_thread on Windows - TLS servers may request a certificate from the client. This request - includes a list of 0 or more acceptable issuer DNs. The client may use - this list to determine which certificate to send. GnuTLS's default - behavior is to not send a client certificate if there is no - match. However, OpenSSL's default behavior is to send the configured - certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL - behavior. + Assisted-by: Jay Satiro + Reviewed-by: Daniel Stenberg - Authored-by: jethrogb on github - Fixes #1411 - Closes #4958 - -- [Leo Neat brought this change] + Closes #5086 - github action: add CIFuzz +Daniel Stenberg (26 Mar 2020) +- docs/make: generate curl.1 from listed files only - Closes #4960 - -- cleanup: comment typos + Previously it rendered the page from files matching "*.d" in the correct + directory, which worked fine in git builds when the files were added but + made it easy to forget adding the files to the dist. - Spotted by 'codespell' + Now, only man page sections listed in DPAGES in Makefile.inc will be + used, thus "forcing" us to update this to get the man page right and get + it included in the dist at the same time. - Closes #4957 + Ref: #5146 + Closes #5149 -Steve Holme (20 Feb 2020) -- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions +- openssl: adapt to functions marked as deprecated since version 3 - Whilst lib\md4.c used this pre-processor, lib\md5.c and - src\tool_metalink.c did not and simply relied on the WIN32 - pre-processor directive. + OpenSSL 3 deprecates SSL_CTX_load_verify_locations and the MD4, DES + functions we use. - Reviewed-by: Marcel Raad - Closes #4955 - -Daniel Stenberg (19 Feb 2020) -- connect: remove some spurious infof() calls + Fix the MD4 and SSL_CTX_load_verify_locations warnings. - As they were added primarily for debugging, they provide little use for - users. + In configure, detect OpenSSL v3 and if so, inhibit the deprecation + warnings. OpenSSL v3 deprecates the DES functions we use for NTLM and + until we rewrite the code to use non-deprecated functions we better + ignore these warnings as they don't help us. - Closes #4951 + Closes #5139 -- HTTP-COOKIES: mention that a trailing newline is required +- dist: add mail-rcpt-allowfails.d to the tarball - ... so that we know we got the whole and not a partial line. + Reported-by: Maksim Stsepanenka + Reviewed-by: Jat Satiro - Also, changed the formatting of the fields away from a table again since - the table format requires a github-markdown tool version that we don't - run on the web server atm. + Closes #5146 + +- travis: update the ngtcp2 build to use the latest OpenSSL patch - Reported-by: Sunny Bean - Fixes #4946 - Closes #4947 + ... which also makes it OpenSSL 1.1.1d based and not v3. -- nit: Copyright year out of date +Marc Hoersken (24 Mar 2020) +- CI: remove default Ubuntu build from GitHub Actions - Follow-up to 1fc0617dcc + We are already running a very similar Ubuntu build on Travis CI. + The macOS variant of this default build is kept on Github Actions. -Jay Satiro (18 Feb 2020) -- tool_util: Improve Windows version of tvnow() +- CI: bring GitHub Actions fuzzing job in line with macOS jobs - - Change tool_util.c tvnow() for Windows to match more closely to - timeval.c Curl_now(). + Update YAML formatting, job naming and triggers. + +- CI: migrate macOS jobs from Azure and Travis CI to GitHub Actions - - Create a win32 init function for the tool, since some initialization - is required for the tvnow() changes. + Reduce workload on Azure Pipelines and Travis CI while + consolidating macOS jobs onto less utilized GitHub Actions. - Prior to this change the monotonic time function used by curl in Windows - was determined at build-time and not runtime. That was a problem because - when curl was built targeted for compatibility with old versions of - Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps - every 49.7 days that Windows has been running. + Reviewed-by: Daniel Stenberg - This change makes curl behave similar to libcurl's tvnow function, which - determines at runtime whether the OS is Vista+ and if so calls - QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used - because it has higher resolution than the more obvious candidate - GetTickCount64). The changes to tvnow are basically a copy and paste but - the types in some cases are different. + Closes #5124 + +Daniel Stenberg (24 Mar 2020) +- config: remove all defines of HAVE_DES_H - Ref: https://github.com/curl/curl/issues/3309 + As there's no code using it. - Closes https://github.com/curl/curl/pull/4847 + Closes #5144 -Daniel Stenberg (18 Feb 2020) -- SOCKS: fix typo in printf formatting +- copyright: fix out-of-date copyright ranges and missing headers - Follow-up to 4a4b63daa + Reported by the new script 'scripts/copyright.pl'. The script has a + regex whitelist for the files that don't need copyright headers. - Reported-by: Peter Piekarski - Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 - -- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section + Removed three (mostly usesless) README files from docs/ - to be in sync with the description above + Closes #5141 + +- packages: add OS400/chkstrings.c to the dist - Reported-by: Joonas Kuorilehto - Fixes #4943 - Closes #4945 + Reported-by: Jon Rumsey + Fixes #5142 + Closes #5143 -- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" +- [Clément Notin brought this change] -- altsvc: make saving the cache an atomic operation + nghttp2: 1.12.0 required - ... by writing the file to temp name then rename to the final when done. + since nghttp2_session_set_local_window_size is needed - Assisted-by: Jay Satiro - Fixes #4936 - Closes #4942 + Closes #5140 -- rename: a new file for Curl_rename() +- RELEASE-NOTES: synced + +- [Calvin Buckley brought this change] + + OS400: Update strings for ccsid-ifier - And make the cookie save function use it. + Fixes build. + + Closes #5132 -- cookies: make saving atomic with a rename +- cirrus: make freebsd ignore the tests instead of skipping - Saves the file as "[filename].[8 random hex digits].tmp" and renames - away the extension when done. + To allow us to see in the CI logs how they actually behave - Co-authored-by: Jay Satiro - Reported-by: Mike Frysinger - Fixes #4914 - Closes #4926 + Closes #5091 -- RELEASE-NOTES: synced +- cirrus: move the sanitizer build from freebsd 13 to freebsd 12 -- socks: make the connect phase non-blocking - - Removes two entries from KNOWN_BUGS. +- Revert "cirrus-ci: disable the FreeBSD 13 builds" - Closes #4907 + This reverts commit 691b71be930f0e285c8f7a76efd56bbe0576cda6. -- multi: if Curl_readwrite sets 'comeback' use expire, not loop +- getinfo: provide CURLINFO_HEADER_SIZE and CURLINFO_REQUEST_SIZE override - Otherwise, a very fast single transfer ricks starving out other - concurrent transfers. + To let debug-builds return fake values, like in test 970. - Closes #4927 + Ref: #5131 + Closes #5136 -- ftp: convert 'sock_accepted' to a plain boolean +- test970: improve the test - This was an array indexed with sockindex but it was only ever used for - the secondary socket. + - send more data to make problems more obvious + - don't start the data with minus, it makes diffs harder to read + - skip the headers in the stdout comparison + - save to a file name to also verify 'filename_effective' - Closes #4929 + Ref: #5131 -Jay Satiro (15 Feb 2020) -- CURLINFO_COOKIELIST.3: Fix example +- CURLINFO_NUM_CONNECTS: improve accuracy - Prior to this change the example would try to import cookies from stdin, - which wasn't what was intended. - - Reported-by: 3dyd@users.noreply.github.com + The counter was not bumped in all cases correctly. - Fixes https://github.com/curl/curl/issues/4930 + Reported-by: Marcel Raad + Ref: #5131 + Closes #5135 -Daniel Stenberg (14 Feb 2020) -- TODO: Paged searches on LDAP server - - Closes #4452 +- TODO: Use "random" ports for the test servers -- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS +- lib/curl_setup: adjust the copyright year range - Closes #4108 + Follow-up from d820224b8 -- azure: disable brotli on the macos debug-builds +Jay Satiro (21 Mar 2020) +- curl_setup: define _WIN32_WINNT_[OS] symbols - Because of: + .. because not all Windows build systems have those symbols, and even + those that do may be missing newer symbols (eg the Windows 7 SDK does + not define _WIN32_WINNT_WIN10). - brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] - const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], + Those symbols are used in build-time logic to decide which API to use + and prior to this change if the symbols were missing it would have + resulted in deprecated API being used when more recent functions were + available (eg GetVersionEx used instead of VerifyVersionInfo). - Closes #4925 - -Steve Holme (13 Feb 2020) -- tool_home: Fix the copyright year being out of date + Reported-by: FuccDucc@users.noreply.github.com - Follow up to 9dc350b6. + Probably fixes https://github.com/curl/curl/issues/4995 + Closes https://github.com/curl/curl/pull/5057 -Jay Satiro (12 Feb 2020) -- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() +- [Ross Burton brought this change] + + curl-functions.m4: remove inappropriate AC_REQUIRE - - Deduplicate GetEnv() code. + AC_REQUIRE means "if this macro hasn't been executed already, execute + it". So in a wrapper around AC_RUN_IFELSE, AC_REQUIRE(AC_RUN_IFELSE) + isn't correct at that will execute AC_RUN_IFELSE without any arguments. - - On Windows change ultimate call to use Windows API - GetEnvironmentVariable() instead of C runtime getenv(). + With autoconf 2.69 this is basically a no-op, but with autoconf 2.70, + AC_RUN_IFELSE without a default value when cross-compiling is fatal. + The result is that curl with autoconf 2.70 cannot cross-compile. - Prior to this change both libcurl and the tool had their own GetEnv - which over time diverged. Now the tool's GetEnv is a wrapper around - curl_getenv (libcurl API function which is itself a wrapper around - libcurl's GetEnv). + Fixes https://github.com/curl/curl/issues/5126 + Closes https://github.com/curl/curl/pull/5130 + +Marc Hoersken (20 Mar 2020) +- ci/tests: fix Azure Pipelines not running Windows containers - Furthermore this change fixes a bug in that Windows API - GetEnvironmentVariable() is called instead of C runtime getenv() to get - the environment variable since some changes aren't always visible to the - latter. + Workaround posted here: microsoft/azure-pipelines-agent#2864 - Reported-by: Christoph M. Becker + Assisted-by: Simon Chalifoux + Assisted-by: Tommy Petty - Fixes https://github.com/curl/curl/issues/4774 - Closes https://github.com/curl/curl/pull/4863 + Fixes #5117 + Closes #5129 -Daniel Stenberg (12 Feb 2020) -- strerror.h: Copyright year out of date +Daniel Stenberg (20 Mar 2020) +- tests: add test 430, 431 and 432 to verify the --config fix - Follow-up to 1c4fa67e8a8fcf6 + Verify the fixes in 4e0b4fee4 -Jay Satiro (12 Feb 2020) -- strerror: Increase STRERROR_LEN 128 -> 256 +- [Rici Lake brought this change] + + cmdline: fix handling of OperationConfig linked list (--next) - STRERROR_LEN is the constant used throughout the library to set the size - of the buffer on the stack that the curl strerror functions write to. + Ensures that -K/--config inserts new items at the end of the list + instead of overwriting the second item, and that after a -K/--config + option has been parsed, the option parser's view of the current config + is update. - Prior to this change some extended length Windows error messages could - be truncated. + Fixes #5120 + Closes #5123 + +Marc Hoersken (20 Mar 2020) +- test2100: fix static port instead of dynamic value being used + +- test970: fix static ip:port instead of dynamic values being used + +Daniel Stenberg (19 Mar 2020) +- secure transport: remove the BACKEND define kludge - Closes https://github.com/curl/curl/pull/4920 + Closes #5122 -- multi: fix outdated comment +- mbedtls: remove the BACKEND define kludge + +- bearssl: remove the BACKEND define kludge + +- wolfssl: remove the BACKEND define kludge + +- nss: remove the BACKEND define kludge + +- gnutls: remove the BACKEND define kludge + +- openssl: remove the BACKEND define kludge - - Do not say that conn->data is "cleared" by multi_done(). + Use a proper variable instead to make it easier to use a debugger and + read the code. + +Marc Hoersken (19 Mar 2020) +- tests: make Python-based servers compatible with Python 2 and 3 - If the connection is in use then multi_done assigns another easy handle - still using the connection to conn->data, therefore in that case it is - not cleared. + Update smbserver.py and negtelnetserver.py to be compatible with + Python 3 while staying backwards-compatible to support Python 2. - Closes https://github.com/curl/curl/pull/4901 - -- easy: remove dead code + Fix string encoding and handling of echoed and transferred data. - multi is already assigned to data->multi by curl_multi_add_handle. + Tested with both Python 2.7.17 and Python 3.7.7 - Closes https://github.com/curl/curl/pull/4900 + Reported-by: Daniel Stenberg + Assisted-by: Kamil Dudka + Reviewed-by: Marcel Raad + + Fixes #5104 + Closes #5110 -Daniel Stenberg (12 Feb 2020) -- create-dirs.d: mention the mode +Daniel Stenberg (18 Mar 2020) +- writeout_json: use curl_off_t printf() option for the time output - Reported-by: Dan Jacobson - Fixes #4766 - Closes #4916 + Follow-up to: 04c03416e68fd635a15 + + Closes #5115 -- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording +- RELEASE-NOTES: synced - Assisted-by: Jay Satiro - Reported-by: Craig Andrews - Fixes #4909 - Closes #4910 + Uh, I missed this in 1a46b218db - RELEASE-NOTES: synced + + ... and bumped curlver.h to 7.70.0 -Steve Holme (9 Feb 2020) -- smtp: Simplify the MAIL command and avoid a duplication of send strings +Jay Satiro (18 Mar 2020) +- http2: Fix erroneous debug message that h2 connection closed - This avoids the duplication of strings when the optional AUTH and SIZE - parameters are required. It also assists with the modifications that - are part of #4892. + Prior to this change in libcurl debug builds http2 stream closure was + erroneously referred to as connection closure. - Closes #4903 - -Daniel Stenberg (9 Feb 2020) -- altsvc: keep a copy of the file name to survive handle reset + Before: + * nread <= 0, server closed connection, bailing - The alt-svc cache survives a call to curl_easy_reset fine, but the file - name to use for saving the cache was cleared. Now the alt-svc cache has - a copy of the file name to survive handle resets. + After: + * nread == 0, stream closed, bailing - Added test 1908 to verify. + Closes https://github.com/curl/curl/pull/5118 + +Daniel Stenberg (18 Mar 2020) +- tool_setopt: correct the copyright year range - Reported-by: Craig Andrews - Fixes #4898 - Closes #4902 + Follow-up to 5450428491 -Steve Holme (9 Feb 2020) -- url: Include the failure reason when curl_win32_idn_to_ascii() fails +Jay Satiro (18 Mar 2020) +- [Johannes Schindelin brought this change] + + schannel: add "best effort" revocation check option - Provide the failure reason in the failf() info just as we do for the - libidn2 version of code. + - Implement new option CURLSSLOPT_REVOKE_BEST_EFFORT and + --ssl-revoke-best-effort to allow a "best effort" revocation check. - Closes #4899 - -Jay Satiro (9 Feb 2020) -- asyn-thread: remove dead code - -Daniel Stenberg (8 Feb 2020) -- [Emil Engler brought this change] - - github: Instructions to post "uname -a" on Unix systems in issues + A best effort revocation check ignores errors that the revocation check + was unable to take place. The reasoning is described in detail below and + discussed further in the PR. - Closes #4896 - -- [Cristian Greco brought this change] - - configure.ac: fix comments about --with-quiche - - A simple s/nghttp3/quiche in some comments of --with-quiche. - Looks like a copy-paste error from --with-nghttp3. + --- - Closes #4897 - -Steve Holme (7 Feb 2020) -- checksrc.bat: Fix not being able to run script from the main curl directory + When running e.g. with Fiddler, the schannel backend fails with an + unhelpful error message: - If the script was ran from the main curl directory rather then the - projects directory then the script would simply exit without error: + Unknown error (0x80092012) - The revocation function was unable + to check revocation for the certificate. - C:\url> projects\checksrc.bat + Sadly, many enterprise users who are stuck behind MITM proxies suffer + the very same problem. - The user would either need to change to the projects directory, - explicitly specify the current working directory, or perform a - oneline hacky workaround: + This has been discussed in plenty of issues: + https://github.com/curl/curl/issues/3727, + https://github.com/curl/curl/issues/264, for example. - C:\url> cd projects - C:\url\projects> checksrc.bat + In the latter, a Microsoft Edge developer even made the case that the + common behavior is to ignore issues when a certificate has no recorded + distribution point for revocation lists, or when the server is offline. + This is also known as "best effort" strategy and addresses the Fiddler + issue. - C:\url> checksrc.bat %cd% + Unfortunately, this strategy was not chosen as the default for schannel + (and is therefore a backend-specific behavior: OpenSSL seems to happily + ignore the offline servers and missing distribution points). - C:\url> pushd projects & checksrc.bat & popd + To maintain backward-compatibility, we therefore add a new flag + (`CURLSSLOPT_REVOKE_BEST_EFFORT`) and a new option + (`--ssl-revoke-best-effort`) to select the new behavior. - Closes #4894 - -Daniel Stenberg (7 Feb 2020) -- [Pierre-Yves Bigourdan brought this change] - - digest: Do not quote algorithm in HTTP authorisation + Due to the many related issues Git for Windows and GitHub Desktop, the + plan is to make this behavior the default in these software packages. - RFC 7616 section 3.4 (The Authorization Header Field) states that "For - historical reasons, a sender MUST NOT generate the quoted string syntax - for the following parameters: algorithm, qop, and nc". This removes the - quoting for the algorithm parameter. + The test 2070 was added to verify this behavior, adapted from 310. - Reviewed-by: Steve Holme - Closes #4890 - -- ftp: remove the duplicated user/password struct fields + Based-on-work-by: georgeok + Co-authored-by: Markus Olsson + Signed-off-by: Johannes Schindelin - Closes #4887 + Closes https://github.com/curl/curl/pull/4981 -- ftp: remove superfluous checking for crlf in user or pwd +- multi: Improve parameter check for curl_multi_remove_handle - ... as this is already done much earlier in the URL parser. + - If an easy handle is owned by a multi different from the one specified + then return CURLM_BAD_EASY_HANDLE. - Also add test case 894 that verifies that pop3 with an encodedd CR in - the user name is rejected. + Prior to this change I assume user error could cause corruption. - Closes #4887 + Closes https://github.com/curl/curl/pull/5116 -Steve Holme (6 Feb 2020) -- ntlm_wb: Use Curl_socketpair() for greater portability +Viktor Szakats (17 Mar 2020) +- windows: suppress UI in all CryptAcquireContext() calls - Reported-by: Daniel Stenberg - Closes #4886 - -Daniel Stenberg (5 Feb 2020) -- [Frank Gevaerts brought this change] + Ref: https://docs.microsoft.com/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontexta#parameters + Reviewed-by: Marc Hörsken + Closes https://github.com/curl/curl/pull/5088 - contributors: Also include people who contributed to curl-www +Daniel Stenberg (17 Mar 2020) +- writeout_json: add missing comma to fix the HTTP version - Closes #4884 - -- [Frank Gevaerts brought this change] + Follow-up to 04c03416e68fd635a15 - contrithanks: Use the most recent tag by default +- test 970: verify --write-out '%{json}' - (similar to 5296abe) + Makes curl_easy_getinfo() of "variable" numerical content instead return + the number set in the env variable `CURL_TIME`. - Closes #4883 - -- scripts: use last set tag if none given + Makes curl_version() of "variable" textual content. This guarantees a + stable version string which can be tested against. Environment variable + `CURL_VERSION` defines the content. - Makes 'delta' and 'contributors.sh' easier to use. + Assisted-by: Mathias Gumz + +- [Mathias Gumz brought this change] + + writeout: support to generate JSON output - Make the delta script invoke contrithanks to get current number of - contributors instead of counting THANKS, for accuracy. + This commit adds support to generate JSON via the writeout feature: - Closes #4881 - -- ftp: shrink temp buffers used for PORT + -w "%{json}" - These two stack based buffers only need to be 46 + 66 bytes instead of - 256 + 1024. + It leverages the existing infrastructure as much as possible. Thus, + generating the JSON on STDERR is possible by: - Closes #4880 - -- curl: error on --alt-svc use w/o support + -w "%{stderr}%{json}" - Make the tool check for alt-svc support at run-time and return error - accordingly if not present when the option is used. + This implements a variant of + https://github.com/curl/curl/wiki/JSON#--write-out-json. - Reported-by: Harry Sintonen - Closes #4878 + Closes #4870 -- docs/HTTP3: add --enable-alt-svc to curl's configure +- CI: stop ignoring 323, it is disabled -- RELEASE-PROCEDURE: feature win is closed post-release a few days +- DISABLED: disable test 323 - We've tried to uphold this already but let's make it official by - publicly stating this is the way we do it. + The test uses SRP to "a server not supporting it" but modern stunnel + versions will silently accept it and remain happy. The test is therefore + faulty. - Closes #4877 + I haven't figured out how to make stunnel explicitly reject SRP-using + connects. + + Reported-by: Marc Hörsken + Fixes #5105 + Closes #5113 -- altsvc: set h3 version at a common single spot +Marc Hoersken (17 Mar 2020) +- ci/tests: increase timeouts for torture builds on Azure Pipelines - ... and move the #ifdefs out of the functions. Addresses the fact they - were different before this change. + For some reason the torture builds have slowed down recently. - Reported-by: Harry Sintonen - Closes #4876 - -- [Harry Sintonen brought this change] + Reported-by: Daniel Stenberg - altsvc: improved header parser - - - Fixed the flag parsing to apply to specific alternative entry only, as - per RFC. The earlier code would also get totally confused by - multiprotocol header, parsing flags from the wrong part of the header. +Daniel Stenberg (16 Mar 2020) +- cmake: add support for building with wolfSSL - - Fixed the parser terminating on unknown protocols, instead of skipping - them. + My working build cmdline: - - Fixed a busyloop when protocol-id was present without an equal sign. + $ cmake -DCMAKE_PREFIX_PATH=$HOME/build-wolfssl -DCMAKE_USE_WOLFSSL=ON . - Closes #4875 - -- [Harry Sintonen brought this change] + Assisted-by: Brad King + Closes #5095 - ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 - -- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 +- tool_operate: fix add_parallel_transfers when more are in queue - Reported-by: James Fuller - -Steve Holme (4 Feb 2020) -- ntlm: Pass the Curl_easy structure to the private winbind functions + Trying to return early from the function if no new transfers were added + would break the "morep" argument and cause issues. This could lead to + zero content "transfers" (within quotes since they would never be + started) when parallel-max was reduced. - ...rather than the full conndata structure. + Reported-by: Gavin Wong + Analyzed-by: Jay Satiro + Fixes #4937 + Closes #5112 -Daniel Stenberg (4 Feb 2020) -- RELEASE-NOTES: synced +- vtls: free ssl_config leftovers on out-of-memory + + Torture testing 2034 and 2037 found this. + + Reported-by: Marc Hörsken + Fixes #5108 + Closes #5109 -- tool_operhlp: Copyright year out of date, should be 2020 +Marc Hoersken (16 Mar 2020) +- ci/tests: fix Azure Pipelines not running for pull requests - Follow-up from 2bc373740a3 + Closes #5111 -- [Orgad Shaneh brought this change] +Daniel Stenberg (15 Mar 2020) +- gskit: update the copyright year range + + Follow-up from 083603c63a3 - curl: avoid using strlen for testing if a string is empty +Marc Hoersken (15 Mar 2020) +- gskit: use our internal select wrapper for portability - Closes #4873 + Follow up to c52b342 + Closes #5106 -Steve Holme (3 Feb 2020) -- ntlm: Ensure the HTTP header data is not stored in the challenge/response +- tests: fix verification of stdout in test 1452 due to newline + + Fixes test1452:41:1: error: missing tag before -Marcel Raad (3 Feb 2020) -- openssl: remove redundant assignment +- ci/tests: install impacket for SMB tests on FreeBSD using CirrusCI - Fixes a scan-build failure on Bionic. + Also force the package index/cache to be updated before installing. - Closes https://github.com/curl/curl/pull/4872 + Closes #5103 -- travis: update non-OpenSSL Linux jobs to Bionic +- tests/README: add note about manually installing python-impacket - For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is - failing with "curl returned 52, when expecting 35". + Follow up to 4be2560 + +Daniel Stenberg (15 Mar 2020) +- transfer: cap retries of "dead connections" to 5 - Closes https://github.com/curl/curl/pull/4872 + When libcurl retries a connection due to it being "seemingly dead" or by + REFUSED_STREAM, it will now only do it up five times before giving up, + to avoid never-ending loops. + + Reported-by: Dima Tisnek + Bug: https://curl.haxx.se/mail/lib-2020-03/0044.html + Closes #5074 -Dan Fandrich (3 Feb 2020) -- cirrus: Add some missing semicolons +- TODO: TLS-PSK with OpenSSL - Newlines aren't preserved in this section so they're needed to separate - commands. The exports luckily worked anyway as a single long line, but - erroneously exported a variable called "export" - [skip ci] + Closes #5081 -Daniel Gustafsson (2 Feb 2020) -- [Pedro Monreal brought this change] +Marc Hoersken (15 Mar 2020) +- select: add 'timeout_ms' wrap-around precaution to Curl_select - cleanup: fix typos and wording in docs and comments +- select: fix 'pending_ms' is assigned a value that is never used - Closes #4869 - Reviewed-by: Emil Engler and Daniel Gustafsson + Detected by Codacy -Steve Holme (2 Feb 2020) -- ntlm: Move the winbind data into the NTLM data structure +- select: move duplicate select preparation code into Curl_select - To assist with adding winbind support to the SASL NTLM authentication, - move the winbind specific data out of conndata into ntlmdata. + Reviewed by Daniel Stenberg + Reviewed by Marcel Raad + Closes #5078 -Daniel Stenberg (30 Jan 2020) -- quiche: Copyright year out of date +Daniel Stenberg (15 Mar 2020) +- connect: happy eyeballs cleanup - Follow-up to 7fc63d72333a - -- altsvc: use h3-25 + Make sure each separate index in connn->tempaddr[] is used for a fixed + family (and only that family) during the connection process. - Closes #4868 + If family one takes a long time and family two fails immediately, the + previous logic could misbehave and retry the same family two address + repeatedly. + + Reported-by: Paul Vixie + Reported-by: Jay Satiro + Fixes #5083 + Fixes #4954 + Closes #5089 -- [Alessandro Ghedini brought this change] +Marc Hoersken (15 Mar 2020) +- ci/tests: fix and align setting TFLAGS for make test-nonflaky - quiche: update to draft-25 - - Closes #4867 +- ci/tests: install test suite dependencies stunnel and impacket -- ngtcp2: update to git master and its draft-25 support +- tests: remove python_dependencies for smbserver from our tree - Closes #4865 + Users of the SMB tests will have to install impacket manually. + + Reasoning: our in-tree version of impacket was quite outdated + and only compatible with Python 2 which is already end-of-life. + Upgrading to Python 3 and a compatible impacket version would + require to import additional Python-only and CPython-extension + dependencies. This would have hindered portability enormously. + + Closes #5094 -- cookie: check __Secure- and __Host- case sensitively +Jay Satiro (14 Mar 2020) +- Makefile.m32: Improve windres parameter compatibility - While most keywords in cookies are case insensitive, these prefixes are - specified explicitly to get checked "with a case-sensitive match". + - s/COFF/coff/ - (From the 6265bis document in progress) + Some versions of windres do not recognize uppercase COFF as a valid + way to specify the COFF output format. - Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 - Closes #4864 - -- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header - -- oauth2-bearer.d: works for HTTP too + Reported-by: Steven Penny - Reported-by: Mischa Salle - Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html - Closes #4862 + Fixes https://github.com/curl/curl/issues/5099 + Closes https://github.com/curl/curl/pull/5101 -- multi_done: if multiplexed, make conn->data point to another transfer +- easy: Fix curl_easy_duphandle for builds missing IPv6 that use c-ares - ... since the current transfer is being killed. Setting to NULL is - wrong, leaving it pointing to 'data' is wrong since that handle might be - about to get freed. + - Ignore CURLE_NOT_BUILT_IN errors returned by c-ares functions in + curl_easy_duphandle. - Fixes #4845 - Closes #4858 - Reported-by: dmitrmax on github + Prior to this change if c-ares was used as the resolver backend and + either it was too old or libcurl was built without IPv6 support then + some of our resolver functions could return CURLE_NOT_BUILT_IN to + curl_easy_duphandle causing it to fail. + + Caused by c8f086b which shipped in 7.69.1. + + Reported-by: Karl Chen + + Fixes https://github.com/curl/curl/issues/5097 + Closes https://github.com/curl/curl/pull/5100 -- location.d: the method change is from POST to GET only +Daniel Stenberg (13 Mar 2020) +- docs: add warnings about FILE: URLs on Windows - Not from generic non-GET to GET. + - --url man page section + - libcurl-security.3 gets the full text + - CURLOPT_URL.3 - Reported-by: Andrius Merkys - Ref: #4859 - Closes #4861 + Reported-by: Tim Sedlmeyer -- urlapi: guess scheme correct even with credentials given +- server/getpart: make the "XML-parser" stricter - In the "scheme-less" parsing case, we need to strip off credentials - first before we guess scheme based on the host name! + When extracting a
and there's no before +
, this now outputs an error and returns a wrong string to + make users spot the mistake. - Assisted-by: Jay Satiro - Fixes #4856 - Closes #4857 + Ref: #5070 + Closes #5071 -- global_init: move the IPv6 works status bool to multi handle +Marc Hoersken (13 Mar 2020) +- impacket: some more Python 3 code compatibility updates - Previously it was stored in a global state which contributed to - curl_global_init's thread unsafety. This boolean is now instead figured - out in curl_multi_init() and stored in the multi handle. Less effective, - but thread safe. + This makes smbserver load on Python 3, but still not work completely. + +- smbserver: pin Python version to 2 since we are not yet 3 compatible - Closes #4851 + Even though the existing code can be fixed to run on Python 3, the + tests will fail due to the Unicode transition the protocol is invalid. + + Follow up to ee63837 + Closes #5085 -- [Jay Satiro brought this change] +Daniel Stenberg (12 Mar 2020) +- [Viktor Szakats brought this change] - README: mention that the docs is in docs/ + cleanup: fix some text/comment typos - Reported-by: Austin Green - Fixes #4830 - Closes #4853 + Closes #5087 -- curl.h: define CURL_WIN32 on windows +Marc Hoersken (12 Mar 2020) +- smbserver: fix Python version specific ConfigParser import - ... so that the subsequent logic below can use a single known define to know - when built on Windows (as we don't define WIN32 anymore). + Follow up to ee63837 and 8c7c4a6 + Fixes #5077 + +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: synced - Follow-up to 1adebe7886ddf20b + bumped to 7.69.2 + +Dan Fandrich (11 Mar 2020) +- tests/data: Fix some XML formatting issues in test cases - Reported-by: crazydef on github - Assisted-by: Marcel Raad - Fixes #4854 - Closes #4855 + This allows these test files to pass xmllint. -- RELEASE-NOTES: synced +Daniel Stenberg (11 Mar 2020) +- [Muhammad Herdiansyah brought this change] -- [Jon Rumsey brought this change] + Makefile: run the cd commands in a subshell + + In bmake, if the directory is changed (with cd or anything else), bmake + won't return to the "root directory" on the next command (in the same + Makefile rule). This commit runs the cd command in a subshell so it + would work in bmake. + + Closes #5073 - urldata: do string enums without #ifdefs for build scripts +- configure: convert -I to -isystem as a last step - ... and check for inconsistencies for OS400 at build time with the new - chkstrings tool. + As all the -I uses in CFLAGS at that point are for system headers and + third party libraries this helps us remove/ignore warnings on those! - Closes #4822 + Closes #5060 -- curl: make the -# spaceship bar not wrap the line +- configure: fix -pedantic-errors for GCC 5 and later - The fixed-point math made us lose precision and thus a too high index - value could be used for outputting the hashtags which could overwrite - the newline. + If --enable-werror is used. - The fix increases the precision in the sine table (*100) and the - associated position math. + Follow-up to d5c0351055d5709da which added it too early in the configure + script before $compiler_num was set correctly and thus this option was + never used. - Reported-by: Andrew Potter - Fixes #4849 - Closes #4850 + Reported-by: Stepan Efremov + Fixes #5067 + Closes #5068 -- global_init: assume the EINTR bit by default +- configure: document 'compiler_num' for gcc - - Removed from global_init since it isn't thread-safe. The symbol will - still remain to not break compiles, it just won't have any effect going - forward. - - - make the internals NOT loop on EINTR (the opposite from previously). - It only risks returning from the select/poll/wait functions early, and that - should be risk-free. + The CURL_CHECK_COMPILER_GNU_C function sets the number to MAJOR*100 + + MINOR and ignores the patch version, and since gcc version 7 it only + sets it to MAJOR*100. - Closes #4840 + Reported-by: Stepan Efremov + Ref: #5067 + Closes #5069 -- [Peter Piekarski brought this change] +Version 7.69.1 (11 Mar 2020) - conn: do not reuse connection if SOCKS proxy credentials differ - - Closes #4835 +Daniel Stenberg (11 Mar 2020) +- RELEASE-NOTES: 7.69.1 -- llist: removed unused Curl_llist_move() - - (and the corresponding unit test) - - Closes #4842 +- THANKS: from the 7.69.1 release -- conncache: removed unused Curl_conncache_bundle_size() +- [Marc Hoersken brought this change] -- strcase: turn Curl_raw_tolower into static + test1129: fix invalid case of closing XML-tag and Content-Length - Only ever used from within this file. - -- singleuse.pl: support new API functions, fix curl_dbg_ handling + Fixes #5070 + Closes #5072 -- wolfssh: make it init properly via Curl_ssh_init() +Marc Hoersken (10 Mar 2020) +- tests/data: fix static ip instead of dynamic value being used - Closes #4846 - -- [Aron Rotteveel brought this change] + Follow up to 94ced8e - form.d: fix two minor typos +- tests/data: fix static ip:port instead of dynamic values being used - Closes #4843 + Closes #5065 -- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields - - Avoid "reparsing" the content and instead deliver more exactly what is - provided in the certificate and avoid truncating the data after 512 - bytes as done previously. This no longer removes embedded newlines. +- tests/server: fix missing use of exe_ext helper function - Fixes #4837 - Reported-by: bnfp on github - Closes #4841 + Follow up to 9819984 and 3dce984 + Reviewed-By: Daniel Stenberg + Closes #5064 -Jay Satiro (23 Jan 2020) -- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 - - - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. - - Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the - CURLOPT_PROXY_SSL_OPTIONS description. +- runtests: log minimal and maximal used port numbers -Daniel Stenberg (22 Jan 2020) -- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER +Daniel Stenberg (9 Mar 2020) +- [Jim Fuller brought this change] + + sftp: fix segfault regression introduced by #4747 - For now, no cert in the bundle actually sets a date there... + This fix adds a defensive check for the case where the char *name in + struct libssh2_knownhost is NULL - Co-Authored-by: Jay Satiro - Reported-by: Christian Heimes - Fixes #4834 - Closes #4836 + Fixes #5041 + Closes #5062 - RELEASE-NOTES: synced -- [Pavel Volgarev brought this change] - - smtp: Allow RCPT TO command to fail for some recipients +- socks4: fix host resolve regression - Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. + 1. The socks4 state machine was broken in the host resolving phase - Verified with the new tests 3002-3007 + 2. The code now insists on IPv4-only when using SOCKS4 as the protocol + only supports that. - Closes #4816 - -- copyright: fix year ranges + Regression from #4907 and 4a4b63d, shipped in 7.69.0 - follow-up from dea17b519d (one of these days I'll learn to check before - I push) + Reported-by: amishmm on github + Bug: https://github.com/curl/curl/issues/5053#issuecomment-596191594 + Closes #5061 -- [nao brought this change] +- [Patrick Monnerat brought this change] - http: move "oauth_bearer" from connectdata to Curl_easy + silly web server: silent a compilation warning - Fixes the bug where oauth_bearer gets deallocated when we re-use a - connection. + Recent gcc warns when byte count of strncpy() equals the destination + buffer size. Since the destination buffer is previously cleared and + the source string is always shorter, reducing the byte count by one + silents the warning without affecting the result. - Closes #4824 + Closes #5059 -- [Emil Engler brought this change] +- [Patrick Monnerat brought this change] - curl: Let -D merge headers in one file again + cookie: get_top_domain() sets zero length for null domains - Closes #4762 - Fixes #4753 + This silents a compilation warning with gcc -O3. -- data.d: remove "Multiple files can also be specified" - - It is superfluous and could even be misleading. - - Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html - Reported-by: Mike Norton - Closes #4832 +- [Patrick Monnerat brought this change] -Marcel Raad (20 Jan 2020) -- CMake: support specifying the target Windows version + test 1560: avoid valgrind false positives - Previously, it was only possible to set it to Windows Vista or XP by - setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. - Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to - explicitly set the target Windows version. `ENABLE_INET_PTON` is - ignored in this case. + When using maximum code optimization level (-O3), valgrind wrongly + detects uses of uninitialized values in strcmp(). - Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 - Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 - Closes https://github.com/curl/curl/pull/4815 + Preset buffers with all zeroes to avoid that. -Daniel Stenberg (20 Jan 2020) -- http.h: Copyright year out of date, should be 2020 +Steve Holme (8 Mar 2020) +- sha256: Added WinCrypt implementation - Follow-up to 7ff9222ced8c + Closed #5030 -- [加藤郁之 brought this change] +- sha256: Added SecureTransport implementation - HTTP: increase EXPECT_100_THRESHOLD to 1Mb +Daniel Stenberg (7 Mar 2020) +- lib1564: reduce number of mid-wait wakeup calls - Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html + This test does A LOT of *wakeup() calls and then calls curl_multi_poll() + twice. The first *poll() is then expected to return early and the second + not - as the first is supposed to drain the socketpair pipe. - Closes #4814 - -- ROADMAP: thread-safe `curl_global_init()` + It turns out however that when given "excessive" amounts of writes to + the pipe, some operating systems (the Solaris based are known) will + return EAGAIN before the pipe is drained, which in our test case causes + the second *poll() call to also abort early. - I'd like to see this happen. + This change attempts to avoid the OS-specific behaviors in the test by + reducing the amount of wakeup calls from 1234567 to 10. + + Reported-by: Andy Fiddaman + Fixes #5037 + Closes #5058 -- RELEASE-NOTES: synced +- [Patrick Monnerat brought this change] -- wolfssl: use the wc-prefixed symbol alternatives - - The symbols without wc_ prefix are not always provided. + mime: fix the binary encoder to handle large data properly - Ref: https://github.com/wolfSSL/wolfssl/issues/2744 + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Closes #4827 + Fixes #4860 + Closes #4833 + Reported-by: RuurdBeerstra on github -- polarssl: removed +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - As detailed in DEPRECATE.md, the polarssl support is now removed after - having been disabled for 6 months and nobody has missed it. + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix - instead of the former 'polarssl' and the common functions that - previously were shared between mbedtls and polarssl and contained the - name 'polarssl' have now all been renamed to instead say 'mbedtls'. + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 667 checks mimepost using single-byte read callback with encoder. + New test 668 checks the end of part data early detection. - Closes #4825 + Fixes #4826 + Reported-by: MrdUkk on github -Marcel Raad (16 Jan 2020) -- libssh2: fix variable type - - This led to a conversion warning on 64-bit MinGW, which has 32-bit - `long` but 64-bit `size_t`. - - Closes https://github.com/curl/curl/pull/4823 +- [Patrick Monnerat brought this change] -Daniel Stenberg (16 Jan 2020) -- curl:progressbarinit: ignore column width from terminals < 20 + mime: latch last read callback status. - To avoid division by zero - or other issues. + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Reported-by: Daniel Marjamäki - Closes #4818 + Fixes #4813 + Reported-by: MrdUkk on github -- wolfssh: set the password correctly for PASSWORD auth +Marc Hoersken (7 Mar 2020) +- runtests: fix missing use of exe_ext helper function -- wolfssh: remove fprintf() calls (and uses of __func__) +Daniel Stenberg (7 Mar 2020) +- [Ernst Sjöstrand brought this change] -Marcel Raad (14 Jan 2020) -- CMake: use check_symbol_exists also for inet_pton - - It doesn't make much sense to only check if the function can be linked - when it's not declared in any header and that is treated as an error. - With the correct target Windows version set, the function is declared - in ws2tcpip.h and the comment above the modified block is invalid. + ares: store dns parameters for duphandle - Also, move the definition of `_WIN32_WINNT` up to before all symbol - availability checks so that we don't have to care which ones must be - done after it. + With c-ares the dns parameters lives in ares_channel. Store them in the + curl handle and set them again in easy_duphandle. - Tested with Visual Studio 2019 and current MinGW-w64. + Regression introduced in #3228 (6765e6d), shipped in curl 7.63.0. - Closes https://github.com/curl/curl/pull/4808 + Fixes #4893 + Closes #5020 + Signed-off-by: Ernst Sjöstrand -Jay Satiro (13 Jan 2020) -- schannel_verify: Fix alt names manual verify for UNICODE builds - - Follow-up to 29e40a6 from two days ago, which added that feature for - Windows 7 and earlier. The bug only occurred in same. +- version: make curl_version* thread-safe without using global context - Ref: https://github.com/curl/curl/pull/4761 + Closes #5010 -Daniel Stenberg (13 Jan 2020) -- HTTP-COOKIES.md: describe the cookie file format - - ... and refer to that file from from CURLOPT_COOKIEFILE.3 and - CURLOPT_COOKIELIST.3 +- RELEASE-NOTES: synced + +Marc Hoersken (7 Mar 2020) +- tests: use native Sleep function as fallback on Windows - Assisted-by: Jay Satiro - Reported-by: bsammon on github - Fixes #4805 - Closes #4806 + Reviewed-By: Daniel Stenberg + Closes #5054 -- [Tobias Hieta brought this change] +- perl: align order and completeness of Windows OS checks - CMake: Add support for CMAKE_LTO option. - - This enables Link Time Optimization. LTO is a proven technique for - optimizing across compilation units. +Daniel Stenberg (7 Mar 2020) +- tool_cb_see: set correct copyright year range - Closes #4799 - -- RELEASE-NOTES: synced + Follow-up to a39e5bfb9 -- ConnectionExists: respect the max_concurrent_streams limits +Marc Hoersken (7 Mar 2020) +- seek: fix fallback for missing ftruncate on Windows - A regression made the code use 'multiplexed' as a boolean instead of the - counter it is intended to be. This made curl try to "over-populate" - connections with new streams. + This fixes test 198 on versions of MinGW-w64 without ftruncate - This regression came with 41fcdf71a1, shipped in curl 7.65.0. + Reviewed-By: Daniel Stenberg + Reviewed-By: Marcel Raad + Closes #5055 + +- config-win32: Windows does not have ftruncate + +Daniel Stenberg (7 Mar 2020) +- pause: force a connection (re-)check after unpausing - Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same - check. + There might be data available that was already read off the socket, for + example in the TLS layer. - Reported-by: Kunal Ekawde - Fixes #4779 - Closes #4784 + Reported-by: Anders Berg + Fixes #4966 + Closes #5049 -- curl: make #0 not output the full URL +- socks5: switch state properly when the resolve is done - It was not intended nor documented! + Regression from 4a4b63d (and #4907) + Reported-by: vitaha85 on github + Fixes #5053 + Closes #5056 + +Jay Satiro (7 Mar 2020) +- libssh: Fix matching user-specified MD5 hex key - Added test 1176 to verify. + Prior to this change a match would never be successful because it + was mistakenly coded to compare binary data from libssh to a + user-specified hex string (ie CURLOPT_SSH_HOST_PUBLIC_KEY_MD5). - Reported-by: vshmuk on hackerone + Reported-by: fds242@users.noreply.github.com - Closes #4812 + Fixes https://github.com/curl/curl/issues/4971 + Closes https://github.com/curl/curl/pull/4974 -- wolfSSH: new SSH backend +Daniel Stenberg (6 Mar 2020) +- pause: bail out on bad input - Adds support for SFTP (not SCP) using WolfSSH. + A NULL easy handle or an easy handle without an associated connection + cannot be paused or unpaused. - Closes #4231 + Closes #5050 -- curl: remove 'config' field from OutStruct +Steve Holme (6 Mar 2020) +- unit1612: fixed the inclusion and compilation of the HMAC unit test - As it was just unnecessary duplicated information already stored in the - 'per_transfer' struct and that's around mostly anyway. + Follow up to 3f74e5e6 to fix: - The duplicated pointer caused problems when the code flow was aborted - before the dupe was filled in and could cause a NULL pointer access. + - A typo in Makefile.inc where unit1611 was used instead + - Some compilation issues in unit1612.c - Reported-by: Brian Carpenter - Fixes #4807 - Closes #4810 + Closes #5024 -- misc: Copyright year out of date, should be 2020 - - Follow-up to recent commits +Daniel Stenberg (6 Mar 2020) +- pause: return early for calls that don't change pause state - [skip ci] - -Jay Satiro (11 Jan 2020) -- [Santino Keupp brought this change] + Reviewed-by: Patrick Monnerat + Ref: #4833 + Closes #5026 - libssh2: add support for forcing a hostkey type +Jay Satiro (6 Mar 2020) +- curl_share_setopt.3: Note sharing cookies doesn't enable the engine - - Allow forcing the host's key type found in the known_hosts file. + Follow-up to d0a7ee3 which fixed a bug in 7.66.0 that caused + CURL_LOCK_DATA_COOKIE to enable the easy handle's cookie engine. - Currently, curl (with libssh2) does not take keys from your known_hosts - file into account when talking to a server. With this patch the - known_hosts file will be searched for an entry matching the hostname - and, if found, libssh2 will be told to claim this key type from the - server. + Bug: https://curl.haxx.se/mail/lib-2020-03/0019.html + Reported-by: Felipe Gasper - Closes https://github.com/curl/curl/pull/4747 - -- [Nicolas Guillier brought this change] + Closes https://github.com/curl/curl/pull/5048 - cmake: Improve libssh2 check on Windows - - - Add "libssh2" name to FindLibSSH2 library search. +- multi: skip EINTR check on wakeup socket if it was closed - On Windows systems, libSSH2 CMake installation may name the library - "LibSSH2". + - Don't check errno on wakeup socket if sread returned 0 since sread + doesn't set errno in that case. - Prior to this change cmake only checked for name "ssh2". On Linux that - works fine because it will prepend the "lib", but it doesn't do that on - Windows. + This is a follow-up to cf7760a from several days ago which fixed + Curl_multi_wait to stop busy looping sread on the non-blocking wakeup + socket if it was closed (ie sread returns 0). Due to a logic error it + was still possible to busy loop in that case if errno == EINTR. - Closes https://github.com/curl/curl/pull/4804 + Closes https://github.com/curl/curl/pull/5047 -- [Faizur Rahman brought this change] +Daniel Stenberg (6 Mar 2020) +- transfer: set correct copyright year range - schannel: Make CURLOPT_CAINFO work better on Windows 7 - - - Support hostname verification via alternative names (SAN) in the - peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. - - CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a - result CertGetNameString doesn't quite work on those versions of - Windows. This change provides an alternative solution for - CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier - versions of Windows. - - Prior to this change many certificates failed the hostname validation - when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates - now represent multiple hostnames and rely on the alternative names field - exclusively to represent their hostnames. +- urldata: remove the 'stream_was_rewound' connectdata struct member - Reported-by: Jeroen Ooms + ... as it is never set anywhere. - Fixes https://github.com/curl/curl/issues/3711 - Closes https://github.com/curl/curl/pull/4761 - -- [Emil Engler brought this change] + Follow-up to 2f44e94ef + Closes #5046 - ngtcp2: Add an error code for QUIC connection errors +- Revert "pause: force-drain the transfer on unpause" - - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection - errors. + This reverts commit fa0216b294af4c7113a9040ca65eefc7fc18ac1c (from #5000) - Prior to this change CURLE_FAILED_INIT was used, but that was not - correct. + Clearly that didn't solve the problem correctly. - Closes https://github.com/curl/curl/pull/4754 + Reported-by: Christopher Reid + Reopens #4966 + Fixes #5044 -- multi: Change curl_multi_wait/poll to error on negative timeout - - - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when - curl_multi_wait/poll is passed timeout param < 0. +- RELEASE-NOTES: synced - Prior to this change passing a negative value to curl_multi_wait/poll - such as -1 could cause the function to wait forever. + and bumped curlver.h + +- MANUAL: update a dict-using command line - Reported-by: hamstergene@users.noreply.github.com + The 'web1913' database is now invalid, use 'gcide' instead. + +- KNOWN_BUGS: configure --with-gssapi with Heimdal is ignored on macOS - Fixes https://github.com/curl/curl/issues/4763 + Closes #3841 + +- polarssl: remove more references and mentions - Closes https://github.com/curl/curl/pull/4765 + Assisted-by: Jay Satiro + Follow-up to 6357a19ff29dac04 + Closes #5036 -- [Marc Aldorasi brought this change] +Marc Hoersken (4 Mar 2020) +- tests: wrap ignored test failures in braces - cmake: Enable SMB for Windows builds +- tests: align some Windows sleep defines with each other + +- tests: try to make sleeping portable by avoiding select - - Define USE_WIN32_CRYPTO by default. This enables SMB. + select does not support just waiting on Windows: + https://perldoc.perl.org/perlport.html#select - - Show whether SMB is enabled in the "Enabled features" output. + Reviewed-By: Daniel Stenberg + Closes #5035 + +Daniel Stenberg (4 Mar 2020) +- runtests.1: rephrase how to specify what tests to run - - Fix mingw compiler warning for call to CryptHashData by casting away - const param. mingw CryptHashData prototype is wrong. + Also mention the new tilde-prefixed way to ignore test results. - Closes https://github.com/curl/curl/pull/4717 + Reviewed-By: Marc Hoersken + Closes #5033 -- vtls: Refactor Curl_multissl_version to make the code clearer +- cirrus-ci: disable the FreeBSD 13 builds - Reported-by: Johannes Schindelin + FreeBSD 13.0 is apparently close to a year away from a stable release + and has proven to cause intermittent builds failures recently. - Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 + Assisted-by: Dan Fandrich + Assisted-by: Fedor Korotkov + Fixes #5028 + Closes #5029 + +Version 7.69.0 (4 Mar 2020) + +Daniel Stenberg (4 Mar 2020) +- RELEASE-NOTES: 7.69.0 + +- THANKS: from 7.69.0 - Closes https://github.com/curl/curl/pull/4803 + Now sorted case insensitive -Daniel Stenberg (10 Jan 2020) -- fix: Copyright year out of date, should be 2020 +Marc Hoersken (3 Mar 2020) +- ci/tests: fix escaping of testnames and disable proxy for CI APIs - Follow-up to 875314ed0bf3b + Follow up to ada581f and c0d8b96 + Closes #5031 -Marcel Raad (10 Jan 2020) -- hostip: move code to resolve IP address literals to `Curl_resolv` +Jay Satiro (3 Mar 2020) +- cmake: Show HTTPS-proxy in the features output - The code was duplicated in the various resolver backends. + - Show HTTPS-proxy in the features output for those backends that + support it: OpenSSL, GnuTLS and NSS. - Also, it was called after the call to `Curl_ipvalid`, which matters in - case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This - caused test 1048 to fail on classic MinGW. - - The code ignores `conn->ip_version` as done previously in the - individual resolver backends. + Prior to this change HTTPS-proxy was missing from the cmake features + output even if curl was built with it. Only cmake output was affected. + Both the library and tool correctly reported the feature. - Move the call to the `resolver_start` callback up to appease test 655, - which wants it to be called also for literal addresses. + Bug: https://curl.haxx.se/mail/lib-2020-03/0008.html + Reported-by: David Lopes - Closes https://github.com/curl/curl/pull/4798 - -Daniel Stenberg (9 Jan 2020) -- scripts/delta: adapt to new public header layout + Closes https://github.com/curl/curl/pull/5025 -- test1167: verify global symbols in public headers are curl prefixed +Marc Hoersken (3 Mar 2020) +- ci/tests: Make it possible to still run but ignore failing tests - ... using the new badsymbols.pl perl script + This enables the development of a solution for the failing tests by + running them on CI while ignoring their result for the overall status. - Fixes #4793 - Closes #4794 + Closes #4994 -- libtest/mk-lib1521: adapt to new public header layout +- README.md: add Azure DevOps Pipelines build status badge -- include: remove non-curl prefixed defines +- ci/tests: Move CI test result creation above environment setup - ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ - enums. - -- curl.h: remove WIN32 define + This avoids using our test servers as proxy to the AppVeyor API. - It isn't our job to define this in a public header - and it defines a - name outside of our naming scope. + Closes #5022 -- tool_dirhie.c: fix the copyright year range +- ci/tests: Send test results to AppVeyor for status overview - Follow-up to: 4027bd72d9 - -- bump: work towards 7.69.0 is started + Closes #5021 -Jay Satiro (9 Jan 2020) -- tool_dirhie: Allow directory traversal during creation - - - When creating a directory hierarchy do not error when mkdir fails due - to error EACCESS (13) "access denied". - - Some file systems allow for directory traversal; in this case that it - should be possible to create child directories when permission to the - parent directory is restricted. - - This is a regression caused by me in f16bed0 (precedes curl-7_61_1). - Basically I had assumed that if a directory already existed it would - fail only with error EEXIST, and not error EACCES. The latter may - happen if the directory exists but has certain restricted permissions. +Daniel Stenberg (3 Mar 2020) +- Revert "sha256: Added SecureTransport implementation" - Reported-by: mbeifuss@users.noreply.github.com + This reverts commit 4feb38deed33fed14ff7c370a6a9153c661dbb9c (from #4956) - Fixes https://github.com/curl/curl/issues/4796 - Closes https://github.com/curl/curl/pull/4797 - -Daniel Stenberg (9 Jan 2020) -- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers + That commit broke test 1610 on macos builds without TLS. - Closes #4080 + Closes #5027 -- docs/RELEASE-PROCEDURE.md: pushed some release dates +- dist: include tests/azure.pm in the tarball - Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html + Bug: https://github.com/curl/curl/commit/ada581f2cc32f48c1629b729707ac19208435b27#commitcomment-37601589 + Reported-by: Marcel Raad -- runtests: make random seed fixed for a month - - When using randomized features of runtests (-R and --shallow) it is - useful to have a fixed random seed to make sure for example extra - commits in a branch or a rebase won't change the seed that would make - repeated runs work differently. - - As it is also useful to change seed sometimes, the default seed is now - determined based on the current month (and first line curl -V - output). When the month changes, so will the random seed. +Steve Holme (3 Mar 2020) +- configure.ac: Disable metalink if mbedTLS is specified - The specific seed is also shown in the standard test suite top header - and it can be set explictly with the new --seed=[num] option so that the - exact order of a previous run can be achieved. + Follow up to cdcc9df1 and #5006. Even though I mentioned mbedTLS as + being one of the backends that metalink needs to be disabled for, I + seem to have included it in the list of allowed SSL/TLS backends in + comnfigure.ac :( - Closes #4734 + Closes #5013 -- RELEASE-PROCEDURE.md: fix next release date (Feb 26) +- sha256: Tidy up following recent changes - [skip ci] - -Version 7.68.0 (8 Jan 2020) + Reviewed-by: Daniel Stenberg + Closes #4956 -Daniel Stenberg (8 Jan 2020) -- RELEASE-NOTES: 7.68.0 +- sha256: Added WinCrypt implementation -- THANKS: updated with names from the 7.68.0 release +- sha256: Added SecureTransport implementation -- RELEASE-PROCEDURE: add four future release dates - - and remove four past release dates - - [skip ci] +- sha256: Added mbedtls implementation -Marcel Raad (6 Jan 2020) -- TrackMemory tests: always remove CR before LF - - It was removed for output containing ' =' via `s/ =.*//`. With classic - MinGW, this made lines with `free()` end with CRLF, but lines with e.g. - `malloc()` end with only LF. The tests expect LF only. - - Closes https://github.com/curl/curl/pull/4788 +- sha256: Added GNU TLS gcrypt implementation -Daniel Stenberg (6 Jan 2020) -- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header - - ... to the private multihhandle.h. It is not for public use and it - wasn't prefixed correctly anyway! - - Closes #4790 +- sha256: Added GNU TLS Nettle implementation -- file: fix copyright year range +Jay Satiro (2 Mar 2020) +- curl_escape.3: Add a link to curl_free - Follow-up to 1b71bc532bd + Ref: https://github.com/curl/curl/pull/5016#issuecomment-593628582 -- curl -w: handle a blank input file correctly +- curl_getenv.3: Fix the memory handling description - Previously it would end up with an uninitialized memory buffer that - would lead to a crash or junk getting output. + - Tell the user to call curl_free() to free the pointer returned by + curl_getenv(). - Added test 1271 to verify. + Prior to this change the user was directed to call free(), but that + would not work in cases where the library and application use separate C + runtimes and therefore have separate heap memory management. - Reported-by: Brian Carpenter - Closes #4786 + Closes https://github.com/curl/curl/pull/5016 -- file: on Windows, refuse paths that start with \\ +Daniel Stenberg (2 Mar 2020) +- [Nick Zitzmann brought this change] + + md4: use init/update/final functions in Secure Transport - ... as that might cause an unexpected SMB connection to a given host - name. + We can use CC_MD4_Init/Update/Final without having to allocate memory + directly. - Reported-by: Fernando Muñoz - CVE-2019-15601 - Bug: https://curl.haxx.se/docs/CVE-2019-15601.html + Closes #4979 -Jay Satiro (6 Jan 2020) -- CURLOPT_READFUNCTION.3: fix fopen params in example +Marc Hoersken (2 Mar 2020) +- ci/tests: some MacOS builds randomly take longer than 20min -- CURLOPT_READFUNCTION.3: fix variable name in example +Daniel Stenberg (2 Mar 2020) +- multi_wait: stop loop when sread() returns zero - Reported-by: Paul Joyce + It's unclear why it would ever return zero here, but this change fixes + Robert's problem and it shouldn't loop forever... - Fixes https://github.com/curl/curl/issues/4787 + Reported-by: Robert Dunaj + Bug: https://curl.haxx.se/mail/archive-2020-02/0011.html + Closes #5019 -Daniel Stenberg (5 Jan 2020) -- curl:getparameter return error for --http3 if libcurl doesn't support +- http: mark POSTs with no body as "upload done" from the start - Closes #4785 + As we have logic that checks if we get a >= 400 reponse code back before + the upload is done, which then got confused since it wasn't "done" but + yet there was no data to send! + + Reported-by: IvanoG on github + Fixes #4996 + Closes #5002 -- docs: mention CURL_MAX_INPUT_LENGTH restrictions +- tests: disable 962, 963 and 964 on Windows - ... for curl_easy_setopt() and curl_url_set(). + These tests are also doing UTF-8 SMTP. - [skip ci] + Follow-up to df207d2dd93b9e73 + +Marc Hoersken (2 Mar 2020) +- ci/tests: fine-tune Azure Pipeline timeouts with a small puffer + +Daniel Stenberg (2 Mar 2020) +- configure: bump the AC_COPYRIGHT year range + +- [Steve Holme brought this change] + + tests: disable SMTP UTF-8 tests on Windows - Closes #4783 + Fixes #4988 + Closes #4992 -- curl: properly free mimepost data +- formdata/mime: copyright year range update - ... as it could otherwise leak memory when a transfer failed. + Due to the merge/revert cycle + +- Revert "mime: latch last read callback status." - Added test 1293 to verify. + This reverts commit 87869e38d7afdec3ef1bb4965711458b088e254f. - Reported-by: Brian Carpenter - Fixes #4781 - Closes #4782 + Fixes #5014 + Closes #5015 + Reopens #4833 -- curl: cleanup multi handle on failure - - ... to fix memory leak in error path. +- Revert "mime: do not perform more than one read in a row" - Fixes #4772 - Closes #4780 - Reported-by: Brian Carpenter + This reverts commit ed0f357f7d25566110d4302f33759f4ffb5a6f83. -Marcel Raad (3 Jan 2020) -- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` +- Revert "mime: fix the binary encoder to handle large data properly" - Closes https://github.com/curl/curl/pull/4775 + This reverts commit b2caaa0681f329eed317ffb6ae6927f4a539f0c1. -Daniel Stenberg (3 Jan 2020) -- COPYING: it's 2020! +- altsvc: both h3 backends now speak h3-27 - [skip ci] + ... also updated the HTTP3 build description for ngtcp2 accordingly. -Jay Satiro (3 Jan 2020) -- [Marc Aldorasi brought this change] +- [Patrick Monnerat brought this change] - tests: Fix bounce requests with truncated writes + mime: fix the binary encoder to handle large data properly - Prior to this change the swsbounce check in service_connection could - fail because prevtestno and prevpartno were not set, which would cause - the wrong response data to be sent to some tests and cause them to fail. + New test 666 checks this is effective. + As upload buffer size is significant in this kind of tests, shorten it + in similar test 652. - Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 + Fixes #4860 + Reported-by: RuurdBeerstra on github -Marcel Raad (31 Dec 2019) -- tool: make a few char pointers point to const char instead +- [Patrick Monnerat brought this change] + + mime: do not perform more than one read in a row - These are read-only. - - Closes https://github.com/curl/curl/pull/4771 - -Jay Satiro (31 Dec 2019) -- tests: Change NTLM tests to require SSL - - Prior to this change tests that required NTLM feature did not require - SSL feature. - - There are pending changes to cmake builds that will allow enabling NTLM - in non-SSL builds in Windows. In that case the NTLM auth strings created - are different from what is expected by the NTLM tests and they fail: - - "The issue with NTLM is that previous non-SSL builds would not enable - NTLM and so the NTLM tests would be skipped." - - Assisted-by: marc-groundctl@users.noreply.github.com + Input buffer filling may delay the data sending if data reads are slow. + To overcome this problem, file and callback data reads do not accumulate + in buffer anymore. All other data (memory data and mime framing) are + considered as fast and still concatenated in buffer. + As this may highly impact performance in terms of data overhead, an early + end of part data check is added to spare a read call. + When encoding a part's data, an encoder may require more bytes than made + available by a single read. In this case, the above rule does not apply + and reads are performed until the encoder is able to deliver some data. - Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 + Tests 643, 644, 645, 650 and 654 have been adapted to the output data + changes, with test data size reduced to avoid the boredom of long lists of + 1-byte chunks in verification data. + New test 664 checks mimepost using single-byte read callback with encoder. + New test 665 checks the end of part data early detection. - Closes https://github.com/curl/curl/pull/4768 + Fixes #4826 + Reported-by: MrdUkk on github -- [Michael Forney brought this change] +- [Patrick Monnerat brought this change] - bearssl: Improve I/O handling + mime: latch last read callback status. - Factor out common I/O loop as bearssl_run_until, which reads/writes TLS - records until the desired engine state is reached. This is now used for - the handshake, read, write, and close. + In case a read callback returns a status (pause, abort, eof, + error) instead of a byte count, drain the bytes read so far but + remember this status for further processing. + Takes care of not losing data when pausing, and properly resume a + paused mime structure when requested. + New tests 670-673 check unpausing cases, with easy or multi + interface and mime or form api. - Match OpenSSL SSL_write behavior, and don't return the number of bytes - written until the corresponding records have been completely flushed - across the socket. This involves keeping track of the length of data - buffered into the TLS engine, and assumes that when CURLE_AGAIN is - returned, the write function will be called again with the same data - and length arguments. This is the same requirement of SSL_write. + Fixes #4813 + Reported-by: MrdUkk on github + Closes #4833 + +Steve Holme (1 Mar 2020) +- unit1651: Fixed conversion compilation warning - Handle TLS close notify as EOF when reading by returning 0. + 371:17: warning: conversion to 'unsigned char' from 'int' may alter its + value [-Wconversion] - Closes https://github.com/curl/curl/pull/4748 + Closes #5008 -- travis: Fix error detection +- configure.ac: Disable metalink support if an incompatible SSL/TLS specified - - Stop using inline shell scripts for before_script and script sections. + tool_metalink only supports cryptography from OpenSSL, GnuTLS, NSS, + The Win32 Crypto library and Apple's Common Crypto library. - Prior to this change Travis could ignore errors from commands in inline - scripts. I don't understand how or why it happens. This is a workaround. + If an TLS backend such as mbedTLS or WolfSSL is specified then the + following error is given during compilation along, with a load of + unresolved extern errors: - Assisted-by: Simon Warta + Can't compile METALINK support without a crypto library. - Ref: https://github.com/travis-ci/travis-ci/issues/1066 + Reviewed-by: Daniel Stenberg + Closes #5006 + +Marc Hoersken (1 Mar 2020) +- ci/tests: Update Azure DevOps pipeline job display names - Fixes https://github.com/curl/curl/issues/3730 - Closes https://github.com/curl/curl/pull/3755 + Make the configure step more descriptive and align others. -- tool_operate: fix mem leak when failed config parse +- ci/tests: Fix typo in previous commit 597cf2 + +- ci/tests: Make sure that the AZURE_ACCESS_TOKEN is available - Found by fuzzing the config file. + For security reasons the access token is not available to PR builds. + Therefore we should not try to use the DevOps API with an empty token. + +Daniel Stenberg (1 Mar 2020) +- build: remove all HAVE_OPENSSL_ENGINE_H defines - Reported-by: Geeknik Labs + ... as there's nothing in the code that actually uses the define! The + last reference was removed in 38203f158. - Fixes https://github.com/curl/curl/issues/4767 + Closes #5007 -- [Xiang Xiao brought this change] +Jay Satiro (29 Feb 2020) +- [Rolf Eike Beer brought this change] - lib: remove erroneous +x file permission on some c files + CMake: clean up and improve build procedures - Modified by commit eb9a604 accidentally. + - remove check for unsupported old CMake versions - Closes https://github.com/curl/curl/pull/4756 - -- [Xiang Xiao brought this change] - - lib: fix warnings found when porting to NuttX + - do not link to c-ares library twice - - Undefine DEBUGASSERT in curl_setup_once.h in case it was already - defined as a system macro. + - modernize custom Find modules - - Don't compile write32_le in curl_endian unless - CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. + - FindLibSSH2: + - pass version to FPHSA to show it in the output + - use LIBSSH2_VERSION define to extract the version number in + one shot. This variable exists in the header for 10 years. + - remove unneeded code - - Include in socketpair.c. + - FindNGHTTP2.cmake: + - drop needless FPHSA argument + - mark found variables as advanced - Closes https://github.com/curl/curl/pull/4756 - -- os400: Add missing CURLE error constants + - FindNSS.cmake: + - show version number - Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 - Reported-by: Emil Engler - -- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 + - FindCARES.cmake: + - drop default paths + - use FPHSA instead of checking things by hand - For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback - is passed two `size_t` parameters which, when multiplied, designate the - number of bytes of data passed in. In practice, CURL always sets the - first parameter (`size`) to 1. + - remove needless explict variable dereference - This practice is also enshrined in documentation and cannot be changed - in future. The documentation states that the default callback is - `fwrite`, which means `fwrite` must be a suitable function for this - purpose. However, the documentation also states that the callback must - return the number of *bytes* it successfully handled, whereas ISO C - `fwrite` returns the number of items (each of size `size`) which it - wrote. The only way these numbers can be equal is if `size` is 1. + - simplify count_true() - Since `size` is 1 and can never be changed in future anyway, document - that fact explicitly and let users rely on it. + - allow all policies up to version 3.16 to be set to NEW - Reported-by: Frank Gevaerts - Commit-message-by: Christopher Head + - do not rerun check for -Wstrict-aliasing=3 every time - Ref: https://github.com/curl/curl/pull/2787 + In contrast to every other compiler flag this has a = in it, which CMake + can't have in a variable name. - Fixes https://github.com/curl/curl/issues/4758 + - only read the interesting strings from curlver.h + + Reviewed-by: Peter Wu + + Closes https://github.com/curl/curl/pull/4975 -- examples/postinmemory.c: Call curl_global_cleanup always +- runtests: fix output to command log - Prior to this change curl_global_cleanup was not called if - curl_easy_init failed. + - Record only the command of the most recently ran test in the command + log. - Reported-by: kouzhudong@users.noreply.github.com + This is a follow-up to 02988b7 from several weeks ago which fixed + writing to the command log, however it saved all commands for all tests + instead of just the most recently ran test as we would now expect. - Fixes https://github.com/curl/curl/issues/4751 + Fixes https://github.com/curl/curl/commit/02988b7#commitcomment-37546876 + Closes https://github.com/curl/curl/pull/5001 -Daniel Stenberg (21 Dec 2019) -- url2file.c: fix copyright year +Steve Holme (1 Mar 2020) +- polarssl: Additional removal - Follow-up to 525787269599b5 + Follow up to 6357a19f. + + Reviewed-by: Daniel Stenberg + Closes #5004 -- [Rickard Hallerbäck brought this change] +- [Jonathan Cardoso Machado brought this change] - examples/url2file.c: corrected a comment - - The comment was confusing and suggested that setting CURLOPT_NOPROGRESS - to 0L would both enable and disable debug output at the same time, like - a Schrödinger's cat of CURLOPTs. + docs: fix typo on CURLINFO_RETRY_AFTER - alwaus -> always - Closes #4745 + Reviewed-by: Steve Holme + Closes #5005 -- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 +- md5: Added implementation for mbedTLS + + Reviewed-by: Jay Satiro + Closes #4980 -- RELEASE-NOTES: synced +- md5: Use pointer notation for array parameters in GnuTLS implementation -Jay Satiro (20 Dec 2019) -- ngtcp2: Support the latest update key callback type +- md4: Use non-deprecated functions in mbedTLS >= 2.7.0 - - Remove our cb_update_key in favor of ngtcp2's new - ngtcp2_crypto_update_key_cb which does the same thing. + Closes #4983 + +Marc Hoersken (29 Feb 2020) +- ci/tests: Send test results to Azure DevOps for reporting + +Daniel Stenberg (29 Feb 2020) +- pause: force-drain the transfer on unpause - Several days ago the ngtcp2_update_key callback function prototype was - changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to - fix up our cb_update_key for that change they also added - ngtcp2_crypto_update_key_cb which does the same thing so we'll use that - instead. + ... since the socket might not actually be readable anymore when for + example the data is already buffered in the TLS layer. - Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c + Fixes #4966 + Reported-by: Anders Berg + Closes #5000 + +- TODO: curl --proxycommand - Closes https://github.com/curl/curl/pull/4735 + Suggested-by: Kristian Mide + Closes #4941 -Daniel Stenberg (19 Dec 2019) -- sws: search for "Testno:" header uncondtionally if no testno +- smtp: overwriting 'from' leaks memory - Even if the initial request line wasn't found. With the fix to 1455, the - test number is now detected correctly. + Detected by Coverity. CID 1418139. - (Problem found when running tests in random order.) + Also, make sure to return error if the new 'from' allocation fails. - Closes #4744 + Closes #4997 -- tests: set LC_ALL in more tests +- CIfuzz: switch off 'dry_run' mode - Follow-up to 23208e330ac0c21 + Follow-up from #4960: now make it fail if it detects problems. - Closes #4743 + Closes #4998 -- test165: set LC_ALL=en_US.UTF-8 too +Marc Hoersken (28 Feb 2020) +- ci/tests: Increase timeouts of Windows builds due to new tests - On my current Debian Unstable with libidn2 2.2.0, I get an error if - LC_ALL is set to blank. Then curl errors out with: - - curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 - - Closes #4738 + Recently added tests increased their runtime above the limit of 60min. -- curl.h: add two defines for the "pre ISO C" case - - Without this fix, this caused a compilation failure on AIX with IBM xlc - 13.1.3 compiler. - - Reported-by: Ram Krushna Mishra - Fixes #4739 - Closes #4740 +- ci/tests: align Azure Pipeline job names with each other -- create_conn: prefer multiplexing to using new connections +- ci/tests: Add Windows builds via Azure Pipelines using Docker + +- tests: fix Python 3 compatibility of smbserver.py + +Daniel Stenberg (27 Feb 2020) +- runtests: restore the command log - ... as it would previously prefer new connections rather than - multiplexing in most conditions! The (now removed) code was a leftover - from the Pipelining code that was translated wrongly into a - multiplex-only world. + The log file with all command lines for the invoked command lines is now + called logs/commands.log - Reported-by: Kunal Ekawde - Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html - Closes #4732 + Fixes #4911 + Closes #4989 -- test1456: remove the use of a fixed local port - - Fixup the test to instead not compare the port number. It sometimes - caused problems like this: +- smtp: fix memory leak on exit path - "curl: (45) bind failed with errno 98: Address already in use" + Detected by Coverity. CID 1418139. "leaked_storage: Variable 'from' + going out of scope leaks the storage it points to" - Closes #4733 + Closes #4990 -Jay Satiro (18 Dec 2019) -- CURLOPT_QUOTE.3: fix typos - - Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man - pages would not compile because a variable name was incorrect. +Steve Holme (27 Feb 2020) +- gtls: Fixed compilation when using GnuTLS < 3.5.0 - Reported-by: Bylon2@users.noreply.github.com + Reverts the functionality from 41fcb4f when compiling with GnuTLS older + than 3.5.0. - Fixes https://github.com/curl/curl/issues/4736 + Reviewed-by: Daniel Stenberg + Closes #4984 -- [Gisle Vanem brought this change] +- RELEASE-NOTES: Corrected the link to issue #4892 - strerror: Fix compiler warning "empty expression" +Daniel Stenberg (27 Feb 2020) +- Curl_is_ASCII_name: handle a NULL argument - - Remove the final semi-colon in the SEC2TXT() macro definition. + Make the function tolerate a NULL pointer input to avoid dereferencing + that pointer. - Before: #define SEC2TXT(sec) case sec: txt = #sec; break; + Follow-up to efce3ea5a85126d + Detected by OSS-Fuzz + Reviewed-By: Steve Holme + Bug: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=20907 + Fixes #4985 + Closes #4986 + +- RELEASE-NOTES: synced + +- http2: make pausing/unpausing set/clear local stream window - After: #define SEC2TXT(sec) case sec: txt = #sec; break + This reduces the HTTP/2 window size to 32 MB since libcurl might have to + buffer up to this amount of data in memory and yet we don't want it set + lower to potentially impact tranfer performance on high speed networks. - Prior to this change SEC2TXT(foo); would generate break;; which caused - the empty expression warning. + Requires nghttp2 commit b3f85e2daa629 + (https://github.com/nghttp2/nghttp2/pull/1444) to work properly, to end + up in the next release after 1.40.0. - Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 + Fixes #4939 + Closes #4940 -Daniel Stenberg (18 Dec 2019) -- curl/parseconfig: use curl_free() to free memory allocated by libcurl - - Reported-by: bxac on github - Fixes #4730 - Closes #4731 +- [Anderson Toshiyuki Sasaki brought this change] -- curl/parseconfig: fix mem-leak + libssh: improve known hosts handling - When looping, first trying '.curlrc' and then '_curlrc', the function - would not free the first string. + Previously, it was not possible to get a known hosts file entry due to + the lack of an API. ssh_session_get_known_hosts_entry(), introduced in + libssh-0.9.0, allows libcurl to obtain such information and behave the + same as when compiled with libssh2. - Closes #4731 - -- CURLOPT_URL.3: "curl supports SMB version 1 (only)" + This also tries to avoid the usage of deprecated functions when the + replacements are available. The behaviour will not change if versions + older than libssh-0.8.0 are used. - [skip ci] - -- test1270: a basic -w redirect_url test + Signed-off-by: Anderson Toshiyuki Sasaki - Closes #4728 - -- HISTORY: the SMB(S) support landed in 2014 + Fixes #4953 + Closes #4962 -- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore +Steve Holme (27 Feb 2020) +- tests: Automatically deduce the tool name from the test case for unit tests - It is covered by USE_OPENSSL_ENGINE now. + It is still possible to override the executable to run during the test, + using the tag, but this patch removes the requirement that the + tag must be present for unit tests. - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 + It also removes the possibility of human error when existing test cases + are used as the basis for new tests, as recently witnessed in 81c37124. - Closes #4725 + Reviewed-by: Daniel Stenberg + Closes #4976 -- lib: remove ASSIGNWITHINCONDITION exceptions, use our code style +- test1323: Added the missing 'unit test' feature requirement in the test case + +Daniel Stenberg (26 Feb 2020) +- cookie: remove unnecessary check for 'out != 0' - ... even for macros + ... as it will always be non-NULL at this point. - Reviewed-by: Daniel Gustafsson - Reviewed-by: Jay Satiro - Reported-by: Jay Satiro - Fixes #4683 - Closes #4722 - -- tests: make sure checksrc runs on header files too + Detected by Coverity: CID 1459009 -- Revert "checksrc: fix regexp for ASSIGNWITHINCONDITION" +- http: added 417 response treatment - This reverts commit ba82673dac3e8d00a76aa5e3779a0cb80e7442af. + When doing a request with a body + Expect: 100-continue and the server + responds with a 417, the same request will be retried immediately + without the Expect: header. - Bug: #4683 - -- KNOWN_BUGS: TLS session cache doesn't work with TFO + Added test 357 to verify. - [skip ci] - Closes #4301 - -- KNOWN_BUGS: Connection information when using TCP Fast Open + Also added a control instruction to tell the sws test server to not read + the request body if Expect: is present, which the new test 357 uses. - Also point to #4296 for more details - Closes #4296 + Reported-by: bramus on github + Fixes #4949 + Closes #4964 -- KNOWN_BUGS: LDAP on Windows doesn't work +Steve Holme (26 Feb 2020) +- smtp: Tidy up, following recent changes, to maintain the coding style - Closes #4261 + Closes #4892 -- docs: TLS SRP doesn't work with TLS 1.3 +- smtp: Support the SMTPUTF8 extension for the EXPN command - Reported-by: sayrer on github - Closes #4262 - [skip ci] + Simply notify the server we support the SMTPUTF8 extension if it does. -Dan Fandrich (16 Dec 2019) -- cirrus: Switch to the FreeBSD 12.1 point release & enable more tests. - - A few tests are now passing on FreeBSD, so no longer skip them. - [skip ci] +- smtp: Support the SMTPUTF8 extension in the VRFY command -Daniel Stenberg (16 Dec 2019) -- azure: the macos cmake doesn't need to install cmake - - Error: cmake 3.15.5 is already installed - To upgrade to 3.16.1, run `brew upgrade cmake`. +- smtp: Support the SMTPUTF8 extension in the RCPT TO command - Closes #4723 + Note: The RCPT TO command isn't required to advertise to the server that + it contains UTF-8 characters, instead the server is told that a mail may + contain UTF-8 in any envelope command via the MAIL command. -Jay Satiro (15 Dec 2019) -- winbuild: Document CURL_STATICLIB requirement for static libcurl - - A static libcurl (ie winbuild mode=static) requires that the user define - CURL_STATICLIB when using it in their application. This is already - covered in the FAQ and INSTALL.md, but is a pretty important point so - now it's noted in the BUILD.WINDOWS.txt as well. +- smtp: Support the SMTPUTF8 extension in the MAIL command - Assisted-by: Michael Vittiglio + Support the SMTPUTF8 extension when sending mailbox information in the + MAIL command (FROM and AUTH parameters). Non-ASCII domain names will + be ACE encoded, if IDN is supported, whilst non-ASCII characters in + the local address part are passed to the server. - Closes https://github.com/curl/curl/pull/4721 + Reported-by: ygthien on github + Fixes #4828 -Daniel Stenberg (15 Dec 2019) -- [Santino Keupp brought this change] +- smtp: Detect server support for the UTF-8 extension as defined in RFC-6531 - libssh2: add support for ECDSA and ed25519 knownhost keys - - ... if a new enough libssh2 version is present. - - Source: https://curl.haxx.se/mail/archive-2019-12/0023.html - Co-Authored-by: Daniel Stenberg - Closes #4714 +- smtp: Support UTF-8 based host names in the VRFY command -- lib1591: free memory properly on OOM, in the trailers callback - - Detected by torture tests. - - Closes #4720 +- smtp: Support UTF-8 based host names in the RCPT TO command -- runtests: --repeat=[num] to repeat tests +- smtp: Support UTF-8 based host names in the MAIL command - Closes #4715 + Non-ASCII host names will be ACE encoded if IDN is supported. -- RELEASE-NOTES: synced +- url: Make the IDN conversion functions available to others -- azure: add a torture test on mac - - Uses --shallow=25 to keep it small enough to get through in time. - - Closes #4712 +- smtp: Added UTF-8 mailbox tests to verify existing behaviour -- multi: free sockhash on OOM +- ftpserver: Updated VRFY_smtp() so the response isn't necessary in the test case + +- ftpserver: Corrected the e-mail address regex in MAIL_smtp() and RCTP_smtp() - This would otherwise leak memory in the error path. + The dot character between the host and the tld was not being escaped, + which meant it specified a match of 'any' character rather than an + explicit dot separator. - Detected by torture test 1540. + Additionally removed the dot character from the host name as it allowed + the following to be specified as a valid address in our test cases: - Closes #4713 - -Marcel Raad (13 Dec 2019) -- tests: use DoH feature for DoH tests + - Previously, http/2 was used instead. - - Assisted-by: Jay Satiro - Closes https://github.com/curl/curl/pull/4692 - -- hostip: suppress compiler warning - - With `--disable-doh --disable-threaded-resolver`, the `dns` parameter - is not used. + Both are typos from 98f7ca7 and 8880f84 :( - Closes https://github.com/curl/curl/pull/4692 - -- tests: fix build with `CURL_DISABLE_DOH` + I can't remember whether my intention was to allow sub-domains to be + specified in the host or not with these additional dots, but by placing + it outside of the host means it can only be specified once per domain + and by placing a + after the new grouping support for sub-domains is + kept. - Closes https://github.com/curl/curl/pull/4692 + Closes #4912 -Daniel Stenberg (13 Dec 2019) -- azure: add a torture test - - Skipping all FTP tests for speed reasons. +- hmac: Added a unit test for the HMAC hash generation - Closes #4697 - -- azure: make the default build use --enable-debug --enable-werror + Closes #4973 -- ntlm_wb: fix double-free in OOM - - Detected by torture testing test 1310 - - Closes #4710 +- ntlm: Moved the HMAC MD5 function into the HMAC module as a generic function -Dan Fandrich (13 Dec 2019) -- cirrus: Drop the FreeBSD 10.4 build +- tests: Added a unit test for MD4 digest generation - Upstream support for 10.4 ended a year ago, and it looks like the image - is now gone, too. - [skip ci] + Closes #4970 -Daniel Stenberg (13 Dec 2019) -- unit1620: fix bad free in OOM +- md4: Use const for the length input parameter - Closes #4709 + This keeps the interface the same as md5 and sha256. -- unit1609: fix mem-leak in OOM +- test1610: Fixed the link to the unit test - Closes #4709 + Typo from 81c37124. -- unit1607: fix mem-leak in OOM +- ntlm: Removed the dependency on the TLS libaries when using MD5 - Closes #4709 - -- lib1559: fix mem-leak in OOM + As we have our own MD5 implementation use the MD5 wrapper to remove the + TLS dependency. - Closes #4709 + Closes #4967 -- lib1557: fix mem-leak in OOM - - Closes #4709 +- md5/sha256: Updated the functions to allow non-string data to be hashed -- altsvc: make the save function ignore NULL filenames - - It might happen in OOM situations. Detected bv torture tests. +- digest: Corrected the name of the local HTTP digest function - Closes #4707 + Follow up to 2b5b37cb. Local static functions do not require the Curl + prefix. -- curl: fix memory leak in OOM in etags logic +- tests: Added a unit test for SHA256 digest generation - Detected by torture tests + Follow up to 2b5b37c. - Closes #4706 + Closes #4968 -- doh: make it behave when built without proxy support - - Reported-by: Marcel Raad - Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734 +- md4: Fixed compilation issues when using GNU TLS gcrypt - Closes #4704 - -- curl: improved cleanup in upload error path + * Don't include 'struct' in the gcrypt MD4_CTX typedef + * The call to gcry_md_read() should use a dereferenced ctx + * The call to gcry_md_close() should use a dereferenced ctx - Memory leak found by torture test 58 + Additional minor whitespace issue in the USE_WIN32_CRYPTO code. - Closes #4705 - -- mailmap: fix Andrew Ishchuk + Closes #4959 -- travis: make torture use --shallow=40 - - As a first step to enable it to run over a more diverse set of tests in - a reasonable time. +Daniel Stenberg (21 Feb 2020) +- RELEASE-NOTES: synced -- runtests: introduce --shallow to reduce huge torture tests +- http2: now require nghttp2 >= 1.12.0 - When set, shallow mode limits runtests -t to make no more than NUM fails - per test case. If more are found, it will randomly discard entries until - the number is right. The random seed can also be set. + To simplify our code and since earlier versions lack important function + calls libcurl needs to function correctly. - This is particularly useful when running MANY tests as then most torture - failures will already fail the same functions over and over and make the - total operation painfully tedious. + nghttp2 1.12.0 was relased on June 26, 2016. - Closes #4699 + Closes #4961 -- conncache: CONNECT_ONLY connections assumed always in-use - - This makes them never to be considered "the oldest" to be discarded when - reaching the connection cache limit. The reasoning here is that - CONNECT_ONLY is primarily used in combination with using the - connection's socket post connect and since that is used outside of - curl's knowledge we must assume that it is in use until explicitly - closed. +- gtls: fix the copyright year - Reported-by: Pavel Pavlov - Reported-by: Pavel Löbl - Fixes #4426 - Fixes #4369 - Closes #4696 + Follow-up from 41fcb4f609 -- [Gisle Vanem brought this change] +- [jethrogb brought this change] - vtls: make BearSSL possible to set with CURL_SSL_BACKEND + GnuTLS: Always send client cert - Ref: https://github.com/curl/curl/commit/9b879160df01e7ddbb4770904391d3b74114302b#commitcomment-36355622 + TLS servers may request a certificate from the client. This request + includes a list of 0 or more acceptable issuer DNs. The client may use + this list to determine which certificate to send. GnuTLS's default + behavior is to not send a client certificate if there is no + match. However, OpenSSL's default behavior is to send the configured + certificate. The `GNUTLS_FORCE_CLIENT_CERT` flag mimics OpenSSL + behavior. - Closes #4698 + Authored-by: jethrogb on github + Fixes #1411 + Closes #4958 -- RELEASE-NOTES: synced +- [Leo Neat brought this change] -- travis: remove "coverage", make it "torture" - - The coveralls service and test coverage numbers are just too unreliable. - Removed badge from README.md as well. + github action: add CIFuzz - Fixes #4694 - Closes #4695 + Closes #4960 -- azure: add libssh2 and cmake macos builds +- cleanup: comment typos - Removed the macos libssh2 build from travis + Spotted by 'codespell' - Closes #4686 + Closes #4957 -- curl: use errorf() better - - Change series of error outputs to use errorf(). +Steve Holme (20 Feb 2020) +- win32: USE_WIN32_CRYPTO to enable Win32 based MD4, MD5 and SHA256 functions - Only errors that are due to mistakes in command line option usage should - use helpf(), other types of errors in the tool should rather use - errorf(). + Whilst lib\md4.c used this pre-processor, lib\md5.c and + src\tool_metalink.c did not and simply relied on the WIN32 + pre-processor directive. - Closes #4691 - -Jay Satiro (9 Dec 2019) -- [Marc Hoersken brought this change] + Reviewed-by: Marcel Raad + Closes #4955 - tests: make it possible to set executable extensions +Daniel Stenberg (19 Feb 2020) +- connect: remove some spurious infof() calls - This enables the use of Windows Subsystem for Linux (WSL) to run the - testsuite against Windows binaries while using Linux servers. + As they were added primarily for debugging, they provide little use for + users. - This commit introduces the following environment variables: - - CURL_TEST_EXE_EXT: set the executable extension for all components - - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only - - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only + Closes #4951 + +- HTTP-COOKIES: mention that a trailing newline is required - Later testcurl.pl could be adjusted to make use of those variables. - - CURL_TEST_EXE_EXT_SRV: set it for the test servers only + ... so that we know we got the whole and not a partial line. - (This is one of several commits to support use of WSL for the tests.) + Also, changed the formatting of the fields away from a table again since + the table format requires a github-markdown tool version that we don't + run on the web server atm. - Closes https://github.com/curl/curl/pull/3899 + Reported-by: Sunny Bean + Fixes #4946 + Closes #4947 -- [Marc Hoersken brought this change] +- nit: Copyright year out of date + + Follow-up to 1fc0617dcc - tests: fix permissions of ssh keys in WSL +Jay Satiro (18 Feb 2020) +- tool_util: Improve Windows version of tvnow() - Keys created on Windows Subsystem for Linux (WSL) require it for some - reason. + - Change tool_util.c tvnow() for Windows to match more closely to + timeval.c Curl_now(). - (This is one of several commits to support use of WSL for the tests.) + - Create a win32 init function for the tool, since some initialization + is required for the tvnow() changes. - Ref: https://github.com/curl/curl/pull/3899 - -- [Marc Hoersken brought this change] - - tests: use \r\n for log messages in WSL + Prior to this change the monotonic time function used by curl in Windows + was determined at build-time and not runtime. That was a problem because + when curl was built targeted for compatibility with old versions of + Windows (eg _WIN32_WINNT < 0x0600) it would use GetTickCount which wraps + every 49.7 days that Windows has been running. - Bash in Windows Subsystem for Linux (WSL) requires it for some reason. + This change makes curl behave similar to libcurl's tvnow function, which + determines at runtime whether the OS is Vista+ and if so calls + QueryPerformanceCounter instead. (Note QueryPerformanceCounter is used + because it has higher resolution than the more obvious candidate + GetTickCount64). The changes to tvnow are basically a copy and paste but + the types in some cases are different. - (This is one of several commits to support use of WSL for the tests.) + Ref: https://github.com/curl/curl/issues/3309 - Ref: https://github.com/curl/curl/pull/3899 - -- [Andrew Ishchuk brought this change] + Closes https://github.com/curl/curl/pull/4847 - winbuild: Define CARES_STATICLIB when WITH_CARES=static +Daniel Stenberg (18 Feb 2020) +- SOCKS: fix typo in printf formatting - When libcurl is built with MODE=static, c-ares is forced into static - linkage too. That doesn't happen when MODE=dll so linker would break - over undefined symbols. + Follow-up to 4a4b63daa - closes https://github.com/curl/curl/pull/4688 + Reported-by: Peter Piekarski + Bug: https://github.com/curl/curl/commit/4a4b63daaa01ef59b131d91e8e6e6dfe275c0f08#r37351330 -Daniel Stenberg (9 Dec 2019) -- conn: always set bits.close with connclose() +- CURLOPT_REDIR_PROTOCOLS.3: update the DEFAULT section - Closes #4690 - -- cirrus: enable clang sanitizers on freebsd 13 - -- conncache: fix multi-thread use of shared connection cache - - It could accidentally let the connection get used by more than one - thread, leading to double-free and more. + to be in sync with the description above - Reported-by: Christopher Reid - Fixes #4544 - Closes #4557 + Reported-by: Joonas Kuorilehto + Fixes #4943 + Closes #4945 -- azure: add a vanilla macos build - - Closes #4685 +- docs/GOVERNANCE: refreshed + added "donations" and "commercial support" -- curl: make the etag load logic work without fseek +- altsvc: make saving the cache an atomic operation - The fseek()s were unnecessary and caused Coverity warning CID 1456554 + ... by writing the file to temp name then rename to the final when done. - Closes #4681 - -- mailmap: Mohammad Hasbini + Assisted-by: Jay Satiro + Fixes #4936 + Closes #4942 -- [Mohammad Hasbini brought this change] +- rename: a new file for Curl_rename() + + And make the cookie save function use it. - docs: fix some typos +- cookies: make saving atomic with a rename - Closes #4680 + Saves the file as "[filename].[8 random hex digits].tmp" and renames + away the extension when done. + + Co-authored-by: Jay Satiro + Reported-by: Mike Frysinger + Fixes #4914 + Closes #4926 - RELEASE-NOTES: synced -Jay Satiro (5 Dec 2019) -- lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN - - Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS - and OS400 package spec. - - Also I added the option to the NameValue list in the tool even though it - isn't exposed as a command-line option (...yet?). (NameValue stringizes - the option name for the curl cmd -> libcurl source generator) +- socks: make the connect phase non-blocking - Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN. + Removes two entries from KNOWN_BUGS. - Ref: https://github.com/curl/curl/pull/4655 + Closes #4907 -- setopt: Fix ALPN / NPN user option when built without HTTP2 +- multi: if Curl_readwrite sets 'comeback' use expire, not loop - - Stop treating lack of HTTP2 as an unknown option error result for - CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN. + Otherwise, a very fast single transfer ricks starving out other + concurrent transfers. - Prior to this change it was impossible to disable ALPN / NPN if libcurl - was built without HTTP2. Setting either option would result in - CURLE_UNKNOWN_OPTION and the respective internal option would not be - set. That was incorrect since ALPN and NPN are used independent of - HTTP2. + Closes #4927 + +- ftp: convert 'sock_accepted' to a plain boolean - Reported-by: Shailesh Kapse + This was an array indexed with sockindex but it was only ever used for + the secondary socket. - Fixes https://github.com/curl/curl/issues/4668 - Closes https://github.com/curl/curl/pull/4672 + Closes #4929 -Daniel Stenberg (5 Dec 2019) -- etag: allow both --etag-compare and --etag-save in same cmdline +Jay Satiro (15 Feb 2020) +- CURLINFO_COOKIELIST.3: Fix example - Fixes #4669 - Closes #4678 - -Marcel Raad (5 Dec 2019) -- curl_setup: fix `CURLRES_IPV6` condition + Prior to this change the example would try to import cookies from stdin, + which wasn't what was intended. - Move the definition of `CURLRES_IPV6` to before undefining - `HAVE_GETADDRINFO`. Regression from commit 67a08dca27a which caused - some tests to fail and others to be skipped with c-ares. + Reported-by: 3dyd@users.noreply.github.com - Fixes https://github.com/curl/curl/issues/4673 - Closes https://github.com/curl/curl/pull/4677 - -Daniel Stenberg (5 Dec 2019) -- test342: make it return a 304 as the tag matches + Fixes https://github.com/curl/curl/issues/4930 -Peter Wu (4 Dec 2019) -- CMake: add support for building with the NSS vtls backend +Daniel Stenberg (14 Feb 2020) +- TODO: Paged searches on LDAP server - Options are cross-checked with configure.ac and acinclude.m4. - Tested on Arch Linux, untested on other platforms like Windows or macOS. + Closes #4452 + +- TODO: CURLOPT_SSL_CTX_FUNCTION for LDAPS - Closes #4663 - Reviewed-by: Kamil Dudka + Closes #4108 -Daniel Stenberg (4 Dec 2019) -- azure: add more builds +- azure: disable brotli on the macos debug-builds - ... removed two from travis (that now runs on azure instead) + Because of: - Closes #4671 - -- CURLOPT_VERBOSE.3: see also ERRORBUFFER + brotli/decode.h:204:33: error: variable length array used [-Werror,-Wvla] + const uint8_t encoded_buffer[BROTLI_ARRAY_PARAM(encoded_size)], + + Closes #4925 -- hostip4.c: bump copyright year range +Steve Holme (13 Feb 2020) +- tool_home: Fix the copyright year being out of date + + Follow up to 9dc350b6. -Marcel Raad (3 Dec 2019) -- configure: enable IPv6 support without `getaddrinfo` +Jay Satiro (12 Feb 2020) +- tool_homedir: Change GetEnv() to use libcurl's curl_getenv() - This makes it possible to recognize and connect to literal IPv6 - addresses when `getaddrinfo` is not available, which is already the - case for the CMake build. This affects e.g. classic MinGW because it - still targets Windows 2000 by default, where `getaddrinfo` is not - available, but general IPv6 support is. + - Deduplicate GetEnv() code. - Instead of checking for `getaddrinfo`, check for `sockaddr_in6` as the - CMake build does. + - On Windows change ultimate call to use Windows API + GetEnvironmentVariable() instead of C runtime getenv(). - Closes https://github.com/curl/curl/pull/4662 - -- curl_setup: disable IPv6 resolver without `getaddrinfo` + Prior to this change both libcurl and the tool had their own GetEnv + which over time diverged. Now the tool's GetEnv is a wrapper around + curl_getenv (libcurl API function which is itself a wrapper around + libcurl's GetEnv). - Also, use `CURLRES_IPV6` only for actual DNS resolution, not for IPv6 - address support. This makes it possible to connect to IPv6 literals by - setting `ENABLE_IPV6` even without `getaddrinfo` support. It also fixes - the CMake build when using the synchronous resolver without - `getaddrinfo` support. + Furthermore this change fixes a bug in that Windows API + GetEnvironmentVariable() is called instead of C runtime getenv() to get + the environment variable since some changes aren't always visible to the + latter. - Closes https://github.com/curl/curl/pull/4662 - -Daniel Stenberg (3 Dec 2019) -- github action/azure pipeline: run 'make test-nonflaky' for tests + Reported-by: Christoph M. Becker - To match travis and give more info on failures. + Fixes https://github.com/curl/curl/issues/4774 + Closes https://github.com/curl/curl/pull/4863 -- openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains +Daniel Stenberg (12 Feb 2020) +- strerror.h: Copyright year out of date - Closes #4655 + Follow-up to 1c4fa67e8a8fcf6 -- openssl: set X509_V_FLAG_PARTIAL_CHAIN +Jay Satiro (12 Feb 2020) +- strerror: Increase STRERROR_LEN 128 -> 256 - Have intermediate certificates in the trust store be treated as - trust-anchors, in the same way as self-signed root CA certificates - are. This allows users to verify servers using the intermediate cert - only, instead of needing the whole chain. + STRERROR_LEN is the constant used throughout the library to set the size + of the buffer on the stack that the curl strerror functions write to. - Other TLS backends already accept partial chains. + Prior to this change some extended length Windows error messages could + be truncated. - Reported-by: Jeffrey Walton - Bug: https://curl.haxx.se/mail/lib-2019-11/0094.html + Closes https://github.com/curl/curl/pull/4920 -- curl: show better error message when no homedir is found +- multi: fix outdated comment - Reported-by: Vlastimil Ovčáčík - Fixes #4644 - Closes #4665 - -- OPENSOCKETFUNCTION.3: correct the purpose description + - Do not say that conn->data is "cleared" by multi_done(). - Reported-by: Jeff Mears - Bug: https://curl.haxx.se/mail/lib-2019-12/0007.html + If the connection is in use then multi_done assigns another easy handle + still using the connection to conn->data, therefore in that case it is + not cleared. - Closes #4667 - -- [Peter Wu brought this change] + Closes https://github.com/curl/curl/pull/4901 - travis: do not use OVERRIDE_CC or OVERRIDE_CXX if empty +- easy: remove dead code - Fixes the macOS builds where OVERRIDE_CC and OVERRIDE_CXX are not set. + multi is already assigned to data->multi by curl_multi_add_handle. - Reported-by: Jay Satiro - Fixes #4659 - Closes #4661 - Closes #4664 + Closes https://github.com/curl/curl/pull/4900 -- azure-pipelines: fix the test script +Daniel Stenberg (12 Feb 2020) +- create-dirs.d: mention the mode + + Reported-by: Dan Jacobson + Fixes #4766 + Closes #4916 -- Azure Pipelines: initial CI setup +- CURLOPT_ALTSVC_CTRL.3: fix the DEFAULT wording - [skip ci] + Assisted-by: Jay Satiro + Reported-by: Craig Andrews + Fixes #4909 + Closes #4910 -- docs: add "added: 7.68.0" to the --etag-* docs +- RELEASE-NOTES: synced -- copyright: fix the year ranges for two files +Steve Holme (9 Feb 2020) +- smtp: Simplify the MAIL command and avoid a duplication of send strings - Follow-up to 9c1806ae + This avoids the duplication of strings when the optional AUTH and SIZE + parameters are required. It also assists with the modifications that + are part of #4892. + + Closes #4903 -Jay Satiro (1 Dec 2019) -- build: Disable Visual Studio warning "conditional expression is constant" +Daniel Stenberg (9 Feb 2020) +- altsvc: keep a copy of the file name to survive handle reset - - Disable warning C4127 "conditional expression is constant" globally - in curl_setup.h for when building with Microsoft's compiler. + The alt-svc cache survives a call to curl_easy_reset fine, but the file + name to use for saving the cache was cleared. Now the alt-svc cache has + a copy of the file name to survive handle resets. - This mainly affects building with the Visual Studio project files found - in the projects dir. + Added test 1908 to verify. - Prior to this change the cmake and winbuild build systems already - disabled 4127 globally for when building with Microsoft's compiler. - Also, 4127 was already disabled for all build systems in the limited - circumstance of the WHILE_FALSE macro which disabled the warning - specifically for while(0). This commit removes the WHILE_FALSE macro and - all other cruft in favor of disabling globally in curl_setup. + Reported-by: Craig Andrews + Fixes #4898 + Closes #4902 + +Steve Holme (9 Feb 2020) +- url: Include the failure reason when curl_win32_idn_to_ascii() fails - Background: - - We have various macros that cause 0 or 1 to be evaluated, which would - cause warning C4127 in Visual Studio. For example this causes it: + Provide the failure reason in the failf() info just as we do for the + libidn2 version of code. - #define Curl_resolver_asynch() 1 + Closes #4899 + +Jay Satiro (9 Feb 2020) +- asyn-thread: remove dead code + +Daniel Stenberg (8 Feb 2020) +- [Emil Engler brought this change] + + github: Instructions to post "uname -a" on Unix systems in issues - Full behavior is not clearly defined and inconsistent across versions. - However it is documented that since VS 2015 Update 3 Microsoft has - addressed this somewhat but not entirely, not warning on while(true) for - example. + Closes #4896 + +- [Cristian Greco brought this change] + + configure.ac: fix comments about --with-quiche - Prior to this change some C4127 warnings occurred when I built with - Visual Studio using the generated projects in the projects dir. + A simple s/nghttp3/quiche in some comments of --with-quiche. + Looks like a copy-paste error from --with-nghttp3. - Closes https://github.com/curl/curl/pull/4658 + Closes #4897 -- openssl: retrieve reported LibreSSL version at runtime - - - Retrieve LibreSSL runtime version when supported (>= 2.7.1). +Steve Holme (7 Feb 2020) +- checksrc.bat: Fix not being able to run script from the main curl directory - For earlier versions we continue to use the compile-time version. + If the script was ran from the main curl directory rather then the + projects directory then the script would simply exit without error: - Ref: https://man.openbsd.org/OPENSSL_VERSION_NUMBER.3 + C:\url> projects\checksrc.bat - Closes https://github.com/curl/curl/pull/2425 - -- strerror: Add Curl_winapi_strerror for Win API specific errors + The user would either need to change to the projects directory, + explicitly specify the current working directory, or perform a + oneline hacky workaround: - - In all code call Curl_winapi_strerror instead of Curl_strerror when - the error code is known to be from Windows GetLastError. + C:\url> cd projects + C:\url\projects> checksrc.bat - Curl_strerror prefers CRT error codes (errno) over Windows API error - codes (GetLastError) when the two overlap. When we know the error code - is from GetLastError it is more accurate to prefer the Windows API error - messages. + C:\url> checksrc.bat %cd% - Reported-by: Richard Alcock + C:\url> pushd projects & checksrc.bat & popd - Fixes https://github.com/curl/curl/issues/4550 - Closes https://github.com/curl/curl/pull/4581 + Closes #4894 -Daniel Stenberg (2 Dec 2019) -- global_init: undo the "intialized" bump in case of failure - - ... so that failures in the global init function don't count as a - working init and it can then be called again. - - Reported-by: Paul Groke - Fixes #4636 - Closes #4653 +Daniel Stenberg (7 Feb 2020) +- [Pierre-Yves Bigourdan brought this change] -- parsedate: offer a getdate_capped() alternative + digest: Do not quote algorithm in HTTP authorisation - ... and use internally. This function will return TIME_T_MAX instead of - failure if the parsed data is found to be larger than what can be - represented. TIME_T_MAX being the largest value curl can represent. + RFC 7616 section 3.4 (The Authorization Header Field) states that "For + historical reasons, a sender MUST NOT generate the quoted string syntax + for the following parameters: algorithm, qop, and nc". This removes the + quoting for the algorithm parameter. - Reviewed-by: Daniel Gustafsson - Reported-by: JanB on github - Fixes #4152 - Closes #4651 + Reviewed-by: Steve Holme + Closes #4890 -- docs: add more references to curl_multi_poll +- ftp: remove the duplicated user/password struct fields - Fixes #4643 - Closes #4652 + Closes #4887 -- sha256: bump the copyright year range +- ftp: remove superfluous checking for crlf in user or pwd - Follow-up from 66e21520f + ... as this is already done much earlier in the URL parser. + + Also add test case 894 that verifies that pop3 with an encodedd CR in + the user name is rejected. + + Closes #4887 -Daniel Gustafsson (28 Nov 2019) -- curl_setup_once: consistently use WHILE_FALSE in macros +Steve Holme (6 Feb 2020) +- ntlm_wb: Use Curl_socketpair() for greater portability - The WHILE_FALSE construction is used to avoid compiler warnings in - macro constructions. This fixes a few instances where it was not - used in order to keep the code consistent. + Reported-by: Daniel Stenberg + Closes #4886 + +Daniel Stenberg (5 Feb 2020) +- [Frank Gevaerts brought this change] + + contributors: Also include people who contributed to curl-www - Closes #4649 - Reviewed-by: Daniel Stenberg + Closes #4884 -Daniel Stenberg (28 Nov 2019) -- [Steve Holme brought this change] +- [Frank Gevaerts brought this change] - http_ntlm: Remove duplicate NSS initialisation + contrithanks: Use the most recent tag by default - Given that this is performed by the NTLM code there is no need to - perform the initialisation in the HTTP layer. This also keeps the - initialisation the same as the SASL based protocols and also fixes a - possible compilation issue if both NSS and SSPI were to be used as - multiple SSL backends. + (similar to 5296abe) - Reviewed-by: Kamil Dudka - Closes #3935 + Closes #4883 -Daniel Gustafsson (28 Nov 2019) -- checksrc: fix regexp for ASSIGNWITHINCONDITION +- scripts: use last set tag if none given - The regexp looking for assignments within conditions was too greedy - and matched a too long string in the case of multiple conditionals - on the same line. This is basically only a problem in single line - macros, and the code which exemplified this was essentially: + Makes 'delta' and 'contributors.sh' easier to use. - do { if((x) != NULL) { x = NULL; } } while(0) + Make the delta script invoke contrithanks to get current number of + contributors instead of counting THANKS, for accuracy. - ..where the final parenthesis of while(0) matched the regexp, and - the legal assignment in the block triggered the warning. Fix by - making the regexp less greedy by matching for the tell-tale signs - of the if statement ending. + Closes #4881 + +- ftp: shrink temp buffers used for PORT - Also remove the one occurrence where the warning was disabled due - to a construction like the above, where the warning didn't apply - when fixed. + These two stack based buffers only need to be 46 + 66 bytes instead of + 256 + 1024. - Closes #4647 - Reviewed-by: Daniel Stenberg - -Daniel Stenberg (28 Nov 2019) -- RELEASE-NOTES: synced - -- [Maros Priputen brought this change] + Closes #4880 - curl: two new command line options for etags +- curl: error on --alt-svc use w/o support - --etag-compare and --etag-save + Make the tool check for alt-svc support at run-time and return error + accordingly if not present when the option is used. - Suggested-by: Paul Hoffman - Fixes #4277 - Closes #4543 - -Daniel Gustafsson (28 Nov 2019) -- docs: fix typos - -Daniel Stenberg (28 Nov 2019) -- mailmap: Niall O'Reilly's name + Reported-by: Harry Sintonen + Closes #4878 -- [Niall O'Reilly brought this change] +- docs/HTTP3: add --enable-alt-svc to curl's configure - doh: use dedicated probe slots +- RELEASE-PROCEDURE: feature win is closed post-release a few days - ... to easier allow additional DNS transactions. + We've tried to uphold this already but let's make it official by + publicly stating this is the way we do it. - Closes #4629 + Closes #4877 -- travis: build ngtcp2 with --enable-lib-only +- altsvc: set h3 version at a common single spot - ... makes it skip the examples and other stuff we don't neeed. + ... and move the #ifdefs out of the functions. Addresses the fact they + were different before this change. - Closes #4646 + Reported-by: Harry Sintonen + Closes #4876 -- [David Benjamin brought this change] +- [Harry Sintonen brought this change] - ngtcp2: fix thread-safety bug in error-handling - - ERR_error_string(NULL) should never be called. It places the error in a - global buffer, which is not thread-safe. Use ERR_error_string_n with a - local buffer instead. + altsvc: improved header parser - Closes #4645 - -- travis: export the CC/CXX variables when set + - Fixed the flag parsing to apply to specific alternative entry only, as + per RFC. The earlier code would also get totally confused by + multiprotocol header, parsing flags from the wrong part of the header. - Suggested-by: Peter Wu - Fixes #4637 - Closes #4640 - -Marcel Raad (26 Nov 2019) -- dist: add error-codes.pl + - Fixed the parser terminating on unknown protocols, instead of skipping + them. - Follow-up to commit 74f441c6d31. - This should fix test 1175 when run via the daily source tarballs. + - Fixed a busyloop when protocol-id was present without an equal sign. - Closes https://github.com/curl/curl/pull/4638 + Closes #4875 -Daniel Stenberg (26 Nov 2019) -- [John Schroeder brought this change] +- [Harry Sintonen brought this change] - curl: fix --upload-file . hangs if delay in STDIN + ngtcp2: fixed to only use AF_INET6 when ENABLE_IPV6 + +- docs/HTTP3: update the OpenSSL branch to use for ngtcp2 - Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION. + Reported-by: James Fuller + +Steve Holme (4 Feb 2020) +- ntlm: Pass the Curl_easy structure to the private winbind functions - When uploading from stdin in non-blocking mode, a delay in reading - the stream (EAGAIN) causes curl to pause sending data - (CURL_READFUNC_PAUSE). Prior to this change, a busy read was - detected and unpaused only in the CURLOPT_WRITEFUNCTION handler. - This change performs the same busy read handling in a - CURLOPT_XFERINFOFUNCTION handler. + ...rather than the full conndata structure. + +Daniel Stenberg (4 Feb 2020) +- RELEASE-NOTES: synced + +- tool_operhlp: Copyright year out of date, should be 2020 - Fixes #2051 - Closes #4599 - Reported-by: bdry on github + Follow-up from 2bc373740a3 -- [John Schroeder brought this change] +- [Orgad Shaneh brought this change] - XFERINFOFUNCTION: support CURL_PROGRESSFUNC_CONTINUE + curl: avoid using strlen for testing if a string is empty - (also for PROGRESSFUNCTION) - - By returning this value from the callback, the internal progress - function call is still called afterward. - - Closes #4599 + Closes #4873 -- [Michael Forney brought this change] +Steve Holme (3 Feb 2020) +- ntlm: Ensure the HTTP header data is not stored in the challenge/response - TLS: add BearSSL vtls implementation +Marcel Raad (3 Feb 2020) +- openssl: remove redundant assignment - Closes #4597 - -- curl_multi_wakeup.3: add example and AVAILABILITY + Fixes a scan-build failure on Bionic. - Reviewed-by: Gergely Nagy - Closes #4635 - -- [Gergely Nagy brought this change] + Closes https://github.com/curl/curl/pull/4872 - multi: add curl_multi_wakeup() +- travis: update non-OpenSSL Linux jobs to Bionic - This commit adds curl_multi_wakeup() which was previously in the TODO - list under the curl_multi_unblock name. + For the OpenSSL builds, test 323 [TLS-SRP to non-TLS-SRP server] is + failing with "curl returned 52, when expecting 35". - On some platforms and with some configurations this feature might not be - available or can fail, in these cases a new error code - (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). + Closes https://github.com/curl/curl/pull/4872 + +Dan Fandrich (3 Feb 2020) +- cirrus: Add some missing semicolons - Fixes #4418 - Closes #4608 + Newlines aren't preserved in this section so they're needed to separate + commands. The exports luckily worked anyway as a single long line, but + erroneously exported a variable called "export" + [skip ci] -Jay Satiro (24 Nov 2019) -- [Xiaoyin Liu brought this change] +Daniel Gustafsson (2 Feb 2020) +- [Pedro Monreal brought this change] - schannel: fix --tls-max for when min is --tlsv1 or default - - Prior to this change schannel ignored --tls-max (CURL_SSLVERSION_MAX_ - macros) when --tlsv1 (CURL_SSLVERSION_TLSv1) or default TLS - (CURL_SSLVERSION_DEFAULT), using a max of TLS 1.2 always. + cleanup: fix typos and wording in docs and comments - Closes https://github.com/curl/curl/pull/4633 + Closes #4869 + Reviewed-by: Emil Engler and Daniel Gustafsson -- checksrc.bat: Add a check for vquic and vssh directories +Steve Holme (2 Feb 2020) +- ntlm: Move the winbind data into the NTLM data structure - Ref: https://github.com/curl/curl/pull/4607 + To assist with adding winbind support to the SASL NTLM authentication, + move the winbind specific data out of conndata into ntlmdata. -- projects: Fix Visual Studio projects SSH builds - - - Generate VQUIC and VSSH filenames in Visual Studio project files. +Daniel Stenberg (30 Jan 2020) +- quiche: Copyright year out of date - Prior to this change generated Visual Studio project configurations that - enabled SSH did not build properly. Broken since SSH files were moved to - lib/vssh 3 months ago in 5b2d703. + Follow-up to 7fc63d72333a + +- altsvc: use h3-25 - Fixes https://github.com/curl/curl/issues/4492 - Fixes https://github.com/curl/curl/issues/4630 - Closes https://github.com/curl/curl/pull/4607 + Closes #4868 -Daniel Stenberg (23 Nov 2019) -- RELEASE-NOTES: synced +- [Alessandro Ghedini brought this change] -Jay Satiro (22 Nov 2019) -- openssl: Revert to less sensitivity for SYSCALL errors - - - Disable the extra sensitivity except in debug builds (--enable-debug). - - - Improve SYSCALL error message logic in ossl_send and ossl_recv so that - "No error" / "Success" socket error text isn't shown on SYSCALL error. - - Prior to this change 0ab38f5 (precedes 7.67.0) increased the sensitivity - of OpenSSL's SSL_ERROR_SYSCALL error so that abrupt server closures were - also considered errors. For example, a server that does not send a known - protocol termination point (eg HTTP content length or chunked encoding) - _and_ does not send a TLS termination point (close_notify alert) would - cause an error if it closed the connection. + quiche: update to draft-25 - To be clear that behavior made it into release build 7.67.0 - unintentionally. Several users have reported it as an issue. + Closes #4867 + +- ngtcp2: update to git master and its draft-25 support - Ultimately the idea is a good one, since it can help prevent against a - truncation attack. Other SSL backends may already behave similarly (such - as Windows native OS SSL Schannel). However much more of our user base - is using OpenSSL and there is a mass of legacy users in that space, so I - think that behavior should be partially reverted and then rolled out - slowly. + Closes #4865 + +- cookie: check __Secure- and __Host- case sensitively - This commit changes the behavior so that the increased sensitivity is - disabled in all curl builds except curl debug builds (DEBUGBUILD). If - after a period of time there are no major issues then it can be enabled - in dev and release builds with the newest OpenSSL (1.1.1+), since users - using the newest OpenSSL are the least likely to have legacy problems. + While most keywords in cookies are case insensitive, these prefixes are + specified explicitly to get checked "with a case-sensitive match". - Bug: https://github.com/curl/curl/issues/4409#issuecomment-555955794 - Reported-by: Bjoern Franke + (From the 6265bis document in progress) - Fixes https://github.com/curl/curl/issues/4624 - Closes https://github.com/curl/curl/pull/4623 + Ref: https://tools.ietf.org/html/draft-ietf-httpbis-rfc6265bis-04 + Closes #4864 -- [Daniel Stenberg brought this change] +- KNOWN_BUGS: Multiple methods in a single WWW-Authenticate: header - openssl: improve error message for SYSCALL during connect - - Reported-by: Paulo Roberto Tomasi - Bug: https://curl.haxx.se/mail/archive-2019-11/0005.html +- oauth2-bearer.d: works for HTTP too - Closes https://github.com/curl/curl/pull/4593 + Reported-by: Mischa Salle + Bug: https://curl.haxx.se/mail/lib-2020-01/0070.html + Closes #4862 -Daniel Stenberg (22 Nov 2019) -- test1175: verify symbols-in-versions and libcurl-errors.3 in sync +- multi_done: if multiplexed, make conn->data point to another transfer - Closes #4628 + ... since the current transfer is being killed. Setting to NULL is + wrong, leaving it pointing to 'data' is wrong since that handle might be + about to get freed. + + Fixes #4845 + Closes #4858 + Reported-by: dmitrmax on github -- include: make CURLE_HTTP3 use a new error code +- location.d: the method change is from POST to GET only - To avoid potential issues with error code reuse. + Not from generic non-GET to GET. - Reported-by: Christoph M. Becker - Assisted-by: Dan Fandrich - Fixes #4601 - Closes #4627 - -- bump: next release will be 7.68.0 + Reported-by: Andrius Merkys + Ref: #4859 + Closes #4861 -- curl: add --parallel-immediate +- urlapi: guess scheme correct even with credentials given - Starting with this change when doing parallel transfers, without this - option set, curl will prefer to create new transfers multiplexed on an - existing connection rather than creating a brand new one. + In the "scheme-less" parsing case, we need to strip off credentials + first before we guess scheme based on the host name! - --parallel-immediate can be set to tell curl to prefer to use new - connections rather than to wait and try to multiplex. + Assisted-by: Jay Satiro + Fixes #4856 + Closes #4857 + +- global_init: move the IPv6 works status bool to multi handle - libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default - on parallel transfers. + Previously it was stored in a global state which contributed to + curl_global_init's thread unsafety. This boolean is now instead figured + out in curl_multi_init() and stored in the multi handle. Less effective, + but thread safe. - Suggested-by: Tom van der Woerdt - Closes #4500 + Closes #4851 -Daniel Gustafsson (20 Nov 2019) -- [Victor Magierski brought this change] +- [Jay Satiro brought this change] - docs: fix typos - - Change 'experiemental' to 'experimental'. + README: mention that the docs is in docs/ - Closes #4618 - Reviewed-by: Daniel Gustafsson + Reported-by: Austin Green + Fixes #4830 + Closes #4853 -Jay Satiro (18 Nov 2019) -- projects: Fix Visual Studio wolfSSL configurations +- curl.h: define CURL_WIN32 on windows - - s/USE_CYASSL/USE_WOLFSSL/ + ... so that the subsequent logic below can use a single known define to know + when built on Windows (as we don't define WIN32 anymore). - - Remove old compatibility macros. + Follow-up to 1adebe7886ddf20b - Follow-up to 1c6c59a from several months ago when CyaSSL named symbols - were renamed to wolfSSL. The wolfSSL library was formerly named CyaSSL - and we kept using their old name for compatibility reasons, until - earlier this year. + Reported-by: crazydef on github + Assisted-by: Marcel Raad + Fixes #4854 + Closes #4855 -Daniel Stenberg (18 Nov 2019) - RELEASE-NOTES: synced -- [Javier Blazquez brought this change] +- [Jon Rumsey brought this change] - ngtcp2: use overflow buffer for extra HTTP/3 data + urldata: do string enums without #ifdefs for build scripts - Fixes #4525 - Closes #4603 + ... and check for inconsistencies for OS400 at build time with the new + chkstrings tool. + + Closes #4822 -- altsvc: bump to h3-24 +- curl: make the -# spaceship bar not wrap the line - ... as both ngtcp2 and quiche now support that in their master branches + The fixed-point math made us lose precision and thus a too high index + value could be used for outputting the hashtags which could overwrite + the newline. - Closes #4604 - -- ngtcp2: free used resources on disconnect + The fix increases the precision in the sine table (*100) and the + associated position math. - Fixes #4614 - Closes #4615 + Reported-by: Andrew Potter + Fixes #4849 + Closes #4850 -- ngtcp2: handle key updates as ngtcp2 master branch tells us +- global_init: assume the EINTR bit by default - Reviewed-by: Tatsuhiro Tsujikawa + - Removed from global_init since it isn't thread-safe. The symbol will + still remain to not break compiles, it just won't have any effect going + forward. - Fixes #4612 - Closes #4613 + - make the internals NOT loop on EINTR (the opposite from previously). + It only risks returning from the select/poll/wait functions early, and that + should be risk-free. + + Closes #4840 -Jay Satiro (17 Nov 2019) -- [Gergely Nagy brought this change] +- [Peter Piekarski brought this change] - multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds + conn: do not reuse connection if SOCKS proxy credentials differ - Prior to this change: + Closes #4835 + +- llist: removed unused Curl_llist_move() - The check if an extra wait is necessary was based not on the - number of extra fds but on the pointer. - - If a non-null pointer was given in extra_fds, but extra_nfds - was zero, then the wait was skipped even though poll was not - called. + (and the corresponding unit test) - Closes https://github.com/curl/curl/pull/4610 + Closes #4842 -- lib: Move lib/ssh.h -> lib/vssh/ssh.h - - Follow-up to 5b2d703 which moved ssh source files to vssh. +- conncache: removed unused Curl_conncache_bundle_size() + +- strcase: turn Curl_raw_tolower into static - Closes https://github.com/curl/curl/pull/4609 + Only ever used from within this file. -Daniel Stenberg (16 Nov 2019) -- [Andreas Falkenhahn brought this change] +- singleuse.pl: support new API functions, fix curl_dbg_ handling - INSTALL.md: provide Android build instructions +- wolfssh: make it init properly via Curl_ssh_init() - Closes #4606 + Closes #4846 -- [Niall O'Reilly brought this change] +- [Aron Rotteveel brought this change] - doh: improced both encoding and decoding - - Improved estimation of expected_len and updated related comments; - increased strictness of QNAME-encoding, adding error detection for empty - labels and names longer than the overall limit; avoided treating DNAME - as unexpected; - - updated unit test 1655 with more thorough set of proofs and tests + form.d: fix two minor typos - Closes #4598 + Closes #4843 -- ngtcp2: increase QUIC window size when data is consumed +- openssl: make CURLINFO_CERTINFO not truncate x509v3 fields - Assisted-by: Javier Blazquez - Ref #4525 (partial fix) - Closes #4600 - -- [Melissa Mears brought this change] + Avoid "reparsing" the content and instead deliver more exactly what is + provided in the certificate and avoid truncating the data after 512 + bytes as done previously. This no longer removes embedded newlines. + + Fixes #4837 + Reported-by: bnfp on github + Closes #4841 - config-win32: cpu-machine-OS for Windows on ARM +Jay Satiro (23 Jan 2020) +- CURLOPT_PROXY_SSL_OPTIONS.3: Sync with CURLOPT_SSL_OPTIONS.3 - Define the OS macro properly for Windows on ARM builds. Also, we might - as well add the GCC-style IA-64 macro. + - Copy CURLOPT_SSL_OPTIONS.3 description to CURLOPT_PROXY_SSL_OPTIONS.3. - Closes #4590 + Prior to this change CURLSSLOPT_NO_PARTIALCHAIN was missing from the + CURLOPT_PROXY_SSL_OPTIONS description. -- examples: add multi-poll.c +Daniel Stenberg (22 Jan 2020) +- mk-ca-bundle: add support for CKA_NSS_SERVER_DISTRUST_AFTER - Show how curl_multi_poll() makes it even easier to use the multi - interface. + For now, no cert in the bundle actually sets a date there... - Closes #4596 + Co-Authored-by: Jay Satiro + Reported-by: Christian Heimes + Fixes #4834 + Closes #4836 -- multi_poll: avoid busy-loop when called without easy handles attached - - Fixes #4594 - Closes #4595 - Reported-by: 3dyd on github +- RELEASE-NOTES: synced -- curl: fix -T globbing - - Regression from e59371a4936f8 (7.67.0) +- [Pavel Volgarev brought this change] + + smtp: Allow RCPT TO command to fail for some recipients - Added test 490, 491 and 492 to verify the functionality. + Introduces CURLOPT_MAIL_RCPT_ALLLOWFAILS. - Reported-by: Kamil Dudka - Reported-by: Anderson Sasaki + Verified with the new tests 3002-3007 - Fixes #4588 - Closes #4591 - -- HISTORY: added cmake, HTTP/3 and parallel downloads with curl + Closes #4816 -- quiche: reject headers in the wrong order - - Pseudo header MUST come before regular headers or cause an error. +- copyright: fix year ranges - Reported-by: Cynthia Coan - Fixes #4571 - Closes #4584 + follow-up from dea17b519d (one of these days I'll learn to check before + I push) -- openssl: prevent recursive function calls from ctx callbacks - - Follow the pattern of many other callbacks. - - Ref: #4546 - Closes #4585 +- [nao brought this change] -- CURL-DISABLE: initial docs for the CURL_DISABLE_* defines + http: move "oauth_bearer" from connectdata to Curl_easy - The disable-scan script used in test 1165 is extended to also verify - that the docs cover all used defines and all defines offered by - configure. + Fixes the bug where oauth_bearer gets deallocated when we re-use a + connection. - Reported-by: SLDiggie on github - Fixes #4545 - Closes #4587 + Closes #4824 -- remove_handle: clear expire timers after multi_done() - - Since 59041f0, a new timer might be set in multi_done() so the clearing - of the timers need to happen afterwards! - - Reported-by: Max Kellermann - Fixes #4575 - Closes #4583 +- [Emil Engler brought this change] -Marcel Raad (10 Nov 2019) -- test1558: use double slash after file: - - Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this - test unnecessarily failed when using `file:/` instead of `file:///`. + curl: Let -D merge headers in one file again - Closes https://github.com/curl/curl/pull/4554 + Closes #4762 + Fixes #4753 -Daniel Stenberg (10 Nov 2019) -- pause: avoid updating socket if done was already called +- data.d: remove "Multiple files can also be specified" - ... avoids unnecesary recursive risk when the transfer is already done. + It is superfluous and could even be misleading. - Reported-by: Richard Bowker - Fixes #4563 - Closes #4574 + Bug: https://curl.haxx.se/mail/archive-2020-01/0016.html + Reported-by: Mike Norton + Closes #4832 -Jay Satiro (9 Nov 2019) -- strerror: Fix an error looking up some Windows error strings - - - Use FORMAT_MESSAGE_IGNORE_INSERTS to ignore format specifiers in - Windows error strings. +Marcel Raad (20 Jan 2020) +- CMake: support specifying the target Windows version - Since we are not in control of the error code we don't know what - information may be needed by the error string's format specifiers. + Previously, it was only possible to set it to Windows Vista or XP by + setting the option `ENABLE_INET_PTON` to `ON` resp. `OFF`. + Use a new cache variable `CURL_TARGET_WINDOWS_VERSION` to be able to + explicitly set the target Windows version. `ENABLE_INET_PTON` is + ignored in this case. - Prior to this change Windows API error strings which contain specifiers - (think specifiers like similar to printf specifiers) would not be shown. - The FormatMessage Windows API call which turns a Windows error code into - a string could fail and set error ERROR_INVALID_PARAMETER if that error - string contained a format specifier. FormatMessage expects a va_list for - the specifiers, unless inserts are ignored in which case no substitution - is attempted. + Ref: https://github.com/curl/curl/pull/1639#issuecomment-313039352 + Ref: https://github.com/curl/curl/pull/4607#issuecomment-557541456 + Closes https://github.com/curl/curl/pull/4815 + +Daniel Stenberg (20 Jan 2020) +- http.h: Copyright year out of date, should be 2020 - Ref: https://devblogs.microsoft.com/oldnewthing/20071128-00/?p=24353 + Follow-up to 7ff9222ced8c -- [r-a-sattarov brought this change] +- [加藤郁之 brought this change] - system.h: fix for MCST lcc compiler - - Fixed build by MCST lcc compiler on MCST Elbrus 2000 architecture and do - some code cleanup. - - e2k (Elbrus 2000) - this is VLIW/EPIC architecture, like Intel Itanium - architecture. + HTTP: increase EXPECT_100_THRESHOLD to 1Mb - Ref: https://en.wikipedia.org/wiki/Elbrus_2000 + Mentioned: https://curl.haxx.se/mail/lib-2020-01/0050.html - Closes https://github.com/curl/curl/pull/4576 + Closes #4814 -Daniel Stenberg (8 Nov 2019) -- TODO: curl_multi_unblock +- ROADMAP: thread-safe `curl_global_init()` - Closes #4418 + I'd like to see this happen. -- TODO: Run web-platform-tests url tests - - Closes #4477 +- RELEASE-NOTES: synced -- TODO: 1.4 alt-svc sharing +- wolfssl: use the wc-prefixed symbol alternatives - Closes #4476 - -- test1560: require IPv6 for IPv6 aware URL parsing + The symbols without wc_ prefix are not always provided. - The URL parser function can't reject a bad IPv6 address properly when - curl was built without IPv6 support. + Ref: https://github.com/wolfSSL/wolfssl/issues/2744 - Reported-by: Marcel Raad - Fixes #4556 - Closes #4572 + Closes #4827 -- checksrc: repair the copyrightyear check - - - Consider a modified file to be committed this year. - - - Make the travis CHECKSRC also do COPYRIGHTYEAR scan in examples and - includes +- polarssl: removed - - Ignore 0 parents when getting latest commit date of file. + As detailed in DEPRECATE.md, the polarssl support is now removed after + having been disabled for 6 months and nobody has missed it. - since in the CI we're dealing with a truncated repo of last 50 commits, - the file's most recent commit may not be available. when this happens - git log and rev-list show the initial commit (ie first commit not to be - truncated) but that's incorrect so ignore it. + The threadlock files used by mbedtls are renamed to an 'mbedtls' prefix + instead of the former 'polarssl' and the common functions that + previously were shared between mbedtls and polarssl and contained the + name 'polarssl' have now all been renamed to instead say 'mbedtls'. - Ref: https://github.com/curl/curl/pull/4547 + Closes #4825 + +Marcel Raad (16 Jan 2020) +- libssh2: fix variable type - Closes https://github.com/curl/curl/pull/4549 + This led to a conversion warning on 64-bit MinGW, which has 32-bit + `long` but 64-bit `size_t`. - Co-authored-by: Jay Satiro + Closes https://github.com/curl/curl/pull/4823 -- copyrights: fix copyright year range - - .. because checksrc's copyright year check stopped working. +Daniel Stenberg (16 Jan 2020) +- curl:progressbarinit: ignore column width from terminals < 20 - Ref: https://github.com/curl/curl/pull/4547 + To avoid division by zero - or other issues. - Closes https://github.com/curl/curl/pull/4549 + Reported-by: Daniel Marjamäki + Closes #4818 -- RELEASE-NOTES: synced +- wolfssh: set the password correctly for PASSWORD auth -- curlver: bump to 7.67.1 +- wolfssh: remove fprintf() calls (and uses of __func__) -- mailmap: fixup Massimiliano Fantuzzi - -- scripts/contributors: make committers get included too +Marcel Raad (14 Jan 2020) +- CMake: use check_symbol_exists also for inet_pton - in addition to authors - -Jay Satiro (8 Nov 2019) -- [Massimiliano Fantuzzi brought this change] - - configure: fix typo in help text + It doesn't make much sense to only check if the function can be linked + when it's not declared in any header and that is treated as an error. + With the correct target Windows version set, the function is declared + in ws2tcpip.h and the comment above the modified block is invalid. - Closes https://github.com/curl/curl/pull/4570 - -Daniel Stenberg (7 Nov 2019) -- [Christian Schmitz brought this change] - - ntlm: USE_WIN32_CRYPTO check removed to get USE_NTLM2SESSION set + Also, move the definition of `_WIN32_WINNT` up to before all symbol + availability checks so that we don't have to care which ones must be + done after it. - Closes #3704 - -Jay Satiro (6 Nov 2019) -- [Wyatt O'Day brought this change] - - build: fix for CURL_DISABLE_DOH + Tested with Visual Studio 2019 and current MinGW-w64. - Fixes https://github.com/curl/curl/issues/4565 - Closes https://github.com/curl/curl/pull/4566 + Closes https://github.com/curl/curl/pull/4808 -- [Leonardo Taccari brought this change] +Jay Satiro (13 Jan 2020) +- schannel_verify: Fix alt names manual verify for UNICODE builds + + Follow-up to 29e40a6 from two days ago, which added that feature for + Windows 7 and earlier. The bug only occurred in same. + + Ref: https://github.com/curl/curl/pull/4761 - configure: avoid unportable `==' test(1) operator +Daniel Stenberg (13 Jan 2020) +- HTTP-COOKIES.md: describe the cookie file format - Closes https://github.com/curl/curl/pull/4567 + ... and refer to that file from from CURLOPT_COOKIEFILE.3 and + CURLOPT_COOKIELIST.3 + + Assisted-by: Jay Satiro + Reported-by: bsammon on github + Fixes #4805 + Closes #4806 -Version 7.67.0 (5 Nov 2019) +- [Tobias Hieta brought this change] -Daniel Stenberg (5 Nov 2019) -- RELEASE-NOTES: synced + CMake: Add support for CMAKE_LTO option. - The 7.67.0 release + This enables Link Time Optimization. LTO is a proven technique for + optimizing across compilation units. + + Closes #4799 -- THANKS: add new names from 7.67.0 +- RELEASE-NOTES: synced -- configure: only say ipv6 enabled when the variable is set +- ConnectionExists: respect the max_concurrent_streams limits - Previously it could say "IPv6: enabled" at the end of the configure run - but the define wasn't set because of a missing getaddrinfo(). + A regression made the code use 'multiplexed' as a boolean instead of the + counter it is intended to be. This made curl try to "over-populate" + connections with new streams. - Reported-by: Marcel Raad - Fixes #4555 - Closes #4560 - -Marcel Raad (2 Nov 2019) -- certs/Server-localhost-lastSAN-sv: regenerate with sha256 + This regression came with 41fcdf71a1, shipped in curl 7.65.0. - All other certificates were regenerated in commit ba782baac30, but - this one was missed. - Fixes test3001 on modern systems. + Also, respect the CURLMOPT_MAX_CONCURRENT_STREAMS value in the same + check. - Closes https://github.com/curl/curl/pull/4551 - -Daniel Stenberg (2 Nov 2019) -- [Vilhelm Prytz brought this change] + Reported-by: Kunal Ekawde + Fixes #4779 + Closes #4784 - copyrights: update all copyright notices to 2019 on files changed this year +- curl: make #0 not output the full URL - Closes #4547 - -- [Bastien Bouclet brought this change] - - mbedtls: add error message for cert validity starting in the future + It was not intended nor documented! - Closes #4552 - -Jay Satiro (1 Nov 2019) -- schannel_verify: Fix concurrent openings of CA file + Added test 1176 to verify. - - Open the CA file using FILE_SHARE_READ mode so that others can read - from it as well. + Reported-by: vshmuk on hackerone - Prior to this change our schannel code opened the CA file without - sharing which meant concurrent openings (eg an attempt from another - thread or process) would fail during the time it was open without - sharing, which in curl's case would cause error: - "schannel: failed to open CA file". + Closes #4812 + +- wolfSSH: new SSH backend - Bug: https://curl.haxx.se/mail/lib-2019-10/0104.html - Reported-by: Richard Alcock + Adds support for SFTP (not SCP) using WolfSSH. + + Closes #4231 -Daniel Stenberg (31 Oct 2019) -- gtls: make gnutls_bye() not wait for response on shutdown +- curl: remove 'config' field from OutStruct - ... as it can make it wait there for a long time for no good purpose. + As it was just unnecessary duplicated information already stored in the + 'per_transfer' struct and that's around mostly anyway. - Patched-by: Jay Satiro - Reported-by: Bylon2 on github - Adviced-by: Nikos Mavrogiannopoulos + The duplicated pointer caused problems when the code flow was aborted + before the dupe was filled in and could cause a NULL pointer access. - Fixes #4487 - Closes #4541 - -- [Michał Janiszewski brought this change] + Reported-by: Brian Carpenter + Fixes #4807 + Closes #4810 - appveyor: publish artifacts on appveyor +- misc: Copyright year out of date, should be 2020 - This allows obtaining upstream builds of curl directly from appveyor for - all the available configurations + Follow-up to recent commits - Closes #4509 + [skip ci] -- url: make Curl_close() NULLify the pointer too +Jay Satiro (11 Jan 2020) +- [Santino Keupp brought this change] + + libssh2: add support for forcing a hostkey type - This is the common pattern used in the code and by a unified approach we - avoid mistakes. + - Allow forcing the host's key type found in the known_hosts file. - Closes #4534 + Currently, curl (with libssh2) does not take keys from your known_hosts + file into account when talking to a server. With this patch the + known_hosts file will be searched for an entry matching the hostname + and, if found, libssh2 will be told to claim this key type from the + server. + + Closes https://github.com/curl/curl/pull/4747 -- [Trivikram Kamat brought this change] +- [Nicolas Guillier brought this change] - INSTALL: add missing space for configure commands + cmake: Improve libssh2 check on Windows - Closes #4539 - -- url: Curl_free_request_state() should also free doh handles + - Add "libssh2" name to FindLibSSH2 library search. - ... or risk DoH memory leaks. + On Windows systems, libSSH2 CMake installation may name the library + "LibSSH2". - Reported-by: Paul Dreik - Fixes #4463 - Closes #4527 - -- examples: remove the "this exact code has not been verified" + Prior to this change cmake only checked for name "ssh2". On Linux that + works fine because it will prepend the "lib", but it doesn't do that on + Windows. - ... as really confuses the reader to not know what to believe! + Closes https://github.com/curl/curl/pull/4804 -- [Trivikram Kamat brought this change] +- [Faizur Rahman brought this change] - HTTP3: fix typo somehere1 > somewhere1 + schannel: Make CURLOPT_CAINFO work better on Windows 7 - Closes #4535 - -Jay Satiro (28 Oct 2019) -- [Javier Blazquez brought this change] - - HTTP3: fix invalid use of sendto for connected UDP socket + - Support hostname verification via alternative names (SAN) in the + peer certificate when CURLOPT_CAINFO is used in Windows 7 and earlier. - On macOS/BSD, trying to call sendto on a connected UDP socket fails - with a EISCONN error. Because the singleipconnect has already called - connect on the socket when we're trying to use it for QUIC transfers - we need to use plain send instead. + CERT_NAME_SEARCH_ALL_NAMES_FLAG doesn't exist before Windows 8. As a + result CertGetNameString doesn't quite work on those versions of + Windows. This change provides an alternative solution for + CertGetNameString by iterating through CERT_ALT_NAME_INFO for earlier + versions of Windows. - Fixes #4529 - Closes https://github.com/curl/curl/pull/4533 - -Daniel Stenberg (28 Oct 2019) -- RELEASE-NOTES: synced + Prior to this change many certificates failed the hostname validation + when CURLOPT_CAINFO was used in Windows 7 and earlier. Most certificates + now represent multiple hostnames and rely on the alternative names field + exclusively to represent their hostnames. + + Reported-by: Jeroen Ooms + + Fixes https://github.com/curl/curl/issues/3711 + Closes https://github.com/curl/curl/pull/4761 -- [Javier Blazquez brought this change] +- [Emil Engler brought this change] - HTTP3: fix Windows build + ngtcp2: Add an error code for QUIC connection errors - The ngtcp2 QUIC backend was using the MSG_DONTWAIT flag for send/recv - in order to perform nonblocking operations. On Windows this flag does - not exist. Instead, the socket must be set to nonblocking mode via - ioctlsocket. + - Add new error code CURLE_QUIC_CONNECT_ERROR for QUIC connection + errors. - This change sets the nonblocking flag on UDP sockets used for QUIC on - all platforms so the use of MSG_DONTWAIT is not needed. + Prior to this change CURLE_FAILED_INIT was used, but that was not + correct. - Fixes #4531 - Closes #4532 + Closes https://github.com/curl/curl/pull/4754 -Marcel Raad (27 Oct 2019) -- appveyor: add --disable-proxy autotools build - - This would have caught issue #3926. +- multi: Change curl_multi_wait/poll to error on negative timeout - Also make formatting more consistent. + - Add new error CURLM_BAD_FUNCTION_ARGUMENT and return that error when + curl_multi_wait/poll is passed timeout param < 0. - Closes https://github.com/curl/curl/pull/4526 - -Daniel Stenberg (25 Oct 2019) -- appveyor: make winbuilds with DEBUG=no/yes and VS 2015/2017 + Prior to this change passing a negative value to curl_multi_wait/poll + such as -1 could cause the function to wait forever. - ... and invoke "curl -V" once done + Reported-by: hamstergene@users.noreply.github.com - Co-Authored-By: Jay Satiro + Fixes https://github.com/curl/curl/issues/4763 - Closes #4523 + Closes https://github.com/curl/curl/pull/4765 -- [Francois Rivard brought this change] +- [Marc Aldorasi brought this change] - schannel: reverse the order of certinfo insertions + cmake: Enable SMB for Windows builds - Fixes #4518 - Closes #4519 - -Marcel Raad (24 Oct 2019) -- test1591: fix spelling of http feature + - Define USE_WIN32_CRYPTO by default. This enables SMB. - The test never got run because the feature name is `http` in lowercase. + - Show whether SMB is enabled in the "Enabled features" output. - Closes https://github.com/curl/curl/pull/4520 - -Daniel Stenberg (23 Oct 2019) -- [Michał Janiszewski brought this change] + - Fix mingw compiler warning for call to CryptHashData by casting away + const param. mingw CryptHashData prototype is wrong. + + Closes https://github.com/curl/curl/pull/4717 - appveyor: Use two parallel compilation on appveyor with CMake +- vtls: Refactor Curl_multissl_version to make the code clearer - Appveyor provides 2 CPUs for each builder[1], make sure to use parallel - compilation, when running with CMake. CMake learned this new option in - version 3.12[2] and the version provided by appveyor is fresh enough. + Reported-by: Johannes Schindelin - Curl doesn't really take that long to build and it is using the slowest - builder available, msbuild, so expect only a moderate improvement in - build times. + Ref: https://github.com/curl/curl/pull/3863#pullrequestreview-241395121 - [1] https://www.appveyor.com/docs/build-environment/ - [2] https://cmake.org/cmake/help/v3.12/release/3.12.html + Closes https://github.com/curl/curl/pull/4803 + +Daniel Stenberg (10 Jan 2020) +- fix: Copyright year out of date, should be 2020 - Closes #4508 + Follow-up to 875314ed0bf3b -- conn-reuse: requests wanting NTLM can reuse non-NTLM connections +Marcel Raad (10 Jan 2020) +- hostip: move code to resolve IP address literals to `Curl_resolv` - Added test case 338 to verify. + The code was duplicated in the various resolver backends. - Reported-by: Daniel Silverstone - Fixes #4499 - Closes #4514 - -Marcel Raad (23 Oct 2019) -- tests: add missing proxy features + Also, it was called after the call to `Curl_ipvalid`, which matters in + case of `CURLRES_IPV4` when called from `connect.c:bindlocal`. This + caused test 1048 to fail on classic MinGW. + + The code ignores `conn->ip_version` as done previously in the + individual resolver backends. + + Move the call to the `resolver_start` callback up to appease test 655, + which wants it to be called also for literal addresses. + + Closes https://github.com/curl/curl/pull/4798 -Daniel Stenberg (22 Oct 2019) -- RELEASE-NOTES: synced +Daniel Stenberg (9 Jan 2020) +- scripts/delta: adapt to new public header layout -Marcel Raad (21 Oct 2019) -- tests: use %FILE_PWD for file:// URLs +- test1167: verify global symbols in public headers are curl prefixed - This way, we always have exactly one slash after the host name, making - the tests pass when curl is compiled with the MSYS GCC. + ... using the new badsymbols.pl perl script - Closes https://github.com/curl/curl/pull/4512 + Fixes #4793 + Closes #4794 -- tests: add `connect to non-listen` keywords - - These tests try to connect to ports nothing is listening on. +- libtest/mk-lib1521: adapt to new public header layout + +- include: remove non-curl prefixed defines - Closes https://github.com/curl/curl/pull/4511 + ...requires some rearranging of the setup of CURLOPT_ and CURLMOPT_ + enums. -- runtests: get textaware info from curl instead of perl +- curl.h: remove WIN32 define - The MSYS system on Windows can run the test suite for curl built with - any toolset. When built with the MSYS GCC, curl uses Unix line endings, - while it uses Windows line endings when built with the MinGW GCC, and - `^O` reports 'msys' in both cases. Use the curl executable itself to - determine the line endings instead, which reports 'x86_64-pc-msys' when - built with the MSYS GCC. + It isn't our job to define this in a public header - and it defines a + name outside of our naming scope. + +- tool_dirhie.c: fix the copyright year range - Closes https://github.com/curl/curl/pull/4506 + Follow-up to: 4027bd72d9 -Daniel Stenberg (20 Oct 2019) -- [Michał Janiszewski brought this change] +- bump: work towards 7.69.0 is started - appveyor: Add MSVC ARM64 build +Jay Satiro (9 Jan 2020) +- tool_dirhie: Allow directory traversal during creation - Closes #4507 - -- http2_recv: a closed stream trumps pause state + - When creating a directory hierarchy do not error when mkdir fails due + to error EACCESS (13) "access denied". - ... and thus should return 0, not EAGAIN. + Some file systems allow for directory traversal; in this case that it + should be possible to create child directories when permission to the + parent directory is restricted. - Reported-by: Tom van der Woerdt - Fixes #4496 - Closes #4505 - -- http2: expire a timeout at end of stream + This is a regression caused by me in f16bed0 (precedes curl-7_61_1). + Basically I had assumed that if a directory already existed it would + fail only with error EEXIST, and not error EACCES. The latter may + happen if the directory exists but has certain restricted permissions. - To make sure that transfer is being dealt with. Streams without - Content-Length need a final read to notice the end-of-stream state. + Reported-by: mbeifuss@users.noreply.github.com - Reported-by: Tom van der Woerdt - Fixes #4496 + Fixes https://github.com/curl/curl/issues/4796 + Closes https://github.com/curl/curl/pull/4797 -Dan Fandrich (18 Oct 2019) -- travis: Add an ARM64 build +Daniel Stenberg (9 Jan 2020) +- KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers - Test 323 is failing for some reason, so disable it there for now. + Closes #4080 -Marcel Raad (18 Oct 2019) -- examples/sslbackend: fix -Wchar-subscripts warning +- docs/RELEASE-PROCEDURE.md: pushed some release dates - With the `isdigit` implementation that comes with MSYS2, the argument - is used as an array subscript, resulting in a -Wchar-subscripts - warning. `isdigit`'s behavior is undefined if the argument is negative - and not EOF [0]. As done in lib/curl_ctype.h, cast the `char` variable - to `unsigned char` to avoid that. + Ref: https://curl.haxx.se/mail/lib-2020-01/0031.html + +- runtests: make random seed fixed for a month - [0] https://en.cppreference.com/w/c/string/byte/isdigit + When using randomized features of runtests (-R and --shallow) it is + useful to have a fixed random seed to make sure for example extra + commits in a branch or a rebase won't change the seed that would make + repeated runs work differently. - Closes https://github.com/curl/curl/pull/4503 - -Daniel Stenberg (18 Oct 2019) -- configure: remove all cyassl references + As it is also useful to change seed sometimes, the default seed is now + determined based on the current month (and first line curl -V + output). When the month changes, so will the random seed. - In particular, this removes the case where configure would find an old - cyall installation rather than a wolfssl one if present. The library is - named wolfssl in modern days so there's no real need to keep support for - the former. + The specific seed is also shown in the standard test suite top header + and it can be set explictly with the new --seed=[num] option so that the + exact order of a previous run can be achieved. - Reported-by: Jacob Barthelmeh - Closes #4502 + Closes #4734 -Marcel Raad (17 Oct 2019) -- test1162: disable MSYS2's POSIX path conversion +- RELEASE-PROCEDURE.md: fix next release date (Feb 26) - This avoids MSYS2 converting the backslasb in the URL to a slash, - causing the test to fail. + [skip ci] -Daniel Stenberg (17 Oct 2019) -- RELEASE-NOTES: synced +Version 7.68.0 (8 Jan 2020) -Jay Satiro (16 Oct 2019) -- CURLOPT_TIMEOUT.3: Clarify transfer timeout time includes queue time - - Prior to this change some users did not understand that the "request" - starts when the handle is added to the multi handle, or probably they - did not understand that some of those transfers may be queued and that - time is included in timeout. - - Reported-by: Jeroen Ooms - - Fixes https://github.com/curl/curl/issues/4486 - Closes https://github.com/curl/curl/pull/4489 +Daniel Stenberg (8 Jan 2020) +- RELEASE-NOTES: 7.68.0 -- [Stian Soiland-Reyes brought this change] +- THANKS: updated with names from the 7.68.0 release - tool_operate: Fix retry sleep time shown to user when Retry-After - - - If server header Retry-After is being used for retry sleep time then - show that value to the user instead of the normal retry sleep time. +- RELEASE-PROCEDURE: add four future release dates - This is a follow-up to 640b973 (7.66.0) which changed curl tool so that - the value from Retry-After header overrides other retry timing options. + and remove four past release dates - Closes https://github.com/curl/curl/pull/4498 + [skip ci] -Daniel Stenberg (16 Oct 2019) -- url: normalize CURLINFO_EFFECTIVE_URL - - The URL extracted with CURLINFO_EFFECTIVE_URL was returned as given as - input in most cases, which made it not get a scheme prefixed like before - if the URL was given without one, and it didn't remove dotdot sequences - etc. - - Added test case 1907 to verify that this now works as intended and as - before 7.62.0. +Marcel Raad (6 Jan 2020) +- TrackMemory tests: always remove CR before LF - Regression introduced in 7.62.0 + It was removed for output containing ' =' via `s/ =.*//`. With classic + MinGW, this made lines with `free()` end with CRLF, but lines with e.g. + `malloc()` end with only LF. The tests expect LF only. - Reported-by: Christophe Dervieux - Fixes #4491 - Closes #4493 + Closes https://github.com/curl/curl/pull/4788 -Marcel Raad (16 Oct 2019) -- tests: line ending fixes for Windows +Daniel Stenberg (6 Jan 2020) +- multi.h: move INITIAL_MAX_CONCURRENT_STREAMS from public header - Mark some files as text. + ... to the private multihhandle.h. It is not for public use and it + wasn't prefixed correctly anyway! - Closes https://github.com/curl/curl/pull/4490 + Closes #4790 -- tests: use proxy feature - - This makes the tests succeed when using --disable-proxy. +- file: fix copyright year range - Closes https://github.com/curl/curl/pull/4488 + Follow-up to 1b71bc532bd -- smbserver: fix Python 3 compatibility - - Python 2's `ConfigParser` module is spelled `configparser` in Python 3. +- curl -w: handle a blank input file correctly - Closes https://github.com/curl/curl/pull/4484 - -- security: silence conversion warning + Previously it would end up with an uninitialized memory buffer that + would lead to a crash or junk getting output. - With MinGW-w64, `curl_socket_t` is is a 32 or 64 bit unsigned integer, - while `read` expects a 32 bit signed integer. - Use `sread` instead of `read` to use the correct parameter type. + Added test 1271 to verify. - Closes https://github.com/curl/curl/pull/4483 + Reported-by: Brian Carpenter + Closes #4786 -- connect: silence sign-compare warning +- file: on Windows, refuse paths that start with \\ - With MinGW-w64 using WinSock, `curl_socklen_t` is signed, while the - result of `sizeof` is unsigned. + ... as that might cause an unexpected SMB connection to a given host + name. - Closes https://github.com/curl/curl/pull/4483 - -Daniel Stenberg (13 Oct 2019) -- TODO: Handle growing SFTP files - - Closes #4344 + Reported-by: Fernando Muñoz + CVE-2019-15601 + Bug: https://curl.haxx.se/docs/CVE-2019-15601.html -- KNOWN_BUGS: remove "CURLFORM_CONTENTLEN in an array" - - The curl_formadd() function is deprecated and shouldn't be used so the - real fix for applications is to switch to the curl_mime_* API. +Jay Satiro (6 Jan 2020) +- CURLOPT_READFUNCTION.3: fix fopen params in example -- KNOWN_BUGS: "LDAP on Windows does authentication wrong" +- CURLOPT_READFUNCTION.3: fix variable name in example - Closes #3116 - -- appveyor: add a winbuild that uses VS2017 + Reported-by: Paul Joyce - Closes #4482 - -- [Harry Sintonen brought this change] + Fixes https://github.com/curl/curl/issues/4787 - socketpair: fix include and define for older TCP header systems - - fixed build for systems that need netinet/in.h for IPPROTO_TCP and are - missing INADDR_LOOPBACK +Daniel Stenberg (5 Jan 2020) +- curl:getparameter return error for --http3 if libcurl doesn't support - Closes #4480 + Closes #4785 -- socketpair: fix double-close in error case +- docs: mention CURL_MAX_INPUT_LENGTH restrictions - Follow-up to bc2dbef0afc08 - -- gskit: use the generic Curl_socketpair - -- asyn-thread: make use of Curl_socketpair() where available - -- socketpair: an implemention for Windows and more + ... for curl_easy_setopt() and curl_url_set(). - Curl_socketpair() is designed to be used and work everywhere if there's - no native version or the native version isn't good enough. + [skip ci] - Closes #4466 - -- RELEASE-NOTES: synced + Closes #4783 -- connect: return CURLE_OPERATION_TIMEDOUT for errno == ETIMEDOUT +- curl: properly free mimepost data - Previosly all connect() failures would return CURLE_COULDNT_CONNECT, no - matter what errno said. + ... as it could otherwise leak memory when a transfer failed. - This makes for example --retry work on these transfer failures. + Added test 1293 to verify. - Reported-by: Nathaniel J. Smith - Fixes #4461 - Clsoes #4462 - -- cirrus: switch off blackhole status on the freebsd CI machines + Reported-by: Brian Carpenter + Fixes #4781 + Closes #4782 -- tests: use port 2 instead of 60000 for a safer non-listening port +- curl: cleanup multi handle on failure - ... when the tests want "connection refused". + ... to fix memory leak in error path. + + Fixes #4772 + Closes #4780 + Reported-by: Brian Carpenter -- KNOWN_BUGS: IDN tests failing on Windows +Marcel Raad (3 Jan 2020) +- lib: fix compiler warnings with `CURL_DISABLE_VERBOSE_STRINGS` - Closes #3747 + Closes https://github.com/curl/curl/pull/4775 -Dan Fandrich (9 Oct 2019) -- cirrus: Increase the git clone depth. +Daniel Stenberg (3 Jan 2020) +- COPYING: it's 2020! - If more commits are submitted to master between the time of triggering - the first Cirrus build and the time the final build gets started, the - desired commit is no longer at HEAD and the build will error out. [skip ci] -Daniel Stenberg (9 Oct 2019) -- docs: make sure the --no-progress-meter docs file is in dist too +Jay Satiro (3 Jan 2020) +- [Marc Aldorasi brought this change] -- docs: document it as --no-progress-meter instead of the reverse + tests: Fix bounce requests with truncated writes - Follow-up to 93373a960c3bb4 + Prior to this change the swsbounce check in service_connection could + fail because prevtestno and prevpartno were not set, which would cause + the wrong response data to be sent to some tests and cause them to fail. - Reported-by: infinnovation-dev on github - Fixes #4474 - Closes #4475 + Ref: https://github.com/curl/curl/pull/4717#issuecomment-570240785 -Dan Fandrich (9 Oct 2019) -- cirrus: Switch the FreeBSD 11.x build to 11.3 and add a 13.0 build. +Marcel Raad (31 Dec 2019) +- tool: make a few char pointers point to const char instead - Also, select the images using image_family to get the latest snapshots - automatically. - [skip ci] + These are read-only. + + Closes https://github.com/curl/curl/pull/4771 -Daniel Stenberg (8 Oct 2019) -- curl: --no-progress-meter +Jay Satiro (31 Dec 2019) +- tests: Change NTLM tests to require SSL - New option that allows a user to ONLY switch off curl's progress meter - and leave everything else in "talkative" mode. + Prior to this change tests that required NTLM feature did not require + SSL feature. - Reported-by: Piotr Komborski - Fixes #4422 - Closes #4470 - -- TODO: Consult %APPDATA% also for .netrc + There are pending changes to cmake builds that will allow enabling NTLM + in non-SSL builds in Windows. In that case the NTLM auth strings created + are different from what is expected by the NTLM tests and they fail: - Closes #4016 - -- CURLOPT_TIMEOUT.3: remove the mention of "minutes" + "The issue with NTLM is that previous non-SSL builds would not enable + NTLM and so the NTLM tests would be skipped." - ... just say that limiting operations risk aborting otherwise fine - working transfers. If that means seconds, minutes or hours, we leave to - the user. + Assisted-by: marc-groundctl@users.noreply.github.com - Reported-by: Martin Gartner - Closes #4469 + Ref: https://github.com/curl/curl/pull/4717#issuecomment-566218729 + + Closes https://github.com/curl/curl/pull/4768 -- [Andrei Valeriu BICA brought this change] +- [Michael Forney brought this change] - docs: added multi-event.c example + bearssl: Improve I/O handling - Similar to multi-uv.c but using libevent 2. This is a simpler libevent - integration example then hiperfifo.c. + Factor out common I/O loop as bearssl_run_until, which reads/writes TLS + records until the desired engine state is reached. This is now used for + the handshake, read, write, and close. - Closes #4471 - -Jay Satiro (5 Oct 2019) -- [Nicolas brought this change] - - ldap: fix OOM error on missing query string + Match OpenSSL SSL_write behavior, and don't return the number of bytes + written until the corresponding records have been completely flushed + across the socket. This involves keeping track of the length of data + buffered into the TLS engine, and assumes that when CURLE_AGAIN is + returned, the write function will be called again with the same data + and length arguments. This is the same requirement of SSL_write. - - Allow missing queries, don't return NO_MEMORY error in such a case. + Handle TLS close notify as EOF when reading by returning 0. - It is acceptable for there to be no specified query string, for example: + Closes https://github.com/curl/curl/pull/4748 + +- travis: Fix error detection - curl ldap://ldap.forumsys.com + - Stop using inline shell scripts for before_script and script sections. - A regression bug in 1b443a7 caused this issue. + Prior to this change Travis could ignore errors from commands in inline + scripts. I don't understand how or why it happens. This is a workaround. - This is a partial fix for #4261. + Assisted-by: Simon Warta - Bug: https://github.com/curl/curl/issues/4261#issuecomment-525543077 - Reported-by: Jojojov@users.noreply.github.com - Analyzed-by: Samuel Surtees + Ref: https://github.com/travis-ci/travis-ci/issues/1066 - Closes https://github.com/curl/curl/pull/4467 - -- [Paul B. Omta brought this change] + Fixes https://github.com/curl/curl/issues/3730 + Closes https://github.com/curl/curl/pull/3755 - build: Remove unused HAVE_LIBSSL and HAVE_LIBCRYPTO defines +- tool_operate: fix mem leak when failed config parse - Closes https://github.com/curl/curl/pull/4460 - -Daniel Stenberg (5 Oct 2019) -- RELEASE-NOTES: synced + Found by fuzzing the config file. + + Reported-by: Geeknik Labs + + Fixes https://github.com/curl/curl/issues/4767 -- [Stian Soiland-Reyes brought this change] +- [Xiang Xiao brought this change] - curl: ensure HTTP 429 triggers --retry - - This completes #3794. + lib: remove erroneous +x file permission on some c files - Also make sure the new tests from #4195 are enabled + Modified by commit eb9a604 accidentally. - Closes #4465 + Closes https://github.com/curl/curl/pull/4756 -Marcel Raad (4 Oct 2019) -- [apique brought this change] +- [Xiang Xiao brought this change] - winbuild: add ENABLE_UNICODE option + lib: fix warnings found when porting to NuttX - Fixes https://github.com/curl/curl/issues/4308 - Closes https://github.com/curl/curl/pull/4309 - -Daniel Stenberg (4 Oct 2019) -- ngtcp2: adapt to API change + - Undefine DEBUGASSERT in curl_setup_once.h in case it was already + defined as a system macro. - Closes #4457 - -- cookies: change argument type for Curl_flush_cookies + - Don't compile write32_le in curl_endian unless + CURL_SIZEOF_CURL_OFF_T > 4, since it's only used by Curl_write64_le. - The second argument is really a 'bool' so use that and pass in TRUE/FALSE - to make it clear. + - Include in socketpair.c. - Closes #4455 + Closes https://github.com/curl/curl/pull/4756 -- http2: move state-init from creation to pre-transfer - - To make sure that the HTTP/2 state is initialized correctly for - duplicated handles. It would otherwise easily generate "spurious" - PRIORITY frames to get sent over HTTP/2 connections when duplicated easy - handles were used. +- os400: Add missing CURLE error constants - Reported-by: Daniel Silverstone - Fixes #4303 - Closes #4442 + Bug: https://github.com/curl/curl/pull/4754#issuecomment-569126922 + Reported-by: Emil Engler -- urlapi: fix use-after-free bug +- CURLOPT_HEADERFUNCTION.3: Document that size is always 1 - Follow-up from 2c20109a9b5d04 + For compatibility with `fwrite`, the `CURLOPT_HEADERFUNCTION` callback + is passed two `size_t` parameters which, when multiplied, designate the + number of bytes of data passed in. In practice, CURL always sets the + first parameter (`size`) to 1. - Added test 663 to verify. + This practice is also enshrined in documentation and cannot be changed + in future. The documentation states that the default callback is + `fwrite`, which means `fwrite` must be a suitable function for this + purpose. However, the documentation also states that the callback must + return the number of *bytes* it successfully handled, whereas ISO C + `fwrite` returns the number of items (each of size `size`) which it + wrote. The only way these numbers can be equal is if `size` is 1. - Reported by OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/17954 + Since `size` is 1 and can never be changed in future anyway, document + that fact explicitly and let users rely on it. - Closes #4453 - -- [Paul Dreik brought this change] - - cookie: avoid harmless use after free + Reported-by: Frank Gevaerts + Commit-message-by: Christopher Head - This fix removes a use after free which can be triggered by - the internal cookie fuzzer, but otherwise is probably - impossible to trigger from an ordinary application. + Ref: https://github.com/curl/curl/pull/2787 - The following program reproduces it: + Fixes https://github.com/curl/curl/issues/4758 + +- examples/postinmemory.c: Call curl_global_cleanup always - curl_global_init(CURL_GLOBAL_DEFAULT); - CURL* handle=curl_easy_init(); - CookieInfo* info=Curl_cookie_init(handle,NULL,NULL,false); - curl_easy_setopt(handle, CURLOPT_COOKIEJAR, "/dev/null"); - Curl_flush_cookies(handle, true); - Curl_cookie_cleanup(info); - curl_easy_cleanup(handle); - curl_global_cleanup(); + Prior to this change curl_global_cleanup was not called if + curl_easy_init failed. - This was found through fuzzing. + Reported-by: kouzhudong@users.noreply.github.com - Closes #4454 - -- [Denis Chaplygin brought this change] + Fixes https://github.com/curl/curl/issues/4751 - docs: add note on failed handles not being counted by curl_multi_perform +Daniel Stenberg (21 Dec 2019) +- url2file.c: fix copyright year - Closes #4446 - -- CURLMOPT_MAX_CONCURRENT_STREAMS.3: fix SEE ALSO typo + Follow-up to 525787269599b5 -- [Niall O'Reilly brought this change] +- [Rickard Hallerbäck brought this change] - ESNI: initial build/setup + examples/url2file.c: corrected a comment - Closes #4011 + The comment was confusing and suggested that setting CURLOPT_NOPROGRESS + to 0L would both enable and disable debug output at the same time, like + a Schrödinger's cat of CURLOPTs. + + Closes #4745 + +- HISTORY: OSS-Fuzz started fuzzing libcurl in 2017 - RELEASE-NOTES: synced -- redirect: when following redirects to an absolute URL, URL encode it +Jay Satiro (20 Dec 2019) +- ngtcp2: Support the latest update key callback type - ... to make it handle for example (RFC violating) embeded spaces. + - Remove our cb_update_key in favor of ngtcp2's new + ngtcp2_crypto_update_key_cb which does the same thing. - Reported-by: momala454 on github - Fixes #4445 - Closes #4447 - -- urlapi: fix URL encoding when setting a full URL - -- tool_operate: rename functions to make more sense - -- curl: create easy handles on-demand and not ahead of time + Several days ago the ngtcp2_update_key callback function prototype was + changed in ngtcp2/ngtcp2@42ce09c. Though it would be possible to + fix up our cb_update_key for that change they also added + ngtcp2_crypto_update_key_cb which does the same thing so we'll use that + instead. - This should again enable crazy-large download ranges of the style - [1-10000000] that otherwise easily ran out of memory starting in 7.66.0 - when this new handle allocating scheme was introduced. + Ref: https://github.com/ngtcp2/ngtcp2/commit/42ce09c - Reported-by: Peter Sumatra - Fixes #4393 - Closes #4438 - -- [Kunal Ekawde brought this change] + Closes https://github.com/curl/curl/pull/4735 - CURLMOPT_MAX_CONCURRENT_STREAMS: new setopt +Daniel Stenberg (19 Dec 2019) +- sws: search for "Testno:" header uncondtionally if no testno - Closes #4410 - -- chunked-encoding: stop hiding the CURLE_BAD_CONTENT_ENCODING error + Even if the initial request line wasn't found. With the fix to 1455, the + test number is now detected correctly. - Unknown content-encoding would get returned as CURLE_WRITE_ERROR if the - response is chunked-encoded. + (Problem found when running tests in random order.) - Reported-by: Ilya Kosarev - Fixes #4310 - Closes #4449 + Closes #4744 -Marcel Raad (1 Oct 2019) -- checksrc: fix uninitialized variable warning +- tests: set LC_ALL in more tests - The loop doesn't need to be executed without a file argument. + Follow-up to 23208e330ac0c21 - Closes https://github.com/curl/curl/pull/4444 + Closes #4743 -- urlapi: fix unused variable warning +- test165: set LC_ALL=en_US.UTF-8 too - `dest` is only used with `ENABLE_IPV6`. + On my current Debian Unstable with libidn2 2.2.0, I get an error if + LC_ALL is set to blank. Then curl errors out with: - Closes https://github.com/curl/curl/pull/4444 - -- lib: silence conversion warnings + curl: (3) Failed to convert www.åäö.se to ACE; could not convert string to UTF-8 - Closes https://github.com/curl/curl/pull/4444 + Closes #4738 -- AppVeyor: add 32-bit MinGW-w64 build - - With WinSSL and testing enabled so that it would have detected most of - the warnings fixed in [0] and [1]. +- curl.h: add two defines for the "pre ISO C" case - [0] https://github.com/curl/curl/pull/4398 - [1] https://github.com/curl/curl/pull/4415 + Without this fix, this caused a compilation failure on AIX with IBM xlc + 13.1.3 compiler. - Closes https://github.com/curl/curl/pull/4433 + Reported-by: Ram Krushna Mishra + Fixes #4739 + Closes #4740 -- AppVeyor: remove MSYS2_ARG_CONV_EXCL for winbuild +- create_conn: prefer multiplexing to using new connections - It's only used for MSYS2 with MinGW. + ... as it would previously prefer new connections rather than + multiplexing in most conditions! The (now removed) code was a leftover + from the Pipelining code that was translated wrongly into a + multiplex-only world. - Closes - -Daniel Stenberg (30 Sep 2019) -- [Emil Engler brought this change] + Reported-by: Kunal Ekawde + Bug: https://curl.haxx.se/mail/lib-2019-12/0060.html + Closes #4732 - git: add tests/server/disabled to .gitignore +- test1456: remove the use of a fixed local port - Closes #4441 - -- altsvc: accept quoted ma and persist values + Fixup the test to instead not compare the port number. It sometimes + caused problems like this: - As mandated by the spec. Test 1654 is extended to verify. + "curl: (45) bind failed with errno 98: Address already in use" - Closes #4443 - -- mailmap: a Lucas fix - -Alessandro Ghedini (29 Sep 2019) -- [Lucas Pardue brought this change] - - quiche: update HTTP/3 config creation to new API - -Daniel Stenberg (29 Sep 2019) -- BINDINGS: PureBasic, Net::Curl for perl and Nim - -- BINDINGS: Kapito is an Erlang library, basically a binding + Closes #4733 -- BINDINGS: added clj-curl +Jay Satiro (18 Dec 2019) +- CURLOPT_QUOTE.3: fix typos - Reported-by: Lucas Severo - -- [Jay Satiro brought this change] - - docs: disambiguate CURLUPART_HOST is for host name (ie no port) + Prior to this change the EXAMPLE in the QUOTE/PREQUOTE/POSTQUOTE man + pages would not compile because a variable name was incorrect. - Closes #4424 + Reported-by: Bylon2@users.noreply.github.com + + Fixes https://github.com/curl/curl/issues/4736 -- cookies: using a share with cookies shouldn't enable the cookie engine +- [Gisle Vanem brought this change] + + strerror: Fix compiler warning "empty expression" - The 'share object' only sets the storage area for cookies. The "cookie - engine" still needs to be enabled or activated using the normal cookie - options. + - Remove the final semi-colon in the SEC2TXT() macro definition. - This caused the curl command line tool to accidentally use cookies - without having been told to, since curl switched to using shared cookies - in 7.66.0. + Before: #define SEC2TXT(sec) case sec: txt = #sec; break; - Test 1166 verifies + After: #define SEC2TXT(sec) case sec: txt = #sec; break - Updated test 506 + Prior to this change SEC2TXT(foo); would generate break;; which caused + the empty expression warning. - Fixes #4429 - Closes #4434 + Ref: https://github.com/curl/curl/commit/5b22e1a#r36458547 -- setopt: handle ALTSVC set to NULL +Daniel Stenberg (18 Dec 2019) +- curl/parseconfig: use curl_free() to free memory allocated by libcurl + + Reported-by: bxac on github + Fixes #4730 + Closes #4731 -- RELEASE-NOTES: synced +- curl/parseconfig: fix mem-leak + + When looping, first trying '.curlrc' and then '_curlrc', the function + would not free the first string. + + Closes #4731 -- [grdowns brought this change] +- CURLOPT_URL.3: "curl supports SMB version 1 (only)" + + [skip ci] - INSTALL: add vcpkg installation instructions +- test1270: a basic -w redirect_url test - Closes #4435 + Closes #4728 -- [Zenju brought this change] +- HISTORY: the SMB(S) support landed in 2014 - FTP: add test for FTPFILE_NOCWD: Avoid redundant CWDs +- define: remove HAVE_ENGINE_LOAD_BUILTIN_ENGINES, not used anymore - Add libtest 661 + It is covered by USE_OPENSSL_ENGINE now. - Closes #4417 - -- [Zenju brought this change] - - FTP: url-decode path before evaluation + Reported-by: Gisle Vanem + Bug: https://github.com/curl/curl/commit/87b9337c8f76c21c57b204e88b68c6ecf3bd1ac0#commitcomment-36447951 - Closes #4428 + Closes #4725 -Marcel Raad (27 Sep 2019) -- tests: fix narrowing conversion warnings +- lib: remove ASSIGNWITHINCONDITION exceptions, use our code style - `timediff_t` is 64 bits wide also on 32-bit systems since - commit b1616dad8f0. + ... even for macros - Closes https://github.com/curl/curl/pull/4415 + Reviewed-by: Daniel Gustafsson + Reviewed-by: Jay Satiro + Reported-by: Jay Satiro + Fixes #4683 + Closes #4722 -Jay Satiro (27 Sep 2019) -- [julian brought this change] +- tests: make sure checksrc runs on header files too - vtls: Fix comment typo about macosx-version-min compiler flag +- Revert "checksrc: fix regexp for ASSIGNWITHINCONDITION" - Closes https://github.com/curl/curl/pull/4425 - -Daniel Stenberg (26 Sep 2019) -- [Yechiel Kalmenson brought this change] - - README: minor grammar fix + This reverts commit ba82673dac3e8d00a76aa5e3779a0cb80e7442af. - Closes #4431 + Bug: #4683 -- [Spezifant brought this change] +- KNOWN_BUGS: TLS session cache doesn't work with TFO + + [skip ci] + Closes #4301 - HTTP3: fix prefix parameter for ngtcp2 build +- KNOWN_BUGS: Connection information when using TCP Fast Open - Closes #4430 + Also point to #4296 for more details + Closes #4296 -- quiche: don't close connection at end of stream! +- KNOWN_BUGS: LDAP on Windows doesn't work + + Closes #4261 -- quiche: set 'drain' when returning without having drained the queues +- docs: TLS SRP doesn't work with TLS 1.3 + + Reported-by: sayrer on github + Closes #4262 + [skip ci] -- Revert "FTP: url-decode path before evaluation" +Dan Fandrich (16 Dec 2019) +- cirrus: Switch to the FreeBSD 12.1 point release & enable more tests. - This reverts commit 2f036a72d543e96128bd75cb0fedd88815fd42e2. + A few tests are now passing on FreeBSD, so no longer skip them. + [skip ci] -- HTTP3: merged and simplified the two 'running' sections +Daniel Stenberg (16 Dec 2019) +- azure: the macos cmake doesn't need to install cmake + + Error: cmake 3.15.5 is already installed + To upgrade to 3.16.1, run `brew upgrade cmake`. + + Closes #4723 -- HTTP3: show an --alt-svc using example too +Jay Satiro (15 Dec 2019) +- winbuild: Document CURL_STATICLIB requirement for static libcurl + + A static libcurl (ie winbuild mode=static) requires that the user define + CURL_STATICLIB when using it in their application. This is already + covered in the FAQ and INSTALL.md, but is a pretty important point so + now it's noted in the BUILD.WINDOWS.txt as well. + + Assisted-by: Michael Vittiglio + + Closes https://github.com/curl/curl/pull/4721 -- [Zenju brought this change] +Daniel Stenberg (15 Dec 2019) +- [Santino Keupp brought this change] - FTP: url-decode path before evaluation + libssh2: add support for ECDSA and ed25519 knownhost keys - Closes #4423 + ... if a new enough libssh2 version is present. + + Source: https://curl.haxx.se/mail/archive-2019-12/0023.html + Co-Authored-by: Daniel Stenberg + Closes #4714 -- openssl: use strerror on SSL_ERROR_SYSCALL +- lib1591: free memory properly on OOM, in the trailers callback - Instead of showing the somewhat nonsensical errno number, use strerror() - to provide a more relatable error message. + Detected by torture tests. - Closes #4411 + Closes #4720 -- HTTP3: update quic.aiortc.org + add link to server list +- runtests: --repeat=[num] to repeat tests - Reported-by: Jeremy Lainé + Closes #4715 -Jay Satiro (26 Sep 2019) -- url: don't set appconnect time for non-ssl/non-ssh connections +- RELEASE-NOTES: synced + +- azure: add a torture test on mac - Prior to this change non-ssl/non-ssh connections that were reused set - TIMER_APPCONNECT [1]. Arguably that was incorrect since no SSL/SSH - handshake took place. + Uses --shallow=25 to keep it small enough to get through in time. - [1]: TIMER_APPCONNECT is publicly known as CURLINFO_APPCONNECT_TIME in - libcurl and %{time_appconnect} in the curl tool. It is documented as - "the time until the SSL/SSH handshake is completed". + Closes #4712 + +- multi: free sockhash on OOM - Reported-by: Marcel Hernandez + This would otherwise leak memory in the error path. - Ref: https://github.com/curl/curl/issues/3760 + Detected by torture test 1540. - Closes https://github.com/curl/curl/pull/3773 + Closes #4713 -Daniel Stenberg (25 Sep 2019) -- ngtcp2: remove fprintf() calls +Marcel Raad (13 Dec 2019) +- tests: use DoH feature for DoH tests - - convert some of them to H3BUF() calls to infof() - - remove some of them completely - - made DEBUG_HTTP3 defined only if CURLDEBUG is set for now + Previously, http/2 was used instead. - Closes #4421 + Assisted-by: Jay Satiro + Closes https://github.com/curl/curl/pull/4692 -- [Jay Satiro brought this change] +- hostip: suppress compiler warning + + With `--disable-doh --disable-threaded-resolver`, the `dns` parameter + is not used. + + Closes https://github.com/curl/curl/pull/4692 - url: fix the NULL hostname compiler warning case +- tests: fix build with `CURL_DISABLE_DOH` - Closes #4403 + Closes https://github.com/curl/curl/pull/4692 -- [Jay Satiro brought this change] +Daniel Stenberg (13 Dec 2019) +- azure: add a torture test + + Skipping all FTP tests for speed reasons. + + Closes #4697 - travis: move the go install to linux-only +- azure: make the default build use --enable-debug --enable-werror + +- ntlm_wb: fix double-free in OOM - ... to repair the build again - Closes #4403 + Detected by torture testing test 1310 + + Closes #4710 -- altsvc: correct the #ifdef for the ngtcp2 backend +Dan Fandrich (13 Dec 2019) +- cirrus: Drop the FreeBSD 10.4 build + + Upstream support for 10.4 ended a year ago, and it looks like the image + is now gone, too. + [skip ci] -- altsvc: save h3 as h3-23 +Daniel Stenberg (13 Dec 2019) +- unit1620: fix bad free in OOM - Follow-up to d176a2c7e5 + Closes #4709 -- urlapi: question mark within fragment is still fragment +- unit1609: fix mem-leak in OOM - The parser would check for a query part before fragment, which caused it - to do wrong when the fragment contains a question mark. + Closes #4709 + +- unit1607: fix mem-leak in OOM - Extended test 1560 to verify. + Closes #4709 + +- lib1559: fix mem-leak in OOM - Reported-by: Alex Konev - Fixes #4412 - Closes #4413 + Closes #4709 -- [Alex Samorukov brought this change] +- lib1557: fix mem-leak in OOM + + Closes #4709 - HTTP3.md: move -p for mkdir, remove -j for make +- altsvc: make the save function ignore NULL filenames - - mkdir on OSX/Darwin requires `-p` argument before dir + It might happen in OOM situations. Detected bv torture tests. - - portabbly figuring out number of cores is an exercise for somewhere - else + Closes #4707 + +- curl: fix memory leak in OOM in etags logic - Closes #4407 + Detected by torture tests + + Closes #4706 -Patrick Monnerat (24 Sep 2019) -- os400: getpeername() and getsockname() return ebcdic AF_UNIX sockaddr, +- doh: make it behave when built without proxy support - As libcurl now uses these 2 system functions, wrappers are needed on os400 - to convert returned AF_UNIX sockaddrs to ascii. + Reported-by: Marcel Raad + Bug: https://github.com/curl/curl/pull/4692#issuecomment-564115734 - This is a follow-up to commit 7fb54ef. - See also #4037. - Closes #4214 + Closes #4704 -Jay Satiro (24 Sep 2019) -- [Lucas Pardue brought this change] +- curl: improved cleanup in upload error path + + Memory leak found by torture test 58 + + Closes #4705 - strcase: fix raw lowercasing the letter X +- mailmap: fix Andrew Ishchuk + +- travis: make torture use --shallow=40 - Casing mistake in Curl_raw_tolower 'X' wasn't lowercased as 'x' prior to - this change. + As a first step to enable it to run over a more diverse set of tests in + a reasonable time. + +- runtests: introduce --shallow to reduce huge torture tests - Follow-up to 0023fce which added the function several days ago. + When set, shallow mode limits runtests -t to make no more than NUM fails + per test case. If more are found, it will randomly discard entries until + the number is right. The random seed can also be set. - Ref: https://github.com/curl/curl/pull/4401#discussion_r327396546 + This is particularly useful when running MANY tests as then most torture + failures will already fail the same functions over and over and make the + total operation painfully tedious. - Closes https://github.com/curl/curl/pull/4408 + Closes #4699 -Daniel Stenberg (23 Sep 2019) -- http2: Expression 'stream->stream_id != - 1' is always true +- conncache: CONNECT_ONLY connections assumed always in-use - PVS-Studio warning - Fixes #4402 + This makes them never to be considered "the oldest" to be discarded when + reaching the connection cache limit. The reasoning here is that + CONNECT_ONLY is primarily used in combination with using the + connection's socket post connect and since that is used outside of + curl's knowledge we must assume that it is in use until explicitly + closed. + + Reported-by: Pavel Pavlov + Reported-by: Pavel Löbl + Fixes #4426 + Fixes #4369 + Closes #4696 -- http2: A value is being subtracted from the unsigned variable +- [Gisle Vanem brought this change] + + vtls: make BearSSL possible to set with CURL_SSL_BACKEND - PVS-Studio warning - Fixes #4402 + Ref: https://github.com/curl/curl/commit/9b879160df01e7ddbb4770904391d3b74114302b#commitcomment-36355622 + + Closes #4698 -- libssh: part of conditional expression is always true: !result +- RELEASE-NOTES: synced + +- travis: remove "coverage", make it "torture" - PVS-Studio warning - Fixed #4402 + The coveralls service and test coverage numbers are just too unreliable. + Removed badge from README.md as well. + + Fixes #4694 + Closes #4695 -- libssh: part of conditional expression is always true +- azure: add libssh2 and cmake macos builds - PVS-Studio warning - Fixes #4402 + Removed the macos libssh2 build from travis + + Closes #4686 + +- curl: use errorf() better + + Change series of error outputs to use errorf(). + + Only errors that are due to mistakes in command line option usage should + use helpf(), other types of errors in the tool should rather use + errorf(). + + Closes #4691 + +Jay Satiro (9 Dec 2019) +- [Marc Hoersken brought this change] + + tests: make it possible to set executable extensions + + This enables the use of Windows Subsystem for Linux (WSL) to run the + testsuite against Windows binaries while using Linux servers. + + This commit introduces the following environment variables: + - CURL_TEST_EXE_EXT: set the executable extension for all components + - CURL_TEST_EXE_EXT_TOOL: set it for the curl tool only + - CURL_TEST_EXE_EXT_SSH: set it for the SSH tools only + + Later testcurl.pl could be adjusted to make use of those variables. + - CURL_TEST_EXE_EXT_SRV: set it for the test servers only + + (This is one of several commits to support use of WSL for the tests.) + + Closes https://github.com/curl/curl/pull/3899 + +- [Marc Hoersken brought this change] + + tests: fix permissions of ssh keys in WSL + + Keys created on Windows Subsystem for Linux (WSL) require it for some + reason. + + (This is one of several commits to support use of WSL for the tests.) + + Ref: https://github.com/curl/curl/pull/3899 + +- [Marc Hoersken brought this change] + + tests: use \r\n for log messages in WSL + + Bash in Windows Subsystem for Linux (WSL) requires it for some reason. + + (This is one of several commits to support use of WSL for the tests.) + + Ref: https://github.com/curl/curl/pull/3899 + +- [Andrew Ishchuk brought this change] + + winbuild: Define CARES_STATICLIB when WITH_CARES=static + + When libcurl is built with MODE=static, c-ares is forced into static + linkage too. That doesn't happen when MODE=dll so linker would break + over undefined symbols. + + closes https://github.com/curl/curl/pull/4688 + +Daniel Stenberg (9 Dec 2019) +- conn: always set bits.close with connclose() + + Closes #4690 + +- cirrus: enable clang sanitizers on freebsd 13 + +- conncache: fix multi-thread use of shared connection cache + + It could accidentally let the connection get used by more than one + thread, leading to double-free and more. + + Reported-by: Christopher Reid + Fixes #4544 + Closes #4557 + +- azure: add a vanilla macos build + + Closes #4685 + +- curl: make the etag load logic work without fseek + + The fseek()s were unnecessary and caused Coverity warning CID 1456554 + + Closes #4681 + +- mailmap: Mohammad Hasbini + +- [Mohammad Hasbini brought this change] + + docs: fix some typos + + Closes #4680 + +- RELEASE-NOTES: synced + +Jay Satiro (5 Dec 2019) +- lib: fix some loose ends for recently added CURLSSLOPT_NO_PARTIALCHAIN + + Add support for CURLSSLOPT_NO_PARTIALCHAIN in CURLOPT_PROXY_SSL_OPTIONS + and OS400 package spec. + + Also I added the option to the NameValue list in the tool even though it + isn't exposed as a command-line option (...yet?). (NameValue stringizes + the option name for the curl cmd -> libcurl source generator) + + Follow-up to 564d88a which added CURLSSLOPT_NO_PARTIALCHAIN. + + Ref: https://github.com/curl/curl/pull/4655 + +- setopt: Fix ALPN / NPN user option when built without HTTP2 + + - Stop treating lack of HTTP2 as an unknown option error result for + CURLOPT_SSL_ENABLE_ALPN and CURLOPT_SSL_ENABLE_NPN. + + Prior to this change it was impossible to disable ALPN / NPN if libcurl + was built without HTTP2. Setting either option would result in + CURLE_UNKNOWN_OPTION and the respective internal option would not be + set. That was incorrect since ALPN and NPN are used independent of + HTTP2. + + Reported-by: Shailesh Kapse + + Fixes https://github.com/curl/curl/issues/4668 + Closes https://github.com/curl/curl/pull/4672 + +Daniel Stenberg (5 Dec 2019) +- etag: allow both --etag-compare and --etag-save in same cmdline + + Fixes #4669 + Closes #4678 + +Marcel Raad (5 Dec 2019) +- curl_setup: fix `CURLRES_IPV6` condition + + Move the definition of `CURLRES_IPV6` to before undefining + `HAVE_GETADDRINFO`. Regression from commit 67a08dca27a which caused + some tests to fail and others to be skipped with c-ares. + + Fixes https://github.com/curl/curl/issues/4673 + Closes https://github.com/curl/curl/pull/4677 + +Daniel Stenberg (5 Dec 2019) +- test342: make it return a 304 as the tag matches + +Peter Wu (4 Dec 2019) +- CMake: add support for building with the NSS vtls backend + + Options are cross-checked with configure.ac and acinclude.m4. + Tested on Arch Linux, untested on other platforms like Windows or macOS. + + Closes #4663 + Reviewed-by: Kamil Dudka + +Daniel Stenberg (4 Dec 2019) +- azure: add more builds + + ... removed two from travis (that now runs on azure instead) + + Closes #4671 + +- CURLOPT_VERBOSE.3: see also ERRORBUFFER + +- hostip4.c: bump copyright year range + +Marcel Raad (3 Dec 2019) +- configure: enable IPv6 support without `getaddrinfo` + + This makes it possible to recognize and connect to literal IPv6 + addresses when `getaddrinfo` is not available, which is already the + case for the CMake build. This affects e.g. classic MinGW because it + still targets Windows 2000 by default, where `getaddrinfo` is not + available, but general IPv6 support is. + + Instead of checking for `getaddrinfo`, check for `sockaddr_in6` as the + CMake build does. + + Closes https://github.com/curl/curl/pull/4662 + +- curl_setup: disable IPv6 resolver without `getaddrinfo` + + Also, use `CURLRES_IPV6` only for actual DNS resolution, not for IPv6 + address support. This makes it possible to connect to IPv6 literals by + setting `ENABLE_IPV6` even without `getaddrinfo` support. It also fixes + the CMake build when using the synchronous resolver without + `getaddrinfo` support. + + Closes https://github.com/curl/curl/pull/4662 + +Daniel Stenberg (3 Dec 2019) +- github action/azure pipeline: run 'make test-nonflaky' for tests + + To match travis and give more info on failures. + +- openssl: CURLSSLOPT_NO_PARTIALCHAIN can disable partial cert chains + + Closes #4655 + +- openssl: set X509_V_FLAG_PARTIAL_CHAIN + + Have intermediate certificates in the trust store be treated as + trust-anchors, in the same way as self-signed root CA certificates + are. This allows users to verify servers using the intermediate cert + only, instead of needing the whole chain. + + Other TLS backends already accept partial chains. + + Reported-by: Jeffrey Walton + Bug: https://curl.haxx.se/mail/lib-2019-11/0094.html + +- curl: show better error message when no homedir is found + + Reported-by: Vlastimil Ovčáčík + Fixes #4644 + Closes #4665 + +- OPENSOCKETFUNCTION.3: correct the purpose description + + Reported-by: Jeff Mears + Bug: https://curl.haxx.se/mail/lib-2019-12/0007.html + + Closes #4667 + +- [Peter Wu brought this change] + + travis: do not use OVERRIDE_CC or OVERRIDE_CXX if empty + + Fixes the macOS builds where OVERRIDE_CC and OVERRIDE_CXX are not set. + + Reported-by: Jay Satiro + Fixes #4659 + Closes #4661 + Closes #4664 + +- azure-pipelines: fix the test script + +- Azure Pipelines: initial CI setup + + [skip ci] + +- docs: add "added: 7.68.0" to the --etag-* docs + +- copyright: fix the year ranges for two files + + Follow-up to 9c1806ae + +Jay Satiro (1 Dec 2019) +- build: Disable Visual Studio warning "conditional expression is constant" + + - Disable warning C4127 "conditional expression is constant" globally + in curl_setup.h for when building with Microsoft's compiler. + + This mainly affects building with the Visual Studio project files found + in the projects dir. + + Prior to this change the cmake and winbuild build systems already + disabled 4127 globally for when building with Microsoft's compiler. + Also, 4127 was already disabled for all build systems in the limited + circumstance of the WHILE_FALSE macro which disabled the warning + specifically for while(0). This commit removes the WHILE_FALSE macro and + all other cruft in favor of disabling globally in curl_setup. + + Background: + + We have various macros that cause 0 or 1 to be evaluated, which would + cause warning C4127 in Visual Studio. For example this causes it: + + #define Curl_resolver_asynch() 1 + + Full behavior is not clearly defined and inconsistent across versions. + However it is documented that since VS 2015 Update 3 Microsoft has + addressed this somewhat but not entirely, not warning on while(true) for + example. + + Prior to this change some C4127 warnings occurred when I built with + Visual Studio using the generated projects in the projects dir. + + Closes https://github.com/curl/curl/pull/4658 + +- openssl: retrieve reported LibreSSL version at runtime + + - Retrieve LibreSSL runtime version when supported (>= 2.7.1). + + For earlier versions we continue to use the compile-time version. + + Ref: https://man.openbsd.org/OPENSSL_VERSION_NUMBER.3 + + Closes https://github.com/curl/curl/pull/2425 + +- strerror: Add Curl_winapi_strerror for Win API specific errors + + - In all code call Curl_winapi_strerror instead of Curl_strerror when + the error code is known to be from Windows GetLastError. + + Curl_strerror prefers CRT error codes (errno) over Windows API error + codes (GetLastError) when the two overlap. When we know the error code + is from GetLastError it is more accurate to prefer the Windows API error + messages. + + Reported-by: Richard Alcock + + Fixes https://github.com/curl/curl/issues/4550 + Closes https://github.com/curl/curl/pull/4581 + +Daniel Stenberg (2 Dec 2019) +- global_init: undo the "intialized" bump in case of failure + + ... so that failures in the global init function don't count as a + working init and it can then be called again. + + Reported-by: Paul Groke + Fixes #4636 + Closes #4653 + +- parsedate: offer a getdate_capped() alternative + + ... and use internally. This function will return TIME_T_MAX instead of + failure if the parsed data is found to be larger than what can be + represented. TIME_T_MAX being the largest value curl can represent. + + Reviewed-by: Daniel Gustafsson + Reported-by: JanB on github + Fixes #4152 + Closes #4651 + +- docs: add more references to curl_multi_poll + + Fixes #4643 + Closes #4652 + +- sha256: bump the copyright year range + + Follow-up from 66e21520f + +Daniel Gustafsson (28 Nov 2019) +- curl_setup_once: consistently use WHILE_FALSE in macros + + The WHILE_FALSE construction is used to avoid compiler warnings in + macro constructions. This fixes a few instances where it was not + used in order to keep the code consistent. + + Closes #4649 + Reviewed-by: Daniel Stenberg + +Daniel Stenberg (28 Nov 2019) +- [Steve Holme brought this change] + + http_ntlm: Remove duplicate NSS initialisation + + Given that this is performed by the NTLM code there is no need to + perform the initialisation in the HTTP layer. This also keeps the + initialisation the same as the SASL based protocols and also fixes a + possible compilation issue if both NSS and SSPI were to be used as + multiple SSL backends. + + Reviewed-by: Kamil Dudka + Closes #3935 + +Daniel Gustafsson (28 Nov 2019) +- checksrc: fix regexp for ASSIGNWITHINCONDITION + + The regexp looking for assignments within conditions was too greedy + and matched a too long string in the case of multiple conditionals + on the same line. This is basically only a problem in single line + macros, and the code which exemplified this was essentially: + + do { if((x) != NULL) { x = NULL; } } while(0) + + ..where the final parenthesis of while(0) matched the regexp, and + the legal assignment in the block triggered the warning. Fix by + making the regexp less greedy by matching for the tell-tale signs + of the if statement ending. + + Also remove the one occurrence where the warning was disabled due + to a construction like the above, where the warning didn't apply + when fixed. + + Closes #4647 + Reviewed-by: Daniel Stenberg + +Daniel Stenberg (28 Nov 2019) +- RELEASE-NOTES: synced + +- [Maros Priputen brought this change] + + curl: two new command line options for etags + + --etag-compare and --etag-save + + Suggested-by: Paul Hoffman + Fixes #4277 + Closes #4543 + +Daniel Gustafsson (28 Nov 2019) +- docs: fix typos + +Daniel Stenberg (28 Nov 2019) +- mailmap: Niall O'Reilly's name + +- [Niall O'Reilly brought this change] + + doh: use dedicated probe slots + + ... to easier allow additional DNS transactions. + + Closes #4629 + +- travis: build ngtcp2 with --enable-lib-only + + ... makes it skip the examples and other stuff we don't neeed. + + Closes #4646 + +- [David Benjamin brought this change] + + ngtcp2: fix thread-safety bug in error-handling + + ERR_error_string(NULL) should never be called. It places the error in a + global buffer, which is not thread-safe. Use ERR_error_string_n with a + local buffer instead. + + Closes #4645 + +- travis: export the CC/CXX variables when set + + Suggested-by: Peter Wu + Fixes #4637 + Closes #4640 + +Marcel Raad (26 Nov 2019) +- dist: add error-codes.pl + + Follow-up to commit 74f441c6d31. + This should fix test 1175 when run via the daily source tarballs. + + Closes https://github.com/curl/curl/pull/4638 + +Daniel Stenberg (26 Nov 2019) +- [John Schroeder brought this change] -- libssh: The expression is excessive or contains a misprint + curl: fix --upload-file . hangs if delay in STDIN - PVS-Studio warning - Fixes #4402 - -- quiche: The expression must be surrounded by parentheses + Attempt to unpause a busy read in the CURLOPT_XFERINFOFUNCTION. - PVS-Studio warning - Fixes #4402 - -- vauth: The parameter 'status' must be surrounded by parentheses + When uploading from stdin in non-blocking mode, a delay in reading + the stream (EAGAIN) causes curl to pause sending data + (CURL_READFUNC_PAUSE). Prior to this change, a busy read was + detected and unpaused only in the CURLOPT_WRITEFUNCTION handler. + This change performs the same busy read handling in a + CURLOPT_XFERINFOFUNCTION handler. - PVS-Studio warning - Fixes #4402 + Fixes #2051 + Closes #4599 + Reported-by: bdry on github -- [Paul Dreik brought this change] +- [John Schroeder brought this change] - doh: allow only http and https in debug mode + XFERINFOFUNCTION: support CURL_PROGRESSFUNC_CONTINUE - Otherwise curl may be told to use for instance pop3 to - communicate with the doh server, which most likely - is not what you want. + (also for PROGRESSFUNCTION) - Found through fuzzing. + By returning this value from the callback, the internal progress + function call is still called afterward. - Closes #4406 + Closes #4599 -- [Paul Dreik brought this change] +- [Michael Forney brought this change] - doh: return early if there is no time left + TLS: add BearSSL vtls implementation - Closes #4406 - -- [Barry Pollard brought this change] + Closes #4597 - http: lowercase headernames for HTTP/2 and HTTP/3 +- curl_multi_wakeup.3: add example and AVAILABILITY - Closes #4401 - Fixes #4400 + Reviewed-by: Gergely Nagy + Closes #4635 -Marcel Raad (23 Sep 2019) -- vtls: fix narrowing conversion warnings +- [Gergely Nagy brought this change] + + multi: add curl_multi_wakeup() - Curl_timeleft returns `timediff_t`, which is 64 bits wide also on - 32-bit systems since commit b1616dad8f0. + This commit adds curl_multi_wakeup() which was previously in the TODO + list under the curl_multi_unblock name. - Closes https://github.com/curl/curl/pull/4398 + On some platforms and with some configurations this feature might not be + available or can fail, in these cases a new error code + (CURLM_WAKEUP_FAILURE) is returned from curl_multi_wakeup(). + + Fixes #4418 + Closes #4608 -Daniel Stenberg (23 Sep 2019) -- [Joel Depooter brought this change] +Jay Satiro (24 Nov 2019) +- [Xiaoyin Liu brought this change] - winbuild: Add manifest to curl.exe for proper OS version detection + schannel: fix --tls-max for when min is --tlsv1 or default - This is a small fix to commit ebd213270a017a6830928ee2e1f4a9cabc799898 - in pull request #1221. That commit added the CURL_EMBED_MANIFEST flag to - CURL_RC_FLAGS. However, later in the file CURL_RC_FLAGS is - overwritten. The fix is to append values to CURL_RC_FLAGS instead of - overwriting + Prior to this change schannel ignored --tls-max (CURL_SSLVERSION_MAX_ + macros) when --tlsv1 (CURL_SSLVERSION_TLSv1) or default TLS + (CURL_SSLVERSION_DEFAULT), using a max of TLS 1.2 always. - Closes #4399 + Closes https://github.com/curl/curl/pull/4633 -- RELEASE-NOTES: synced +- checksrc.bat: Add a check for vquic and vssh directories + + Ref: https://github.com/curl/curl/pull/4607 -Marcel Raad (22 Sep 2019) -- openssl: fix compiler warning with LibreSSL +- projects: Fix Visual Studio projects SSH builds - It was already fixed for BoringSSL in commit a0f8fccb1e0. - LibreSSL has had the second argument to SSL_CTX_set_min_proto_version - as uint16_t ever since the function was added in [0]. + - Generate VQUIC and VSSH filenames in Visual Studio project files. - [0] https://github.com/libressl-portable/openbsd/commit/56f107201baefb5533486d665a58d8f57fd3aeda + Prior to this change generated Visual Studio project configurations that + enabled SSH did not build properly. Broken since SSH files were moved to + lib/vssh 3 months ago in 5b2d703. - Closes https://github.com/curl/curl/pull/4397 + Fixes https://github.com/curl/curl/issues/4492 + Fixes https://github.com/curl/curl/issues/4630 + Closes https://github.com/curl/curl/pull/4607 -Daniel Stenberg (22 Sep 2019) -- curl: exit the create_transfers loop on errors - - When looping around the ranges and given URLs to create transfers, all - errors should exit the loop and return. Previously it would keep - looping. - - Reported-by: SumatraPeter on github - Bug: #4393 - Closes #4396 +Daniel Stenberg (23 Nov 2019) +- RELEASE-NOTES: synced -Jay Satiro (21 Sep 2019) -- socks: Fix destination host shown on SOCKS5 error +Jay Satiro (22 Nov 2019) +- openssl: Revert to less sensitivity for SYSCALL errors - Prior to this change when a server returned a socks5 connect error then - curl would parse the destination address:port from that data and show it - to the user as the destination: + - Disable the extra sensitivity except in debug builds (--enable-debug). - curld -v --socks5 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to IPv4 172.217.12.206 (locally resolved) - * Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) - curl: (7) Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) + - Improve SYSCALL error message logic in ossl_send and ossl_recv so that + "No error" / "Success" socket error text isn't shown on SYSCALL error. - That's incorrect because the address:port included in the connect error - is actually a bind address:port (typically unused) and not the - destination address:port. This fix changes curl to show the destination - information that curl sent to the server instead: + Prior to this change 0ab38f5 (precedes 7.67.0) increased the sensitivity + of OpenSSL's SSL_ERROR_SYSCALL error so that abrupt server closures were + also considered errors. For example, a server that does not send a known + protocol termination point (eg HTTP content length or chunked encoding) + _and_ does not send a TLS termination point (close_notify alert) would + cause an error if it closed the connection. - curld -v --socks5 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to IPv4 172.217.7.14:99 (locally resolved) - * Can't complete SOCKS5 connection to 172.217.7.14:99. (1) - curl: (7) Can't complete SOCKS5 connection to 172.217.7.14:99. (1) + To be clear that behavior made it into release build 7.67.0 + unintentionally. Several users have reported it as an issue. - curld -v --socks5-hostname 10.0.3.1:1080 http://google.com:99 - * SOCKS5 communication to google.com:99 - * SOCKS5 connect to google.com:99 (remotely resolved) - * Can't complete SOCKS5 connection to google.com:99. (1) - curl: (7) Can't complete SOCKS5 connection to google.com:99. (1) + Ultimately the idea is a good one, since it can help prevent against a + truncation attack. Other SSL backends may already behave similarly (such + as Windows native OS SSL Schannel). However much more of our user base + is using OpenSSL and there is a mass of legacy users in that space, so I + think that behavior should be partially reverted and then rolled out + slowly. - Ref: https://tools.ietf.org/html/rfc1928#section-6 + This commit changes the behavior so that the increased sensitivity is + disabled in all curl builds except curl debug builds (DEBUGBUILD). If + after a period of time there are no major issues then it can be enabled + in dev and release builds with the newest OpenSSL (1.1.1+), since users + using the newest OpenSSL are the least likely to have legacy problems. - Closes https://github.com/curl/curl/pull/4394 - -Daniel Stenberg (21 Sep 2019) -- travis: enable ngtcp2 h3-23 builds - -- altsvc: both backends run h3-23 now + Bug: https://github.com/curl/curl/issues/4409#issuecomment-555955794 + Reported-by: Bjoern Franke - Closes #4395 + Fixes https://github.com/curl/curl/issues/4624 + Closes https://github.com/curl/curl/pull/4623 -- http: fix warning on conversion from int to bit - - Follow-up from 03ebe66d70 +- [Daniel Stenberg brought this change] -- urldata: use 'bool' for the bit type on MSVC compilers + openssl: improve error message for SYSCALL during connect - Closes #4387 - Fixes #4379 - -- appveyor: upgrade VS2017 to VS2019 + Reported-by: Paulo Roberto Tomasi + Bug: https://curl.haxx.se/mail/archive-2019-11/0005.html - Closes #4383 - -- [Zenju brought this change] + Closes https://github.com/curl/curl/pull/4593 - FTP: FTPFILE_NOCWD: avoid redundant CWDs +Daniel Stenberg (22 Nov 2019) +- test1175: verify symbols-in-versions and libcurl-errors.3 in sync - Closes #4382 + Closes #4628 -- cookie: pass in the correct cookie amount to qsort() +- include: make CURLE_HTTP3 use a new error code - As the loop discards cookies without domain set. This bug would lead to - qsort() trying to sort uninitialized pointers. We have however not found - it a security problem. + To avoid potential issues with error code reuse. - Reported-by: Paul Dreik - Closes #4386 + Reported-by: Christoph M. Becker + Assisted-by: Dan Fandrich + Fixes #4601 + Closes #4627 -- [Paul Dreik brought this change] +- bump: next release will be 7.68.0 - urlapi: avoid index underflow for short ipv6 hostnames +- curl: add --parallel-immediate - If the input hostname is "[", hlen will underflow to max of size_t when - it is subtracted with 2. + Starting with this change when doing parallel transfers, without this + option set, curl will prefer to create new transfers multiplexed on an + existing connection rather than creating a brand new one. - hostname[hlen] will then cause a warning by ubsanitizer: + --parallel-immediate can be set to tell curl to prefer to use new + connections rather than to wait and try to multiplex. - runtime error: addition of unsigned offset to 0x overflowed to - 0x + libcurl-wise, this means that curl will set CURLOPT_PIPEWAIT by default + on parallel transfers. - I think that in practice, the generated code will work, and the output - of hostname[hlen] will be the first character "[". + Suggested-by: Tom van der Woerdt + Closes #4500 + +Daniel Gustafsson (20 Nov 2019) +- [Victor Magierski brought this change] + + docs: fix typos - This can be demonstrated by the following program (tested in both clang - and gcc, with -O3) + Change 'experiemental' to 'experimental'. - int main() { - char* hostname=strdup("["); - size_t hlen = strlen(hostname); + Closes #4618 + Reviewed-by: Daniel Gustafsson + +Jay Satiro (18 Nov 2019) +- projects: Fix Visual Studio wolfSSL configurations - hlen-=2; - hostname++; - printf("character is %d\n",+hostname[hlen]); - free(hostname-1); - } + - s/USE_CYASSL/USE_WOLFSSL/ - I found this through fuzzing, and even if it seems harmless, the proper - thing is to return early with an error. + - Remove old compatibility macros. - Closes #4389 + Follow-up to 1c6c59a from several months ago when CyaSSL named symbols + were renamed to wolfSSL. The wolfSSL library was formerly named CyaSSL + and we kept using their old name for compatibility reasons, until + earlier this year. + +Daniel Stenberg (18 Nov 2019) +- RELEASE-NOTES: synced -- [Tatsuhiro Tsujikawa brought this change] +- [Javier Blazquez brought this change] - ngtcp2: compile with latest ngtcp2 + nghttp3 draft-23 + ngtcp2: use overflow buffer for extra HTTP/3 data - Closes #4392 - -- THANKS-filter: deal with my typos 'Jat' => 'Jay' + Fixes #4525 + Closes #4603 -- travis: use go master +- altsvc: bump to h3-24 - ... as the boringssl builds needs a very recent version + ... as both ngtcp2 and quiche now support that in their master branches - Co-authored-by: Jat Satiro - Closes #4361 + Closes #4604 -- tool_operate: removed unused variable 'done' +- ngtcp2: free used resources on disconnect - Fixes warning detected by PVS-Studio - Fixes #4374 + Fixes #4614 + Closes #4615 -- tool_operate: Expression 'config->resume_from' is always true +- ngtcp2: handle key updates as ngtcp2 master branch tells us - Fixes warning detected by PVS-Studio - Fixes #4374 - -- tool_getparam: remove duplicate switch case + Reviewed-by: Tatsuhiro Tsujikawa - Fixes warning detected by PVS-Studio - Fixes #4374 + Fixes #4612 + Closes #4613 -- libssh2: part of conditional expression is always true: !result - - Fixes warning detected by PVS-Studio - Fixes #4374 +Jay Satiro (17 Nov 2019) +- [Gergely Nagy brought this change] -- urlapi: Expression 'storep' is always true + multi: Fix curl_multi_poll wait when extra_fds && !extra_nfds - Fixes warning detected by PVS-Studio - Fixes #4374 - -- urlapi: 'scheme' is always true + Prior to this change: - Fixes warning detected by PVS-Studio - Fixes #4374 - -- urlapi: part of conditional expression is always true: (relurl[0] == '/') + The check if an extra wait is necessary was based not on the + number of extra fds but on the pointer. - Fixes warning detected by PVS-Studio - Fixes #4374 - -- setopt: store CURLOPT_RTSP_SERVER_CSEQ correctly + If a non-null pointer was given in extra_fds, but extra_nfds + was zero, then the wait was skipped even though poll was not + called. - Fixes bug detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/4610 -- mime: make Curl_mime_duppart() assert if called without valid dst +- lib: Move lib/ssh.h -> lib/vssh/ssh.h - Fixes warning detected by PVS-Studio - Fixes #4374 - -- http_proxy: part of conditional expression is always true: !error + Follow-up to 5b2d703 which moved ssh source files to vssh. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes https://github.com/curl/curl/pull/4609 -- imap: merged two case-branches performing the same action - - Fixes warning detected by PVS-Studio - Fixes #4374 +Daniel Stenberg (16 Nov 2019) +- [Andreas Falkenhahn brought this change] -- multi: value '2L' is assigned to a boolean + INSTALL.md: provide Android build instructions - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4606 -- easy: part of conditional expression is always true: !result - - Fixes warning detected by PVS-Studio - Fixes #4374 +- [Niall O'Reilly brought this change] -- netrc: part of conditional expression is always true: !done + doh: improced both encoding and decoding - Fixes warning detected by PVS-Studio - Fixes #4374 - -- version: Expression 'left > 1' is always true + Improved estimation of expected_len and updated related comments; + increased strictness of QNAME-encoding, adding error detection for empty + labels and names longer than the overall limit; avoided treating DNAME + as unexpected; - Fixes warning detected by PVS-Studio - Fixes #4374 - -- url: remove dead code + updated unit test 1655 with more thorough set of proofs and tests - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4598 -- url: part of expression is always true: (bundle->multiuse == 0) +- ngtcp2: increase QUIC window size when data is consumed - Fixes warning detected by PVS-Studio - Fixes #4374 + Assisted-by: Javier Blazquez + Ref #4525 (partial fix) + Closes #4600 -- ftp: the conditional expression is always true +- [Melissa Mears brought this change] + + config-win32: cpu-machine-OS for Windows on ARM - ... both !result and (ftp->transfer != FTPTRANSFER_BODY)! + Define the OS macro properly for Windows on ARM builds. Also, we might + as well add the GCC-style IA-64 macro. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4590 -- ftp: Expression 'ftpc->wait_data_conn' is always false +- examples: add multi-poll.c - Fixes warning detected by PVS-Studio - Fixes #4374 - -- ftp: Expression 'ftpc->wait_data_conn' is always true + Show how curl_multi_poll() makes it even easier to use the multi + interface. - Fixes warning detected by PVS-Studio - Fixes #4374 + Closes #4596 -- ftp: part of conditional expression is always true: !result +- multi_poll: avoid busy-loop when called without easy handles attached - Fixes warning detected by PVS-Studio - Fixes #4374 + Fixes #4594 + Closes #4595 + Reported-by: 3dyd on github -- http: fix Expression 'http->postdata' is always false +- curl: fix -T globbing - Fixes warning detected by PVS-Studio - Fixes #4374 - Reported-by: Valerii Zapodovnikov - -- [Niall O'Reilly brought this change] - - doh: avoid truncating DNS QTYPE to lower octet + Regression from e59371a4936f8 (7.67.0) - Closes #4381 - -- [Jens Finkhaeuser brought this change] - - urlapi: CURLU_NO_AUTHORITY allows empty authority/host part + Added test 490, 491 and 492 to verify the functionality. - CURLU_NO_AUTHORITY is intended for use with unknown schemes (i.e. not - "file:///") to override cURL's default demand that an authority exists. + Reported-by: Kamil Dudka + Reported-by: Anderson Sasaki - Closes #4349 - -- version: next release will be 7.67.0 + Fixes #4588 + Closes #4591 -- RELEASE-NOTES: synced +- HISTORY: added cmake, HTTP/3 and parallel downloads with curl -- url: only reuse TLS connections with matching pinning - - If the requests have different CURLOPT_PINNEDPUBLICKEY strings set, the - connection should not be reused. +- quiche: reject headers in the wrong order - Bug: https://curl.haxx.se/mail/lib-2019-09/0061.html - Reported-by: Sebastian Haglund + Pseudo header MUST come before regular headers or cause an error. - Closes #4347 + Reported-by: Cynthia Coan + Fixes #4571 + Closes #4584 -- README: add OSS-Fuzz badge [skip ci] +- openssl: prevent recursive function calls from ctx callbacks - Closes #4380 + Follow the pattern of many other callbacks. + + Ref: #4546 + Closes #4585 -Michael Kaufmann (18 Sep 2019) -- http: merge two "case" statements +- CURL-DISABLE: initial docs for the CURL_DISABLE_* defines + + The disable-scan script used in test 1165 is extended to also verify + that the docs cover all used defines and all defines offered by + configure. + + Reported-by: SLDiggie on github + Fixes #4545 + Closes #4587 -Daniel Stenberg (18 Sep 2019) -- [Zenju brought this change] +- remove_handle: clear expire timers after multi_done() + + Since 59041f0, a new timer might be set in multi_done() so the clearing + of the timers need to happen afterwards! + + Reported-by: Max Kellermann + Fixes #4575 + Closes #4583 - FTP: remove trailing slash from path for LIST/MLSD +Marcel Raad (10 Nov 2019) +- test1558: use double slash after file: - Closes #4348 + Classic MinGW / MSYS 1 doesn't support `MSYS2_ARG_CONV_EXCL`, so this + test unnecessarily failed when using `file:/` instead of `file:///`. + + Closes https://github.com/curl/curl/pull/4554 -- mime: when disabled, avoid C99 macro +Daniel Stenberg (10 Nov 2019) +- pause: avoid updating socket if done was already called - Closes #4368 + ... avoids unnecesary recursive risk when the transfer is already done. + + Reported-by: Richard Bowker + Fixes #4563 + Closes #4574 -- url: cleanup dangling DOH request headers too +Jay Satiro (9 Nov 2019) +- strerror: Fix an error looking up some Windows error strings - Follow-up to 9bc44ff64d9081 + - Use FORMAT_MESSAGE_IGNORE_INSERTS to ignore format specifiers in + Windows error strings. - Credit to OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/17269 + Since we are not in control of the error code we don't know what + information may be needed by the error string's format specifiers. - Closes #4372 + Prior to this change Windows API error strings which contain specifiers + (think specifiers like similar to printf specifiers) would not be shown. + The FormatMessage Windows API call which turns a Windows error code into + a string could fail and set error ERROR_INVALID_PARAMETER if that error + string contained a format specifier. FormatMessage expects a va_list for + the specifiers, unless inserts are ignored in which case no substitution + is attempted. + + Ref: https://devblogs.microsoft.com/oldnewthing/20071128-00/?p=24353 -- [Christoph M. Becker brought this change] +- [r-a-sattarov brought this change] - http2: relax verification of :authority in push promise requests + system.h: fix for MCST lcc compiler - If the :authority pseudo header field doesn't contain an explicit port, - we assume it is valid for the default port, instead of rejecting the - request for all ports. + Fixed build by MCST lcc compiler on MCST Elbrus 2000 architecture and do + some code cleanup. - Ref: https://curl.haxx.se/mail/lib-2019-09/0041.html + e2k (Elbrus 2000) - this is VLIW/EPIC architecture, like Intel Itanium + architecture. - Closes #4365 - -- doh: clean up dangling DOH handles and memory on easy close + Ref: https://en.wikipedia.org/wiki/Elbrus_2000 - If you set the same URL for target as for DoH (and it isn't a DoH - server), like "https://example.com" in both, the easy handles used for - the DoH requests could be left "dangling" and end up not getting freed. + Closes https://github.com/curl/curl/pull/4576 + +Daniel Stenberg (8 Nov 2019) +- TODO: curl_multi_unblock - Reported-by: Paul Dreik - Closes #4366 + Closes #4418 -- unit1655: make it C90 compliant +- TODO: Run web-platform-tests url tests - Unclear why this was not detected in the CI. + Closes #4477 + +- TODO: 1.4 alt-svc sharing - Follow-up to b7666027296a + Closes #4476 -- smb: check for full size message before reading message details +- test1560: require IPv6 for IPv6 aware URL parsing - To avoid reading of uninitialized data. + The URL parser function can't reject a bad IPv6 address properly when + curl was built without IPv6 support. - Assisted-by: Max Dymond - Bug: https://crbug.com/oss-fuzz/16907 - Closes #4363 + Reported-by: Marcel Raad + Fixes #4556 + Closes #4572 -- quiche: persist connection details +- checksrc: repair the copyrightyear check - ... like we do for other protocols at connect time. This makes "curl -I" - and other things work. + - Consider a modified file to be committed this year. - Reported-by: George Liu - Fixes #4358 - Closes #4360 - -- openssl: fix warning with boringssl and SSL_CTX_set_min_proto_version + - Make the travis CHECKSRC also do COPYRIGHTYEAR scan in examples and + includes - Follow-up to ffe34b7b59 - Closes #4359 - -- [Paul Dreik brought this change] - - doh: fix undefined behaviour and open up for gcc and clang optimization + - Ignore 0 parents when getting latest commit date of file. - The undefined behaviour is annoying when running fuzzing with - sanitizers. The codegen is the same, but the meaning is now not up for - dispute. See https://cppinsights.io/s/516a2ff4 + since in the CI we're dealing with a truncated repo of last 50 commits, + the file's most recent commit may not be available. when this happens + git log and rev-list show the initial commit (ie first commit not to be + truncated) but that's incorrect so ignore it. - By incrementing the pointer first, both gcc and clang recognize this as - a bswap and optimizes it to a single instruction. See - https://godbolt.org/z/994Zpx + Ref: https://github.com/curl/curl/pull/4547 - Closes #4350 - -- [Paul Dreik brought this change] + Closes https://github.com/curl/curl/pull/4549 + + Co-authored-by: Jay Satiro - doh: fix (harmless) buffer overrun +- copyrights: fix copyright year range - Added unit test case 1655 to verify. - Close #4352 + .. because checksrc's copyright year check stopped working. - the code correctly finds the flaws in the old code, - if one temporarily restores doh.c to the old version. + Ref: https://github.com/curl/curl/pull/4547 + + Closes https://github.com/curl/curl/pull/4549 -Alessandro Ghedini (15 Sep 2019) -- docs: remove trailing ':' from section names in CURLOPT_TRAILER* man +- RELEASE-NOTES: synced -- docs: fix typo in CURLOPT_HTTP_VERSION man +- curlver: bump to 7.67.1 -GitHub (14 Sep 2019) -- [Daniel Stenberg brought this change] +- mailmap: fixup Massimiliano Fantuzzi - CI: inintial github action job +- scripts/contributors: make committers get included too - First shot at a CI build on github actions + in addition to authors -Daniel Stenberg (13 Sep 2019) -- appveyor: add a winbuild - - Assisted-by: Marcel Raad - Assisted-by: Jay Satiro - - Closes #4324 +Jay Satiro (8 Nov 2019) +- [Massimiliano Fantuzzi brought this change] -- FTP: allow "rubbish" prepended to the SIZE response - - This is a protocol violation but apparently there are legacy proprietary - servers doing this. - - Added test 336 and 337 to verify. + configure: fix typo in help text - Reported-by: Philippe Marguinaud - Closes #4339 + Closes https://github.com/curl/curl/pull/4570 -- [Zenju brought this change] +Daniel Stenberg (7 Nov 2019) +- [Christian Schmitz brought this change] - FTP: skip CWD to entry dir when target is absolute + ntlm: USE_WIN32_CRYPTO check removed to get USE_NTLM2SESSION set - Closes #4332 + Closes #3704 -Kamil Dudka (13 Sep 2019) -- curl: fix memory leaked by parse_metalink() - - This commit fixes a regression introduced by curl-7_65_3-5-gb88940850. - Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind - and libmetalink enabled. +Jay Satiro (6 Nov 2019) +- [Wyatt O'Day brought this change] + + build: fix for CURL_DISABLE_DOH - Closes #4326 + Fixes https://github.com/curl/curl/issues/4565 + Closes https://github.com/curl/curl/pull/4566 -Daniel Stenberg (13 Sep 2019) -- parsedate: still provide the name arrays when disabled +- [Leonardo Taccari brought this change] + + configure: avoid unportable `==' test(1) operator - If FILE or FTP are enabled, since they also use them! + Closes https://github.com/curl/curl/pull/4567 + +Version 7.67.0 (5 Nov 2019) + +Daniel Stenberg (5 Nov 2019) +- RELEASE-NOTES: synced - Reported-by: Roland Hieber - Fixes #4325 - Closes #4343 + The 7.67.0 release -- [Gilles Vollant brought this change] +- THANKS: add new names from 7.67.0 - curl:file2string: load large files much faster +- configure: only say ipv6 enabled when the variable is set - ... by using a more efficient realloc scheme. + Previously it could say "IPv6: enabled" at the end of the configure run + but the define wasn't set because of a missing getaddrinfo(). - Bug: https://curl.haxx.se/mail/lib-2019-09/0045.html - Closes #4336 + Reported-by: Marcel Raad + Fixes #4555 + Closes #4560 -- openssl: close_notify on the FTP data connection doesn't mean closure - - For FTPS transfers, curl gets close_notify on the data connection - without that being a signal to close the control connection! +Marcel Raad (2 Nov 2019) +- certs/Server-localhost-lastSAN-sv: regenerate with sha256 - Regression since 3f5da4e59a556fc (7.65.0) + All other certificates were regenerated in commit ba782baac30, but + this one was missed. + Fixes test3001 on modern systems. - Reported-by: Zenju on github - Reviewed-by: Jay Satiro - Fixes #4329 - Closes #4340 + Closes https://github.com/curl/curl/pull/4551 -- [Jimmy Gaussen brought this change] +Daniel Stenberg (2 Nov 2019) +- [Vilhelm Prytz brought this change] - docs/HTTP3: fix `--with-ssl` ngtcp2 configure flag + copyrights: update all copyright notices to 2019 on files changed this year - Closes #4338 - -- RELEASE-NOTES: synced + Closes #4547 -- curlver: bump to 7.66.1 +- [Bastien Bouclet brought this change] -- [Zenju brought this change] + mbedtls: add error message for cert validity starting in the future + + Closes #4552 - setopt: make it easier to add new enum values +Jay Satiro (1 Nov 2019) +- schannel_verify: Fix concurrent openings of CA file - ... by using the *_LAST define names better. + - Open the CA file using FILE_SHARE_READ mode so that others can read + from it as well. - Closes #4321 - -- asyn-thread: s/AF_LOCAL/AF_UNIX for Solaris + Prior to this change our schannel code opened the CA file without + sharing which meant concurrent openings (eg an attempt from another + thread or process) would fail during the time it was open without + sharing, which in curl's case would cause error: + "schannel: failed to open CA file". - Reported-by: Dagobert Michelsen - Fixes #4328 - Closes #4333 - -- [Bernhard Walle brought this change] + Bug: https://curl.haxx.se/mail/lib-2019-10/0104.html + Reported-by: Richard Alcock - winbuild/MakefileBuild.vc: Add vssh +Daniel Stenberg (31 Oct 2019) +- gtls: make gnutls_bye() not wait for response on shutdown - Without that modification, the Windows build using the makefiles doesn't - work. + ... as it can make it wait there for a long time for no good purpose. - Signed-off-by: Bernhard Walle + Patched-by: Jay Satiro + Reported-by: Bylon2 on github + Adviced-by: Nikos Mavrogiannopoulos - Fixes #4322 - Closes #4323 + Fixes #4487 + Closes #4541 -Bernhard Walle (11 Sep 2019) -- winbuild/MakefileBuild.vc: Fix line endings - - The file had mixed line endings. - - Signed-off-by: Bernhard Walle +- [Michał Janiszewski brought this change] -Jay Satiro (11 Sep 2019) -- ldap: Stop using wide char version of ldapp_err2string + appveyor: publish artifacts on appveyor - Despite ldapp_err2string being documented by MS as returning a - PCHAR (char *), when UNICODE it is mapped to ldap_err2stringW and - returns PWCHAR (wchar_t *). + This allows obtaining upstream builds of curl directly from appveyor for + all the available configurations - We have lots of code that expects ldap_err2string to return char *, - most of it failf used like this: + Closes #4509 + +- url: make Curl_close() NULLify the pointer too - failf(data, "LDAP local: Some error: %s", ldap_err2string(rc)); + This is the common pattern used in the code and by a unified approach we + avoid mistakes. - Closes https://github.com/curl/curl/pull/4272 - -Version 7.66.0 (10 Sep 2019) + Closes #4534 -Daniel Stenberg (10 Sep 2019) -- RELEASE-NOTES: curl 7.66.0 +- [Trivikram Kamat brought this change] -- THANKS: from the 7.66.0 release + INSTALL: add missing space for configure commands + + Closes #4539 -- curl: make sure the parallel transfers do them all +- url: Curl_free_request_state() should also free doh handles - The logic could erroneously break the loop too early before all - transfers had been transferred. + ... or risk DoH memory leaks. - Reported-by: Tom van der Woerdt - Fixes #4316 - Closes #4317 - -- urlapi: one colon is enough for the strspn() input (typo) + Reported-by: Paul Dreik + Fixes #4463 + Closes #4527 -- urlapi: verify the IPv6 numerical address +- examples: remove the "this exact code has not been verified" - It needs to parse correctly. Otherwise it could be tricked into letting - through a-f using host names that libcurl would then resolve. Like - '[ab.be]'. + ... as really confuses the reader to not know what to believe! + +- [Trivikram Kamat brought this change] + + HTTP3: fix typo somehere1 > somewhere1 - Reported-by: Thomas Vegas - Closes #4315 + Closes #4535 -- [Clément Notin brought this change] +Jay Satiro (28 Oct 2019) +- [Javier Blazquez brought this change] - openssl: use SSL_CTX_set__proto_version() when available + HTTP3: fix invalid use of sendto for connected UDP socket - OpenSSL 1.1.0 adds SSL_CTX_set__proto_version() that we now use - when available. Existing code is preserved for older versions of - OpenSSL. + On macOS/BSD, trying to call sendto on a connected UDP socket fails + with a EISCONN error. Because the singleipconnect has already called + connect on the socket when we're trying to use it for QUIC transfers + we need to use plain send instead. - Closes #4304 - -- [Clément Notin brought this change] + Fixes #4529 + Closes https://github.com/curl/curl/pull/4533 - openssl: indent, re-organize and add comments +Daniel Stenberg (28 Oct 2019) +- RELEASE-NOTES: synced -- [migueljcrum brought this change] +- [Javier Blazquez brought this change] - sspi: fix memory leaks + HTTP3: fix Windows build - Closes #4299 - -- travis: disable ngtcp2 builds (again) - -- Curl_fillreadbuffer: avoid double-free trailer buf on error + The ngtcp2 QUIC backend was using the MSG_DONTWAIT flag for send/recv + in order to perform nonblocking operations. On Windows this flag does + not exist. Instead, the socket must be set to nonblocking mode via + ioctlsocket. - Reviewed-by: Jay Satiro - Reported-by: Thomas Vegas + This change sets the nonblocking flag on UDP sockets used for QUIC on + all platforms so the use of MSG_DONTWAIT is not needed. - Closes #4307 + Fixes #4531 + Closes #4532 -- tool_setopt: handle a libcurl build without netrc support +Marcel Raad (27 Oct 2019) +- appveyor: add --disable-proxy autotools build - Reported-by: codesniffer13 on github - Fixes #4302 - Closes #4305 - -- security:read_data fix bad realloc() + This would have caught issue #3926. - ... that could end up a double-free + Also make formatting more consistent. - CVE-2019-5481 - Bug: https://curl.haxx.se/docs/CVE-2019-5481.html - -- [Thomas Vegas brought this change] + Closes https://github.com/curl/curl/pull/4526 - tftp: Alloc maximum blksize, and use default unless OACK is received +Daniel Stenberg (25 Oct 2019) +- appveyor: make winbuilds with DEBUG=no/yes and VS 2015/2017 - Fixes potential buffer overflow from 'recvfrom()', should the server - return an OACK without blksize. + ... and invoke "curl -V" once done - Bug: https://curl.haxx.se/docs/CVE-2019-5482.html - CVE-2019-5482 + Co-Authored-By: Jay Satiro + + Closes #4523 -- [Thomas Vegas brought this change] +- [Francois Rivard brought this change] - tftp: return error when packet is too small for options + schannel: reverse the order of certinfo insertions + + Fixes #4518 + Closes #4519 -- KNOWN_BUGS/TODO: cleanup and remove outdated issues +Marcel Raad (24 Oct 2019) +- test1591: fix spelling of http feature + + The test never got run because the feature name is `http` in lowercase. + + Closes https://github.com/curl/curl/pull/4520 -- RELEASE-NOTES: synced +Daniel Stenberg (23 Oct 2019) +- [Michał Janiszewski brought this change] -- netrc: free 'home' on error + appveyor: Use two parallel compilation on appveyor with CMake - Follow-up to f9c7ba9096ec2 + Appveyor provides 2 CPUs for each builder[1], make sure to use parallel + compilation, when running with CMake. CMake learned this new option in + version 3.12[2] and the version provided by appveyor is fresh enough. - Coverity CID 1453474 + Curl doesn't really take that long to build and it is using the slowest + builder available, msbuild, so expect only a moderate improvement in + build times. - Closes #4291 + [1] https://www.appveyor.com/docs/build-environment/ + [2] https://cmake.org/cmake/help/v3.12/release/3.12.html + + Closes #4508 -- urldata: avoid 'generic', use dedicated pointers +- conn-reuse: requests wanting NTLM can reuse non-NTLM connections - For the 'proto' union within the connectdata struct. + Added test case 338 to verify. - Closes #4290 + Reported-by: Daniel Silverstone + Fixes #4499 + Closes #4514 -- cleanup: move functions out of url.c and make them static - - Closes #4289 +Marcel Raad (23 Oct 2019) +- tests: add missing proxy features -- smtp: check for and bail out on too short EHLO response - - Otherwise, a three byte response would make the smtp_state_ehlo_resp() - function misbehave. - - Credit to OSS-Fuzz - Bug: https://crbug.com/oss-fuzz/16918 +Daniel Stenberg (22 Oct 2019) +- RELEASE-NOTES: synced + +Marcel Raad (21 Oct 2019) +- tests: use %FILE_PWD for file:// URLs - Assisted-by: Max Dymond + This way, we always have exactly one slash after the host name, making + the tests pass when curl is compiled with the MSYS GCC. - Closes #4287 + Closes https://github.com/curl/curl/pull/4512 -- smb: init *msg to NULL in smb_send_and_recv() - - ... it might otherwise return OK from this function leaving that pointer - uninitialized. +- tests: add `connect to non-listen` keywords - Bug: https://crbug.com/oss-fuzz/16907 + These tests try to connect to ports nothing is listening on. - Closes #4286 + Closes https://github.com/curl/curl/pull/4511 -- ROADMAP: updated after recent user poll +- runtests: get textaware info from curl instead of perl - In rough prio order + The MSYS system on Windows can run the test suite for curl built with + any toolset. When built with the MSYS GCC, curl uses Unix line endings, + while it uses Windows line endings when built with the MinGW GCC, and + `^O` reports 'msys' in both cases. Use the curl executable itself to + determine the line endings instead, which reports 'x86_64-pc-msys' when + built with the MSYS GCC. + + Closes https://github.com/curl/curl/pull/4506 -- THANKS: remove duplicate +Daniel Stenberg (20 Oct 2019) +- [Michał Janiszewski brought this change] -- Curl_addr2string: take an addrlen argument too + appveyor: Add MSVC ARM64 build - This allows the function to figure out if a unix domain socket has a - file name or not associated with it! When a socket is created with - socketpair(), as done in the fuzzer testing, the path struct member is - uninitialized and must not be accessed. + Closes #4507 + +- http2_recv: a closed stream trumps pause state - Bug: https://crbug.com/oss-fuzz/16699 + ... and thus should return 0, not EAGAIN. - Closes #4283 - -- [Rolf Eike Beer brought this change] - - CMake: remove needless newlines at end of gss variables - -- [Rolf Eike Beer brought this change] + Reported-by: Tom van der Woerdt + Fixes #4496 + Closes #4505 - CI: remove duplicate configure flag for LGTM.com +- http2: expire a timeout at end of stream + + To make sure that transfer is being dealt with. Streams without + Content-Length need a final read to notice the end-of-stream state. + + Reported-by: Tom van der Woerdt + Fixes #4496 -- [Rolf Eike Beer brought this change] +Dan Fandrich (18 Oct 2019) +- travis: Add an ARM64 build + + Test 323 is failing for some reason, so disable it there for now. - CMake: use platform dependent name for dlopen() library +Marcel Raad (18 Oct 2019) +- examples/sslbackend: fix -Wchar-subscripts warning - Closes #4279 + With the `isdigit` implementation that comes with MSYS2, the argument + is used as an array subscript, resulting in a -Wchar-subscripts + warning. `isdigit`'s behavior is undefined if the argument is negative + and not EOF [0]. As done in lib/curl_ctype.h, cast the `char` variable + to `unsigned char` to avoid that. + + [0] https://en.cppreference.com/w/c/string/byte/isdigit + + Closes https://github.com/curl/curl/pull/4503 -- quiche: expire when poll returned data +Daniel Stenberg (18 Oct 2019) +- configure: remove all cyassl references - ... to make sure we continue draining the queue until empty + In particular, this removes the case where configure would find an old + cyall installation rather than a wolfssl one if present. The library is + named wolfssl in modern days so there's no real need to keep support for + the former. - Closes #4281 + Reported-by: Jacob Barthelmeh + Closes #4502 -- quiche: decrease available buffer size, don't assign it! +Marcel Raad (17 Oct 2019) +- test1162: disable MSYS2's POSIX path conversion - Found-by: Jeremy Lainé + This avoids MSYS2 converting the backslasb in the URL to a slash, + causing the test to fail. +Daniel Stenberg (17 Oct 2019) - RELEASE-NOTES: synced -- [Kyohei Kadota brought this change] - - curl: fix include conditions - -- [Kyohei Kadota brought this change] - - plan9: fix installation instructions +Jay Satiro (16 Oct 2019) +- CURLOPT_TIMEOUT.3: Clarify transfer timeout time includes queue time - Closes #4276 - -- ngtcp2: on h3 stream close, call expire + Prior to this change some users did not understand that the "request" + starts when the handle is added to the multi handle, or probably they + did not understand that some of those transfers may be queued and that + time is included in timeout. - ... to trigger a new read to detect the stream close! + Reported-by: Jeroen Ooms - Closes #4275 + Fixes https://github.com/curl/curl/issues/4486 + Closes https://github.com/curl/curl/pull/4489 -- [Tatsuhiro Tsujikawa brought this change] +- [Stian Soiland-Reyes brought this change] - ngtcp2: build latest ngtcp2 and ngtcp2_crypto_openssl + tool_operate: Fix retry sleep time shown to user when Retry-After - Closes #4278 - -- ngtcp2: set flow control window to stream buffer size + - If server header Retry-After is being used for retry sleep time then + show that value to the user instead of the normal retry sleep time. - Closes #4274 - -- [Christopher Head brought this change] - - CURLOPT_HEADERFUNCTION.3: clarify + This is a follow-up to 640b973 (7.66.0) which changed curl tool so that + the value from Retry-After header overrides other retry timing options. - Closes #4273 + Closes https://github.com/curl/curl/pull/4498 -- CURLINFO docs: mention that in redirects times are added +Daniel Stenberg (16 Oct 2019) +- url: normalize CURLINFO_EFFECTIVE_URL - Suggested-by: Brandon Dong - Fixes #4250 - Closes #4269 - -- travis: enable ngtcp2 builds again + The URL extracted with CURLINFO_EFFECTIVE_URL was returned as given as + input in most cases, which made it not get a scheme prefixed like before + if the URL was given without one, and it didn't remove dotdot sequences + etc. - Switched to the openssl-quic-draft-22 openssl branch. + Added test case 1907 to verify that this now works as intended and as + before 7.62.0. - Closes #4271 - -- HTTP3: switched openssl branch to use - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: Build with latest ngtcp2 and ngtcp2_crypto_openssl + Regression introduced in 7.62.0 - Closes #4270 + Reported-by: Christophe Dervieux + Fixes #4491 + Closes #4493 -- http2: when marked for closure and wanted to close == OK +Marcel Raad (16 Oct 2019) +- tests: line ending fixes for Windows - It could otherwise return an error even when closed correctly if GOAWAY - had been received previously. + Mark some files as text. - Reported-by: Tom van der Woerdt - Fixes #4267 - Closes #4268 - -- RELEASE-NOTES: synced + Closes https://github.com/curl/curl/pull/4490 -- build-openssl: fix build with Visual Studio 2019 +- tests: use proxy feature - Reviewed-by: Marcel Raad - Contributed-by: osabc on github - Fixes #4188 - Closes #4266 + This makes the tests succeed when using --disable-proxy. + + Closes https://github.com/curl/curl/pull/4488 -Kamil Dudka (26 Aug 2019) -- vauth: return CURLE_AUTH_ERROR on gss_init_sec_context() failure +- smbserver: fix Python 3 compatibility - This is a follow-up to https://github.com/curl/curl/pull/3864 . + Python 2's `ConfigParser` module is spelled `configparser` in Python 3. - Closes #4224 + Closes https://github.com/curl/curl/pull/4484 -Daniel Stenberg (26 Aug 2019) -- KNOWN_BUGS: USE_UNIX_SOCKETS on Windows +- security: silence conversion warning - Closes #4040 - -- quiche: send the HTTP body correctly on callback uploads + With MinGW-w64, `curl_socket_t` is is a 32 or 64 bit unsigned integer, + while `read` expects a 32 bit signed integer. + Use `sread` instead of `read` to use the correct parameter type. - Closes #4265 + Closes https://github.com/curl/curl/pull/4483 -- travis: disable ngtcp2 builds (temporarily) +- connect: silence sign-compare warning - Just too many API changes right now + With MinGW-w64 using WinSock, `curl_socklen_t` is signed, while the + result of `sizeof` is unsigned. - Closes #4264 + Closes https://github.com/curl/curl/pull/4483 -- ngtcp2: add support for SSLKEYLOGFILE +Daniel Stenberg (13 Oct 2019) +- TODO: Handle growing SFTP files - Closes #4260 + Closes #4344 -- ngtcp2: improve h3 response receiving +- KNOWN_BUGS: remove "CURLFORM_CONTENTLEN in an array" - Closes #4259 + The curl_formadd() function is deprecated and shouldn't be used so the + real fix for applications is to switch to the curl_mime_* API. -- ngtcp2: use nghttp3_version() +- KNOWN_BUGS: "LDAP on Windows does authentication wrong" + + Closes #3116 -- ngtcp2: sync with upstream API changes +- appveyor: add a winbuild that uses VS2017 - Assisted-by: Tatsuhiro Tsujikawa + Closes #4482 -- [Kyle Abramowitz brought this change] +- [Harry Sintonen brought this change] - scp: fix directory name length used in memcpy + socketpair: fix include and define for older TCP header systems - Fix read off end of array due to bad pointer math in getworkingpath for - SCP home directory case. + fixed build for systems that need netinet/in.h for IPPROTO_TCP and are + missing INADDR_LOOPBACK - Closes #4258 + Closes #4480 -- http: the 'closed' struct field is used by both ngh2 and ngh3 +- socketpair: fix double-close in error case - and remove 'header_recvbuf', not used for anything + Follow-up to bc2dbef0afc08 + +- gskit: use the generic Curl_socketpair + +- asyn-thread: make use of Curl_socketpair() where available + +- socketpair: an implemention for Windows and more - Reported-by: Jeremy Lainé + Curl_socketpair() is designed to be used and work everywhere if there's + no native version or the native version isn't good enough. - Closes #4257 + Closes #4466 -- ngtcp2: accept upload via callback - - Closes #4256 +- RELEASE-NOTES: synced -- defines: avoid underscore-prefixed defines - - Double-underscored or underscore plus uppercase letter at least. +- connect: return CURLE_OPERATION_TIMEDOUT for errno == ETIMEDOUT - ... as they're claimed to be reserved. + Previosly all connect() failures would return CURLE_COULDNT_CONNECT, no + matter what errno said. - Reported-by: patnyb on github + This makes for example --retry work on these transfer failures. - Fixes #4254 - Closes #4255 + Reported-by: Nathaniel J. Smith + Fixes #4461 + Clsoes #4462 + +- cirrus: switch off blackhole status on the freebsd CI machines -- travis: add a build using ngtcp2 + nghttp3 (and a patched OpenSSL) +- tests: use port 2 instead of 60000 for a safer non-listening port - Runs no tests + ... when the tests want "connection refused". + +- KNOWN_BUGS: IDN tests failing on Windows - Closes #4253 + Closes #3747 -- travis: bump to using nghttp2 version 1.39.2 +Dan Fandrich (9 Oct 2019) +- cirrus: Increase the git clone depth. - Closes #4252 + If more commits are submitted to master between the time of triggering + the first Cirrus build and the time the final build gets started, the + desired commit is no longer at HEAD and the build will error out. + [skip ci] -- [Gisle Vanem brought this change] +Daniel Stenberg (9 Oct 2019) +- docs: make sure the --no-progress-meter docs file is in dist too - docs/examples/curlx: fix errors +- docs: document it as --no-progress-meter instead of the reverse - Initialise 'mimetype' and require the -p12 arg. + Follow-up to 93373a960c3bb4 - Closes #4248 + Reported-by: infinnovation-dev on github + Fixes #4474 + Closes #4475 -- cleanup: remove DOT_CHAR completely - - Follow-up to f9c7ba9096ec - - The use of DOT_CHAR for ".ssh" was probably a mistake and is removed - now. - - Pointed-out-by: Gisle Vanem - Bug: https://github.com/curl/curl/pull/4230#issuecomment-522960638 +Dan Fandrich (9 Oct 2019) +- cirrus: Switch the FreeBSD 11.x build to 11.3 and add a 13.0 build. - Closes #4247 + Also, select the images using image_family to get the latest snapshots + automatically. + [skip ci] -- spnego_sspi: add typecast to fix build warning +Daniel Stenberg (8 Oct 2019) +- curl: --no-progress-meter - Reported in build "Win32 target on Debian Stretch (64-bit) - - i686-w64-mingw32 - gcc-20170516" + New option that allows a user to ONLY switch off curl's progress meter + and leave everything else in "talkative" mode. - Closes #4245 + Reported-by: Piotr Komborski + Fixes #4422 + Closes #4470 -- openssl: build warning free with boringssl +- TODO: Consult %APPDATA% also for .netrc - Closes #4244 + Closes #4016 -- curl: make --libcurl use CURL_HTTP_VERSION_3 +- CURLOPT_TIMEOUT.3: remove the mention of "minutes" - Closes #4243 - -- ngtcp2: make postfields-set posts work + ... just say that limiting operations risk aborting otherwise fine + working transfers. If that means seconds, minutes or hours, we leave to + the user. - Closes #4242 - -- http: remove chunked-encoding and expect header use for HTTP/3 + Reported-by: Martin Gartner + Closes #4469 -- [Alessandro Ghedini brought this change] +- [Andrei Valeriu BICA brought this change] - configure: use pkg-config to detect quiche - - This removes the need to hard-code the quiche target path in - configure.ac. + docs: added multi-event.c example - This depends on https://github.com/cloudflare/quiche/pull/128 + Similar to multi-uv.c but using libevent 2. This is a simpler libevent + integration example then hiperfifo.c. - Closes #4237 + Closes #4471 -- CURLOPT_SSL_VERIFYHOST: treat the value 1 as 2 - - For a long time (since 7.28.1) we've returned error when setting the - value to 1 to make applications notice that we stopped supported the old - behavior for 1. Starting now, we treat 1 and 2 exactly the same. - - Closes #4241 +Jay Satiro (5 Oct 2019) +- [Nicolas brought this change] -- curl: use .curlrc (with a dot) on Windows as well + ldap: fix OOM error on missing query string - Fall-back to _curlrc if the dot-version is missing. + - Allow missing queries, don't return NO_MEMORY error in such a case. - Co-Authored-By: Steve Holme + It is acceptable for there to be no specified query string, for example: - Closes #4230 - -- netrc: make the code try ".netrc" on Windows as well + curl ldap://ldap.forumsys.com - ... but fall back and try "_netrc" too if the dot version didn't work. + A regression bug in 1b443a7 caused this issue. - Co-Authored-By: Steve Holme - -- ngtcp2: use ngtcp2_version() to get the run-time version + This is a partial fix for #4261. - ... which of course doesn't have to be the same used at build-time. + Bug: https://github.com/curl/curl/issues/4261#issuecomment-525543077 + Reported-by: Jojojov@users.noreply.github.com + Analyzed-by: Samuel Surtees - Function just recently merged in ngtcp2. + Closes https://github.com/curl/curl/pull/4467 -- ngtcp2: move the h3 initing to immediately after the rx key - - To fix a segfault and to better deal with 0-RTT +- [Paul B. Omta brought this change] + + build: Remove unused HAVE_LIBSSL and HAVE_LIBCRYPTO defines - Assisted-by: Tatsuhiro Tsujikawa + Closes https://github.com/curl/curl/pull/4460 -- [Alessandro Ghedini brought this change] +Daniel Stenberg (5 Oct 2019) +- RELEASE-NOTES: synced - quiche: register debug callback once and earlier - - The quiche debug callback is global and can only be initialized once, so - make sure we don't do it multiple times (e.g. if multiple requests are - executed). +- [Stian Soiland-Reyes brought this change] + + curl: ensure HTTP 429 triggers --retry - In addition this initializes the callback before the connection is - created, so we get logs for the handshake as well. + This completes #3794. - Closes #4236 - -- ssh: add a generic Curl_ssh_version function for SSH backends + Also make sure the new tests from #4195 are enabled - Closes #4235 - -- base64: check for SSH, not specific SSH backends - -- vssh: move ssh init/cleanup functions into backend code + Closes #4465 -- vssh: create directory for SSH backend code +Marcel Raad (4 Oct 2019) +- [apique brought this change] -- TODO/ROADMAP: remove "refuse downgrade redirects" and HTTP/3 - - HTTP3 is now already in full progress + winbuild: add ENABLE_UNICODE option - Downgrade redirects can be achived almost exactly like that by setting - CURLOPT_REDIR_PROTOCOLS. + Fixes https://github.com/curl/curl/issues/4308 + Closes https://github.com/curl/curl/pull/4309 -- RELEASE-NOTES: synced +Daniel Stenberg (4 Oct 2019) +- ngtcp2: adapt to API change + + Closes #4457 -- travis: add a quiche build +- cookies: change argument type for Curl_flush_cookies + + The second argument is really a 'bool' so use that and pass in TRUE/FALSE + to make it clear. - Closes #4207 + Closes #4455 -- http: fix use of credentials from URL when using HTTP proxy +- http2: move state-init from creation to pre-transfer + + To make sure that the HTTP/2 state is initialized correctly for + duplicated handles. It would otherwise easily generate "spurious" + PRIORITY frames to get sent over HTTP/2 connections when duplicated easy + handles were used. - When a username and password are provided in the URL, they were wrongly - removed from the stored URL so that subsequent uses of the same URL - wouldn't find the crendentials. This made doing HTTP auth with multiple - connections (like Digest) mishave. + Reported-by: Daniel Silverstone + Fixes #4303 + Closes #4442 + +- urlapi: fix use-after-free bug - Regression from 46e164069d1a5230 (7.62.0) + Follow-up from 2c20109a9b5d04 - Test case 335 added to verify. + Added test 663 to verify. - Reported-by: Mike Crowe + Reported by OSS-Fuzz + Bug: https://crbug.com/oss-fuzz/17954 - Fixes #4228 - Closes #4229 + Closes #4453 -- [Mike Crowe brought this change] +- [Paul Dreik brought this change] - tests: Replace outdated test case numbering documentation + cookie: avoid harmless use after free - Tests are no longer grouped by numeric range[1]. Let's stop saying that - and provide some alternative advice for numbering tests. + This fix removes a use after free which can be triggered by + the internal cookie fuzzer, but otherwise is probably + impossible to trigger from an ordinary application. - [1] https://curl.haxx.se/mail/lib-2019-08/0043.html + The following program reproduces it: - Closes #4227 - -- travis: reduce number of torture tests in 'coverage' + curl_global_init(CURL_GLOBAL_DEFAULT); + CURL* handle=curl_easy_init(); + CookieInfo* info=Curl_cookie_init(handle,NULL,NULL,false); + curl_easy_setopt(handle, CURLOPT_COOKIEJAR, "/dev/null"); + Curl_flush_cookies(handle, true); + Curl_cookie_cleanup(info); + curl_easy_cleanup(handle); + curl_global_cleanup(); - ... to make it complete in time. This cut seems not almost not affect - the coverage percentage and yet completes within 35 minutes on travis - where the previous runs recently always timed out after 50. + This was found through fuzzing. - Closes #4223 + Closes #4454 -- [Igor Makarov brought this change] +- [Denis Chaplygin brought this change] - configure: use -lquiche to link to quiche + docs: add note on failed handles not being counted by curl_multi_perform - Closes #4226 + Closes #4446 -- ngtcp2: provide the callbacks as a static struct - - ... instead of having them in quicsocket +- CURLMOPT_MAX_CONCURRENT_STREAMS.3: fix SEE ALSO typo -- [Tatsuhiro Tsujikawa brought this change] +- [Niall O'Reilly brought this change] - ngtcp2: add missing nghttp3_conn_add_write_offset call + ESNI: initial build/setup - Closes #4225 - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: deal with stream close - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: Consume QUIC STREAM data properly + Closes #4011 -- [Tatsuhiro Tsujikawa brought this change] +- RELEASE-NOTES: synced - ngtcp2: don't reinitialize SSL on Retry +- redirect: when following redirects to an absolute URL, URL encode it + + ... to make it handle for example (RFC violating) embeded spaces. + + Reported-by: momala454 on github + Fixes #4445 + Closes #4447 -- multi: getsock improvements for QUIC connecting +- urlapi: fix URL encoding when setting a full URL -- connect: connections are persistent by default for HTTP/3 +- tool_operate: rename functions to make more sense -- quiche: happy eyeballs +- curl: create easy handles on-demand and not ahead of time - Closes #4220 - -- ngtcp2: do QUIC connections happy-eyeballs friendly - -- curl_version: bump string buffer size to 250 + This should again enable crazy-large download ranges of the style + [1-10000000] that otherwise easily ran out of memory starting in 7.66.0 + when this new handle allocating scheme was introduced. - With HTTP/3 libs and plenty TLS libs, I manged to hit the limit (which - causes a truncated output). + Reported-by: Peter Sumatra + Fixes #4393 + Closes #4438 -- CURLOPT_ALTSVC.3: use a "" file name to not load from a file +- [Kunal Ekawde brought this change] -Jay Satiro (14 Aug 2019) -- vauth: Use CURLE_AUTH_ERROR for auth function errors + CURLMOPT_MAX_CONCURRENT_STREAMS: new setopt - - Add new error code CURLE_AUTH_ERROR. + Closes #4410 + +- chunked-encoding: stop hiding the CURLE_BAD_CONTENT_ENCODING error - Prior to this change auth function errors were signaled by - CURLE_OUT_OF_MEMORY and CURLE_RECV_ERROR, and neither one was - technically correct. + Unknown content-encoding would get returned as CURLE_WRITE_ERROR if the + response is chunked-encoded. - Ref: https://github.com/curl/curl/pull/3848 + Reported-by: Ilya Kosarev + Fixes #4310 + Closes #4449 + +Marcel Raad (1 Oct 2019) +- checksrc: fix uninitialized variable warning - Co-authored-by: Dominik Hölzl + The loop doesn't need to be executed without a file argument. - Closes https://github.com/curl/curl/pull/3864 + Closes https://github.com/curl/curl/pull/4444 -Daniel Stenberg (13 Aug 2019) -- curl_version_info: make the quic_version a const +- urlapi: fix unused variable warning - Follow-up from 1a2df1518ad8653f + `dest` is only used with `ENABLE_IPV6`. - Closes #4222 + Closes https://github.com/curl/curl/pull/4444 -- examples: add http3.c, altsvc.c and http3-present.c +- lib: silence conversion warnings - Closes #4221 + Closes https://github.com/curl/curl/pull/4444 -Peter Wu (13 Aug 2019) -- nss: use TLSv1.3 as default if supported +- AppVeyor: add 32-bit MinGW-w64 build - SSL_VersionRangeGetDefault returns (TLSv1.0, TLSv1.2) as supported - range in NSS 3.45. It looks like the intention is to raise the minimum - version rather than lowering the maximum, so adjust accordingly. Note - that the caller (nss_setup_connect) initializes the version range to - (TLSv1.0, TLSv1.3), so there is no need to check for >= TLSv1.0 again. + With WinSSL and testing enabled so that it would have detected most of + the warnings fixed in [0] and [1]. - Closes #4187 - Reviewed-by: Daniel Stenberg - Reviewed-by: Kamil Dudka - -Daniel Stenberg (13 Aug 2019) -- quic.h: remove unused proto + [0] https://github.com/curl/curl/pull/4398 + [1] https://github.com/curl/curl/pull/4415 + + Closes https://github.com/curl/curl/pull/4433 -- curl_version_info.3: mentioned ALTSVC and HTTP3 +- AppVeyor: remove MSYS2_ARG_CONV_EXCL for winbuild - ... and sorted the list alphabetically + It's only used for MSYS2 with MinGW. + + Closes -- lib/quic.c: unused - removed +Daniel Stenberg (30 Sep 2019) +- [Emil Engler brought this change] -- CURLOPT_ALTSVC_CTRL.3: remove CURLALTSVC_ALTUSED + git: add tests/server/disabled to .gitignore - Follow-up to 98c3f148 that removed it from the header file - -- [Junho Choi brought this change] + Closes #4441 - docs/HTTP3: simplify quiche build instruction +- altsvc: accept quoted ma and persist values - Use --recursive to get boringssl in one line + As mandated by the spec. Test 1654 is extended to verify. - Closes #4219 + Closes #4443 -- altsvc: make it use h3-22 with ngtcp2 as well +- mailmap: a Lucas fix -- ngtcp2: initial h3 request work - - Closes #4217 +Alessandro Ghedini (29 Sep 2019) +- [Lucas Pardue brought this change] -- curl_version_info: offer quic (and h3) library info - - Closes #4216 + quiche: update HTTP/3 config creation to new API -- HTTP3: use ngtcp2's draft-22 branch +Daniel Stenberg (29 Sep 2019) +- BINDINGS: PureBasic, Net::Curl for perl and Nim -- RELEASE-NOTES: synced +- BINDINGS: Kapito is an Erlang library, basically a binding -- CURLOPT_READFUNCTION.3: provide inline example +- BINDINGS: added clj-curl - ... instead of mentioning one in another place + Reported-by: Lucas Severo -- [Tatsuhiro Tsujikawa brought this change] +- [Jay Satiro brought this change] - ngtcp2: send HTTP/3 request with nghttp3 - - This commit makes sending HTTP/3 request with nghttp3 work. It - minimally receives HTTP response and calls nghttp3 callbacks, but no - processing is made at the moment. + docs: disambiguate CURLUPART_HOST is for host name (ie no port) - Closes #4215 - -- nghttp3: initial h3 template code added + Closes #4424 -- nghttp3: required when ngtcp2 is used for QUIC +- cookies: using a share with cookies shouldn't enable the cookie engine - - checked for by configure - - updated docs/HTTP3.md - - shown in the version string + The 'share object' only sets the storage area for cookies. The "cookie + engine" still needs to be enabled or activated using the normal cookie + options. - Closes #4210 - -- [Eric Wong brought this change] - - asyn-thread: issue CURL_POLL_REMOVE before closing socket + This caused the curl command line tool to accidentally use cookies + without having been told to, since curl switched to using shared cookies + in 7.66.0. - This avoids EBADF errors from EPOLL_CTL_DEL operations in the - ephiperfifo.c example. EBADF is dangerous in multi-threaded - applications where I rely on epoll_ctl to operate on the same - epoll description from different threads. + Test 1166 verifies - Follow-up to eb9a604f8d7db8 + Updated test 506 - Bug: https://curl.haxx.se/mail/lib-2019-08/0026.html - Closes #4211 + Fixes #4429 + Closes #4434 -- [Carlo Marcelo Arenas Belón brought this change] +- setopt: handle ALTSVC set to NULL - configure: avoid undefined check_for_ca_bundle - - instead of using a "greater than 0" test, check for variable being - set, as it is always set to 1, and could be left unset if non of - OPENSSL MBEDTLS GNUTLS WOLFSSL is being configured for. - - Closes #4213 +- RELEASE-NOTES: synced -- [Tatsuhiro Tsujikawa brought this change] +- [grdowns brought this change] - ngtcp2: Send ALPN h3-22 + INSTALL: add vcpkg installation instructions - Closes #4212 - -- [Tatsuhiro Tsujikawa brought this change] - - ngtcp2: use ngtcp2_settings_default and specify initial_ts - -- curl_global_init_mem.3: mention it was added in 7.12.0 + Closes #4435 -- [Tatsuhiro Tsujikawa brought this change] +- [Zenju brought this change] - ngtcp2: make the QUIC handshake work + FTP: add test for FTPFILE_NOCWD: Avoid redundant CWDs + + Add libtest 661 - Closes #4209 + Closes #4417 -- [Alex Mayorga brought this change] +- [Zenju brought this change] - HTTP3.md: Update quiche build instructions - - Added cloning for quiche and BoringSSL and modified the build - instructions so they work on a clean folder. + FTP: url-decode path before evaluation - Closes #4208 + Closes #4428 -- CURLOPT_H3: removed +Marcel Raad (27 Sep 2019) +- tests: fix narrowing conversion warnings - There's no use for this anymore and it was never in a release. + `timediff_t` is 64 bits wide also on 32-bit systems since + commit b1616dad8f0. - Closes #4206 + Closes https://github.com/curl/curl/pull/4415 + +Jay Satiro (27 Sep 2019) +- [Julian Z brought this change] -- http3: make connection reuse work + vtls: Fix comment typo about macosx-version-min compiler flag - Closes #4204 + Closes https://github.com/curl/curl/pull/4425 -- quiche: add SSLKEYLOGFILE support +Daniel Stenberg (26 Sep 2019) +- [Yechiel Kalmenson brought this change] -- cleanup: s/curl_debug/curl_dbg_debug in comments and docs - - Leftovers from the function rename back in 76b63489495 - - Reported-by: Gisle Vanem - Bug: https://github.com/curl/curl/commit/f3e0f071b14fcb46a453f69bdf4e062bcaacf362#com - mitcomment-34601751 + README: minor grammar fix - Closes #4203 + Closes #4431 -- RELEASE-NOTES: synced +- [Spezifant brought this change] -- alt-svc: add protocol version selection masking - - So that users can mask in/out specific HTTP versions when Alt-Svc is - used. - - - Removed "h2c" and updated test case accordingly - - Changed how the altsvc struct is laid out - - Added ifdefs to make the unittest run even in a quiche-tree + HTTP3: fix prefix parameter for ngtcp2 build - Closes #4201 + Closes #4430 -- http3: fix the HTTP/3 in the request, make alt-svc set right versions - - Closes #4200 +- quiche: don't close connection at end of stream! -- alt-svc: send Alt-Used: in redirected requests - - RFC 7838 section 5: - - When using an alternative service, clients SHOULD include an Alt-Used - header field in all requests. - - Removed CURLALTSVC_ALTUSED again (feature is still EXPERIMENTAL thus - this is deemed ok). - - You can disable sending this header just like you disable any other HTTP - header in libcurl. - - Closes #4199 +- quiche: set 'drain' when returning without having drained the queues -- CURLOPT_HTTP_VERSION: seting this to 3 forces HTTP/3 use directly - - Even though it cannot fall-back to a lower HTTP version automatically. The - safer way to upgrade remains via CURLOPT_ALTSVC. - - CURLOPT_H3 no longer has any bits that do anything and might be removed - before we remove the experimental label. - - Updated the curl tool accordingly to use "--http3". +- Revert "FTP: url-decode path before evaluation" - Closes #4197 + This reverts commit 2f036a72d543e96128bd75cb0fedd88815fd42e2. -- docs/ALTSVC: remove what works and the experimental explanation - - Also, put the TODO items at the bottom. - - Closes #4198 +- HTTP3: merged and simplified the two 'running' sections -- docs/EXPERIMENTAL: explain what it means and what's experimental now +- HTTP3: show an --alt-svc using example too -- curl: make use of CURLINFO_RETRY_AFTER when retrying - - If a Retry-After: header was used in the response, that value overrides - other retry timing options. - - Fixes #3794 - Closes #4195 +- [Zenju brought this change] -- curl: use CURLINFO_PROTOCOL to check for HTTP(s) + FTP: url-decode path before evaluation - ... instead of CURLINFO_EFFECTIVE_URL to avoid string operations. + Closes #4423 -- CURLINFO_RETRY_AFTER: parse the Retry-After header value - - This is only the libcurl part that provides the information. There's no - user of the parsed value. This change includes three new tests for the - parser. +- openssl: use strerror on SSL_ERROR_SYSCALL - Ref: #3794 - -- docs/ALTSVC.md: first basic file format description - -- curl: have -w's 'http_version' show '3' for HTTP/3 + Instead of showing the somewhat nonsensical errno number, use strerror() + to provide a more relatable error message. - Closes #4196 + Closes #4411 -- curl.h: add CURL_HTTP_VERSION_3 to the version enum +- HTTP3: update quic.aiortc.org + add link to server list - It can't be set for CURLOPT_HTTP_VERSION, but it can be extracted with - CURLINFO_HTTP_VERSION. - -- quiche: make use of the connection timeout API properly - -- quiche: make POSTFIELDS posts work - -- quiche: improved error handling and memory cleanups - -- quiche: flush egress in h3_stream_recv() too - -- RELEASE-NOTES: synced - -Jay Satiro (6 Aug 2019) -- [Patrick Monnerat brought this change] + Reported-by: Jeremy Lainé - os400: take care of CURLOPT_SASL_AUTHZID in curl_easy_setopt_ccsid(). +Jay Satiro (26 Sep 2019) +- url: don't set appconnect time for non-ssl/non-ssh connections - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + Prior to this change non-ssl/non-ssh connections that were reused set + TIMER_APPCONNECT [1]. Arguably that was incorrect since no SSL/SSH + handshake took place. - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + [1]: TIMER_APPCONNECT is publicly known as CURLINFO_APPCONNECT_TIME in + libcurl and %{time_appconnect} in the curl tool. It is documented as + "the time until the SSL/SSH handshake is completed". - Closes https://github.com/curl/curl/pull/4186 - -- tests: Fix the line endings for the SASL alt-auth tests + Reported-by: Marcel Hernandez - - Change data and protocol sections to CRLF line endings. + Ref: https://github.com/curl/curl/issues/3760 - Prior to this change the tests would fail or hang, which is because - certain sections such as protocol require CRLF line endings. + Closes https://github.com/curl/curl/pull/3773 + +Daniel Stenberg (25 Sep 2019) +- ngtcp2: remove fprintf() calls - Follow-up to grandparent commit which added the tests. + - convert some of them to H3BUF() calls to infof() + - remove some of them completely + - made DEBUG_HTTP3 defined only if CURLDEBUG is set for now - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + Closes #4421 + +- [Jay Satiro brought this change] + + url: fix the NULL hostname compiler warning case - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + Closes #4403 + +- [Jay Satiro brought this change] + + travis: move the go install to linux-only - Closes https://github.com/curl/curl/pull/4186 + ... to repair the build again + Closes #4403 + +- altsvc: correct the #ifdef for the ngtcp2 backend -- [Steve Holme brought this change] +- altsvc: save h3 as h3-23 + + Follow-up to d176a2c7e5 - examples: Added SASL PLAIN authorisation identity (authzid) examples +- urlapi: question mark within fragment is still fragment - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + The parser would check for a query part before fragment, which caused it + to do wrong when the fragment contains a question mark. - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + Extended test 1560 to verify. - Closes https://github.com/curl/curl/pull/4186 + Reported-by: Alex Konev + Fixes #4412 + Closes #4413 -- [Steve Holme brought this change] +- [Alex Samorukov brought this change] - curl: --sasl-authzid added to support CURLOPT_SASL_AUTHZID from the tool + HTTP3.md: move -p for mkdir, remove -j for make - Ref: https://github.com/curl/curl/issues/3653 - Ref: https://github.com/curl/curl/pull/3790 + - mkdir on OSX/Darwin requires `-p` argument before dir - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + - portabbly figuring out number of cores is an exercise for somewhere + else - Closes https://github.com/curl/curl/pull/4186 - -- [Steve Holme brought this change] + Closes #4407 - sasl: Implement SASL authorisation identity via CURLOPT_SASL_AUTHZID - - Added the ability for the calling program to specify the authorisation - identity (authzid), the identity to act as, in addition to the - authentication identity (authcid) and password when using SASL PLAIN - authentication. - - Fixes #3653 - Closes #3790 +Patrick Monnerat (24 Sep 2019) +- os400: getpeername() and getsockname() return ebcdic AF_UNIX sockaddr, - NOTE: This commit was cherry-picked and is part of a series of commits - that added the authzid feature for upcoming 7.66.0. The series was - temporarily reverted in db8ec1f so that it would not ship in a 7.65.x - patch release. + As libcurl now uses these 2 system functions, wrappers are needed on os400 + to convert returned AF_UNIX sockaddrs to ascii. - Closes https://github.com/curl/curl/pull/4186 - -Daniel Stenberg (6 Aug 2019) -- docs/HTTP3: refreshed as it is now in master and HTTP/3 can be tested - -- [Yiming Jing brought this change] + This is a follow-up to commit 7fb54ef. + See also #4037. + Closes #4214 - mesalink: implement client authentication - - Closes #4184 +Jay Satiro (24 Sep 2019) +- [Lucas Pardue brought this change] -- curl_multi_poll: a sister to curl_multi_wait() that waits more - - Repeatedly we see problems where using curl_multi_wait() is difficult or - just awkward because if it has no file descriptor to wait for - internally, it returns immediately and leaves it to the caller to wait - for a small amount of time in order to avoid occasional busy-looping. + strcase: fix raw lowercasing the letter X - This is often missed or misunderstood, leading to underperforming - applications. + Casing mistake in Curl_raw_tolower 'X' wasn't lowercased as 'x' prior to + this change. - This change introduces curl_multi_poll() as a replacement drop-in - function that accepts the exact same set of arguments. This function - works identically to curl_multi_wait() - EXCEPT - for the case when - there's nothing to wait for internally, as then this function will by - itself wait for a "suitable" short time before it returns. This - effectiely avoids all risks of busy-looping and should also make it less - likely that apps "over-wait". + Follow-up to 0023fce which added the function several days ago. - This also changes the curl tool to use this funtion internally when - doing parallel transfers and changes curl_easy_perform() to use it - internally. + Ref: https://github.com/curl/curl/pull/4401#discussion_r327396546 - Closes #4163 + Closes https://github.com/curl/curl/pull/4408 -- quiche:h3_stream_recv return 0 at end of stream +Daniel Stenberg (23 Sep 2019) +- http2: Expression 'stream->stream_id != - 1' is always true - ... and remove some verbose messages we don't need. Made transfers from - facebook.com work better. - -- altsvc: make quiche use h3-22 now - -- quiche: show the actual version number + PVS-Studio warning + Fixes #4402 -- quiche: first working HTTP/3 request - - - enable debug log - - fix use of quiche API - - use download buffer - - separate header/body +- http2: A value is being subtracted from the unsigned variable - Closes #4193 + PVS-Studio warning + Fixes #4402 -- http09: disable HTTP/0.9 by default in both tool and library - - As the plan has been laid out in DEPRECATED. Update docs accordingly and - verify in test 1174. Now requires the option to be set to allow HTTP/0.9 - responses. +- libssh: part of conditional expression is always true: !result - Closes #4191 - -- quiche: initial h3 request send/receive - -- lib/Makefile.am: make checksrc run in vquic too + PVS-Studio warning + Fixed #4402 -- altsvc: fix removal of expired cache entry +- libssh: part of conditional expression is always true - Closes #4192 - -- RELEASE-NOTES: synced + PVS-Studio warning + Fixes #4402 -Steve Holme (4 Aug 2019) -- md4: Use our own MD4 implementation when no crypto libraries are available +- libssh: The expression is excessive or contains a misprint - Closes #3780 - -- md4: No need to include Curl_md4.h for each TLS library + PVS-Studio warning + Fixes #4402 -- md4: No need for the NTLM code to call Curl_md4it() for each TLS library +- quiche: The expression must be surrounded by parentheses - As the NTLM code no longer calls any of TLS libraries' specific MD4 - functions, there is no need to call this function for each #ifdef. - -- md4: Move the mbed TLS MD4 implementation out of the NTLM code - -- md4: Move the WinCrypt implementation out of the NTLM code - -- md4: Move the SecureTransport implementation out of the NTLM code - -- md4: Use the Curl_md4it() function for OpenSSL based NTLM - -- md4: Move the GNU TLS gcrypt MD4 implementation out of the NTLM code - -- md4: Move the GNU TLS Nettle MD4 implementation out of the NTLM code + PVS-Studio warning + Fixes #4402 -Jay Satiro (4 Aug 2019) -- OS400: Add CURLOPT_H3 symbols - - Follow-up to 3af0e76 which added experimental H3 support. +- vauth: The parameter 'status' must be surrounded by parentheses - Closes https://github.com/curl/curl/pull/4185 - -Daniel Stenberg (3 Aug 2019) -- url: make use of new HTTP version if alt-svc has one + PVS-Studio warning + Fixes #4402 -- url: set conn->transport to default TCP at init time +- [Paul Dreik brought this change] -- altsvc: with quiche, use the quiche h3 alpn string + doh: allow only http and https in debug mode - Closes #4183 - -- alt-svc: more liberal ALPN name parsing + Otherwise curl may be told to use for instance pop3 to + communicate with the doh server, which most likely + is not what you want. - Allow pretty much anything to be part of the ALPN identifier. In - particular minus, which is used for "h3-20" (in-progress HTTP/3 - versions) etc. + Found through fuzzing. - Updated test 356. - Closes #4182 + Closes #4406 -- quiche: use the proper HTTP/3 ALPN +- [Paul Dreik brought this change] -- quiche: add failf() calls for two error cases - - To aid debugging + doh: return early if there is no time left - Closes #4181 + Closes #4406 -- mailmap: added Kyohei Kadota +- [Barry Pollard brought this change] -Kamil Dudka (1 Aug 2019) -- http_negotiate: improve handling of gss_init_sec_context() failures - - If HTTPAUTH_GSSNEGOTIATE was used for a POST request and - gss_init_sec_context() failed, the POST request was sent - with empty body. This commit also restores the original - behavior of `curl --fail --negotiate`, which was changed - by commit 6c6035532383e300c712e4c1cd9fdd749ed5cf59. - - Add regression tests 2077 and 2078 to cover this. + http: lowercase headernames for HTTP/2 and HTTP/3 - Fixes #3992 - Closes #4171 + Closes #4401 + Fixes #4400 -Daniel Stenberg (1 Aug 2019) -- mailmap: added 4 more names +Marcel Raad (23 Sep 2019) +- vtls: fix narrowing conversion warnings + + Curl_timeleft returns `timediff_t`, which is 64 bits wide also on + 32-bit systems since commit b1616dad8f0. - Evgeny Grin, Peter Pih, Anton Malov and Marquis de Muesli + Closes https://github.com/curl/curl/pull/4398 -- mailmap: add Giorgos Oikonomou +Daniel Stenberg (23 Sep 2019) +- [Joel Depooter brought this change] -- src/makefile: fix uncompressed hugehelp.c generation + winbuild: Add manifest to curl.exe for proper OS version detection - Regression from 5cf5d57ab9 (7.64.1) + This is a small fix to commit ebd213270a017a6830928ee2e1f4a9cabc799898 + in pull request #1221. That commit added the CURL_EMBED_MANIFEST flag to + CURL_RC_FLAGS. However, later in the file CURL_RC_FLAGS is + overwritten. The fix is to append values to CURL_RC_FLAGS instead of + overwriting - Fixed-by: Lance Ware - Fixes #4176 - Closes #4177 + Closes #4399 -- appveyor: pass on -k to make +- RELEASE-NOTES: synced -- timediff: make it 64 bit (if possible) even with 32 bit time_t +Marcel Raad (22 Sep 2019) +- openssl: fix compiler warning with LibreSSL - ... to make it hold microseconds too. + It was already fixed for BoringSSL in commit a0f8fccb1e0. + LibreSSL has had the second argument to SSL_CTX_set_min_proto_version + as uint16_t ever since the function was added in [0]. - Fixes #4165 - Closes #4168 - -- ROADMAP: parallel transfers are merged now - -- getenv: support up to 4K environment variable contents on windows + [0] https://github.com/libressl-portable/openbsd/commit/56f107201baefb5533486d665a58d8f57fd3aeda - Reported-by: Michal Čaplygin - Fixes #4174 - Closes #4175 - -- [Kyohei Kadota brought this change] + Closes https://github.com/curl/curl/pull/4397 - plan9: add support for running on Plan 9 +Daniel Stenberg (22 Sep 2019) +- curl: exit the create_transfers loop on errors - Closes #3701 - -- [Kyohei Kadota brought this change] - - ntlm: explicit type casting - -- [Justin brought this change] - - curl.h: fix outdated comment + When looping around the ranges and given URLs to create transfers, all + errors should exit the loop and return. Previously it would keep + looping. - Closes #4167 + Reported-by: SumatraPeter on github + Bug: #4393 + Closes #4396 -- curl: remove outdated comment +Jay Satiro (21 Sep 2019) +- socks: Fix destination host shown on SOCKS5 error - Turned bad with commit b8894085000 + Prior to this change when a server returned a socks5 connect error then + curl would parse the destination address:port from that data and show it + to the user as the destination: - Reported-by: niallor on github - Fixes #4172 - Closes #4173 - -- cleanup: remove the 'numsocks' argument used in many places + curld -v --socks5 10.0.3.1:1080 http://google.com:99 + * SOCKS5 communication to google.com:99 + * SOCKS5 connect to IPv4 172.217.12.206 (locally resolved) + * Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) + curl: (7) Can't complete SOCKS5 connection to 253.127.0.0:26673. (1) - It was used (intended) to pass in the size of the 'socks' array that is - also passed to these functions, but was rarely actually checked/used and - the array is defined to a fixed size of MAX_SOCKSPEREASYHANDLE entries - that should be used instead. + That's incorrect because the address:port included in the connect error + is actually a bind address:port (typically unused) and not the + destination address:port. This fix changes curl to show the destination + information that curl sent to the server instead: - Closes #4169 - -- readwrite_data: repair setting the TIMER_STARTTRANSFER stamp + curld -v --socks5 10.0.3.1:1080 http://google.com:99 + * SOCKS5 communication to google.com:99 + * SOCKS5 connect to IPv4 172.217.7.14:99 (locally resolved) + * Can't complete SOCKS5 connection to 172.217.7.14:99. (1) + curl: (7) Can't complete SOCKS5 connection to 172.217.7.14:99. (1) - Regression, broken in commit 65eb65fde64bd5f (curl 7.64.1) + curld -v --socks5-hostname 10.0.3.1:1080 http://google.com:99 + * SOCKS5 communication to google.com:99 + * SOCKS5 connect to google.com:99 (remotely resolved) + * Can't complete SOCKS5 connection to google.com:99. (1) + curl: (7) Can't complete SOCKS5 connection to google.com:99. (1) - Reported-by: Jonathan Cardoso Machado - Assisted-by: Jay Satiro + Ref: https://tools.ietf.org/html/rfc1928#section-6 - Fixes #4136 - Closes #4162 + Closes https://github.com/curl/curl/pull/4394 -- mailmap: Amit Katyal +Daniel Stenberg (21 Sep 2019) +- travis: enable ngtcp2 h3-23 builds -- asyn-thread: removed unused variable +- altsvc: both backends run h3-23 now - Follow-up to eb9a604f. Mistake caused by me when I edited the commit - before push... - -- RELEASE-NOTES: synced - -- [Amit Katyal brought this change] + Closes #4395 - asyn-thread: create a socketpair to wait on +- http: fix warning on conversion from int to bit - Closes #4157 + Follow-up from 03ebe66d70 -- curl: cap the maximum allowed values for retry time arguments - - ... to avoid integer overflows later when multiplying with 1000 to - convert seconds to milliseconds. - - Added test 1269 to verify. +- urldata: use 'bool' for the bit type on MSVC compilers - Reported-by: Jason Lee - Closes #4166 + Closes #4387 + Fixes #4379 -- progress: reset download/uploaded counter - - ... to make CURLOPT_MAX_RECV_SPEED_LARGE and - CURLOPT_MAX_SEND_SPEED_LARGE work correctly on subsequent transfers that - reuse the same handle. +- appveyor: upgrade VS2017 to VS2019 - Fixed-by: Ironbars13 on github - Fixes #4084 - Closes #4161 + Closes #4383 -- http2_recv: trigger another read when the last data is returned - - ... so that end-of-stream is detected properly. +- [Zenju brought this change] + + FTP: FTPFILE_NOCWD: avoid redundant CWDs - Reported-by: Tom van der Woerdt - Fixes #4043 - Closes #4160 + Closes #4382 -- curl: avoid uncessary libcurl timeouts (in parallel mode) +- cookie: pass in the correct cookie amount to qsort() - When curl_multi_wait() returns OK without file descriptors to wait for, - it might already have done a long timeout. + As the loop discards cookies without domain set. This bug would lead to + qsort() trying to sort uninitialized pointers. We have however not found + it a security problem. - Closes #4159 + Reported-by: Paul Dreik + Closes #4386 -- [Balazs Kovacsics brought this change] +- [Paul Dreik brought this change] - HTTP: use chunked Transfer-Encoding for HTTP_POST if size unknown + urlapi: avoid index underflow for short ipv6 hostnames - If using the read callback for HTTP_POST, and POSTFIELDSIZE is not set, - automatically add a Transfer-Encoding: chunked header, same as it is - already done for HTTP_PUT, HTTP_POST_FORM and HTTP_POST_MIME. Update - test 1514 according to the new behaviour. + If the input hostname is "[", hlen will underflow to max of size_t when + it is subtracted with 2. - Closes #4138 - -Jay Satiro (29 Jul 2019) -- [Daniel Stenberg brought this change] - - winbuild: add vquic to list of build directories + hostname[hlen] will then cause a warning by ubsanitizer: - This fixes the winbuild build method which broke several days ago - when experimental quic support was added in 3af0e76. + runtime error: addition of unsigned offset to 0x overflowed to + 0x - Reported-by: Michael Lee + I think that in practice, the generated code will work, and the output + of hostname[hlen] will be the first character "[". - Fixes https://github.com/curl/curl/issues/4158 - -- easy: resize receive buffer on easy handle reset + This can be demonstrated by the following program (tested in both clang + and gcc, with -O3) - - In curl_easy_reset attempt to resize the receive buffer to its default - size. If realloc fails then continue using the previous size. + int main() { + char* hostname=strdup("["); + size_t hlen = strlen(hostname); - Prior to this change curl_easy_reset did not properly handle resetting - the receive buffer (data->state.buffer). It reset the variable holding - its size (data->set.buffer_size) to the default size (READBUFFER_SIZE) - but then did not actually resize the buffer. If a user resized the - buffer by using CURLOPT_BUFFERSIZE to set the size smaller than the - default, later called curl_easy_reset and attempted to reuse the handle - then a heap overflow would very likely occur during that handle's next - transfer. + hlen-=2; + hostname++; + printf("character is %d\n",+hostname[hlen]); + free(hostname-1); + } - Reported-by: Felix Hädicke + I found this through fuzzing, and even if it seems harmless, the proper + thing is to return early with an error. - Fixes https://github.com/curl/curl/issues/4143 - Closes https://github.com/curl/curl/pull/4145 + Closes #4389 -- [Brad Spencer brought this change] +- [Tatsuhiro Tsujikawa brought this change] - examples: Avoid reserved names in hiperfifo examples - - - Trade in __attribute__((unused)) for the classic (void)x to silence - unused symbols. - - Because the classic way is not gcc specific. Also because the prior - method mapped to symbol _Unused, which starts with _ and a capital - letter which is reserved. - - Assisted-by: The Infinnovation team - - Bug: https://github.com/curl/curl/issues/4120#issuecomment-512542108 + ngtcp2: compile with latest ngtcp2 + nghttp3 draft-23 - Closes https://github.com/curl/curl/pull/4153 - -Daniel Stenberg (25 Jul 2019) -- RELEASE-NOTES: synced + Closes #4392 -- [Felix Hädicke brought this change] +- THANKS-filter: deal with my typos 'Jat' => 'Jay' - ssh-libssh: do not specify O_APPEND when not in append mode +- travis: use go master - Specifying O_APPEND in conjunction with O_TRUNC and O_CREAT does not - make much sense. And this combination of flags is not accepted by all - SFTP servers (at least not Apache SSHD). + ... as the boringssl builds needs a very recent version - Fixes #4147 - Closes #4148 - -- [Gergely Nagy brought this change] + Co-authored-by: Jat Satiro + Closes #4361 - multi: call detach_connection before Curl_disconnect +- tool_operate: removed unused variable 'done' - Curl_disconnect bails out if conn->easyq is not empty, detach_connection - needs to be called first to remove the current easy from the queue. + Fixes warning detected by PVS-Studio + Fixes #4374 + +- tool_operate: Expression 'config->resume_from' is always true - Fixes #4144 - Closes #4151 + Fixes warning detected by PVS-Studio + Fixes #4374 -Jay Satiro (23 Jul 2019) -- tool_operate: fix implicit call to easysrc_cleanup +- tool_getparam: remove duplicate switch case - easysrc_cleanup is only defined when CURL_DISABLE_LIBCURL_OPTION is not - defined, and prior to this change would be called regardless. + Fixes warning detected by PVS-Studio + Fixes #4374 + +- libssh2: part of conditional expression is always true: !result - Bug: https://github.com/curl/curl/pull/3804#issuecomment-513922637 - Reported-by: Marcel Raad + Fixes warning detected by PVS-Studio + Fixes #4374 + +- urlapi: Expression 'storep' is always true - Closes https://github.com/curl/curl/pull/4142 + Fixes warning detected by PVS-Studio + Fixes #4374 -Daniel Stenberg (22 Jul 2019) -- curl:create_transfers check return code from curl_easy_setopt - - From commit b8894085 - - Pointed out by Coverity CID 1451703 +- urlapi: 'scheme' is always true - Closes #4134 + Fixes warning detected by PVS-Studio + Fixes #4374 -- HTTP3: initial (experimental) support +- urlapi: part of conditional expression is always true: (relurl[0] == '/') - USe configure --with-ngtcp2 or --with-quiche + Fixes warning detected by PVS-Studio + Fixes #4374 + +- setopt: store CURLOPT_RTSP_SERVER_CSEQ correctly - Using either option will enable a HTTP3 build. - Co-authored-by: Alessandro Ghedini + Fixes bug detected by PVS-Studio + Fixes #4374 + +- mime: make Curl_mime_duppart() assert if called without valid dst - Closes #3500 + Fixes warning detected by PVS-Studio + Fixes #4374 -- curl: remove dead code +- http_proxy: part of conditional expression is always true: !error - The loop never loops (since b889408500), pointed out by Coverity (CID - 1451702) + Fixes warning detected by PVS-Studio + Fixes #4374 + +- imap: merged two case-branches performing the same action - Closes #4133 + Fixes warning detected by PVS-Studio + Fixes #4374 -- docs/PARALLEL-TRANSFERS: correct the version number +- multi: value '2L' is assigned to a boolean + + Fixes warning detected by PVS-Studio + Fixes #4374 -- docs/PARALLEL-TRANSFERS: added +- easy: part of conditional expression is always true: !result + + Fixes warning detected by PVS-Studio + Fixes #4374 -- curl: support parallel transfers +- netrc: part of conditional expression is always true: !done - This is done by making sure each individual transfer is first added to a - linked list as then they can be performed serially, or at will, in - parallel. + Fixes warning detected by PVS-Studio + Fixes #4374 + +- version: Expression 'left > 1' is always true - Closes #3804 + Fixes warning detected by PVS-Studio + Fixes #4374 -- docs/MANUAL.md: converted to markdown from plain text +- url: remove dead code - ... will make it render as a nicer web page. + Fixes warning detected by PVS-Studio + Fixes #4374 + +- url: part of expression is always true: (bundle->multiuse == 0) - Closes #4131 + Fixes warning detected by PVS-Studio + Fixes #4374 -- curl_version_info: provide nghttp2 details +- ftp: the conditional expression is always true - Introducing CURLVERSION_SIXTH with nghttp2 info. + ... both !result and (ftp->transfer != FTPTRANSFER_BODY)! - Closes #4121 + Fixes warning detected by PVS-Studio + Fixes #4374 -- bump: start working on 7.66.0 +- ftp: Expression 'ftpc->wait_data_conn' is always false + + Fixes warning detected by PVS-Studio + Fixes #4374 -- source: remove names from source comments +- ftp: Expression 'ftpc->wait_data_conn' is always true - Several reasons: + Fixes warning detected by PVS-Studio + Fixes #4374 + +- ftp: part of conditional expression is always true: !result - - we can't add everyone who's helping out so its unfair to just a few - selected ones. - - we already list all helpers in THANKS and in RELEASE-NOTES for each - release - - we don't want to give the impression that some parts of the code is - "owned" or "controlled" by specific persons + Fixes warning detected by PVS-Studio + Fixes #4374 + +- http: fix Expression 'http->postdata' is always false - Assisted-by: Daniel Gustafsson - Closes #4129 + Fixes warning detected by PVS-Studio + Fixes #4374 + Reported-by: Valerii Zapodovnikov -Version 7.65.3 (19 Jul 2019) +- [Niall O'Reilly brought this change] -Daniel Stenberg (19 Jul 2019) -- RELEASE-NOTES: 7.65.3 + doh: avoid truncating DNS QTYPE to lower octet + + Closes #4381 -- THANKS: 7.65.3 status +- [Jens Finkhaeuser brought this change] -- progress: make the progress meter appear again + urlapi: CURLU_NO_AUTHORITY allows empty authority/host part - Fix regression caused by 21080e1 + CURLU_NO_AUTHORITY is intended for use with unknown schemes (i.e. not + "file:///") to override cURL's default demand that an authority exists. - Reported-by: Chih-Hsuan Yen - Fixes #4122 - Closes #4124 - -- version: bump to 7.65.3 - -- RELEASE-NOTES: Contributors or now 1990 - -Version 7.65.2 (17 Jul 2019) - -Daniel Stenberg (17 Jul 2019) -- RELEASE-NOTES: 7.65.2 - -- THANKS: add contributors from 7.65.2 + Closes #4349 -Jay Satiro (17 Jul 2019) -- [aasivov brought this change] +- version: next release will be 7.67.0 - cmake: Fix finding Brotli on case-sensitive file systems - - - Find package "Brotli" instead of "BROTLI" since the former is the - casing used for CMake/FindBrotli.cmake, and otherwise find_package - may fail on a case-sensitive file system. - - Fixes https://github.com/curl/curl/issues/4117 +- RELEASE-NOTES: synced -- CURLOPT_RANGE.3: Caution against using it for HTTP PUT +- url: only reuse TLS connections with matching pinning - AFAICT CURLOPT_RANGE does not support ranged HTTP PUT uploads so I've - cautioned against using it for that purpose and included a workaround. + If the requests have different CURLOPT_PINNEDPUBLICKEY strings set, the + connection should not be reused. - Bug: https://curl.haxx.se/mail/lib-2019-04/0075.html - Reported-by: Christopher Head + Bug: https://curl.haxx.se/mail/lib-2019-09/0061.html + Reported-by: Sebastian Haglund - Closes https://github.com/curl/curl/issues/3814 - -- [Stefano Simonelli brought this change] + Closes #4347 - CURLOPT_SEEKDATA.3: fix variable name +- README: add OSS-Fuzz badge [skip ci] - Closes https://github.com/curl/curl/pull/4118 - -- [Giorgos Oikonomou brought this change] + Closes #4380 - CIPHERS.md: Explain Schannel error SEC_E_ALGORITHM_MISMATCH - - If the SSL backend is Schannel and the user specifies an Schannel CALG_ - that is not supported by the protocol or the server then curl returns - CURLE_SSL_CONNECT_ERROR (35) SEC_E_ALGORITHM_MISMATCH. - - Fixes https://github.com/curl/curl/issues/3389 - Closes https://github.com/curl/curl/pull/4106 +Michael Kaufmann (18 Sep 2019) +- http: merge two "case" statements -- [Daniel Gustafsson brought this change] +Daniel Stenberg (18 Sep 2019) +- [Zenju brought this change] - nss: inspect returnvalue of token check - - PK11_IsPresent() checks for the token for the given slot is available, - and sets needlogin flags for the PK11_Authenticate() call. Should it - return false, we should however treat it as an error and bail out. + FTP: remove trailing slash from path for LIST/MLSD - Closes https://github.com/curl/curl/pull/4110 + Closes #4348 -- docs: Explain behavior change in --tlsv1. options since 7.54 - - Since 7.54 --tlsv1. options use the specified version or later, however - older versions of curl documented it as using just the specified version - which may or may not have happened depending on the TLS library. - Document this discrepancy to allay confusion for users familiar with the - old documentation that expect just the specified version. +- mime: when disabled, avoid C99 macro - Fixes https://github.com/curl/curl/issues/4097 - Closes https://github.com/curl/curl/pull/4119 + Closes #4368 -- libcurl: Restrict redirect schemes (follow-up) - - - Allow FTPS on redirect. - - - Update default allowed redirect protocols in documentation. +- url: cleanup dangling DOH request headers too - Follow-up to 6080ea0. + Follow-up to 9bc44ff64d9081 - Ref: https://github.com/curl/curl/pull/4094 + Credit to OSS-Fuzz + Bug: https://crbug.com/oss-fuzz/17269 - Closes https://github.com/curl/curl/pull/4115 + Closes #4372 -Daniel Stenberg (16 Jul 2019) -- test1173: make it also check all libcurl option man pages - - ... and adjust those that cause errors - - Closes #4116 +- [Christoph M. Becker brought this change] -- curl: only accept COLUMNS less than 10000 + http2: relax verification of :authority in push promise requests - ... as larger values would rather indicate something silly (and could - potentially cause buffer problems). + If the :authority pseudo header field doesn't contain an explicit port, + we assume it is valid for the default port, instead of rejecting the + request for all ports. - Reported-by: pendrek at hackerone - Closes #4114 - -- dist: add manpage-syntax.pl + Ref: https://curl.haxx.se/mail/lib-2019-09/0041.html - follow-up to 7fb66c403 + Closes #4365 -- test1173: detect some basic man page format mistakes +- doh: clean up dangling DOH handles and memory on easy close - Triggered by PR #4111 + If you set the same URL for target as for DoH (and it isn't a DoH + server), like "https://example.com" in both, the easy handles used for + the DoH requests could be left "dangling" and end up not getting freed. - Closes #4113 - -Jay Satiro (15 Jul 2019) -- [Bjarni Ingi Gislason brought this change] + Reported-by: Paul Dreik + Closes #4366 - docs: Fix missing lines caused by undefined macros - - - Escape apostrophes at line start. - - Some lines begin with a "'" (apostrophe, single quote), which is then - interpreted as a control character in *roff. - - Such lines are interpreted as being a call to a macro, and if - undefined, the lines are removed from the output. - - Bug: https://bugs.debian.org/926352 - Signed-off-by: Bjarni Ingi Gislason +- unit1655: make it C90 compliant - Submitted-by: Alessandro Ghedini + Unclear why this was not detected in the CI. - Closes https://github.com/curl/curl/pull/4111 + Follow-up to b7666027296a -Daniel Stenberg (14 Jul 2019) -- libcurl-security.3: update to new CURLOPT_REDIR_PROTOCOLS defaults +- smb: check for full size message before reading message details - follow-up to 6080ea098 - -- [Linos Giannopoulos brought this change] + To avoid reading of uninitialized data. + + Assisted-by: Max Dymond + Bug: https://crbug.com/oss-fuzz/16907 + Closes #4363 - libcurl: Add testcase for gopher redirects +- quiche: persist connection details - The testcase ensures that redirects to CURLPROTO_GOPHER won't be - allowed, by default, in the future. Also, curl is being used - for convenience while keeping the testcases DRY. + ... like we do for other protocols at connect time. This makes "curl -I" + and other things work. - The expected error code is CURLE_UNSUPPORTED_PROTOCOL when the client is - redirected to CURLPROTO_GOPHER + Reported-by: George Liu + Fixes #4358 + Closes #4360 + +- openssl: fix warning with boringssl and SSL_CTX_set_min_proto_version - Signed-off-by: Linos Giannopoulos + Follow-up to ffe34b7b59 + Closes #4359 -- [Linos Giannopoulos brought this change] +- [Paul Dreik brought this change] - libcurl: Restrict redirect schemes - - All protocols except for CURLPROTO_FILE/CURLPROTO_SMB and their TLS - counterpart were allowed for redirect. This vastly broadens the - exploitation surface in case of a vulnerability such as SSRF [1], where - libcurl-based clients are forced to make requests to arbitrary hosts. - - For instance, CURLPROTO_GOPHER can be used to smuggle any TCP-based - protocol by URL-encoding a payload in the URI. Gopher will open a TCP - connection and send the payload. - - Only HTTP/HTTPS and FTP are allowed. All other protocols have to be - explicitly enabled for redirects through CURLOPT_REDIR_PROTOCOLS. - - [1]: https://www.acunetix.com/blog/articles/server-side-request-forgery-vulnerability/ + doh: fix undefined behaviour and open up for gcc and clang optimization - Signed-off-by: Linos Giannopoulos + The undefined behaviour is annoying when running fuzzing with + sanitizers. The codegen is the same, but the meaning is now not up for + dispute. See https://cppinsights.io/s/516a2ff4 - Closes #4094 - -- [Zenju brought this change] - - openssl: define HAVE_SSL_GET_SHUTDOWN based on version number + By incrementing the pointer first, both gcc and clang recognize this as + a bswap and optimizes it to a single instruction. See + https://godbolt.org/z/994Zpx - Closes #4100 + Closes #4350 -- [Peter Simonyi brought this change] +- [Paul Dreik brought this change] - http: allow overriding timecond with custom header + doh: fix (harmless) buffer overrun - With CURLOPT_TIMECONDITION set, a header is automatically added (e.g. - If-Modified-Since). Allow this to be replaced or suppressed with - CURLOPT_HTTPHEADER. + Added unit test case 1655 to verify. + Close #4352 - Fixes #4103 - Closes #4109 + the code correctly finds the flaws in the old code, + if one temporarily restores doh.c to the old version. -Jay Satiro (11 Jul 2019) -- [Juergen Hoetzel brought this change] +Alessandro Ghedini (15 Sep 2019) +- docs: remove trailing ':' from section names in CURLOPT_TRAILER* man - smb: Use the correct error code for access denied on file open - - - Return CURLE_REMOTE_ACCESS_DENIED for SMB access denied on file open. - - Prior to this change CURLE_REMOTE_FILE_NOT_FOUND was returned instead. - - Closes https://github.com/curl/curl/pull/4095 +- docs: fix typo in CURLOPT_HTTP_VERSION man -- [Daniel Gustafsson brought this change] +GitHub (14 Sep 2019) +- [Daniel Stenberg brought this change] - DEPRECATE: fixup versions and spelling - - Correctly set the July 17 version to 7.65.2, and update spelling to - be consistent. Also fix a typo. + CI: inintial github action job - Closes https://github.com/curl/curl/pull/4107 - -- [Gisle Vanem brought this change] + First shot at a CI build on github actions - system_win32: fix clang warning +Daniel Stenberg (13 Sep 2019) +- appveyor: add a winbuild - - Declare variable in header as extern. + Assisted-by: Marcel Raad + Assisted-by: Jay Satiro - Bug: https://github.com/curl/curl/commit/48b9ea4#commitcomment-34084597 + Closes #4324 -Daniel Gustafsson (10 Jul 2019) -- headers: Remove no longer exported functions - - There were a leftover few prototypes of Curl_ functions that we used to - export but no longer do, this removes those prototypes and cleans up any - comments still referring to them. +- FTP: allow "rubbish" prepended to the SIZE response - Curl_write32_le(), Curl_strcpy_url(), Curl_strlen_url(), Curl_up_free() - Curl_concat_url(), Curl_detach_connnection(), Curl_http_setup_conn() - were made static in 05b100aee247bb9bec8e9a1b0166496aa4248d1c. - Curl_http_perhapsrewind() made static in 574aecee208f79d391f10d57520b3. + This is a protocol violation but apparently there are legacy proprietary + servers doing this. - For the remainder, I didn't trawl the Git logs hard enough to capture - their exact time of deletion, but they were all gone: Curl_splayprint(), - Curl_http2_send_request(), Curl_global_host_cache_dtor(), - Curl_scan_cache_used(), Curl_hostcache_destroy(), Curl_second_connect(), - Curl_http_auth_stage() and Curl_close_connections(). + Added test 336 and 337 to verify. - Closes #4096 - Reviewed-by: Daniel Stenberg - -- CMake: fix typos and spelling + Reported-by: Philippe Marguinaud + Closes #4339 -- [Kyle Edwards brought this change] +- [Zenju brought this change] - CMake: Convert errant elseif() to else() - - CMake interprets an elseif() with no arguments as elseif(FALSE), - resulting in the elseif() block not being executed. That is not what - was intended here. Change the empty elseif() to an else() as it was - intended. + FTP: skip CWD to entry dir when target is absolute - Closes #4101 - Reported-by: Artalus - Reviewed-by: Daniel Gustafsson + Closes #4332 -- buildconf: fix header filename +Kamil Dudka (13 Sep 2019) +- curl: fix memory leaked by parse_metalink() - The header file inclusion had a typo, it should be .h and not .hd. - Fix by renaming. + This commit fixes a regression introduced by curl-7_65_3-5-gb88940850. + Detected by tests 2005, 2008, 2009, 2010, 2011, and 2012 with valgrind + and libmetalink enabled. - Fixes #4102 - Reported-by: AceCrow on Github - -- [Jan Chren brought this change] + Closes #4326 - configure: fix --disable-code-coverage +Daniel Stenberg (13 Sep 2019) +- parsedate: still provide the name arrays when disabled - This fixes the case when --disable-code-coverage supplied to ./configure - would result in coverage="yes" being set. + If FILE or FTP are enabled, since they also use them! - Closes #4099 - Reviewed-by: Daniel Gustafsson - -- cleanup: fix typo in comment - -- RELEASE-NOTES: synced + Reported-by: Roland Hieber + Fixes #4325 + Closes #4343 -Jay Satiro (6 Jul 2019) -- [Daniel Gustafsson brought this change] +- [Gilles Vollant brought this change] - nss: support using libnss on macOS + curl:file2string: load large files much faster - The file suffix for dynamically loadable objects on macOS is .dylib, - which need to be added for the module definitions in order to get the - NSS TLS backend to work properly on macOS. + ... by using a more efficient realloc scheme. - Closes https://github.com/curl/curl/pull/4046 - -- [Daniel Gustafsson brought this change] + Bug: https://curl.haxx.se/mail/lib-2019-09/0045.html + Closes #4336 - nss: don't set unused parameter - - The value of the maxPTDs parameter to PR_Init() has since at least - NSPR 2.1, which was released sometime in 1998, been marked ignored - as is accordingly not used in the initialization code. Setting it - to a value when calling PR_Init() is thus benign, but indicates an - intent which may be misleading. Reset the value to zero to improve - clarity. +- openssl: close_notify on the FTP data connection doesn't mean closure - Closes https://github.com/curl/curl/pull/4054 - -- [Daniel Gustafsson brought this change] - - nss: only cache valid CRL entries + For FTPS transfers, curl gets close_notify on the data connection + without that being a signal to close the control connection! - Change the logic around such that we only keep CRLs that NSS actually - ended up caching around for later deletion. If CERT_CacheCRL() fails - then there is little point in delaying the freeing of the CRL as it - is not used. + Regression since 3f5da4e59a556fc (7.65.0) - Closes https://github.com/curl/curl/pull/4053 + Reported-by: Zenju on github + Reviewed-by: Jay Satiro + Fixes #4329 + Closes #4340 -- [Gergely Nagy brought this change] +- [Jimmy Gaussen brought this change] - lib: Use UTF-8 encoding in comments - - Some editors and IDEs assume that source files use UTF-8 file encodings. - It also fixes the build with MSVC when /utf-8 command line option is - used (this option is mandatory for some other open-source projects, this - is useful when using the same options is desired for building all - libraries of a project). + docs/HTTP3: fix `--with-ssl` ngtcp2 configure flag - Closes https://github.com/curl/curl/pull/4087 + Closes #4338 -- [Caleb Raitto brought this change] +- RELEASE-NOTES: synced - CURLOPT_HEADEROPT.3: Fix example - - Fix an issue where example builds a curl_slist, but fails to actually - use it, or free it. - - Closes https://github.com/curl/curl/pull/4090 +- curlver: bump to 7.66.1 -- [Shankar Jadhavar brought this change] +- [Zenju brought this change] - winbuild: Change Makefile to honor ENABLE_OPENSSL_AUTO_LOAD_CONFIG - - - Made changes so that ENABLE_OPENSSL_AUTO_LOAD_CONFIG will be honored. - - - Also removed some ^M chars from file. + setopt: make it easier to add new enum values - Prior to this change while building on Windows platform even if we pass - the ENABLE_OPENSSL_AUTO_LOAD_CONFIG option with value as "no" it does - not set the CURL_DISABLE_OPENSSL_AUTO_LOAD_CONFIG flag. + ... by using the *_LAST define names better. - Closes https://github.com/curl/curl/pull/4086 - -Daniel Stenberg (4 Jul 2019) -- doh-url.d: added in 7.62.0 + Closes #4321 -Jay Satiro (30 Jun 2019) -- docs: Fix links to OpenSSL docs - - OpenSSL changed their manual locations and does not redirect to the new - locations. +- asyn-thread: s/AF_LOCAL/AF_UNIX for Solaris - Bug: https://curl.haxx.se/mail/lib-2019-06/0056.html - Reported-by: Daniel Stenberg + Reported-by: Dagobert Michelsen + Fixes #4328 + Closes #4333 -Daniel Stenberg (26 Jun 2019) -- [Gaël PORTAY brought this change] +- [Bernhard Walle brought this change] - curl_multi_wait.3: escape backslash in example - - The backslash in the character Line Feed must be escaped. - - The current man-page outputs the code as following: - - fprintf(stderr, "curl_multi failed, code %d.0, mc); + winbuild/MakefileBuild.vc: Add vssh - The commit fixes it as follow: + Without that modification, the Windows build using the makefiles doesn't + work. - fprintf(stderr, "curl_multi failed, code %d\n", mc); + Signed-off-by: Bernhard Walle - Closes #4079 + Fixes #4322 + Closes #4323 -- openssl: disable engine if OPENSSL_NO_UI_CONSOLE is defined +Bernhard Walle (11 Sep 2019) +- winbuild/MakefileBuild.vc: Fix line endings - ... since that needs UI_OpenSSL() which isn't provided when OpenSSL is - built with OPENSSL_NO_UI_CONSOLE which happens when OpenSSL is built for - UWP (with "VC-WIN32-UWP"). + The file had mixed line endings. - Reported-by: Vasily Lobaskin - Fixes #4073 - Closes #4077 + Signed-off-by: Bernhard Walle -- test1521: adapt to SLISTPOINT +Jay Satiro (11 Sep 2019) +- ldap: Stop using wide char version of ldapp_err2string - The header now has the slist-using options marked as SLISTPOINT so this - makes sure test 1521 understands that. + Despite ldapp_err2string being documented by MS as returning a + PCHAR (char *), when UNICODE it is mapped to ldap_err2stringW and + returns PWCHAR (wchar_t *). - Follow-up to ae99b4de1c443ae989 + We have lots of code that expects ldap_err2string to return char *, + most of it failf used like this: - Closes #4074 - -- win32: make DLL loading a no-op for UWP + failf(data, "LDAP local: Some error: %s", ldap_err2string(rc)); - Reported-by: Michael Brehm - Fixes #4060 - Closes #4072 + Closes https://github.com/curl/curl/pull/4272 -- [1ocalhost brought this change] +Version 7.66.0 (10 Sep 2019) - configure: fix typo '--disable-http-uath' - - Closes #4076 +Daniel Stenberg (10 Sep 2019) +- RELEASE-NOTES: curl 7.66.0 -- [Niklas Hambüchen brought this change] +- THANKS: from the 7.66.0 release - docs: fix string suggesting HTTP/2 is not the default - - Commit 25fd1057c9c86e3 made HTTP2 the default, and further down in the - man page that new default is mentioned, but the section at the top - contradicted it until now. +- curl: make sure the parallel transfers do them all - Also remove claim that setting the HTTP version is not sensible. + The logic could erroneously break the loop too early before all + transfers had been transferred. - Closes #4075 - -- RELEASE-NOTES: synced + Reported-by: Tom van der Woerdt + Fixes #4316 + Closes #4317 -- [Stephan Szabo brought this change] +- urlapi: one colon is enough for the strspn() input (typo) - tests: update fixed IP for hostip/clientip split +- urlapi: verify the IPv6 numerical address - These tests give differences for me on linux when using a hostip - pointing to the external ip address for the local machine. + It needs to parse correctly. Otherwise it could be tricked into letting + through a-f using host names that libcurl would then resolve. Like + '[ab.be]'. - Closes #4070 + Reported-by: Thomas Vegas + Closes #4315 + +- [Clément Notin brought this change] -Daniel Gustafsson (24 Jun 2019) -- http: clarify header buffer size calculation + openssl: use SSL_CTX_set__proto_version() when available - The header buffer size calculation can from static analysis seem to - overlow as it performs an addition between two size_t variables and - stores the result in a size_t variable. Overflow is however guarded - against elsewhere since the input to the addition is regulated by - the maximum read buffer size. Clarify this with a comment since the - question was asked. + OpenSSL 1.1.0 adds SSL_CTX_set__proto_version() that we now use + when available. Existing code is preserved for older versions of + OpenSSL. - Reviewed-by: Daniel Stenberg + Closes #4304 + +- [Clément Notin brought this change] + + openssl: indent, re-organize and add comments + +- [migueljcrum brought this change] -Daniel Stenberg (24 Jun 2019) -- KNOWN_BUGS: Don't clear digest for single realm + sspi: fix memory leaks - Closes #3267 + Closes #4299 + +- travis: disable ngtcp2 builds (again) -- KNOWN_BUGS: Schannel disable CURLOPT_SSL_VERIFYPEER and verify hostname +- Curl_fillreadbuffer: avoid double-free trailer buf on error + + Reviewed-by: Jay Satiro + Reported-by: Thomas Vegas - Closes #3284 + Closes #4307 -- http2: call done_sending on end of upload +- tool_setopt: handle a libcurl build without netrc support - To make sure a HTTP/2 stream registers the end of stream. + Reported-by: codesniffer13 on github + Fixes #4302 + Closes #4305 + +- security:read_data fix bad realloc() - Bug #4043 made me find this problem but this fix doesn't correct the - reported issue. + ... that could end up a double-free - Closes #4068 + CVE-2019-5481 + Bug: https://curl.haxx.se/docs/CVE-2019-5481.html -- [James Brown brought this change] +- [Thomas Vegas brought this change] - c-ares: honor port numbers in CURLOPT_DNS_SERVERS + tftp: Alloc maximum blksize, and use default unless OACK is received - By using ares_set_servers_ports_csv on new enough c-ares. + Fixes potential buffer overflow from 'recvfrom()', should the server + return an OACK without blksize. - Fixes #4066 - Closes #4067 - -Daniel Gustafsson (24 Jun 2019) -- CURLMOPT_SOCKETFUNCTION.3: fix typo + Bug: https://curl.haxx.se/docs/CVE-2019-5482.html + CVE-2019-5482 -Daniel Stenberg (24 Jun 2019) -- [Koen Dergent brought this change] +- [Thomas Vegas brought this change] - curl: skip CURLOPT_PROXY_CAPATH for disabled-proxy builds - - Closes #4061 + tftp: return error when packet is too small for options -- test153: fix content-length to avoid occasional hang - - Closes #4065 +- KNOWN_BUGS/TODO: cleanup and remove outdated issues - RELEASE-NOTES: synced -- multi: enable multiplexing by default (again) +- netrc: free 'home' on error - It was originally made default in d7c4213bd0c (7.62.0) but mistakenly - reverted in commit 2f44e94efb3d (7.65.0). Now enabled again. + Follow-up to f9c7ba9096ec2 - Closes #4051 - -- typecheck: add 3 missing strings and a callback data pointer + Coverity CID 1453474 - Closes #4050 + Closes #4291 -- tests: add disable-scan.pl to dist +- urldata: avoid 'generic', use dedicated pointers - follow-up from 29177f422a5 + For the 'proto' union within the connectdata struct. - Closes #4059 + Closes #4290 -- http2: don't call stream-close on already closed streams +- cleanup: move functions out of url.c and make them static - Closes #4055 + Closes #4289 -Marcel Raad (20 Jun 2019) -- travis: enable alt-svc for coverage build +- smtp: check for and bail out on too short EHLO response - Closes - -- travis: enable libssh2 for coverage build + Otherwise, a three byte response would make the smtp_state_ehlo_resp() + function misbehave. - It was enabled by default before commit c92d2e14cfb. + Credit to OSS-Fuzz + Bug: https://crbug.com/oss-fuzz/16918 - Disable torture tests 600 and 601 because of - https://github.com/curl/curl/issues/1678. + Assisted-by: Max Dymond - Closes + Closes #4287 -- travis: disable threaded resolver for coverage build - - This enables more tests. +- smb: init *msg to NULL in smb_send_and_recv() - Closes - -- travis: enable brotli for all xenial jobs + ... it might otherwise return OK from this function leaving that pointer + uninitialized. - There's no need for a separate job, and no need to build it from source - with Xenial. + Bug: https://crbug.com/oss-fuzz/16907 - Closes + Closes #4286 -- travis: enable warnings-as-errors for coverage build +- ROADMAP: updated after recent user poll - Closes - -GitHub (20 Jun 2019) -- [Gisle Vanem brought this change] + In rough prio order - system_win32: fix typo +- THANKS: remove duplicate -Daniel Stenberg (20 Jun 2019) -- typecheck: CURLOPT_CONNECT_TO takes an slist too +- Curl_addr2string: take an addrlen argument too + + This allows the function to figure out if a unix domain socket has a + file name or not associated with it! When a socket is created with + socketpair(), as done in the fuzzer testing, the path struct member is + uninitialized and must not be accessed. - Additionally, add an alias in curl.h for slist-using options so that - we can grep/parse those out at will. + Bug: https://crbug.com/oss-fuzz/16699 - Closes #4042 + Closes #4283 -- [Stephan Szabo brought this change] +- [Rolf Eike Beer brought this change] - tests: support non-localhost HOSTIP for dict/smb servers - - smbserver.py/dictserver.py were explicitly using localhost/127.0.0.1 for - binding the server which when we were running the tests with a separate - HOSTIP and CLIENTIP had failures verifying the server from the device we - were testing. - - This changes them to take the address from runtests.py and default to - localhost/127.0.0.1 if none is given. - - Closes #4048 + CMake: remove needless newlines at end of gss variables -- test1523: basic test of CURLOPT_LOW_SPEED_LIMIT +- [Rolf Eike Beer brought this change] -- configure: --disable-progress-meter - - Builds libcurl without support for the built-in progress meter. - - Closes #4023 + CI: remove duplicate configure flag for LGTM.com -- curl: improved skip-setopt-options when built with disabled features - - Reduces #ifdefs in src/tool_operate.c +- [Rolf Eike Beer brought this change] + + CMake: use platform dependent name for dlopen() library - Follow-up from 4e86f2fc4e6 - Closes #3936 + Closes #4279 -Steve Holme (18 Jun 2019) -- netrc: Return the correct error code when out of memory +- quiche: expire when poll returned data - Introduced in 763c5178. + ... to make sure we continue draining the queue until empty - Closes #4036 + Closes #4281 -Daniel Stenberg (18 Jun 2019) -- config-os400: add getpeername and getsockname defines +- quiche: decrease available buffer size, don't assign it! - Reported-by: jonrumsey on github - Fixes #4037 - Closes #4039 + Found-by: Jeremy Lainé -- runtests: keep logfiles around by default - - Make '-k' a no-op. The singletest function now clears the log directory - BEFORE each individual test and not after, which makes it possible to - always keep the logfiles around after a test has been run. No need to - specify -k anymore. Keeping the option parsing around to work with users - of old habits. - - Some tests also didn't work properly when -k was used (since the old - logs would be kep when a new test starts) which this change also fixes. - - Closes #4035 +- RELEASE-NOTES: synced -- [Gergely Nagy brought this change] +- [Kyohei Kadota brought this change] - openssl: fix pubkey/signature algorithm detection in certinfo - - Certinfo gives the same result for all OpenSSL versions. - Also made printing RSA pubkeys consistent with older versions. - - Reported-by: Michael Wallner - Fixes #3706 - Closes #4030 + curl: fix include conditions -- conn_maxage: move the check to prune_dead_connections() - - ... and avoid the locking issue. - - Reported-by: Kunal Ekawde - Fixes #4029 - Closes #4032 +- [Kyohei Kadota brought this change] -- tests: have runtests figure out disabled features - - ... so that runtests can skip individual test cases that test features - that are explicitly disabled in this build. This new logic is intended - for disabled features that aren't otherwise easily visible through the - curl_version_info() or other API calls. - - tests/server/disabled is a newly built executable that will output a - list of disabled features. Outputs nothing for a default build. + plan9: fix installation instructions - Closes #3950 + Closes #4276 -- test188/189: fix Content-Length +- ngtcp2: on h3 stream close, call expire - This cures the flaky test results + ... to trigger a new read to detect the stream close! - Closes #4034 + Closes #4275 -- [Thomas Gamper brought this change] +- [Tatsuhiro Tsujikawa brought this change] - winbuild: use WITH_PREFIX if given + ngtcp2: build latest ngtcp2 and ngtcp2_crypto_openssl - Closes #4031 + Closes #4278 -Daniel Gustafsson (17 Jun 2019) -- openssl: remove outdated comment - - OpenSSL used to call exit(1) on syntax errors in OPENSSL_config(), - which is why we switched to CONF_modules_load_file() and introduced - a comment stating why. This behavior was however changed in OpenSSL - commit abdd677125f3a9e3082f8c5692203590fdb9b860, so remove the now - outdated and incorrect comment. The mentioned commit also declares - OPENSSL_config() deprecated so keep the current coding. +- ngtcp2: set flow control window to stream buffer size - Closes #4033 - Reviewed-by: Daniel Stenberg + Closes #4274 -Daniel Stenberg (16 Jun 2019) -- RELEASE-NOTES: synced +- [Christopher Head brought this change] -Patrick Monnerat (16 Jun 2019) -- os400: make vsetopt() non-static as Curl_vsetopt() for os400 support. - - Use it in curl_easy_setopt_ccsid(). + CURLOPT_HEADERFUNCTION.3: clarify - Reported-by: jonrumsey on github - Fixes #3833 - Closes #4028 + Closes #4273 -Daniel Stenberg (15 Jun 2019) -- runtests: report single test time + total duration - - ... after each successful test. +- CURLINFO docs: mention that in redirects times are added - Closes #4027 + Suggested-by: Brandon Dong + Fixes #4250 + Closes #4269 -- multi: fix the transfer hash function +- travis: enable ngtcp2 builds again - Follow-up from 8b987cc7eb + Switched to the openssl-quic-draft-22 openssl branch. - Reported-by: Tom van der Woerdt - Fixes #4018 - Closes #4024 + Closes #4271 -- unit1654: cleanup on memory failure - - ... to make it handle torture tests properly. - - Reported-by: Marcel Raad - Fixes #4021 - Closes #4022 +- HTTP3: switched openssl branch to use -Marcel Raad (13 Jun 2019) -- krb5: fix compiler warning - - Even though the variable was used in a DEBUGASSERT, GCC 8 warned in - debug mode: - krb5.c:324:17: error: unused variable 'maj' [-Werror=unused-variable] - - Just suppress the warning and declare the variable unconditionally - instead of only for DEBUGBUILD (which also missed the check for - HAVE_ASSERT_H). - - Closes https://github.com/curl/curl/pull/4020 +- [Tatsuhiro Tsujikawa brought this change] -Daniel Stenberg (13 Jun 2019) -- quote.d: asterisk prefix works for SFTP as well + ngtcp2: Build with latest ngtcp2 and ngtcp2_crypto_openssl - Reported-by: Ben Voris - Fixes #4017 - Closes #4019 + Closes #4270 -- multi: fix the transfer hashes in the socket hash entries - - - The transfer hashes weren't using the correct keys so removing entries - failed. +- http2: when marked for closure and wanted to close == OK - - Simplified the iteration logic over transfers sharing the same socket and - they now simply are set to expire and thus get handled in the "regular" - timer loop instead. + It could otherwise return an error even when closed correctly if GOAWAY + had been received previously. Reported-by: Tom van der Woerdt - Fixes #4012 - Closes #4014 + Fixes #4267 + Closes #4268 -Jay Satiro (12 Jun 2019) -- [Cliff Crosland brought this change] +- RELEASE-NOTES: synced - url: Fix CURLOPT_MAXAGE_CONN time comparison - - Old connections are meant to expire from the connection cache after - CURLOPT_MAXAGE_CONN seconds. However, they actually expire after 1000x - that value. This occurs because a time value measured in milliseconds is - accidentally divided by 1M instead of by 1,000. +- build-openssl: fix build with Visual Studio 2019 - Closes https://github.com/curl/curl/pull/4013 + Reviewed-by: Marcel Raad + Contributed-by: osabc on github + Fixes #4188 + Closes #4266 -Daniel Stenberg (11 Jun 2019) -- test1165: verify that CURL_DISABLE_ symbols are in sync +Kamil Dudka (26 Aug 2019) +- vauth: return CURLE_AUTH_ERROR on gss_init_sec_context() failure + + This is a follow-up to https://github.com/curl/curl/pull/3864 . - between configure.ac and source code. They should be possible to switch - on/off in configure AND be used in source code. + Closes #4224 -- configure: remove CURL_DISABLE_TLS_SRP +Daniel Stenberg (26 Aug 2019) +- KNOWN_BUGS: USE_UNIX_SOCKETS on Windows - It isn't used by code so stop providing the define. + Closes #4040 + +- quiche: send the HTTP body correctly on callback uploads - Closes #4010 + Closes #4265 -- Revert "cmake: add SMB to list of disabled protocols if HTTP_ONLY is specified" +- travis: disable ngtcp2 builds (temporarily) - This reverts commit 36738caeb78603ce24e3ea089a167b8c216fb938. + Just too many API changes right now - Apparently several of the appveyor windows builds broke. + Closes #4264 -- [sergey-raevskiy brought this change] +- ngtcp2: add support for SSLKEYLOGFILE + + Closes #4260 - cmake: add SMB to list of disabled protocols if HTTP_ONLY is specified +- ngtcp2: improve h3 response receiving - Reviewed-by: Jakub Zakrzewski - Closes #3770 + Closes #4259 -- RELEASE-NOTES: synced +- ngtcp2: use nghttp3_version() -- http2: remove CURL_DISABLE_TYPECHECK define +- ngtcp2: sync with upstream API changes - ... in http2-less builds as it served no use. + Assisted-by: Tatsuhiro Tsujikawa -- configure: more --disable switches to toggle off individual features - - ... actual support in the code for disabling these has already landed. - - Closes #4009 +- [Kyle Abramowitz brought this change] -- wolfssl: fix key pinning build error + scp: fix directory name length used in memcpy - follow-up from deb9462ff2de8 + Fix read off end of array due to bad pointer math in getworkingpath for + SCP home directory case. + + Closes #4258 -- cgit v1.2.3