It seems the generated nginx configuration for the redirection (server listening on the TCP/80) is wrong:
it's using $server_name in the redirect, while this value is set to "_" (default vhost in nginx).
It should be using $host instead, since this variable will contain the actual Host header content (among other possibilities).
Here's a simple way to prove it:
curl -I <my-vyos-router>
HTTP/1.1 301 Moved Permanently
Date: Thu, 29 Oct 2020 10:58:48 GMT
Location: https://_/ <- this is wrong.
I'll submit a patch soon with the correction.