Ok so I open a bug report for V1.2.x. As closing https://phabricator.vyos.net/Q79 was not very helpfull. In VyOS blog I read cloud installs are very important for this project. (I agree in this)
As wrote in Q79 the problem persists in the latest nightly build. Details see https://phabricator.vyos.net/Q79
further updates on https://wiki.strongswan.org/issues/1220.
There is a kernel patch for the xen-netfront bug: https://patchwork.kernel.org/patch/9338979/. Maybe a backport to 4.4 would be the solution for VyOS.
I can help with tests are welcome.