Strict https site behind SSL Offloader - Grav not always using https urls

Hi
I've got a site that's full https but behind a SSL offloader, I've uncommented out the x-forward-proto line in the htacess file and the site in general works fine.

However it looks like bits that use grav uri which I think uses PHP SERVER_PORT (which will be port 80) instead of the x-fordward-proto port. I've noticed this mostly with the sitemap and SEO plugins.

I don't have full access to the server. Is there any way to tackle this? I've tried setting the Force SSL field in the admin panel but it has not effect.

Could do with some more information about which bits exactly are not https. Also which version of Grav and assocaited plugins?