I've been digging on Grav the last month! I've slogged through many issues on my own and the documentation is fantastic, but I've run into something that has me stumped.

I've got my sites running on multisite through a series of folders at the same level as the user folder:
-user
-user-www.site1.com
-user-www.site2.com

This part works great - no problems there!
I have several other sites that need to get built - sometimes before the DNS entries are made. In order to show them to internal and external stakeholders, I wanted to allow subdirectory multisite as well. Something like this:
www.existingdomain.com/www.site3.com
And I wanted to use the exact same install directory.
Crazy? Maybe.
So, I got subdirectory multisite working in setup.php. I'm setting a cookie if there's a query string like this:
www.existingdomain.com/www.site3.com?preview=true
Works great in my localhost (apache)! Yay! And it appeared to work perfectly in production (Nginx)! However, here's what happens:

1. Go here: www.existingdomain.com/www.site3.com?preview=true and the page shows up
2. Refresh the page and I get these errors in my grav log:

grav.CRITICAL: Undefined index: /var/www/html/grav/user-www.site3.com/pages - Trace:

0 /var/www/html/grav/system/src/Grav/Common/Page/Pages.php(340): Whoops\Run->handleError(8, 'Undefined index...', '/var/www/html/m...', 340, Array)

1 /var/www/html/grav/system/src/Grav/Common/Twig/Twig.php(305): Grav\Common\Page\Pages->root()

2 /var/www/html/grav/system/src/Grav/Common/Grav.php(157): Grav\Common\Twig\Twig->processSite(NULL)

3 /var/www/html/grav/vendor/pimple/pimple/src/Pimple/Container.php(113): Grav\Common\Grav::Grav\Common{closure}(Object(Grav\Common\Grav))

4 /var/www/html/grav/system/src/Grav/Common/Grav.php(257): Pimple\Container->offsetGet('output')

5 /var/www/html/grav/index.php(37): Grav\Common\Grav->process() #6 {main} [] []

This last bit of set up is so tantalizingly close, but I've reached the end of my ability to troubleshoot this - what am I doing wrong?

Jason you are treading in uncharted territory my friend! I'm impressed you have already tweaked the multisite configuration to get as far as you have, but I'm really unsure why Nginx would behave differently than Apache.

The only thing that springs to mind is that Nginx typically uses $_SERVER['HTTP_HOST'] where as Apache uses $_SERVER['SERVER_NAME'] to identify the host. Perhaps you are not taking these both into account?

That's possible, but the only place I use $_SERVER['SERVER_NAME'] is for the multi-domain part - and that works fine in both environments (although, now I'm curious as to why that is!).
Here's what my setup.php looks like:

<?php 
use Grav\Common\Filesystem\Folder;

if (isset($_GET["preview"])) {
    $preview = $_GET["preview"];
    if ($preview === 'true') {
        setcookie("preview", 'true', time()+3000, "/");
    } else {
        unset($_COOKIE["preview"]);
    }
}

if (isset($_COOKIE["preview"])) {

    // ------------- subdirectory setup -------------------```

    // Get relative path from Grav root.
    $path = isset($_SERVER['PATH_INFO'])
       ? $_SERVER['PATH_INFO']
       : Folder::getRelativePath($_SERVER['REQUEST_URI'], ROOT_DIR); 

    // Extract name of subsite from path
    $name = Folder::shift($path);
    $folder = "user-{$name}";
    $prefix = "{$name}";

    if (!$name || !is_dir(ROOT_DIR . "{$folder}")) {
        return [];
    }

    // Prefix all pages with the name of the subsite
    $container['pages']->base($prefix);
    // ------------- end subdirectory setup -------------------```
} else {
    // ------------- regular setup -------------------```

    $name = isset($_SERVER['SERVER_NAME']) ? $_SERVER['SERVER_NAME'] : "localhost";
    $folder = "user-" . str_ireplace('.local', '', $name);

    if ($name === 'localhost' || !is_dir(ROOT_DIR . "{$folder}")) {
        return [];
    }
// ------------- end regular setup -------------------```
}

return [
    'environment' => $name,
    'streams' => [
        'schemes' => [
            'common' => [
                'prefixes' => [
                    '' => ['user'],
                ]
            ],
            'user' => [
               'type' => 'ReadOnlyStream',
               'prefixes' => [
                   '' => ["{$folder}"],
               ]
            ],
            'plugins' => [
                'type' => 'ReadOnlyStream',
                'prefixes' => [
                    '' => ['user://plugins', 'common://plugins'],
                ]
            ],
            'themes' => [
                'type' => 'ReadOnlyStream',
                'prefixes' => [
                    '' => ['user://themes','common://themes'],
                ]
             ]
        ]
    ]
];

Hurrah! It's readable!

Ha ha! Only took a few tries!

One more clarification, if I clear the cache with bin/grav clear-cache then the page shows up again - at least until I refresh.

I think the problem might relate to the way Nginx handles rewrites. It has a much more manual approach to rewriting that relies on each 'site' having its own configuration setting, where as Apache can handle rewrites dynamically.

I'm just not sure what your doing is possible on Nginx without a custom rewrite configuration for each site. Unfortunately i'm somewhat limited in my nginx skills. I know enough to get a platform like Grav working, however have no clue about any kind of multisite setup on Nginx.

You might want to look at running Nginx in a reverse proxy configuration to Apache. This way you get the benefits of Nginx for fast response and load, and the benefits of Apache for .htaccess support and simpler setup.

And now the foot is on the other hand! I'd be willing to write custom server blocks for each site, but like you, my Nginx is nascent at best (this is my first Nginx project). I think, at this point, I'm as much curious about what going on as I am to get it working. What would you recommend as the next step in troubleshooting?

In my experience Nginx is best left for single sites, or a few sites. I have MANY installs on my local setup and would never inflict Nginx on myself.

If your looking to do multiple sites, I would definitely look to install Apache either by itself, or in the reverse Proxy configuration I mentioned above.

I really can't give you much more advice than that because I just don't have the multi-site experience for Nginx, so i'm not even sure what to tell you regarding that.

The issue does appear to lie at the feet of the webserver though. It works for you in your Apache configuration, so it's highly likely its an Nginx configuration issue. Hence, use what you know works, and that's Apache.