Well, someone could quickly look behind them. It is all about viewing area, and not predictive of your position.
Jon, I think some bgls are probably thrown out shortly after they are loaded. It is more useful not to think in terms of BGL, but in terms of runtime data. A lot of data is loaded from the bgl, processed into some other form, and the bgl is thrown out immediately even though that runtime derived data is still in memory. The terrain system also makes a lot of use of LRU cache of data, so even though no data from a file is being used, it might still be loaded just because some newer data hasn't been loaded in yet.
Autogen is a good example of this. When an AGN file is needed for a particular level 15 qmid it gets loaded and put into an LRU cache. It is then used to generate autogen placements for that particular spot in the world using the AGN file as a template. Afterwards the AGN is no longer used, but it is still in memory because it is in the LRU cache. The generated autogen placements for the qmid is in memory as long as you are in the ring of autogen, even if the AGN gets unloaded. If enough other AGN files are loaded, eventually the original one will be removed form the cache and unloaded. Unless there is another 15 qmid cell in the area that uses the same AGN, in which case it is either used from LRU cache if it is still there, or it just gets loaded from disk again.
Terrain textures work in a similar way. DEM, OBX, and vector data probably don't use a cache since they are specific to a certain spot on Earth.