W.I.P. Tool for creating Elevation using Rectangles

[FoxtrotScenery]

Hey everyone,

With the disappearance of existing tools for creating elevation rectangles from a DEM source as well as some recent problems with the newly introduced terraform method, I've started work on a small tool which can convert a DEM source image to Rectangle objects with a heightmap. I currently have a process which works with a WGS84 raster as input and it can convert said raster to a Rectangle with customisable resolution. At a resolution of 25 metres, it took about 5 seconds to process the entire island of Lesvos, which is just over 4000 square km of data. With a lot of headroom for optimisation, I think it's promising for processing large sets of elevation data.

Now the question arises, is anyone interested in such a tool and what features (specifically related to elevation, not yet planning on including anything else) would you really need to have included in the tool? This is a side project to my scenery development but since I really needed to edit the elevation, I created this tool to streamline the process and I want to share this with the community.

Here's some of the features I am considering myself right now:

• Process all raster projections instead of just WGS84
• Merge and blend raster files at different resolutions (for localised high res terrain)
• Import 3D objects and convert to Rectangles (for manual sculpting in 3D software)
• Merge generated Rectangles into MSFS project XML
• Drape runways over customised terrain
• Export existing rectangles to a 3D object for object placement

Any thoughts? Let me know!

 

[jonmkj]

Just now seeing this. LOVE that you're moving on this project. Looking forward to trying it out.

 

[FoxtrotScenery]

Seeing the votes come in the past two weeks, I have been working steadily towards a very basic first version which is purely aimed at generating evenly distributed elevation rectangles without any fancy additional things. That being said, I will try and release the tool in its current state in the next two weeks so I can also see how others experience the tool and so I can track down any bugs. After that works, I want to add more functionality to the tool. As it stands things look like this:

I know, it's not the fanciest UI you're ever going to see but it does the job for now and will be expanded upon in the future as well! So for now I'm tracking down all bugs I can find for this version and I'll update this thread as soon as I have big news or a release version.

 

[FoxtrotScenery]

It seems the tool has just been approved as a resource, so for those who want to try it you can now do so! Please be aware this is the earliest version and I only know it works on my own laptop and desktop. That being said, please give it a try and let me know about any bugs and/or problems with the tool. It's expected to break here and there although that will only help me out with future versions. Nonetheless, enjoy and look forward to updates!

Edit: Please make a backup of your scenery xml file. I haven't encountered any problems but you might.

 

[superspud]

Superb.

Just to double check I'm not doing something stupid, it can only load one raster image at a time?

Edit - that's confirmed in the description.

It certainly works quickly and smoothly. I had to convert my raster image to vanilla WGS84 in Qgis for it to digest.

I can't use any of the results, in my area at least, because of Asobo's stunning mediocrity in creating a terraforming falloff system that works, but hopefully one day I can.

 

[FoxtrotScenery]

Superb.

Just to double check I'm not doing something stupid, it can only load one raster image at a time?

Edit - that's confirmed in the description.

It certainly works quickly and smoothly. I had to convert my raster image to vanilla WGS84 in Qgis for it to digest.

I can't use any of the results, in my area at least, because of Asobo's stunning mediocrity in creating a terraforming falloff system that works, but hopefully one day I can.

Thanks for checking out the tool!

The falloff problem annoys me big time as well. I am looking into something to solve that problem by creating elevation rectangles beyond the region of interest, sampling the default terrain and then using that to blend between the user's and the default terrain.

However, there's no real quick way to do this without user input (i.e. the user has to manually add a heightmap to all these "blending rectangles"). Alternatively, there is a way to control applications like MSFS by hooking into Windows and basically clicking for the user. This feels kind of intrusive since the tool would technically be able to perform actions on your PC. I personally don't like the idea of that at all but I would like other people's opinion on that as well.

For now though, I'm focusing on multi raster and projection support.

 

[superspud]

Thanks so much for making it. It's been a long wait for an alternative.

In some areas I have managed to get DEM rectangles to behave smoothly in the past, but that's the minority and I'm not doing anything different to my knowledge. It either works or it doesn't.

Since the system is so godawful and unpredictable I wonder whether it's possible to design around it. You will go down in history if you can.

The created rectangles do usually respond well to manual adjustment after creation and I've managed to disguise the falloff cliffs, but sometimes the area will be too big and it's horribly fiddly.

 

[Bavarello]

Thank you for this tool of yours.
I went to test it immediately with a 1° x 1° extension raster file that I use to create terrain meshes on large areas. Briefly, what I got.
I loaded a DEM file with these features
Width 11449
Height 6375
Data type Float32 - Thirty two bit floating point
GDAL Driver Description GTiff
GDAL Driver Metadata GeoTIFF

More information
AREA_OR_POINT=Area
Dimensions X: 11449 Y: 6375 Bands: 1
Origin 35.9869569360279442,47.0204720954516944
Pixel Size 8.983152656583115458e-05,-8.983152656583115458e-05

Result: I had to interrupt the process after almost half an hour as processing was not completed (I had an .xml file of almost 2 GB)

So I tried with this DEM:
Width 859
Height 1267
Data type Byte - Eight bit unsigned integer
GDAL Driver Description GTiff
GDAL Driver Metadata GeoTIFF

More information
AREA_OR_POINT=Area
Dimensions X: 859 Y: 1267 Bands: 1
Origin 34.9684470668932335,47.0326891833194125
Pixel Size 0.001236570687689999117,-0.0008535054894481241281

The processing was successful, generating the .xml file and subsequently the bgl.

Looking at the structure of the .xml file in particular the "priority" attribute instead of being set to 0 (zero) autoincrements in relation to rectangles... it's not quite like in the SDK.

 

[FoxtrotScenery]

Thank you for trying out the tool and giving feedback. I'm actually glad you found a problem so I can make it more robust. Could you maybe privately share the DEM with me so I can take a look since I haven't had problems with large files yet.

Now in terms of XML size, large elevation rasters will produce large xml file due to the way the data is stored using strings instead of 32 bit floating points. If I oversimplify it a bit, a floating point number represented as a string like this "62.47328906921808" requires 18 characters or 18 bytes, but representing it as a 32 bit floating point in a raster file only requires 4 bytes, just like in code. Right now I can maybe see if I can truncate the number after a certain amount since I don't see how this amount of precision is necessary so thanks for mentioning that!

Right now I recommend working in a separate project for elevation so it doesn't overload your scenery XML for airports or model libraries.

Edit:

Looking at the structure of the .xml file in particular the "priority" attribute instead of being set to 0 (zero) autoincrements in relation to rectangles... it's not quite like in the SDK.

This is some leftover code from testing I guess, will fix that in a patch the coming days.

 

[Bavarello]

Hi @FoxtrotScenery,

I successfully tested vers.1.1 alpha with a GeoTiff, horizontal spatial resolution at 10 meters, Data type Float32 - Thirty two bit floating point
The whole process only took 14 minutes with my hardware.

 

[thetford569]

Just tried this tool out this week and it worked well. Only thing is the rectangles generated deleted all the photogrammetry underneath. But when opening in the editor there isn’t anything checked that would delete the photogrammetry. I checked the XML as well and didn’t see anything about removing TIN or photogrammetry. I don’t want to lose the photogrammetry buildings around my airport so is this expected behavior?

 

[salud]

Hi FoxtrotScenery, thank you so much for this tool! It is exactly what I was looking for recently while working on some helipad sceneries. The tool works perfectly, and the result with 1m DEM is stunning.

 

[peacefarm]

Hi FoxtrotScenery, thank you so much for this tool! It is exactly what I was looking for recently while working on some helipad sceneries. The tool works perfectly, and the result with 1m DEM is stunning.

Hi there! Would you mind sharing a basic over of how you obtain and process the raster image or maybe some tutorials links?

This seems like a wonderful tool, but I am stuck at the basics. Any help would be appreciated!

Thanks!

 

[rhumbaflappy]

There is a problem if the GeoTiff has nodata values. This is usually caused by a sea-level mask on the data, setting the value to -9999. It's a problem more with the tiff than with the app. The nodata in the tiff would need to be changed to 0.0 to solve the problem.

 

[salud]

Hi there! Would you mind sharing a basic over of how you obtain and process the raster image or maybe some tutorials links?

This seems like a wonderful tool, but I am stuck at the basics. Any help would be appreciated!

Thanks!

Sure, I'll try to walk you though my process, though I am sure there are many different paths that one may go.

Step 1: Get raster data
This may already become quite difficult. For the US, the USGS National Map Viewer is a great source, and depending on the are you may find 1 arc data (~24m resolution), 1/3 arc (~8m resolution) or 1 m data. Another source is Opentopography. Once you found an appropriate data set, you can usually download a GeoTIFF.

Step 2:
The downloaded file usually covers a much larger area than what you need. Now comes a part that is a bit tricky: extracting a section of the GeoTIFF. For this purpose I use the free tool QGIS (https://www.qgis.org/). There you can import a raster layer such as the downloaded GeoTIFF via Layer -> Add Layer -> Raster.

Step 2+:
Then it may be helpful to import some OpenStreetMap data for orientation and reference via the QuickOSM plugin (can be installed right from QGIS -> Plugins), by Quick Query -> change In.. to Canvas extent.

Step 3:
Zoom to the section of the terrain that you want to use in MSFS, then right click on the Layer -> Export -> Save as..., then specify an export file name, choose WGS84 under CRS, click once on Map Canvas Extent, then Ok.

Step 4:
Now you have a smaller GeoTIFF file of your area of interest, which you can import into this tool, so from here follow Foxtrot's instructions.

This process works for me, though it may not be the simplest, and for sure it is not the only one.

 

[peacefarm]

Sure, I'll try to walk you though my process, though I am sure there are many different paths that one may go.

Step 1: Get raster data
This may already become quite difficult. For the US, the USGS National Map Viewer is a great source, and depending on the are you may find 1 arc data (~24m resolution), 1/3 arc (~8m resolution) or 1 m data. Another source is Opentopography. Once you found an appropriate data set, you can usually download a GeoTIFF.

Step 2:
The downloaded file usually covers a much larger area than what you need. Now comes a part that is a bit tricky: extracting a section of the GeoTIFF. For this purpose I use the free tool QGIS (https://www.qgis.org/). There you can import a raster layer such as the downloaded GeoTIFF via Layer -> Add Layer -> Raster.

Step 2+:
Then it may be helpful to import some OpenStreetMap data for orientation and reference via the QuickOSM plugin (can be installed right from QGIS -> Plugins), by Quick Query -> change In.. to Canvas extent.

Step 3:
Zoom to the section of the terrain that you want to use in MSFS, then right click on the Layer -> Export -> Save as..., then specify an export file name, choose WGS84 under CRS, click once on Map Canvas Extent, then Ok.

Step 4:
Now you have a smaller GeoTIFF file of your area of interest, which you can import into this tool, so from here follow Foxtrot's instructions.

This process works for me, though it may not be the simplest, and for sure it is not the only one.

Hello Salud, thank you so very much for taking the time to share such a detailed explanation - it definitely gets me up and going!

Im exploring using LiDAR data and converting that to a DEM file. There is some awesome free data out there (USGS) that has the finest of detail like small canals and bumps along the airport surfaces.

I'll be sure to share my findings with you all if I do manage to make it work.

@FoxtrotScenery thank you so much for your efforts and for sharing this gift with us!

 

[spatialpro]

There is a problem if the GeoTiff has nodata values. This is usually caused by a sea-level mask on the data, setting the value to -9999. It's a problem more with the tiff than with the app. The nodata in the tiff would need to be changed to 0.0 to solve the problem.

Easy to do in GlobalMapper. QGIS might do this easily too, but I haven't investigated (anyone know how already?).

 

[rhumbaflappy]

QGIS might do this easily too, but I haven't investigated (anyone know how already?).

 

[salud]

Hello Salud, thank you so very much for taking the time to share such a detailed explanation - it definitely gets me up and going!

Im exploring using LiDAR data and converting that to a DEM file. There is some awesome free data out there (USGS) that has the finest of detail like small canals and bumps along the airport surfaces.

I'll be sure to share my findings with you all if I do manage to make it work.

@FoxtrotScenery thank you so much for your efforts and for sharing this gift with us!

Sounds interesting, I kept wondering whether Lidar data may be useful for this kind of work. As I understand, it is basically like precise photogrammetry but without the imagery, so I am very curious to hear about your findings. Good luck!

 

[peacefarm]

Sounds interesting, I kept wondering whether Lidar data may be useful for this kind of work. As I understand, it is basically like precise photogrammetry but without the imagery, so I am very curious to hear about your findings. Good luck!

Hello again. I managed to create a georeferenced tiff from the lidar LAS file with WGS84 CS. I followed the instructions by the dev, but running into issues when I try to open the XML up in the scenery editor. Process in the app says it created 9 rectangles, but when I open the XML in the sim, I get the following error message. I also opened the XML with notepad and do see all the points written to it. I feel I'm doing something wrong ... perhaps a tiny setting somewhere. Have you or anyone else run into this before?

Thank you!