Saturday, May 9, 2020

Skymodels With rcp: A Couple Quick Notes

Morgan Hite's great write-up about using rcp and skymodelling in Linux has reminded me that there were a couple "Oh, I'll just throw that in there for now" programming decisions that have taken on longer lives of their own. I wanted to list some of them here for your edification.

Several of these will be out-of-date at some point as I improve the program, but for now I hope this helps alleviate any gotchas. 

A random snip from a current project, because the rest of this post is going to be text heavy...

Cross-Platform Compatibility
The core of rcp is platform independent thanks to Python's os and subprocess modules doing all the heavy lifting for us. The normal blur methods work just fine, as does TPI and the actual processing bits of skymodelling. However, the mdenoise.exe and skylum.exe programs are obviously compiled for windows.

Dealing with mdenoise in Linux or MacOS
The big gotcha here is that rcp currently expects you to set a path to the mdenoise executable in settings.py and will complain if that file does not exist. You can compile mdenoise for yourself if you want to. Or, if you don't want to use mdenoise, you can simply point it to any file that exists (though as I type that, I realize there may be security complications of pointing it at any file). On *nix you could do a touch ~/fake_md to create a fake, empty file and then point to that file in settings.py. However, be aware of the security implications of that file becoming an executable. chmod may help here, but there's still a risk.

I'll try to rewrite the check so that it only looks for the file if you actually run -m mdenoise.

SkyLum via Wine
Morgan Hite was able to run SkyLum in Ubutu via Wine. It seems like a simple enough program that is tightly compiled (very little reliance on outside dll's) so I imagine it shouldn't be too difficult to get it to run under Wine in other Linux distros or on MacOS, but I haven't verified this.

Skymodel Elevation Exaggeration Factor
Kennelly and Stewart's demo program applied a multiplier of 5 to the elevation before processing the hillshades/shadows. As I recreated their example in a parallel processing-capable environment, I kept this value hard-coded in. However, you can play around with this to your hearts content by modifying that line of code in methods.skymodel() (it uses the literal value 5 near the top of the method).

In the future, I'll expose this as an optional command line switch and value.

Shadows Optimization
By far the most computationally intensive part of the skymodel is calculating shadows. My current algorithm loops through each cell of the chunk (ie, not the areas read in as part of the overlap) and starts walking back towards the light source defined by the azimuth and elevation for that row in the luminance file, checking if the elevation of the cell for each step is high enough to block the view of the sun from our current cell (fun with trigonometry!).

Because this could go on forever, I had to choose a ending point at which it stops looking for each cell. After some very un-scientific trial and error, I came up with 600 steps, where each step is the distance of the source raster's cell size in the direction defined by the azimuth.

Once it finds a cell that shadows our source cell, it stops checking for that cell and marks every cell between the source and the shadowing cell as being in shadow. Then, as part of the loop that checks every cell, if it determines that source cell is already shadowed, it skips the shadow check. This helps reduce the number of ray tracing loops that have to be run or the length of each loop.

Shadows and Overlaps
Because the shadowing method is limited to 600 steps, and thus the highest number of possible cells that could be checked is 600 (if the light source is directly N, S, E, or W), the overlap for skymodel should always be set at 600. Higher values will just consume more resources with no gain, and lower values will produce inconsistent shading at the chunk edges.

In the future, I'll set this overlap automatically so the user doesn't need to worry about that.

A Final Anecdote About Azimuth
One of the challenges in implementing the hillshade algorithm was sorting out the difference between compass bearings and mathematical angles. On a compass, 0 degrees is due north, and the angle increases as you go clockwise. East is 90 degrees, south east is 135, and so on. This is how we usually think of angular directions on a map.

However, the unit circle that all trig functions are based on works differently. It puts 0 degrees (or 0 radians) on the x-axis one unit away from the origin (or, in map terms, due east). It then increases as you go counter clockwise- north is 90 degrees, west is 180, etc.

It took me a lot longer than it should have to realize that the conversion from our map degrees to mathematical degrees is 90 - map_az. That is all.