Skymodels With rcp: A Couple Quick Notes

Morgan Hite's great write-up about using rcp and skymodelling in Linux has reminded me that there were a couple "Oh, I'll just throw that in there for now" programming decisions that have taken on longer lives of their own. I wanted to list some of them here for your edification.

Several of these will be out-of-date at some point as I improve the program, but for now I hope this helps alleviate any gotchas. 

A random snip from a current project, because the rest of this post is going to be text heavy...

Cross-Platform Compatibility
The core of rcp is platform independent thanks to Python's os and subprocess modules doing all the heavy lifting for us. The normal blur methods work just fine, as does TPI and the actual processing bits of skymodelling. However, the mdenoise.exe and skylum.exe programs are obviously compiled for windows.

Dealing with mdenoise in Linux or MacOS
The big gotcha here is that rcp currently expects you to set a path to the mdenoise executable in settings.py and will complain if that file does not exist. You can compile mdenoise for yourself if you want to. Or, if you don't want to use mdenoise, you can simply point it to any file that exists (though as I type that, I realize there may be security complications of pointing it at any file). On *nix you could do a touch ~/fake_md to create a fake, empty file and then point to that file in settings.py. However, be aware of the security implications of that file becoming an executable. chmod may help here, but there's still a risk.

I'll try to rewrite the check so that it only looks for the file if you actually run -m mdenoise.

SkyLum via Wine
Morgan Hite was able to run SkyLum in Ubutu via Wine. It seems like a simple enough program that is tightly compiled (very little reliance on outside dll's) so I imagine it shouldn't be too difficult to get it to run under Wine in other Linux distros or on MacOS, but I haven't verified this.

Skymodel Elevation Exaggeration Factor
Kennelly and Stewart's demo program applied a multiplier of 5 to the elevation before processing the hillshades/shadows. As I recreated their example in a parallel processing-capable environment, I kept this value hard-coded in. However, you can play around with this to your hearts content by modifying that line of code in methods.skymodel() (it uses the literal value 5 near the top of the method).

In the future, I'll expose this as an optional command line switch and value.

Shadows Optimization
By far the most computationally intensive part of the skymodel is calculating shadows. My current algorithm loops through each cell of the chunk (ie, not the areas read in as part of the overlap) and starts walking back towards the light source defined by the azimuth and elevation for that row in the luminance file, checking if the elevation of the cell for each step is high enough to block the view of the sun from our current cell (fun with trigonometry!).

Because this could go on forever, I had to choose a ending point at which it stops looking for each cell. After some very un-scientific trial and error, I came up with 600 steps, where each step is the distance of the source raster's cell size in the direction defined by the azimuth.

Once it finds a cell that shadows our source cell, it stops checking for that cell and marks every cell between the source and the shadowing cell as being in shadow. Then, as part of the loop that checks every cell, if it determines that source cell is already shadowed, it skips the shadow check. This helps reduce the number of ray tracing loops that have to be run or the length of each loop.

Shadows and Overlaps
Because the shadowing method is limited to 600 steps, and thus the highest number of possible cells that could be checked is 600 (if the light source is directly N, S, E, or W), the overlap for skymodel should always be set at 600. Higher values will just consume more resources with no gain, and lower values will produce inconsistent shading at the chunk edges.

In the future, I'll set this overlap automatically so the user doesn't need to worry about that.

A Final Anecdote About Azimuth
One of the challenges in implementing the hillshade algorithm was sorting out the difference between compass bearings and mathematical angles. On a compass, 0 degrees is due north, and the angle increases as you go clockwise. East is 90 degrees, south east is 135, and so on. This is how we usually think of angular directions on a map.

However, the unit circle that all trig functions are based on works differently. It puts 0 degrees (or 0 radians) on the x-axis one unit away from the origin (or, in map terms, due east). It then increases as you go counter clockwise- north is 90 degrees, west is 180, etc.

It took me a lot longer than it should have to realize that the conversion from our map degrees to mathematical degrees is 90 - map_az. That is all.

Wasatch Front Water, Part 1: Experimentation and Finding Focus

This is the first post in (what should be) a series of posts about my latest map, That the Desert May Blossom as the Rose. Today I'm introducing the map and reviewing the background and inspiration behind it. In the future I'll write a little bit more about the techniques I used to realize different ideas and messages in the map.

Click here for a larger, compressed-quality image

Experimentation
About a year ago, right after fixing my skymodel implementation, I started exploring different options for a map of the Wasatch Front. I find that sometimes I need to just load different datasets and explore them until I get inspiration for a map.

I started with the National Land Cover Dataset (NLCD), which has the potential to be a fun basemap if you can do something with the visually-strong reds they use for urban areas. I sandwiched that between a slope-combined hillshade and a skymodel with strong shadows and got excited about how the mountains looked. I put a roads layer down to help me get my bearings.

Water Flows Downhill
Adding the NHD water basin layer was what gave me my first flash of inspiration for this map. I realized the water basins didn't always line up with my mental map of relationships between areas. It's amazing just how much overland transportation, especially the modern interstate system, hides natural boundaries in our mental maps.

For example, Park City is more related to Ogden, hydrologically speaking, than Salt Lake City. Downtown Park City drains into Silver Creek, which runs into the upper Weber River just below Rockport Reservoir. Kimball Junction and Jeremy Ranch drain into East Canyon and then into the lower Weber River 30 miles downstream. At the top of Parley's summit, the canyons drain down to the Jordan River. Water on the west side of the summit doesn't meet Park City water until the middle of the Great Salt Lake.

If you were to graph out the dendritic drainage structure of the Great Salt Lake Basin, Park City and Kimball Junction would be on hydrologically separate branches despite being physically right next to each other. Parley's Canyon is on completely the opposite side of the trunk, and its branch doesn't meet up until the very end of the graph.

However, my mental relationship map of Park City is almost exclusively focused on I-80 and is completely linear: Salt Lake, Parley's Canyon, Parley's Summit, Jeremy Ranch, Kimball Junction, Park City. Because of how steep the climb up to Parley's Summit is (especially if you've got a 15-year-old four-cylinder station wagon), anything after that is "downhill". The slight elevation difference between Kimball Junction and downtown Park City doesn't even register, but it's enough to send water on two different paths dozens of miles apart.

Deadlines: The Mother of Productivity
Life got busy and months passed before I did anything more with this concept. Finally, at the end of 2019 I saw the announcement that submissions for the 2020 Atlas of Design were due in late January. I thought this map was my best shot, despite still not really having a solid plan. Four weeks to do a map in my spare time? Why not.

Just like photography, a good map needs to have a single focus, a single story it's trying to tell, a single theme it's trying to illuminate. Both photography and cartography are really the art of removing as much extraneous information as possible, similar to the not-really-Einstein quote "make everything as simple as possible, but not simpler." (This also applies to writing, though the length of this post is evidence that I'm much better at simplifying maps than words)

I spent two weeks working on the map with the vague theme of showing the basins. I classified streams, I cleaned up basin boundaries (the NHD doesn't know what to do with the manufactured wetlands in the Jordan River delta), I played with elevation color ramps.

Finding Focus
The NHD flowlines include a couple of major artificial pathways, like the Salt Lake City Aqueduct running from the mouth of Provo Canyon to the mouth of Parley's Canyon. Being an avid user of the Murdock Canal Trail, I knew there were more aqueducts and pipelines that aren't in the NHD. I started hunting down alignments from water conservancy district webpages.

It was after a couple days down this rabbit hole that I realized I was more interested in making a map about all the water works delivering mountain water to the valley cities. This became my new, sharper focus.

I was struck with renewed awe that we move water not just between different branches of the Great Salt Lake Basin (like the Weber-Provo Canal in Kamas) but between the Colorado/Pacific Basin and the Great Basin via the Strawberry and Diamond Fork systems. I wanted to show this, while also showing off the majesty of the Wasatch Front.

So, I guess, in the end, I really have two goals: a) Show the extensive water systems, and b) Show off my hillshading/terrain techniques. I hope that I managed to do b) without impacting a) too much. I can definitely do a different style of map for a), and I've got some ideas for that already.

But for now, such as it is, it is That the Desert May Blossom as the Rose.