• Whiter glaciers/permanent snow with blue contour lines. I talked about this in a previous post.
• Change forest color depending on density (only in Yosemite and Sequoia/King’s Canyon NP so far). I also mentioned this in a previous post.
• Non-SPS peak names.  Discussed earlier as well.
• Pass names.
• Trail names, mostly in the NPs.
• Removed many bogus ‘lakes’ which were actually mischaracterized permanent snow.
• Added styling for scree, talus, and meadow/marshes.
• Changed font for SPS peaks to slightly larger, darker, and italic to set them apart from non-SPS peaks.
• Not a tile change, but I added UTM coordinate display in lower right.

Here’s an image showing most of these new features:

v2 Sample

Note that I screwed up the land cover in Yosemite National Park by accidentally styling it with two sets of landcover data.  Obvious side effects of this are that the whole area is generally too green and the permanent snow sections look funny.  It will take me a few days to fix this.

Enjoy the new tiles!

This all started with California GNIS data.  After filtering down to just the summits, I converted it to a shapefile and threw it on the map expecting it to work out pretty well.  Not so much.  Here’s a good example of what I got:

Bad GNIS Locations

As you can see the GNIS summit locations of Columbine Peak and Isosceles Peak are nowhere near where they should be.  While this example is particularly bad, I’d say that almost all of the summits were unacceptably displaced from their true locations.

I knew early on that I’d want the ability to manually modify features on the map so this became the impetus to make that happen. First, I needed a back-end to store my modifiable point data.  Borrowing some ideas from OpenStreetMap’s schema, I went with a simple two-table design that stores points and key/value data.  I’d really like to have something like OSM’s revisioning system but at this point that’s overly complex.  Here’s my simple schema:

mysql> desc points;
+-----------+---------+------+-----+---------+----------------+
| Field     | Type    | Null | Key | Default | Extra          |
+-----------+---------+------+-----+---------+----------------+
| point_id  | int(11) | NO   | PRI | NULL    | auto_increment |
| latitude  | double  | NO   |     | NULL    |                |
| longitude | double  | NO   |     | NULL    |                |
+-----------+---------+------+-----+---------+----------------+

mysql> desc point_tags;
+----------+--------------+------+-----+---------+-------+
| Field    | Type         | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| point_id | int(11)      | NO   | PRI | 0       |       |
| tag      | varchar(255) | NO   | PRI | NULL    |       |
| value    | varchar(255) | YES  |     | NULL    |       |
+----------+--------------+------+-----+---------+-------+

Next I coded up some CRUD functions to bridge the gap between Python and MySQL.  This allows me to treat objects in my point database as simple Python dictionaries.  From here it was pretty trivial to import whatever GNIS data I wanted and stuff it in my database.

So far OpenLayers had proven to be pretty easy to use so I decided to give it a go as my editing front-end.  Next step was a layer for getting between the front- and back-ends.  A friend of mine suggested that writing a WSGI application would be about as simple as it gets and he was right.  After about another 100 lines of code I had a mechanism for loading and storing data from a web-page using JSON as the intermediary format.

After a bit of hacking on OpenLayers I had my end-to-end (albeit a bit fragile) solution.  I could select any point, drag it to it’s correct location (oftentimes referring to the USGS topos), and save it back to the server.  Getting to this point was fun as I got to toy around with several new technologies.  But now the grunt work…

The problem is that there are about 2000 named summits in the bounds of my map.  Ideally I would look at each one and decide where it should be plotted on the map.  I started this process and realized that it was going to take a while without help (want to help? send me an email at dan at closed contour dot com).  Narrowing my focus to summits in the vicinity of SPS peaks allowed me to actually finish the job without going insane.  In the process I realized that my existing SPS peak locations weren’t ideal (see below) so I fixed those too.  (Random side note: it turns out that there are only 11 SPS peaks that aren’t officially named in GNIS.)

Bad Mount Humphreys Location

With that work behind me I moved on to locating all of the GNIS ‘Gap’ entries — passes, saddles, etc. There are only a couple hundred of these guys so it wasn’t so time consuming.  I should have new tiles with these new point features up by the end of the weekend.

At this point I feel like I have a workable system for modifying point-based data on the map.  There’s a bunch of point features I want to add in the near future: trailheads, campgrounds, and glaciers (all13 of them).  Then, it’s on to line data…

Here’s a prime example: contour lines on glaciers (or, in California, permanent snow).  Take a look at the area around North Palisade:

USGS Topo

And now the same area as rendered in the v1 SPS Map tiles:

Closed Contour

See the problem?  On the USGS map, contour lines on the Palisade Glacier are light blue, as is the outline.  Even though the glacier is white on the Closed Contour map you can barely see it.  I decided that my maps needed this feature, both for usability and aesthetics.

Another motivation for making this map was to learn some new technologies.  Sure, I’d written a little SQL before and I’m pretty familiar with geospatial software and techniques, but I hadn’t ever really used PostgreSQL and/or PostGIS.  This would be a great opportunity to try to do something useful purely in SQL.

A little background: at this point all of the map data (except the hillshade raster)  are stored in a haphazardly organized PostgreSQL with PostGIS database; each data source with it’s own table.  Actually, in the case of contours I have them separated into 3 tables, one for 40′, one for 200′, and one for 1000′ intervals.  This is an artifact of the contour generation/import process and something I’ll hopefully clean up.  There’s also a polygon land-cover table.

So all I need to do is to iterate over the snow/glacier land-cover polygons, find all contours inside them, slice them to be exactly inside the snow and mark them as such, and then subtract out the contours from the original data set.  No problem, right?  Here’s how I accomplished that in SQL:

CREATE OR REPLACE FUNCTION find_snow_contours()
    RETURNS  void AS $$
DECLARE
   contour_line RECORD;
   snow_poly RECORD;
   g geometry;
BEGIN
   -- Holding place for contours that are purely contained by snow/glacier
   DROP TABLE IF EXISTS temp_snow_contours;
   CREATE TABLE temp_snow_contours (like contours_40);

   -- Iterate over all snow polygons ('snow' is a temporary table with just snow polygons')
   FOR snow_poly IN SELECT * FROM snow LOOP
      -- Iterate over all contour lines that intersect this snow polygon
      FOR contour_line IN SELECT * from contours_40 WHERE ST_Intersects(the_geom, snow_poly.the_geom) LOOP

         -- Subtract off the snow section and save resulting lines as non-snow contours
         SELECT INTO g ST_Difference(contour_line.the_geom, snow_poly.the_geom);

         -- Delete the original contour line and replace it with the remaining segments
         -- It's common for there to be many segments after the difference, hence the 'ST_Dump'
         DELETE FROM contours_40 WHERE gid = contour_line.gid;
         INSERT INTO contours_40 (elev, is_snow, the_geom)
            SELECT contour_line.elev, 0, (ST_Dump(g)).geom;

         -- Add the intersection as snow contours
         SELECT INTO g ST_Intersection(contour_line.the_geom, snow_poly.the_geom);
         INSERT INTO temp_snow_contours (gid, elev, is_snow, the_geom) VALUES (contour_line.gid, contour_line.elev, 1, g);

      END LOOP;
   END LOOP;

   -- Finally put the snow contours back in the main table.
   INSERT INTO contours_40 (elev, the_geom, is_snow)
      SELECT  elev, (ST_Dump(the_geom)).geom, is_snow FROM temp_snow_contours;
   DROP TABLE temp_snow_contours;
   RETURN;
END;
$$

I imagine that a PostGIS SQL-guru would look at this function and shudder… forgive me, I’m a newb. Anyway, this technique worked out nicely and I’m pretty happy with the result:

(L) Before, (R) After

Not sure when I’ll generate them but I’ll definitely be rolling this feature into v2 of the tiles.

1. Data
2. Tools and Infrastructure
3. Cartography
4. Publishing

Data

Finding geodata for a specific area, especially a popular area like the Sierra Nevada, is generally not too difficult.  The problem I’ve had is piecing together a cohesive dataset from many sources.  Take hiking trails for example, I put together the hiking trails from three different data sources: a Yosemite trails shapefile from the NPS, a Sequoia/King’s Canyon trails shapefile from the NPS, and a generic trails ‘extract’ from the Forest Service.

Of course, the schema for these shapefiles were all completely different.  The Yosemite and Sequoia/King’s has trail names, the Forest Service stuff had none.  While the Forest Service data had large chunks missing in the National Park regions there was still considerable overlap that I manually cleaned up.  After a few hours of work I have a single shapefile that covers the trails of my region of interest but that’s just the beginning.  I want all of these trails (at least the big ones) to be labeled nicely and I’m going to have to go through and do that manually.

Similar story for just about every other substantive data layer:

• TIGER road data is variable by year with some of the earlier stuff having more detail in the dirt/private roads that I care about and I still haven’t found a public data source that indicates whether a stretch of road is paved or not.
• I found many sources of populated places, some have way too many places, some not enough, some with population, some without.
• Vegetation, hydrology, GNIS — all have consistency and accuracy problems.

Hopefully I’ll get a chance to go into detail on each of the above data layers in separate blog posts.  Suffice it to say that data wrangling, processing, converting, munging, and fact-checking has taken a large chunk of time so far and will continue to do so for the foreseeable future.

Tools and Infrastructure

The next challenge is the ‘engineering’ part, dealing with all of the software to create approximately 220,000 map tiles.  The software I’m relying on:

• Mapnik — the backbone of the operation, it generates the map tiles,
• GDAL/OGR — for data munging,
• Postgres/PostGIS — for storing my processed data for Mapnik,
• Eclipse with PyDev — for writing my Mapnik scripts,
• GlobalMapper — it may have a painful UI at times and it’s not free but it’s a quick way to view data and can convert just about any geodata format.

I feel like that, for the most part, the bulk of my work is done in this department.  I’ve got a decent setup where I can test out cartographic ideas on my home machine and do the processing ‘in the cloud’ (see below).

Cartography

I think munging data will be the most time consuming piece of the process but that’s generally easy work.  The hard part is the cartography.  A map is only as good as each of the decisions that was made in creating it.

Should the hillshading be darker/lighter?  Should I use ‘Google Mercator‘ or something better?  Should I include 200′ contour lines at this zoom level?  At what scale should the river flow stroke get smaller?  How much should I accent the SPS summits versus other summits?  There are thousands of such decisions to make.  I plan on writing about these decisions and why I made them one way or another.

Publishing

How do I go about creating my tiles and publishing them?  This falls in the ‘engineering’ category as well and has essentially nothing to do with cartography.  Personally I quite enjoyed learning about all of the potential options but I can easily imagine would-be cartographers not sharing their work because of the technological burden.

I’ll write more about it later but the bottom line is that I’m using virtual machines on Amazon’s EC2 to generate the tiles (takes about 6 hours on a medium instance) and storing them on S3 (takes another 6 hours to upload them) where they are served via CloudFront.  I then have a cheap web host to serve the static OpenLayers-based map viewer.  Time will tell how expensive this is…

So there you have it.  I feel like I have the infrastructure and publishing pieces solved so now I can focus my energy and time on the data and cartography and eventually produce a map that achieves my goals.

• User-specific data.  I’m into peak climbing and I want my maps to be focused on that.
• Aesthetically pleasing.  A topographic map should look good at every scale and viewing distance.
• Editable.  Ideally the map should be editable in a wiki style (like openstreetmap.org) but I’ll settle for just me being able to make changes.
• Consistent.  The units, contour interval, typography, coloring, scale, etc. should all be consistent across the entire mapped area.
• Printable.  The map should be printable at an accurate scale.
• Web interface.  This isn’t too much to ask, there’s plenty of good mapping interfaces (even open source) out there and you can already find the USGS topos in them.

There’s no single solution that meets all of my needs right now so I’m going to try to build one.  This blog will chronicle how that goes.