Data Points

Imagine your typical birthday cake:  2 cake layers, separated by a layer of cream filling and topped with icing, maybe some berries and candles. Now imagine spatial data – it's better for you than cake anyways (sugar is bad for you) and it has layers too.    

In data/mapping we think and work in layers all the time. We typically source, produce, and perform quality control based on individual data layers.  A layer of POIs, another of streets, yet another of parcels.  Entire products can be a single layer. We might source a 'layer' of property attributes from a company x, then a 'layer' of consumer data from company y.

We sourced a homemade birthday cake similarly (we buy some cake mix, buy some icing ingredients, some berries from a local farm and candles). But when the candles are blown out we then eat the entire cake, not as layers, but as the entire cake.  Sure, everyone has a favorite layer of the cake, but the quality of the cake depends on all those layers being thoughtfully and properly integrated.

Similarly, spatial data rarely consume one layer of data at a time; instead they integrate multiple layers to solve business problems.  While we cannot ignore the sourcing, production and quality control of the data elements in an individual layer, we need to do that next (frankly more difficult) step and check the interplay between the layers as a consumer uses them.  To do this properly, we must think like that data consumer who uses multiple layers at a time, and that's where things tend to break down.  Organizational lines (or product lines) frequently are drawn along data layers, and you can bet that folks are going to focus on their part of the business. This means that over time quality gaps will develop between the cooperation of the layers. You don't want the customer that tries to integrate the layers to be the one to discover that the layers are not properly integrated… 

I offer 3 types of checks to help detect cross-layer issues: 

1. Spatial coherence between the layers. Do the data elements in the layers appear to have acceptable absolute and relative accuracy between them? Relative accuracy issues increase in criticality as you add layers, as all the layers need to work well together.   For example, do your buildings, address points and POIs spatially align, or do they look like 3 disjointed layers?  What do your roads, water polygons and parcels combined reveal?  Specifications and checks that are designed to check spatial relationships such as these should be considered.
2. ID interoperability between the layers. Do you have shared IDs between the layers? Ideally you would and that allows them to be easily linked to each other.  If you do, how well do the shared IDs match between data layers?  What is the completeness between them?    Do they reveal completeness issues in an individual layer?
3. Correctness issues revealed by the combination of the layers. A lack of standardization and duplication of data elements in multiple layers both can be easily seen.  For example, an address delivered on each of 3 layers (for example, address points, POIs, and property attributes) may be different for each layer and reveal correctness issues.  A parcel with a data value indicating vacant land combined with a layer showing a building and a POI associated with that parcel shows something is wrong. Specifications and checks that are designed to check correctness issues such as these should be considered.

In the end, both the cake eater & the data consumer really care only about the end product – not just the individual layers.  We should always keep that foremost in mind.

Imagine your typical birthday cake:  2 cake layers, separated by a layer of cream filling and topped with icing, maybe some berries and candles. Now imagine spatial data – it's better for you than cake anyways (sugar is bad for you) and it has layers too.    

In data/mapping we think and work in layers all the time. We typically source, produce, and perform quality control based on individual data layers.  A layer of POIs, another of streets, yet another of parcels.  Entire products can be a single layer. We might source a 'layer' of property attributes from a company x, then a 'layer' of consumer data from company y.

We sourced a homemade birthday cake similarly (we buy some cake mix, buy some icing ingredients, some berries from a local farm and candles). But when the candles are blown out we then eat the entire cake, not as layers, but as the entire cake.  Sure, everyone has a favorite layer of the cake, but the quality of the cake depends on all those layers being thoughtfully and properly integrated.

Similarly, spatial data rarely consume one layer of data at a time; instead they integrate multiple layers to solve business problems.  While we cannot ignore the sourcing, production and quality control of the data elements in an individual layer, we need to do that next (frankly more difficult) step and check the interplay between the layers as a consumer uses them.  To do this properly, we must think like that data consumer who uses multiple layers at a time, and that's where things tend to break down.  Organizational lines (or product lines) frequently are drawn along data layers, and you can bet that folks are going to focus on their part of the business. This means that over time quality gaps will develop between the cooperation of the layers. You don't want the customer that tries to integrate the layers to be the one to discover that the layers are not properly integrated… 

I offer 3 types of checks to help detect cross-layer issues: 

1. Spatial coherence between the layers. Do the data elements in the layers appear to have acceptable absolute and relative accuracy between them? Relative accuracy issues increase in criticality as you add layers, as all the layers need to work well together.   For example, do your buildings, address points and POIs spatially align, or do they look like 3 disjointed layers?  What do your roads, water polygons and parcels combined reveal?  Specifications and checks that are designed to check spatial relationships such as these should be considered.
2. ID interoperability between the layers. Do you have shared IDs between the layers? Ideally you would and that allows them to be easily linked to each other.  If you do, how well do the shared IDs match between data layers?  What is the completeness between them?    Do they reveal completeness issues in an individual layer?
3. Correctness issues revealed by the combination of the layers. A lack of standardization and duplication of data elements in multiple layers both can be easily seen.  For example, an address delivered on each of 3 layers (for example, address points, POIs, and property attributes) may be different for each layer and reveal correctness issues.  A parcel with a data value indicating vacant land combined with a layer showing a building and a POI associated with that parcel shows something is wrong. Specifications and checks that are designed to check correctness issues such as these should be considered.

In the end, both the cake eater & the data consumer really care only about the end product – not just the individual layers.  We should always keep that foremost in mind.