MapInfo Pro

My first foray into using the new Multi-Resolution Raster (MRR) format was to merge the asc digital terrain model of Great Britain from Ordnance Survey? - OS Terrain 50. A 50m model made up of 8,571 tiles. Resulting in one MRR file of 188 MB.

Then I moved onto OS Terrain 5 - a 5m model with 35,649 tiles (1 million values in each) and the result was a 26.5 GB MRR file.

You can also use this to merge raster imagery and mapping. Rather than having a Seamless Table comprising 815 files of 1:50,000 scale mapping, have one MRR. Or merge 3,500 high res aerial images (Leicestershire) into a 135 GB file.

I'm guessing someone will have bigger than that if they have satellite imagery of a much larger area??

I have the Danish DSM 2015 with a resolution of 40 cm.

It was "destilled" in Australia and has travelled all over the World, from Australia via USA to the UK where I got a copy.

It's close to 200GB - a 194.5 GB to be precise.

It originates from close to 50,000 GeoTIFF files with a total size of 650GB.

As you could imagine working with 50,000 individual files would be a nightmare. But one big single MRR - works smoothly

I knew yours would be bigger!

Who's counting, John 😁

A further question is "Who can make the smallest MRR from the same source data?". I have also put together the OS Terrain 50 and my MRR is 132 MB.

There are a variety of techniques available to make your MRR files smaller, although some are not yet exposed in the Pro user interface. The size of the MRR file depends on the geometry of the storage, the treatment of the data being stored and the compression codec employed.

Geometry includes things like the tile size (for base level and overview levels) and the amount of tile wastage. Tiles don't have to exist if there is no valid data. Small tiles may reduce wastage, but large tiles may reduce overhead and improve compression ratios.

Data treatment includes - choosing the most appropriate and compact data type, appropriate per-band validity settings, decimal accuracy restriction (for floating point data types), storing floating point data as integers by employing a transformation, predictive encoding (which can make the data stream more compressible and is generally only used when the storage data type is integer).

Finally choose the best compression codec. In general, you should always choose some compression codec over having no compression at all.

Amongst the lossless codecs, LZMA generally produces the highest level of compression. But setting the compression level for this codec is vital - levels 1 - 4 behave very differently to levels 5 - 9. For imagery, PNG will often produce a smaller file than LZMA.

We currently only offer JPEG as a lossy codec for imagery. The size of the file depends on the compression level - from 0 (largest, highest quality) to 9 (smallest, lowest quality). Level 0 will generally halve the file size or better without any visible loss of quality. Quality remains very high up to level 3. For levels 4 - 9 there will be an increasing noticeable level of image quality degradation.

That's very interesting, Sam.

I would hope that we have document describing the "best" choice for a couple of scenatios.

I noticed that we now offer a more simple choice for example in the Merge window: Data Balanced, Data Speed and Data Space. And then you can also select Advanced and make your own choices.

Did you just pick Data Space?

Will the "other" techniques be exposed to the user sometime soon Sam? Thanks for the tips as I haven't experimented beyond the defaults.

@Sam Roberts? Did some testing with various compression methods on a representative aerial tile ?to test the performance of the various compression options.

I'm not sure why there was such little variation in the LZMA and ZIP output files sizes I was expecting more range. I had tested Jpeg compression previously at various levels to determine highest compression whilst retaining acceptable quality and came to the same conclusion with Level 3.

As it stands we have all our imagery available for SSA MRR consumption now with Jpeg 3 compression;

Combined our normal imagery consists of two main datasets, regional coverage @25cm GSD and towns @10cm GSD totalling 45Gb and 34Gb respectively. Pretty happy with the filesizes, about 15% larger than the previous ECWs though I'm not sure of the compression used there, before my time.

LZMA levels 1 - 4 will provide a similar level of compression to ZIP. LZMA Levels 5 - 9 will generally provide a step-wise improvement over the highest level of compression you can achieve with ZIP. We are talking percentages here, not twice as small or anything like it. You pay a high CPU cost for the gain but I generally consider that to be worth it.

Some comments on ECW :

ECW is a first class member of the Advanced raster product. The MRR format ought to be considered as a potential alternative to ECW, but not an automatic replacement. ECW is a mature and highly optimised product. It is fast and robust. ECW uses wavelet compression. We have not supplied a similar codec in MRR (one was trialled but not shipped).

I can provide some guidelines to help you choose. Firstly, if you have imagery like aerial photography, ECW will generally compress to a smaller file for a given level of image quality. Period - no arguments. But you can assess for yourself what "image quality" means. For lossy compression it simply means - what kind of noise pattern do you prefer? ECW introduces high frequency noise evenly through the entire image. JPEG compression tends to introduce higher levels of noise in areas of high variability and lower levels of noise in areas of low variability.

If you have imagery derived from some vector product, then an MRR with a lossless compression (PNG or LZMA) can produce very much higher quality imagery and also smaller file sizes. MRR is recommended in this case.

If you plan to do any processing on the raster then you will find working with MRR is more efficient than ECW.

Final note - please do not try to convert from ECW to MRR! Always go back to the original high quality source data, or you will just add noise to noise. Or, stick with your existing ECW files. We will do our best to support them as well as possible.

@Sam Roberts?  I somehow missed this excellent reply at the time, thank you for taking the time to write it up. Very informative. FWIW we have gone with MRR Jpeg 3 for the aerial imagery and we do no analysis processing of our imagery, it is simply reference imagery. Having it as MRR has simplified display across departmental workspaces and into SSA.