MapInfo Pro

At work, often our MapInfo tables link to files in other locations using relative file paths. This is great, but requires users to use a fixed folder structure and essentially makes tables static without the file system behind it.

Ideally users would be able to store their files within a MapInfo table, preferentially 1 file per record. Furthermore, it'd also be nice to have the ability to store a BLOB in the Metadata of a table, allowing users to store documentation for the table within the table itself.

What is a BLOB?

For those who don't know, here is the definition of a BLOB:

A Binary Large OBject (BLOB) is a collection of binary data stored as a single entity in a database management system. Blobs are typically images, audio, PDFs or other multimedia objects, though sometimes binary executable code (.exe) is stored as a blob.

The new TAB Extended (NATIVEX) format has several improvements such as expanding beyond the 2GB file size limitation and the ability to store Unicode strings. We do plan to extend this format further over time with corresponding enhancments to our product suite. Variable length strings and NULL values are examples of enhancements we would like to add in the future. BLOB/CLOB data could also be added (as well as fixed length binary types). BLOB types would most likely require a new file in the table's family of files since the DAT file is fixed width.

Indeed, today I briefly looked at how I could implement it myself with MapBasic. It doesn't look like it would be too difficult. I'd need to implement the 64-bit integer data type in MapBasic (to store the size of a file (otherwise I'd have the same issue as 2GB limit of TAB files)), but that's not too hard given that kernel32.dll has some functions for operations on 64-bit integer data type... The main thing I'd have to worry about is speed of streaming passed a multigigabyte file.

And indeed my approach would be using a seperate .BLOB file with structure:

HEADER:

ID: 32-Bit Int; Name: Null terminated string; Size of blob: 64-bit Int; line-feed

...

DATA:

<<BLOB>><<BLOB>><<BLOB>>...

This way one could quickly identify the blob required and the cumulative offset required to stream to, then jump to that position in the file and scrape the blob data to a new file for opening. It'd make for relatively efficient MapBasic code anyway...

After the great support for Unicode (which I've used extensively already) I think support for NULL in TAB Extended would be a great addition. I've lost track of how many times I've argued that NULL Island should not really be at (0,0) of your preferred coordinate system. :-) Blobs would be nice, but less important to me.

MapInfo Pro/MapBasic already has support for 64-bit Integer - we call it LargeInt

Oh it does? Didn't know that, Is there a list of MapInfo types somewhere other than the MapBasic reference? I did look in the reference initially and didn't see LargeInt. Did see IntPtr is 64-bit in 64-bit though, so that helps.

I also started writing MapInfo-BLOB. It's on Github :) Hopefully others would find a use for it if it ever gets off the ground! Currently very little is implemented as my windows Laptop has a dead battery (no more working on trains!)

I have created a task for us to get the documentation updated on the supported types as I also were struggling to find LargeInt in the documentation.