MapInfo Pro

 View Only

MapInfo Monday: Loading OSM Data into MapInfo Pro via Python

  • 1.  MapInfo Monday: Loading OSM Data into MapInfo Pro via Python

    Employee
    Posted 09-23-2024 03:52
      |   view attached

    OpenStreetMap (also known as OSM) is a free, editable map of the world volunteers. About a month ago, on August 22nd, 2024, they had their 20th anniversary. Read more about OpenStreetMap on their website.

    Most of us use OSM regularly. The Precisely Maps in MapInfo Pro are based on OSM. Many websites use OSM as their base map, and some of you probably have contributed to OSM in some way or form.

    Getting OSM in raster form into MapInfo Pro has been easy for many years via the Tile Server tables. Getting OSM as vector data into MapInfo Pro has always been tricky.

    You can download precompiled datasets from OpenStreetMap in ESRI Shape format from GeoFabrik. Depending on the amount of data, you may have to go sub-state level to download the data as there is a limit to the file size.
    Last week, I came across a LinkedIn post by Milan Janosov on using OSMnx to extract data from OpenStreetMaps using Python. I decided to see if this also would work from within MapInfo Pro.
    Hang on to see if I was able to make it work. Happy #MapInfoMonday!

    OpenStreetMap tagging

    Before we dive into using the OSMnx module to extract data, I wanted to introduce the OpenStreetMap tagging system. This is important to have an understanding of to be able to query OpenStreetMap data.

    The data in OpenStreetMap isn't grouped into multiple datasets or tables as you normally would work with your data in MapInfo Pro. This also means that you can't just download all the buildings or roads from OpenStreetMap. Instead, all records are stored in a database and "grouped" using a tagging system. This makes it highly flexible as you can add as many or as few tags to each element. It also makes it easy to implement new ways to group your data as you just start adding a new tag.

    The OSM community has decided on certain key-value pair combinations for the most commonly used tags.

    To get an understanding of the various tags, I'd recommend that you read more about the tagging and the OSM map features here: OSM Map Features. I have also personally used this OpenStreetMap map interface to inspect various map features. This also helps you understand what tags you can use to filter out the map features you are looking for.

    Installing the OSMnx Python Module

    I tried installing the OSMnx Python module using Pip and that worked fine for me.

    From the Python310 folder in your MapInfo Pro installation folder, right-click on the file prompt.bat and then click on Run as Administrator.

    Use the DOS command cd to position yourself in the Python310 subfolder in the MapInfo Pro installation folder.

    Now use this command to download and install the OSMnx module:

    python -m pip install OSMnx

    If this works you will see how the module and its dependencies are being downloaded and installed.
    You can read more here: MapInfo Monday: Installing Additional Python Modules for MapInfo Pro

    Writing Python in MapInfo Pro

    I'm using the SQL Window to write my Python script. From the New dropdown in the SQL Window, select Python.
    The OSMnx module comes with a long list of capabilities. I'll not go through all these here. You can find the full documentation on Read the Docs.
    My small example here does 3 things:
    1. It downloads data from OpenStreetMap into a GeoPandas Data Frame.
    2. It saves the content from the GeoPandas Data Frame into a GeoPackage database.
    3. It opens the GeoPackage tables into a map in MapInfo Pro.

    To make it easier for, and you, to download additional data, I have created two Python functions to help out. One function, requestInto_GPKGtable, downloads and saves the data into a table in GeoPackage. The other function, open_GPKGtable, opens a GeoPackage table into a map in MapInfo Pro.

    With those two functions, it is easy to download and open data from OpenStreetMap.

    These 3 lines of code download the data into Geopackage:

    tags = {"building": True}
    tabName = 'building'
    requestInto_GPKGtable(filename_gpkg, tabName, tags, point, dist)

    The tags variable controls which features I download from OpenStreetMap. The tabName variable controls what table I save the data to. The last line calls the function to download and save the OpenStreetMap data to GeoPackage.

    These 3 lines of code open the GeoPackage table into a map in MapInfo Pro:

    tabName = 'building'
    fileTab = os.path.join(path_temp, tabName + '.tab')
    open_GPKGtable(filename_gpkg, tabName, fileTab, map)

    The tabName variable holds the name of the table in the GeoPackage database and the fileTab variable holds the file name of the MapInfo TAB file. In the third line, I call the function to open the GeoPackage table into a map in MapInfo Pro.

    Let's also inspect the two functions a bit more closely.

    def requestInto_GPKGtable(fileGPKG, tabName, tags, point, distance):
        print('{0}: features_from_point for {1}...'.format(getTime(), tags))
        gdf = ox.features_from_point(point, tags, dist=distance)
        print('{0} Returned records: {1} as {2}'.format(getTime(), len(gdf), tabName))
        gdf.to_file(fileGPKG, driver='GPKG', layer=tabName, OVERWRITE=True)
        print('{0} Saved to layer: {1}'.format(getTime(), tabName))
        del gdf

    The function requestInto_GPKGtable takes five parameters: The name of the GeoPackage database, fileGPKG, and the table, tabName, to save the data into. The tags parameter controls what map features to download from OpenStreetMap. The point and distance parameters are used to limit the map features geographically.

    The function ox.features_from_point downloads map features from OpenStreetMap into a GeoPandas Data Frame, gdf.

    The line gdf.to_file saves the data from the GeoPandas Data Frame into a file, in this case, a GeoPackage database. You can also use this function to save the data into other spatial formats, say ESRI Shape or GeoJSON file.

    Finally, I delete the DataFrame with this statement: del gdf.

    def open_GPKGtable(fileGPKG, tabName, tabFileName, map):
        pro.RunMapBasicCommand('Register Table "{0}" TYPE "GPKG" TABLE "{1}" Into "{2}"'.format(fileGPKG, tabName, tabFileName))
        tab = pro.Catalog.OpenTable(tabFileName)
        map.Layers.AddLayer(tab.Alias)

    The function openPPKGtable takes four parameters. The name of the GeoPackage database, fileGPKG, and the table, tabName, holding the data to open into MapInfo Pro. The name of the MapInfo TAB file, tabFileName, to create pointing to the table in the GeoPackage database. The map to add the table to.

    Not all MapBasic statements have matching Python statements. I use the RunMapBasicCommand function to create a MapInfo TAB file referencing the table in the GeoPackage database using a MapBasic statement.

    The variable pro refers to the running instance of MapInfo Pro. For this instance of MapInfo Pro, there is a Catalog which helps you work with and manage the datasets. I can use the method OpenTable on the Catalog to open another table into MapInfo Pro.

    For the map, the property Layers has a method called AddLayer that allows me to add a new layer to the map.

    The main part of the Python code sets the name of the GeoPackage database to use, it grabs the center and the zoom from the map and it configure the OSMnx module.

    path_temp = pro.EvalMapBasicCommandEx('PathToDirectory$(TempFileName$(""))')
    filename_gpkg = 'osm.gpkg'
    filename_gpkg = os.path.join(path_temp, filename_gpkg)
    
    map = pro.Maps.MostRecentMap
    
    pro.RunMapBasicCommand('Set CoordSys Earth Projection 1, 104')
    pro.RunMapBasicCommand('Set Distance Units "m"')
    
    lat = map.Center.Y
    long = map.Center.X 
    point = (lat, long)
    
    dist = map.Zoom
    print('{0}: Center Lat: {1} Long: {2} Zoom, m: {3}'.format(getTime(), lat, long, dist))
    
    ox.settings.use_cache = True
    ox.settings.cache_folder = path_temp
    ox.settings.log_console = True

    I also close the tables that I'm about to open as they may already be open and in this example, I delete the Geopackage database.

    tab = pro.Catalog.Tables['building']
    if tab:
        tab.Close()
    tab = pro.Catalog.Tables['forest']
    if tab:
        tab.Close()
    tab = pro.Catalog.Tables['roads']
    if tab:
        tab.Close()
    
    if os.path.exists(filename_gpkg):
        os.remove(filename_gpkg)

    Finally, I can download and open the map features from OpenStreetMap into the number of tables that make sense to me.

    #--- Downloading Features from OSM to GPKG ---------------------------
    tags = {"building": True}
    tabName = 'building'
    requestInto_GPKGtable(filename_gpkg, tabName, tags, point, dist)
    
    tags = {"landuse": 'forest'}
    tabName = 'forest'
    requestInto_GPKGtable(filename_gpkg, tabName, tags, point, dist)
    
    tags = {"highway": True}
    tabName = 'roads'
    requestInto_GPKGtable(filename_gpkg, tabName, tags, point, dist)
    
    #--- Opening tables pointing to GPKG tables ---------------------------
    tabName = 'building'
    fileTab = os.path.join(path_temp, tabName + '.tab')
    open_GPKGtable(filename_gpkg, tabName, fileTab, map)
    
    tabName = 'forest'
    fileTab = os.path.join(path_temp, tabName + '.tab')
    open_GPKGtable(filename_gpkg, tabName, fileTab, map)
    
    tabName = 'roads'
    fileTab = os.path.join(path_temp, tabName + '.tab')
    open_GPKGtable(filename_gpkg, tabName, fileTab, map)

    Below you can see the data that I have downloaded into MapInfo Pro using the script above. The data comes without styling so this part is up to you.

    I have attached the full script in the form of a Python file. You can open this file into a text editor, copy the content, and paste it into the Python Console or in the SQL Window in a new Python script in MapInfo Pro.


    ------------------------------
    Peter Horsbøll Møller
    Principal Presales Consultant | Distinguished Engineer
    Precisely | Trust in Data
    ------------------------------

    Attachment(s)