All Things Location

 View Only
Back to discussions
Expand all | Collapse all

Uber's H3 Hexagon Library

  • 1.  Uber's H3 Hexagon Library

    Employee
    Posted 09-13-2019 08:31
    Edited by Peter Møller 09-13-2019 10:04
    You might have seen my earlier post on creating hexagons using MapInfo Pro.

    After building this Hexagon library using MapBasic, I came across a different Hexagon library build by Uber: H3.

    Here is a #Video from a presentation where Joseph Gilley from Uber talks about why they chose to use hexagons.


    Read more about the H3 by Uber in this post.

    ------------------------------
    Peter Horsbøll Møller
    Pitney Bowes
    ------------------------------​


  • 2.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 09-13-2019 08:51
    From the presentation: "Pentagons er just hexagons without one side"

    ------------------------------
    Peter Horsbøll Møller
    Pitney Bowes
    ------------------------------

    Original Message


  • 3.  RE: Uber's H3 Hexagon Library

    Posted 08-13-2021 12:29
    Peter (or anyone else),
    Has anyone been successful in creating the H3 hex grid in MapInfo or other program to create shapefiles?  I don't even care if it's in MapInfo or a standalone program that can generate shapefiles in a specific bounding area, for different size H3 grids that I can then convert and bring info MapInfo.  We may have an upcoming project that will require using this grid system, however, no shapefiles exist that I can find, yet.  Plus, files for the entire planet would be a massive file size anyway.  Being a non-programmer and will have little prep time, I'm a little overwhelmed on how to proceed to use open source code to generate what I will need.  Any help or pointing in the right direction would be greatly appreciated!  If someone has already invented this wheel, to generate the H3 grids in specific areas, that would be even better (if you're willing to share).  Thanks.

    ------------------------------
    Jim Coleman
    Monte R. Lee & Co
    Oklahoma City OK
    ------------------------------

    Original Message


  • 4.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 08-16-2021 08:28
    Hi JIm

    Thanks for reaching out.

    When I looked at the H3 library the last time, I got stuck because the library that I could use from .NET was compiled with 32-bit which didn't match up with the 64-bit versions of MapInfo Pro.

    Looking at it again now, I guess the Python version could be a good choice as MapInfo Pro now has support for Python.

    I guess you may already have come across these sites: H3 and h3-py on Github.

    I also found this description of creating a shapefile with H3 hexagons for a given shapefile using Python.

    ------------------------------
    Peter Horsbøll Møller
    Principal Presales Consultant | Distinguished Engineer
    Precisely | Trust in Data
    ------------------------------

    Original Message


  • 5.  RE: Uber's H3 Hexagon Library

    Posted 08-16-2021 08:33
    Thanks for getting back Peter.

    I did see those, but I get lost in the "how" to put all of their example pieces together to make it work.  I haven't been able to dedicate much time to it yet, so hopefully I'm just overthinking it.

    ------------------------------
    Jim Coleman
    ------------------------------

    Original Message


  • 6.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 08-19-2021 07:23
    Here is the python script for create Uber H3 Grid Hexagons for a MapInfo Table in MapInfo Pro.

    ### Description: Shows how to create Uber H3 Grid Hexagons on a table opened in MapInfo Pro.
### Category: Table

from osgeo import ogr
from osgeo import osr
import os 
import json
from MapInfo.Types import MapInfoDictionary
from MapInfo.Types.Data import ITable
from System import String, Object
from System.Collections.Generic import List

try:
    import h3
except ModuleNotFoundError:
    print("module 'h3' is not installed. It will be installed")
    install_module("h3")

try:
    import h3
    tables = List[ITable]()
    for t in pro.Catalog.Tables:
        if t.CoordSysName == 'Longitude / Latitude (WGS 84)':
            tables.Add(t)

    if tables.Count == 0:
        exit()

    vTable = MapInfoDictionary[String, Object]()
    vTable['Name'] = 'Table'
    vTable['Type'] = 'Table'
    vTable['Prompt'] = 'Input _Table:'
    vTable['Value'] = tables[0]
    vTable['Values'] = tables

    variables = List[MapInfoDictionary[String, Object]]()
    variables.Add(vTable)

    if not pro.Input('Select Table To Create H3 Grid', variables, None):
        exit()
    srctable = vTable['Value']

    vTable = None
    variables = None
    tables = None

    fullPath = srctable.FullPath
    dir_path = os.path.dirname(os.path.realpath(fullPath))
           
    ds = ogr.Open(fullPath)
    ls = ds.GetLayer()

    if srctable.CoordSysName == 'Longitude / Latitude (WGS 84)':
        h3_list = []
        
        out_file = "{}\\{}_h3.tab".format(dir_path, srctable.Alias)
        h3_resolution = 3
        vFile = MapInfoDictionary[String, Object]()
        vFile['Name'] = 'File'
        vFile['Type'] = 'File'
        vFile['Prompt'] = 'Output _File:'
        vFile['Value'] = out_file
        vFile['Extension'] = 'TAB'


        vResolution = MapInfoDictionary[String, Object]()
        vResolution['Name'] = 'Integer'
        vResolution['Type'] = 'Integer'
        vResolution['Prompt'] = 'H3 _Resolution:'
        vResolution['Value'] = h3_resolution
        vResolution['MinimumValue'] = 1
        vResolution['MaximumValue'] = 15

        variables = List[MapInfoDictionary[String, Object]]()
        variables.Add(vFile)
        variables.Add(vResolution)
        if not pro.Input('Select Table To Create H3 Grid', variables, None):
            exit()

        out_file = vFile['Value']
        h3_resolution = int(vResolution['Value'])

        outtable = pro.Catalog.Tables.GetByPath(out_file)
        if outtable:
            outtable.Close()

        print("Creating H3 Grid on Table {} With Resolution {}".format(srctable.Alias, h3_resolution))

        for feature in ls:
            geom = feature.GetGeometryRef()
            if geom.GetGeometryName() == 'MULTIPOLYGON':
                for geom_part in geom:
                    fj = geom_part.ExportToJson()
                    geojson = json.loads(fj)
                    hex = h3.polyfill(geojson, res=h3_resolution, geo_json_conformant=True)
                    h3_list.extend(hex)
            elif geom.GetGeometryName() == 'POLYGON':
                fj = geom.ExportToJson()
                geojson = json.loads(fj)
                hex = h3.polyfill(geojson, res=h3_resolution, geo_json_conformant=True)
                h3_list.extend(hex)
        ls = None
        ds = None
        
        
        print("Dumping H3 Hexagons in {}".format(out_file))

        # Dump the output H3 hexagons to NativeX table using GDAL OGR
        driver = ogr.GetDriverByName("MapInfo EFAL")
        data_source = driver.CreateDataSource(out_file, options=['FORMAT=NATIVEX'])
        
        srs = osr.SpatialReference()
        srs.ImportFromEPSG(4326)
        
        layer = data_source.CreateLayer("hexgrid", srs, ogr.wkbPolygon)
        field_hex = ogr.FieldDefn("HexId", ogr.OFTString)
        field_hex.SetWidth(32)
        layer.CreateField(field_hex)

        for hexid in h3_list:
            feature = ogr.Feature(layer.GetLayerDefn())
            feature.SetField("HexId", hexid)
            #lng/lat tuple of tuples
            wkt = "POLYGON (("
            h3hexPoly = h3.h3_to_geo_boundary(hexid, geo_json=True)
            if h3hexPoly:
                for cord in h3hexPoly:
                    wkt += "{} {},".format(cord[0], cord[1])
                wkt = wkt[:-1]
                wkt += "))"
                poly = ogr.CreateGeometryFromWkt(wkt)
                feature.SetGeometry(poly)
            layer.CreateFeature(feature)
            feature = None
        
        layer = None
        srs = None
        data_source = None
        driver = None
        
        print("Done creating!")

        hextable = pro.Catalog.OpenTable(out_file)
        map = pro.Maps.AddMap([hextable, srctable])
        map.ZoomEntire()

except Exception as e:
    print("Error: {}".format(e))​


    ------------------------------
    Anshul Goel
    Knowledge Community Shared Account
    Shelton CT
    ------------------------------

    Original Message


  • 7.  RE: Uber's H3 Hexagon Library

    Posted 08-19-2021 10:34
    Thank you, Anshul.  I really appreciate it.

    ------------------------------
    Jim Coleman
    ------------------------------

    Original Message


  • 8.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 08-19-2021 15:27
    For those new to Python in Pro, it is an optional install but checked by default so you should have it if you have 2019.3.

    Save the above script to a .py file say hex.py
    Then you can run it by using run program dialog (Ctrl+U) and select hex.py.

    ------------------------------
    Bob Fortin
    Software Architect and Distinguished Engineer
    MapInfo Pro Development Team
    ------------------------------

    Original Message


  • 9.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 08-19-2021 15:44
    Another thing to note is that the input table has to be in projection (epsg:4326) 
    Longitude / Latitude (WGS 84)'​

    The script only lists these tables and exits if none are found.

    ------------------------------
    Bob Fortin
    Software Architect and Distinguished Engineer
    MapInfo Pro Development Team
    ------------------------------

    Original Message


  • 10.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 08-19-2021 16:01
      |   view attached
    Here is a sample file that works for me.

    ------------------------------
    Bob Fortin
    Software Architect and Distinguished Engineer
    MapInfo Pro Development Team
    ------------------------------

    Attachment(s)

    zip
    us_nation.zip   280 KB 1 version
    Original Message


  • 11.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 08-19-2021 16:07
    Partial screen shot with results


    ------------------------------
    Bob Fortin
    Software Architect and Distinguished Engineer
    MapInfo Pro Development Team
    ------------------------------

    Original Message


  • 12.  RE: Uber's H3 Hexagon Library

    Posted 08-19-2021 16:10
    It worked using your sample us nation file.  My county was showing as a native file, but it must have an issue.  Thank you very much for taking the time to help get me going.

    ------------------------------
    Jim Coleman
    ------------------------------

    Original Message


  • 13.  RE: Uber's H3 Hexagon Library

    Posted 23 days ago

    When attempting to run the PY script with the States data or my own boundaries I get this error message.

    Error: module 'h3' has no attribute 'polyfill'



    ------------------------------
    Jay Russell
    Lead Systems Analyst
    Centerpoint Energy, Inc.
    Houston TX
    ------------------------------

    Original Message


  • 14.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 23 days ago

    Hi Jay

    Have you tried using the tool in the MapInfo Marketplace, Create Uber H3 Hexagons?

    I'd recommend that you run MapInfo Pro as administrator the first time as it will install the h3 module which requires elevated permissions.



    ------------------------------
    Peter Horsbøll Møller
    Principal Presales Consultant | Distinguished Engineer
    Precisely | Trust in Data
    ------------------------------

    Original Message


  • 15.  RE: Uber's H3 Hexagon Library

    Posted 22 days ago

    After loading the tool from marketplace it's giving the same error as the independent PY script.  

    06:55:00: Creating H3 Grid on Table States With Resolution 7
    Error: module 'h3' has no attribute 'polyfill'

    Is there a way to verify the H3 module installed correctly?  I recently installed the 2023.1 patch and have elevated permissions but didn't run as admin.



    ------------------------------
    Jay Russell
    Lead Systems Analyst
    Centerpoint Energy, Inc.
    Houston TX
    ------------------------------

    Original Message


  • 16.  RE: Uber's H3 Hexagon Library

    Employee
    Posted 22 days ago

    Hey Jay

    I had a closer look and it seems that the Python H3 module has been changed and the methods we used are no longer in the module.

    I was able to find some alternative methods and I was able to make the script work again.

    That's using the Python console in the SQL Window using this modified script:
    ### Description: Shows how to create Uber H3 Grid Hexagons on a table opened in MapInfo Pro.
### Category: Table

from osgeo import ogr
from osgeo import osr
import os 
import json
from MapInfo.Types import MapInfoDictionary
from MapInfo.Types.Data import ITable
from System import String, Object
from System.Collections.Generic import List

try:
    import h3
except ModuleNotFoundError:
    print("module 'h3' is not installed. It will be installed")
    install_module("h3")

try:
    import h3
    tables = List[ITable]()
    for t in pro.Catalog.Tables:
        if t.CoordSysName == 'Longitude / Latitude (WGS 84)':
            tables.Add(t)

    if tables.Count == 0:
        exit()

    vTable = MapInfoDictionary[String, Object]()
    vTable['Name'] = 'Table'
    vTable['Type'] = 'Table'
    vTable['Prompt'] = 'Input _Table:'
    vTable['Value'] = tables[0]
    vTable['Values'] = tables

    variables = List[MapInfoDictionary[String, Object]]()
    variables.Add(vTable)

    if not pro.Input('Select Table To Create H3 Grid', variables, None):
        exit()
    srctable = vTable['Value']

    vTable = None
    variables = None
    tables = None

    fullPath = srctable.FullPath
    dir_path = os.path.dirname(os.path.realpath(fullPath))
           
    ds = ogr.Open(fullPath)
    ls = ds.GetLayer()

    if srctable.CoordSysName == 'Longitude / Latitude (WGS 84)':
        h3_list = []
        
        out_file = "{}\\{}_h3.tab".format(dir_path, srctable.Alias)
        h3_resolution = 3
        vFile = MapInfoDictionary[String, Object]()
        vFile['Name'] = 'File'
        vFile['Type'] = 'File'
        vFile['Prompt'] = 'Output _File:'
        vFile['Value'] = out_file
        vFile['Extension'] = 'TAB'


        vResolution = MapInfoDictionary[String, Object]()
        vResolution['Name'] = 'Integer'
        vResolution['Type'] = 'Integer'
        vResolution['Prompt'] = 'H3 _Resolution:'
        vResolution['Value'] = h3_resolution
        vResolution['MinimumValue'] = 1
        vResolution['MaximumValue'] = 15

        variables = List[MapInfoDictionary[String, Object]]()
        variables.Add(vFile)
        variables.Add(vResolution)
        if not pro.Input('Select Table To Create H3 Grid', variables, None):
            exit()

        out_file = vFile['Value']
        h3_resolution = int(vResolution['Value'])

        outtable = pro.Catalog.Tables.GetByPath(out_file)
        if outtable:
            outtable.Close()

        print("Creating H3 Grid on Table {} With Resolution {}".format(srctable.Alias, h3_resolution))

        for feature in ls:
            geom = feature.GetGeometryRef()
            if geom.GetGeometryName() == 'MULTIPOLYGON':
                for geom_part in geom:
                    fj = geom_part.ExportToJson()
                    geojson = json.loads(fj)
                    hex = h3.geo_to_cells(geojson, h3_resolution)
                    h3_list.extend(hex)
            elif geom.GetGeometryName() == 'POLYGON':
                fj = geom.ExportToJson()
                geojson = json.loads(fj)
                hex = h3.geo_to_cells(geojson, h3_resolution)
                h3_list.extend(hex)
        ls = None
        ds = None
             
        print("Dumping H3 Hexagons in {}".format(out_file))

        # Dump the output H3 hexagons to NativeX table using GDAL OGR
        driver = ogr.GetDriverByName("MapInfo EFAL")
        data_source = driver.CreateDataSource(out_file, options=['FORMAT=NATIVEX'])
        
        srs = osr.SpatialReference()
        srs.ImportFromEPSG(4326)
        
        layer = data_source.CreateLayer("hexgrid", srs, ogr.wkbPolygon)
        field_hex = ogr.FieldDefn("HexId", ogr.OFTString)
        field_hex.SetWidth(32)
        layer.CreateField(field_hex)

        for hexid in h3_list:
            feature = ogr.Feature(layer.GetLayerDefn())
            feature.SetField("HexId", hexid)
            #lng/lat tuple of tuples
            wkt = "POLYGON (("
            h3hexPoly = h3.cell_to_boundary(hexid)
            if h3hexPoly:
                wktFirst = ''
                for cord in h3hexPoly:
                    wkt += "{} {},".format(cord[1], cord[0])
                    if len(wktFirst) == 0:
                        wktFirst = "{} {},".format(cord[1], cord[0])
                wkt += wktFirst
                wkt = wkt[:-1]
                wkt += "))"
                poly = ogr.CreateGeometryFromWkt(wkt)
                feature.SetGeometry(poly)
            layer.CreateFeature(feature)
            feature = None
        
        layer = None
        srs = None
        data_source = None
        driver = None
        
        print("Done creating!")

        hextable = pro.Catalog.OpenTable(out_file)
        map = pro.Maps.AddMap([hextable, srctable])
        map.ZoomEntire()

except Exception as e:
    print("Error: {}".format(e))
    I wasn't able to just replace the script for the add-in as well and make that work too.
    I'll pass this onto engineering for a fix.
    Thanks for raising this issue, Jay



    ------------------------------
    Peter Horsbøll Møller
    Principal Presales Consultant | Distinguished Engineer
    Precisely | Trust in Data
    ------------------------------

    Original Message