MapInfo Pro

 View Only
  • 1.  Raster API: Interpolation

    Posted 08-24-2019 14:50
    Hi. I'm having some problems using the raster interpolation operations from C#.
    Now, I'm not sure if I'm using the API the way it's meant to be used, so any help pointing me in the right direction is appreciated.

    Let's say I have some point data files and I want to interpolate the data and create a raster from it.
    Easy, just use the Create Raster tool.

    Here you can see the result from the create raster operation (and the parameters used)

    As the result, I get the raster image I want. Everything is fine.
    Now, the thing is, I want to do the same but from C# to automate some operations.

    First, I have the data loaded in memory and apply some operations to it.
    To invoke the InterpolateMinimumCurvature method, I first need to write my data to a file, so I write an ascii file with the data and then call the interpolation method.



    If I use the exact same data as before (without modifying it at all) I get the following result:


    The result is... somewhat similar. But it has that horrible horizontal noise lines.
    I have checked the parameters, they are the same (i think) and even checked the numbers written to the ascii file and I could not find any sort of rounding error or anything suspicious.

    Here is the interpolation code used:

    private void Interpolate(string strInputFile, string strOutputFile)
    {
        RasterInterpolationOptions interpolationOptions = new RasterInterpolationOptions
        {
            Preferences =
                        {
                            AutoCacheSize = true,
                            CacheSize = 0,
                            TemporaryDirectory = ""
                        }
        };
    
        string strCoordSystem = "CoordSys Earth Projection 1, 104";
        // setup source data file parameters, can add more than one file
        RasterInterpolationInputFileDetails file = new RasterInterpolationInputFileDetails();
        file.DataFieldIndexes.Add(2);
        file.XFieldIndex = 0;
        file.YFieldIndex = 1;
        file.FileType = RasterInterpolationInputFileType.ASCII_SPACE;
        file.FilePath = strInputFile;
        file.HeaderRows = 0;
        file.CoordSysString = strCoordSystem;
        file.SubFileName = "*.*";
        interpolationOptions.InputDetails.Add(file);
    
        // setup output raster parameters
        interpolationOptions.OutputDetails.FilePath = strOutputFile;
        interpolationOptions.OutputDetails.DriverId = DriverID.MRR.GetString();
        interpolationOptions.OutputDetails.AutoDataType = true;
    
        interpolationOptions.OutputDetails.CoordSysString = strCoordSystem;
    
        // setup raster geometry parameters
        interpolationOptions.Geometry.Extent.Auto = true;
    
        interpolationOptions.Geometry.CellSize.Auto = true;
    
        // setup minimum curvature method parameters
        RasterInterpolationMinimumCurvature minimumCurvatureOptions = new RasterInterpolationMinimumCurvature();
        minimumCurvatureOptions.ParameterType.UnitsType = RasterInterpolationParameterUnitsType.Cell;
        minimumCurvatureOptions.ParameterType.UnitCode = MapInfoUnitCode.Meters;
        minimumCurvatureOptions.Iterations = RasterInterpolationIterationIntensity.Normal;
    
        minimumCurvatureOptions.PercentChange = 0.001;
    
        minimumCurvatureOptions.SplineTension = 0;
        minimumCurvatureOptions.StampMethod = RasterInterpolationMinimumCurvatureStampingMethod.AverageLastInWeighted;
    
        minimumCurvatureOptions.Smoothing.Type = RasterInterpolationSmoothingType.Gaussian;
        minimumCurvatureOptions.Smoothing.Level = 5;
        minimumCurvatureOptions.Clip.Method = RasterInterpolationClipMethod.None;
    
        RasterApiOptions apiOptions = new RasterApiOptions();
    
        RasterCreationOptions creationOptions = new RasterCreationOptions
        {
            CompressionOptions = new RasterCompressionOptions(RasterCompressionType.DataBalanced, 3),
            PredictiveEncoding = EncodingType.PreviousColumnLinear
        };
        apiOptions.CreationOptions = creationOptions;
    
        RasterInterpolation.InterpolateMinimumCurvature(interpolationOptions, minimumCurvatureOptions, apiOptions);
    }


    Here is another sample:

    The left image was created from MapInfo using CreateRaster the right one was created using the same code as before.

    Any Ideas of what could be happening?
    (Using MapInfo Pro 17.0 64 bits, Windows 7 SP1)

    ------------------------------
    Lucas Pandolfo
    Knowledge Community Shared Account
    Shelton CT
    ------------------------------


  • 2.  RE: Raster API: Interpolation

    Employee
    Posted 08-27-2019 06:09
    @Sam Roberts something you could advise on? ​

    ------------------------------
    Chris Jenkins
    Pitney Bowes Software Ltd
    HENLEY-ON-THAMES
    ------------------------------



  • 3.  RE: Raster API: Interpolation

    Posted 08-27-2019 08:52
    I don't know what is causing this problem. It looks very odd. Perhaps you can share the source data and we can investigate.

    ------------------------------
    Sam Roberts
    Engineer, MapInfo Pro Advanced (Raster)
    Australia
    ------------------------------



  • 4.  RE: Raster API: Interpolation

    Posted 08-28-2019 09:14
      |   view attached
    I tried isolating the offending code. The zip contains the Visual Studio project files, the code and the compiled files, and the data files.
    I removed all the unnecessary code. It adds a button into a new ribbon and, when clicked, it hangs for a while (all progress bars were removed) and after a few seconds it opens the interpolated raster.



    ------------------------------
    Lucas Pandolfo
    Knowledge Community Shared Account
    Shelton CT
    ------------------------------

    Attachment(s)

    zip
    Example.zip   1.65 MB 1 version


  • 5.  RE: Raster API: Interpolation

    Posted 08-29-2019 01:33
    Hi Lucas,

    Thanks for sharing your data and code.

    You need to turn off Predictive Encoding - just change the setting you use (below) to None.

    PredictiveEncoding = EncodingType.PreviousColumnLinear​


    We were able to duplicate your problem and when we turn off this option it is resolved. It appears there is a bug. However, it may only appear when you have a floating-point output data type. In these cases, I recommend you do not use predictive encoding anyway. I use predictive encoding for integer data types because it does not introduce any changes in the data. Also, the only reason to use this option is to improve the data compression on very large rasters. This is not a problem you are facing.

    Also, if you are using Minimum Curvature then you should not be using smoothing. It ought to produce a very smooth raster without any additional kernel smoothing operations. These operations will damage your data honouring and I never use them.

    I noticed your data is a SHP file and that when we try to grid this in Pro it simply does nothing, which is not a useful output... I will take a look at that.

    Your data is interesting because clearly it came from a raster in the first place. The points are all on aligned rows and columns with uniform spacing of 0.25 degrees. Another approach for you might be to turn this into a grid using nearest neighbour or stamp methods and then, if you want a cell size smaller than 0.25 degrees, you could use the resample operation. This will do a smooth cubic interpolation and will give a result as good (or better than) minimum curvature. 

    Best wishes,
    Sam.



    ------------------------------
    Sam Roberts
    Engineer, MapInfo Pro Advanced (Raster)
    Australia
    ------------------------------



  • 6.  RE: Raster API: Interpolation

    Posted 08-29-2019 17:37
    Ok, I disabled Predictiveencoding and now it's working perfectly. I feel dumb for not trying that. Thanks.

    The data comes from the noaa weather shape files and I need to get both a interpolated raster and then compute some contours using said raster.
    The current approach (even without smoothing) creates some ripples at zero level.


    In the image above, the blue area should be solid. All the data points within this area have 0 value.


    The raster values varies and has values like 0.01, 0.0005, -5.33e-9, 3.54e-8, 0, etc. 
    In the end this causes some problems when i use the contour tool and specify 0 as min contour level and throws what seems to be nonsense data.


    Anyways, thanks for your time, my immediate problem is solved. I'll try looking into that grid thing. 
    Do you refer to this function?
    Create Grid 
    	From tablename 
    	With expression [ Ignore value_to_ignore ]
    	Into filespec [ Type grid_type ]
    		[ Coordsys... ]
    	[ Clipping { Object obj } | { Table tablename } ]
    	Inflect num_inflections [ By Percent ] at
    	color : inflection_value [ color : inflection_value ...] 
    	[ Round rounding_factor ]
    	{[ Cell Size cell_size [ Units distance_unit ]] | [ Cell Min n_cells ]}
    		 [ Border numcells ]
    	Interpolate With interpolator_name Version version_string 
    		Using num_parameters parameter_name : parameter_value 
    		[ parameter_name : parameter_value ... ]


    Can I get a raster image and compute the contours using this method? (I've never worked with grids before)
    I suppose I have to dive into the help files again.

    Cheers.

    ------------------------------
    Lucas Pandolfo
    Knowledge Community Shared Account
    Shelton CT
    ------------------------------



  • 7.  RE: Raster API: Interpolation

    Posted 08-29-2019 23:39
    I am certainly familiar with the ripple effect you are observing (in a different context). We see this effect in MapInfo ProAdvanced when you render rasters with certain kinds of features, at certain levels of zoom and with certain interpolation options.

    When you have a raster that has areas of smoothly varying data (like terrain for example) and those areas are adjacent to (or surrounded by) flat areas where all the cells are the same value (for example if the terrain raster includes the ocean and all offshore cells are zero) then you can see this effect. It occurs when you zoom in and the raster engine begins to interpolate the raster to a higher resolution in order to render it. 

    You can use cubic, cubic operator, bilinear or nearest neighbour interpolation. If you use cubic then the ripples appear between the cell centres - the cubic splines are "ringing" after they drop off the smoothly varying data into the flat data. You get a ripple between every cell in the base resolution raster. The cubic method is very smooth and this ringing can persist right out to the edge of the raster tile. If you use "cubic operator" the effect is more local and you will see it curtailed faster. If you use bilinear or nearest neighbour the effect goes away.

    I guess it is possible that you could see this effect in minimum curvature gridding as well although I can't say I have ever seen it. I will try to duplicate your results with some data that ought to show a similar effect when I get back to my office next week.

    In MapInfo Pro Advanced there is a tool that generates contours in the UI. You can also generate contours using C#, similar to what you were doing with gridding.

    ------------------------------
    Sam Roberts
    Engineer, MapInfo Pro Advanced (Raster)
    Australia
    ------------------------------



  • 8.  RE: Raster API: Interpolation

    Posted 09-02-2019 06:56
    I investigated the problem with predictive encoding in this instance and found that the issue is not a bug. In this case, you were using a 32-bit floating-point data type (for output) and the data you were gridding had cell-to-cell differences that were generally about 1e-7 of the average magnitude. In this scenario, predictive encoding with floating-point numbers generates round-off errors. We try to mitigate this by resetting the prediction every 128 cells - this causes the pattern you saw in the raster.

    I reiterate my previous advice - use predictive encoding on integer data types only. If you choose to use it on floating-point numbers you need to be careful that the data type has enough decimal precision to represent the deltas between cells accurately without causing significant round-off.

    ------------------------------
    Sam Roberts
    Engineer, MapInfo Pro Advanced (Raster)
    Australia
    ------------------------------



  • 9.  RE: Raster API: Interpolation

    Posted 09-02-2019 10:43
    Indeed, I've been using the same data without predictive encoding and no problems so far.

    Now the only annoyance that remains is the rippling effect. For the time being I just ignore it and generate the contours from -1 instead of 0 and it seems to work fine. Not a big deal.

    Anyways, I have some more questions about the different Mapinfo Pro /Raster APIs that I'm having problems understanding reading the docs, so this is not the last you'll hear from me. Maybe.

    Thanks for your help.

    ------------------------------
    Lucas Pandolfo
    Knowledge Community Shared Account
    Shelton CT
    ------------------------------