MapInfo Pro

Last week, my colleague asked me for a favor: "Can you help me cut out objects within this polygon from these layers and save them as a new dataset?"

Of course, I said yes. How hard could it be, right? Well, it was Friday afternoon, and my mind had already wandered off to the weekend.

There were around 40 layers, and many of them had tens of thousands of records. This was the Precisely StreetPro Display dataset for the United Kingdom.

Read on to see how it went.

Happy #MapInfoMonday!

How not to do it

I opened the tables into a map together with the polygon I should use as the cutter.

Next, I used the Invert Selection tool to select all objects outside the polygon. In this case, all the records I hadn't currently selected.

This also worked quickly. In the statusbar you can see the number of selected records.

Finally, I made the layer editable and pressed the Delete key to delete the currently almost 366,000 selected records...

Did I mention this was Friday afternoon?

Even though I had set all the other layers to be not visible so they wouldn't take up rendering time, it was still slow. Very slow. And this wasn't even the largest of tables.

This wouldn't work. I could see my weekend used on selecting and deleting objects from tables. And I had other plans for my weekend.

My mind had to postpone the weekend and get back to work.

How to do it

With my mind back at work, we quickly came up with another strategy. Since I only needed a small subset of the objects in each table, I concluded it would be much faster to save them to a new table, then overwrite the existing table with the new one. The problem was that this involved a lot of steps where I could go wrong and overwrite the wrong table.

When overwriting existing tables, I always recommend making a copy of the data and working on that copy.

The many steps in this process prompted me to write a small script to ensure I used the correct table names when overwriting. I also built the query into the script.

Dim sTab As String
sTab = TableInfo(1, TAB_INFO_NAME)
Print "Table: " + sTab
Dim sFile As String
sFile = TableInfo(sTab, TAB_INFO_TABFILE)

Fetch First From District
Dim oSel As Object
oSel = District.OBJ

Select * From sTab
    Where OBJ Intersects oSel
    Into _qResult NoSelect

Dim sFileTmp As String
sFileTmp = PathToDirectory$(TempFileName$("")) & "_temp.tab"
Commit Table _qResult As sFileTmp
Close Table sTab
Open Table sFileTmp
Print Time(24) + "   Records: " + TableInfo("_temp", TAB_INFO_NROWS)
Rename Table _temp As sFile
Close Table sTab

Using the method and the semi-automatic loop allowed me to select and save the objects within the polygon in less than 3 minutes.

That's a lot faster than just deleting the 366,000 records from just the first table.

The final steps

I then reopened all the tables in a map window.

As you can see, the polylines and polygons extend beyond the polygon of interest. They have not been cut hard at the polygon boundary.

The point tables and the empty tables are fine. I can identify these using the icon in the Table list.

The first icon indicates that this table contains a mix of object types. But it is also used for an empty table when no object type is present. Since none of the tables contained a mixture of object types from the start, I can safely assume they are empty.

So I use the Table List to close all these empty tables and tables with points.

Now, I only have to cut the remaining polygon and polyline tables against the polygon of interest.

I start by making all the layers not visible.

Then I clicked on the Editable icon in the Layer list for the top layer. This also makes the layer visible. I use the layer's context menu to Select All records in this layer.

And then I can set these selected objects as the target for my next action using the Set Target control on the Spatial tab. I could also have used Ctrl + T.

I can now select my polygon as the cutter object and select the Ease Outside control from the Spatial tab. This will delete all parts of the objects that are not inside my polygon of interest.

When the erase outside process finishes for a layer, I remove that layer from the map and repeat the same actions on the next layer.

In this way, I can keep track of how far I have gotten in the process.

Finally, I saved all the tables and packed them too. I don't need to pack the point tables.

And that's it. In good time for the weekend.

Have you come across a process that took forever - and found a faster workaround?

Good example Peter.  We've noticed improved performance when using FastEdit setting when editing tables.  Would this also speed up this edit if the FastEdit command was used prior to the Erase Outside?

Thanks, Jay.

Good point! FastEdit can certainly speed up processing. It's not that the processing as such goes faster; it's more that writing to the TAB file is more performant.

FastEdit is faster due to two reasons:

1. You avoid creating the transaction files, 
2. and you write the changes to the data (DAT and MAP) files directly.

FastEdit also comes with a drawback: It has no regret. You can't undo your changes as they are written directly to the TAB file.

I often use FastEdit in my applications, and sometimes when scripting in the SQL or MapBasic window.

I rarely, or never, use it when using the tools in MapInfo Pro by hand. This is because when I do things by hand, I sometimes make mistakes.

If I had scripted the Erase Outside, I may have used FastEdit. But then again, it was small tables I was working on, so it wouldn't have benefited much in terms of performance.

I tried using FastEdit to delete points outside my polygon, but it only had a limited effect on the delete process.

But yes, FastEdit does have its moments.

In case some of you wonder, this is the statement used to turn on FastEdit for a table:

Set Table tablename FastEdit On

There is no button in the interface for activating it.