MapInfo Pro

I could be going mad, or becoming incompetent in my old age, but it seems ever since the recent update I have been experiencing problems I have never come across before. 

The first problem I encountered was where I exported some tables to MIF to share with our client. To check if the exports were correct, I re-imported one back into MapInfo and was greeted with the error "unsupported character set: UTF-16". For starters MapInfo exported in this charset, how can it not be supported? Second, I didn't recall specifying such a charset so went back through the process – seems I chose to export in the source's charset. Right, so the source turns out to be UTF-16 for some reason. Again, if it's IN UTF-16 how is this not supported when I'm pulling it back in? More to the point, WHY is the source file suddenly in this charset when I'm pretty sure windowsLatin1 has been the standard. 

But then next I'm pulling in a CSV of float values, some of which are over 999, as a lookup table to update another table. Confidently using the add column function to do this, I run it my program without error, search for 0s, and was satisfied everything had populated as it should. But then I notice several records have values of 1 in them. That's not right. I check the lookup and the value that is supposed to being pulled is 1,876. Whatever I do mapinfo interprets this as 1. Both source and destination fields are floats. Both accept vales of over 999 if I type directly into them, and MapInfo even adds commas to separate them by thousands. But pulling the number from one table into the other and it truncates it at 1: at the comma. 

So I look at the original CSV and obviously as its comma delimited, numerical values WITH commas in them have escape quotes around them. Right. Clearly this value in escape quotes is being interpreted differently or something. Which is utterly baffling, as how on earth have I never experienced this problem in 20 years of MapInfo use? I must have imported a billion CSVs into MapInfo over my lifetime, and certainly a significant proportion of those will have contained values over 999. I work with coordinates ffs! Not once have I ever noticed an instance of MapInfo changing these values to 1. I SURELY would have noticed.

So I grab an old CSV at random which I know contains floats of 1,000+. And the pop-up I get is defaulting to UTF-8 charset. Eh? Since when? And not even UTF-16? But 8? So I venture into Options and system settings say UTF-16 for new tables. Hmm. It may be just one of those things I have been incredibly unobservant about but that doesn't seem typical to me. Could this be why the initial table in my first encounter was in this format? And if MapInfo is literally defaulting to that format, how come it says it is not supported when I'm trying to open a MIF? And are all these problems such as the truncating of 1,000 to 1 related? 

Am I having a breakdown, or is something going on here? I mean, are we supposed to remove all thousand separators from our raw data now? How long has this been a thing? Dazed and confused.  

------------------------------
Ryan Cook
Knowledge Community Shared Account
------------------------------

I could be going mad, or becoming incompetent in my old age, but it seems ever since the recent update I have been experiencing problems I have never come across before. 

The first problem I encountered was where I exported some tables to MIF to share with our client. To check if the exports were correct, I re-imported one back into MapInfo and was greeted with the error "unsupported character set: UTF-16". For starters MapInfo exported in this charset, how can it not be supported? Second, I didn't recall specifying such a charset so went back through the process – seems I chose to export in the source's charset. Right, so the source turns out to be UTF-16 for some reason. Again, if it's IN UTF-16 how is this not supported when I'm pulling it back in? More to the point, WHY is the source file suddenly in this charset when I'm pretty sure windowsLatin1 has been the standard. 

But then next I'm pulling in a CSV of float values, some of which are over 999, as a lookup table to update another table. Confidently using the add column function to do this, I run it my program without error, search for 0s, and was satisfied everything had populated as it should. But then I notice several records have values of 1 in them. That's not right. I check the lookup and the value that is supposed to being pulled is 1,876. Whatever I do mapinfo interprets this as 1. Both source and destination fields are floats. Both accept vales of over 999 if I type directly into them, and MapInfo even adds commas to separate them by thousands. But pulling the number from one table into the other and it truncates it at 1: at the comma. 

So I look at the original CSV and obviously as its comma delimited, numerical values WITH commas in them have escape quotes around them. Right. Clearly this value in escape quotes is being interpreted differently or something. Which is utterly baffling, as how on earth have I never experienced this problem in 20 years of MapInfo use? I must have imported a billion CSVs into MapInfo over my lifetime, and certainly a significant proportion of those will have contained values over 999. I work with coordinates ffs! Not once have I ever noticed an instance of MapInfo changing these values to 1. I SURELY would have noticed.

So I grab an old CSV at random which I know contains floats of 1,000+. And the pop-up I get is defaulting to UTF-8 charset. Eh? Since when? And not even UTF-16? But 8? So I venture into Options and system settings say UTF-16 for new tables. Hmm. It may be just one of those things I have been incredibly unobservant about but that doesn't seem typical to me. Could this be why the initial table in my first encounter was in this format? And if MapInfo is literally defaulting to that format, how come it says it is not supported when I'm trying to open a MIF? And are all these problems such as the truncating of 1,000 to 1 related? 

Am I having a breakdown, or is something going on here? I mean, are we supposed to remove all thousand separators from our raw data now? How long has this been a thing? Dazed and confused.  

------------------------------
Ryan Cook
Knowledge Community Shared Account
------------------------------

Hi Ryan

I'm also on the latest release for v2021.1, build 33.

I tried to follow what you did, Ryan. Let me know if I missed a step somewhere. Now my data is stored as UTF-8.

Here's how this fake demo data looks in a browser. Note the floating value at the end.

Export "Customers" Into "C:\3. Demo\2. Maps\2023\Site Selection\Customers\Customers.mif" Type "MIF"

I also tried exporting it to CSV

Export "Customers" Into "C:\3. Demo\2. Maps\2023\Site Selection\Customers\Customers.csv" Type "ASCII" Overwrite Delimiter "," Titles

And again it all looks fine

I also tried importing the MIF/MID file again and that also worked nicely.

I also tried saving my UTF-8 table into a UTF-16 and exported this to MIF/MID. The data still looks fine and the reimport also works fine.

I have for a while been using UTF-8 as my default charset and this seems to be maintained after I upgraded (yesterday in fact).

Do keep in mind that MapInfo Pro relies on the Regional Settings when showing numbers and when writing and reading these from other formats. I'm however not sure how this can affect the way it seems to treat your numbers internally.

Not sure if any of the above is helpful to you, Ryan, If MapInfo Pro is still misbehaving when you get in Monday morning, it might be worth reaching out to our support team, or maybe if possible try sharing the MIF/MID file that you were able to import.

------------------------------
Peter Horsbøll Møller
Principal Presales Consultant | Distinguished Engineer
Precisely | Trust in Data

Original Message:
Sent: 02-23-2023 10:50
From: Ryan Cook
Subject: Problems since recent update

I could be going mad, or becoming incompetent in my old age, but it seems ever since the recent update I have been experiencing problems I have never come across before. 

The first problem I encountered was where I exported some tables to MIF to share with our client. To check if the exports were correct, I re-imported one back into MapInfo and was greeted with the error "unsupported character set: UTF-16". For starters MapInfo exported in this charset, how can it not be supported? Second, I didn't recall specifying such a charset so went back through the process – seems I chose to export in the source's charset. Right, so the source turns out to be UTF-16 for some reason. Again, if it's IN UTF-16 how is this not supported when I'm pulling it back in? More to the point, WHY is the source file suddenly in this charset when I'm pretty sure windowsLatin1 has been the standard. 

But then next I'm pulling in a CSV of float values, some of which are over 999, as a lookup table to update another table. Confidently using the add column function to do this, I run it my program without error, search for 0s, and was satisfied everything had populated as it should. But then I notice several records have values of 1 in them. That's not right. I check the lookup and the value that is supposed to being pulled is 1,876. Whatever I do mapinfo interprets this as 1. Both source and destination fields are floats. Both accept vales of over 999 if I type directly into them, and MapInfo even adds commas to separate them by thousands. But pulling the number from one table into the other and it truncates it at 1: at the comma. 

So I look at the original CSV and obviously as its comma delimited, numerical values WITH commas in them have escape quotes around them. Right. Clearly this value in escape quotes is being interpreted differently or something. Which is utterly baffling, as how on earth have I never experienced this problem in 20 years of MapInfo use? I must have imported a billion CSVs into MapInfo over my lifetime, and certainly a significant proportion of those will have contained values over 999. I work with coordinates ffs! Not once have I ever noticed an instance of MapInfo changing these values to 1. I SURELY would have noticed.

So I grab an old CSV at random which I know contains floats of 1,000+. And the pop-up I get is defaulting to UTF-8 charset. Eh? Since when? And not even UTF-16? But 8? So I venture into Options and system settings say UTF-16 for new tables. Hmm. It may be just one of those things I have been incredibly unobservant about but that doesn't seem typical to me. Could this be why the initial table in my first encounter was in this format? And if MapInfo is literally defaulting to that format, how come it says it is not supported when I'm trying to open a MIF? And are all these problems such as the truncating of 1,000 to 1 related? 

Am I having a breakdown, or is something going on here? I mean, are we supposed to remove all thousand separators from our raw data now? How long has this been a thing? Dazed and confused.  

------------------------------
Ryan Cook
Knowledge Community Shared Account
------------------------------

Apologies for the late reply, Peter - I have been on holiday and the reminders of this thread were pushed down the queue in my inbox. Thanks for taking a look at this. The "unsupported character set: UTF-16" error still persists (but I have found a workaround) but the odd comma problem has vanished mysteriously. I will put it down to a bug on the day. 

However, a new problem since the update: new employees are being met with a request for login details whenever they try to add a Precisely background map. I can't recall ever having to login, I can't find any details for doing so and I can't find any way to register the new users. Any ideas?

Ryan

------------------------------
Ryan Cook
Knowledge Community Shared Account

Original Message:
Sent: 02-24-2023 04:37
From: Peter Møller
Subject: Problems since recent update

Hi Ryan

I'm also on the latest release for v2021.1, build 33.

I tried to follow what you did, Ryan. Let me know if I missed a step somewhere. Now my data is stored as UTF-8.

Here's how this fake demo data looks in a browser. Note the floating value at the end.

Export "Customers" Into "C:\3. Demo\2. Maps\2023\Site Selection\Customers\Customers.mif" Type "MIF"

I also tried exporting it to CSV

Export "Customers" Into "C:\3. Demo\2. Maps\2023\Site Selection\Customers\Customers.csv" Type "ASCII" Overwrite Delimiter "," Titles

And again it all looks fine

I also tried importing the MIF/MID file again and that also worked nicely.

I also tried saving my UTF-8 table into a UTF-16 and exported this to MIF/MID. The data still looks fine and the reimport also works fine.

I have for a while been using UTF-8 as my default charset and this seems to be maintained after I upgraded (yesterday in fact).

Do keep in mind that MapInfo Pro relies on the Regional Settings when showing numbers and when writing and reading these from other formats. I'm however not sure how this can affect the way it seems to treat your numbers internally.

Not sure if any of the above is helpful to you, Ryan, If MapInfo Pro is still misbehaving when you get in Monday morning, it might be worth reaching out to our support team, or maybe if possible try sharing the MIF/MID file that you were able to import.

------------------------------
Peter Horsbøll Møller
Principal Presales Consultant | Distinguished Engineer
Precisely | Trust in Data

Original Message:
Sent: 02-23-2023 10:50
From: Ryan Cook
Subject: Problems since recent update

I could be going mad, or becoming incompetent in my old age, but it seems ever since the recent update I have been experiencing problems I have never come across before. 

The first problem I encountered was where I exported some tables to MIF to share with our client. To check if the exports were correct, I re-imported one back into MapInfo and was greeted with the error "unsupported character set: UTF-16". For starters MapInfo exported in this charset, how can it not be supported? Second, I didn't recall specifying such a charset so went back through the process – seems I chose to export in the source's charset. Right, so the source turns out to be UTF-16 for some reason. Again, if it's IN UTF-16 how is this not supported when I'm pulling it back in? More to the point, WHY is the source file suddenly in this charset when I'm pretty sure windowsLatin1 has been the standard. 

But then next I'm pulling in a CSV of float values, some of which are over 999, as a lookup table to update another table. Confidently using the add column function to do this, I run it my program without error, search for 0s, and was satisfied everything had populated as it should. But then I notice several records have values of 1 in them. That's not right. I check the lookup and the value that is supposed to being pulled is 1,876. Whatever I do mapinfo interprets this as 1. Both source and destination fields are floats. Both accept vales of over 999 if I type directly into them, and MapInfo even adds commas to separate them by thousands. But pulling the number from one table into the other and it truncates it at 1: at the comma. 

So I look at the original CSV and obviously as its comma delimited, numerical values WITH commas in them have escape quotes around them. Right. Clearly this value in escape quotes is being interpreted differently or something. Which is utterly baffling, as how on earth have I never experienced this problem in 20 years of MapInfo use? I must have imported a billion CSVs into MapInfo over my lifetime, and certainly a significant proportion of those will have contained values over 999. I work with coordinates ffs! Not once have I ever noticed an instance of MapInfo changing these values to 1. I SURELY would have noticed.

So I grab an old CSV at random which I know contains floats of 1,000+. And the pop-up I get is defaulting to UTF-8 charset. Eh? Since when? And not even UTF-16? But 8? So I venture into Options and system settings say UTF-16 for new tables. Hmm. It may be just one of those things I have been incredibly unobservant about but that doesn't seem typical to me. Could this be why the initial table in my first encounter was in this format? And if MapInfo is literally defaulting to that format, how come it says it is not supported when I'm trying to open a MIF? And are all these problems such as the truncating of 1,000 to 1 related? 

Am I having a breakdown, or is something going on here? I mean, are we supposed to remove all thousand separators from our raw data now? How long has this been a thing? Dazed and confused.  

------------------------------
Ryan Cook
Knowledge Community Shared Account
------------------------------

I think the login the the services was updated in March forcing users to login again.

You can login and also create new accounts from the Services tab on the backstage (via the Pro tab)

Apologies for the late reply, Peter - I have been on holiday and the reminders of this thread were pushed down the queue in my inbox. Thanks for taking a look at this. The "unsupported character set: UTF-16" error still persists (but I have found a workaround) but the odd comma problem has vanished mysteriously. I will put it down to a bug on the day. 

However, a new problem since the update: new employees are being met with a request for login details whenever they try to add a Precisely background map. I can't recall ever having to login, I can't find any details for doing so and I can't find any way to register the new users. Any ideas?

Ryan

------------------------------
Ryan Cook
Knowledge Community Shared Account

Original Message:
Sent: 02-24-2023 04:37
From: Peter Møller
Subject: Problems since recent update

Hi Ryan

I'm also on the latest release for v2021.1, build 33.

I tried to follow what you did, Ryan. Let me know if I missed a step somewhere. Now my data is stored as UTF-8.

Here's how this fake demo data looks in a browser. Note the floating value at the end.

Export "Customers" Into "C:\3. Demo\2. Maps\2023\Site Selection\Customers\Customers.mif" Type "MIF"

I also tried exporting it to CSV

Export "Customers" Into "C:\3. Demo\2. Maps\2023\Site Selection\Customers\Customers.csv" Type "ASCII" Overwrite Delimiter "," Titles

And again it all looks fine

I also tried importing the MIF/MID file again and that also worked nicely.

I also tried saving my UTF-8 table into a UTF-16 and exported this to MIF/MID. The data still looks fine and the reimport also works fine.

I have for a while been using UTF-8 as my default charset and this seems to be maintained after I upgraded (yesterday in fact).

Do keep in mind that MapInfo Pro relies on the Regional Settings when showing numbers and when writing and reading these from other formats. I'm however not sure how this can affect the way it seems to treat your numbers internally.

Not sure if any of the above is helpful to you, Ryan, If MapInfo Pro is still misbehaving when you get in Monday morning, it might be worth reaching out to our support team, or maybe if possible try sharing the MIF/MID file that you were able to import.

------------------------------
Peter Horsbøll Møller
Principal Presales Consultant | Distinguished Engineer
Precisely | Trust in Data

Original Message:
Sent: 02-23-2023 10:50
From: Ryan Cook
Subject: Problems since recent update

I could be going mad, or becoming incompetent in my old age, but it seems ever since the recent update I have been experiencing problems I have never come across before. 

The first problem I encountered was where I exported some tables to MIF to share with our client. To check if the exports were correct, I re-imported one back into MapInfo and was greeted with the error "unsupported character set: UTF-16". For starters MapInfo exported in this charset, how can it not be supported? Second, I didn't recall specifying such a charset so went back through the process – seems I chose to export in the source's charset. Right, so the source turns out to be UTF-16 for some reason. Again, if it's IN UTF-16 how is this not supported when I'm pulling it back in? More to the point, WHY is the source file suddenly in this charset when I'm pretty sure windowsLatin1 has been the standard. 

But then next I'm pulling in a CSV of float values, some of which are over 999, as a lookup table to update another table. Confidently using the add column function to do this, I run it my program without error, search for 0s, and was satisfied everything had populated as it should. But then I notice several records have values of 1 in them. That's not right. I check the lookup and the value that is supposed to being pulled is 1,876. Whatever I do mapinfo interprets this as 1. Both source and destination fields are floats. Both accept vales of over 999 if I type directly into them, and MapInfo even adds commas to separate them by thousands. But pulling the number from one table into the other and it truncates it at 1: at the comma. 

So I look at the original CSV and obviously as its comma delimited, numerical values WITH commas in them have escape quotes around them. Right. Clearly this value in escape quotes is being interpreted differently or something. Which is utterly baffling, as how on earth have I never experienced this problem in 20 years of MapInfo use? I must have imported a billion CSVs into MapInfo over my lifetime, and certainly a significant proportion of those will have contained values over 999. I work with coordinates ffs! Not once have I ever noticed an instance of MapInfo changing these values to 1. I SURELY would have noticed.

So I grab an old CSV at random which I know contains floats of 1,000+. And the pop-up I get is defaulting to UTF-8 charset. Eh? Since when? And not even UTF-16? But 8? So I venture into Options and system settings say UTF-16 for new tables. Hmm. It may be just one of those things I have been incredibly unobservant about but that doesn't seem typical to me. Could this be why the initial table in my first encounter was in this format? And if MapInfo is literally defaulting to that format, how come it says it is not supported when I'm trying to open a MIF? And are all these problems such as the truncating of 1,000 to 1 related? 

Am I having a breakdown, or is something going on here? I mean, are we supposed to remove all thousand separators from our raw data now? How long has this been a thing? Dazed and confused.  

------------------------------
Ryan Cook
Knowledge Community Shared Account
------------------------------