Standardized data is a humongous feat to achieve. There are articles written by data specialists around the globe, speaking of the complexity incurred in standardizing a dataset.
Business-trade-name, being the most important identifier of a POI, needs to be standardized.
For example, here are just a few ways of how the famous retail chain 7 ELEVEN is represented by our major vendors like Dun & Bradstreet and TomTom:-
- 7/11
- 7 ELEVEN
- 7-ELEVEN
- SEVEN 11
- 7ELEVEN
- 7-ELEVEN
- 7-11
- 711
No matter which DBMS is being used, taking all the above occurrences into consideration would require the end customer to manipulate his SQL query with the usage of wild cards and the keywords in all possible forms.
With the help of latest technologies like Natural Language Processing, String-Matching Algorithms like Cosine & Levenshtein, pySpark, Jupiter notebook, etc, we introduced the BRANDNAME column which provides the customer with a single standardized business-trade-name for a POI.
Brandname Standardization has been successfully applied in 24 major Countries and 3000 distinct Brandnames.
------------------------------
Surojeet Ghosh,
Senior Data Engineer,
Data Development
Noida, India
------------------------------
Original Message:
Sent: 06-13-2019 14:43
From: Keith Wright
Subject: Whats your favorite data attribute?
Very surprising, why is that Dan?
------------------------------
Keith Wright
Pitney Bowes Software Ltd
HENLEY-ON-THAMES
Original Message:
Sent: 06-13-2019 14:38
From: Dan Adams
Subject: Whats your favorite data attribute?
Keith-- awesome question. Because I was just looking at the POI product, I'll say POI Brand name. It's much harder to get that right than you'd think.
------------------------------
Dan Adams
Pitney Bowes
White River Junction VT
Original Message:
Sent: 06-13-2019 14:37
From: Keith Wright
Subject: Whats your favorite data attribute?
Mine is 'Outdoor Activities' from Consumer Data!
------------------------------
Keith Wright
Pitney Bowes Software Ltd
HENLEY-ON-THAMES
------------------------------