LAE

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

I have comma-delimited acquisition (input) file with numbers with format like below examples:
2103667XXX
21055074XX
210289732X


The above mean that X represents all possible numbers within those ranges from 0 to 9 for single X, from 00 to 99 for XX, from 000 to 999 for XXX.
Is there a way to Trim those and transform the input to contain ALL individual numbers below such ranges ???

Thank you.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ryeh

I think the attached is what you're asking for. I made that assumption that all ID's will be 10 digits in length. If not, we can add more logic, but otherwise, we're stripping out the X's and then matching the base (number without the X's) with actual ID's.
Attachments:
joinGroupIDs.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

First of all, thank you for your quick reply!

I have adjusted to my project, but, unfortunately, it appears this is running for very long time (or forever, since it is still running now.....).

How should I better change to override emit or default (so that it might work) ?

Thank you.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ryeh

How many records do you have in each input? Since we are doing a cartesian join, we are essentially doing m-by-n (m and n are the number of records in each intput) matches. Two ways around this:
1. Use a hash split. If you're not on the server version, then this won't be available.
2. Break it into a multi-stage match. See Stony's example posted on 3/18: https://infogix.zendesk.com/hc/en-us/community/posts/360051801493

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

I have 45899 records from single numbers (like your ID input) and 51218 records from ranges with the X, XX, or XXX at end of 10-digit number, representing the ranges, as explained before and as you correctly have at your example Group input.
My answers to your questions:
1. My version is not the Server version, it is the Windows version (i.e. installed and running at Windows PC, not at Server, but using Linux Server for Input/Output/tmp files etc.). So, I guess I do not have the hash split option, I cannot see it at Acquisition anyway.
2. Which BRG exactly are you referring to, the test.brg? I have downloaded and I am reviewing now, but I am not confident that this is dealing with the task I need to do..

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

After more careful review of your BRG, I can understand that at first input with the ID:string, you provide a "mapping" for the ranges (groups) at 2nd input.
This is not very practical to do, since I have tens of thousands of range numbers (ending with XXXX, XXX, XX, or X), so I cannot create other input for all those cases at another input like your ID:string.
I need an intelligent algorithm to replace all Xs with values from 0 to 9. For single X could be simple, but for XX, XXX and XXXX it is be more tricky, since, as said at first post, I will need all XX from 00 to 99, all XXX from 000 to 999 and all XXXX from 0000 to 9999 !

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi here is a graph that expands out number ranges into all the individual numbers in each range. Does this meet your needs?

Best regards,
Adrian

Number_Range_Expansion-XXXX_Support.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ppapazaf

Hi, what about replacing XXX with wildcards like "_" and asterix "*" . However I noticed that with x-ref this does not work, matching eg 2103667___ or 2103667*** with 2103667123. It considers wildcards as part of the string. How can we make it the node understand the wildcards and matches the inputs correctly?

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

Hi, Here is a version of the graph that contains two new nodes that support multiple wildcards. The first additional node permits *any* of the specified wildcard characters to be present in the records (in any combination, e.g. 'X_*'). The second additional node permits the use of any one of the specified wildcard characters. All records have to use the specified wildcard (if one is present).

The match for the 'X' wildcard is also now case-insensitive and the logic validates that each record has exactly 10 characters in the 'number'.

Regards,
Adrian

Number_Range_Expansion-Multiple_Wildcards.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: ryeh

So there's no 'cross referencing with wildcards' capability. The closest you could do is with strFind(I) or regexIsMatch(I) within a cross reference. But you would need to do a cartesian join (every record on one input is checked against every record the other input).

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

Hi.
What if I have following requirement: Use ranges extract to exclude all numbers related to ranges and exclude them from any comparison to other input with numbers that belong to those ranges.
For example, for below ranges, I have a 1st ranges extract with following numbers:
2103667XXX
21055074XX
210289732X
Then, I have another 2nd extract with numbers that could or could not belong to the above 1st extract, like:
2103667015
2101234567
2102897328

So:
The 2103667015 of 2nd extract belongs to the 2103667XXX range of 1st extract.
The 2101234567 of 2nd extract does not belong to any range of 1st extract.
The 2102897328 of 2nd extract belongs to the 210289732X range of 1st extract.

Therefore, I want to ignore the 2103667015 and 2102897328 at any comparison with any other provided extract.
So, I am trying to construct either a filter or (most probably) an X-Ref to do the job.

Hope the requirement is clear. Please for help at any way. Thank you.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: awilliams1024

The following should split out individual numbers if they are contained within a number range that is also included in the input data.
The node assumes the extract data sets have been concatenated into a single input data set.

Split_Out_Numbers_Contained_In_Ranges.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

@awilliams1024: Thank you for your reply and the BRG provided. This is close to what I need, but not exactly. To explain with a few words: I want to compare your 1 and 2 and finally keep only the 2101234567, which is the only number of 2 that does not belong to any of the ranges defined at 1.

(at next stage, I will compare result of what we are trying to do here, like the 2101234567, with a 3rd extract containing only clear 10-digit numbers and simply then trying to find: matches, orphans of result here, orphans of 3rd extract).

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: stonysmith

Here is an alternative way to process the data.
This solution is not very dynamic, but it should work for this specific case.
Attachments:
Number_Range_Expansion-XXXX_Support_v2.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

I'm not sure if I've correctly interpreted your requirements, but try the attached graph for an example of a different approach. It may be a bit faster since it's not actually exploding the matching key set into individual records (at least not on disk).
Attachments:
Range Expression Match.brg

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: Kostas75

Originally posted by: mmarinelli
					

I'm not sure if I've correctly interpreted your requirements, but try the attached graph for an example of a different approach. It may be a bit faster since it's not actually exploding the matching key set into individual records (at least not on disk).
			

Cannot open your BRG, because it is from version 6. Can you please save as version 5 and send again?
Thank you.

Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

Originally posted by: mmarinelli

Apologies for the delay; I'm just able to get back to this now. I've attached a v5 BRG, and also made a change to the logic: I changed the comparison operator in my join node from LTE to LT, to properly bound the matching. Hope this works for you.
Attachments:
Range Expression Match v5.brg