LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only
  • 1.  Breaking up data

    Employee
    Posted 11-22-2011 19:18

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: mwillett

    Hi,

    Here's what I'm looking to do.

    I have data in a form like this:
    Key,Value
    1,A
    1,B
    2,C
    2,D
    2,E

    What I want to do is pull out every pair of values held by each key. So the out for the above dataset, I would like this output:
    Key, Val1, Val2
    1, A, B
    2, C, D
    2, C, E
    2, D, E

    The number of values in each key may vary so don't assume this will always be two or three.

    I've attached my graph to date that does this but the issue seems to be on the "emit" statement in that where I want it to emit errors, and if I put the emit at the end then I get the wrong result.

    Can someone assist?
    Attachments:
    BreakData.brg


  • 2.  RE: Breaking up data

    Employee
    Posted 11-23-2011 15:50

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: mwillett

    OK, I worked out a way to do this though it's not the most elegant.

    FYI - if anyone is interested in my solution and also on how it could be improved.
    Attachments:
    BreakData2.brg


  • 3.  RE: Breaking up data

    Employee
    Posted 11-25-2011 11:04

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    Attached is a much less complicated version that takes adavantage of the ability of the JOIN node to produce cartesian result sets.

    I used this condition:
    where 1:value < 2:value
    to avoid these possibly undesirable results:
    A,A
    A,B
    B,A
    B,B
    Attachments:
    BreakData2.brg


  • 4.  RE: Breaking up data

    Employee
    Posted 12-01-2011 08:58

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: rpigneri

    Dear Mike,

    Your original solution required a trick called "do output" allows you to output the more than a single output row for a given input row or rows. This form of output is a special case for many nodes so you have to provide some additional information than in a normal output situation.

    In order to signal to the node that you want to output multiple rows, you need to wrap the emit statement in question in a "do output" block. However, before you can output any data, you also need to set the output metadata for the pin that will be used in the do output block. This metadata setup allows you easily to output the input record and then augment that record with some specific data that comes from the while loop. You can see this feature demonstrated in the attached graph in the node entitled "Do Output Example".

    To show this "do output" feature in action, I replaced the code in your Agg with the following:

    if firstInGroup then {
    ValueList = list()
    }
    
    # Set up the metadata so the node is ready for the extra output rows.
    output 1
    {
    	emit "" as "Val1", "" as "Val2" where false
    }
    
    ValueList = ValueList.append(value)
    
    if lastInGroup then {
    	counterA = 0
    
    	while counterA < len(ValueList) {
    		
    		counterB = counterA + 1
    		while counterB < len(ValueList){
    			Val1 = str(getItem(ValueList, counterA))
    			Val2 = str(getItem(ValueList, counterB))
    			do output 1 {
    				emit Val1, Val2 # You no longer get an error
    						# since this statement is in
    						# a do output block!
    			}
    			counterB = counterB + 1
    	}
    
    	counterA = counterA + 1}
    }
    In magenta--you can see that my skills do not extend to visual art --I have added two blocks of code to address the output issue. The first block sets up the metadata of the output pin for use in the second block. The second block then outputs the data. The green block was added to avoid duplicate output records that were output before all the elements in the list were added.

    I have attached an updated version of the original graph with the changes explained above for your reference.

    Hope that helps,

    Rocco
    Attachments:
    BreakData.brg


  • 5.  RE: Breaking up data

    Employee
    Posted 12-08-2011 00:31

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: mikewillett

    Great, thanks Rocco, Stony,

    In fact, it seemed my code some sort of error in it as when I ran yours and mine, i got different outcomes (albeit minor).

    A bit of checking showed yours to be right - though I haven't fully debugged where mine went astray,

    Mike