LAE

Welcome to the LAE community!  Please feel free to start a discussion in the discussion tab or join in a conversation.

Discussions

Members

Resources

Events

 View Only
  • 1.  Read and ReWrite TEXT Files - Python

    Employee
    Posted 09-02-2015 08:37

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: jpstory

    Hi

    I have the following code written to:
    1. Skip the pre-header lines and grab the real header line (where a string of "XYZ" is found)
    2. ReWrite the entire text file with the real header line as the first line so I can use CSV node to import the files in later step

    It just does not work and I couldn't figure out what went wrong, please kindly advise, thanks.

    import braininfo
    import csv 
    
    def setup(brainNodeControlObj, BrainNodeClass):
    	class BrainNode(BrainNodeClass):
    		def initialize(self):
    			super(BrainNode, self).initialize()
    
    			metadata = self.newMetadata()
    			metadata.append("FileNames", "String")
    			self.outputs[0].metadata = metadata
    
    		def finalize(self, val):
    			super(BrainNode, self).finalize(val)
    
    		def pump(self, quant):
    			while quant.permitsRunning(self):
    				inRec = self.inputs[0].read()
    				if not inRec:
    					return False
    				filename = inRec["FileName"]
    				r = 0
    				with open(filename,'rU') as f:
    					reader = csv.reader(f)
    					for row in reader:
    						if row.find("XYZ") <> -1:
    							hd = row.split(",")
    							break
    				success = self.Process(filename,f,hd,reader)
    				
    		def Process(self, filename,f,hd,reader):
    			success = False
    			outRec = self.outputs[0].newRecord()
    			
    			#OverWrite the source file 
    			with open(filename,'w') as f:
    				writer = csv.DictWriter(f,fieldnames=hd)
    				writer.writeheader()
    				for data in reader
    					writer.writerow(data)
    					
    			outRec["FileNames"]=filename
    			self.outputs[0].write(outRec)
    			success = True
    	return BrainNode


  • 2.  RE: Read and ReWrite TEXT Files - Python

    Employee
    Posted 09-03-2015 12:25

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: Bryt Park

    I am looking for the answer to this question as well, could anyone please point us to the right direction, thanks.


  • 3.  RE: Read and ReWrite TEXT Files - Python

    Employee
    Posted 09-03-2015 13:20

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: stonysmith

    The only thing I see 'wrong' here (and it may not be wrong) is that you open the file for reading and you don't close the file before trying to open it for writing.
    You may be running into an operating system block.. can't have it open for both read and write at the same time. I would approach it by writhing out a temporary file, and then deleting the old one, renaming the new one in it's place.

    The other thing I see is that you copy the header to the top of the file. Personally, I see that as a bit of a problem, the 'junk' lines at the top won't read in the necessary number of columns and you're likely to get an error.

    For what it's worth, I built a similar process using LAE nodes. I read in the raw text with DelimitedFile (using a field separator of newline), then a filter to discard records until I find the header, and then use DelimitedOutput to write the file... then pass it to a CSV reader.


  • 4.  RE: Read and ReWrite TEXT Files - Python

    Employee
    Posted 09-04-2015 06:22

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: jpstory

    Can I have the graph you built please, thanks!


  • 5.  RE: Read and ReWrite TEXT Files - Python

    Employee
    Posted 09-04-2015 14:38

    Note: This was originally posted by an inactive account. Content was preserved by moving under an admin account.

    Originally posted by: ryeh

    See attached for an example. First read in a file using the Delimited File node. You want each line to be a new record, but you only want one column. So I put in a random string (@@$$%%) as the FieldDelimiter. Check for the XYZ marker and output from that record onwards. Save as a delimited file, and re-read as a CSV.
    Attachments:
    ReadRewrite.zip