List of Contributions

Gerry Mullin

Contact Details

My Content

1 to 19 of 19 total

RE: How to read CSV files within a ZIP file

Posted By Gerry Mullin 11-12-2024 12:33
Found In Egroup: Data360 Analyze \ view thread
The only thing I can think of then is that I usually replace the \ with a / for all file names but I didn't in this instance for absolutely no particular reason. I think I saw before shutil.move() doesn't work with a backslash, I could be wrong just going off memory of the backslash causing problems ...

RE: How to read CSV files within a ZIP file

Posted By Gerry Mullin 11-12-2024 11:24
Found In Egroup: Data360 Analyze \ view thread
The file is on my c drive. Did you edit it the ZipFile and ExtractDirectory properties?

RE: Curl command to HTTP node

Posted By Gerry Mullin 11-11-2024 10:15
Found In Egroup: Data360 Analyze \ view thread
Thanks for the data flow Adrian. I did not realize the body had to be created like that. I'm still having problems with getting the upload to S3 to work but I'll spend some more time on how this format is and hopefully get it to work! I appreciate the help!

RE: How to read CSV files within a ZIP file

Posted By Gerry Mullin 11-11-2024 10:14
Found In Library: Data360 Analyze

RE: How to read CSV files within a ZIP file

Posted By Gerry Mullin 11-11-2024 10:14
Found In Egroup: Data360 Analyze \ view thread
Extracting from a zip is pretty straight forward in python this will do it: import zipfile with zipfile.ZipFile(path_to_zip_file, 'r') as zip_ref: zip_ref.extractall(directory_to_extract_to) I did a small flow that will delete the directory you want to extract to and recreated it, ...

Curl command to HTTP node

Posted By Gerry Mullin 10-09-2024 09:42
Found In Egroup: Data360 Analyze \ view thread
I have this curl command for posting a file to an S3 bucket which works fine, but I would rather use the HTTP node. I couldn't figure out the right syntax as I constantly get bad request back. For example I tried adding to the body something like this: key=xyz&bucket=name&policy=xyz I also tried ...

RE: End of month

Posted By Gerry Mullin 09-05-2024 10:44
Found In Egroup: Data360 Analyze \ view thread
I've used this. Peter's way looks to work too, the only difference I see is you don't need to import calendar but really they both do the same thing. def last_day_of_month(any_day): # this will never fail # get close to the end of the month for any day, and add 4 days 'over' next_month = any_ ...

Data Flows that are "Locked for Editing"

Posted By Gerry Mullin 03-29-2024 09:24
Found In Egroup: Data360 Analyze \ view thread
Any time a user has made a change to a data flow and exits without saving, you will get a warning saying its locked for editing by the user. Is there any good way, even as an admin, to remove this message and revert to the last saved copy? I haven't found any easy way to get around this without deleting ...

RE: Batch insert query optimization

Posted By Gerry Mullin 03-29-2024 09:20
Found In Egroup: Data360 Analyze \ view thread
Just to give my experience on loading to tables, I have found the JDBC Store to be generally quicker. I can tell you my experience with loading to Snowflake, I had an extremely slow loading table that took hours to load less than 100k rows. Other tables would load far more than that in minutes so I couldn't ...

Paging on HTTP node

Posted By Gerry Mullin 03-18-2024 13:37
Found In Egroup: Data360 Analyze \ view thread
We currently using the requests package in a Python node rather than the HTTP node to pull back data from an API call that has an unknown number of pages. I would like to move to the HTTP node and it looks like there are Paging options but I can't seem to get it to work. Here is what the URL looks ...

RE: S3 Put with KMS keys

Posted By Gerry Mullin 03-06-2024 09:36
Found In Egroup: Data360 Analyze \ view thread
Thanks Adrian. ------------------------------ Gerry Mullin Avalara Inc Seattle WA ------------------------------

S3 Put with KMS keys

Posted By Gerry Mullin 03-05-2024 10:36
Found In Egroup: Data360 Analyze \ view thread
We are trying to upload a file to an S3 bucket. Using the standard encryption works fine, but when we change to use Server-side encryption with AWS Key Management Service keys (SSE-KMS) and specify a Encryption key ARN, it will continue to upload but the encryption type is Server-side encryption with ...

RE: Desktop version

Posted By Gerry Mullin 09-11-2023 14:35
Found In Egroup: Data360 Analyze \ view thread
I removed java.exe from Path and JAVA_HOME from my Environment Variables and it installed. ------------------------------ Gerry Mullin Avalara Inc Seattle WA ------------------------------

Desktop version

Posted By Gerry Mullin 09-07-2023 13:10
Found In Egroup: Data360 Analyze \ view thread
I just installed the free desktop version of 3.12.2 and this message popped up at the end. I did not restore from backup or perform an upgrade, there was no other instance of Analyze on my laptop at the time of installing. HTTP node won't run, says its missing Furl module, other nodes to seem to work. ...

RE: Identify records containing cyrillic characters

Posted By Gerry Mullin 04-18-2023 13:16
Found In Egroup: Data360 Analyze \ view thread
You could just try to convert the value to a string and if it fails it would indicate its not a windows-1252 character. This is on a transform node where I added a second out2 pin. ------------------------------ Gerry Mullin Avalara Inc Seattle WA ------------------------------

RE: Auto-defined run property for Analyze Username

Posted By Gerry Mullin 03-01-2023 12:27
Found In Egroup: Data360 Analyze \ view thread
In a Transform node, this will get the username of whoever is running the data flow. username = node.properties.getString("","graph.username") ------------------------------ Gerry Mullin Avalara Inc Seattle WA ------------------------------ ------------------------------------------- ...

RE: Count rows

Posted By Gerry Mullin 01-13-2023 08:49
Found In Library: Data360 Analyze

RE: Count rows

Posted By Gerry Mullin 01-13-2023 08:49
Found In Egroup: Data360 Analyze \ view thread
@ Precisely - ​​I tried to upload an LNA here but I get told its not supported. Considering the forum perhaps we should allow LNA files? Uploading as a zip. ------------------------------ Gerry Mullin Avalara Inc Seattle WA ------------------------------ ------------------------------------------- ...

RE: Count rows

Posted By Gerry Mullin 01-13-2023 08:47
Found In Egroup: Data360 Analyze \ view thread
You can count the rows separately and bring them into the main dataset via a lookup. Then its just a case of filtering what you want. ------------------------------ Gerry Mullin Avalara Inc Seattle WA ------------------------------ ------------------------------------------- Original Message: ...