Data360 Analyze

Hi Folks

I was hoping that someone could advise me if/how to import a pdf document within D360

Importing multi-structured data from PDF files is inherently a tricky and error-prone process. PDF files may also be images rather than contain any structured data and Data360 Analyze does not include any Optical Character Recognition (OCR) capabilities.

One option would be to use the Transform node with a third party Python module (e.g. PyPDF2)  that can extract text information from a PDF file.

Here is a basic example of how the PyPDF2 module can be leveraged to convert the textual data in a PDF document into a text file.

The attached zip contains the data flow and requires Data360 Analyze v.3.12.1 or higher.

The Python scripts used in the Transform node are also included in archive, together with the input PDF document used by the example data flow.

The data flow assumes the PyPDF2 package has been imported into the 'lib/jython2' directory within the Analyze system's 'site' directory (and is hence on the default search path used when the system attempts to locate the module to be imported).

In practice the PyPDF2 module does extract the structured data from the sample PDF file. However, the original structure is not maintained and there appears to be some aberrations in the field names that were identified by the module. Writing the extracted data to  a text file at least enables you to examine the data before attempting to import it and convert it to structured data in a second step.

As stated at the start of this post, attempting to convert PDF files into structured data is not straightforward and your mileage may vary.

Note, Precisely do not provide support for third party Python packages 

Hi Folks

I was hoping that someone could advise me if/how to import a pdf document within D360

HI Adrian

Thank You for supplying the graph, however I will need some further context on how to run it.

I've uploaded the pdf file to my server, however I am not sure what I need to do to get the first node looking for the pdf?

Looking forward to your reply

Importing multi-structured data from PDF files is inherently a tricky and error-prone process. PDF files may also be images rather than contain any structured data and Data360 Analyze does not include any Optical Character Recognition (OCR) capabilities.

One option would be to use the Transform node with a third party Python module (e.g. PyPDF2)  that can extract text information from a PDF file.

Here is a basic example of how the PyPDF2 module can be leveraged to convert the textual data in a PDF document into a text file.

The attached zip contains the data flow and requires Data360 Analyze v.3.12.1 or higher.

The Python scripts used in the Transform node are also included in archive, together with the input PDF document used by the example data flow.

The data flow assumes the PyPDF2 package has been imported into the 'lib/jython2' directory within the Analyze system's 'site' directory (and is hence on the default search path used when the system attempts to locate the module to be imported).

In practice the PyPDF2 module does extract the structured data from the sample PDF file. However, the original structure is not maintained and there appears to be some aberrations in the field names that were identified by the module. Writing the extracted data to  a text file at least enables you to examine the data before attempting to import it and convert it to structured data in a second step.

As stated at the start of this post, attempting to convert PDF files into structured data is not straightforward and your mileage may vary.

Note, Precisely do not provide support for third party Python packages 

Hi Folks

I was hoping that someone could advise me if/how to import a pdf document within D360

Importing multi-structured data from PDF files is inherently a tricky and error-prone process. PDF files may also be images rather than contain any structured data and Data360 Analyze does not include any Optical Character Recognition (OCR) capabilities.

One option would be to use the Transform node with a third party Python module (e.g. PyPDF2)  that can extract text information from a PDF file.

Here is a basic example of how the PyPDF2 module can be leveraged to convert the textual data in a PDF document into a text file.

The attached zip contains the data flow and requires Data360 Analyze v.3.12.1 or higher.

The Python scripts used in the Transform node are also included in archive, together with the input PDF document used by the example data flow.

The data flow assumes the PyPDF2 package has been imported into the 'lib/jython2' directory within the Analyze system's 'site' directory (and is hence on the default search path used when the system attempts to locate the module to be imported).

In practice the PyPDF2 module does extract the structured data from the sample PDF file. However, the original structure is not maintained and there appears to be some aberrations in the field names that were identified by the module. Writing the extracted data to  a text file at least enables you to examine the data before attempting to import it and convert it to structured data in a second step.

As stated at the start of this post, attempting to convert PDF files into structured data is not straightforward and your mileage may vary.

Note, Precisely do not provide support for third party Python packages 

Hi Folks

I was hoping that someone could advise me if/how to import a pdf document within D360