Automate

 View Only

  • 1.  Running MM01 script for large data set, over 1M lines of data

    Posted 06-19-2020 11:45
    Hi Community,
    Our Master Data team is involved in a project where they would like us to extend a very large number of materials from multiple plants to multiple storage locations via MM01.  The problem is the excel sheet they have given us contains over 1 million lines of data.  Has anyone ever loaded that many lines of data using Winshuttle?  If so, what was your approach?  Any Solution Engineers have a suggestion?

    Thanks in Advance!


    ------------------------------
    Monica Anicito | Master Data Analyst
    Ortho-Clinical Diagnostics, Inc., Raritan, NJ
    ------------------------------


  • 2.  RE: Running MM01 script for large data set, over 1M lines of data

    Posted 06-22-2020 08:28
      |   view attached
    Hi Monica -

    We recommend splitting up that data across multiple Excel files or at least multiple Excel sheets.  If you split it up across files, then you could run them in parallel also, provided there is no overlap on the material numbers being extended at the same time.  I would also recommend setting your Studio options so that the excel file is saved after [n] number of records, in case of excel time outs or loss of VPN, etc.  

    Thank you!

    saveaftern


    ------------------------------
    Jennifer Hwang | SE
    Winshuttle North America
    ------------------------------

    Original Message


  • 3.  RE: Running MM01 script for large data set, over 1M lines of data

    Posted 01-14-2021 10:46
    Hello - I have a similar example where I am running an MM01 script for thousands of materials and I can run about 500 materials an hour.  To speed this up I'd like to split into multiple files and run in parallel.  I tried this however and when I run the first template and it begins processing, my second template using the Winshuttle Add on, the "Run" button is greyed out.  Is there something specific I need to do to run two scripts at the same time?  Or could I achieve the same using the scheduler to run on my machine at the same time?

    ------------------------------
    Krista Zinna | Master Data Specialist
    Timken Company | 234-262-2008
    ------------------------------

    Original Message


  • 4.  RE: Running MM01 script for large data set, over 1M lines of data

    Posted 01-14-2021 11:11
    Hi Krista -

    You need to make sure that you are running separate instances of Excel, not multiple excel files from the same instance of Excel.  You will be able to find numerous articles online on how to launch separate instances. The screenshot below shows 2 separate instances of Excel. 

    multexcel

    Thank you!

    ------------------------------
    JENNIFER HWANG | SE
    Winshuttle North America
    ------------------------------

    Original Message


  • 5.  RE: Running MM01 script for large data set, over 1M lines of data

    Employee
    Posted 06-25-2020 20:10
    Hi Monica

    Jennifer is correct - splitting it up in Excel workbooks would work well.

    Another options is to load data into an Access database and run your script from Studio.  It also has size limitations but not number of record limitations.

    Just a thought.

    Hope you're well!
    Sigrid

    ------------------------------
    Sigrid Kok
    PSE | Winshuttle NA
    ------------------------------

    Original Message


  • 6.  RE: Running MM01 script for large data set, over 1M lines of data

    Posted 01-14-2021 11:07
    Hi, another thing I do when loading a large number of records: switching off auto calculation in Excel (if you have calculated values in the workbook) and showing only the title row when running the script. These actions both improve processing speed.

    ------------------------------
    jan ketele | Sr engineer release mgt
    Xeikon Manufacturing NV | +3234431875
    ------------------------------

    Original Message