5 Replies Latest reply: May 15, 2014 10:15 AM by Olly Bond RSS

    Maximum number of projects in a process

    carpy _

      Could you please tell me if there is a maximum # of projects that you can have in a process.  The process would be monitoring its own folder for text files w/a wildcard character (ie: D:\Monarch\Monarch Reports\process1_all\*.txt) Each project in that process would then have an input folder in that folder for its text files (ie: D:\Monarch\Monarch Reports\process1_all\project1\process1*.txt). 

       

      Are there any limitations or concerns that we need to be aware of?  For example, can an individual process still only monitor up to 60 file specifications (ours above would only be monitoring for 1 file specification)...one of Gareth's responses to another post said that the admin app will enforce this on the Monitor page of Process Properties when adding or editing a process?

       

      Are there any similar limitations that we need to be aware of?

       

      We are running MDP Std version 10.5.0.2 and Monarch Pro version 10.5

        • Maximum number of projects in a process
          Olly Bond

          Hello Carpy,

           

          I think I've seen this request before - from a customer with an ERP system that just dropped all the reports into a big bucket as spool123.txt, spool124.txt, and then you needed to have one process to look at the header of page 1, work out who needed the report and which model to use, and then a different process to attack each type of report.

           

          You could make the case for using DatawatchES instead - have the ERP run the reports in batch overnight, and DatawatchES would ingest them as different document groups and manage the models and security settings for you.

           

          In DataPump, I think you might be able to avoid having too much monitoring (which appears to slow down performance) by using the API to call DataPump itself. So project1 reads the spool123.txt, sees that it needs to use Project1, and so has a post process script which triggers a call to DataPump to execute a process that consists of project1.xprj.

           

          Hope this helps,

           

          Olly

            • Maximum number of projects in a process
              carpy _

              Olly,

              Thanks for replying.  We have 1 process that is monitoring 1 folder.  This process is working right now and contains many projects.  We were just wondering if there is a limit to the number of projects that it could contain.

                • Maximum number of projects in a process
                  Olly Bond

                  Hello Carpy,

                   

                  I've just exported and imported an XML file from DataPump v11.3 to quickly create a process with 200+ projects, set Monitored to Enabled, and DataPump seemed OK.

                   

                  I've run the process without any complaint from DataPump. How many projects do you need in the one process? Perhaps the API would let you trigger a second process after the first if you hit a limit.

                   

                  Note, after running the process, the job log reported successful execution of the first 41 projects, then it reported an I/O error that it couldn't write to the job log itself. I guess that having 200+ small projects all running in one job was optimistic. Please let me know if you want me to run some tests with other options, like one project per job...

                   

                  Best wishes,

                   

                  Olly