23 Replies Latest reply: May 15, 2014 10:01 AM by Tim Racht RSS

    Monitoring Problem

    Jeff C

      New to DP, had it for 2 days now and trying to get my first process to work.  I have a process called 8200 that works great when run manually but if I enable monitoring nothing happens.  Being the day before the 4th of July there is no one at support so I left them a message and am asking for help here.  Don't know what to ask because I don't know what the problem is.  Any assistance will be appreciated!

      Jeff C

        • Monitoring Problem
          joey

          Several questions that I'm sure Tech suport will be asking as well.

           

           

          - Does the process work fine if you kick it off manually through Data Pump. 

           

          - Is there an input distribution defined?

           

          - What file are you monitoring for?  What is the input to the project?  Does the file exist?  You might try double clicking on the monitored file specification to select the file name and click start->run and paste the string of the file you are looking for.  If windows can't find the file, chances are you have a typo in the monitored location.

            • Monitoring Problem
              Jeff C

              Yes, process works perfectly if selected for manual processing.

               

              Yes, there is an input distribution defined...file extension is renamed to .first

               

              Monitoring for the data file rbir8200.lis, a report generated out of our Collier Jackson Circulation software.  File is pushed by a job in UDMS to a server which is suppose to be watched.

               

              Yes, the file exists.  I keep copying it to the folder to test the monitoring.  Nothing seems to work.

               

              Thanks for your help.  Let me know if you have any more ideas.

               

              Jeff C

                • Monitoring Problem
                  Jeff C

                  While I have you here Joey, could you explain to me what the script tab is?  Am I able to export a monarch summary to excel then using the script open the file and run a macro?  That would be most useful.

                   

                  Jeff C

                    • Monitoring Problem
                      joey

                      Is the monitoring specification for rbir8200.lis using a UNC path or is it using a drive letter that has been mapped to the server? 

                       

                      Also, when you open the process and click the monitoring tab, does it say "Monitoring has been disabled due to setup errors"

                       

                      Is the checkbox checked in the Monitor tab?

                       

                      If you are using Data Pump 9, does the rightmost column have MD, MD, SE, SD, or AH in it? 

                       

                       

                      As far as scripting, you can run scripts at several points in the execution cycle.  I almost always use the Post Export script on my last distribution to do some advanced Excel formatting features. I also use it to compact Access databases and other routine tasks. It really opens Data Pump up possibilities limited only by your creativity (or VB .Net / batch script coding ability).

                       

                      Give it a try sometime and add code to call an external batch script.

                        • Monitoring Problem
                          Jeff C

                          It has been determined by our IT department that the Services and Applications\Services Monarch Data Pump 9 Service is stopping even thought it shows Startup Type of Automatic after something that is monitored has ran.  IT is talking with Customer Support about the issue.  I will leave that one up to the professionals.

                           

                          I am about to search for a thread on having one process wait on another process to finish before it starts.  Is there a easy way to do this?  I have one process that must export a table from Monarch before the other process can begin and the two processes are trying to access the file at the same time. 

                           

                          Still on the upside of the learning curve!

                           

                          Thanks for you help.

                          Jeff C

                            • Monitoring Problem
                              joey

                              I've had the service crash also, but they fixed most of issues by V9.  Try the event viewer, and there should be meaningful codes listed.  You can shutdown/restart the server and check the event viewer for errors caused by the Data Pump.

                               

                              Try http://www.monarchforums.com/showthread.php?t=2146[/URL] on kicking off one process after another.  I haven't tried this myself.  Usually I can have one process output a file that is used as input or a trigger to start another process.

                                • Monitoring Problem
                                  Jeff C

                                  I have tried to do that but even though I have the "multiple projects per job" selected process 2 tries to open the file before process 1 creates it.  Its coming along.  At least I can now run the processes manually in a fraction of the time it took to do it in Monarch.  Here is the error log.  Process 1 creates PRODUCT_DRAW.XLS for process 2.

                                   

                                  + 07-09-2008 13:09:58 Created.

                                       

                                  time  2008-07-09T13:09:58

                                   

                                   

                                  + 07-09-2008 13:09:58 Process prerequisites were not met: File '
                                  pmcserver\e\CIS FILES\DataPump\MANIFEST\PRODUCT_DRAW.XLS' does not exist.

                                       

                                  time  2008-07-09T13:09:58

                                   

                                   

                                  + 07-09-2008 13:09:58 Failed: DwchServer.NoRetryInfoException: Process prerequisites failed, but no retry information is available. Process will not be retried. ---> DwchServer.SpecifiedFileDoesNotExistPrerequisiteException: File '
                                  pmcserver\e\CIS FILES\DataPump\MANIFEST\PRODUCT_DRAW.XLS' does not exist. at DwchServer.FilePrerequisite.a(String A_0) at DwchServer.FilePrerequisite.Test(String strProcessName) at DwchServer.ProjectInputs.TestPrerequisites() at DwchServer.ProcessHandler.b(ServerDB A_0, JobRow A_1, JobLog A_2) --- End of inner exception stack trace --- at DwchServer.ProcessHandler.b(ServerDB A_0, JobRow A_1, JobLog A_2)

                                       

                                  time  2008-07-09T13:09:58

                                  alert  SystemAlertProcessFailed

                                   

                                   

                                  + 07-09-2008 13:09:58 Distribution started

                                       

                                  time  2008-07-09T13:09:58

                                  source  distributor

                                   

                                   

                                  + 07-09-2008 13:09:58 Distribution completed

                                    • Monitoring Problem
                                      joey

                                      Try removing the prerequisites for the projects.  Just set the input files to Input Required.

                                        • Monitoring Problem
                                          Jeff C

                                          No prerequisites set.  Input files just must exist.  And they do.

                                           

                                          This is what our IT guy sent to Customer Service.  They had me try setting up a process with local data and models.  Still does not work.  Any ideas will be appreciated.

                                           

                                           

                                          What I’ve observed from working with Jeff is that the monitor stops working – the service is still running, the proper file is in the directory being monitored, but the application does not pick up the file for processing.  After we stop & restart the service the monitor acts normally, for a while (I’m not sure exactly how long yet – maybe minutes, maybe hours).

                                            • Monitoring Problem
                                              joey

                                              Are you saying you have two separate processes, one for each project? 

                                               

                                              What I suggest is to have two projects in one process. The first one should be first and the second one second.

                                               

                                               

                                               

                                               

                                              In the Monarch data Pump console, you should have three folders under Monarch data Pump on the left: Serve Settings, Event Viewer, and Computer Management.  Expand the Event Viewer folder.  You should have a monarch Data Pump Events and a Monarch Data pump Service Events listed.

                                               

                                              Try to shut down and restart the Pump using the buttons in the Server Status screen.  Then check the Monarch Data Pump Events list.  You can look at the most recent few entries to look for something like your file monitoring stopped.

                                               

                                              Or you could try sorting the list by the Event number and look for 210.  I've recieved messages such as this:

                                               

                                              DwchServer.ReloadMonitoringTablesException: Error occurred while loading the monitoring tables. ---> DwchServer.InvalidFindFirstChangeNotificationException: Error setting up initial change notification for
                                              central1\sharedat\mfinfo - error 56

                                              at DwchServer.ChangeNotification..ctor(String strPath, Int32 iWatchSubtree, FileNotifyChange notificationType)

                                              at DwchServer.w.b()

                                              at DwchServer.d.d()

                                              --- End of inner exception stack trace ---

                                              at DwchServer.d.d()

                                              at DwchServer.d.e()

                                              /SIZE

                                               

                                              Hopefully you can find an event in the event viewer that will give a clue or more information for Tech support on why file monitoring is crashing.

                                                • Monitoring Problem
                                                  Jeff C

                                                  You are not going to believe what I found the error to be....  When the IT department set up the Data Pump and Monarch software on the server it did not occur to me that Data Pump will be using the Monarch on the server for the processing.  I finally figured out that the Monarch did not "see" the reports our system generates.  Monarch was looking for .txt and other file types, I just set the file type to "*" and everything works perfectly!  DUH!

                                                   

                                                  Thank you for your help though.  From now on anytime Monarch is install I will be sure to make that one change!

                                                   

                                                  Jeff C

                                                   

                                                  PS  I have three projects named in one process.  The problem seems to be that the first process has not finished writing the file before the second process tries to read it.  Any suggestions on how to slow things down a few seconds?  I get an error that the second process can not access the file because it is in use by the first. 

                                                   

                                                  I have also set it up so that the second process is monitoring for the file generated by the first process and the third process is monitoring for the file generated by the second.  Still with the same error.  Not sure how to set up a multiple pass with Monarch using Data Pump.  Any suggestions?

                                                    • Monitoring Problem
                                                      Tim Racht

                                                      I am also receiving the same message on a process that we are trying to run.  The process is scheduled for an input file database (dbf file).  The rpocess fails and indicates that the Process prerequisites were not met file does not exist.

                                                      I have verified the file exists and that I have access.  I am not sure where you have changed the file types but I think we changed ours in Monarch.  If anyone has any ideas to resolve this issue would be helpful.

                                                        • Monitoring Problem
                                                          Chickenman _

                                                          This has happened on mullti-project processes so I set the retry to minute and retry 5 times for all processes. That gets the first project completed and the subsequent ones finish.

                                                            • Monitoring Problem
                                                              joey

                                                              Another option is to log into the server where Data Pump is, unmap all network drives, and try running the process manually.  If you have anything that is a mapped letter, Data Pump won't take it.

                                                                • Monitoring Problem
                                                                  Tim Racht

                                                                  I am still unable to have our process run successfully.  The process fails and still indicates that the Process Prerequisiteswere not met file does not exist.  Would I need full read and write access to this file?  If any one has any other ideas to help us resolve this issue it would be greatly appreciated.

                                                                    • Monitoring Problem
                                                                      joey

                                                                      - The data pump service should run as an administrator account and have access to the files.

                                                                       

                                                                      - Does the monarch project work when the files cannot be found, if you unmap all network drives?

                                                                        • Monitoring Problem
                                                                          Tim Racht

                                                                          We have all of the network drives unmapped as we are using the unc paths and the process will not work.  The data pump service does have access to the file as I can access the project and it does open the database (dbf file) and displays the information we are looking for.

                                                                            • Monitoring Problem
                                                                              joey

                                                                              Does the data pump service run under your user account?  Ours runs as an administrative account.

                                                                                • Monitoring Problem
                                                                                  Tim Racht

                                                                                  As an administrative account as we have a separate server set up for these processes.

                                                                                    • Monitoring Problem
                                                                                      joey

                                                                                      Have you tried logging into the server as the administrator that the service runs under and seeing if it will run manually?

                                                                                        • Monitoring Problem
                                                                                          Tim Racht

                                                                                          We are logged into the server as the administrator now and we are actually trying to run this manually to see if we can have this process run successfully.  We have tried to schedule it and run it manually with the same prerequisite error, file not found.  We have verified that we have the unc paths for all network files and and we do.  I am not sure what else we can verify to have this process run.  If you have any other ideas that would be appreciated.

                                                                                            • Monitoring Problem
                                                                                              joey

                                                                                              When you logged in as the admin, did you open the projects and run the exports manually?  What was the result?

                                                                                               

                                                                                              Are there any prerequisites for any of the projects?  How many projects are there?  What are the input files to each project?

                                                                                                • Monitoring Problem
                                                                                                  Tim Racht

                                                                                                  We can open the project and run the exports manually successfully.  There are no prerequisites for this project and this is the only project.  We have tried to set the prerequisites to the input file but this just gives us another error message. The input file is a lotus approach database (dbf file) located on our network, which is set up with the unc path's. A have included the error message below.

                                                                                                  Process: AR APPROACH TEST Job ID: 17618

                                                                                                   

                                                                                                   

                                                                                                   

                                                                                                  Events  Show Detail   Hide Detail 

                                                                                                   

                                                                                                   

                                                                                                  Date/Time Event Description

                                                                                                  + 12-22-2008 11:38:34 Created.

                                                                                                       

                                                                                                  time  2008-12-22T11:38:34

                                                                                                   

                                                                                                   

                                                                                                  + 12-22-2008 11:38:34 Process prerequisites were not met: File '
                                                                                                  casualty\sys\USERS\COLLECT\Approach\AssignedRisk\policy.dbf' does not exist.

                                                                                                       

                                                                                                  time  2008-12-22T11:38:34

                                                                                                   

                                                                                                   

                                                                                                  + 12-22-2008 11:38:34 Failed: DwchServer.NoRetryInfoException: Process prerequisites failed, but no retry information is available. Process will not be retried. ---> DwchServer.SpecifiedFileDoesNotExistPrerequisiteException: File '
                                                                                                  casualty\sys\USERS\COLLECT\Approach\AssignedRisk\policy.dbf' does not exist. at DwchServer.ProjectInputs.TestPrerequisites() at DwchServer.ProcessHandler.b(ServerDB A_0, JobRow A_1, JobLog A_2) --- End of inner exception stack trace --- at DwchServer.ProcessHandler.b(ServerDB A_0, JobRow A_1, JobLog A_2)

                                                                                                       

                                                                                                  time  2008-12-22T11:38:34

                                                                                                  alert  SystemAlertProcessFailed

                                                                                                   

                                                                                                   

                                                                                                  + 12-22-2008 11:38:34 Distribution started

                                                                                                       

                                                                                                  time  2008-12-22T11:38:34

                                                                                                  source  distributor

                                                                                                   

                                                                                                   

                                                                                                  + 12-22-2008 11:38:35 Distribution completed

                                                                                                       

                                                                                                  time  2008-12-22T11:38:35

                                                                                                  source  distributor

                                                                                                  value  complete