5 Replies Latest reply: Dec 26, 2014 9:27 AM by Rebekah T RSS

    How do you manage holidays in Data Pump/Automator?

    Rebekah T

      As we create more automated processes, we are running into the issue of being inundated with error notifications whenever there is a holiday and the input data is not created. Is there a way to script for the holidays, since we know in advance when they will occur? Would like to know how others are dealing with this issue.

      Thanks!

      Rebekah T

        • Re: How do you manage holidays in Data Pump/Automator?
          Olly Bond

          Hello Rebekah,

           

          My first approach would be to suspend the schedules. There isn't an option to avoid holidays that I can see - it's based on the basic Windows Scheduler, so you have just hourly, daily, weekly and monthly options. There are tricks involving Working Days functions and an extra step in each process, but I doubt they would be worth the effort to avoid a few false positive error messages.

           

          Best wishes

           

          Olly

            • Re: How do you manage holidays in Data Pump/Automator?
              Rebekah T

              Thanks, Olly - I appreciate the response. Since Automator has the option to script I was hoping I could point the system to a calendar set up just for running processes, but I see that the script is on the process level, and not the schedule level. I wonder if that might be an idea for an enhancement?

              Thanks again and warmest regards -

              Rebekah T

                • Re: How do you manage holidays in Data Pump/Automator?
                  Olly Bond

                  Hi Rebekah,

                   

                  I'm allergic to scripts or coding of any kind - the other Monarch Experts are much more flexible but I like to do everything through the interface if I can. But one easy option might be deeper inside the Windows Server to simply stop the Automator service on the days you don't want it to run. I don't know if he reads this forum, but Rami Chahine is the gentleman in charge of Automator developments, and he's pretty responsive to good suggestions.

                   

                  Best wishes,

                   

                  Olly

              • Re: How do you manage holidays in Data Pump/Automator?
                Mo Abdolrahim

                Dear Rebekah,

                It is possible to script for holidays.  The example below has been tested in the Automator v12.2+.  A similar technique can be used if you are using Data Pump v11-.   I hope you can follow the following descriptions.

                Happy New Year

                Regards

                Mo

                Create a text file and name it holidays.txt and add all holiday dates in this file.  For example the following are sample holiday dates:

                 

                01/01/2014

                02/17/2014 

                05/26/2014 

                06/04/2014 

                09/01/2014 

                11/27/2014 

                11/28/2014 

                12/25/2014 

                12/26/2014

                 

                Create two processes. Schedule the first process to run every day.  Let’s say you expect the input file to be available at 8:00am, therefore schedule the first process to run every day at 8:30am.  Add a dummy project file to the first process.  The dummy project file should only have a reference to a dummy model and an input, but no export.

                 

                Create an ad-hoc process which uses the actual model and input file and generate an export.

                Add script in the first process pre-export script.  If the current date is not a holiday, then the pre-process script should invoke the second process.

                How to run a process when the current date it is not a holiday (Automator v12+)

                 

                The IsHoliday Process invokes Dummy Project.  The DummyProject does not have any export, just a model and an input file.

                 

                Schedule the IsHoliday process to run every day.  The Pre-Process script reads the holidays.txt file and compare it with the current date.  If a match was found, it will not run the next process (in my example it is the training1 process).  If the current date is not a holiday, then the next process will run.

                   

                 
                 

                 

                 

                Pre-process script

                   

                 

                 

                Dim InstanceName As String = "Automator"   ' see the configurator below 

                Dim ConfigServer As String ="net.tcp://W2008R2DES" ' see the configurator below 

                Dim ProcessName As String = "training1"              ' an existing ad-hoc process 

                Dim HolidayFile As String = "C:\demo\holiday\holidays.txt"   'change the path if different on your system 

                Dim Holidays() As Date = Nothing

                Dim TrackId As String = "" 

                Dim msg As String = "" 

                Dim ok As Boolean = False 

                Dim CurrentDate As Date = Now.Date

                Dim HolidayFound As Boolean = False 

                Try 

                  ok = readHolidays(HolidayFile, Holidays, msg) 

                  If Not ok Then 

                    log.AddEvent("Error...reading holiday file.  " & msg) 

                    Exit Function 

                  End If

                 

                  For each HD As Date in Holidays 

                    If HD=CurrentDate Then   

                      HolidayFound = True 

                      Exit For 

                    End If  

                  Next

                 

                  If HolidayFound Then 

                    Log.AddEvent("Info...Today, " & CurrentDate & " is a holiday, therefore, " & ProcessName & " process will not run.") 

                    Exit Function 

                  End If

                 

                  ok = StartaProcess(ProcessName, msg, TrackId) 

                  If Not ok Then 

                    log.AddEvent("Error in RunaProcess. " & msg) 

                  Else 

                    log.AddEvent("TrackId: " & TrackId) 

                  End If 

                catch ex as exception 

                   log.addevent("Error.." & ex.message) 

                end try

                 

                   

                 

                 
                 

                 

                 

                Global Declarations

                   

                Function readHolidays(byVal FileName as String, ByRef HolidayArray() As Date, ByRef msg As String) As Boolean 

                  Dim ok As Boolean = True 

                  Dim i As Integer = 0 

                  Try 

                      Using reader As StreamReader = New StreamReader(FileName) 

                                While reader.Peek() >= 0 

                                    ReDim Preserve HolidayArray(i) 

                                    HolidayArray(i) = reader.ReadLine 

                                    i += 1 

                                End While 

                                reader.Close() 

                        End Using 

                  Catch ex As Exception 

                    ok = False 

                    msg = ex.message 

                  End Try 

                  Return ok 

                End Function

                   

                Function StartaProcess(byVal ProcessName As String, byRef msg As String, ByRef TrackId As String, Optional byVal InstanceName As String = "Automator", Optional ByVal ConfigServer As String = "net.tcp://W2008R2DES") As Boolean 

                  Dim oPump As PumpAPI = Nothing 

                  Dim ok As Boolean = True 

                  Try 

                    oPump = New PumpAPI(InstanceName, ConfigServer) 

                    TrackId = oPump.StartProcess(ProcessName) 

                  catch ex as exception 

                     ok = False 

                     msg = ex.message 

                  End Try 

                  If Not IsNothing(oPump) Then oPump = Nothing 

                  Return ok 

                End Function

                   

                References & imports

                Add the following references in the script (global script).  The Automator references should be available in the Datapump tools.  Ensure that you use the correct path. 

                 

                1. System.io
                2. DwchServer.PumpAPI

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\DwchServer.PumpAPI.dll

                1. Datawatch.Services.DataPump.Administrator.Contracts

                C:\mo\DW\DES\12.2\DataPumpTools\x64\Datawatch.DataPump.PumpAPI\Datawatch.Services.DataPump.Administrator.Contracts.dll

                 

                1. Datawatch.LoadBalancing.WcfLoadBalancing

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\Datawatch.LoadBalancing.WcfLoadBalancing.dll

                 

                1. Datawatch.LoadBalancing.Contracts

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\Datawatch.LoadBalancing.Contracts.dll

                 

                1. Datawatch.LoadBalancing

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\Datawatch.LoadBalancing.dll

                 

                1. Microsoft.Practices.Unity

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\Microsoft.Practices.Unity.dll

                 

                1. Microsoft.Practices.Unity.Interception

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\Microsoft.Practices.Unity.Interception.dll

                 

                1. Datawatch.ServiceClients

                C:\mo\DW\DES\12.2\DataPump Tools\x64\Datawatch.DataPump.PumpAPI\Datawatch.ServiceClients.dll