3 Replies Latest reply: May 15, 2014 10:15 AM by Olly Bond RSS

    Data Pump Input File Name Monitoring

    econ664 _

      Hi, I'm using MDP v 11.3.3 and Monarch v11.

       

      I'm sort of new to using Data Pump and I had a question about folder monitoring.

       

      Is it possible to have a process monitor a folder for differently named input files? Like, say I get the same file every month, 201301.csv for January, 201302.csv for February, and so on.

       

      Right now, I can have monitoring for 2013*.csv, but I have to select a single input file, so it always looks for 201301.csv whenever the monitoring conditions are met. I realize that even if it worked the way I wanted, it would still output to the same file.

       

      Is there any way to set it up in a single process to monitor for all twelve of these files, and export to differently named files? Or should I just copy the process twelve times, one for each month?

       

      Thanks,

        • Data Pump Input File Name Monitoring
          Olly Bond

          Hello econ, and welcome,

           

          Monarch can open up to 1024 report files in any project - that includes fixed-width text, PDF, HTML and CSV files. To define a project to open multiple named CSV files at once, go to File, Open, Open Database, browse to the folder, and CTRL + click the files you want. Then save the project.

           

          If you want to change from the literal filenames that you have there, you can edit the project not in Monarch, but in the XPRJ Editor provided with DataPump, which allows you to handle wildcards like *.csv in the Input tab.

           

          Exporting to differently named files is possible, making use of the File() function. There's certainly no need to duplicate the project twelve times if you've got DataPump, as this allows for pretty flexible handling of export file names based on parameters.

           

          Best wishes,

           

          Olly

            • Data Pump Input File Name Monitoring
              econ664 _

              Olly,

               

              Thank you so much for your response. Could you please expand upon the use of the File() function, or perhaps point me in the direction of some reference documentation?

               

              Also, is opening multiple files only possible with delimited or flat files? What about database files like excel?

               

              Thanks,

                • Data Pump Input File Name Monitoring
                  Olly Bond

                  Hello econ,

                   

                  No, you can't open lots of Excel files, or even worksheets, at once. Some people have been known to script their way around this and export each Excel file to CSV.

                   

                  You can perform left outer joins - in Monarch these are called "external lookups" and called from the Table window. A single model can support up to nine external lookups.

                   

                  File() just returns the full path from Windows of the File() being worked on. To get rid of the path, rsplit(File();2;"\";1) will return just the filename.

                   

                  File(), ID(), Page(), Line(), Rowno(), Recno(), TextLine(), and Column() are covered in a webinar I recorded for v10.

                   

                  I think it's still online here: https://www.livemeeting.com/cc/datawatch/view?id=QPC3BW-1[/url]

                   

                  Best wishes,

                   

                  Olly