1 Reply Latest reply: May 15, 2014 9:54 AM by Grant Perkins RSS

    Limiting data imports from an external database

    JoeB _

      Is there any way to set up limits in Monarch Pro when importing data from an external database?   The file that I am accessing with Monarch Pro has multiple fiscal years in it.  When I run the model the results table is so large that it is unmanageable.  If there was a way to limit the year field to only 2002 the result section would be much smaller and the model would run faster.

        • Limiting data imports from an external database
          Grant Perkins

          Joe,

           

          I have always found Monarch to be pretty quick at processing stuff (relative to other software reading the same data file) although 100Mb files over a network can offer challenges. Mainly from network priority parameters.

           

          Is it reasonable to assume you have already tried using a filter in the model when you open the database?

           

          If so I suspect that the solution may be either to pre-filter the database to another DB that gives only the year you want, then run Monarch against that.

           

          Alternatively take the process time hit in a first pass using Monarch to filter for you, then save the resulting data set (export it)and then use it with the main part of your model functionality.

           

          You don't say how you are accessing the original database or what application it is. I have found some applications can be much slower than expected. Similarly ODBC links can seem very slow from time to time - not just within Monarch. In fact performance is worse, in my experience, using Excel or Access. But then even that depends on versions of the O/S, ODBC drivers and all that stuff. Sorting the database into date order using the field you want to select by may help if it is not already set up as a keyed field in the originating file.

           

          Hope this helps somehow.

           

          Grant

           

          Originally posted by JoeB:

          Is there any way to set up limits in Monarch Pro when importing data from an external database?   The file that I am accessing with Monarch Pro has multiple fiscal years in it.  When I run the model the results table is so large that it is unmanageable.  If there was a way to limit the year field to only 2002 the result section would be much smaller and the model would run faster.[/b][/quote]