3 Replies Latest reply: May 15, 2014 10:00 AM by Nigel Winton RSS

    Limit to file size

    nadineschaefer _

      I did a quick search for my problem and did not find anything.

       

      I have Monarch Pro 7 on my local computer and access to Monarch Pro 8 via Citrix server.

       

      I have a text file that is 18GB. That has data date form 1989 to 2002.

       

      At the moment I have a database that is opening this file and running it through Monarch with model that has run time parameter on it. This works but takes for every (about a day). One of the reasons why is the file is on an external hard drive.

       

      What I want to do (and tried to do) is extract each year’s data and put into a MS Access Database.

       

      I keep running into the problem I am not extracting all the data from this large file by for each year. When I did extracting the data for one year I did it as the following:

       

      There is a text filed that has the data mmddyyyy.

      Set up a filter with *yyyy

      Then scrolled to the very bottom of the data (or what I thought was the end)

      Export to an access database.

       

      So my questions are: Is there a limit to the file size monarch can work with? Is there any limit with exporting from monarch to access (other then access 2GB size limit)?

       

       

      Any insight would help

        • Limit to file size
          Grant Perkins

          Nadine,

           

          The process will create temporary work files in the folder specified for such things, the largest of which  is limited to just a touch over 1Gb which does in fact realte to the Access 2 Gb constraint.

           

          It's not a limit for the number of records as such but rather one for the total data capacity in the process. A lot of fields, and especially a lot of calculated fields, can eat into the space available.

           

          It may not be easy to come up with a usable alternative view.

           

          HTH.

           

           

          Grant

            • Limit to file size
              Olly Bond

              Dear Nadine,

               

              I've come up against the same problem at a recent customer. One option is to run a script against the file to break it into smaller chunks. It might also be worth looking at using Datawatch|ES rather than desktop Monarch.  Rather than create an Access database of each year, you could preserve the report archive and be able to query across it.

               

              Best wishes,

               

              Olly

                • Limit to file size
                  Nigel Winton

                  Nadine

                  Don't know whether this will help, but have you tried openining the text file as a database in Monarch and applying Source Filtering as it is opened. I have used this on somewhat smaller text files, but do not have anything like the size you are working with.

                  If it works you should get a Table of the filtered data you can then export to Access.

                   

                  Best of Luck

                   

                  Nigel