11 Replies Latest reply: May 15, 2014 10:04 AM by Grant Perkins RSS

    Parsing a directory file structure

    keithy _

      I am trying to parse out file names and dates from a multi level directory text report.

       

      I want to parse out any files that are in a 3rd level Sub Directory. I can select the path of the directory OK as the structure is fixed. I can also fix the filename and date in the "detail" section. The trouble is, that I get all the files and file dates in the report, and not just the files that are under the sub-directory header I have already selected.

       

      How do I get Monach to only select records "inside" a found header.

       

      Has anyone done any parsing of file structures.

       

      Any help or pointers would be most appreciated.

        • Parsing a directory file structure
          Grant Perkins

          Hello keithy and welcome to the forum.

           

          Is it correct to assume that you are using something like the floating trap or slicing and dicing using one or more of the SPLIT() functions to identify the fields in your directories listing and then adding a filter based on the sub-directory name to select records for the chosen sub-directory?

           

          If so are you able to pre-specify the sub-directory name for the filter or are you seeking to pick the filter definition content on the fly?

           

          Is the third level subdirectory name unique compared to other 3rd level subdirectories in the list?

           

          If so your description of the problem suggests that the filter is not working as expected for some reason. In which case we made need to look at the specific 'report' and model to be in a position to offer useful advice but lets get to that when we know the answers to the questions.

           

          HTH.

           

           

          Grant

            • Parsing a directory file structure
              Olly Bond

              Hello keithy,

               

              (First, a tip of the hat to Steve Caiels who taught me this trick a few years ago...)

               

              It sounds like you've redirected your dir output to a text file, say listing.prn. That's great, and now you've opened it in Monarch.

               

              Directory of C:\Documents and Settings\ob84363\UserData

               

              01-07-2009  10:36            32.768 index.dat

                             1 File(s)         32.768 bytes

               

              Directory of C:\Documents and Settings\ob84363\UserData\GX49E3GT

               

              05-03-2009  18:21                46 IsOnIE6tbPromo[1].xml

                             1 File(s)             46 bytes

               

              Directory of C:\Documents and Settings\ob84363\UserData\M7UZUXYF

               

              23-04-2009  16:55                40 cpa[1].xml

              16-06-2009  15:45                42 pmocntr[1].xml

                             2 File(s)             82 bytes

               

              Directory of C:\Documents and Settings\ob84363\UserData\Q1SBI5E5

               

              15-04-2009  17:25                42 tba[1].xml

                             1 File(s)             42 bytes

               

                   Total Files Listed:

                         20900 File(s)  5.779.097.674 bytes

                          2915 Dir(s)  59.329.363.968 bytes free                        

              /CODE

               

              Now, it sounds from your post, at least to me, that you've trapped the detail of the files correctly, but that you may have trapped your append for the relevant directory with a literal rather than a wildcard trap?

               

              If so, trap the directory names with a wildcard and filter down to the relevant results in the table. If you're using v10, you could make a stopper template after the data you want but that seems like a trickier approach.

               

              If, instead of wanting to show only the files from a particular directory, your challenge is to filter out all files from 3rd level and below, then it's simple.

               

              Create a calculated field Level that counts the number of \ characters in the Directory name, and filter on this.

               

              Level =

               

              len(strip(Directory;strip(Directory;"\"))

               

              Then just a filter of Level<3 should do the trick.

               

              Hope this helps,

               

              Olly

                • Parsing a directory file structure
                  Data Kruncher

                  Olly, That's a nice simplification of my regular character counting formula:

                  len(Directory)-len(strip(Directory,"\"))[/CODE]

                    • Parsing a directory file structure
                      keithy _

                      Thanks for your prompt response guys. I will go through these over the next couple of days and update you on my efforts ! ( It is very hot here in the UK, and the old brain is getting seriously overheated !! )

                       

                      Keithy

                        • Parsing a directory file structure
                          Nick Osdale-Popa

                          Awesome approach to this!

                           

                          I was actually using a 3rd party util that would give me a list of files and their sizes along with folders (and folder size) as well. This output was in text so I would then analyze it through Monarch.

                           

                          Silly me for taking a complicated approach to it when the simple DOS DIR command gives me the desired results as well!

                           

                          And the solution to getting the directory level is also innovative.

                           

                          Thanks!

                           

                          EDIT:

                          Here's some formulas to convert the bytes to various sizes:

                          KB = round(/1024*100000)/100000  

                          MB = round(/1048576*100000)/100000  

                          GB = round(/1073741824*100000)/100000  

                          TB = round(/1099511627776*100000)/100000 

                          /code

                            • Parsing a directory file structure
                              keithy _

                              Well, I tried !!  I think my newly aquired Monarch skills are not up to fully understanding all of this, but it did lead me to solution to match my ability:

                               

                              I used AutoIT to filter, concatenate and then export the details to a text file for Monarch to work on. Not as elegant at that point, but I have extended the AutoIT script to open Monarch, load the project and export the data to an Access Table.

                               

                              Next phase is to use AutoIT to fire off a Crystal Report pdf to the recipient, and then compile the whole lot into 1 exe, and run it on Windows scheduler. Now I am sure someone can do all that with Monarch, but I need to work with the brains I've got !

                               

                              Thanks for making me think differently !!

                               

                              Regards

                               

                              Keithy

                                • Parsing a directory file structure
                                  Grant Perkins

                                  Keith,

                                   

                                  Is there a reason that you need to resort to Crystal Reports to produce a PDF output from the Access Table?

                                   

                                  I'm thinking you could consider creating the output as a PDF directly from Monarch - either from the original report analysis or as a second step from the access table if you need that for some other purpose anyway.

                                   

                                  Is this something that needs to be run on a regular basis? There are a number of ideas that come to mind that may[/I] be useful to you for simplifying the overall process. I say may[/I] because, obviously, I don't know about all that you are seeking to deliver from this.

                                   

                                  HTH.

                                   

                                   

                                  Grant.

                                    • Parsing a directory file structure
                                      keithy _

                                      There is a second "static data" table in the Access Database that I need to link into the data from the Monarch parsed detail.

                                       

                                      I am doubtless that this can be achieved in Monarch, but this task is a "quick and dirty" fix which I can't spend too much time or new learning on just now. The Crystal Route is familiar and quick. 

                                       

                                      My MD has a bee in his bonnet about this data just now, and wants it NOW ! . If he maintains or extends his interest, or I get a quiet couple of hours, I will re-visit Monarch and learn a bit more.

                                       

                                      Regards

                                       

                                      Keithy

                                        • Parsing a directory file structure
                                          keithy _

                                          To pick up the second point about frequency, Grant, the plan is to run daily. This task is to monitor CRM (Maximiser) sychronised packets for each of our remote salesman. A packet is generated for all salesmen, of all database activity over the last 12 hours. The salesman are supposed to update their local copies on their laptops, and then connect in to synchronise on a daily basis, sending in their changes at the same time.

                                           

                                          I'm dropping it into Access, just in case I need to hold historical performance data.

                                           

                                          Regards

                                           

                                          Keithy

                                            • Parsing a directory file structure
                                              Olly Bond

                                              Hello Keithy,

                                               

                                              I know that kind of requirement - MDs' whims are responsible for some very creative quick bodges...

                                               

                                              The more applications, the more points of failure, IMHO. If you are working from reports, and so are working in Monarch, do as much as you can there. Monarch Pro will let you join external data sources (anything ODBC or OLEDB, as well as CSV, Excel of all flavours, Access, etc). And you can then automate the job very easily...

                                               

                                              I'm worried that an Access database of packets might exceed the internal row count or data size fairly shortly - but I guess that depends on how busy your salesmen are

                                               

                                              There are lots of posts here on external lookups - or have a play with the manual - or get in touch with Rob (rob.page@datawatch-europe.com) to find the next UK training course dates. The second afternoon of the course is spent working on your own data, so you'll get a real life fix for your problem there.

                                               

                                              Best wishes,

                                               

                                              Olly

                                                • Parsing a directory file structure
                                                  Grant Perkins

                                                  Keith,

                                                   

                                                  So many inputs and possibilities.

                                                   

                                                  I second Olly's points and would add that having an overview of the potential for Monarch Data Pump and perhaps even BI Server might be useful for you when responding to the instant requests for repetitious jobs. Develop once, automate once and then leave it to run might have some appeal for all concerned.

                                                   

                                                  Have fun with it and don't forget that there is willing help available here on the forum to smooth your Monarch familiarisation path.

                                                   

                                                   

                                                  Grant