12 Replies Latest reply: May 15, 2014 10:12 AM by Grant Perkins RSS

    can you "lock in" a detail or any other template?

    monarchman7 _

      Here is what I have

       

       

       

       

       

       

       

       

       

       

                               ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

                               dfkljdfsljdfsljdfsljdfsljdlfjsljdsfj

       

                               dsfk;fds;dfsjdfsdfdljdljdldlddldldl

       

                               200-xxxxxxxxxxxxxxxx

       

       

       

       

       

       

       

      pickup1                   ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                           pickup2

       

                                ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

                                ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

       

       

       

       

       

       

                               ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

                               dfkljdfsljdfsljdfsljdfsljdlfjsljdsfj

       

                               dsfk;fds;dfsjdfsdfdljdljdldlddldldl

       

                               200-xxxxxxxxxxxxxxxx

       

       

       

       

       

       

       

      pickup1                   ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                           pickup2

       

                                ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

                                ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

       

       

       

       

       

       

                               ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

                               dfkljdfsljdfsljdfsljdfsljdlfjsljdsfj

       

                               dsfk;fds;dfsjdfsdfdljdljdldlddldldl

       

                               200-xxxxxxxxxxxxxxxx

       

       

       

       

       

       

       

      pickup1                   ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                           pickup2

       

                                ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

                                ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf

       

       

       

       

      /code

       

      The information between "pickup1" and "pickup2" goes with the information above it (the lines before the 200-xxxx stuff, those lines can vary). What I want is the following:

       

      1 column: info in pickup1

      2nd column: info in pickup2

      3rd column: paragraph in between (moved together)

      4th column: 200-xxxxxx stuff

      5th column: paragraph above the 200-xxxxx stuff

       

      Grabbing 1-3 isn't a problem because I can set it to blank field values (2 or 3 spaces) to get the paragraph in-between. I can also easily append column 4. Column 5 is where I get the issue because if I try to append that in a similar way which I did with column 3 it will bleed into that template (since I can only trap by placing non blanks in the middle and blanks outside, there is nothing outside of those paragraphs to grab from).

       

      What I want to do (if possible) is tell monarch to "lock in" what it is grabbing when I make the model to grab the first 4 columns, then append the 5th after, so that it doesn't mess with the others.

       

      As always, any help would be great, thanks guys!

        • can you "lock in" a detail or any other template?
          Grant Perkins

          Hi Monarchman7,

           

          I'm not certain I really understand adequately how your report is structured but I'll start out by saying that the order of columns in the database is flexible (as you likely already know) and so the extractions of data do not of necessity have to be achieved in any order or, in fact, in a single action. (Calculated fields can be used to manipulate the structure of the extract from a generic capture when required.)

           

          Am I correct in thinking that each record in the report (for which we need to define a trap for the DETAIL template as a starting point) is in 2 blocks? The first block always ends with the 200-xxxxxxxxxxx line and the second block always starts with the pickup 1 and pickup 2 fields?

           

          What are you using as the detail template trap?

           

          Are the data blocks always a fixed number of lines (and blank lines between the blocks)?

           

          I think we need to establish the 'architecture' of the report first in order to best work out how it can deconstructed to give the best result for you. It doesn't look really nasty as presented .... but sometimes these things can turn out to be not quite as simple as portrayed.

           

          HTH.

           

           

          Grant

           

          ETA: Also which version of Monarch are we working with here?

            • can you "lock in" a detail or any other template?
              monarchman7 _

              Hi Grant, thank you for the reply.

               

              "Am I correct in thinking that each record in the report (for which we need to define a trap for the DETAIL template as a starting point) is in 2 blocks? The first block always ends with the 200-xxxxxxxxxxx line and the second block always starts with the pickup 1 and pickup 2 fields?

               

              Yes to both

               

              "What are you using as the detail template trap?"

               

              I was going to trap using the "pickup1" and "pickup2" so in the detail template would be those two along with the paragraph in-between. This obviously isn't set in stone though.

               

              "Are the data blocks always a fixed number of lines (and blank lines between the blocks)?"

               

              The data blocks are not always fixed, they vary between 2 and 3 text lines (there may be 4 in some cases, but haven't found one yet). There is a single blank line between each text line (within each block), and usually 3 blank lines between each block, or in this case it would be between the 200-xxxxxxxxxxxxxxxx stuff and the block below.

               

              I'm using version 10.5, thank you again for the help.

                • can you "lock in" a detail or any other template?
                  Grant Perkins

                  If the pickup fields ae always on a line that is a known number of lines below the first line of the first (200-xxxx......) block, including blank line, then is can be successfully used for a detail trap. But if not it's not so good. the 200-xxxxx line might also be an option if the 200-xxxxx line is always at a known line in the record.

                   

                  With a fixed structure things are usually relatively straightforward. With variables this is less likely.

                    • can you "lock in" a detail or any other template?
                      monarchman7 _

                      the pickup lines are always a fixed number of lines below the 200-xxxxxxx stuff, but neither of those are a fixed number below the very first line of what I would use as an append

                       

                      I forgot to mention that 200-xxxxxxx stuff and above applies to multiple stuff below it at times, meaning some of those will have multiple "pickup" stuff below it to put it with. This is why I have the pickup and below as the detail, and tried to append the stuff above.

                       

                      At best right now I can create a 5 line sample for the stuff above (using the 200-xxxxxx to trap) but then I am missing the first line for some of the records. Is there any way I can tell monarch to grab two lines above without it impacting anything else? It's fine for that line to be in a different column, but since the only thing I can trap on from that line is a nonblank or a letter character it screws up the rest of it if I create that append.

                        • can you "lock in" a detail or any other template?
                          Grant Perkins

                          the pickup lines are always a fixed number of lines below the 200-xxxxxxx stuff, but neither of those are a fixed number below the very first line of what I would use as an append

                           

                          I forgot to mention that 200-xxxxxxx stuff and above applies to multiple stuff below it at times, meaning some of those will have multiple "pickup" stuff below it to put it with. This is why I have the pickup and below as the detail, and tried to append the stuff above.

                           

                          At best right now I can create a 5 line sample for the stuff above (using the 200-xxxxxx to trap) but then I am missing the first line for some of the records. Is there any way I can tell monarch to grab two lines above without it impacting anything else? It's fine for that line to be in a different column, but since the only thing I can trap on from that line is a nonblank or a letter character it screws up the rest of it if I create that append.[/QUOTE]

                           

                          OK so that makes it almost inevitable that the Pickup line block has to be a detail template (as you already have it I think) and that the 200-xxxx has somehow to be an Append. That would be fine IF we could tap on the start of the Append block rather than the end since we also have to deal with a variable number of lines. If the number of lines were consistent the trap on the last line would not be a problem. (As you likely already know of course.)

                           

                          Is there absolutely nothing - no character pattern or specific data of anything like that - that would help with finding the first line of that block rather than the last line?

                           

                          At the maximum, including preceding empty lines, how many lines could we allocated to the Append Template sample counting back up from the 200-xxxxx line (which would be the last line, before it would be overlapped by the previous detail record?

                           

                          One option to conside may be the re-ordering of the report file as a pre-process step that makes it easier to identivy and extract the Append Template data required. But let's try to exhaust other possibilities first

                           

                           

                          Grant

                            • can you "lock in" a detail or any other template?
                              monarchman7 _

                              "At the maximum, including preceding empty lines, how many lines could we allocated to the Append Template sample counting back up from the 200-xxxxx line (which would be the last line, before it would be overlapped by the previous detail record?"

                               

                              I think this is the same thinking which led to my quasi solution. I created a 7 line sample which trapped on the 7th line (the 200 stuff). However, this only works if my sample starts with 2 blank lines, the highlighting starts at the top line and ends at blank field values (2). I guess I could have done a 7 liner as well with the same result. I think this being the only type of sample working has to do with how monarch deals with blank lines. The ones with 3 text lines above the 200-xxxx stuff instead of 2 get picked up fine.

                               

                              As I was scrolling through I found some 4 text liners as well, however we are talking about 50-100 in a 10k set so no big deal making manual edits there. I can't make the sample any bigger without screwing bigger things up.

                               

                              On another note, the reason there is a blank line between each line in each block is because of how monarch extracted the text. The PDFs themselves don't have those blank lines (the have big gaps between the blocks themselves, but not blank lines within the blocks). I have heard of two different methods of saving PDFs to possibly make them better for monarch. CutePDF and saving as an XPS. CutePDF doesn't help here, and all I see online about saving as an XPS costs a lot of money (which may be fine if I can be confident that it will work). Have you had any experience with this?

                                • can you "lock in" a detail or any other template?
                                  Olly Bond

                                  Hello Monarchman,

                                   

                                  This may sound a little left-field, but it strikes me that your report would be easier to process upside down. You could trap on the 200xxx lines and stop the template on blank field values of 2, and this would let you cope with the variable lengths of the blocks of text.

                                   

                                  You can turn text upside down in Monarch - by trapping every line of data as one big field, then sorting the table in descending order of Recno(), then exporting the table as fixed width text. Then open this reversed report and trap as above.

                                   

                                  Hope this helps,

                                   

                                  Olly

                                    • can you "lock in" a detail or any other template?
                                      monarchman7 _

                                      hmm, very interesting idea Olly, would work better since the 2nd block varies very little (if at all)

                                       

                                      Kinda wondering if the export to access would be upside down or backwards or both.

                                        • can you "lock in" a detail or any other template?
                                          Olly Bond

                                          Hello MonarchMan,

                                           

                                          Just to clarify, it wouldn't be exported to Access, but to fixed width text, so that you can then open it as a report again (and make use of the templates and advanced field options in Monarch).

                                           

                                          If your data looked like this, would it be easier to extract what you wanted?

                                           

                                          RECORD          BLOCK                                                                               

                                          78                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                     

                                          77                                                                               

                                          76                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                     

                                          75                                                                               

                                          74              pickup1                   ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                           pickup2   

                                          73                                                                               

                                          72                                                                               

                                          71                                                                               

                                          70                                                                               

                                          69                                                                               

                                          68                                                                               

                                          67                                                                               

                                          66                                       200-xxxxxxxxxxxxxxxx                                                      

                                          65                                                                               

                                          64                                       dsfk;fds;dfsjdfsdfdljdljdldlddldldl                                       

                                          63                                                                               

                                          62                                       dfkljdfsljdfsljdfsljdfsljdlfjsljdsfj                                      

                                          61                                                                               

                                          60                                       ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                      

                                          59                                                                               

                                          58                                                                               

                                          57                                                                               

                                          56                                                                               

                                          55                                                                               

                                          54                                                                               

                                          53                                                                               

                                          52                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                     

                                          51                                                                               

                                          50                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                     

                                          49                                                                               

                                          48              pickup1                   ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                           pickup2   

                                          47                                                                               

                                          46                                                                               

                                          45                                                                               

                                          44                                                                               

                                          43                                                                               

                                          42                                                                               

                                          41                                                                               

                                          40                                       200-xxxxxxxxxxxxxxxx                                                      

                                          39                                                                               

                                          38                                       dsfk;fds;dfsjdfsdfdljdljdldlddldldl                                       

                                          37                                                                               

                                          36                                       dfkljdfsljdfsljdfsljdfsljdlfjsljdsfj                                      

                                          35                                                                               

                                          34                                       ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                      

                                          33                                                                               

                                          32                                                                               

                                          31                                                                               

                                          30                                                                               

                                          29                                                                               

                                          28                                                                               

                                          27                                                                               

                                          26                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                     

                                          25                                                                               

                                          24                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                     

                                          23                                                                               

                                          22              pickup1                   ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                           pickup2   

                                          21                                                                               

                                          20                                                                               

                                          19                                                                               

                                          18                                                                               

                                          17                                                                               

                                          16                                                                               

                                          15                                                                               

                                          14                                       200-xxxxxxxxxxxxxxxx                                                      

                                          13                                                                               

                                          12                                       dsfk;fds;dfsjdfsdfdljdljdldlddldldl                                       

                                          11                                                                               

                                          10                                       dfkljdfsljdfsljdfsljdfsljdlfjsljdsfj                                      

                                          9                                                                               

                                          8                                        ddfhdfskjkhdfssdfdfskhjdfskjdfskhjdf                                      

                                          7                                                                               

                                          6                                                                               

                                          5                                                                               

                                          4                                                                               

                                          3                                                                               

                                          2                                                                               

                                          1                                                                                /CODE

                                           

                                          Best wishes,

                                           

                                          Olly

                                            • can you "lock in" a detail or any other template?
                                              monarchman7 _

                                              easier to extract, but I would need the paragraphs to be in the same order they came in, ie can't start with line 3

                                               

                                              pretty awesome that monarch can edit pdfs like this

                                                • can you "lock in" a detail or any other template?
                                                  Olly Bond

                                                  Hello MonarchMan,

                                                   

                                                  I think this is going to get too in-depth to be much benefit to other users in a forum thread, but if you can email me your report I'll happily take a look at it in detail.

                                                   

                                                  Best wishes,

                                                   

                                                  Olly

                                                    • can you "lock in" a detail or any other template?
                                                      Grant Perkins

                                                      A couple of days ago I prepped a fairly full comment on this but somewhere arond the time of hitting the submit button it was lost ...

                                                       

                                                      Suffice it to say that my suggestion was in line with Olly's and that, at least in theory, the sort order could be reverted by some means during the process.

                                                       

                                                      It would be nice if the text blocks could be trimmed (ie, strip out any blank lines that result from the pdf extraction) and captured in their correct line order as MEMO fields. But if that is not practicable it can be dealt with using calculated fields.

                                                       

                                                      I do wonder if the blank lines are simple a 'wrapping' issue on the extract that could be avoided, but if not the extracted file couold be pre-processed to remove the excess blank lines (but not necessarily all! There may be a strong case for keeping or even extending the number of blank lines between records) before being loaded into Monarch.

                                                       

                                                      At this point things do get a little specific so looking at the report off-line seems like a sensible suggestion.

                                                       

                                                      HTH.

                                                       

                                                       

                                                      Grant