0 Replies Latest reply: Mar 30, 2009 8:41 AM by MonUserCJ _ RSS

    Understanding floating traps

    MonUserCJ _

      Hello,

       

         I have been using floating traps to try to extract data from reports, and I'm having some trouble getting sufficiently accurate reports. One thing I don't understand is--can one define a field across a trap character in a floating trap? It seems that sometimes I can, and sometimes I can't. Also, sometimes I'll drag a field across some trap characters next to a floating trap, and, although the trap doesn't expand to include the trap characters, the field widens to variably capture field lengths leading up to the trap characterl

       

         I have run into some problems, because, in the PDF I'm workinng on, the last column of data gets pushed further and further right as one goes down in the PDF document. Can someone explain floating trap characters and floating trap fields to me, or point me to a good resource for learning about this method?

       

         Thanks.

        • Understanding floating traps
          Grant Perkins

          MUCJ,

           

          If you have read all the Help entries and studied the relevent sections of the Learning Guide you are probably about as well equipped as anyone to understand the complexity that might occur with Floating traps. Multiply that exponentially if you are bringing PDF files of variable consistency to the party.

           

          The concept of the floating trap, bearing in mind the original intention was as per the demo file to provide a method of splitting things like log file lines where you could expect certain strings to occur on the line but not easily know where they might be, is excellent. However lines with a lot of fields that need to be split up soon become overly complex and often prove difficult to process because the potential trap characters and positioning are not consistent enough to allow the rules to work.

           

          Add in your unpredictable PDFs and all bets are off.

           

          Two basic concepts to consider.

           

          The trap positions MUST MATCH the Template Sample line. This implies that the sample line must represent the maximum fields sizes that may ever be found for a given field in a specific column. In other words if you were reading a data base the optimum sample line would be one that used the max field sizes as a template and ensured that there was a usable trap character between each field. A SPACE, for example, may not be a useful character in that respect, at least not for some databases. (Actually your trap line may not need to have the full width fields, though that would be the easiest way to work on screen, but you would need to be able to tell the model the maximum size for the field so the effect is the same. If you have a highly variable input source that may not be successful consistently.)

           

          Secondly things tend to get interesting if a field in a given line is unpopulated and ends up with no character width at all. Not good.

           

          What happens is that you end up being unable to define rules that will work consistently. So if will work, maybe, but not provide the results you expect or need.

           

          If I look at a report that may need a floating trap and see more then 3 or 4 fields (or blocks of data that may then need further processing to get to the fields) I tend to start thinking about alternative strategies. Somewhere around 6 fields, unless there is some very evident potential (a known complete maximum size line for a sample AND always a usable field separator), I just don't consider Floating Traps since there is a risk that even if it works for the sample report used for model developement it might not work for the next report that comes along.

           

          An exception would be some form of CSV file - or a SAP "|" separated file -PROVIDING there was a master field width line available to create the model against. But even then I would first consider reading the file as a database, an approach that is likely to be sensible in most such cases.

           

          (On the other hand if you are a guaranteed contract at a daily rate and the contract is to be paid until you manage to find a solution, I would heartily recommend sticking with the PDF and floating trap idea since it should provide guaranteed income for a working lifetime ... just be sure to keep asking for new sample PDF reports to test against so that people know you are busy ... )

           

          Sorry it's not better guidance - perhaps there are others who have found a different approach that is successful - but I think considering all your influencing factors you will have greater success looking at alternative approaches, based on what you have previously described.

           

          HTH.

           

           

          Grant

            • Understanding floating traps
              Data Kruncher

              Grant is quite correct; sometimes using a floating trap approach as a solution just isn't enough.

               

              That's when it's time to [URL is no longer valid].

                • Understanding floating traps
                  MonUserCJ _

                  Thanks for the replies, guys. I decided to just use fixed width fields, and parse those using other means to deal with data split between fields(it's proving to be somewhat tedious, but workable).

                   

                  I appreciate the suggestion on using the maximum size field widths to create a floating trap. I have often been frustrating when I seem to define a floating trap, click OK, only to receive the "Floating trap must match sample exactly". When I shift trap to match the sample, often the results are quite frustrating (I guess that comes from having a report with occasionally empty fields separated by spaces).

                   

                  Also, thanks for the link. That page looks good, and finding out good links makes me feel like part of the data conversion club. :cool: