3 Replies Latest reply: May 15, 2014 10:08 AM by Olly Bond RSS

    Were there any left overs?

    Lee Gillie

      I am doing pre-sale evaluation.


      Can Monarch tell if there was anything left over on the page? Such as cause an error condition if there was.


      We are currently scraping data with manually created programs. We parse text statements and reformat them with stylization for printing and producing PDF.


      To protect ourselves, a general approach has ALWAYS been to look for anchors or traps as I think they are referred to by Monarch, and pull the anchor from the page, and pull the data items from the page. At the end of it all we check to see if there is anything left on the page. If there is, it is often either a scenario in the report we had never seen before. Or possibly the customers software vendor has varied the report format, or added new parts or sections. In what we do, this is a fatal condition. It requires investigation, and resolution to where there are once again no left overs on the page after scraping data and known labels a column headers, and so on.


      Is it possible, using Monarch to put this kind of left overs protection in the scraping process?


      BTW - It would be HUGELY appreciated if the product documentation were available to better evaluate purchase.

        • Were there any left overs?
          Data Kruncher

          Hi Lee and welcome to the forum.


          Monarch offers a "verify" process. It checks to see if the painted fields which have been defined as the areas of the report that you want to extract are wide enough to capture the text and/or values that appear in the report.


          Basically it prompts you and displays values that seem to be adjacent to the painted field. At which point you may expand the width of the painted field or ignore the prompt.


          This is not an automatic process however. You must run the "verify" process manually. You'd click an icon in the toolbar to run it. And while Monarch offers a suite of object based methods and properties for automating Monarch via a COM interface, you cannot automate the verify process.

            • Were there any left overs?
              thompssc _

              If you also spring for Monarch Data Pump I believe it does allow automation of the "Verify" feature embedded in Monarch.

                • Were there any left overs?
                  Olly Bond

                  Hello Lee,


                  DataPump allows for automatic verification of field boundaries, and can also trigger messages when the number of rows meeting a certain filter is zero, or under or over a limit. Combining these options with multiple models it's possible to be pretty sure that you've got all the data you need.


                  One approach is to trap the report in a second model with an empty trap, so selecting every line of every page, then to use a calculated field to determine whether the line should be considered as a valid detail, and to use an external lookup to the table from your main model as a check that you have got the expected data.


                  With Monarch 10.5 and Excel 2007 or above, you can also use the Monarch Audit feature to export the source report embedded into the Excel table. This won't give you automatic verification, but will make it easier for users to check the source of records.