4 Replies Latest reply: May 15, 2014 10:09 AM by Bill Watson RSS

    Any Free Data Screen Scraping Tools

    adonis _

      does anyone know of any free and simple data screen scraping tools that let you get data from a list which would access data via one of the mainframe apps in a telnet 3270?

       

      have a list of security identifiers which i need to extract a particular field in a telnet 3270 session which is about 1000 long and i dont want to make any mistakes and potentially copy paste errors to add.

       

      any help would be appreciated.

        • Any Free Data Screen Scraping Tools
          Grant Perkins

          Some years ago a company I was working with had a similar requirement. There was certain information we needed about operational things that was held by a third party outsource supplier who seemed unable or unwilling (or both) to make it properly available.

           

          The only access, for one employee only, was via a dial-up line using terminal emulation on a PC. The record we needed were displayed as cluttered 'database' screens and had a few anomalies in layout and could cover multiple 'pages'.

           

          Fortunately the emulate had a screen dump facility to a text file. So our guy wrote a script that would grab the screens for as many pages as were required for each record, concatenate each 'screen' into a single file by a job reference number and then move on to the next record and repeat the process.

           

          I was able to produce a Monarch model that could interpret and re-map the extracted information into the form we needed it and produce a master input file from which we could then export cleansed data split up as required by our systems. I managed to find rules to interpret the anomalies except for about 10 records where the incoming information was just mangled so we simply tagged them and correct them manually.

           

          So I guess that if you are running a PC with an emulation package to access the mainframe you might find what you need there.

           

          There may be free tools around but I have not had a need to be involved with that sort of activity for some years so have no current view of the options available.

           

          I assume there is no utility within the mainframe that can dump the information you need to a file or report of some sort? When you mention the word 'security' I assume it is system security (not stock market) in which case simple methods of access are probably not a good idea!

           

          Grant

            • Any Free Data Screen Scraping Tools
              adonis _

              thank grant, looked online but from what i see they are mostly vb scripts that can do it, i will let you know if i need any insight, have a good weekend

                • Any Free Data Screen Scraping Tools
                  Grant Perkins

                  thank grant, looked online but from what i see they are mostly vb scripts that can do it, i will let you know if i need any insight, have a good weekend[/quote]

                   

                  There are some commercial packages that might be usable in various ways to collect the information you need. Obviously not free though.

                   

                  On that subject if we assume that this is an important task and has to be completed with no error in order to avoid the risk of knock on costs or cleint dissatisfaction, how much is the cost penalty for buying helpful and capable software compared to a 'manual method?

                   

                  It's a rhetorical question that, of course, would also apply to decisions about implementing Monarch.

                   

                   

                  Grant Perkins