5 Replies Latest reply: May 15, 2014 10:03 AM by wendy! _ RSS

    Last record truncated/data lost!

    wendy! _

      Not sure which details are relevant so I will briefly include basic details.

      I get a file with approximately 5000 records, txt format, fixed width, 804 characters per line for each record, "packed" file where I must convert the data:

       

      SAMPLE of unconverted data[/B]

       

      00000000A00000000{00000000{00000000C

      {=0

      A=1

      B=2 etc to 9

      }=-0

      J=-1

      k=-2 etc to -9

       

      I run the report through monarch to separate the records into fields and then import into Access and convert, and do all the reporting that is required with the file.

       

      I have been doing this for several years, no problems. 

      I recently upgraded to Monarch 9.0 - this may be relevant.

       

      When I open the file in Monarch now, the final record at approximately 230 the "data" stops and the rest of the characters display as spaces in "report view", export as spaces, show up in "Table view" as spaces. (total character count for the record is still 804)

       

      HOWEVER when I view the data in notepad or similar, the data is there - the final record is a complete record, all the way out to 804 characters.

       

      This happened on 7 out of the last 11 months files and I do not see a difference on the files that had the "last record" problem and those that did not.

       

      This makes me question the integrity of the data as a whole.

       

      Does anyone have any clue what is going on?:confused:

        • Last record truncated/data lost!
          wendy! _

          Nobody has any ideas I guess...

          sniff sniff:(

            • Last record truncated/data lost!
              Grant Perkins

              Hi Wendy! and welcome to the forum.

               

              Your problem sort of sounds like the report file might be lacking an identifiable end of file marker for some reason. In theory upgrading from 8 to 9 should not change aything much in that respect but there were a few enhancements to the input options that may have changed the way Monarch handles such a situation or recognises any markers that are there. So playing with the input options may be a place to start since defaults may have been reinstated. I can't be more specific I'm afraid since I have not had to dive in there for a long time.

               

              A quick test of the theory could be undertaken. Edit one of the problem files (Probably using Wordpad if it is too big for Notepad)zand add a Carrage Return right at the end. Save the result, then open in Monarch.

               

              If the line is now complete you know what the problem is even if the workable solution is not yet clear!

               

              Also consider whether anything has changed at the source file production stage.

               

               

              HTH.

               

               

              Grant

                • Last record truncated/data lost!
                  Joe Berry

                  I had a problem a few years ago with the data input from our hospital information system.  Someone had inadvertantly hit an escape character which embeded a non-printable character in the data.  When our report writer hit that record it did some stange things.

                   

                  The fact that seven out of eleven months have the issue leads me to think that it is possible the system may be creating this issue when it creates the report.  You can check this out fairly quickly by opening the report in a hex editor or any editor that will display all characters - even non-printable (invisible) characters.  I think that Microsoft Word will also do this, as does Ultra Edit and many more.

                   

                  I would only suggest going this deeply if Grant's suggestion does not get you to the root of the problem.

                   

                  Joe

                    • Last record truncated/data lost!
                      Grant Perkins

                      I had a problem a few years ago with the data input from our hospital information system. Someone had inadvertantly hit an escape character which embeded a non-printable character in the data. When our report writer hit that record it did some stange things.

                       

                      The fact that seven out of eleven months have the issue leads me to think that it is possible the system may be creating this issue when it creates the report. You can check this out fairly quickly by opening the report in a hex editor or any editor that will display all characters - even non-printable (invisible) characters. I think that Microsoft Word will also do this, as does Ultra Edit and many more.

                       

                      I would only suggest going this deeply if Grant's suggestion does not get you to the root of the problem.

                       

                      Joe[/quote]

                       

                      Good advice Joe.

                       

                      Datawatch also provides a character analysis program for free download should it be useful - it has helped me out in the past.

                       

                      http://www.datawatch.com/downloads/Examine.zip[/URL]

                       

                      It runs from a command line so may not be an attraction for everyone but I thought it worth mentioning.

                       

                       

                      Grant