2 Replies Latest reply: May 15, 2014 9:53 AM by takos _ RSS

    data migration process

    takos _

      As a last year student at the Chalmers University of Technology in Sweden I chose to write about data migration process in my final term paper. I wonder if you can help me with that. First of all I have to investigate tools concerning data migration, i.e. data analysis, data parsing, different work processes for data collection etc.

       

      Second part of my assignment is to find some products that can move data from one database to another. Source data should be in any format as xml, different text formats, while target system is an Oracle database.

       

      I really hope that someone can help me in some part of my task!

      Thank You

      takos  :confused:

        • data migration process
          Grant Perkins

          takos,

           

          You have selected a very big subject which, in my occasional experiences over the last 20 years or more, often has as many approaches, methods and preferred software tools as there are people involved in the process.

           

          In outline I think it is often best to work backwards from the result required and seek the easiest way to get what is required.

           

          The approach will be different if it is a one-way, one time migration to an approach which will be used for a regular production activity. Regular events need to consider operational requirements, security, auditability, process speed, backups, flexibility for adapting to changing requirements over their period of usage, maintenance and so on.

           

          The approach will also depend on the resources available. If the project is populated with programmers it will surely result in a lot of programmed solutions - maybe in several languages.

           

          I have also seem a lot of work performed using editors like vi/edt to manipulate the information available.

           

          I also know of organisations who regularly use  scripting tools to simulate the operator actions to extract information from one system (usually via a report or reports) and the 'key' that automatically into the target system. Sometimes there will be manipulation of the information in the middle of that process.

           

          On the Datawatch web site you should find some interesting white papers and user stories concerning moving data between systems.

           

          Monarch and Data Pump (Monarch data extraction and transformation with automated scheduling and distribution added) are particularly useful tools because they can start with report prints allowing non-technical staff to be easily invlved in the process. Very useful if your technical people are under pressure (and expensive). Yet at the other end of the process - the output in the format required by the target system - Monarch offers a wide range of formats to be selected from a list. Very easy. No programming required unless people choose to extend the automation.

           

          VorteXML extends the possibilities where a dedicted XML type output is required. As a product it is a little more technical than Monarch in some aspects but is still usable by non-technical people.

           

          But as I wrote earlier, if you ask the questions  in a room filled with 1000 programmers my guess is that you would get 1000 (or more) different answers.    

           

          Good luck with your paper.

           

          Grant

          • data migration process
            takos _

            Well, now I understand that this is too big area to cover in just few weeks, but I am going to do my best.

            Data migration that I am writing about is not one time migration; it is in a way continuously updating the source database with data from different target databases.

            Grant, thank you for your help! I am going to check those products you wrote about.

            takos