If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > Data replication algorithm

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 09-28-03, 21:37
A340-313X A340-313X is offline
Registered User
 
Join Date: Aug 2003
Posts: 11
Data replication algorithm

Hello

I am writing a database replicator software to replicate new records from a source DB2 database to a target MySQL database. However, the replication seems to be rather slow, cuz the software basically grabs records from the DB2 database 100,000 at a time and does comparison of the primary keys to see if they exist in the target table.

I would like to ask if anyone has a better method for doing this job? I was actually thinking of running a small daemon program on the source machine which detects if new records have been added and then zaps these records over to the target machine but the source machine seems a little overloaded at the moment.

Thanks a lot for the help! Really appreciate it.

Kind regards,
A340-313X
Reply With Quote
  #2 (permalink)  
Old 09-28-03, 22:23
Marcus_A Marcus_A is offline
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,198
You could create triggers on the DB2 tables which would log inserts, updates, and deletes to another DB2 table (with an extra column to indicate insert, update, or delete). Then you would have a table of just the items that changed on the table. Obviously you would want to delete the rows off the DB2 log table as you posted them to MySQL.
Reply With Quote
  #3 (permalink)  
Old 09-29-03, 03:16
A340-313X A340-313X is offline
Registered User
 
Join Date: Aug 2003
Posts: 11
Thanks for your prompt reply!

Is there any other way other than triggers? The only problem is I don't seem to have sufficient control over the DB2 database to create stuff like triggers. If not, thanks a lot anyway!
Reply With Quote
  #4 (permalink)  
Old 09-29-03, 03:27
Marcus_A Marcus_A is offline
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,198
There may be software packages available that read the logs and output them in a user friendly way. But I don't have any specifics on that.

IBM offers a product called DataPropagator, but I believe it only works on DB2 mainframe. I would check out the DB2 Replication Guide. Maybe others have an idea that might help you.
Reply With Quote
  #5 (permalink)  
Old 09-30-03, 15:36
Marcus_A Marcus_A is offline
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,198
Here is an IBM Redbook that may be of Interest:
"A Practical Guide to DB2 UDB Data Replication V8"

http://publib-b.boulder.ibm.com/Redb...=0,replication
Reply With Quote
  #6 (permalink)  
Old 09-30-03, 16:33
cchattoraj cchattoraj is offline
Registered User
 
Join Date: Mar 2003
Posts: 343
There's a tool called recovery expert which can read db2 logs and create undo statements - I do not know whether it can generate redo statements. I's worth a shot.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On