Unanswered: Tranferring data from SQL Server in incremental way
I have this table which has about 30 thousand records
I want to send all the records everyday at night to the other interface(TIBCO) in form of XML.
I can send all the records at once or maybe 10 records at a time
1. If I make one XML with all the records and send it, its risky as XML would be quite big, also it would be time consuming and would generate a big load on the program(.NET program will create XML). Also, any abnormal error would kill the whole process and we would have to start again
2. Processing 10 or 20 records at a time and then sending the XML(with these 10 or 20 records) and doing this till all the records are sent. If there is any error in between in the program(.NET), the program can start from that step again. Now the problem is how I can keep track of the record number already sent to the other interface
Can someone help?
I think the answer might be something similar to the "checkbit" (and similar) transfer methods.
Basically you send some information along with the packet and check the packet against the "checkbit". I suppose you can then send the checkbit back and... check it... If it's right then the packet sent correctly, if not, re-send.
Wow, that's a confusing mess - have fun deciphering