If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > Performance problem on mass delete

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 12-21-11, 03:26
olel olel is offline
Registered User
 
Join Date: Oct 2009
Posts: 23
Performance problem on mass delete

Hi,

we have a table with about 120 millions of records that grows at 2.5 millions of records per week.

For not letting this table explode we would like to delete old records once a week. Therefore we have an SQL like

Code:
delete from table where creationDate < someDate
As this delete affects about 2.5 million rows it is really slow which seems to be up to the transaction log that gets really big.

Which is the best, so the most performant, way for scuch a mass delete operation?
Reply With Quote
  #2 (permalink)  
Old 12-21-11, 03:37
tonkuma tonkuma is online now
Registered User
 
Join Date: Feb 2008
Location: Japan
Posts: 2,193
First came in my mind was to create an index on creationDate column.
Reply With Quote
  #3 (permalink)  
Old 12-21-11, 03:51
amitrai4 amitrai4 is offline
Registered User
 
Join Date: Aug 2011
Posts: 46
Have you given a thought to range partition this table ?
Reply With Quote
  #4 (permalink)  
Old 12-21-11, 03:56
olel olel is offline
Registered User
 
Join Date: Oct 2009
Posts: 23
Quote:
Originally Posted by tonkuma View Post
First came in my mind was to create an index on creationDate column.
The creationDate column is indexed of course. What slows the statement down seems to be the writing to the transaction log which is independent of the time for selecting the rows.
Reply With Quote
  #5 (permalink)  
Old 12-21-11, 03:58
olel olel is offline
Registered User
 
Join Date: Oct 2009
Posts: 23
Quote:
Originally Posted by amitrai4 View Post
Have you given a thought to range partition this table ?
As the application on this database is a Hibernate one (which should be independent of the db system used) partitioning is not an option as it is not supported by all db systems.
Reply With Quote
  #6 (permalink)  
Old 12-21-11, 04:07
tonkuma tonkuma is online now
Registered User
 
Join Date: Feb 2008
Location: Japan
Posts: 2,193
How about UNION ALL view?
Reply With Quote
  #7 (permalink)  
Old 12-21-11, 04:16
olel olel is offline
Registered User
 
Join Date: Oct 2009
Posts: 23
Quote:
Originally Posted by tonkuma View Post
How about UNION ALL view?
Can you please give me some more information on how a UNION ALL view works and might help. Unfortunately I'm not really a DB expert.
Reply With Quote
  #8 (permalink)  
Old 12-21-11, 04:29
amitrai4 amitrai4 is offline
Registered User
 
Join Date: Aug 2011
Posts: 46
Quote:
Originally Posted by tonkuma View Post
How about UNION ALL view?
Or MDC with DB2_MDC_ROLLOUT registry variable/ CURRENT MDC ROLLOUT MODE special register
Reply With Quote
  #9 (permalink)  
Old 12-21-11, 04:41
fengsun2 fengsun2 is offline
Registered User
 
Join Date: Nov 2011
Posts: 124
I think using a table with data patition instead of your table will be a better solution.
You can use detach command to rollout data quickly .
Reply With Quote
  #10 (permalink)  
Old 12-21-11, 04:42
przytula_guy przytula_guy is offline
Registered User
 
Join Date: Apr 2006
Location: Belgium
Posts: 1,159
all doc/info is at
IBM DB2 9.5 Information Center for Linux, UNIX, and Windows
I really do not understand that company's have their db handled by people that do not know anything about it.. and just executing what people from the forum suggest..
__________________
Best Regards, Guy Przytula
Database Software Consultant
DB2 UDB LUW Certified V7-V8-V9-V9.7 DB Admin - Dprop..
Information Server Datastage Certified
http://www.infocura.be
Reply With Quote
  #11 (permalink)  
Old 12-21-11, 05:21
wolaos123 wolaos123 is offline
Registered User
 
Join Date: Nov 2011
Location: Shen Zhen,China
Posts: 37
temporary alter the table with ACTIVATE NOT LOGGED INITIALLY attribute
Reply With Quote
  #12 (permalink)  
Old 12-21-11, 06:29
dr_te_z dr_te_z is offline
Registered User
 
Join Date: Jan 2009
Location: Zoetermeer, Holland
Posts: 555
Quote:
Originally Posted by amitrai4 View Post
Have you given a thought to range partition this table ?
Yes agree. On 9.7 the DETACH is much more smooth. The local indexes take away your REORG worries.
Reply With Quote
  #13 (permalink)  
Old 12-22-11, 11:18
stolze stolze is offline
Registered User
 
Join Date: Jan 2007
Location: Jena, Germany
Posts: 2,662
You want to have a general solution that works with lot's of different database systems (because you use Hibernate)? Then you have to stick to the common functionality, which is DELETE and which logs all changes. This will be slower than the here proposed solutions to do some sort of partitioning or deactivating logging. All those are approaches specific to DB2. So you have to decide what you want.
__________________
Knut Stolze
IBM DB2 Analytics Accelerator
IBM Germany Research & Development
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On