If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > Best way to offload data from a huge table?

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 01-16-12, 12:48
db2user24 db2user24 is offline
Registered User
 
Join Date: Nov 2007
Posts: 248
Best way to offload data from a huge table?

Hi,

We have a READINGS table that currently has around 828,027,973 records. The table has these fields --

ID INTEGER
RDATE TIMESTAMP
RVALUE DOUBLE


Is there an 'ideal' way to export data for a certain ID and then delete the records from the READINGS table? As an example, one of the ID's has 935122 records and we would like to put the data for this ID in a file and then remove it from the table itself. Any tips would be appreciated..thanks!!
Reply With Quote
  #2 (permalink)  
Old 01-16-12, 13:41
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by db2user24 View Post
Is there an 'ideal' way
What are your criteria of "ideal"?
Reply With Quote
  #3 (permalink)  
Old 01-16-12, 14:36
db2user24 db2user24 is offline
Registered User
 
Join Date: Nov 2007
Posts: 248
Ideal meaning what is the best way to accomplish this.. basically the fastest method that is easy if there is such a way..thanks!
Reply With Quote
  #4 (permalink)  
Old 01-16-12, 14:49
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
The fastest would probably be Optim High Performance Unload for extraction, then a partition detach if your data are partitioned by ID.
Reply With Quote
  #5 (permalink)  
Old 01-16-12, 15:15
db2user24 db2user24 is offline
Registered User
 
Join Date: Nov 2007
Posts: 248
Thanks, will Optim High Performance Unload put a copy of the data in a file or actually move the data to a file? If I were to use the export command to put the data in a file, what would be the fastest way to delete records without filling up the logs, etc.
Reply With Quote
  #6 (permalink)  
Old 01-16-12, 15:25
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by db2user24 View Post
what would be the fastest way to delete records without filling up the logs, etc.
That would certainly depend on how much log space you have, whether the table was created with the NOT LOGGED option, and your "etc" as well.

Why do you think you need the "fastest" way?
Reply With Quote
  #7 (permalink)  
Old 01-16-12, 16:13
db2user24 db2user24 is offline
Registered User
 
Join Date: Nov 2007
Posts: 248
I don't necessarily need the 'fastest' way but just don't want it to take forever, that's all I don't see the NOT LOGGED option in the schema for the table..here is the schema :

CREATE TABLE READINGS (
"ID" INTEGER NOT NULL ,
"RDATE" TIMESTAMP NOT NULL ,
"RVAL" DOUBLE NOT NULL )
DATA CAPTURE CHANGES
PARTITIONING KEY ("ID") USING HASHING
IN "R_TBLSP" ;

-- DDL Statements for primary key on Table "READINGS"

ALTER TABLE READINGS
ADD PRIMARY KEY
("ID", "RDATE");


ALTER TABLE READINGS
ADD CONSTRAINT "SQL0502120241xxxxx" FOREIGN KEY
("ID")
REFERENCES TABLE2
("ID")
ON DELETE NO ACTION
ON UPDATE NO ACTION
ENFORCED
ENABLE QUERY OPTIMIZATION;


here are the config paramaters of the db that gives some information on the primary / secondary log files --

Catalog cache size (4KB) (CATALOGCACHE_SZ) = 4096
Log buffer size (4KB) (LOGBUFSZ) = 256
Log file size (4KB) (LOGFILSIZ) = 8000
Number of primary log files (LOGPRIMARY) = 10
Number of secondary log files (LOGSECOND) = 5
Reply With Quote
  #8 (permalink)  
Old 01-16-12, 17:01
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by n_i
Like I said, just detach the partition with the ID that you want to lose, then drop the detached table.
That was wrong, disregard.

Last edited by n_i; 01-17-12 at 21:08.
Reply With Quote
  #9 (permalink)  
Old 01-17-12, 09:46
stolze stolze is offline
Registered User
 
Join Date: Jan 2007
Location: Jena, Germany
Posts: 2,662
If you want to avoid 2 table scans or so, you could use "SELECT ... FROM OLD TABLE ( DELETE ... )". But that will do the regular logging for DML operations.
__________________
Knut Stolze
IBM DB2 Analytics Accelerator
IBM Germany Research & Development
Reply With Quote
  #10 (permalink)  
Old 01-17-12, 20:57
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by db2user24 View Post
I don't see the NOT LOGGED option in the schema for the table
You can alter the table to active NOT LOGGED INITIALLY and execute DELETE in the same UOW. But be careful with NLI, you can loose access to the table if you encounter some error during this operation.
Reply With Quote
  #11 (permalink)  
Old 01-17-12, 20:59
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by n_i View Post
Like I said, just detach the partition with the ID that you want to lose, then drop the detached table.
How do you detach a partition from this table?
Reply With Quote
  #12 (permalink)  
Old 01-17-12, 21:05
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by db2girl View Post
How do you detach a partition from this table?
Uhm.. I don't know why I said that... I must have been thinking of something else.
Reply With Quote
  #13 (permalink)  
Old 01-17-12, 21:08
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by n_i View Post
I must have been thinking of something else.
Beer, I guess
Reply With Quote
  #14 (permalink)  
Old 01-17-12, 21:12
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by db2girl View Post
Beer, I guess
I am predictable, aren't I?
Reply With Quote
  #15 (permalink)  
Old 01-17-12, 21:17
db2girl db2girl is offline
∞∞∞∞∞∞
 
Join Date: Aug 2008
Location: Toronto, Canada
Posts: 1,816
Quote:
Originally Posted by n_i View Post
I am predictable, aren't I?
By now, I know what you like - beer and dogs
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On