If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > Removing 50 percent of data

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 09-18-07, 10:09
dba_udb dba_udb is offline
Registered User
 
Join Date: Mar 2005
Posts: 73
Removing 50 percent of data

I have a question about Removing 50 percent od database from a big tables which has got more than one million rows.

In how many ways i can do this .
Reply With Quote
  #2 (permalink)  
Old 09-18-07, 12:24
loquin loquin is offline
Super Moderator
 
Join Date: Jun 2004
Location: Arizona, USA
Posts: 1,797
The general concept would be to use a DELETE SQL statement, with a where condition that excludes the 50% that you need to keep.

A common approach would be to use a timestamp field to decied what stayes or goes.

Code:
Delete from YourTable Where TimeStampField < '1-JAN-2004'
might be just the ticket, for instance.

Now, you should possibly keep the old data in an archival table; if so, first copy the records from the active table to the archive table.

You don't say why you feel you need to remove some of the data, but if it's because of data access delays, you should be able to index the table so that this issue should largely go away. I'm not familiar with DB2, it should allow you to add views to limit the amount of data returned. (you could define a view of your table with a where clause to filter out the old data, for instance)

If you have a front-end client that access the data, but is slow in retrieving the table, you should take a close look at the design - in particular, make sure that you're not pulling a copy of the entire table to the client if the entire table contents aren't needed - instead, allow the database server to filter the results first.
__________________
Lou
使大吃一惊
"Lisa, in this house, we obey the laws of thermodynamics!" - Homer Simpson
"I have my standards. They may be low, but I have them!" - Bette Middler
"It's a book about a Spanish guy named Manual. You should read it." - Dilbert

Reply With Quote
  #3 (permalink)  
Old 09-19-07, 08:41
grofaty grofaty is offline
Registered User
 
Join Date: Jan 2003
Posts: 1,570
Quote:
Originally Posted by dba_udb
I have a question about Removing 50 percent od database from a big tables which has got more than one million rows.

In how many ways i can do this .
Hi,
what is the problem? Performance? One million rows is not a big table... What is the number of pages and page size (default is 4 kB)?
Code:
runstats on table schema.table

select npages, card from syscat.tables where tabschema='schema' and tabname='table'
What is your db edition and version? OS name and version?
Regards,
Grofaty
Reply With Quote
  #4 (permalink)  
Old 09-19-07, 12:00
dba_udb dba_udb is offline
Registered User
 
Join Date: Mar 2005
Posts: 73
grotafy-

It was not a problem. But it was one of the questions i was asked in an interview.

The question is like this.

How could you delete 50 percent of data in a huge table?
Which has got about 10 million or so.

My answer was to export the data which you need ( if it is based on search condition) . and trunce the table data by importing zero bytes and reloading the previouly exported data.

I would like to know the many more easy options.

tx
Reply With Quote
  #5 (permalink)  
Old 09-19-07, 12:54
jsharon1248 jsharon1248 is offline
Registered User
 
Join Date: Apr 2007
Location: Chicago
Posts: 57
For z/OS DB2, you would want to run the REORG with the DISCARD option. Run it with the STATISTICS, LOG NO, and COPY options to get the biggest bang for the buck.
Reply With Quote
  #6 (permalink)  
Old 09-21-07, 03:35
grofaty grofaty is offline
Registered User
 
Join Date: Jan 2003
Posts: 1,570
Quote:
Originally Posted by dba_udb
grotafy-

It was not a problem. But it was one of the questions i was asked in an interview.

The question is like this.

How could you delete 50 percent of data in a huge table?
Which has got about 10 million or so.

My answer was to export the data which you need ( if it is based on search condition) . and trunce the table data by importing zero bytes and reloading the previouly exported data.

I would like to know the many more easy options.

tx
Hi,
what is operating system name and version? DB2 edition and version, fixpack level?

I would do the following on DB2/LUW:
1. create new table with the same definition: create table schema.table_name like new_table
2. insert rows needed into new table: insert into schema.new_table select * from schema.table where rows_you_need
3. drop original table: drop table schema.table_name
4. rename new table to old table name: rename table schema.new_table to table

Hope this helps,
Grofaty

Last edited by grofaty; 09-21-07 at 03:39.
Reply With Quote
  #7 (permalink)  
Old 09-21-07, 07:12
grofaty grofaty is offline
Registered User
 
Join Date: Jan 2003
Posts: 1,570
Hi,
there is even faster way:
I would do the following on DB2/LUW:
1. create new table with the same definition: create table schema.table_name like new_table
2. declare cursor for records you would like to keep:
declare c1 cursor for select * from schema.table where rows_you_would_like_to_keep
3. load from cursor: load from c1 of cursor insert into schema.new_table
4. drop original table: drop table schema.table_name
5. rename new table to old table name: rename table schema.new_table to table

Hope this helps,
Grofaty
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On