If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > Delete ALL! duplicate rows from table.

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 06-13-07, 07:15
grofaty grofaty is offline
Registered User
 
Join Date: Jan 2003
Posts: 1,570
Delete ALL! duplicate rows from table.

Hi,
I would like to delete all duplicate rows from table.

Sample:
CREATE TABLE DB2ADMIN.TABLE1 (COL1 INT)
INSERT INTO DB2ADMIN.TABLE1 VALUES (1),(1),(1),(2),(2),(3)
SELECT * FROM DB2ADMIN.TABLE1
Code:
COL1
-----------
          1
          1
          1
          2
          2
          3
After deleting the result should be without duplicates, so:
Code:
COL1
-----------
          1
          2
          3
I know I can write an SELECT that "marks" all rows that are duplicate and should be deleted - mycount column is greater than 1. (Marked bold in following table)

SELECT COL1, ROW_NUMBER() OVER (PARTITION BY COL1 ORDER BY COL1) AS MYCOUNT FROM DB2ADMIN.TABLE1
Code:
COL1        MYCOUNT
----------- --------------------
          1                    1
          1                    2
          1                    3
          2                    1
          2                    2
          3                    1
But how to write a delete statement? Any idea?

My system: db2 v8.2 fp 9 on Windows XP SP2
Thanks,
Grofaty

Last edited by grofaty; 06-13-07 at 07:21.
Reply With Quote
  #2 (permalink)  
Old 06-13-07, 07:47
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,524
create table db2admin.table2 (col1 int);

insert into db2admin.table2
select distinct col1 from db2admin.table1;

drop table db2admin.table1;

rename table db2admin.table2 to db2admin.table1;

__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #3 (permalink)  
Old 06-13-07, 18:00
Peter.Vanroose Peter.Vanroose is offline
Registered User
 
Join Date: Sep 2004
Location: Belgium
Posts: 1,079
use a stored procedure

It's clear that some sequential logic is needed in this case, since because two identical rows cannot be distinguished, either both or none of them will be deleted.
row_number() is essentially "sequential logic", but I don't immediately see a way to use it in a delete statement.

An other option (using sequential logic) is to write a stored procedure:
- Open a cursor: DECLARE c CURSOR FOR SELECT col1 FROM table1 FOR UPDATE
- set prevv1 to 0
- iteratively FETCH c INTO :v1
- in the iteration, compare v1 with prevv1, and DELETE WHERE CURRENT OF c only when v1 = prevv1
- in the iteration, copy v1 to prevv1.
__________________
--_Peter Vanroose,
__IBM Certified Database Administrator, DB2 9 for z/OS
__IBM Certified Application Developer
__ABIS Training and Consulting
__http://www.abis.be/
Reply With Quote
  #4 (permalink)  
Old 06-14-07, 07:34
umayer umayer is offline
Registered User
 
Join Date: Dec 2005
Posts: 273
Quote:
Originally Posted by Peter.Vanroose
- Open a cursor: DECLARE c CURSOR FOR SELECT col1 FROM table1 FOR UPDATE
- set prevv1 to 0
- iteratively FETCH c INTO :v1
- in the iteration, compare v1 with prevv1, and DELETE WHERE CURRENT OF c only when v1 = prevv1
- in the iteration, copy v1 to prevv1.

I don't believe this will work. The order of the rows is random so it is mere chance that two identical rows are fetched in succession
Reply With Quote
  #5 (permalink)  
Old 06-14-07, 07:44
umayer umayer is offline
Registered User
 
Join Date: Dec 2005
Posts: 273
Is it possible to EXPORT the table with SELECT DISTINCT ...
and then reIMPORT the data into the table ?

or EXPORT , DELETE all rows, CREATE an unique index and then IMPORT the data using the INSERT_UPDATE option ?
Reply With Quote
  #6 (permalink)  
Old 06-14-07, 07:57
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,524
or SELECT DISTINCT, DROP, RENAME as outlined in post #2?
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #7 (permalink)  
Old 06-14-07, 11:30
Peter.Vanroose Peter.Vanroose is offline
Registered User
 
Join Date: Sep 2004
Location: Belgium
Posts: 1,079
Quote:
Originally Posted by umayer
I don't believe this will work. The order of the rows is random so it is mere chance that two identical rows are fetched in succession
Of course, you're right. Sorry.
Just add "ORDER BY col1" to the cursor declaration.
(Note that in this case an index MUST be defined in col1, since only and index-based access will allow the FOR UPDATE in combination with ORDER BY.)
__________________
--_Peter Vanroose,
__IBM Certified Database Administrator, DB2 9 for z/OS
__IBM Certified Application Developer
__ABIS Training and Consulting
__http://www.abis.be/

Last edited by Peter.Vanroose; 06-14-07 at 11:35.
Reply With Quote
  #8 (permalink)  
Old 06-14-07, 11:53
ankur02018 ankur02018 is offline
Registered User
 
Join Date: Jun 2007
Posts: 189
there no concept of ROWID as in oracle you cannot differentiate similar
rows

is there any way to delete from using sql
Reply With Quote
  #9 (permalink)  
Old 06-14-07, 12:05
r937 r937 is offline
SQL Consultant
 
Join Date: Apr 2002
Location: Toronto, Canada
Posts: 19,524
post #2 is sql...
__________________
r937.com | rudy.ca
please visit Simply SQL and buy my book
Reply With Quote
  #10 (permalink)  
Old 06-14-07, 12:26
Peter.Vanroose Peter.Vanroose is offline
Registered User
 
Join Date: Sep 2004
Location: Belgium
Posts: 1,079
Quote:
Originally Posted by ankur02018
is there any way to delete from using sql
Yes: you can use REORG with a DISCARD WHEN condition. (At least, on z/OS; don't know about LUW.)
__________________
--_Peter Vanroose,
__IBM Certified Database Administrator, DB2 9 for z/OS
__IBM Certified Application Developer
__ABIS Training and Consulting
__http://www.abis.be/

Last edited by Peter.Vanroose; 06-14-07 at 12:32.
Reply With Quote
  #11 (permalink)  
Old 06-14-07, 12:40
Peter.Vanroose Peter.Vanroose is offline
Registered User
 
Join Date: Sep 2004
Location: Belgium
Posts: 1,079
Quote:
Originally Posted by umayer
Is it possible to EXPORT the table with SELECT DISTINCT ...
and then reIMPORT the data into the table ?
Yes:
Code:
db2 export to temp.ixf of ixf select distinct * from table1
db2 import from temp.ixf of ixf replace into table1
__________________
--_Peter Vanroose,
__IBM Certified Database Administrator, DB2 9 for z/OS
__IBM Certified Application Developer
__ABIS Training and Consulting
__http://www.abis.be/
Reply With Quote
  #12 (permalink)  
Old 06-14-07, 23:43
ankur02018 ankur02018 is offline
Registered User
 
Join Date: Jun 2007
Posts: 189
Quote:
Originally Posted by Peter.Vanroose
Yes: you can use REORG with a DISCARD WHEN condition. (At least, on z/OS; don't know about LUW.)
can you please give full syntax
Reply With Quote
  #13 (permalink)  
Old 06-15-07, 02:22
Peter.Vanroose Peter.Vanroose is offline
Registered User
 
Join Date: Sep 2004
Location: Belgium
Posts: 1,079
Quote:
Originally Posted by ankur02018
can you please give full syntax
Example: remove all rows from mytable for which col1 > 7:
Code:
REORG TABLESPACE dbname.tsname DISCARD FROM TABLE mytable WHEN (col1 > 7)
(See page 487 of the v8 Utility Guide for an other example.)
__________________
--_Peter Vanroose,
__IBM Certified Database Administrator, DB2 9 for z/OS
__IBM Certified Application Developer
__ABIS Training and Consulting
__http://www.abis.be/
Reply With Quote
  #14 (permalink)  
Old 06-15-07, 10:25
ankur02018 ankur02018 is offline
Registered User
 
Join Date: Jun 2007
Posts: 189
Quote:
Originally Posted by r937
create table db2admin.table2 (col1 int);

insert into db2admin.table2
select distinct col1 from db2admin.table1;

drop table db2admin.table1;

rename table db2admin.table2 to db2admin.table1;


gr8 work done

but only last statement was not working
it work as below:--

rename table table2 to table1:
Reply With Quote
  #15 (permalink)  
Old 06-15-07, 10:35
ankur02018 ankur02018 is offline
Registered User
 
Join Date: Jun 2007
Posts: 189
Quote:
Originally Posted by Peter.Vanroose
Example: remove all rows from mytable for which col1 > 7:
Code:
REORG TABLESPACE dbname.tsname DISCARD FROM TABLE mytable WHEN (col1 > 7)
(See page 487 of the v8 Utility Guide for an other example.)
its not working please check
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On