If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > about loading large data into DB2 table

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 11-11-08, 12:53
cetzhbo cetzhbo is offline
Registered User
 
Join Date: May 2008
Posts: 7
about loading large data into DB2 table

Hello Gurus,

I want to load large data( about 500,000 row) into DB2 table. using following two methods:

(1) alter table activate not logged initially with empty table

turn off all the related index

insert into <table> select ... from ...

but running very slow.

(2) export to <flat file>
import from <flat file > insert into <table>

but exporting operation is very slow.

can you tell me how to improve inserting or exporting ?

Many thanks,
Frank Zhang
Reply With Quote
  #2 (permalink)  
Old 11-11-08, 12:54
ARWinner ARWinner is offline
Registered User
 
Join Date: Jan 2003
Posts: 3,575
Try the LOAD utility.

Andy
Reply With Quote
  #3 (permalink)  
Old 11-11-08, 12:59
cetzhbo cetzhbo is offline
Registered User
 
Join Date: May 2008
Posts: 7
Quote:
Originally Posted by ARWinner
Try the LOAD utility.

Andy
but my following export statement is very slow, how to make it fast:


export to /homecomm/carbatch/frank/backup/AM_REWARD_PRG_BONUS_MONTHLY_F.dat of del
with temp1 (AM_REWARD_PRG_NUMBER) as

(select distinct AM_REWARD_PRG_NUMBER from AMCP.AC_D Q1 inner join AMCP.AM_BONUS_ACCOUNT_MONTHLY_F Q2 on Q1.ACCOUNT_DIM_KEY = Q2.ACCOUNT_DIM_KEY
where Q2.AM_BONUS_ELIGIBILITY_FLAG = 'Y' and TIME_DIM_KEY = 155) ,

temp2 ( AM_REWARD_PRG_NUMBER, AM_BONUS_CATEGORY_ID) as

(select AM_REWARD_PRG_NUMBER, 'BUS' from temp1
UNION
select AM_REWARD_PRG_NUMBER, 'LOC' from temp1
UNION
select AM_REWARD_PRG_NUMBER, 'MTG' from temp1
UNION
select AM_REWARD_PRG_NUMBER, 'SAV' from temp1
except all
select AM_REWARD_PRG_NUMBER,AM_BONUS_CATEGORY_ID from AMCP.AC_D Q1 inner join AMCP.AM_BONUS_ACCOUNT_MONTHLY_F Q2 on Q1.ACCOUNT_DIM_KEY = Q2.ACCOUNT_DIM_KEY
where Q2.AM_BONUS_ELIGIBILITY_FLAG = 'Y' and TIME_DIM_KEY = 155 group by AM_REWARD_PRG_NUMBER,AM_BONUS_CATEGORY_ID,AM_BONUS _ELIGIBILITY_FLAG),

temp3 (AM_REWARD_PRG_NUMBER, AM_BONUS_CATEGORY_ID,AM_BONUS_ELIGIBILITY_FLAG ) as

(select AM_REWARD_PRG_NUMBER, AM_BONUS_CATEGORY_ID,'N' from temp2),

temp4 (AM_REWARD_PRG_NUMBER, TIME_DIM_KEY, AM_BONUS_CATEGORY_ID,AM_BONUS_ELIGIBILITY_FLAG ) as

(select AM_REWARD_PRG_NUMBER, 155, AM_BONUS_CATEGORY_ID, AM_BONUS_ELIGIBILITY_FLAG from AMCP.AC_D Q1 inner join AMCP.AM_BONUS_ACCOUNT_MONTHLY_F Q2 on Q1.ACCOUNT_DIM_KEY =
Q2.ACCOUNT_DIM_KEY where Q2.AM_BONUS_ELIGIBILITY_FLAG = 'Y' and TIME_DIM_KEY = 155 group by AM_REWARD_PRG_NUMBER,AM_BONUS_CATEGORY_ID,AM_BONUS _ELIGIBILITY_FLAG
union
select AM_REWARD_PRG_NUMBER,155, AM_BONUS_CATEGORY_ID,AM_BONUS_ELIGIBILITY_FLAG from temp3)

select * from temp4
Reply With Quote
  #4 (permalink)  
Old 11-11-08, 13:01
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
Quote:
Originally Posted by cetzhbo

but exporting operation is very slow.
That usually boils down to the optimization of the query used to select data (unless you do the export over the network, in which case the network performance also matters).
Reply With Quote
  #5 (permalink)  
Old 11-11-08, 13:05
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,449
You will have to explain what you are trying to achieve with your query, otherwise it does not make a lot of sense.
Reply With Quote
  #6 (permalink)  
Old 11-11-08, 13:06
cetzhbo cetzhbo is offline
Registered User
 
Join Date: May 2008
Posts: 7
Quote:
Originally Posted by n_i
That usually boils down to the optimization of the query used to select data (unless you do the export over the network, in which case the network performance also matters).
you are right, but my select statement in the export took less than half a minute to get the result.

so I don't think this query slowed down the exporting.
Reply With Quote
  #7 (permalink)  
Old 11-11-08, 13:09
cetzhbo cetzhbo is offline
Registered User
 
Join Date: May 2008
Posts: 7
Quote:
Originally Posted by n_i
You will have to explain what you are trying to achieve with your query, otherwise it does not make a lot of sense.

Hi friend, this is a ETL processing in real data warehouse.

I just want to extract data from a few data source, and then transform and loading into the target DB table , those operations are based on business requirement ( or mapping rule).

please help me to resolve this loading issue.

thanks very much!
Reply With Quote
  #8 (permalink)  
Old 11-11-08, 13:56
ARWinner ARWinner is offline
Registered User
 
Join Date: Jan 2003
Posts: 3,575
If the source and target tables are in the same DB, you can load from cursor. If not, then maybe you can set up federation and still load from cursor.

Andy
Reply With Quote
  #9 (permalink)  
Old 11-11-08, 14:40
cetzhbo cetzhbo is offline
Registered User
 
Join Date: May 2008
Posts: 7
thanks guys,

I have solved this problem.

Many thanks
Frank
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On