If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > DB2 > export of large table

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 06-04-04, 17:01
jfkuser jfkuser is offline
Registered User
 
Join Date: Mar 2003
Posts: 59
export of large table

Hi all,
version 7.2 on aix 5.

We are trying to export a table with 115 million rows. As soo as it reaches 64GB it abends giving folowing error message
"SQL3002C An I/O error occurred while writing to the output data file."

Is 64 GB limit of DB2 UDB or AIX file? If yes, How do we export the complete data?

Thanks in advance
j
Reply With Quote
  #2 (permalink)  
Old 06-04-04, 18:29
Marcus_A Marcus_A is online now
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,196
It is probably an operationg system limit, although I believe it can usually be overriden with the proper operating system parameter.

64GB is the maximum size of a table with 4K page size, but that is probably not directly related to your problem.

Check with your AIX system administrator. If that is you, hit the manuals.
__________________
M. A. Feldman
IBM Certified DBA on DB2 for Linux, UNIX, and Windows
IBM Certified DBA on DB2 for z/OS and OS/390
Reply With Quote
  #3 (permalink)  
Old 06-04-04, 18:34
sathyaram_s sathyaram_s is offline
Super Moderator
 
Join Date: Aug 2001
Location: UK
Posts: 4,534
You are right, jfs has a 64Gig upper limit ..

To confirm you filesystem is jfs, do

lsfs <filesystem name>

A jfs2 filesystem (which is available from AIX v5 should help )

A note from IBM Website :

The maximum size of a regular file in a JFS file system enabled for large files is slightly less than 64 gigabytes (68589453312). In other file systems that are enabled for large files and in other JFS file system types, all files not listed as regular in the previous table have a maximum file size of 2 gigabytes minus 1 (2147483647). The maximum size of a file in JFS2 is limited by the size of the file system itself.

The architectural limit on the size of a JFS2 file system is 2^52 bytes, or 4 petabytes. In AIX 5.2, the maximum JFS2 file size supported by the 32-bit kernel is 2^40 - 4096 bytes, or just under 1 terabyte. The maximum file size supported by the 64-bit kernel is 2^44 - 4096 bytes, or just less than 16 terabytes.
__________________
Visit the new-look IDUG Website , register to gain access to the excellent content.
Reply With Quote
  #4 (permalink)  
Old 06-04-04, 19:10
toby25 toby25 is offline
Registered User
 
Join Date: Sep 2003
Posts: 63
Yes, This is a JFS file system. That means it has ;imit of 64 GB.
Now the question is how do I export the data out of this table?
I need to load the data into other database server.

Thanks,
j
Reply With Quote
  #5 (permalink)  
Old 06-04-04, 19:13
Marcus_A Marcus_A is online now
Registered User
 
Join Date: May 2003
Location: USA
Posts: 5,196
You need a mount point with a file system that supports more than 64GB files. This is an AIX issue.
__________________
M. A. Feldman
IBM Certified DBA on DB2 for Linux, UNIX, and Windows
IBM Certified DBA on DB2 for z/OS and OS/390
Reply With Quote
  #6 (permalink)  
Old 06-04-04, 19:28
sathyaram_s sathyaram_s is offline
Super Moderator
 
Join Date: Aug 2001
Location: UK
Posts: 4,534
Define a jfs2 filesystem ...
__________________
Visit the new-look IDUG Website , register to gain access to the excellent content.
Reply With Quote
  #7 (permalink)  
Old 06-07-04, 14:03
J Petruk J Petruk is offline
Registered User
 
Join Date: Mar 2004
Location: Toronto, ON, Canada
Posts: 513
Quote:
Originally Posted by toby25
Yes, This is a JFS file system. That means it has ;imit of 64 GB.
Now the question is how do I export the data out of this table?
I need to load the data into other database server.

Thanks,
j
Can you find something to toss into the WHERE clause of the SELECT to split it into two (or more) EXPORTs?
__________________
--
Jonathan Petruk
DB2 Database Consultant
Reply With Quote
  #8 (permalink)  
Old 06-08-04, 10:20
jthakrar jthakrar is offline
Registered User
 
Join Date: Mar 2004
Posts: 46
There's one more option -

Essentially what you need to do is create a "pipe file" (actually a pipe or fifo file), export to that "pipe file" and then compress the data being pumped into that file.

Here's a sample -

mkfifo pipefile <--- creat a pipe file

db2 'export to pipefile of ixf select * from tablename' & <--- export and put in background !

gzip -f < pipefile > abc.gz <--- compress the file

If you wish, you can send the export output to a message file.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On