If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Database Server Software > Oracle > Loading windows CSV file on Linux without dos2unix

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 09-09-10, 12:35
nitsam nitsam is offline
Registered User
 
Join Date: Sep 2010
Posts: 3
Loading windows CSV file on Linux without dos2unix

I have some very large data files (several 100 GB) that I want to load into a Linux version of Oracle via sqlldr. Unfortunately the files were created on a windows machine and contain the extra windows ^M linefeed. When I try to load the files with the extra newline, sqlldr has an error.

Is there any way for me to use sqlldr without having to run dos2unix on the data files first? The files are large enough that this takes several hours to complete, and I have to do this frequently enough for it to be very annoying. I unfortunately do not have control over the source data, so cannot change it to come with unix newlines.
Reply With Quote
  #2 (permalink)  
Old 09-09-10, 12:51
anacedent anacedent is offline
Registered User
 
Join Date: Aug 2003
Location: Where the Surf Meets the Turf @Del Mar, CA
Posts: 6,415
>I unfortunately do not have control over the source data, so cannot change it to come with unix newlines.
Not necessarily.

If the files are transfered from Windows to *nix using "ftp" in ASCII mode;
then the extra character will be eliminated as part of the transfer process.

Try it; you'll like it!
__________________
You can lead some folks to knowledge, but you can not make them think.
The average person thinks he's above average!
For most folks, they don't know, what they don't know.
Reply With Quote
  #3 (permalink)  
Old 09-09-10, 12:55
nitsam nitsam is offline
Registered User
 
Join Date: Sep 2010
Posts: 3
Thanks for the tip. Unfortunately my data source is coming on a external hard drive, so there's no FTP or network transfer involved at this point. The FTP overhead would actually be more than dos2unix .

Any other suggestions?
Reply With Quote
  #4 (permalink)  
Old 09-09-10, 13:15
n_i n_i is offline
:-)
 
Join Date: Jun 2003
Location: Toronto, Canada
Posts: 4,453
I think you can specify multibyte record delimiters in the control file:

Code:
 ... INFILE file.txt "STR X'0d0a'" ...
Reply With Quote
  #5 (permalink)  
Old 09-09-10, 16:11
nitsam nitsam is offline
Registered User
 
Join Date: Sep 2010
Posts: 3
Quote:
Originally Posted by n_i View Post
I think you can specify multibyte record delimiters in the control file:

Code:
 ... INFILE file.txt "STR X'0d0a'" ...
Excellent, that works wonderfully - thanks!
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On