If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Data Access, Manipulation & Batch Languages > Unix Shell Scripts > Concatenating Files

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 09-23-03, 10:06
bollin99 bollin99 is offline
Registered User
 
Join Date: Sep 2002
Posts: 30
Question Concatenating Files

HI

How can we concatenate files vertically ?
eg:
file a
aaaaaaaaaaaa
aaaaaaaaaaaa

file b
bbbbbbbbbbbb
bbbbbbbbbbbb
bbbbbbbbbbbb

file c
cccccccccccccc
cccccccccccccc
cccccccccccccc

file d should look like
aaaaaaaaaaaabbbbbbbbbbbbbcccccccccccccc
aaaaaaaaaaaabbbbbbbbbbbbbcccccccccccccc
aaaaaaaaaaaabbbbbbbbbbbbbcccccccccccccc

Can somebody help me in this aspect ?

Thanks
Bollin99
Reply With Quote
  #2 (permalink)  
Old 09-23-03, 11:14
LKBrwn_DBA LKBrwn_DBA is offline
Registered User
 
Join Date: Jun 2003
Location: West Palm Beach, FL
Posts: 2,456
Lightbulb

Try this:


paste -d'\0' file_a file_b file_c




note: files have to have the same number of records.
__________________
The person who says it can't be done should not interrupt the person doing it. -- Chinese proverb
Reply With Quote
  #3 (permalink)  
Old 09-24-03, 04:36
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Here's a little trick I learnt using the file descriptors. It will allow you to format your output as you want and so, you can overcome instances where your files have different numbers of records.

# first define the file descriptors for your input files...
exec 3< file1
exec 4< file2
exec 5< file3

#Now read in from all 3 file descriptors simultaneously...
while read file1Line <&3 ; read file2Line <&4; read file3Line <&5
do
# format the output as required
# (I'm assuming that each input line has a fixed width of 15 chars)
printf "%-15s%-15s%-15s\n" "$file1Line" "$file2Line" "$file3Line"
done > newFile # redirect your output to a new file

The "" around the variables read in are important in order that at least a blank string is formatted!

Damian

Last edited by Damian Ibbotson; 09-24-03 at 04:40.
Reply With Quote
  #4 (permalink)  
Old 09-24-03, 11:09
bollin99 bollin99 is offline
Registered User
 
Join Date: Sep 2002
Posts: 30
Concatenation Works great , But ..?

HI
Firstly thank you for your replies. Bot work excellently.
but there is a constraint about reading the files.

I am not sure of the number of files that have to be concatenated.
The filenames will be read from a file and then will be concatenated.

so obviously we cannot give all the filenames if there are 10 or 20 file to be concatenated. Definig the file descriptors will be a problem.

So is there a mechanism to put the file names and the exec commands in a loop/sequence and execute the next set of command ?

eg: there are files 1-10 then
read the file names 1-10 from a file then
define the exec for 1-10 and then concatenate ?

Please let me know about the structre of the program.

Thank you
Bollin99
Reply With Quote
  #5 (permalink)  
Old 09-24-03, 18:13
LKBrwn_DBA LKBrwn_DBA is offline
Registered User
 
Join Date: Jun 2003
Location: West Palm Beach, FL
Posts: 2,456
Lightbulb

Try this:

Code:

#!/bin/ksh
#
echo </dev/null >concat_file
while read f
do
  cat $f| paste -d'\0' concat_file - >temp1
  cp temp1 concat_file
done <MyFiles_File
__________________
The person who says it can't be done should not interrupt the person doing it. -- Chinese proverb
Reply With Quote
  #6 (permalink)  
Old 09-25-03, 12:03
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Re: Concatenation Works great , But ..?

Quote:
Originally posted by bollin99
So is there a mechanism to put the file names and the exec commands in a loop/sequence and execute the next set of command ?

eg: there are files 1-10 then
read the file names 1-10 from a file then
define the exec for 1-10 and then concatenate ?
Hi 'Bollin99'.

Obviously you didn't spot the deliberate mistake in my previous post. The method I gave you would require the file with the most rows to be assigned to the last file descriptor read in the 'while' logic test (because only the result of the final test is used).

I've attached a script that gets round this and which takes between 1 and 7 files as parameters.

If you need to work with greater numbers of files, you would just call the script within a loop. Something like...
------------------------------
touch tempFile
while read fileName
do
fpaste tempFile ${fileName} > newTempFile &&
mv newTempFile tempFile
done < fileListFile
------------------------------

This kind of defeats the object of dynamically allocating the file descriptors in the 'fpaste' script but I couldn't think of an easy way to get round the limitation of 9 file descriptors (of which 2 are used for stderr and stdout).

Damian
Attached Files
File Type: txt fpaste.txt (2.0 KB, 161 views)
Reply With Quote
  #7 (permalink)  
Old 09-25-03, 22:57
bollin99 bollin99 is offline
Registered User
 
Join Date: Sep 2002
Posts: 30
Thumbs up Thank you

HI
Thank you, to both of you for sparing your time to find the best solution for my situation.
I have tested LKbrwn_dba's script, and it has been functioning well.
But i am skeptical whether it would work to concatenate 7-10 file with higher voulmes of date ......say a million rows in each file.
Other wise the script works well with no problem

THANK YOU
Bollin99
Reply With Quote
  #8 (permalink)  
Old 09-26-03, 04:15
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Re: Thank you

Quote:
Originally posted by bollin99

But i am skeptical whether it would work to concatenate 7-10 file with higher voulmes of date ......say a million rows in each file.
I don't see a problem. The only limitation I can envisage is the length of the input and concatanated rows. I could be proved wrong - it wouldn't be the first time!
Reply With Quote
  #9 (permalink)  
Old 09-26-03, 05:23
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
There is a minor bug in the fpaste script.

Line 56 wants to be:

fileWidth=$( awk 'NR==1{print length($0)"."length($0); exit}' < ${fileName} )
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On