Results 1 to 9 of 9
  1. #1
    Join Date
    Sep 2002
    Posts
    30

    Question Unanswered: Concatenating Files

    HI

    How can we concatenate files vertically ?
    eg:
    file a
    aaaaaaaaaaaa
    aaaaaaaaaaaa

    file b
    bbbbbbbbbbbb
    bbbbbbbbbbbb
    bbbbbbbbbbbb

    file c
    cccccccccccccc
    cccccccccccccc
    cccccccccccccc

    file d should look like
    aaaaaaaaaaaabbbbbbbbbbbbbcccccccccccccc
    aaaaaaaaaaaabbbbbbbbbbbbbcccccccccccccc
    aaaaaaaaaaaabbbbbbbbbbbbbcccccccccccccc

    Can somebody help me in this aspect ?

    Thanks
    Bollin99

  2. #2
    Join Date
    Jun 2003
    Location
    West Palm Beach, FL
    Posts
    2,713

    Lightbulb

    Try this:


    paste -d'\0' file_a file_b file_c




    note: files have to have the same number of records.
    The person who says it can't be done should not interrupt the person doing it. -- Chinese proverb

  3. #3
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525
    Here's a little trick I learnt using the file descriptors. It will allow you to format your output as you want and so, you can overcome instances where your files have different numbers of records.

    # first define the file descriptors for your input files...
    exec 3< file1
    exec 4< file2
    exec 5< file3

    #Now read in from all 3 file descriptors simultaneously...
    while read file1Line <&3 ; read file2Line <&4; read file3Line <&5
    do
    # format the output as required
    # (I'm assuming that each input line has a fixed width of 15 chars)
    printf "%-15s%-15s%-15s\n" "$file1Line" "$file2Line" "$file3Line"
    done > newFile # redirect your output to a new file

    The "" around the variables read in are important in order that at least a blank string is formatted!

    Damian
    Last edited by Damian Ibbotson; 09-24-03 at 05:40.

  4. #4
    Join Date
    Sep 2002
    Posts
    30

    Concatenation Works great , But ..?

    HI
    Firstly thank you for your replies. Bot work excellently.
    but there is a constraint about reading the files.

    I am not sure of the number of files that have to be concatenated.
    The filenames will be read from a file and then will be concatenated.

    so obviously we cannot give all the filenames if there are 10 or 20 file to be concatenated. Definig the file descriptors will be a problem.

    So is there a mechanism to put the file names and the exec commands in a loop/sequence and execute the next set of command ?

    eg: there are files 1-10 then
    read the file names 1-10 from a file then
    define the exec for 1-10 and then concatenate ?

    Please let me know about the structre of the program.

    Thank you
    Bollin99

  5. #5
    Join Date
    Jun 2003
    Location
    West Palm Beach, FL
    Posts
    2,713

    Lightbulb

    Try this:

    Code:
    
    #!/bin/ksh
    #
    echo </dev/null >concat_file
    while read f
    do
      cat $f| paste -d'\0' concat_file - >temp1
      cp temp1 concat_file
    done <MyFiles_File
    
    The person who says it can't be done should not interrupt the person doing it. -- Chinese proverb

  6. #6
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525

    Re: Concatenation Works great , But ..?

    Originally posted by bollin99
    So is there a mechanism to put the file names and the exec commands in a loop/sequence and execute the next set of command ?

    eg: there are files 1-10 then
    read the file names 1-10 from a file then
    define the exec for 1-10 and then concatenate ?
    Hi 'Bollin99'.

    Obviously you didn't spot the deliberate mistake in my previous post. The method I gave you would require the file with the most rows to be assigned to the last file descriptor read in the 'while' logic test (because only the result of the final test is used).

    I've attached a script that gets round this and which takes between 1 and 7 files as parameters.

    If you need to work with greater numbers of files, you would just call the script within a loop. Something like...
    ------------------------------
    touch tempFile
    while read fileName
    do
    fpaste tempFile ${fileName} > newTempFile &&
    mv newTempFile tempFile
    done < fileListFile
    ------------------------------

    This kind of defeats the object of dynamically allocating the file descriptors in the 'fpaste' script but I couldn't think of an easy way to get round the limitation of 9 file descriptors (of which 2 are used for stderr and stdout).

    Damian
    Attached Files Attached Files

  7. #7
    Join Date
    Sep 2002
    Posts
    30

    Thumbs up Thank you

    HI
    Thank you, to both of you for sparing your time to find the best solution for my situation.
    I have tested LKbrwn_dba's script, and it has been functioning well.
    But i am skeptical whether it would work to concatenate 7-10 file with higher voulmes of date ......say a million rows in each file.
    Other wise the script works well with no problem

    THANK YOU
    Bollin99

  8. #8
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525

    Re: Thank you

    Originally posted by bollin99

    But i am skeptical whether it would work to concatenate 7-10 file with higher voulmes of date ......say a million rows in each file.
    I don't see a problem. The only limitation I can envisage is the length of the input and concatanated rows. I could be proved wrong - it wouldn't be the first time!

  9. #9
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525
    There is a minor bug in the fpaste script.

    Line 56 wants to be:

    fileWidth=$( awk 'NR==1{print length($0)"."length($0); exit}' < ${fileName} )

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •