Results 1 to 2 of 2
  1. #1
    Join Date
    Feb 2004
    Posts
    143

    Unanswered: multi-thread a code

    I am on a Sun Solaris 5.8
    A piece of my load.ksh script has the following lines :

    cat /data/dev/CODE_LIST1.TXT | while read column1 column2
    do
    genscript.ksh ${dbname} ${schemaname} member $column1 /data/dev/member_scripts/$column2
    done

    The CODE_LIST1.TXT looks something like

    38222 38222_123456789.sql
    48338 48338_438434845.sql


    I am reading a CODE_LIST1.TXT file and passing the contents as variables into the genscript.ksh script
    The problem is that CODE_LIST1.TXT could have 100 lines out there and currently our process if there are
    100 entries in the .TXT file takes about 6 unavoidable hours just due to the processing involved.

    Can I get some pointers on how I can multi-thread this code so that if there are more than 4 entries
    in the CODE_LIST1.TXT, I want to split it up into 4 execution threads - each picking one up 1 line.
    If there are 100 lines, each of the parallel threads need to pick up 25 each. If there are 105 lines,
    then, first 3 picks up 27 each and the last one picks up 24 lines to process.
    Can this be done ?
    Thanks

  2. #2
    Join Date
    Feb 2004
    Posts
    143

    Got it

    I figured this out...

    linecount=`cat CODE_LIST1.TXT |wc -l`
    echo $linecount
    splitcount=`echo "scale=0; $linecount/4+1" |bc -l`
    echo $splitcount
    split -l $splitcount CODE_LIST1.TXT

    ## By the above, I now have 4 files - xaa,xab,xac and xad

    for f in x*
    do
    parallelproc.ksh $dbname $schemaname $f &
    done
    wait


    And in my parallelproc.ksh I have :

    #!/bin/ksh
    dbname=$1
    schemaname=$2
    filename=$3
    cat /data/dev/$filename | while read column1 column2
    do
    genscript.ksh ....(with other parameters as in original posted thread)
    done
    exit

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •