Results 1 to 14 of 14
  1. #1
    Join Date
    May 2003
    Posts
    27

    Question Unanswered: how to translate shell variable inside a file to a new file?

    I have many files that contain multiple shell variables in ${myvar} format. I want to translate all the variables in the its corresponding value and output all content to a new file. however, in lines other than the those contain the shell variable have the double quote or single quote character, and that's why I can use the cat commands to do the translation...........

    does anyone know how to do this?

    thanks in advance!!

  2. #2
    Join Date
    Oct 2003
    Posts
    18
    Can you post an example of what it is that you want?

  3. #3
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525

    Re: how to translate shell variable inside a file to a new file?

    Originally posted by r390gt1
    I have many files that contain multiple shell variables in ${myvar} format. I want to translate all the variables in the its corresponding value and output all content to a new file. however, in lines other than the those contain the shell variable have the double quote or single quote character, and that's why I can use the cat commands to do the translation...........

    does anyone know how to do this?
    If I understand you correctly, you can't do what you are asking. How can you retrieve the associated value for a variable that has not yet been set (i.e. if the script is not actually running)?

    You can use 'set' within your script at some point to display all the variables that are being used.

    ---------------------------------
    var1=a
    var2=b
    var3=c
    set > scriptParametersFile
    ---------------------------------

    Or maybe even source your script in a subshell and use set.

    (. yourScript 2> /dev/null; set > scriptParametersFile)

    These should be fine if your scripts are pretty simple.

    HTH

  4. #4
    Join Date
    Oct 2003
    Location
    Germany
    Posts
    138
    Hello r390gt1,
    if I understand you correctly, all of your scripts use the same variables and you will change the values at a central point and not in every script.
    Values which where changed from script1 should be used from script2, such as an update from your central point values.
    Is this correct ?

    Try this:

    #### create a new file which called VAR_LIST.TXT ( for example )
    #### with following values
    # VAR1|1111
    # VAR2|2222
    # VARx|3333
    #### this file will be the central point for the variables for all your scripts


    # Use this at all your scripts which work with the central pointed variables
    # each script can change the values to be used from the next script.


    # insert the followed rows at the top of every script
    ################################################## #############################
    ################################################## #############################
    VAR1=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
    {
    if ($1 == "VAR1")
    {
    print $2
    }
    }'`
    VAR2=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
    {
    if ($1 == "VAR2")
    {
    print $2
    }
    }'`
    VARx=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
    {
    if ($1 == "VARx")
    {
    print $2
    }
    }'`
    echo $VAR1
    echo $VAR2
    echo $VARx

    # now we change the values from VAR1 - VARx
    # and update the central file with the new values for the next operation
    VAR1=`expr $VAR1 + 1`
    VAR2=`expr $VAR2 + 1`
    VARx=`expr $VARx + 1`
    echo "VAR1|"$VAR1 > VAR_LIST.TXT
    echo "VAR2|"$VAR2 >> VAR_LIST.TXT
    echo "VARx|"$VARx >> VAR_LIST.TXT

    # start the the sript again and you will get the updatet values

    max be I can help you out.

    Peter.F
    Last edited by fla5do; 11-05-03 at 18:00.
    Greetings from germany
    Peter F.

  5. #5
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525
    Originally posted by fla5do
    #### create a new file which called VAR_LIST.TXT ( for example )
    #### with following values
    # VAR1|1111
    # VAR2|2222
    # VARx|3333
    #### this file will be the central point for the variables for all your scripts

    # Use this at all your scripts which work with the central pointed variables
    # each script can change the values to be used from the next script.

    # insert the followed rows at the top of every script
    ################################################## #############################
    ################################################## #############################
    VAR1=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
    {
    if ($1 == "VAR1")
    {
    print $2
    }
    }'`
    VAR2=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
    {
    if ($1 == "VAR2")
    {
    print $2
    }
    }'`
    VARx=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
    {
    if ($1 == "VARx")
    {
    print $2
    }
    }'`
    Isn't this the same as...

    parameterListFile
    ---------------------
    VAR1=1111
    VAR2=2222
    VARx=3333

    yourScript
    ---------------------
    . parameterListFile

    ???

  6. #6
    Join Date
    Oct 2003
    Location
    Germany
    Posts
    138
    Isn't this the same as...

    parameterListFile
    ---------------------
    VAR1=1111
    VAR2=2222
    VARx=3333

    yourScript
    ---------------------
    . parameterListFile

    ???
    -----------------------------------------------------------------------------
    Hello Damian,
    I think its not the same. I describe a way for input and output,
    with a complete example and comments. And in my ParametersFile the
    variables can be named for better reading and understanding later.

    Your solution;
    ------------------
    var1=a
    var2=b
    var3=c
    set > scriptParametersFile

    !!! the command "set" makes an output from all enviromente variables in your shell, but not only "var1 - var3"


    Peter F.
    Last edited by fla5do; 11-06-03 at 15:54.
    Greetings from germany
    Peter F.

  7. #7
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525
    Hello Damian,
    I think its not the same. I describe a way for input and output,
    with a complete example and comments. And in my ParametersFile the
    variables can be named for better reading and understanding later.
    Fair enough! I'd be concerned about the maintainability though.

    And yes, 'set' will output the environment variables as well but I'm sure a work around would be simple enough.
    Last edited by Damian Ibbotson; 11-07-03 at 05:56.

  8. #8
    Join Date
    May 2003
    Posts
    27

    Talking Sorry for keeping you guys waiting, I give a example for my case..

    I exported two variable named VAR1 and VAR2 using the following commands...... all the variable setting are centralized in one shell script.
    export VAR1=myvalue1
    export VAR2=myvalue2

    and I have multiple files that contain these variable and these files are not shell script......
    eg.
    file1.txt
    -----------------------------
    'some text with single quote.......'
    ${VAR1} is bah bah
    "some anthoer text with double quote"
    ${VAR2} is bah bah


    file2.txt
    -----------------------------
    "some anthoer text with double quote"
    This is my var 2 ${VAR2}
    'some text with single quote.......'


    What I want is to intepret the ${VAR1} into myvalue1 and ${VAR2} into myvalue2 and write the output to new file...... I only want to intepret the variable only, all other text should remain unchanged........

    eg.
    file1_new.txt
    -----------------------------
    'some text with single quote.......'
    myvalue1 is bah bah
    "some anthoer text with double quote"
    myvalue2 is bah bah


    Thanks for you help...... :-)
    I will state my question clearly next time........ Thanks, fla5do, Damian Ibbotson ......
    Last edited by r390gt1; 11-07-03 at 08:27.

  9. #9
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525

    Re: Sorry for keeping you guys waiting, I give a example for my case..

    I exported two variable named VAR1 and VAR2 using the following commands...... all the variable setting are centralized in one shell script.
    export VAR1=myvalue1
    export VAR2=myvalue2

    and I have multiple files that contain these variable and these files are not shell script......
    eg.
    file1.txt
    -----------------------------
    'some text with single quote.......'
    ${VAR1} is bah bah
    "some anthoer text with double quote"
    ${VAR2} is bah bah
    Now I understand! The problem is that the order that command line tokens are evaluated by the shell means that variable substitution occurs before command substitution.

    If you tried...

    cat << !!
    ${VAR1}
    !!

    VAR1 would be evaluated to "myvalue1". Whereas...

    cat file1

    or even

    cat << !!
    $(cat file1)
    !!

    ... would not see your variables evaluated.

    You can force the shell to evaluate the line more than once using the 'eval' command but seen as your input lines are fixed and contain quotes I'm not sure it could be done. Something like...

    while read line
    do
    eval echo \"$line\"
    done < file1

    (This would not output all of your quotes correctly!)

    I came up with this solution but I kind of feel like I cheated...

    awk '{system ("cat << !!\n"$0"\n!!")}' file1

    Maybe someone could provide something less hacky?

    Damian

  10. #10
    Join Date
    May 2003
    Posts
    27

    the above solution is working

    while read line
    do
    eval echo \"$line\"
    done < file1
    the above solution is working, thanks Damian Ibbotson

    my version is as below, I trying to intrept only the line contains this pattern '${' ...... however, it run rather slow....... :-(


    while read
    do
    print -R "$REPLY" | grep \$\{ > /dev/null
    if (( $? == 0 ))
    then
    LINE="\"$REPLY\""
    eval "echo $LINE >> ${NEW}/$myfile"
    else
    echo "$REPLY" >> ${NEW}/$myfile
    fi
    done < ${OLD}/$myfile


    is there anyway to improve the intrept speed...???

  11. #11
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525

    Re: the above solution is working

    the above solution is working, thanks Damian Ibbotson

    my version is as below, I trying to intrept only the line contains this pattern '${' ...... however, it run rather slow....... :-(


    while read
    do
    print -R "$REPLY" | grep \$\{ > /dev/null
    if (( $? == 0 ))
    then
    LINE="\"$REPLY\""
    eval "echo $LINE >> ${NEW}/$myfile"
    else
    echo "$REPLY" >> ${NEW}/$myfile
    fi
    done < ${OLD}/$myfile


    is there anyway to improve the intrept speed...???
    while read
    do
    print -r $REPLY | grep "\${" 1> /dev/null && eval echo \\'$REPLY\\' || print -r $REPLY
    done < file1 > newFile

    The above solution does the same with the logic trimmed down a little.

    It does not *work* however! Consider the following data:

    ${VAR1} is bah bah
    ${VAR2} is bah bah
    ${VAR2} is 'bah bah'
    ${VAR2} is "bah bah"

    You are going to have problems with lines that contain variable references along with quotes or backslashes.

    The awk solution I gave you is the most complete (you do need to 'export' your variables initially though).

    awk '{system ("cat << !!\n"$0"\n!!")}' file1 > newFile

  12. #12
    Join Date
    May 2003
    Posts
    27
    Sorry, Damian Ibbotson
    I am a newbie to awk, for the command

    awk '{system ("cat << !!\n"$0"\n!!")}' file1 > newFile

    I am not quite understand this part....... !!\n"$0"\n!!"
    especially for the !!

    what i understand is:
    awk read line by line from file1 and run the system command cat which has a pipelined input from !!\n"$0"\n!!", the $0 mean the each line read from the file..........

    !! means ????
    \n means enter ??
    Last edited by r390gt1; 11-17-03 at 22:29.

  13. #13
    Join Date
    Jun 2002
    Location
    UK
    Posts
    525
    Originally posted by r390gt1
    Sorry, Damian Ibbotson
    I am a newbie to awk, for the command

    awk '{system ("cat << !!\n"$0"\n!!")}' file1 > newFile

    I am not quite understand this part....... !!\n"$0"\n!!"
    especially for the !!

    what i understand is:
    awk read line by line from file1 and run the system command cat which has a pipelined input from !!\n"$0"\n!!", the $0 mean the each line read from the file..........

    !! means ????
    \n means enter ??
    The '!!' is simply an arbitrary identifier for the start and end of a string passed into a command. If a command takes a file as an argument you can (usually) pass a string as if it were a file using this syntax.

    e.g.

    sed 's/a/b/g' << !!
    abc
    !!

    outputs...

    bbc

    You can use any string as an identifier (with some exceptions, e.g. &, -).

    e.g.

    cat << anyString
    hello world!
    anyString

    The "\n" in the awk system call represents a new line, so 'system ("cat << !!\n"$0"\n!!")', is passing the following to the shell...

    cat << !!
    theInputLineFromYourFile
    !!

    So... if your input line was '${VAR1} is bah bah', this would be...

    cat << !!
    ${VAR1} is bah bah
    !!

    Seen as variable substitution would occur in this instance, you would see ${VAR1} evaluated to its corresponding value.

    HTH
    Last edited by Damian Ibbotson; 11-18-03 at 06:05.

  14. #14
    Join Date
    May 2003
    Posts
    27

    Thanks, Damian Ibbotson

    Thanks, Damian Ibbotson

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •