If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Data Access, Manipulation & Batch Languages > Unix Shell Scripts > how to translate shell variable inside a file to a new file?

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 11-05-03, 01:47
r390gt1 r390gt1 is offline
Registered User
 
Join Date: May 2003
Posts: 27
Question how to translate shell variable inside a file to a new file?

I have many files that contain multiple shell variables in ${myvar} format. I want to translate all the variables in the its corresponding value and output all content to a new file. however, in lines other than the those contain the shell variable have the double quote or single quote character, and that's why I can use the cat commands to do the translation...........

does anyone know how to do this?

thanks in advance!!
Reply With Quote
  #2 (permalink)  
Old 11-05-03, 04:11
hacker hacker is offline
Registered User
 
Join Date: Oct 2003
Posts: 18
Can you post an example of what it is that you want?
Reply With Quote
  #3 (permalink)  
Old 11-05-03, 05:23
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Re: how to translate shell variable inside a file to a new file?

Quote:
Originally posted by r390gt1
I have many files that contain multiple shell variables in ${myvar} format. I want to translate all the variables in the its corresponding value and output all content to a new file. however, in lines other than the those contain the shell variable have the double quote or single quote character, and that's why I can use the cat commands to do the translation...........

does anyone know how to do this?
If I understand you correctly, you can't do what you are asking. How can you retrieve the associated value for a variable that has not yet been set (i.e. if the script is not actually running)?

You can use 'set' within your script at some point to display all the variables that are being used.

---------------------------------
var1=a
var2=b
var3=c
set > scriptParametersFile
---------------------------------

Or maybe even source your script in a subshell and use set.

(. yourScript 2> /dev/null; set > scriptParametersFile)

These should be fine if your scripts are pretty simple.

HTH
Reply With Quote
  #4 (permalink)  
Old 11-05-03, 14:50
fla5do fla5do is offline
Registered User
 
Join Date: Oct 2003
Location: Germany
Posts: 138
Hello r390gt1,
if I understand you correctly, all of your scripts use the same variables and you will change the values at a central point and not in every script.
Values which where changed from script1 should be used from script2, such as an update from your central point values.
Is this correct ?

Try this:

#### create a new file which called VAR_LIST.TXT ( for example )
#### with following values
# VAR1|1111
# VAR2|2222
# VARx|3333
#### this file will be the central point for the variables for all your scripts


# Use this at all your scripts which work with the central pointed variables
# each script can change the values to be used from the next script.


# insert the followed rows at the top of every script
################################################## #############################
################################################## #############################
VAR1=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
{
if ($1 == "VAR1")
{
print $2
}
}'`
VAR2=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
{
if ($1 == "VAR2")
{
print $2
}
}'`
VARx=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
{
if ($1 == "VARx")
{
print $2
}
}'`
echo $VAR1
echo $VAR2
echo $VARx

# now we change the values from VAR1 - VARx
# and update the central file with the new values for the next operation
VAR1=`expr $VAR1 + 1`
VAR2=`expr $VAR2 + 1`
VARx=`expr $VARx + 1`
echo "VAR1|"$VAR1 > VAR_LIST.TXT
echo "VAR2|"$VAR2 >> VAR_LIST.TXT
echo "VARx|"$VARx >> VAR_LIST.TXT

# start the the sript again and you will get the updatet values

max be I can help you out.

Peter.F
__________________
Greetings from germany
Peter F.

Last edited by fla5do; 11-05-03 at 17:00.
Reply With Quote
  #5 (permalink)  
Old 11-06-03, 03:56
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Quote:
Originally posted by fla5do
#### create a new file which called VAR_LIST.TXT ( for example )
#### with following values
# VAR1|1111
# VAR2|2222
# VARx|3333
#### this file will be the central point for the variables for all your scripts

# Use this at all your scripts which work with the central pointed variables
# each script can change the values to be used from the next script.

# insert the followed rows at the top of every script
################################################## #############################
################################################## #############################
VAR1=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
{
if ($1 == "VAR1")
{
print $2
}
}'`
VAR2=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
{
if ($1 == "VAR2")
{
print $2
}
}'`
VARx=`cat VAR_LIST.TXT | awk ' BEGIN { FS="|"}
{
if ($1 == "VARx")
{
print $2
}
}'`
Isn't this the same as...

parameterListFile
---------------------
VAR1=1111
VAR2=2222
VARx=3333

yourScript
---------------------
. parameterListFile

???
Reply With Quote
  #6 (permalink)  
Old 11-06-03, 13:15
fla5do fla5do is offline
Registered User
 
Join Date: Oct 2003
Location: Germany
Posts: 138
Isn't this the same as...

parameterListFile
---------------------
VAR1=1111
VAR2=2222
VARx=3333

yourScript
---------------------
. parameterListFile

???
-----------------------------------------------------------------------------
Hello Damian,
I think itīs not the same. I describe a way for input and output,
with a complete example and comments. And in my ParametersFile the
variables can be named for better reading and understanding later.

Your solution;
------------------
var1=a
var2=b
var3=c
set > scriptParametersFile

!!! the command "set" makes an output from all enviromente variables in your shell, but not only "var1 - var3"


Peter F.
__________________
Greetings from germany
Peter F.

Last edited by fla5do; 11-06-03 at 14:54.
Reply With Quote
  #7 (permalink)  
Old 11-07-03, 04:48
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Quote:
Hello Damian,
I think itīs not the same. I describe a way for input and output,
with a complete example and comments. And in my ParametersFile the
variables can be named for better reading and understanding later.
Fair enough! I'd be concerned about the maintainability though.

And yes, 'set' will output the environment variables as well but I'm sure a work around would be simple enough.

Last edited by Damian Ibbotson; 11-07-03 at 04:56.
Reply With Quote
  #8 (permalink)  
Old 11-07-03, 07:13
r390gt1 r390gt1 is offline
Registered User
 
Join Date: May 2003
Posts: 27
Talking Sorry for keeping you guys waiting, I give a example for my case..

I exported two variable named VAR1 and VAR2 using the following commands...... all the variable setting are centralized in one shell script.
export VAR1=myvalue1
export VAR2=myvalue2

and I have multiple files that contain these variable and these files are not shell script......
eg.
file1.txt
-----------------------------
'some text with single quote.......'
${VAR1} is bah bah
"some anthoer text with double quote"
${VAR2} is bah bah


file2.txt
-----------------------------
"some anthoer text with double quote"
This is my var 2 ${VAR2}
'some text with single quote.......'


What I want is to intepret the ${VAR1} into myvalue1 and ${VAR2} into myvalue2 and write the output to new file...... I only want to intepret the variable only, all other text should remain unchanged........

eg.
file1_new.txt
-----------------------------
'some text with single quote.......'
myvalue1 is bah bah
"some anthoer text with double quote"
myvalue2 is bah bah


Thanks for you help...... :-)
I will state my question clearly next time........ Thanks, fla5do, Damian Ibbotson ......

Last edited by r390gt1; 11-07-03 at 07:27.
Reply With Quote
  #9 (permalink)  
Old 11-11-03, 09:22
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Re: Sorry for keeping you guys waiting, I give a example for my case..

Quote:
I exported two variable named VAR1 and VAR2 using the following commands...... all the variable setting are centralized in one shell script.
export VAR1=myvalue1
export VAR2=myvalue2

and I have multiple files that contain these variable and these files are not shell script......
eg.
file1.txt
-----------------------------
'some text with single quote.......'
${VAR1} is bah bah
"some anthoer text with double quote"
${VAR2} is bah bah
Now I understand! The problem is that the order that command line tokens are evaluated by the shell means that variable substitution occurs before command substitution.

If you tried...

cat << !!
${VAR1}
!!

VAR1 would be evaluated to "myvalue1". Whereas...

cat file1

or even

cat << !!
$(cat file1)
!!

... would not see your variables evaluated.

You can force the shell to evaluate the line more than once using the 'eval' command but seen as your input lines are fixed and contain quotes I'm not sure it could be done. Something like...

while read line
do
eval echo \"$line\"
done < file1

(This would not output all of your quotes correctly!)

I came up with this solution but I kind of feel like I cheated...

awk '{system ("cat << !!\n"$0"\n!!")}' file1

Maybe someone could provide something less hacky?

Damian
Reply With Quote
  #10 (permalink)  
Old 11-12-03, 03:40
r390gt1 r390gt1 is offline
Registered User
 
Join Date: May 2003
Posts: 27
the above solution is working

Quote:
while read line
do
eval echo \"$line\"
done < file1
the above solution is working, thanks Damian Ibbotson

my version is as below, I trying to intrept only the line contains this pattern '${' ...... however, it run rather slow....... :-(


while read
do
print -R "$REPLY" | grep \$\{ > /dev/null
if (( $? == 0 ))
then
LINE="\"$REPLY\""
eval "echo $LINE >> ${NEW}/$myfile"
else
echo "$REPLY" >> ${NEW}/$myfile
fi
done < ${OLD}/$myfile


is there anyway to improve the intrept speed...???
Reply With Quote
  #11 (permalink)  
Old 11-12-03, 04:59
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Re: the above solution is working

Quote:
the above solution is working, thanks Damian Ibbotson

my version is as below, I trying to intrept only the line contains this pattern '${' ...... however, it run rather slow....... :-(


while read
do
print -R "$REPLY" | grep \$\{ > /dev/null
if (( $? == 0 ))
then
LINE="\"$REPLY\""
eval "echo $LINE >> ${NEW}/$myfile"
else
echo "$REPLY" >> ${NEW}/$myfile
fi
done < ${OLD}/$myfile


is there anyway to improve the intrept speed...???
while read
do
print -r $REPLY | grep "\${" 1> /dev/null && eval echo \\'$REPLY\\' || print -r $REPLY
done < file1 > newFile

The above solution does the same with the logic trimmed down a little.

It does not *work* however! Consider the following data:

${VAR1} is bah bah
${VAR2} is bah bah
${VAR2} is 'bah bah'
${VAR2} is "bah bah"

You are going to have problems with lines that contain variable references along with quotes or backslashes.

The awk solution I gave you is the most complete (you do need to 'export' your variables initially though).

awk '{system ("cat << !!\n"$0"\n!!")}' file1 > newFile
Reply With Quote
  #12 (permalink)  
Old 11-17-03, 21:17
r390gt1 r390gt1 is offline
Registered User
 
Join Date: May 2003
Posts: 27
Sorry, Damian Ibbotson
I am a newbie to awk, for the command

awk '{system ("cat << !!\n"$0"\n!!")}' file1 > newFile

I am not quite understand this part....... !!\n"$0"\n!!"
especially for the !!

what i understand is:
awk read line by line from file1 and run the system command cat which has a pipelined input from !!\n"$0"\n!!", the $0 mean the each line read from the file..........

!! means ????
\n means enter ??

Last edited by r390gt1; 11-17-03 at 21:29.
Reply With Quote
  #13 (permalink)  
Old 11-18-03, 04:37
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Quote:
Originally posted by r390gt1
Sorry, Damian Ibbotson
I am a newbie to awk, for the command

awk '{system ("cat << !!\n"$0"\n!!")}' file1 > newFile

I am not quite understand this part....... !!\n"$0"\n!!"
especially for the !!

what i understand is:
awk read line by line from file1 and run the system command cat which has a pipelined input from !!\n"$0"\n!!", the $0 mean the each line read from the file..........

!! means ????
\n means enter ??
The '!!' is simply an arbitrary identifier for the start and end of a string passed into a command. If a command takes a file as an argument you can (usually) pass a string as if it were a file using this syntax.

e.g.

sed 's/a/b/g' << !!
abc
!!

outputs...

bbc

You can use any string as an identifier (with some exceptions, e.g. &, -).

e.g.

cat << anyString
hello world!
anyString

The "\n" in the awk system call represents a new line, so 'system ("cat << !!\n"$0"\n!!")', is passing the following to the shell...

cat << !!
theInputLineFromYourFile
!!

So... if your input line was '${VAR1} is bah bah', this would be...

cat << !!
${VAR1} is bah bah
!!

Seen as variable substitution would occur in this instance, you would see ${VAR1} evaluated to its corresponding value.

HTH

Last edited by Damian Ibbotson; 11-18-03 at 05:05.
Reply With Quote
  #14 (permalink)  
Old 11-18-03, 19:56
r390gt1 r390gt1 is offline
Registered User
 
Join Date: May 2003
Posts: 27
Thanks, Damian Ibbotson

Thanks, Damian Ibbotson
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On