If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

 
Go Back  dBforums > Data Access, Manipulation & Batch Languages > Unix Shell Scripts > Can I replace paragraphs using sed?

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
  #1 (permalink)  
Old 08-12-03, 03:14
granite granite is offline
Registered User
 
Join Date: Aug 2003
Posts: 4
Lightbulb Can I replace paragraphs using sed?

Hi,
I have this problem and I wish to know is there an easy way to solve it.

I have a file containing in it couples of lines with a specific format (among many other lines).
The format of the couples look like that:
rocks 20
books 12

or:
rocks 32
books 7

etc..
the numbers follow rocks and books could be any numbers.

I want to get 4 numbers as parameters and than to check these couples at the file for a match, if there is a match than the numbers of these certain couples at the file should be replace.

example:

the file a.txt starts here
the numbers
is it true
rocks 20
books 12
yes
I want to be there
rocks 12
books 20
rocks 65
books 17
why is it
rocks 20
books 12
so difficult
rocks 20
books 78
the file a.txt ends here

Now if I get the parameters : 20 12 100 200

than it means that I should look for the couples:
rocks 20
books 12
and change them to:
rocks 100
books 200

the file after the changes should look like that:

the file a.txt starts here
the numbers
is it true
rocks 100
books 200
yes
I want to be there
rocks 12
books 20
rocks 65
books 17
why is it
rocks 100
books 200
so difficult
rocks 20
books 78
the file a.txt ends here

Is it possible to use the sed command in order to search for a pattern of 2 lines with the first 2 numbers :
rocks 20
books 12

and than to replace them with the last 2 numbers?
rocks 100
books 200

I need to know is there a short easy way with sed or should I write scripts to solve that,

thanks for your help
Reply With Quote
  #2 (permalink)  
Old 08-12-03, 10:13
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
I'm not entirely sure what you are trying to do but I think I get the jist of it.

The following example would do the sort of thing you are asking using awk. Just pass in the type (rocks or books), the existing count and the replacement count.

awk '$1==type && $2==count {sub(/.*/,newCount,$2)}{print}' type=rocks count=40 newCount=100 < yourFile

In sed, you would want to control the parameters using the shell. The example below assumes that the type and the count are separated by a single space and that there is no leading or trailing whitespace.

type=rocks; count=40; newCount=100
sed "/^$type $count$/s/$count/$newcount/" < yourFile
Reply With Quote
  #3 (permalink)  
Old 08-12-03, 10:52
granite granite is offline
Registered User
 
Join Date: Aug 2003
Posts: 4
Quote:
Originally posted by Damian Ibbotson
I'm not entirely sure what you are trying to do but I think I get the jist of it.

The following example would do the sort of thing you are asking using awk. Just pass in the type (rocks or books), the existing count and the replacement count.

awk '$1==type && $2==count {sub(/.*/,newCount,$2)}{print}' type=rocks count=40 newCount=100 < yourFile

In sed, you would want to control the parameters using the shell. The example below assumes that the type and the count are separated by a single space and that there is no leading or trailing whitespace.

type=rocks; count=40; newCount=100
sed "/^$type $count$/s/$count/$newcount/" < yourFile
Hello damian,

Tnx for your quick replay.
However the most important thing is that the two lines appear one after the other
and only in that format exactly :
rocks 12
books 20

than, if that is the situation, I want to replace these 2 lines to:
rocks 100
books 200

If for example the line:
rocks 12

appears and not followed by the line:
books 20

than I don't want to change it at all.

I wanted to know whether sed can find a pattern of 2 lines:
rocks 12
books 20

and replace it by
rocks 100
books 200


Regards meirav
Reply With Quote
  #4 (permalink)  
Old 08-13-03, 06:01
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
Here's a rather low-tech method using a sed script. I've included some comments to help you see what's going on.

Save to a file and execute using sed -f yourSedFile yourInputFile

----------------------------------------------------------------

#if pattern space matches "rocks 20"
/rocks 20/ {
#append next line of input to pattern space
N
#if pattern space matches "books 30"
/books 30/ {
#perform substitutions
s/rocks 20\(.*\)books 30/rocks 88\1books 77/
}
}

-----------------------------------------------------------------
Reply With Quote
  #5 (permalink)  
Old 08-13-03, 07:19
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
A slight tweak so that "rocks 20XXXX" wouldn't also get changed.

/rocks 20$/ {
N
/books 30$/ {
s/rocks 20\(.*\)books 30/rocks 88\1books 77/
}
}
Reply With Quote
  #6 (permalink)  
Old 08-14-03, 02:12
granite granite is offline
Registered User
 
Join Date: Aug 2003
Posts: 4
Cool Can I find&replace paragraphs using sed command?

Hello damian,

Thanks a lot for the answer it works perfectly!
I have for you 2 more questions:

1. Is there a way to pass the 4 numbers as parameters? ( if not, I thought to
write a script that will build the sed file every time with the numbers I will give it).

2. I wrote a C - shell script (attached below) which does the same,
but when I compare the running time of your sed file to my script,
both on a 1M size of file.
well it took yours - 2 seconds while 7 minutes to mine,
do you know why the big difference?

I will appreciate your answers,
Regards meirav



The command line is:



>Convert_lines num1 num2 num3 num4







Convert_lines:



#!/usr/local/bin/tcsh –f



cat a.txt | convert $1 $2 $3 $4









Convert:





#!/usr/bin/tcsh -f

set line1 = ( $< )

set line2 = ( $< )

\rm tmp.txt

touch tmp.txt



while ( ${#line2} != 0 )

if ( ${#line1} == 2 && ${#line2} == 2 ) then

if ($line1[1] =~ "rocks" && $line2[1] =~ "books” && $line1[2] =~ $1 && $line2[2] =~ $2 ) then



set line1=("$line1[1] $3")

set line2=("$line2[1] $4")

echo "$line1" >> tmp.txt

echo "$line2" >> tmp.txt

set line1=( $< )

set line2=( $< )



else

echo "$line1" >> tmp.txt

set line1 = ( $line2 )

set line2=( $< )



endif



else



echo "$line1" >> tmp.txt

set line1 = ( $line2 )

set line2=( $< )

endif



end



echo "$line1" >> tmp.txt
Reply With Quote
  #7 (permalink)  
Old 08-14-03, 04:46
Damian Ibbotson Damian Ibbotson is offline
Padawan
 
Join Date: Jun 2002
Location: UK
Posts: 525
If you place double quotes around your sed command, the shell will evaluate it first and expand any shell variables.

--------------------------------------------------------------------------
# sedUsingTheShell.sh

rocksVal=20
booksVal=30
newRocksVal=88
newBooksVal=77

sed "
/rocks ${rocksVal}$/ {
N
/books ${booksVal}$/ {
s/rocks ${rocksVal}\(.*\)books ${booksVal}/rocks ${newRocksVal}\1books ${newBooksVal}/
}
}" < yourFile

---------------------------------------------------------------------------

As to why yours is slower...

I'm not familar with tcsh but generally shells are notoriously slow when implementing a 'while read' type construct. I won't claim to understand why because I don't. Sed on the other hand, is notoriously quick.

It would probably be a little quicker if you executed it like this...

convert_lines var1 var2 var3 var4 < inputFile

The use of 'cat' is unnecessary.



Last edited by Damian Ibbotson; 08-14-03 at 10:17.
Reply With Quote
  #8 (permalink)  
Old 08-14-03, 13:27
granite granite is offline
Registered User
 
Join Date: Aug 2003
Posts: 4
Smile Can I find&replace paragraphs using sed command?

Hello Damian,

The script works just excellent.
I tested it on 100MB files and it runs very fast.

Thanks a lot
Best regards meirav
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On