sed/regexp: remove empty lines
Web Server forum
Back To The Forum Home!Search!Private Messaging System

Web Server Talk Web Server Talk > Unix and Linux reviews > Free Unix support > Unix Shell > sed/regexp: remove empty lines




  Last Thread   Next Thread Next
  Show Printable Version Email this Page Subscribe to this Thread      Post New Thread    Post A Reply      

    sed/regexp: remove empty lines  
rolf randen


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-19-07 06:18 PM

hello there,

i'm having a problem with a shell-script for removing empty lines from a
logfile.

i do
sed -e '^$//'
and get the error
": invalid command code ^

this is the whole script:

#!/bin/bash
cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |
sed -e 's/xtar.*Icon.*d.*directory//' | sed -e 's/xtar.*Network Trash
Folder.*denied//' | sed -e '^$//' | mail -s "backup-daily"
logcheck@mailserver.local

without removing the empty lines the script runs fine.
i'm working with bash on macos 10.3.9

please help!

rolf





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
Icarus Sparry


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-19-07 06:18 PM

On Fri, 19 Jan 2007 18:16:48 +0100, rolf randen wrote:

> hello there,
>
> i'm having a problem with a shell-script for removing empty lines from a
> logfile.
>
> i do
> sed -e '^$//'
> and get the error
> ": invalid command code ^
>
> this is the whole script:
>
> #!/bin/bash
> cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |
> sed -e 's/xtar.*Icon.*d.*directory//' | sed -e 's/xtar.*Network Trash
> Folder.*denied//' | sed -e '^$//' | mail -s "backup-daily"
> logcheck@mailserver.local
>
> without removing the empty lines the script runs fine.
> i'm working with bash on macos 10.3.9
>
> please help!
>
> rolf

sed is objecting to the ^ character. What you want is
sed -e '/^$/d'

the /..../ matches a pattern. The pattern ^$ is start-of-line followed
immediately be end-of-line. The 'd' is the action to take when the pattern
matches, in this case delete the line.





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
Bill Marcum


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-19-07 06:18 PM

On Fri, 19 Jan 2007 18:16:48 +0100, rolf randen
<randenrolf@gmx.net> wrote:
>
>
> hello there,
>
> i'm having a problem with a shell-script for removing empty lines from a
> logfile.
>
> i do
> sed -e '^$//'
> and get the error
> ": invalid command code ^
>
sed -e '/^$/d'


--
BOFH excuse #92:
Stale file handle (next time use Tupperware(tm)!)





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
Ed Morton


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-19-07 06:18 PM

rolf randen wrote:

> hello there,
>
> i'm having a problem with a shell-script for removing empty lines from a
> logfile.
>
> i do
> sed -e '^$//'
> and get the error
> ": invalid command code ^
>
> this is the whole script:
>
> #!/bin/bash
> cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |
> sed -e 's/xtar.*Icon.*d.*directory//' | sed -e 's/xtar.*Network Trash
> Folder.*denied//' | sed -e '^$//' | mail -s "backup-daily"
> logcheck@mailserver.local
>
> without removing the empty lines the script runs fine.
> i'm working with bash on macos 10.3.9
>
> please help!
>
> rolf


You got your specific answer (/^$/d) but you could simplify the whole
script:

cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |
sed -e 's/xtar.*Icon.*d.*directory//' | sed -e 's/xtar.*Network Trash
Folder.*denied//' | sed -e '^$//' | ...

The "cat" is useless since sed can open a file just as easily as cat
can, so this:

cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |

could be just this:

sed -e 's/xtar.*Icon..d.*directory//' /Volumes/Save/daily.log |

The first of these 2 sed commands:

sed -e 's/xtar.*Icon..d.*directory//' |
sed -e 's/xtar.*Icon.*d.*directory//' |

is a subset of the second since ".." (any 2 characters) is a subset of
".*" (any sequence of characters), so the above can be reduced to just:

sed -e 's/xtar.*Icon.*d.*directory//' |

so, so far, you're left with:

sed -e 's/xtar.*Icon.*d.*directory//' /Volumes/Save/daily.log |
sed -e 's/xtar.*Network Trash Folder.*denied//' |
sed -e '/^$/d' |
mail -s "backup-daily"
logcheck@mailserver.local

sed only supports BREs but if you switch to awk you can take advantage
of EREs to combine the first 2 sed commands to this:

awk '{sub(/xtar.*(Icon.*d.*directory|Network Trash
Folder.*denied)/,"")}1' /Volumes/Save/daily.log

and you can also test the NF (Number of Fields) variable in awk to get
rid of empty lines thus removing the need for the final sed command and
so the whole scipt becomes just:

#!/bin/bash
awk '{sub(/xtar.*(Icon.*d.*directory|Network Trash
Folder.*denied)/,"")}NF' /Volumes/Save/daily.log |
mail -s "backup-daily"
logcheck@mailserver.local

Use of "NF" as above will also remove lines that contain just blank
characters. If you WANT those to be printed, then use "!/^$/" instead:

#!/bin/bash
awk '{sub(/xtar.*(Icon.*d.*directory|Network Trash
Folder.*denied)/,"")}!/^$/' /Volumes/Save/daily.log |
mail -s "backup-daily"
logcheck@mailserver.local

Regards,

Ed.





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
Chris F.A. Johnson


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-19-07 06:18 PM

On 2007-01-19, rolf randen wrote:
> hello there,
>
> i'm having a problem with a shell-script for removing empty lines from a
> logfile.
>
> i do
> sed -e '^$//'

sed -e '/^$/d'

> and get the error
> ": invalid command code ^
>
> this is the whole script:
>
> #!/bin/bash
> cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |
> sed -e 's/xtar.*Icon.*d.*directory//' | sed -e 's/xtar.*Network Trash
> Folder.*denied//' | sed -e '^$//' | mail -s "backup-daily"
> logcheck@mailserver.local

sed -e 's/xtar.*Icon..d.*directory//' \
-e 's/xtar.*Network Trash Folder.*denied//' \
-e '/^$/d' /Volumes/Save/daily.log |
mail -s "backup-daily" logcheck@mailserver.local

> without removing the empty lines the script runs fine.
> i'm working with bash on macos 10.3.9
>
> please help!
>
> rolf


--
Chris F.A. Johnson, author        <http://cfaj.freeshell.org/shell>
Shell Scripting Recipes: A Problem-Solution Approach (2005, Apress)
===== My code in this post, if any, assumes the POSIX locale
===== and is released under the GNU General Public Licence





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
Alan Curry


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-20-07 12:29 AM

In article <hob784-bkj.ln1@don.localnet>,
Bill Marcum  <marcumbill@bellsouth.net> wrote:
>sed -e '/^$/d'

"grep ." is easier to type, easier to read, harder to get wrong; why is
sed '/^$/d' the popular idiom?

--
Alan Curry
pacman@world.std.com





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
Bill Marcum


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-20-07 06:27 AM

On Fri, 19 Jan 2007 21:08:32 +0000 (UTC), Alan Curry
<pacman@TheWorld.com> wrote:
>
>
> In article <hob784-bkj.ln1@don.localnet>,
> Bill Marcum  <marcumbill@bellsouth.net> wrote: 
>
> "grep ." is easier to type, easier to read, harder to get wrong; why is
> sed '/^$/d' the popular idiom?
>
I was just correcting the sed command in the original post. "grep ." is
indeed easier.

--
"The lesser of two evils -- is evil."
-- Seymour (Sy) Leon





[ Post a follow-up to this message ]



    Re: sed/regexp: remove empty lines  
rolf randen


View Ip Address Report This Message To A Moderator Edit/Delete Message


 
01-22-07 06:16 PM

rolf randen schrieb:
> hello there,
>
> i'm having a problem with a shell-script for removing empty lines from a
> logfile.
>
> i do
> sed -e '^$//'
> and get the error
> ": invalid command code ^
>
> this is the whole script:
>
> #!/bin/bash
> cat /Volumes/Save/daily.log | sed -e 's/xtar.*Icon..d.*directory//' |
> sed -e 's/xtar.*Icon.*d.*directory//' | sed -e 's/xtar.*Network Trash
> Folder.*denied//' | sed -e '^$//' | mail -s "backup-daily"
> logcheck@mailserver.local
>
> without removing the empty lines the script runs fine.
> i'm working with bash on macos 10.3.9
>
> please help!
>
> rolf


hey, unbelievable,

this is the first time i get useful answers in a newsgroup... and then a
whole bunch of it! thanks a lot to all of you!

rolf





[ Post a follow-up to this message ]



    Sponsored Links  




 





   All times are GMT. The time now is 01:39 AM.      Post New Thread    Post A Reply      
  Last Thread   Next Thread Next


Most Popular forums 

Forum Jump:
Rate This Thread:

Forum Rules:
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is OFF
vB code is ON
Smilies are ON
[IMG] code is OFF
 
Medical and Health forum | Computer Games Reviews | Graphics design forum

Back To The Top
Home | Usercp | Faq | Register