Unix Programming - Tricky commandline parameters

Author

Tricky commandline parameters

John Smith

2005-04-11, 7:52 am

Lets say you have a file foo.txt and a file bar.txt. Each file contains
textlines with one value at each line like:

foo.txt:
aaaaaa
bbbbbbb

bar.txt:
ddd
aaaaaa
fffffff
bbbbbbb
ggggg

I'd like to make a new file foobar.txt which contains all the unique lines
from both but the common ones must be removed. In other words all the lines
in foo.txt must be removed from bar.txt. How can I acomplish this from
commandline?

Secondly each line represents a commandline setting for strip utility.
These will be used with "strip -N text_symbol". However the problem is that
the list is so long that the shell complains about too long argument list.
How can I fix this? One way I thought of was to run strip utility once for
each line. I know it probably takes much longer but it's a price worth
paying for getting it work.
But how can you execute strip for each line in foobar.txt?

All this is executed from a makefile under Redhat 9.0 (x86).

Thanks in advance.
-- John

SM Ryan

2005-04-11, 7:52 am

# I'd like to make a new file foobar.txt which contains all the unique lines
# from both but the common ones must be removed. In other words all the lines
# in foo.txt must be removed from bar.txt. How can I acomplish this from
# commandline?

Something like sort ... | comm -13 ... | sed 's/^ //' ...

# Secondly each line represents a commandline setting for strip utility.
# These will be used with "strip -N text_symbol". However the problem is that
# the list is so long that the shell complains about too long argument list.
# How can I fix this? One way I thought of was to run strip utility once for
# each line. I know it probably takes much longer but it's a price worth
# paying for getting it work.
# But how can you execute strip for each line in foobar.txt?

Does your strip allow something like -R to read symbol names from a file?

--
SM Ryan http://www.rawbw.com/~wyrmwif/
I think that's kinda of personal; I don't think I should answer that.

Ralf Fassel

2005-04-11, 7:52 am

* "John Smith" <john.smith@x-formation.com>
| In other words all the lines in foo.txt must be removed from
| bar.txt.

Check the `comm' command. `comm' requires sorted files, though.

| But how can you execute strip for each line in foobar.txt?

while read line ; do
echo "line read $line"
done < foobar.txt

`xargs' might also be of some help, though in your example it might
not be applicable due to the -N switch needed for each argument.

HTH
R'

Måns Rullgård

2005-04-11, 7:52 am

Henry Townsend

2005-04-11, 5:59 pm

Måns Rullgård wrote:

> cat foo.txt bar.txt | sort | uniq -u

UUOC (and UUOU).

sort -u foo.txt bar.txt

--
Henry Townsend

Pascal Bourguignon

2005-04-11, 5:59 pm

"John Smith" <john.smith@x-formation.com> writes:

> Lets say you have a file foo.txt and a file bar.txt. Each file contains
> textlines with one value at each line like:
>
> foo.txt:
> aaaaaa
> bbbbbbb
>
> bar.txt:
> ddd
> aaaaaa
> fffffff
> bbbbbbb
> ggggg
>
> I'd like to make a new file foobar.txt which contains all the unique lines
> from both but the common ones must be removed. In other words all the lines
> in foo.txt must be removed from bar.txt. How can I acomplish this from
> commandline?

ldiff bar.txt foo.txt

> Secondly each line represents a commandline setting for strip utility.
> These will be used with "strip -N text_symbol". However the problem is that
> the list is so long that the shell complains about too long argument list.
> How can I fix this?

I hear that xargs can be used in such circonstances...

> One way I thought of was to run strip utility once for
> each line. I know it probably takes much longer but it's a price worth
> paying for getting it work.
> But how can you execute strip for each line in foobar.txt?

while read line ; do strip -N $line ; done << foobar.txt

> All this is executed from a makefile under Redhat 9.0 (x86).
>
> Thanks in advance.
> -- John

------------------------------------------------------------------------
/ ****************************************
**************************************
FILE: ldiff.c
LANGUAGE: ANSI-C
SYSTEM: ANSI
USER-INTERFACE: ANSI
DESCRIPTION
This program build the difference (set operator) between two input files.
ie.:
file1: file2: ldiff file1 file2
aaaa bbb aaaa
bbb ccc ddd
bbb eee <EOF>
ccc <EOF>
ddd
<EOF>
AUTHORS
<PJB> Pascal J. Bourguignon
MODIFICATIONS
1992-12-18 <PJB> Created.
LEGAL
GPL

Copyright Pascal J. Bourguignon 1992 - 2005

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version
2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be
useful, but WITHOUT ANY WARRANTY; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public
License along with this program; if not, write to the Free
Software Foundation, Inc., 59 Temple Place, Suite 330,
Boston, MA 02111-1307 USA
****************************************
**************************************/
#include <stdio.h>
#include <string.h>

#define LineSize (4096)

static void ldiff(FILE* file1,FILE* file2,FILE* output)
{
char line1[LineSize];
char line2[LineSize];
int neof1;
int neof2;
int cmp;

neof1=(fgets(line1,LineSize-1,file1)!=NULL);
neof2=(fgets(line2,LineSize-1,file2)!=NULL);
while((neof1)&&(neof2)){
/*
line1 < line2 => fputs(line1,output); fgets(line1,LineSize-1,file1);
line1 = line2 => fgets(line1,LineSize-1,file1);
line1 > line2 => fgets(line2,LineSize-1,file2);
*/
cmp=strcmp(line1,line2);
if(cmp<0){
fputs(line1,output);
neof1=(fgets(line1,LineSize-1,file1)!=NULL);
}else if(cmp==0){
neof1=(fgets(line1,LineSize-1,file1)!=NULL);
}else{
neof2=(fgets(line2,LineSize-1,file2)!=NULL);
}
}
while(neof1){
fputs(line1,output);
neof1=(fgets(line1,LineSize-1,file1)!=NULL);
}
}/*ldiff*/

int main(int argc,char** argv)
{
FILE* file1;
FILE* file2;

if(argc!=3){
fprintf(stderr,"# usage: %s file1 file2 \n",argv[0]);
fprintf(stderr,"# outputs all lines in file1 not in file2 "
"(sorted files).\n");
return(1);
}

file1=fopen(argv[1],"r");
if(file1==NULL){
fprintf(stderr,"Cannot open file <%s>.\n",argv[1]);
return(8+1);
}

file2=fopen(argv[2],"r");
if(file2==NULL){
fprintf(stderr,"Cannot open file <%s>.\n",argv[2]);
return(8+2);
}

ldiff(file1,file2,stdout);

fclose(file1);
fclose(file2);
return(0);
}/*main*/

/*** ldiff.c -- 2003-11-18 19:41:46 -- pascal ***/
------------------------------------------------------------------------

--
__Pascal Bourguignon__ http://www.informatimago.com/

This is a signature virus. Add me to your signature and help me to live

John W. Krahn

2005-04-11, 5:59 pm

John Smith wrote:
> Lets say you have a file foo.txt and a file bar.txt. Each file contains
> textlines with one value at each line like:
>
> foo.txt:
> aaaaaa
> bbbbbbb
>
> bar.txt:
> ddd
> aaaaaa
> fffffff
> bbbbbbb
> ggggg
>
> I'd like to make a new file foobar.txt which contains all the unique lines
> from both but the common ones must be removed. In other words all the lines
> in foo.txt must be removed from bar.txt. How can I acomplish this from
> commandline?

grep -v -f foo.txt bar.txt > foobar.txt

John
--
use Perl;
program
fulfillment

Ralf Fassel

2005-04-12, 7:49 am

* "John W. Krahn" <someone@example.com>
| grep -v -f foo.txt bar.txt > foobar.txt

Better use fgrep? Otherwise it might strip too much if e.g. a dot
in foo.txt matches any char in bar.txt.

R'