Unix Programming - Best unix programming language or tool for dealing with plaintext DBs?

Author

Best unix programming language or tool for dealing with plaintext DBs?

casioculture@gmail.com

2005-11-03, 8:49 pm

What's the best unix programming language or tool for dealing with
simple plaintext databases.

It should be simple and easy, and should provide convenient handling of
fields and records.

So far I know that Awk does.

Pascal Bourguignon

2005-11-03, 8:49 pm

casioculture@gmail.com writes:
> What's the best unix programming language or tool for dealing with
> simple plaintext databases.
>
> It should be simple and easy, and should provide convenient handling of
> fields and records.
>
> So far I know that Awk does.

For my value of "best", it is Common Lisp.

--
__Pascal Bourguignon__ http://www.informatimago.com/
The mighty hunter
Returns with gifts of plump birds,
Your foot just squashed one.

casioculture@gmail.com

2005-11-04, 2:49 am

Pascal Bourguignon wrote:
> casioculture@gmail.com writes:
>
> For my value of "best", it is Common Lisp.
>

Why lisp?

>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/
> The mighty hunter
> Returns with gifts of plump birds,
> Your foot just squashed one.

Pascal Bourguignon

2005-11-04, 2:49 am

casioculture@gmail.com writes:

> Pascal Bourguignon wrote:
>
> Why lisp?

Because it's a true programming language.
You don't have any limitation or any artificial quirk.

For example, dealing with one file in a filter mode with awk is all
raight. But try to deal with two files with awk (eg. merge them), and
see what I mean.

(You might also consider scsh if you prefer scheme over Common Lisp).

--
__Pascal Bourguignon__ http://www.informatimago.com/
The mighty hunter
Returns with gifts of plump birds,
Your foot just squashed one.

icub3d

2005-11-04, 5:57 pm

In the end, It all depends on what programs or languages you are
familiar with. For me, If this was a program I would be using on a
regular basis for a long duration, I would write a small C program to
deal with it.

If on the otherhand, this was a small problem that I would only use
once or twice, then PERL or php would probably be ok.

Factor in your time (both programming and times you will run the
program) as well as the computers time and make a decision.

casioculture@gmail.com

2005-11-04, 5:57 pm

M=E5ns Rullg=E5rd wrote:
> Pascal Bourguignon <spam@mouse-potato.com> writes:
>
g of[vbcol=seagreen]
>
> PERL might be a reasonable choice, too.
>
> --
> M=E5ns Rullg=E5rd
> mru@inprovide.com

Would you guys suggest I learn PERL before lisp, or lisp before perl?

casioculture@gmail.com

2005-11-04, 5:57 pm

M=E5ns Rullg=E5rd wrote:
>
>
> "LISP is worth learning for the profound enlightenment experience you
> will have when you finally get it; that experience will make you a
> better programmer for the rest of your days." -- Eric S. Raymond
>
> You'll probably find it easier coming to terms with Perl.
>

I'm so sorry, I'm a little thick today, does this mean that learning
perl first would be easier, or that learning lisp first would make it
easier for me to come to terms with PERL when i later learn it?

joe@invalid.address

2005-11-04, 5:57 pm

casioculture@gmail.com writes:

> Måns Rullgård wrote:
>
> I'm so sorry, I'm a little thick today, does this mean that learning
> PERL first would be easier, or that learning lisp first would make
> it easier for me to come to terms with PERL when i later learn it?

Forget about lisp for something like this. This kind of thing is what
perl was made for.

Joe
--
Gort, klatu barada nikto

Pascal Bourguignon

2005-11-04, 5:57 pm

joe@invalid.address writes:

> casioculture@gmail.com writes:
>
>
> Forget about lisp for something like this. This kind of thing is what
> PERL was made for.

perl is unreadable and unwritable. At least, with APL it was
unreadable, but you could write it...

Lisp is readable and writable.

YMMV.

--
__Pascal Bourguignon__ http://www.informatimago.com/
You're always typing.
Well, let's see you ignore my
sitting on your hands.

casioculture@gmail.com

2005-11-04, 5:57 pm

Pascal Bourguignon wrote:
> joe@invalid.address writes:
>
>
> PERL is unreadable and unwritable. At least, with APL it was
> unreadable, but you could write it...
>
> Lisp is readable and writable.
>
> YMMV.
>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/
> You're always typing.
> Well, let's see you ignore my
> sitting on your hands.

I plan to learn both perl, php, and ruby, and also lisp, scheme, guile
and python, just in time; my current self-taught computing curriculum
is for a decade. You see, I'm really impressed with awk's simplicity at
handling textual databases.

For example

Looking at this page
http://stud.wsi.edu.pl/~robert/awk/tutorial_4a.html

BEGINQUOTE

For example, if we have list of second-hand car prices like this...

ford mondeo 1990 5800
ford fiesta 1991 4575
honda accord 1991 6000
toyota tercel 1992 6500
vaxhaull astra 1990 5950
vaxhaull carlton 1991 6450

=2E..we might want to select all the cars which were made after or
during 1991 (column 3) and cost less than 6,250 pounds (column 4). We
can do this with the following awk code:

awk '$3 >=3D1991 && $4 < 6250' cars

ENDQUOTE

How would PERL and lisp deal with this same textfile containing this
same list of second-hand car prices to do the same exact query as the
above? How would other languages do it that would be best for this
purpose?

Pascal Bourguignon

2005-11-04, 5:57 pm

casioculture@gmail.com writes:
> I plan to learn both perl, php, and ruby, and also lisp, scheme, guile
> and python, just in time; my current self-taught computing curriculum
> is for a decade. You see, I'm really impressed with awk's simplicity at
> handling textual databases.
>
> For example
>
> Looking at this page
> http://stud.wsi.edu.pl/~robert/awk/tutorial_4a.html
>
> BEGINQUOTE
>
> For example, if we have list of second-hand car prices like this...
>
> ford mondeo 1990 5800
> ford fiesta 1991 4575
> honda accord 1991 6000
> toyota tercel 1992 6500
> vaxhaull astra 1990 5950
> vaxhaull carlton 1991 6450
>
> ...we might want to select all the cars which were made after or
> during 1991 (column 3) and cost less than 6,250 pounds (column 4). We
> can do this with the following awk code:
>
> awk '$3 >=1991 && $4 < 6250' cars
>
> ENDQUOTE
>
> How would PERL and lisp deal with this same textfile containing this
> same list of second-hand car prices to do the same exact query as the
> above? How would other languages do it that would be best for this
> purpose?

Microbenchmarks won't teach you anything. If all you have to do is
this kind of work, then you don't need a programming language.

Come with a >1000-line script, and then we'll compare.

--
__Pascal Bourguignon__ http://www.informatimago.com/
Litter box not here.
You must have moved it again.
I'll poop in the sink.

casioculture@gmail.com

2005-11-04, 5:57 pm

Pascal Bourguignon wrote:
> casioculture@gmail.com writes:
>
> Microbenchmarks won't teach you anything. If all you have to do is
> this kind of work, then you don't need a programming language.
>
> Come with a >1000-line script, and then we'll compare.
>

Well, you see, I'm just starting - I plan to play with 1 to 10-line
scripts, then when i'm comfortable with that i'll move on to, say,
10-100, and so on. Is this a bad plan?

Giorgos Keramidas

2005-11-04, 5:57 pm

casioculture@gmail.com writes:
> I plan to learn both perl, php, and ruby, and also lisp,
> scheme, guile and python, just in time; my current self-taught
> computing curriculum is for a decade. You see, I'm really
> impressed with awk's simplicity at handling textual databases.
>
> For example
>
> Looking at this page
> http://stud.wsi.edu.pl/~robert/awk/tutorial_4a.html
>
> BEGINQUOTE
>
> For example, if we have list of second-hand car prices like this...
>
> ford mondeo 1990 5800
> ford fiesta 1991 4575
> honda accord 1991 6000
> toyota tercel 1992 6500
> vaxhaull astra 1990 5950
> vaxhaull carlton 1991 6450
>
> ...we might want to select all the cars which were made after
> or during 1991 (column 3) and cost less than 6,250 pounds
> (column 4). We can do this with the following awk code:
>
> awk '$3 >=1991 && $4 < 6250' cars
>
> ENDQUOTE

awk is pretty cool for doing simplistic processing of very simple
text that is clearly formatted in columns. Still, it has a few
serious shortcomings. A lot of things happen before some of the
original data is available to you as a programmer. One of my
usual gripes, even though I'm known for writing rather elaborate
monstrosities in awk one-lines, is that you have *NO* easy way to
find out what the original length of column $1 was, unless you
start playing with substr() and matching of $1 within $0 or use
some other 'hack'.

Pascal is *very* right in suggesting that you learn a language
that is suited for more 'general purpose' hacking.

- Giorgos

Chris F.A. Johnson

2005-11-04, 8:50 pm

On 2005-11-04, casioculture@gmail.com wrote:
>
> What's the best unix programming language or tool for dealing with
> simple plaintext databases.
>
> It should be simple and easy, and should provide convenient handling of
> fields and records.

Any language at all will do.

I'd use shell scripts.

If you were more specific about what you wanted to do with the
databases, what was in the databases, how large they are, etc., a
more specific suggestion might be forthcoming.

> So far I know that Awk does.

--
Chris F.A. Johnson | Author:
<http://cfaj.freeshell.org> | Shell Scripting Recipes:
Any code in this post is released | A Problem-Solution Approach,
under the GNU General Public Licence | 2005, Apress

joe@invalid.address

2005-11-04, 8:50 pm

Pascal Bourguignon <spam@mouse-potato.com> writes:

> joe@invalid.address writes:
>
>
> PERL is unreadable and unwritable. At least, with APL it was
> unreadable, but you could write it...

Funny, lots of people can both read and write perl. Hard to think how
it got to be so popular if it was unreadable and unwriteable.

> Lisp is readable and writable.
>
> YMMV.

So, I suppose, might yours.

Joe
--
Gort, klatu barada nikto

Giorgos Keramidas

2005-11-08, 6:29 pm

Russell Shaw <rjshawN_o@s_pam.netspace.net.au> writes:
> PERL is easy once you rtfm and use comments. I've only ever heard
> of lisp being used for some editor that masquerades as an OS;)

I'm sure you got the joke backwards and it's a bit too old to be
funny anymore

Bruintje Beer

2005-11-08, 6:29 pm

<casioculture@gmail.com> schreef in bericht
news:1131068497.449661.222670@z14g2000cwz.googlegroups.com...
>
> What's the best unix programming language or tool for dealing with
> simple plaintext databases.
>
> It should be simple and easy, and should provide convenient handling of
> fields and records.
>
> So far I know that Awk does.
>

what about cobol ?

John

William James

2005-11-08, 6:29 pm

casioculture@gmail.com wrote:

> I plan to learn both perl, php, and ruby, and also lisp, scheme, guile
> and python, just in time; my current self-taught computing curriculum
> is for a decade. You see, I'm really impressed with awk's simplicity at
> handling textual databases.
>
> For example
>
> Looking at this page
> http://stud.wsi.edu.pl/~robert/awk/tutorial_4a.html
>
> BEGINQUOTE
>
> For example, if we have list of second-hand car prices like this...
>
> ford mondeo 1990 5800
> ford fiesta 1991 4575
> honda accord 1991 6000
> toyota tercel 1992 6500
> vaxhaull astra 1990 5950
> vaxhaull carlton 1991 6450
>
> ...we might want to select all the cars which were made after or
> during 1991 (column 3) and cost less than 6,250 pounds (column 4). We
> can do this with the following awk code:
>
> awk '$3 >=1991 && $4 < 6250' cars
>
> ENDQUOTE
>
> How would PERL and lisp deal with this same textfile containing this
> same list of second-hand car prices to do the same exact query as the
> above? How would other languages do it that would be best for this
> purpose?

Here's a more complex problem. We have a list of books
owned by Bob and Jack. Each line has author, title,
condition, owner.

Print the books owned by Bob but not by Jack.
Sort them by author and title, ignoring "The " at the
beginning of a title. We'll use Ruby.

h = Hash.new( [] )

DATA.each{ |line|
author, title, owner =
/^ ( .*? ) # author
\. \s* "
( .*? ) # title
" .*
Owner: \s+
( .*? ) # Owner
[ \t]*
(?: \( | $ )
/x.match( line ).captures
h[ owner ] += [[ author, title ]]
}

puts ( h['Bob'] - h['Jack'] ).sort_by{ |x|
[ x.first, x.last[ /^(?:The )?(.*)/, 1 ] ] }.map{ |a,b|
a + "; " + b }

__END__
Williams. "The Long Saturday Night" poor Owner: Jack
Adams. "The Desperado" VG Owner: Bob (E. Cantankerous, Il.)
Adams. "Death's Sweet Song" good Owner: Bob
Williams. "Hell Hath No Fury" Fine Owner: Bob
Rogers. "The Red Right Hand" good Owner: Bob
Brean. "Wilders Walk Away" NF Owner: Bob
Thompson. "The Getaway" poor Owner: Bob
Gruber. "Swing Low, Swing Dead" Good Owner: Bob
Gruber. "The French Key" fair Owner: Bob
Gruber."The Limping Goose" good Owner: Bob
Williams. "The Long Saturday Night" fair Owner: Bob
Gruber. "The French Key" good Owner: Jack
Carr. "The Burning Court" good Owner: Jack (Fiendish, Ind.)

And the output is:

Adams; Death's Sweet Song
Adams; The Desperado
Brean; Wilders Walk Away
Gruber; The Limping Goose
Gruber; Swing Low, Swing Dead
Rogers; The Red Right Hand
Thompson; The Getaway
Williams; Hell Hath No Fury

Pascal Bourguignon

2005-11-08, 6:29 pm

"William James" <w_a_x_man@yahoo.com> writes:

> casioculture@gmail.com wrote:
>
>
>
> Here's a more complex problem. We have a list of books
> owned by Bob and Jack. Each line has author, title,
> condition, owner.
>
> Print the books owned by Bob but not by Jack.
> Sort them by author and title, ignoring "The " at the
> beginning of a title. We'll use Ruby.
>
>
> h = Hash.new( [] )
>
> DATA.each{ |line|
> author, title, owner =
> /^ ( .*? ) # author
> \. \s* "
> ( .*? ) # title
> " .*
> Owner: \s+
> ( .*? ) # Owner
> [ \t]*
> (?: \( | $ )
> /x.match( line ).captures
> h[ owner ] += [[ author, title ]]
> }
>
> puts ( h['Bob'] - h['Jack'] ).sort_by{ |x|
> [ x.first, x.last[ /^(?:The )?(.*)/, 1 ] ] }.map{ |a,b|
> a + "; " + b }
>
> __END__
> Williams. "The Long Saturday Night" poor Owner: Jack
> Adams. "The Desperado" VG Owner: Bob (E. Cantankerous, Il.)
> Adams. "Death's Sweet Song" good Owner: Bob
> Williams. "Hell Hath No Fury" Fine Owner: Bob
> Rogers. "The Red Right Hand" good Owner: Bob
> Brean. "Wilders Walk Away" NF Owner: Bob
> Thompson. "The Getaway" poor Owner: Bob
> Gruber. "Swing Low, Swing Dead" Good Owner: Bob
> Gruber. "The French Key" fair Owner: Bob
> Gruber."The Limping Goose" good Owner: Bob
> Williams. "The Long Saturday Night" fair Owner: Bob
> Gruber. "The French Key" good Owner: Jack
> Carr. "The Burning Court" good Owner: Jack (Fiendish, Ind.)
>
>
> And the output is:
>
> Adams; Death's Sweet Song
> Adams; The Desperado
> Brean; Wilders Walk Away
> Gruber; The Limping Goose
> Gruber; Swing Low, Swing Dead
> Rogers; The Red Right Hand
> Thompson; The Getaway
> Williams; Hell Hath No Fury

(defparameter data
"Williams. \"The Long Saturday Night\" poor Owner: Jack
Adams. \"The Desperado\" VG Owner: Bob (E. Cantankerous, Il.)
Adams. \"Death's Sweet Song\" good Owner: Bob
Williams. \"Hell Hath No Fury\" Fine Owner: Bob
Rogers. \"The Red Right Hand\" good Owner: Bob
Brean. \"Wilders Walk Away\" NF Owner: Bob
Thompson. \"The Getaway\" poor Owner: Bob
Gruber. \"Swing Low, Swing Dead\" Good Owner: Bob
Gruber. \"The French Key\" fair Owner: Bob
Gruber.\"The Limping Goose\" good Owner: Bob
Williams. \"The Long Saturday Night\" fair Owner: Bob
Gruber. \"The French Key\" good Owner: Jack
Carr. \"The Burning Court\" good Owner: Jack (Fiendish, Ind.)
")

;; Let's first define our "objects":

(defstruct book author title quality)
(defstruct owner name comment books)

;; Then let's parse the strange ad-hoc file format:

(defun read-books-and-owners (stream)
(loop
:with owners = '()
:with books = '()
:for line = (read-line stream nil nil)
:while line
:do (multiple-value-bind (all author title quality bowner comment-p comment)
(regexp:match "\$[^.]\\+\$. *\"\$[^\"]*\$\" *\$[A-Za-z]\\+\$ *Owner: *\$[A-Za-z]\\+\$ *\$(\\(.*\$)\\)\\?" line)
(unless all (error "Ill formated book line: ~S" line))
(let* ((owner-name (regexp:match-string line bowner))
(owner (find owner-name owners
:key (function owner-name)
:test (function string-equal)))
(book-key (list (regexp:match-string line author)
(regexp:match-string line title)))
(book (find book-key books
:key (lambda (book) (list (function book-author)
(function book-title)))
:test (function equalp))))
(unless book
(setf book (make-book
:author (first book-key)
:title (second book-key)
:quality (regexp:match-string line quality)))
(push book books))
(unless owner
(setf owner (make-owner :name owner-name))
(push owner owners))
(when comment-p
(setf (owner-comment owner) (regexp:match-string line comment)))
(push book (owner-books owner))))

:finally (return-from read-books-and-owners (values books owners))))

(defvar *books* nil)
(defvar *owners* nil)
(multiple-value-setq (*books* *owners*)
(with-input-from-string (stream data) (read-books-and-owners stream)))

;; Now, what's the question?

;; Print the books owned by Bob but not by Jack.
;; Sort them by author and title, ignoring "The " at the
;; beginning of a title.

(defun but-particle (string)
(flet ((remove-prefix (prefix string)
(when (and (< (length prefix) (length string))
(string-equal prefix string :end2 (length prefix)))
(subseq string (length prefix)))))
(cond
((remove-prefix "a " string))
((remove-prefix "an " string))
((remove-prefix "the " string))
(t string))))

(defparameter results
(sort (set-difference (owner-books (find "Bob" *owners*
:key (function owner-name)
:test (function equalp)))
(owner-books (find "Jack" *owners*
:key (function owner-name)
:test (function equalp)))
:key (lambda (book) (list (book-author book)
(book-title book)))
:test (function equalp))
(lambda (b1 b2)
(or (string-lessp (book-author b1) (book-author b2))
(and (string-equal (book-author b1) (book-author b2))
(string-lessp (but-particle (book-title b1))
(but-particle (book-title b2))))))))

results
-->
(#S(BOOK :AUTHOR "Adams" :TITLE "Death's Sweet Song" :QUALITY "good")
#S(BOOK :AUTHOR "Adams" :TITLE "The Desperado" :QUALITY "VG")
#S(BOOK :AUTHOR "Brean" :TITLE "Wilders Walk Away" :QUALITY "NF")
#S(BOOK :AUTHOR "Gruber" :TITLE "The Limping Goose" :QUALITY "good")
#S(BOOK :AUTHOR "Gruber" :TITLE "Swing Low, Swing Dead" :QUALITY "Good")
#S(BOOK :AUTHOR "Rogers" :TITLE "The Red Right Hand" :QUALITY "good")
#S(BOOK :AUTHOR "Thompson" :TITLE "The Getaway" :QUALITY "poor")
#S(BOOK :AUTHOR "Williams" :TITLE "Hell Hath No Fury" :QUALITY "Fine"))

;; Well, assuuming you want ad-hoc formating of the output:

(dolist (book results (values))
(format t "~A; ~A~%" (book-author book) (book-title book)))

Adams; Death's Sweet Song
Adams; The Desperado
Brean; Wilders Walk Away
Gruber; The Limping Goose
Gruber; Swing Low, Swing Dead
Rogers; The Red Right Hand
Thompson; The Getaway
Williams; Hell Hath No Fury

As I said, the advantages of a true programming languages appear when
you program in the large, not in small scripts. But small program
will grow big.

--
__Pascal Bourguignon__ http://www.informatimago.com/
Kitty like plastic.
Confuses for litter box.
Don't leave tarp around.

William James

2005-11-08, 6:29 pm

Pascal Bourguignon wrote:
> "William James" <w_a_x_man@yahoo.com> writes:
>
>
>
> (defparameter data
> "Williams. \"The Long Saturday Night\" poor Owner: Jack
> Adams. \"The Desperado\" VG Owner: Bob (E. Cantankerous, Il.)
> Adams. \"Death's Sweet Song\" good Owner: Bob
> Williams. \"Hell Hath No Fury\" Fine Owner: Bob
> Rogers. \"The Red Right Hand\" good Owner: Bob
> Brean. \"Wilders Walk Away\" NF Owner: Bob
> Thompson. \"The Getaway\" poor Owner: Bob
> Gruber. \"Swing Low, Swing Dead\" Good Owner: Bob
> Gruber. \"The French Key\" fair Owner: Bob
> Gruber.\"The Limping Goose\" good Owner: Bob
> Williams. \"The Long Saturday Night\" fair Owner: Bob
> Gruber. \"The French Key\" good Owner: Jack
> Carr. \"The Burning Court\" good Owner: Jack (Fiendish, Ind.)
> ")
>
> ;; Let's first define our "objects":
>
> (defstruct book author title quality)
> (defstruct owner name comment books)

Thanks for showing a Lisp solution.

>
>
> ;; Then let's parse the strange ad-hoc file format:

Strange? Welcome to the real world, Pascal. Extracting
this data is very easy compared to extracting the data from
many web pages that I have parsed.

>
> (defun read-books-and-owners (stream)
> (loop
> :with owners = '()
> :with books = '()
> :for line = (read-line stream nil nil)
> :while line
> :do (multiple-value-bind (all author title quality bowner comment-p comment)
> (regexp:match "\$[^.]\\+\$. *\"\$[^\"]*\$\" *\$[A-Za-z]\\+\$ *Owner: *\$[A-Za-z]\\+\$ *\$(\\(.*\$)\\)\\?" line)
> (unless all (error "Ill formated book line: ~S" line))
> (let* ((owner-name (regexp:match-string line bowner))
> (owner (find owner-name owners
> :key (function owner-name)
> :test (function string-equal)))
> (book-key (list (regexp:match-string line author)
> (regexp:match-string line title)))
> (book (find book-key books
> :key (lambda (book) (list (function book-author)
> (function book-title)))
> :test (function equalp))))
> (unless book
> (setf book (make-book
> :author (first book-key)
> :title (second book-key)
> :quality (regexp:match-string line quality)))
> (push book books))
> (unless owner
> (setf owner (make-owner :name owner-name))
> (push owner owners))
> (when comment-p
> (setf (owner-comment owner) (regexp:match-string line comment)))
> (push book (owner-books owner))))
>
> :finally (return-from read-books-and-owners (values books owners))))
>
> (defvar *books* nil)
> (defvar *owners* nil)
> (multiple-value-setq (*books* *owners*)
> (with-input-from-string (stream data) (read-books-and-owners stream)))
>
> ;; Now, what's the question?
>
> ;; Print the books owned by Bob but not by Jack.
> ;; Sort them by author and title, ignoring "The " at the
> ;; beginning of a title.
>
> (defun but-particle (string)
> (flet ((remove-prefix (prefix string)
> (when (and (< (length prefix) (length string))
> (string-equal prefix string :end2 (length prefix)))
> (subseq string (length prefix)))))
> (cond
> ((remove-prefix "a " string))
> ((remove-prefix "an " string))
> ((remove-prefix "the " string))
> (t string))))
>
> (defparameter results
> (sort (set-difference (owner-books (find "Bob" *owners*
> :key (function owner-name)
> :test (function equalp)))
> (owner-books (find "Jack" *owners*
> :key (function owner-name)
> :test (function equalp)))
> :key (lambda (book) (list (book-author book)
> (book-title book)))
> :test (function equalp))
> (lambda (b1 b2)
> (or (string-lessp (book-author b1) (book-author b2))
> (and (string-equal (book-author b1) (book-author b2))
> (string-lessp (but-particle (book-title b1))
> (but-particle (book-title b2))))))))
>
>

COBOL and Lisp were invented in the same era; apparently
extreme verbosity was highly valued at that time.

> results
> -->
> (#S(BOOK :AUTHOR "Adams" :TITLE "Death's Sweet Song" :QUALITY "good")
> #S(BOOK :AUTHOR "Adams" :TITLE "The Desperado" :QUALITY "VG")
> #S(BOOK :AUTHOR "Brean" :TITLE "Wilders Walk Away" :QUALITY "NF")
> #S(BOOK :AUTHOR "Gruber" :TITLE "The Limping Goose" :QUALITY "good")
> #S(BOOK :AUTHOR "Gruber" :TITLE "Swing Low, Swing Dead" :QUALITY "Good")
> #S(BOOK :AUTHOR "Rogers" :TITLE "The Red Right Hand" :QUALITY "good")
> #S(BOOK :AUTHOR "Thompson" :TITLE "The Getaway" :QUALITY "poor")
> #S(BOOK :AUTHOR "Williams" :TITLE "Hell Hath No Fury" :QUALITY "Fine"))
>
>
> ;; Well, assuuming you want ad-hoc formating of the output:
>
> (dolist (book results (values))
> (format t "~A; ~A~%" (book-author book) (book-title book)))
>
> Adams; Death's Sweet Song
> Adams; The Desperado
> Brean; Wilders Walk Away
> Gruber; The Limping Goose
> Gruber; Swing Low, Swing Dead
> Rogers; The Red Right Hand
> Thompson; The Getaway
> Williams; Hell Hath No Fury
>
>
>
> As I said, the advantages of a true programming languages appear when
> you program in the large, not in small scripts.

It seems that Lisp forces every program to be large.

> But small program
> will grow big.

Ruby (as in Ruby on Rails) is a true programming language
(it even has lambda). It handles large programming tasks easily,
and doesn't inflate small problems into big ones.

>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/
> Kitty like plastic.
> Confuses for litter box.
> Don't leave tarp around.

Pascal Bourguignon

2005-11-08, 6:29 pm

"William James" <w_a_x_man@yahoo.com> writes:
> Pascal Bourguignon wrote:
>
> It seems that Lisp forces every program to be large.

It may look like it for toy examples, but I assure you that everytime
I've convert normal sized programs either from C or Modula-2 to
Common-Lisp, I've experienced a ten to twelvefold reduction in size.

>
>
> Ruby (as in Ruby on Rails) is a true programming language
> (it even has lambda). It handles large programming tasks easily,
> and doesn't inflate small problems into big ones.

I appreciate the concision of Ruby.

But I note that it's more cryptic: there's more syntax to know to be
able to understand it.

Also, each language can handle more easily a given kind of data. Ruby
is designed to process easily lines of text files (as often found in
unix); Lisp is designed to process easily lists of symbols.

If the input data file had contained:

(("Williams" "The Long Saturday Night" poor "Jack") ("Adams"
"The Desperado" VG "Bob" "(E. Cantankerous, Il.)")
("Adams" "Death's Sweet Song" good "Bob") ("Williams"
"Hell Hath No Fury" Fine "Bob") ("Rogers"
"The Red Right Hand" good "Bob") ("Brean" "Wilders Walk Away" NF
"Bob") ("Thompson" "The Getaway" poor "Bob")
("Gruber" "Swing Low, Swing Dead" Good "Bob") ("Gruber"
"The French Key" fair "Bob") ("Gruber"
"The Limping Goose" good "Bob") ("Williams" "The Long Saturday Night"
fair "Bob") ("Gruber" "The French Key" good "Jack")
("Carr" "The Burning Court" good "Jack" "(Fiendish, Ind.)"))

I'd like to see ruby process it. On the other hand, in lisp you could
do it as easily as the first version in Ruby:

(defun but-particle (string)
(flet ((remove-prefix (prefix string)
(when (and (< (length prefix) (length string))
(string-equal prefix string :end2 (length prefix)))
(subseq string (length prefix)))))
(cond
((remove-prefix "a " string))
((remove-prefix "an " string))
((remove-prefix "the " string))
(t string))))

(defparameter input (with-open-file (in "data") (read in)))

(format t "~:{~A; ~A~%~}"
(sort
(set-difference
(remove "bob" input
:key (function fourth)
:test (complement (function string-equal)))
(remove "jack" input
:key (function fourth)
:test (complement (function string-equal)))
:key (lambda (line) (list (first line) (second line)))
:test (function equalp))
(lambda (b1 b2)
(or (string-lessp (first b1) (first b2))
(and (string-equal (first b1) (first b2))
(string-lessp (but-particle (second b1))
(but-particle (second b2))))))))

But when we talk about programming in the large, the adequation of the
input data with the internal data structures is less important because
it's only a small part of the processing to implement. Then other
features like readability, meta-programmation and maintainability are
more important and pay back. For maintainability and readability,
(BOOK-AUTHOR b) is much better than (FIRST x) or x.first.

And you cannot beat COBOL to write small programs processing fixed
format records and to print fixed format 132x66 reports.

--
__Pascal Bourguignon__ http://www.informatimago.com/
I need a new toy.
Tail of black dog keeps good time.
Pounce! Good dog! Good dog!

William James

2005-11-08, 6:29 pm

Pascal Bourguignon wrote:
> "William James" <w_a_x_man@yahoo.com> writes:
>
> It may look like it for toy examples, but I assure you that everytime
> I've convert normal sized programs either from C or Modula-2 to
> Common-Lisp, I've experienced a ten to twelvefold reduction in size.
>
>
>
> I appreciate the concision of Ruby.
>
> But I note that it's more cryptic: there's more syntax to know to be
> able to understand it.
>
> Also, each language can handle more easily a given kind of data. Ruby
> is designed to process easily lines of text files (as often found in
> unix); Lisp is designed to process easily lists of symbols.
>
>
> If the input data file had contained:
>
> (("Williams" "The Long Saturday Night" poor "Jack") ("Adams"
> "The Desperado" VG "Bob" "(E. Cantankerous, Il.)")
> ("Adams" "Death's Sweet Song" good "Bob") ("Williams"
> "Hell Hath No Fury" Fine "Bob") ("Rogers"
> "The Red Right Hand" good "Bob") ("Brean" "Wilders Walk Away" NF
> "Bob") ("Thompson" "The Getaway" poor "Bob")
> ("Gruber" "Swing Low, Swing Dead" Good "Bob") ("Gruber"
> "The French Key" fair "Bob") ("Gruber"
> "The Limping Goose" good "Bob") ("Williams" "The Long Saturday Night"
> fair "Bob") ("Gruber" "The French Key" good "Jack")
> ("Carr" "The Burning Court" good "Jack" "(Fiendish, Ind.)"))
>
> I'd like to see ruby process it. On the other hand, in lisp you could
> do it as easily as the first version in Ruby:
>
> (defun but-particle (string)
> (flet ((remove-prefix (prefix string)
> (when (and (< (length prefix) (length string))
> (string-equal prefix string :end2 (length prefix)))
> (subseq string (length prefix)))))
> (cond
> ((remove-prefix "a " string))
> ((remove-prefix "an " string))
> ((remove-prefix "the " string))
> (t string))))
>
> (defparameter input (with-open-file (in "data") (read in)))
>
> (format t "~:{~A; ~A~%~}"
> (sort
> (set-difference
> (remove "bob" input
> :key (function fourth)
> :test (complement (function string-equal)))
> (remove "jack" input
> :key (function fourth)
> :test (complement (function string-equal)))
> :key (lambda (line) (list (first line) (second line)))
> :test (function equalp))
> (lambda (b1 b2)
> (or (string-lessp (first b1) (first b2))
> (and (string-equal (first b1) (first b2))
> (string-lessp (but-particle (second b1))
> (but-particle (second b2))))))))
>
>
> But when we talk about programming in the large, the adequation of the
> input data with the internal data structures is less important because
> it's only a small part of the processing to implement. Then other
> features like readability, meta-programmation and maintainability are
> more important and pay back. For maintainability and readability,
> (BOOK-AUTHOR b) is much better than (FIRST x) or x.first.
>
>
> And you cannot beat COBOL to write small programs processing fixed
> format records and to print fixed format 132x66 reports.
>
> --
> __Pascal Bourguignon__ http://www.informatimago.com/

# We have a list of books
# owned by Bob and Jack. Each record has author, title,
# condition, owner.
#
# Print the books owned by Bob but not by Jack.
# Sort them by author and title, ignoring "The " at the
# beginning of a title. We'll use Ruby.

books = Hash.new( [] )

# No quote characters.
nq = /[^"]*/
# No parentheses or quotes.
npq = /[^()"]*/

DATA.read.chomp.sub( / \A $ | $ \Z /x, "").scan(
# Match one record.
%r{ $ #{ npq } (?: "#{ nq }" #{ npq } )* $ }xm
){ |record| author, title, owner =
%r{ "(#{ nq })" \s* "(#{ nq })" .*? "(#{ nq })" }xm\
.match( record ).captures
books[ owner ] += [[ author, title ]]
}

class Array; alias author first; alias title last; end

match_prefix = /^(?:The |An |A )?/

puts ( books['Bob'] - books['Jack'] ).sort_by{ |x|
[ x.author, x.title[ /#{ match_prefix }(.*)/, 1 ] ]
}.map{ |x| x.author + "; " + x.title }

__END__
(("Williams" "The Long Saturday Night" poor "Jack") ("Adams"
"The Desperado" VG "Bob" "(E. Cantankerous, Il.)")
("Adams" "Death's Sweet Song" good "Bob") ("Williams"
"Hell Hath No Fury" Fine "Bob") ("Rogers"
"The Red Right Hand" good "Bob") ("Brean" "Wilders Walk Away" NF
"Bob") ("Thompson" "The Getaway" poor "Bob")
("Gruber" "Swing Low, Swing Dead" Good "Bob") ("Gruber"
"The French Key" fair "Bob") ("Gruber"
"The Limping Goose" good "Bob") ("Williams" "The Long Saturday Night"
fair "Bob") ("Gruber" "The French Key" good "Jack")
("Carr" "The Burning Court" good "Jack" "(Fiendish, Ind.)"))

Pascal Bourguignon

2005-11-08, 6:29 pm

"William James" <w_a_x_man@yahoo.com> writes:
> # We have a list of books
> # owned by Bob and Jack. Each record has author, title,
> # condition, owner.
> #
> # Print the books owned by Bob but not by Jack.
> # Sort them by author and title, ignoring "The " at the
> # beginning of a title. We'll use Ruby.
>
>
> books = Hash.new( [] )
>
> # No quote characters.
> nq = /[^"]*/
> # No parentheses or quotes.
> npq = /[^()"]*/
>
> DATA.read.chomp.sub( / \A $ | $ \Z /x, "").scan(
> # Match one record.
> %r{ $ #{ npq } (?: "#{ nq }" #{ npq } )* $ }xm
> ){ |record| author, title, owner =
> %r{ "(#{ nq })" \s* "(#{ nq })" .*? "(#{ nq })" }xm\
> .match( record ).captures
> books[ owner ] += [[ author, title ]]
> }
>
> class Array; alias author first; alias title last; end
>
> match_prefix = /^(?:The |An |A )?/
>
> puts ( books['Bob'] - books['Jack'] ).sort_by{ |x|
> [ x.author, x.title[ /#{ match_prefix }(.*)/, 1 ] ]
> }.map{ |x| x.author + "; " + x.title }
>
>
> __END__
> (("Williams" "The Long Saturday Night" poor "Jack") ("Adams"
> "The Desperado" VG "Bob" "(E. Cantankerous, Il.)")
> ("Adams" "Death's Sweet Song" good "Bob") ("Williams"
> "Hell Hath No Fury" Fine "Bob") ("Rogers"
> "The Red Right Hand" good "Bob") ("Brean" "Wilders Walk Away" NF
> "Bob") ("Thompson" "The Getaway" poor "Bob")
> ("Gruber" "Swing Low, Swing Dead" Good "Bob") ("Gruber"
> "The French Key" fair "Bob") ("Gruber"
> "The Limping Goose" good "Bob") ("Williams" "The Long Saturday Night"
> fair "Bob") ("Gruber" "The French Key" good "Jack")
> ("Carr" "The Burning Court" good "Jack" "(Fiendish, Ind.)"))

Good, in an APLish way.

--
"Our users will know fear and cower before our software! Ship it!
Ship it and let them flee like the dogs they are!"

William James

2005-11-08, 6:29 pm

Pascal Bourguignon wrote:
> "William James" <w_a_x_man@yahoo.com> writes:
>
> Good, in an APLish way.

Earlier, you said APL was unreadable. I believe that many people
who know neither Lisp nor Ruby will find the Ruby version more
comprehensible. There is no doubt that Lisp is extremely
powerful, but the price for that power is that the language was
designed so that its programs could be easily parsed and
manipulated by computers, and not so that they could be readily
understood by humans.

Using any particular language is a compromise. To each his own.

Pascal Bourguignon

2005-11-08, 6:29 pm

"William James" <w_a_x_man@yahoo.com> writes:
>
> Earlier, you said APL was unreadable. I believe that many people
> who know neither Lisp nor Ruby will find the Ruby version more
> comprehensible. There is no doubt that Lisp is extremely
> powerful, but the price for that power is that the language was
> designed so that its programs could be easily parsed and
> manipulated by computers, and not so that they could be readily
> understood by humans.

I doubt it, because there's almost no syntax in lisp, the structure is
clear because of the explicit parentheses, and the meaning can be made
explicit with the use of good, fully-spelled names. On Ruby I observe
a lot of "syntax", special characters, and I suspect precedence rules,
the meaning of which is harder to infer.

> Using any particular language is a compromise. To each his own.

Indeed.

--
"Our users will know fear and cower before our software! Ship it!
Ship it and let them flee like the dogs they are!"

toby

2005-11-08, 6:29 pm

Giorgos Keramidas wrote:
> Russell Shaw <rjshawN_o@s_pam.netspace.net.au> writes:
>
> I'm sure you got the joke backwards and it's a bit too old to be
> funny anymore

So was your bogus PERL clich=E9 - sorry.

Pascal Bourguignon

2005-11-08, 6:29 pm

Russell Shaw <rjshawN_o@s_pam.netspace.net.au> writes:

> toby wrote:
>
> After using vim for a long time, i tried (x)emacs and learnt lisp a
> bit. I totally dislike waiting for an editor to start. It should be
> there in an instant as soon as <CR> is hit. Also hate relying on an
> editor that brings in 50MB to the system (debian).
> Vim configuration and script syntax is also ugly as hell and i hate
> it too, but i'm only using it because i've got a half useable .vimrc
> file worked out, and it's fast and does regexps easily. I'll get a
> multi-windowing vi working one day that is far better;)

Why? You launch emacs once every reboot. My emacs uptime ~= unix uptime.
And if you want to edit a file from a terminal, you use emacsclient:
alias ec='emacsclient -c --no-wait'

You can get any faster than that: the program is always already launched!

--
__Pascal Bourguignon__ http://www.informatimago.com/

There is no worse tyranny than to force a man to pay for what he does not
want merely because you think it would be good for him. -- Robert Heinlein

Pascal Bourguignon

2005-11-09, 2:49 am

Russell Shaw <rjshawN_o@s_pam.netspace.net.au> writes:
>
> Maybe so, but i don't like such a heavy weight framework just for an editor,
> even if it does do all kinds of other stuff.
>
> Anyway, i don't see why you have to wait for *any* program to start.
>
> There's no reason why mozilla, open-office, emacs, or anything else shouldn't
> start in an instant and be blazingly fast on old hardware. It all comes
> down to the crappiness of the implementation; a lot of that can be due
> to the slow and bloated crappiness of the current graphical widget systems.

Really, the "bloat" is not very big:

$ ls -l /usr/bin/perl /usr/bin/emacs /bin/vim /usr/apps/mozilla/mozilla-bin \
/usr/bin/mutt /usr/bin/irc-4.4
-rwxr-xr-x 1 root 1020164 2003-04-18 10:36 /bin/vim*
-rwxr-xr-x 1 root 4166640 2003-03-28 05:00 /usr/bin/emacs*
-rwxr-xr-x 1 root 563612 2004-02-12 09:46 /usr/bin/mutt*
-rwxr-xr-x 2 root 1257284 2003-03-13 23:33 /usr/bin/perl*
-rwxr-xr-x 1 pjb 1773582 2004-07-15 22:05 /usr/apps/mozilla/mozilla-bin*
-rwxr-xr-x 1 pjb 1365939 2003-12-14 00:41 /usr/bin/irc-4.4*

What's the difference between 1M and 4M when you've got gigabytes of
RAM? Moreover, what's the difference when you amortize it over the
multiple use? If you need to launch vim, mozilla, perl, mutt, irc,
etc, you really fast get to use much more memory than with emacs which
can do everything at once.

--
__Pascal Bourguignon__ http://www.informatimago.com/
Grace personified,
I leap into the window.
I meant to do that.