|
Home > Archive > dBASE Programming > November 2007 > parsing a text file
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
parsing a text file
|
|
| Tom Rathje 2007-11-16, 1:30 pm |
| I am hoping someone can help me with some ideas on how to code for the following:
I have a text file that is an output from a statistical analysis program that contains a block of information I want to extract into a table. The text file contains numerous items, but the block I need begins just after the abbreviation 'ID' and a blank
line appears in the file. The data is then in 4 fixed width columns and continues without any blank lines until the block of data is finished, after which there is another blank line.
My question is, is there a way to code a program in dbase to read this .txt file and extract only the desired block of data, placing it into a predefined table structure that would be set up to match the column widths for the data points?
Any help to this relatively inexperienced user would be appreciated!
TR
| |
| Ken Mayer [dBVIPS] 2007-11-16, 1:30 pm |
| Tom Rathje wrote:
> I am hoping someone can help me with some ideas on how to code for the following:
>
> I have a text file that is an output from a statistical analysis program that contains a block of information I want to extract into a table. The text file contains numerous items, but the block I need begins just after the abbreviation 'ID' and a blan
k line appears in the file. The data is then in 4 fixed width columns and continues without any blank lines until the block of data is finished, after which there is another blank line.
>
> My question is, is there a way to code a program in dbase to read this .txt file and extract only the desired block of data, placing it into a predefined table structure that would be set up to match the column widths for the data points?
>
> Any help to this relatively inexperienced user would be appreciated!
Use the file object. Yes, you can do this sort of thing. It takes a bit
of work, but once you have it working, you'll be surprised at just how
fast it is.
For starters (off the top of my head, no guarantees):
fIn = new file()
fIn.open( "TextFilename.txt", "R" )
// read-only flag so you don't hose anything in the file
bID_Found = false
do while not fIn.eof() // end of file
cLine = fIn.gets() // read to end of line, if text has standard
// line breaks
if "ID" $ cLine
// you have found the line with the "ID" characters you
// wanted
bID_Found := true
exit // out of loop
endif
enddo
// you know there's a blank:
cLine = fIn.gets()
// start parsing the data:
do while not fIn.eof()
cLine = fIn.gets() // need to parse this line
// use left(), right(), substr(), etc. to break this down
// into the data you need, add code to put it into the
// table ...
enddo
fIn.close() // close it
release object fIn
fIn := null // just good cleanup ...
Something along those lines should get you going.
Ken
--
/(Opinions expressed are purely my own, not those of dataBased
Intelligence, Inc.)/
*Ken Mayer* [dBVIPS]
/Golden Stag Productions/
dBASE at goldenstag dot net
http://www.goldenstag.net/GSP
http://www.goldenstag.net/dbase/dBASEBooks.htm
http://www.goldenstag.net/dbase
| |
| Mervyn Bick 2007-11-16, 1:30 pm |
| On Sat, 03 Nov 2007 00:21:10 +0200, Ken Mayer [dBVIPS] =
<dbase@_nospam_goldenstag.net> wrote:
bID_found is not used in the example code so it could have be omitted. =
It =
would, however, be a good idea to use it to decide whether to go ahead =
with the extraction of the data or to show an error message if necessary=
..
> // you know there's a blank:
> cLine =3D fIn.gets()
> // start parsing the data:
> do while not fIn.eof()
> cLine =3D fIn.gets() // need to parse this line
> // use left(), right(), substr(), etc. to break this down
> // into the data you need, add code to put it into the
> // table ...
> enddo
> fIn.close() // close it
> release object fIn
> fIn :=3D null // just good cleanup ...
Unless the blank line that denotes the end of the data block is, in fact=
, =
the last line in the file then one needs to test for the blank line and =
=
then finish up when it is found. There is more than one way to do this =
=
(the Bick way and the better way <g> ) but I would do the following.
if bID_found
// you know there's a blank:
cLine =3D fIn.gets()
// move to the first line of data
cLine =3D fIn.gets()
// start parsing the data:
do while not cLine =3D " "
// use left(), right(), substr(), etc. to break this down
// into the data you need, add code to put it into the
// table ...
cLine =3D fIn.gets() // move to the next line of data
enddo
else
msgBox("Oops, data block not found.")
endif
fIn.close() // close it
release object fIn
fIn :=3D null // just good cleanup ...
With only 4 items of data per line it is easy enough to hard code the =
extraction process but BreakString() in stringex.cc is there if one =
doesn't want to reinvent the wheel.
Mervyn.
| |
| Ivar B. Jessen 2007-11-16, 1:30 pm |
| On Sat, 03 Nov 2007 08:54:15 +0200, in dbase.programming,
Subject: Re: parsing a text file,
Message-ID: <op.t07eqbgf380wm4@localhost>,
"Mervyn Bick" <invalid@invalid.invalid> wrote:
>Unless the blank line that denotes the end of the data block is, in fact,
>the last line in the file then one needs to test for the blank line and
>then finish up when it is found. There is more than one way to do this
>(the Bick way and the better way <g> ) but I would do the following.
>
Mervyn.
Here is a different way, no guarantee that it is better, but it doesn't require parsing anything and is thus easier for
the lazy :-)
1) Open the file and read until the ID and the first blank line is found.
2) Create an output file <MyFile.sdf>, read each line of the blok of text and write the lines
to the output file until second blank line is found.
3) Close input and output files, use destination table and APPEND FROM <MyFile.sdf> TYPE sdf.
All untested.
Ivar B. Jessen
| |
| Ken Mayer [dBVIPS] 2007-11-16, 1:30 pm |
| Mervyn Bick wrote:
> On Sat, 03 Nov 2007 00:21:10 +0200, Ken Mayer [dBVIPS]
> <dbase@_nospam_goldenstag.net> wrote:
>
>
> bID_found is not used in the example code so it could have be omitted.
> It would, however, be a good idea to use it to decide whether to go
> ahead with the extraction of the data or to show an error message if
> necessary.
Well, it was intended to be used with an error trap that said that it
wasn't found, but I was making the code up off the top of my head ...
Ken
--
/(Opinions expressed are purely my own, not those of dataBased
Intelligence, Inc.)/
*Ken Mayer* [dBVIPS]
/Golden Stag Productions/
dBASE at goldenstag dot net
http://www.goldenstag.net/GSP
http://www.goldenstag.net/dbase/dBASEBooks.htm
http://www.goldenstag.net/dbase
| |
| Mervyn Bick 2007-11-16, 1:30 pm |
| On Sat, 03 Nov 2007 10:43:57 +0200, Ivar B. Jessen
<bergishagen@it.notthis.dk> wrote:
> Here is a different way, no guarantee that it is better, but it doesn't
> require parsing anything and is thus easier for
> the lazy :-)
>
> 1) Open the file and read until the ID and the first blank line is found.
> 2) Create an output file <MyFile.sdf>, read each line of the blok of
> text and write the lines
> to the output file until second blank line is found.
> 3) Close input and output files, use destination table and APPEND FROM
> <MyFile.sdf> TYPE sdf.
>
> All untested.
Come, come, Ivar. You're slipping. In the past your way has always been
better than my way - no question. And here you are having doubts. <g>
Mind you, I like the lateral thinking.
Mervyn.
| |
| Ivar B. Jessen 2007-11-16, 1:30 pm |
| On Sat, 03 Nov 2007 17:36:38 +0200, in dbase.programming,
Subject: Re: parsing a text file,
Message-ID: <op.t072zcc5380wm4@mervyn1>,
"Mervyn Bick" <invalid@invalid.invalid> wrote:
>On Sat, 03 Nov 2007 10:43:57 +0200, Ivar B. Jessen
><bergishagen@it.notthis.dk> wrote:
>
>
>Come, come, Ivar. You're slipping. In the past your way has always been
>better than my way - no question. And here you are having doubts. <g>
>Mind you, I like the lateral thinking.
>
Mervyn,
Lateral thinking. This dates you, wasn't that expression popular in management circles about 25 years ago?
Ivar B. Jessen
| |
| Mervyn Bick 2007-11-16, 1:30 pm |
| On Sat, 03 Nov 2007 21:10:41 +0200, Ivar B. Jessen
<bergishagen@it.notthis.dk> wrote:
> Lateral thinking. This dates you, wasn't that expression popular in
> management circles about 25 years ago?
It does, doesn't it. The concept was formalised about 40 years ago but it
certainly was still in vogue when I got to management level about 20 years
ago.
Mervyn.
| |
| Geoff Wass [dBVIPS] 2007-11-16, 1:30 pm |
| In article <8jgpi35ftpiudabjefu36g8fqj668j3r97@4ax.com>,
bergishagen@it.notthis.dk says...
> On Sat, 03 Nov 2007 17:36:38 +0200, in dbase.programming,
> Subject: Re: parsing a text file,
> Message-ID: <op.t072zcc5380wm4@mervyn1>,
> "Mervyn Bick" <invalid@invalid.invalid> wrote:
>
>
> Mervyn,
>
> Lateral thinking. This dates you, wasn't that expression popular in management circles about 25 years ago?
>
>
> Ivar B. Jessen
Ivar,
By the same token, it would also date you.
What is this lateral thinking anyway? It would appear to be before my
time.
--
Geoff Wass [the young dBVIPS]
Montréal, Québec, Canada
..|.|.| dBASE info at http://geocities.com/geoff_wass |.|.|.
..|.|.| ---------------------------------------------------------- |.|.|.
..|.|.| IT Consultant http://Geoff_Wass.com |.|.|.
| |
| Ivar B. Jessen 2007-11-16, 1:30 pm |
| On Sun, 4 Nov 2007 01:53:28 -0400, in dbase.programming,
Subject: Re: parsing a text file,
Message-ID: <MPG.219728632ca7377d989c97@news.dbase.com>,
Geoff Wass [dBVIPS] <gswassREMOVE_ME@attglobal.net> wrote:
>
>
>
>Ivar,
>
>By the same token, it would also date you.
>
>What is this lateral thinking anyway? It would appear to be before my
>time.
Hmm, young people don't know what lateral thinking is, sometimes one doubts if they know what thinking is <g>
Ivar B. Jessen
| |
| Mervyn Bick 2007-11-16, 1:30 pm |
| On Sun, 04 Nov 2007 07:53:28 +0200, Geoff Wass [dBVIPS]
<gswassREMOVE_ME@attglobal.net> wrote:
> What is this lateral thinking anyway? It would appear to be before my
> time.
>
The term was coined by Edward de Bono. Have a look at his website.
http://www.edwdebono.com/debono/lateral.htm
Mervyn.
| |
| Geoff Wass [dBVIPS] 2007-11-16, 1:30 pm |
| In article <vhpqi3pvldrrln192dsb90eofd1omc0lst@4ax.com>,
bergishagen@it.notthis.dk says...
> On Sun, 4 Nov 2007 01:53:28 -0400, in dbase.programming,
> Subject: Re: parsing a text file,
> Message-ID: <MPG.219728632ca7377d989c97@news.dbase.com>,
> Geoff Wass [dBVIPS] <gswassREMOVE_ME@attglobal.net> wrote:
>
>
> Hmm, young people don't know what lateral thinking is, sometimes one doubts if they know what thinking is <g>
>
>
> Ivar B. Jessen
Ivar,
Think?! Who has time for that? We young people have other priorities!
--
Geoff Wass [dBVIPS]
Montréal, Québec, Canada
..|.|.| dBASE info at http://geocities.com/geoff_wass |.|.|.
..|.|.| ---------------------------------------------------------- |.|.|.
..|.|.| IT Consultant http://Geoff_Wass.com |.|.|.
| |
| Geoff Wass [dBVIPS] 2007-11-16, 1:30 pm |
| In article <op.t09aev1z380wm4@mervyn1>, invalid@invalid.invalid says...
> On Sun, 04 Nov 2007 07:53:28 +0200, Geoff Wass [dBVIPS]
> <gswassREMOVE_ME@attglobal.net> wrote:
>
>
>
> The term was coined by Edward de Bono. Have a look at his website.
>
> http://www.edwdebono.com/debono/lateral.htm
>
> Mervyn.
>
Mervyn,
Very interesting site. All that stuff was invented before the iPod.
Impressive!
--
Geoff Wass [dBVIPS]
Montréal, Québec, Canada
..|.|.| dBASE info at http://geocities.com/geoff_wass |.|.|.
..|.|.| ---------------------------------------------------------- |.|.|.
..|.|.| IT Consultant http://Geoff_Wass.com |.|.|.
| |
| Ivar B. Jessen 2007-11-16, 1:30 pm |
| On Mon, 5 Nov 2007 01:06:33 -0500, in dbase.programming,
Subject: Re: parsing a text file,
Message-ID: <MPG.21987cf06494dd8b989ca1@news.dbase.com>,
Geoff Wass [dBVIPS] <gswassREMOVE_ME@attglobal.net> wrote:
>
>
>Ivar,
>
>Think?! Who has time for that? We young people have other priorities!
LOL
Ivar B. Jessen
| |
| Ahmed Talib 2007-11-16, 1:30 pm |
| Tom Rathje wrote:
===========message begin=======
I have a text file that is an output from a statistical analysis program
that
contains a block of information I want to extract into a table. The text
file
contains numerous items, but the block I need begins just after the
abbreviation 'ID' and a blank line appears in the file. The data is then in
4 fixed width columns and continues without any blank lines until the block
of data is finished, after which there is another blank line.
==============message end=======
I have had such a problem before and it was solved.
Can you send a copy of the text file and the structure of your dbf file
across?
One might be able to do something about it.
Ahmed Talib
|
|
|
|
|