|
Home > Archive > Database Theory > September 2004 > First Impressions on Using Alphora's Dataphor
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
First Impressions on Using Alphora's Dataphor
|
|
| Josh Hewitt 2004-08-27, 5:45 pm |
| I have recently downloaded Alphora's Dataphor product and begun
experimenting with it. I tried to write a simple (non-GUI), toy
application based on the principles put forward in the The Third
Manifesto (TTM) and also in the articles published on the Database
Debunkings [http://www.dbdebunk.com] site. I defined scalar types and
scalar operators and I also spent some time designing a normalized
database schema along with integrity constraints. I did not use any
external storage device (that is to say, a SQL database back-end). I
only experimented with the 'in-process' server. (Although I plan to
connect to an Oracle instance later to see how it performs.)
Good things
- Expressing relational algebraic (relalg) queries in D4 was much easier
than it used to be in SQL. The resulting queries were usually clearer,
significantly more compact, and, of course, were seamlessly integrated
with the rest of the language. It is important to mention that several
queries that would have been too cumbersome to bother with in SQL
became expressible in D4, quite often easily expressible. I think this
is mainly due to the orthogonal design of the language which allows
arbitrary nesting of relalg expressions. Needless to say that with D4
being truly relational, or, in other words, with D4 removing duplicates
at each point of the evaluation, one can rest assured that the results
will be relations in the mathematical sense.
- The ability to declaratively define arbitrary constraints in Dataphor's
D4 language was truly impressive. Knowing that the DBMS would enforce
the constraints at all times gave great confidence that the database
could not be modified in an inconsistent way. As it turned out later,
this helped in finding subtle logic errors in the algorithms that tried
to update the database.
Nuisances
- Since the DBMS now can be extended with user defined types (UDTs) along
with their operators it follows that types and operators become
database objects themselves. This means that they must be 'CREATE'-ed,
later, possibly, 'ALTER'-ed, and eventually, 'DROP'-ed, just like
tables (I use the Dataphor terminology here), views, and constraints.
In addition, the strongly-typed nature of Dataphor means that each
database object, in any reasonably large system, will have several
dependencies and several dependents. Types can depend on other types
and operators for their definition; operators depend on types and can
depend on tables and views; tables and views depend on types and
operators; and finally, constraints can depend on everything mentioned
so far. The end result is a complex web of dependencies between
database objects where you cannot simply re-define anything without
re-defining its dependents at the same time. Simply put, the
create/drop cycle became the most annoying part of the development.
For example, when I realized that I have to change an operator I had to
drop all the constraints and operators that used the one being changed.
The situation was even uglier when I realized that I had to change one
of the scalar types. In this case I practically had to drop every
database object and then re-create them again.
(I will further elaborate on this point and its implications a bit more
at the end of the post.)
- Although it is logical after giving it a little thought, it came as a
nasty surprise that Dataphor demands that all scalar types that might
appear in a context where duplicate elimination is required must have
the less-than (Dataphor calls it 'iLess') operator defined on them.
(Probably any of the operators that provide ordering would do: <, >,
>=, or <=). It was a nasty surprise because, at first, I could not
figure out why Dataphor rejected an otherwise perfectly formulated
query, protesting that 'operator iLess' was not defined for one of the
scalar types. The query did not use the 'ORDER BY' clause and did not
even refer to any attributes of the scalar type mentioned in the error
message. As it turned out, some relalg subexpression of the original
query contained a projection that required duplicate elimination during
evaluation and an attribute of the mentioned scalar type was part of
this relation. (The scalar type was describing the operating system of
a node in a network.) I understand that without ordering defined on a
scalar type it is very inefficient to remove duplicates but I did not
feel like defining an arbitrary comparison operator just for this
purpose. I ended up 'projecting away' the troublesome attribute since
I did not need it in the final result anyway. This solved my problem
but it showed that physical implementation does leak through the
logical model. Maybe Dataphor should provide some kind of internally
implemented default ordering for the purposes of duplicate elimination.
Also, I must add that if the resulting relation contains a key
attribute Dataphor is clever enough to recognize this in order to avoid
duplicate elimination so the problem described here is not a really
serious one.
- The help system is a little bit out of sync with what is implemented at
points and I found the number of examples provided too few. This is
especially true of the D4 Language Guide. The sample applications
bundled with the evaluation version do not really show the capabilities
of the system either. In my opinion, developers might benefit more
from a "feature fest" type demo application.
Musings
How can one do software maintenance in a live Dataphor installation?
Since scalar types form the building blocks of the database design,
almost every database object will depend on them, either directly or
indirectly. Relvars and operators are all defined using types while
constraints are defined in terms of expressions involving relvars and
operators. The consequence is that every scalar type will have several
dependencies and several dependents. How can one fix a bug that requires
more than cosmetic changes in this setup? You cannot just drop the type
since everything depends on it. Also, what are you going to do with the
data in your relvars? You cannot just drop your production tables.
What are you going to do with older applications that depend on (the now
obsolete) old types/operators/tables/views. Some suggested creating
views for the old applications based on the new tables. I do not think
that this approach is viable in the long run. It would mean duplicating
the affected parts every time there is a bug fix. Also, the added
complexity of maintaining 'parallel universes' for old and new programs
is not something that one would want.
Now I digress a little bit so the following musings are about UDTs in
RDBMSs in general. Please do understand that I am in full support for
truly relational systems and I pose the following question in order to
provoke a healthy debate that might focus attention to areas neglected by
relational theory but which are still important in practice:
What if major vendors 'do get it' but enterprise environments simply
'squeeze out' anything but atomic types (numbers, character strings,
booleans, etc.) from the database?
Quite often, in enterprise environments, development teams rarely have
access and/or control over applications written by other teams. Given
the tangled web of dependencies in a 'UDT-enabled' RDBMS, achieving
success suddenly becomes a factor of how well-designed those UDTs are in
the first place, and if they need modification, who has the right (and
the resources) to perform those modifications. I think this is the make
or break question of the real life success of truly relational databases
right along with the questions of performance (which I do not want to
address right now.) Most development projects try to have control over
as many factors of their success as possible. (That is why everybody is
building several layers of abstractions in order to 'insulate' themselves
from the ever-changing environment.) Average workplaces have average
developers on average. This means that it is rarely the case that
developers can work on a database that is well-designed. Even in
database design there are a lot of ways to screw up and I do not think
that designing UDTs is easier business in any way. So in a UDT-rich
world developers would be forced to work with someone else's not-so-well
designed UDTs most of the time. Not to mention that fixing a badly
designed mess can be very time-consuming, especially if it is not the job
one were hired to do in the first place. Even fixing serious database
design flows in today's UDT-free databases often meets with severe
resistance because of the (not completely unsubstantiated) fear of
breaking other applications. Is it not better then to forget about UDTs
in the database and only allow them in programming languages where
developers have complete control over them? Is it not the case that
the lack of UDTs in today's databases is more of a blessing then a curse?
Sorry for the long post, but I felt that this might be an interesting
topic to discuss in this news group.
Josh Hewitt
Ph.D. Student
Department of Computer Sciences
Florida Institute of Technology
| |
| Laconic2 2004-08-27, 5:45 pm |
|
"Josh Hewitt" <lajos.nagy@gmail.com> wrote in message
news:1c92edeb.0408260910.60242a59@posting.google.com...
> - Although it is logical after giving it a little thought, it came as a
> nasty surprise that Dataphor demands that all scalar types that might
> appear in a context where duplicate elimination is required must have
> the less-than (Dataphor calls it 'iLess') operator defined on them.
Yes, it is logical.
>
>
> Now I digress a little bit so the following musings are about UDTs in
> RDBMSs in general.
What is the difference, if any, between UDTs and the domains generated by
"CREATE DOMAIN" in some RDBMs
and, I think, now included in the SQL standard?
> Sorry for the long post, but I felt that this might be an interesting
> topic to discuss in this news group.
>
>
> Josh Hewitt
I once knew a Carl Hewitt, at MIT. Are you related?
| |
| mAsterdam 2004-08-27, 5:45 pm |
| Josh Hewitt wrote:
> I have recently downloaded Alphora's Dataphor product and begun
> experimenting with it.
[...]
> Sorry for the long post, but I felt that this might be an interesting
> topic to discuss in this news group.
Thank you for sharing your experience.
| |
| mAsterdam 2004-08-27, 5:45 pm |
| Laconic2 wrote:
> "Josh Hewitt" <lajos.nagy@gmail.com> wrote in message
> news:1c92edeb.0408260910.60242a59@posting.google.com...
>
>
> Yes, it is logical.
It makes some intuitive sense to me. Would you care to elaborate?
I would appreciate that, and I suspect more readers would.
| |
| mAsterdam 2004-08-27, 5:45 pm |
| Josh Hewitt wrote:
> I have recently downloaded Alphora's Dataphor product and begun
> experimenting with it. I tried to write a simple (non-GUI), toy
> application based on the principles put forward in the The Third
> Manifesto (TTM) and also in the articles published on the Database
> Debunkings [http://www.dbdebunk.com] site. I defined scalar types and
> scalar operators and I also spent some time designing a normalized
> database schema along with integrity constraints.
>[...]
> Nuisances
>
> - Since the DBMS now can be extended with user defined types (UDTs) along
> with their operators it follows that types and operators become
> database objects themselves. This means that they must be 'CREATE'-ed,
> later, possibly, 'ALTER'-ed, and eventually, 'DROP'-ed, just like
> tables (I use the Dataphor terminology here), views, and constraints.
Types are really different (from things like tables, columns,
triggers, procedures), aren't they? Once the exclusive realm of
compiler designers, later roll-your-own classes. Neither approach
does the job for datamanagement. Somehow their significance extends
beyonds one shop - much more so than e.g. tables.
> In addition, the strongly-typed nature of Dataphor means that each
> database object, in any reasonably large system, will have several
> dependencies and several dependents.
I don't understand what it is you 'ld want to see/have instead.
> Types can depend on other types
> and operators for their definition; operators depend on types and can
> depend on tables and views; tables and views depend on types and
> operators; and finally, constraints can depend on everything mentioned
> so far. The end result is a complex web of dependencies between
> database objects where you cannot simply re-define anything without
> re-defining its dependents at the same time. Simply put, the
> create/drop cycle became the most annoying part of the development.
Uncharted territory does that to you.
> For example, when I realized that I have to change an operator I had to
> drop all the constraints and operators that used the one being changed.
> The situation was even uglier when I realized that I had to change one
> of the scalar types. In this case I practically had to drop every
> database object and then re-create them again.
Types are very basic to your operation, in an (as yet) elusive way.
Maybe it's like "schema is to development what
type-definition-system (one beer for a better name)
is to schema" - but that is just a guess.
>[...]
> Musings
>
> How can one do software maintenance in a live Dataphor installation?
>
> Since scalar types form the building blocks of the database design,
> almost every database object will depend on them, either directly or
> indirectly.
They always did (depend on them). However, now you can play
with them because of the gained tecnical freedom to find out
it is somehow bounded by other phenomena. Which? (I am not
asking you to solve, just to ponder :-)
> Relvars and operators are all defined using types while
> constraints are defined in terms of expressions involving relvars and
> operators. The consequence is that every scalar type will have several
> dependencies and several dependents. How can one fix a bug that requires
> more than cosmetic changes in this setup? You cannot just drop the type
> since everything depends on it. Also, what are you going to do with the
> data in your relvars? You cannot just drop your production tables.
> What are you going to do with older applications that depend on (the now
> obsolete) old types/operators/tables/views. Some suggested creating
> views for the old applications based on the new tables. I do not think
> that this approach is viable in the long run. It would mean duplicating
> the affected parts every time there is a bug fix. Also, the added
> complexity of maintaining 'parallel universes' for old and new programs
> is not something that one would want.
A type bug is a serious bug indeed.
> Now I digress a little bit so the following musings are about UDTs in
> RDBMSs in general. Please do understand that I am in full support for
> truly relational systems and I pose the following question in order to
> provoke a healthy debate that might focus attention to areas neglected by
> relational theory but which are still important in practice:
>
> What if major vendors 'do get it' but enterprise environments simply
> 'squeeze out' anything but atomic types (numbers, character strings,
> booleans, etc.) from the database?
>
> Quite often, in enterprise environments, development teams rarely have
> access and/or control over applications written by other teams. Given
> the tangled web of dependencies in a 'UDT-enabled' RDBMS, achieving
> success suddenly becomes a factor of how well-designed those UDTs are in
> the first place, and if they need modification, who has the right (and
> the resources) to perform those modifications. I think this is the make
> or break question of the real life success of truly relational databases
> right along with the questions of performance (which I do not want to
> address right now.) Most development projects try to have control over
> as many factors of their success as possible. (That is why everybody is
> building several layers of abstractions in order to 'insulate' themselves
> from the ever-changing environment.) Average workplaces have average
> developers on average. This means that it is rarely the case that
> developers can work on a database that is well-designed. Even in
> database design there are a lot of ways to screw up and I do not think
> that designing UDTs is easier business in any way. So in a UDT-rich
> world developers would be forced to work with someone else's not-so-well
> designed UDTs most of the time. Not to mention that fixing a badly
> designed mess can be very time-consuming, especially if it is not the job
> one were hired to do in the first place. Even fixing serious database
> design flows in today's UDT-free databases often meets with severe
> resistance because of the (not completely unsubstantiated) fear of
> breaking other applications. Is it not better then to forget about UDTs
> in the database and only allow them in programming languages where
> developers have complete control over them? Is it not the case that
> the lack of UDTs in today's databases is more of a blessing then a curse?
Nice thought. I am sorry I don't feel up to go into this right now.
| |
| Gene Wirchenko 2004-08-27, 5:45 pm |
| mAsterdam <mAsterdam@vrijdag.org> wrote:
>Josh Hewitt wrote:
>
>
>[...]
>
You are apologising for posting some of the most interesting
material I have seen in this group?
[vbcol=seagreen]
>Thank you for sharing your experience.
Seconded.
I have heard of Dataphor, but I really do not know enough to dig
in. I would be pleased to read more about it.
Sincerely,
Gene Wirchenko
Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.
| |
| Christopher Browne 2004-08-27, 5:45 pm |
| In the last exciting episode, lajos.nagy@gmail.com (Josh Hewitt) wrote:
> Sorry for the long post, but I felt that this might be an
> interesting topic to discuss in this news group.
No, that's a very interesting set of observations.
The challenges involved in the transformations of domains as you
experiment with them is interesting indeed.
I'm seeing, in this, a tad of need for some dynamic of type
transformations. Two things leap to mind as opposite ends of the
spectrum:
- The CLOS notion of defined "classes," which actually provide a way
to transform instances into the new form should you happen to
redefine them.
<http://www.ai.mit.edu/projects/iiip...ined-class.html>
That's object-oriented, not relational, but it sure points to a way
to update "tuples" on demand...
- The ML notion of polymorphism, where types can be left abstract
somewhat until the last moment; you can define functions that are
strongly typed, but which do not care what they look like 'til the
moment that you actually call the function.
I'm not sure either is the ultimate answer, and it would _definitely_
be an inconvenience to make a little domain change "over here" that
leads to 200GB worth of tuples having to be revised "over there."
--
select 'cbbrowne' || '@' || 'acm.org';
http://www.ntlug.org/~cbbrowne/spreadsheets.html
"We defeated the enemy with teamwork and the hammer of not bickering."
-- The Shoveller, Mystery Men
| |
| Jonathan Leffler 2004-08-27, 5:45 pm |
| Laconic2 wrote:
> What is the difference, if any, between UDTs and the domains
> generated by "CREATE DOMAIN" in some RDBMs and, I think, now
> included in the SQL standard?
The CREATE DOMAIN in standard SQL (well, SQL-92, but these things
usually don't change much) is a pitiful, neutered non-entity, at least
if C J Date is to be believed (and, from what little I know of CREATE
DOMAIN, I believe him).
To a crude approximation, the difference is between a full type and a
sort of shorthand for a built-in type plus possibly a check constraint
which can be dropped even when columns that were defined using the
domain name still exist in the database.
--
Jonathan Leffler #include <disclaimer.h>
Email: jleffler@earthlink.net, jleffler@us.ibm.com
Guardian of DBD::Informix v2003.04 -- http://dbi.perl.org/
| |
| Laconic2 2004-08-27, 5:45 pm |
|
"mAsterdam" <mAsterdam@vrijdag.org> wrote in message
news:412e5c87$0$65124$e4fe514c@news.xs4all.nl...
> It makes some intuitive sense to me. Would you care to elaborate?
> I would appreciate that, and I suspect more readers would.
>
Well, I'm not sure, but here goes.
It seems to me that, if you have a feature that has identity, but not order,
then eliminating duplicates involves
comparisons on the order of (n*n/2). If you have a feature that has
identity and order, then eliminating duplicates
can be done with comparisons on the order of (n*log(n)). The latter is the
order of magnitude of the comparisons needed to sort.
For large n, sorting is clearly preferable to comparing all pairs.
This is really just conjecture, but it sounds good to me.
| |
| Leandro Guimaraens Faria Corsetti Dutra 2004-08-27, 5:45 pm |
| Em Thu, 26 Aug 2004 10:10:56 -0700, Josh Hewitt escreveu:
> Is it not better then to forget about UDTs in the database and only
> allow them in programming languages where developers have complete
> control over them?
Definetely not. There would lie the path to madness.
Why should developers have free reign over UDTs? There are
sure important enough to be centrally managed by DBAs, and specified
by system analysts.
If the company is too bureaucratic and thus managing UDTs
becomes a problem, that’s an organisational problem, not a
technological one.
--
Leandro Guimarães Faria Corsetti Dutra +55 (44) 3028 7467 ext34
Rua Guarani 361 ap 601 – Z4 +55 (44) 3025 6253
87.014-040 Maringá, PR leandro@dutra.fastmail.fm
BRAZIL http://br.geocities.com./lgcdutra/
| |
| Alfredo Novoa 2004-08-27, 5:45 pm |
| On 26 Aug 2004 10:10:56 -0700, lajos.nagy@gmail.com (Josh Hewitt)
wrote:
> so far. The end result is a complex web of dependencies between
> database objects where you cannot simply re-define anything without
> re-defining its dependents at the same time. Simply put, the
> create/drop cycle became the most annoying part of the development.
It seems an insignificant problem to me.
You only have to drop the database and to execute the modified script.
>- Although it is logical after giving it a little thought, it came as a
> nasty surprise that Dataphor demands that all scalar types that might
> appear in a context where duplicate elimination is required must have
> the less-than (Dataphor calls it 'iLess') operator defined on them.
It seems a little bug to me. The equality operator should suffice.
>How can one do software maintenance in a live Dataphor installation?
Export the data, drop the database, execute the new script and import
the data.
>Since scalar types form the building blocks of the database design,
Scalar types don't change often in databases, specially in business
systems.
>more than cosmetic changes in this setup? You cannot just drop the type
>since everything depends on it. Also, what are you going to do with the
>data in your relvars? You cannot just drop your production tables.
Why not?
If it is a 24-7-365 system then you have redundant servers and you can
update them individually while the other servers still run.
>that designing UDTs is easier business in any way. So in a UDT-rich
>world developers would be forced to work with someone else's not-so-well
>designed UDTs most of the time.
The UDTs needed in biz systems tend to be few and trivial to design.
>Is it not better then to forget about UDTs
>in the database and only allow them in programming languages where
>developers have complete control over them?
Definitely not. They can be useful in the good hands, and in the bad
hands the system will be a disaster anyway  .
> Is it not the case that
>the lack of UDTs in today's databases is more of a blessing then a curse?
The lack of UDTs in today's business systems is not a very important
problem, but it is in GIS, CAD and other areas. The niches of the
called OODBMSs (basically network DBMSs with UDTs).
>Sorry for the long post, but I felt that this might be an interesting
>topic to discuss in this news group.
The most interesting post in a long while.
Regards
Alfredo
| |
| Mikito Harakiri 2004-08-27, 5:45 pm |
|
"Leandro Guimaraens Faria Corsetti Dutra" <leandro@dutra.fastmail.fm> wrote
in message news:pan.2004.08.27.12.31.59.829172@dutra.fastmail.fm...
> Em Thu, 26 Aug 2004 10:10:56 -0700, Josh Hewitt escreveu:
>
>
> Why should developers have free reign over UDTs? There are
> sure important enough to be centrally managed by DBAs, and specified
> by system analysts.
>
> If the company is too bureaucratic and thus managing UDTs
> becomes a problem, that's an organisational problem, not a
> technological one.
Leandro, do you really believe that couple of thin paragraphs is enough to
rebuff insightful and quite substantial original message?
To add to UDT problems (which were posted sometime ago in the c.d.t exchange
with Paul Brown, so I plot only summary here):
1. Creating User Defined indexes is hard.
2. User Defined statistics is expert users domain.
3. UDT optimization is virtually non-existent.
Even though 1-3 might be not fundamental problems, they create
unsatisfactory Return-On-Investment when creating a UDT of only limited
utility requires unproportional effort.
| |
| Peter Koch Larsen 2004-08-27, 5:45 pm |
|
"Laconic2" <laconic2@comcast.net> skrev i en meddelelse
news:MLydnffBvNGntLPcRVn-jQ@comcast.com...
>
> "Josh Hewitt" <lajos.nagy@gmail.com> wrote in message
> news:1c92edeb.0408260910.60242a59@posting.google.com...
>
> Yes, it is logical.
>
Why? You only need the operator "same as" (is it "=" or "==" on Alphora?) to
remove duplicates. I would probably implement duplicate removal using some
kind of hashing.
[snip]
Kind regards
Peter
| |
| Tony Douglas 2004-08-30, 5:45 pm |
| Christopher Browne <cbbrowne@acm.org> wrote in message news:<2p7l7sFhrhjuU1@uni-berlin.de>...
<snip>...
> I'm not sure either is the ultimate answer, and it would _definitely_
> be an inconvenience to make a little domain change "over here" that
> leads to 200GB worth of tuples having to be revised "over there."
Alternatively, multiple possible representations. I must admit to not
being a fan of these for the usually pushed purpose (polar vs.
cartesian co-ordinates, say), but I could see a use for them in
version control, such that as and when an update is required on a
given tuple, the representation of the UDT in that tuple can be
updated to the "newer" version (much like in some DBMSs "alter table
.... add column..." adds the column as and when a row is updated or
inserted).
In the ML or Haskell system, where you may opt to describe your UDT
with an algebraic datatype, you might be able to add in extra cases,
and add corresponding cases to the patterns for your operators. Type
variables probably wouldn't be of much assistance to you here as you
are intimately interested in the internal workings of your new
datatype.
- Tony
| |
| Nathan Allan 2004-08-30, 5:45 pm |
| Josh,
> I have recently downloaded Alphora's Dataphor product and begun
> experimenting with it.
First of all, let me thank you. Many people speculate about our
product but few actually give it more than a superficial glance-over.
It is refreshing for us to be able to discuss reasonable and even
valid criticisms.
> [...] I did not use any
> external storage device (that is to say, a SQL database back-end). I
> only experimented with the 'in-process' server. (Although I plan to
> connect to an Oracle instance later to see how it performs.)
Please allow me to correct the terminology here. "In-process" refers
to the ability to host the Dataphor server within the instance of the
client or development environment. I believe what you meant was that
you were using one of the built-in storage devices; either "Memory
Device" or "Simple Device" for in-memory or simple file dump
respectively. Whether or not the Dataphor Server is running
"in-process", it can connect to any number of underlying data stores.
No big deal... just wanted to clarify.
> ... - Since the DBMS now can be extended with user defined types (UDTs) along
> with their operators it follows that types and operators become
> database objects themselves. This means that they must be 'CREATE'-ed,
> later, possibly, 'ALTER'-ed, and eventually, 'DROP'-ed, just like
> tables (I use the Dataphor terminology here), views, and constraints.
> In addition, the strongly-typed nature of Dataphor means that each
> database object, in any reasonably large system, will have several
> dependencies and several dependents. Types can depend on other types
> and operators for their definition; operators depend on types and can
> depend on tables and views; tables and views depend on types and
> operators; and finally, constraints can depend on everything mentioned
> so far. The end result is a complex web of dependencies between
> database objects where you cannot simply re-define anything without
> re-defining its dependents at the same time. Simply put, the
> create/drop cycle became the most annoying part of the development.
> For example, when I realized that I have to change an operator I had to
> drop all the constraints and operators that used the one being changed.
> The situation was even uglier when I realized that I had to change one
> of the scalar types. In this case I practically had to drop every
> database object and then re-create them again.
Let me start by saying that this is indeed a valid observation;
something that a Dataphor developer must deal with. I'll add, in
fact, that it is relatively easy to get into a tangled web of
dependencies with other types of database objects. For example,
imagine attempting to drop a column in a table that is referenced by
an operator that is used in a default of another column of another
table. This really is an issue in all compiled systems, though it
particularly manifests itself in Database systems because the unit of
compilation is more granular than that of a traditional compiler.
Dataphor mostly leaves management of dependencies to the developer,
but it does provide some services for this management:
-Dataphor has several operators for querying the dependency graph for
any object.
-Dataphor has operators for scripting drop and re-creation scripts
for objects, libraries, or even the entire catalog, based on the
dependency graph.
-Dataphor exposes these various operators interactively in the
development environment (Dataphoria)
-Great effort went into the error reporting system in Dataphor.
Dependency related errors identify exactly what dependencies are
problematic.
Though dependency management in Dataphor is mostly a manual process at
present, the problem is not inherently manual. We have ideas for
managing and automating complexities surrounding not only dependency
management, but other schema modification problems as well. Future
versions of Dataphor will address these issues.
One more note regarding this issue, though it may seem difficult
managing a complex network of dependencies, development in Dataphor is
not generally done this way in practice (at least we do not do it this
way) because of its "federated" architecture. Due to the decoupling
of the Dataphor schema from the underlying data store, if more than a
few changes need to be applied, the schema script is modified and the
library re-registered. This way there is no schema alteration concern
for changes that do not affect the schema of the underlying storage
device. Changes that do affect the underlying storage system are
managed through alterations using Dataphor's upgrade subsystem.
> ... - Although it is logical after giving it a little thought, it came as a
> nasty surprise that Dataphor demands that all scalar types that might
> appear in a context where duplicate elimination is required must have
> the less-than (Dataphor calls it 'iLess') operator defined on them.
> (Probably any of the operators that provide ordering would do: <, >,
The operators iEqual and either iLess or iGreater must be provided
unless iCompare is provided. iCompare returns 0, -1, or 1 based on
the comparison of two operands. If iCompare is defined, then Dataphor
will use it in place of the individual comparison operators if they
are not defined. BTW there is also a symbol (?=) for the iCompare
operator due to its usefulness.
[vbcol=seagreen]
> It was a nasty surprise because, at first, I could not
> figure out why Dataphor rejected an otherwise perfectly formulated
> query, protesting that 'operator iLess' was not defined for one of the
> scalar types.
This is admittedly a vague error. We should address this.
> ... I understand that without ordering defined on a
> scalar type it is very inefficient to remove duplicates but I did not
> feel like defining an arbitrary comparison operator just for this
> purpose.
As was mentioned in another post in this thread, there are mechanisms
for duplicate elimination that are at least as fast as sorting (e.g.
hashing) but Dataphor does not currently employ them. I would note
that although Dataphor's compiler detected the possible need for this
operator, the operator would probably not have actually been needed if
the storage device were one of the more sophisticated ones. If, for
example, the execution plan at the point of the query in question were
bound to one of the "big three" DBMSs, that DBMS would decide how to
eliminate those duplicates (Dataphor would ensure that the emitted SQL
statement instructs that they be removed).
> I ended up 'projecting away' the troublesome attribute since
> I did not need it in the final result anyway. This solved my problem
> but it showed that physical implementation does leak through the
> logical model.
Guilty as charged. Dataphor should only require that equal be
defined, as there are critical logical reasons for this requirement.
I would note that there are, however, logical reasons for introducing
the notion of "ordinal" types in a system; the problem is of course
requiring them for operations that do not logically need them.
> Maybe Dataphor should provide some kind of internally
> implemented default ordering for the purposes of duplicate elimination.
We have considered this, but have kept it as a low priority because
most data types can easily be construed to have some sort of ordering
(though a bit artificial at times).
> ... - The help system is a little bit out of sync with what is implemented at
> points and I found the number of examples provided too few. This is
> especially true of the D4 Language Guide. The sample applications
> bundled with the evaluation version do not really show the capabilities
> of the system either. In my opinion, developers might benefit more
> from a "feature fest" type demo application.
Yes, this is all true. Fortunately this is only a weakness of the
beta and will not be the case in the final version 2 release.
> How can one do software maintenance in a live Dataphor installation?
I alluded to some of the answers above and I will skip the details,
but we have found from experience that it is not too difficult.
> Since scalar types form the building blocks of the database design,
> almost every database object will depend on them, either directly or
> indirectly. ... You cannot just drop the type
> since everything depends on it. Also, what are you going to do with the
> data in your relvars? You cannot just drop your production tables.
As was mentioned, there are ways of changing the schema (including
completely dropping and re-creating it) without affecting the stored
data.
> What are you going to do with older applications that depend on (the now
> obsolete) old types/operators/tables/views.
This can be accomplished reasonably using storage device
reconciliation modes, and Dataphor's upgrade mechanism.
> Some suggested creating
> views for the old applications based on the new tables. I do not think
> that this approach is viable in the long run. It would mean duplicating
> the affected parts every time there is a bug fix. Also, the added
> complexity of maintaining 'parallel universes' for old and new programs
> is not something that one would want.
I agree... not a viable solution.
> ... What if major vendors 'do get it' but enterprise environments simply
> 'squeeze out' anything but atomic types (numbers, character strings,
> booleans, etc.) from the database?
>
> Quite often, in enterprise environments, development teams rarely have
> access and/or control over applications written by other teams. Given
> the tangled web of dependencies in a 'UDT-enabled' RDBMS, achieving
> success suddenly becomes a factor of how well-designed those UDTs are in
> the first place, and if they need modification, who has the right (and
> the resources) to perform those modifications. I think this is the make
> or break question of the real life success of truly relational databases
> right along with the questions of performance (which I do not want to
> address right now.)
There are many levels being addressed here so let me start by
suggesting that your ideas seem to be presented from the perspective
of the current state of the art, whereas a system like Dataphor may
completely change the dynamics of the situation. For example, the
reasoning that led to today's typical development approach includes
the dangerously insidious notion that "data" in a database system is
somehow different than "data" in a "programming environment".* The
point is that the entire application development process revolves
around current thinking regarding the "layers" of an application and
their composition. Dataphor is (for better or worse) against the
grain in this regard by its very nature--that of a "middle
(application) tier" that behaves as a database system.
Setting this aside, let's imagine that Dataphor were treated as the
"database tier" in today's paradigm, and that an additional "middle
tier" were being built against it.** In this scenario, I see managing
change of UDTs as not presenting much (if any) more challenge to the
"application tier" than do table changes. Note that UDTs (as well as
system provided types) are translated between the logical layer and
the external one, so an external user is really just concerned with
the values on that side of the translation. If the system allows for
control of this translation within the logical realm (as Dataphor
does), then it could even be argued that UDTs provide additional
indirection to the external level.
*In fact, even the notion of a TRDBMS as being some type of utopian
ideal fuels this thinking. Regardless of its "conformity", a DBMS can
only enable declarative development (the *real* goal) if its services
(such as logical data independence and relation data types and
operators) are exposed and even brought into each of the application
layers.
** This scenario seems pretty backwards considering that Dataphor is
designed to replace and eliminate the need for such an imperative
middle tier, but this seems to be the perspective from which you pose
the issue.
> Most development projects try to have control over
> as many factors of their success as possible. (That is why everybody is
> building several layers of abstractions in order to 'insulate' themselves
> from the ever-changing environment.)
May I point out that ironically this is largely due to the lack of
visibility with regard to dependencies? ;-)
> Average workplaces have average
> developers on average. This means that it is rarely the case that
> developers can work on a database that is well-designed. Even in
> database design there are a lot of ways to screw up and I do not think
> that designing UDTs is easier business in any way.
A few points:
-UDTs certainly can make things easier. If the problem domain makes
use of coordinates, intervals, varying representations/units of
measure, or whatever, then having to deal with encoding within overly
simplistic types makes things more difficult and complex.
-Dataphor makes building trivial UDTs very easy (e.g. create type MyID
like String).
-Dataphor also makes using UDTs extremely easy. There are no
differences between user-defined types and system-provided types in
Dataphor, the system-defined types are built in the same way that UDTs
are, so using values of UDTs is exactly the same as using values of
the system-provided types.
-Because UDTs have the same characteristics as system-provided types,
(indeed, we hardly ever refer to them as UDTs, they are simply scalar
types), whether or not a given development effort takes advantage of
them is largely an issue of comfort level.
-UDTs do not have to be used.
> So in a UDT-rich
> world developers would be forced to work with someone else's not-so-well
> designed UDTs most of the time. Not to mention that fixing a badly
> designed mess can be very time-consuming, especially if it is not the job
> one were hired to do in the first place.
Again this is assuming the too common "DBAs with their sacred tome"
approach to business. As Leandro stated, UDTs are hardly the problem
in such an environment.
> Even fixing serious database
> design flows in today's UDT-free databases often meets with severe
> resistance because of the (not completely unsubstantiated) fear of
> breaking other applications. Is it not better then to forget about UDTs
> in the database and only allow them in programming languages where
> developers have complete control over them?
Again, that is assuming that the "developers" and the "database
architects" are different people. I don't think this should be the
case, especially if we aim for a more declarative approach to
development
Begin flaming here ;-)
> Is it not the case that
> the lack of UDTs in today's databases is more of a blessing then a curse?
A strong but subjective No! As an application developer myself that
has had the pleasure of using them for a couple of years, you would
have to pry me from them. ;-)
> Sorry for the long post, but I felt that this might be an interesting
> topic to discuss in this news group.
Thanks again.
Regards,
--
Nathan Allan
Alphora
| |
| Paul G. Brown 2004-09-02, 5:45 pm |
| lajos.nagy@gmail.com (Josh Hewitt) wrote in message news:<1c92edeb.0408260910.60242a59@posting.google.com>...
[ snip ]
> What if major vendors 'do get it' but enterprise environments simply
> 'squeeze out' anything but atomic types (numbers, character strings,
> booleans, etc.) from the database?
[ snip ]
The dumbest idea in 'database theory' is that 'relational theory' is
about how to build 'databases' (and therefore that the way to advocate
for the 'relational model' is to bash SQL DBMS products, multi-value
DBMS products, and so on).
The Relational Model describes a 'reason machine'.
If you want the SQL:
UPDATE Emps E
FROM GetSalaryPopupWindow G
SET Salary := G.New_Salary
WHERE G.Name = E.Name
AND G.Old_Salary = E.Salary
AND G.UI_Device = CurrentUserUI;
Emps E is a 'table'.
GetSalaryPopupWindow is a 'relvar' that gets its values by popping up
a UI window populated with E.Name and current E.Salary into the device
indicated by the CurrentUserUI, and once that window is closed, yielding
a 'New_Salary' value back to the query.
Or
UPDATE Screen S
FROM File F
SET S.Pixel = F.Pixel
WHERE S.X = F.X
AND S.Y = F.Y;
There is *nothing* in the relational model that precludes doing this kind
of thing. There are inhibitors in the minds of relational thinkers, who
for too long have been fighting a battle they lost in their youth, over
and over and over again.
Forget databases, guys. They're dead. Dead as the dodo.
The kinds of things Dataphor is (ought to be) capable of are far broader,
and more important, than databases. Lord knows, Postgress could run
both of the queries above in 1993 (but all the relational people seemed to
notice was that it implemented references, and then used this
implementation to show that they had no performance advantage at all.)
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
news:57da7b56.0408301732.c200bd5@posting.google.com...
> The dumbest idea in 'database theory' is that 'relational theory' is
> about how to build 'databases' (and therefore that the way to advocate
> for the 'relational model' is to bash SQL DBMS products, multi-value
> DBMS products, and so on).
>
> The Relational Model describes a 'reason machine'.
Here's a quote from the original Codd paper:
<quote>
This paper is concerned with the application of elementary relation theory
to systems which provide shared access to large banks of formatted data.
Except for a paper by Childs [1], the principal application of relations to
data systems has been to deductive question - answering systems. Levein and
Maron [2] provide numerous references to work in this area.
</quote>
It's clear from the above that relational data thory was not originally
about databases as such. To me, "reason machine", "inference engine", and
"deductive question answering systems" all seem like variations on a theme.
But that doesn't mean that using the relational model as the basis for a
database was a bad idea. It was a good idea, when compared to the existing
alternatives in 1970. It's still a good idea.
Your comments regarding Dataphor may well be correct. It may be much more
than a "database", and perhaps it should be.
"systems which provide shared acess to large banks of formatted data" are
still relevant, although in a different context.
I think much of the ideological thrashing and trashing I've seen in this
forum is comparing "databases" to "application development environments" as
if they were the same kind of thing. It's useful to realize that they are
not.
| |
| D Guntermann 2004-09-02, 5:45 pm |
|
"Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
news:57da7b56.0408301732.c200bd5@posting.google.com...
> lajos.nagy@gmail.com (Josh Hewitt) wrote in message
news:<1c92edeb.0408260910.60242a59@posting.google.com>...
> [ snip ]
> [ snip ]
>
> The dumbest idea in 'database theory' is that 'relational theory' is
> about how to build 'databases' (and therefore that the way to advocate
> for the 'relational model' is to bash SQL DBMS products, multi-value
> DBMS products, and so on).
>
> The Relational Model describes a 'reason machine'.
>
> If you want the SQL:
>
> UPDATE Emps E
> FROM GetSalaryPopupWindow G
> SET Salary := G.New_Salary
> WHERE G.Name = E.Name
> AND G.Old_Salary = E.Salary
> AND G.UI_Device = CurrentUserUI;
>
> Emps E is a 'table'.
>
> GetSalaryPopupWindow is a 'relvar' that gets its values by popping up
> a UI window populated with E.Name and current E.Salary into the device
> indicated by the CurrentUserUI, and once that window is closed,
yielding
> a 'New_Salary' value back to the query.
>
> Or
>
> UPDATE Screen S
> FROM File F
> SET S.Pixel = F.Pixel
> WHERE S.X = F.X
> AND S.Y = F.Y;
>
> There is *nothing* in the relational model that precludes doing this
kind
> of thing. There are inhibitors in the minds of relational thinkers, who
> for too long have been fighting a battle they lost in their youth, over
> and over and over again.
>
> Forget databases, guys. They're dead. Dead as the dodo.
Hi Dr. Brown:
I certainly respect your knowledge and opinion, but I'm not sure I agree to
the degree you state databases are dead. The examples you give are
certainly possible and even already implemented, especially in tightly
coupled integrated forms and reports environments with DBMS's. However,
there still seems to be difficulty in convincing procedurally-oriented and
constructionist-oriented developers in the usefulness of declarative
manipulation across code/platforms/and data sources.
We've seen a great deal of research in mediator architectures and other
"integrating" technologies over heterogeneous sources (files, OODBs,
hybrids, hierarchical, and the like), but I have yet to see something that
comes close to overcoming the NP-complete nature of the problem. During the
time (a small amount and in an academic context, granted) I spent looking
and researching developments, it seemed pretty clear that you can't use an
integrating/reconciling technology based on logic if the sources don't
necessarily follow any logic (that is easily and explicitly translatable).
You simply can't make the illogical suddently logical without making a lot
of assumptions, where half the time they might be correct and half the time
they might not be.
I've seen other papers written (such as Bernstein as a follow-on to 'Too
Much Middleware') that predict that middleware will evolve and merge various
technologies (data management, portal, IA, etc) further into a very small
set of monolithinc super-integrating technology products, and sometimes the
trend seems to support this notion. But, still SQL DBMS technology seems to
be thriving and the individuals around me seem comfortable with some sort of
decoupling of a single application from data, especially in consideration of
a larger enterprise context.
Could you give more concrete examples of what upcoming technology
theories/implementations are leading to the impending raising of the
independent DBMS tombstone? Thanks.
- Dan
| |
| Peter Koch Larsen 2004-09-02, 5:45 pm |
|
"Nathan Allan" <nathan@alphora.com> skrev i en meddelelse
news:fedf3d42.0408291030.1641800f@posting.google.com...
> Peter,
>
Alphora?) to[vbcol=seagreen]
some[vbcol=seagreen]
>
> The equal operator in Dataphor is "=" (assignment is ":="). Yes,
> hashing is an excellent duplicate removal mechanism, but thus far
> Dataphor only uses BTrees (and therefore requires ordering).
Okay. This then is one of the places where the model is influenced by the
implementation, which is quite unlucky imho.
With respect to your comment on BTrees, i believed Alphora to be implemented
on top of SQL... therefore i do not understand your explanation. Could you
elaborate?
>
> Regards,
>
> --
> Nathan Allan
> Alphora
Kind regards
Peter
| |
| Paul G. Brown 2004-09-02, 5:45 pm |
| "D Guntermann" <guntermann@hotmail.com> wrote in message news:<I3Aqo2.817@news.boeing.com>...
> "Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
> news:57da7b56.0408301732.c200bd5@posting.google.com...
> news:<1c92edeb.0408260910.60242a59@posting.google.com>...
>
> Hi Dr. Brown:
>
> I certainly respect your knowledge and opinion, but I'm not sure I agree to
> the degree you state databases are dead.
First, I ain't a Dr. And second, I'm not sure I respect my own knowledge
and opinions, so I'm wary when someone else says they do.
[ snip ]
> However,
> there still seems to be difficulty in convincing procedurally-oriented and
> constructionist-oriented developers in the usefulness of declarative
> manipulation across code/platforms/and data sources.
I'd respond by pointing out that it is revolution, not evolution, that
has characterized the development of ideas in computer science. And
specifically, repeated revolutions in our conceptualization of what
constitutes a 'system' or 'program'. Vannevar Bush had dreamed up
hypertext and something like the HTML/HTTP based web in 1945, but along
the way we've endured mainframes, two-tier, three tier,
networked/distributed architectures, the Web and peer-to-peer.
But even VB didn't get so far as Google. And as a 'reason engine', Google
leaves a lot to be desired.
[ big snip of reasonable argument/observations ]
> Could you give more concrete examples of what upcoming technology
> theories/implementations are leading to the impending raising of the
> independent DBMS tombstone? Thanks.
The DBMS tombstone has engraved upon it "6%", which is the growth rate
in license sales over the last few years. This says to people with
money that investment in DBMS technology will not generate
a reasonable rate of return. (By DBMS, I mean software products that
manage a shared, centralized repository or 'data bank'). The market for
these products has become commoditized beneath a rigid standard and
any attempt to break out of that standard is met with contempt by
customers. Want to compete in the DBMS business? Price, price, price.
On the other hand, the theory and practice of information management
continues to be important and to grow new limbs. How are we going to
cope with inputs from a pervasive computing infrastructure? What is the
most useful level of abstraction at which to view distributed hardware
resources? These challenges are driving requirements for which the
DBMS model of the world--either SQL DBMS or TR DBMS--is entirely unsuited.
Now I'm convinced that, because it supports declarative programming,
relational thinking is at least extremely useful (in my bolder moments
I'd even sign up for 'central' or 'necessary'). But sitting around
comparing the merits and demerits of Dataphor and SQL DBMS technologies
seems to be a gigantic waste of time. Like squabbling about whether Greek
or Latin is the true language of religious consciousness.
(Why am I bothering, then? Because I'd like to be convinced otherwise.
It would soothe my worries about my career choices no end.)
| |
| Gene Wirchenko 2004-09-02, 5:45 pm |
| mAsterdam <mAsterdam@vrijdag.org> wrote:
>Josh Hewitt wrote:
>
>
>[...]
>
You are apologising for posting some of the most interesting
material I have seen in this group?
[vbcol=seagreen]
>Thank you for sharing your experience.
Seconded.
I have heard of Dataphor, but I really do not know enough to dig
in. I would be pleased to read more about it.
Sincerely,
Gene Wirchenko
Computerese Irregular Verb Conjugation:
I have preferences.
You have biases.
He/She has prejudices.
| |
| Rene Hartmann 2004-09-02, 5:45 pm |
| On Sat, 28 Aug 2004, Rene Hartmann wrote:
> I would rather suggest using a system-defined comparison operator, which
> could be based on the internal representation of a type.
> This operator must only be consistent with the equality operator, so that
> if A =3D B then A <=3D B must also evaluate to true. Since the operator
> is only used internally for purposes of duplicate elimination, there are
> no special requirements for it.
>
OK, I must admit that things are not as easy as I thought first.
I thought a comparison function which first calls the user-provided
equality operator and then compares the byte representation would do,
but this could lead to a situation where A < B < C but A =3D C.
So it seems that if the user provides an equality operator, s/he
must also provide <=3D.
But I think a user-provided equality operator is rarely possible,
because equality could be based on a possible representation,
and <=3D could too.
--
Rene=B4 Hartmann
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
news:57da7b56.0408310834.382e95e5@posting.google.com...
> First, I ain't a Dr. And second, I'm not sure I respect my own knowledge
> and opinions, so I'm wary when someone else says they do.
Ahh, someone like myself! A refreshing change from the people who know
everything, for small values of everything.
I look forward to discussing things with you. Maybe we'll each learn
something.
>
> I'd respond by pointing out that it is revolution, not evolution, that
> has characterized the development of ideas in computer science. And
> specifically, repeated revolutions in our conceptualization of what
> constitutes a 'system' or 'program'. Vannevar Bush had dreamed up
> hypertext and something like the HTML/HTTP based web in 1945, but along
> the way we've endured mainframes, two-tier, three tier,
> networked/distributed architectures, the Web and peer-to-peer.
I don't agree. I've been through lots of "revolutions" over the course of
my career.
Assembler to structured programming in 3GL to OOP.
Flat files to indexed files to hierachical, codasyl, and relational DBMS.
Raw iron modems to Arpanet to Internet.
Mainframes to Timeshared machines to minicomputers to desktops and palmtops.
And so on.
I think the revolutions are more apparent than real. Most of them gather
momentum outside of the general view, and then suddenly bloom when they are
ready for prime time.
Examples:
When was the floppy disk invented? 1970
When was the first "object oriented language" developed? 1967 (Simula) or
1981 (Smalltalk).
When was the relational model applied to database systems? 1970 or 1982
when the commercial world took over.
When were the first computers connected in the Arpanet? 1970
When was the first Intel microchip porcessor? 1972 (the 4004)
And so on.
All of these things that seemed so "revolutionary" when they burst on the
scene were actually years or decades in the making.
Even the wrold wide web. Between Vannevar Bush and HTML there was a long
progression, including hypertext itself, and markup lanaguages.
> The DBMS tombstone has engraved upon it "6%", which is the growth rate
> in license sales over the last few years. This says to people with
> money that investment in DBMS technology will not generate
> a reasonable rate of return. (By DBMS, I mean software products that
> manage a shared, centralized repository or 'data bank'). The market
for
> these products has become commoditized beneath a rigid standard and
> any attempt to break out of that standard is met with contempt by
> customers. Want to compete in the DBMS business? Price, price, price.
>
You make a good point here. But I think the 6% figure is just a reaction to
the "overselling" of DBMS licenses
during the last 15 years or so. People bought Oracle, DB2 or SQLserver
licenses not because they really wanted to manage a "data bank" in the
classical sense at all. They just used SQL based DBMS software because
"it's the latest thing" or "everybody else is doing it, and they must be
really smart."
Those are lousy reasons for choosing any tool, and a DBMS is no exception.
What's interesting is that it's still growing. Those people whose career
depends on 40% growth rates are toast.
But that's true in every field: insurance, real estate, agriculture, and
even social security.
BTW, I wouldn't call it "centralized" anymore. But I would call it
"integrated". What matters is not whether the data is kept in the same
placed but whether it can be used together.
> On the other hand, the theory and practice of information management
> continues to be important and to grow new limbs. How are we going to
> cope with inputs from a pervasive computing infrastructure? What is
the
> most useful level of abstraction at which to view distributed hardware
> resources? These challenges are driving requirements for which the
> DBMS model of the world--either SQL DBMS or TR DBMS--is entirely
unsuited.
>
You are right on here. Integrating disjoint and fragmented data is going to
be around for a long, long time. Much longer than
the DBMS fad. The thing is, it not only takes someone who is wise,
knowledgeable, and canny to do it. It also takes a reasonably smart manager
to know that it needs to be done. It's going to take a while for the
market to permit those who use stategic long term data strategies to gain
the commercial advantage over those who fly by the seat of their pants.
I'm not sure I can wait that long.
> Now I'm convinced that, because it supports declarative programming,
> relational thinking is at least extremely useful (in my bolder moments
> I'd even sign up for 'central' or 'necessary'). But sitting around
> comparing the merits and demerits of Dataphor and SQL DBMS
technologies
> seems to be a gigantic waste of time. Like squabbling about whether
Greek
> or Latin is the true language of religious consciousness.
A fascinating sub topic. Having grown up in Latin America, I know Latin
much better than Dan Quayle did. ;-)
But Greek has all these neat features like the aorist tense and the middle
voice.
It seems like Greek was invented for philosophers, while Latin was invented
for engineers and generals.
>
> (Why am I bothering, then? Because I'd like to be convinced otherwise.
> It would soothe my worries about my career choices no end.)
There are no easy choices. Don't worry about making the wrong one.
| |
| Christopher Browne 2004-09-02, 5:45 pm |
| In an attempt to throw the authorities off his trail, "Laconic2" <laconic2@comcast.net> transmitted:
> When was the floppy disk invented? 1970
> When was the first "object oriented language" developed? 1967 (Simula) or
> 1981 (Smalltalk).
> When was the relational model applied to database systems? 1970 or 1982
> when the commercial world took over.
> When were the first computers connected in the Arpanet? 1970
> When was the first Intel microchip porcessor? 1972 (the 4004)
>
> And so on.
Actually, this could be considered support the notion that there has
been practically NO genuine innovation since the early 1970s.
Which would support the further thought that it would be a sensible
idea to ignore _all_ of the technologies created more recently than
(say) 1975, and for would-be innovators to look at the things created
between 1965 and 1975 for inspiration to see if things that failed
then due to computers not being powerful enough could be reclaimed
now.
I'm not _entirely_ serious, although I definitely don't think there
has been anything usefully new in either OSes or DBMSes since at best
the '80s.
- Linux is a Unix derivative, and therefore not genuinely
new;
- Windows isn't particularly useful, so being new isn't helpful ;-);
- XML is merely a rehashing of S-exprs, which date back to the '60s;
- Java and C# are merely additional iterations in the retreads of BCPL
and Algol :-)
- Most of the innovation in "relational stuff" ended when Stonebraker
left academia.
You could do far worse than to rummage around the papers of the '70s
to look for mine them for things people have forgotten... Nobody in
IT 'management' would be likely to notice that you were creating
'retreads' of old technologies.
--
"cbbrowne","@","acm.org"
http://www.ntlug.org/~cbbrowne/multics.html
If a logical expression is hard to understand, try transforming it.
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Christopher Browne" <cbbrowne@acm.org> wrote in message
news:2pkepaFls80gU1@uni-berlin.de...
> Actually, this could be considered support the notion that there has
> been practically NO genuine innovation since the early 1970s.
>
> Which would support the further thought that it would be a sensible
> idea to ignore _all_ of the technologies created more recently than
> (say) 1975, and for would-be innovators to look at the things created
> between 1965 and 1975 for inspiration to see if things that failed
> then due to computers not being powerful enough could be reclaimed
> now.
>
> I'm not _entirely_ serious, although I definitely don't think there
> has been anything usefully new in either OSes or DBMSes since at best
> the '80s.
>
I think that's a little too "retro", even for me.
When I finally bought a computer of my own, in 1983...
Spreadsheets were an innovation for me, although VISICALC was already many
years old.
And TURBO PASCAL was a great leap forward in simplicity, power, and, yes
price.
I liked TP so much that I started writing Pascal programs at home, and then
uploading them to the "real computers" at work.
But I digress. In keeping of my theme of "evolution, not revolution", I'm
going to say that progress has been steady.
The only thing I lament is all the stuff that gets invented, debugged,
matured, and then forgotten. Then, some time later
someone from a later generation, "invents it" all over again.
more later..
> - Linux is a Unix derivative, and therefore not genuinely
> new;
>
> - Windows isn't particularly useful, so being new isn't helpful ;-);
>
> - XML is merely a rehashing of S-exprs, which date back to the '60s;
>
> - Java and C# are merely additional iterations in the retreads of BCPL
> and Algol :-)
>
> - Most of the innovation in "relational stuff" ended when Stonebraker
> left academia.
>
> You could do far worse than to rummage around the papers of the '70s
> to look for mine them for things people have forgotten... Nobody in
> IT 'management' would be likely to notice that you were creating
> 'retreads' of old technologies.
> --
> "cbbrowne","@","acm.org"
> http://www.ntlug.org/~cbbrowne/multics.html
> If a logical expression is hard to understand, try transforming it.
| |
| Laconic2 2004-09-02, 5:45 pm |
| I had to step away from the keyboard, so I'm continuing my response now.
"Christopher Browne" <cbbrowne@acm.org> wrote in message
news:2pkepaFls80gU1@uni-berlin.de...
> - Linux is a Unix derivative, and therefore not genuinely
> new;
It's an evolution. Unix itself was an evolution from Multics.
>
> - Windows isn't particularly useful, so being new isn't helpful ;-);
Windows IS useful. It's easy to learn. (not the same thing a s easy to use,
but that's a whole other discussion).
Windows got its look and feel from the Macintosh. And the Mac got its look
and feel from the "office of the future"
that was built at the Palo Alto Research Center of Xerox corp. Again, it's
an evolution.
>
> - XML is merely a rehashing of S-exprs, which date back to the '60s;
Dunno about this one, so I'll pass.
>
> - Java and C# are merely additional iterations in the retreads of BCPL
> and Algol :-)
Well, not really. Algol was actually a "tighter" language than C. When I
left Algol, I went with Pascal.
Then again, I thought Betamax was better than VHS.
>
> - Most of the innovation in "relational stuff" ended when Stonebraker
> left academia.
>
> You could do far worse than to rummage around the papers of the '70s
> to look for mine them for things people have forgotten... Nobody in
> IT 'management' would be likely to notice that you were creating
> 'retreads' of old technologies.
I'm doing a little of that kind of work myself. I like to think of it as
more a "consolidation" of several old ideas than a simple "retread". But
maybe I'm exaggerating my own importance.
| |
| Nathan Allan 2004-09-02, 5:45 pm |
| Peter,
>
> Okay. This then is one of the places where the model is influenced by the
> implementation, which is quite unlucky imho.
I agree. It is on our list of things to address. We haven't found it
to significantly enough affect the overall usability or usefulness of
the product to bump the other tasks on our list.
> With respect to your comment on BTrees, i believed Alphora to be implemented
> on top of SQL... therefore i do not understand your explanation. Could you
> elaborate?
I would be happy to. The Dataphor compiler starts with the assumption
that none of a query's operations can be handled by the storage
device(s), except base table "retrieval" operations, which storage
devices are required to support. The compiler creates an initial
execution plan based on this assumption. The execution plan is later
bound to the various storage devices at which time Dataphor determines
the extent to which each device can support the query. When
performing the initial compilation, the Dataphor server attempts to
resolve comparison operators for duplicate elimination via the BTree
that it assumes will be used. The reason this only comes up in
certain places--such as sorting and duplicate removal--is due to
Dataphor's pipelined query processor. In other words, Dataphor does
not materialize a result set at every operator, and in most cases
avoids materializing the complete set altogether.
There you have it, our laundry hanging there for all to see. :-)
Regards,
--
Nathan Allan
Alphora
| |
| Paul G. Brown 2004-09-02, 5:45 pm |
| "Laconic2" <laconic2@comcast.net> wrote in message news:<mrCdna138YP8cancRVn-pw@comcast.com>...
> "Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
> news:57da7b56.0408310834.382e95e5@posting.google.com...
> All of these things that seemed so "revolutionary" when they burst on the
> scene were actually years or decades in the making.
So were the English Civil War, the French Revolution, the American
Revolution, the events of 1917 in Russia, the Cuban revolution, etc.
Name one revolution which was not 'actually years or decades in the making'.
Contrast with the evolution of British Parliamentary Democracy, the
evolution in suffrage in the US, etc. Or the rather smooth evolution of
automobile technology.
A 'revolution' is simple one big-arse party with knife fights 'round the
stereo an' we all wake up with a really bad hangover an' a tatoo (SQL). You
sayin' that don't characterize the history of computer science?
[ snip ]
> You make a good point here. But I think the 6% figure is just a reaction to
> the "overselling" of DBMS licenses
> during the last 15 years or so.
6% (Gartner's Number, not mine) says "there isn't anyone left t' sell
this stuff to". Are you really suggestin' it's gonna go back to, oh,
10% in two or three years?
DBMSs been sellin' since 1985. 20% growth, year on year, almost ever
since. Today - 6%. That's population numbers, boyo.
[ snip ]
> BTW, I wouldn't call it "centralized" anymore. But I would call it
> "integrated". What matters is not whether the data is kept in the same
> placed but whether it can be used together.
You make a good point. But do you know what the growth rate in
'information integration' tools and products is? 4%.
[ snip ]
> It also takes a reasonably smart manager
> to know that it needs to be done. It's going to take a while for the
> market to permit those who use stategic long term data strategies to gain
> the commercial advantage over those who fly by the seat of their pants.
Me. I'm a geek. Always been one. But lately, I've been actin' as the kind
of 'domesticated' geek managers like to invite to meetings.
Y' know what? IT in general makes about 2% difference to price. Want to
increase demand? Buy an endorsement from Britany. Want to cut cost? Move
production to somewhere that balances labor and transport costs. Want to
argue about how much money we spend on DBMS technology? Lesse. . . Oh!
Here it is! It's the 200th ranked line item. Next!
All managers care about DBMS tech is 'How much is it gonna cost me?' Wasn't
always like that. Wal*Mart, and a few others, built a business model on
top of their ability to manage data. Today, everyone knows those tricks.
[ snip ]
> It seems like Greek was invented for philosophers, while Latin was invented
> for engineers and generals.
Well, all I can say is "Tuis pugis pignore!"
| |
| Marshall Spight 2004-09-02, 5:45 pm |
| "Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message news:57da7b56.0408310834.382e95e5@posting.google.com...
>
> On the other hand, the theory and practice of information management
> continues to be important and to grow new limbs. How are we going to
> cope with inputs from a pervasive computing infrastructure? What is the
> most useful level of abstraction at which to view distributed hardware
> resources? These challenges are driving requirements for which the
> DBMS model of the world--either SQL DBMS or TR DBMS--is entirely unsuited.
An interesting assertion. Do you have anything to back it up? To my mind,
the relational model has a lot of relevance to the questions you ask.
Marshall
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
news:57da7b56.0408312037.260d7269@posting.google.com...
> Me. I'm a geek. Always been one. But lately, I've been actin' as the
kind
> of 'domesticated' geek managers like to invite to meetings.
So why should geeks make lots of money? Go to a sideshow and bite the heads
off of live chickens.
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Marshall Spight" <mspight@dnai.com> wrote in message
news:52dZc.216184$8_6.123794@attbi_s04...
> An interesting assertion. Do you have anything to back it up? To my mind,
> the relational model has a lot of relevance to the questions you ask.
You are right. But keep in mind that the relational model has to do with a
broader scope of information than just DBMS systems.
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message
news:57da7b56.0408312037.260d7269@posting.google.com...
> Contrast with the evolution of British Parliamentary Democracy, the
> evolution in suffrage in the US, etc. Or the rather smooth evolution of
> automobile technology.
Don't know much about British Parliamentary Democracy.
The evolution in suffrage in the US was marked by events like the US Civil
war.
The smooth evolution of automobile technology hit a few bumps in the road,
too. Take Henry Ford and the model T.
The use of assembly lines to make cars, and to make cars that auto workers
could afford to buy was more revolutionary than anything that Marx and Lenin
ever came up with.
But then, as Our Ford used to say, "history is bunk."
| |
| pstnotpd 2004-09-02, 5:45 pm |
| Laconic2 wrote:
>
>
> Well, not really. Algol was actually a "tighter" language than C. When I
> left Algol, I went with Pascal.
>
> Then again, I thought Betamax was better than VHS.
I always heard that Betamax actually was better, but never took off due
to the restrictions on 'adult' material, which could then only be viewed
on VHS.
What does that say about C......? ;-)
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"pstnotpd" <schoenmakers@tpd.tno.nl> wrote in message
news:ch409h$p13$1@voyager.news.surf.net...
> I always heard that Betamax actually was better, but never took off due
> to the restrictions on 'adult' material, which could then only be viewed
> on VHS.
The story I heard was that Betamax delivered a better signal, but you
couldn't fit a standard length movie on one cassette.
The hassle of managing that, for the consumer, was the difference. And for
renters, it was even worse.
>
> What does that say about C......? ;-)
At the time I was ready to make the jump, I had access to a machine with
Pascal on it, so I made the natural choice.
By the time I could choose between C and Pascal, I had become accustomed
to strong typing. For the life of me, I've never been able to grasp why
strong typing didn't catch on in the C and C++ languages.
On the other hand, C offers you the chance to get cAmEl_CaSe symbolic
names wrong. Great.
| |
| pstnotpd 2004-09-02, 5:45 pm |
| Laconic2 wrote:
> The story I heard was that Betamax delivered a better signal, but you
> couldn't fit a standard length movie on one cassette.
> The hassle of managing that, for the consumer, was the difference. And for
> renters, it was even worse.
Sorry, confused it with the Philips V2000 format.
> For the life of me, I've never been able to grasp why
> strong typing didn't catch on in the C and C++ languages.
My guess is that the use of pointers and casting basically defeats
typing anyway. (I've had my share of debugging sessions where people got
these wrong)
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"pstnotpd" <schoenmakers@tpd.tno.nl> wrote in message
news:ch4j9i$1rp$1@voyager.news.surf.net...
> My guess is that the use of pointers and casting basically defeats
> typing anyway. (I've had my share of debugging sessions where people got
> these wrong)
As far as casting goes, I disagree. At least in the dialects of Pascal
that I've used, if you cast, you leave fingerprints in the source code.
It's possible for someone who looks at your source to see what you've done,
without benefit of a debugger.
I learned my first debugger in about 1962. Twenty years later, debuggers
were becoming wildly popular, on machines like the VAX, but I was
interested in going in the other direction. Nickle's worth might have put
it thus:
Instead of learning the debugger, why not invest the same effort in really
learning how to program?
I'm not sure I want to carry that big of a chip on my shoulder, but there
is a certain appeal to getting it right the fist time.
I know that when I tightened up my coding style, it was deliberate. I was
sick of debugging my own code.
Finding bugs created by other people is a whole other story.
The issue of pointers is more complex, and I don't have time to go into it
right now. I will say that I'm not entirely happy with the way Pascal
addressed the issue, but an undisciplined approach to pointers leads to
worse things. You get programs with memory leaks, or the CODASYL database
model. Basically, pointers gone wild will do for data what GOTO does for
process: it makes spaghetti out of it.
| |
| pstnotpd 2004-09-02, 5:45 pm |
| > As far as casting goes, I disagree. At least in the dialects of Pascal
> that I've used, if you cast, you leave fingerprints in the source code.
Are we talking about the same thing? It has been a while but I with C
(type)casting I mean instructing the program to view an entity of one
variable type as a different variable type. It was commonly used in my
case to read file lines into an input buffer and cast it to different
structures in one go. How would this fingerprint? Please explain,
because I've never heard it used as such.
> I learned my first debugger in about 1962.
Way before my time I'm afraid ;-)
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"pstnotpd" <schoenmakers@tpd.tno.nl> wrote in message
news:ch4n8j$d4s$1@voyager.news.surf.net...
> Are we talking about the same thing? It has been a while but I with C
> (type)casting I mean instructing the program to view an entity of one
> variable type as a different variable type. It was commonly used in my
> case to read file lines into an input buffer and cast it to different
> structures in one go. How would this fingerprint? Please explain,
> because I've never heard it used as such.
No, we are not. If I understand C correctly, you can use a variable as if
it contained whatever type value you like.
If you screwed up, well, there's always the debugger.
In Pascal, the cast operator is a "function" that casts a value into
another type.
Example:
program cast_example (output);
var a: char;
begin
a := char(65);
writeln (a)
end.
the function char(65) generates no code, when compiled. It merely instructs
the compiler to allow the resulting
value to be considered as a value of type char, instead of type integer.
If I had said:
program cast_example (output);
var a: char;
begin
a := 65;
writeln (a)
end.
I would have gotten a compiler error, because of the type mismatch between
"a" and "65". These compiler errors sound like an obstacle to rapid
software development, but they are not. By the time we get to Java, the
mainstream is starting to come around to where Pascal was in 1971, I think.
>
> Way before my time I'm afraid ;-)
>
There are only two invariants I've noted in the world of computing.
The price per pound of computers tends to remain roughly constant.
And the "stone age" ended ten years ago, no matter when the remark is made.
These two seem to remain true across the decades. It's not clear whether
price per pound is inflation adjusted or not.
Take my advice: live long and prosper, but don't get old.
| |
| Peter Koch Larsen 2004-09-02, 5:45 pm |
|
"pstnotpd" <schoenmakers@tpd.tno.nl> skrev i en meddelelse
news:ch409h$p13$1@voyager.news.surf.net...
> Laconic2 wrote:
>
I[vbcol=seagreen]
>
> I always heard that Betamax actually was better, but never took off due
> to the restrictions on 'adult' material, which could then only be viewed
> on VHS.
>
> What does that say about C......? ;-)
Wouldn't it be a brillant idea to put your ramblings in a thread where the
subject line has at least some relation to its content? And erhaps you could
continue this discussion somewhere else. Somehow you remind me of people
talking while watching a movie at a cinema. A nuisance and quite rude, come
to think of it.
Kind regards
Peter
| |
| Laconic2 2004-09-02, 5:45 pm |
|
"Peter Koch Larsen" <pklspam@mailme.dk> wrote in message
news:tgmZc.42138$Vf.2242903@news000.worldonline.dk...
> Wouldn't it be a brillant idea to put your ramblings in a thread where the
> subject line has at least some relation to its content?
Guilty as charged. I'll desist, and, if necessary start a new thread.
| |
| Paul G. Brown 2004-09-02, 5:45 pm |
| "Marshall Spight" <mspight@dnai.com> wrote in message news:<52dZc.216184$8_6.123794@attbi_s04>...
> "Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message news:57da7b56.0408310834.382e95e5@posting.google.com...
>
> An interesting assertion. Do you have anything to back it up? To my mind,
> the relational model has a lot of relevance to the questions you ask.
(*sigh*)
This misreading of what I said is *precisely* the kind of 'mental
inhibitor'
I am talkin' about. I wrote (very carefully) that 'These challenges
are
driving requirements for which the DBMS model of the world--either
SQL DBMS
or TR DBMS--is entirely unsuited.'
I carefully defined DBMS. I said 'centralized, shared repository'.
This
systems model (DBMS = DataBase Management System) is unsuited to the
two
challenges I described because the information will be not be
centralized,
nor globally shared, and it will not be placed into a repository to
be
accessed again later. It will be highly decentralized, parts of it
will be
available to different users at differnet points in time, and it
will be
entirely volatile: it will be cheaper to go back to the original
source and
ask 'what was that again?' than to store it. Whatever systems are
created to
cope with these challenges, I am willing to bet that not a single
line of
code from a SQL DBMS or a TR DBMS will be used.
But you make a mild observation about the limits that might apply to
a True
Relational DBMS, and the relational zealots glance up, bleary-eyed,
from
their row of empty shot-glasses, and start flingin' peanuts at yer
head.
Sheesh. If anyone asks after me, tell 'em I'm hangin' out on
alt.cranky-old-ex-DBMS-coots-who-are-working-on-other-problems-now-thank-you-very-much.
| |
| Marshall Spight 2004-09-02, 5:45 pm |
| "Laconic2" <laconic2@comcast.net> wrote in message news:dL6dndJaWqe--KjcRVn-ig@comcast.com...
>
> "Marshall Spight" <mspight@dnai.com> wrote in message
> news:52dZc.216184$8_6.123794@attbi_s04...
>
>
> You are right. But keep in mind that the relational model has to do with a
> broader scope of information than just DBMS systems.
But of course. :-)
Marshall
| |
| Marshall Spight 2004-09-02, 5:45 pm |
| "Paul G. Brown" <paul_geoffrey_brown@yahoo.com> wrote in message news:57da7b56.0409011040.7e14e505@posting.google.com...
>
> (*sigh*)
>
> This misreading of what I said is *precisely* the kind of 'mental
> inhibitor'
> I am talkin' about. I wrote (very carefully) that 'These challenges
> are
> driving requirements for which the DBMS model of the world--either
> SQL DBMS
> or TR DBMS--is entirely unsuited.'
>
> I carefully defined DBMS. I said 'centralized, shared repository'.
> This
> systems model (DBMS = DataBase Management System) is unsuited to the
> two
> challenges I described because the information will be not be
> centralized,
> nor globally shared, and it will not be placed into a repository to
> be
> accessed again later. It will be highly decentralized, parts of it
> will be
> available to different users at differnet points in time, and it
> will be
> entirely volatile: it will be cheaper to go back to the original
> source and
> ask 'what was that again?' than to store it. Whatever systems are
> created to
> cope with these challenges, I am willing to bet that not a single
> line of
> code from a SQL DBMS or a TR DBMS will be used.
>
> But you make a mild observation about the limits that might apply to
> a True
> Relational DBMS, and the relational zealots glance up, bleary-eyed,
> from
> their row of empty shot-glasses, and start flingin' peanuts at yer
> head.
>
> Sheesh. If anyone asks after me, tell 'em I'm hangin' out on
> alt.cranky-old-ex-DBMS-coots-who-are-working-on-other-problems-now-thank-you-very-much.
You have apparently mistaken me for someone else.
Distributed doesn't necessarily mean decentralized. There are
benefits to centralization that are fundamental; administration
for one. On the other hand, loosely-federated distributed systems
have some interesting possibilities that a centralized approach
does not.
It sounds like we might be able to have an interesting and productive
conversation if you are willing to drop the drama queen act.
Marshall
| |
| pstnotpd 2004-09-02, 5:45 pm |
| Yep, sorry 'bout that.
[vbcol=seagreen]
|
|
|
|
|