Javadogs,

I'd like to add a line to the specification that a portable mapping  
for arrays, collections, and maps will include a primary key on the  
join table. There is not always a way for the JDO implementation to  
guess what the primary key constraints should be for a join table.

The issue came up with the TCK where we were trying to get JPOX to  
recognize that an embedded PersistenceCapable in a join table can  
have a unique identifying field. In the case of datastore identity,  
this doesn't work because there is no primary key field in the  
PersistenceCapable.

<proposed 18.14>
A portable mapping for arrays, collections, and maps will include a  
primary key on the join table.
</proposed 18.14>

This is discussed on the Apache JDO JIRA page http:// 
issues.apache.org/jira/browse/JDO-211 if you would like more  
information.

Craig

Test*MapStringValueCollections there is a mismatch between orm and  
schema.

All
Comments
Change History
Subversion Commits

Sort Order: 
Comment by Michelle Caisse [12/Nov/05 02:35 AM]
[ Permlink ]

PC fields that were specified embedded in the orm were a reference to 
a join table in the schema, and vice versa. I fixed the mapping  
problem with revision 332699.

However, JPOX still wants an adapter index column in the schema for  
embedded PC fields (see JDO-121 for a discussion). Since Map does not 
allow duplicates, and MapStringKeyCollections pass without an order  
column such as we used to map Collections, it seems that this is a  
JPOX bug.

Comment by Andy Jefferson [12/Nov/05 07:16 AM]
[ Permlink ]

Hi Michelle,
I'm not convinced that it's a JPOX bug here. I'll explain why

Let's take the example of MapStringValueCollections, and its field  
"MapOfObject_String3". This is defined as
a Map field with join table and with key of SimpleClass, value of  
String. Both key and value are embedded, so the join table will have
* FK back to the owner table (IDENTIFIER)
* column for the value
* column(s) for the key fields
This is all well and good, but is missing one thing. JPOX needs to  
put a PK on the join table. What are you proposing that it uses ? It  
doesnt have an id for the (embedded) SimpleClass key. So it needs  
something else - and in this case it adds this ADPT_PK_IDX column.

Is there a field in the key class (SimpleClass) that would give  
uniqueness ? e.g "ID" ? In that case you should define the <primary- 
key> on the join table to be formed by IDENTIFIER and ID. This should 
be enough to stop JPOX adding its own PK adapter column (and if it  
doesnt prevent it then it is indeed a bug in JPOX). If there isn't a  
field of that nature then you have to have the ADPT_PK_IDX, or the  
JDO impl needs to be told somehow that it should not create a PK on  
the join table (which should be the last option IMHO since its a best 
practice to have one).

I noticed that in the TestHashSetCollections/TestSetCollections you  
added <order> which avoided this issue. This means that you have a  
HashSet/Set that can't contain duplicates (since its a Set), and a  
datastore representation that *can* have dups. Using a join table  
primary-key specification (like that above) there would have been  
better IMHO.

Comment by Craig Russell [13/Nov/05 11:29 PM]
[ Permlink ]

1. Let's take the example of application identity for  
MapStringValueCollections, and its field "MapOfObject_String3".

The key for this map is SimpleClass. The metadata for SimpleClass  
says that the primary key field of the class is "id". So JPOX knows  
that there is a key object embedded in the join table that has a  
primary key field, and that field can be used as the primary key of  
the join table.

2. Take the case of datastore identity for MapStringValueCollections, 
and its field "MapOfObject_String3".

The key for this map is SimpleClass. The metadata for SimpleClass  
doesn't identify a primary key (duh. It's datastore identity). So  
JPOX has no idea that there is a key field in the key object.  
Therefore, the orm metadata needs to identify a key column for the  
join table, and I'd suggest using the id column as the key, as Andy  
suggests.

Sadly, the HashSetCollections and SetCollections have <order> columns 
which is not correct. There should not be an ordering specified for  
the join table since duplicates are not allowed. So the mapping for  
application identity should not need a primary key but the datastore  
identity mapping does need a primary key (and again, I'd use the id  
field of SimpleClass as the join table key column.

Comment by Andy Jefferson [14/Nov/05 07:03 PM]
[ Permlink ]

Agree with Craig wrt app identity. JPOX should (and indeed does in  
JPOX CVS - builds 15/11/2005 onwards) use the app id fields of the  
key as part of the PK when the user hasn't specified the PK, so no  
need to add the PK spec for that identity case.

It shows up some schema mapping issues. For example in  
HashMapStringValueCollections (app identity) there is a field  
HashMapOfSimpleClass_String12 that has an embedded key (SimpleClass), 
which has no <embedded> definition, and which has a schema consisting 
of a single column for that embedded key.

It also gets through to the actual test of expected versus actual on  
MapStringValueCollections (app identity) with the failure being in  
the comparison process - the values are correct by my reckoning

Comment by Michelle Caisse [17/Nov/05 06:42 PM]
[ Permlink ]

Checked in various metadata changes that resolve some of the problems 
in application identity. -- revision 345289.

Comment by Michelle Caisse [17/Nov/05 06:45 PM]
[ Permlink ]

The attached patch adds a pk to the id column of join tables for  
SimpleClass fields for datastore identity. JPOX still gives the same  
error with this patch applied:
SQL Exception: 'ADPT_PK_IDX' is not a column in table or VTI  
'DATASTOREIDENTITY0.HASHMAP_OBJECT3_STR'.
SQL Exception: 'ADPT_PK_IDX' is not a column in table or VTI  
'DATASTOREIDENTITY0.HASHTBL_OBJECT3_STR'.
SQL Exception: 'ADPT_PK_IDX' is not a column in table or VTI  
'DATASTOREIDENTITY0.MAP_OBJECT3_STR'.
SQL Exception: 'ADPT_PK_IDX' is not a column in table or VTI  
'DATASTOREIDENTITY0.TREEMAP_OBJECT3_STR'.

Comment by Craig Russell [18/Nov/05 06:15 PM]
[ Permlink ]

This appears to be a JPOX issue assuming a column that doesn't exist.

Comment by Michelle Caisse [19/Nov/05 01:52 AM]
[ Permlink ]

Committed patch with revision 345615.

Comment by Andy Jefferson [21/Nov/05 07:09 PM]
[ Permlink ]

I see that the MetaData is now, for example
<field name="HashMapOfObject_String3" table="HASHMAP_OBJECT3_STR">
<join column="IDENTIFIER"/>
<key>
<embedded>
<field name="id" primary-key="true" column="ID"/>
<field name="intField" column="INTVAL"/>
<field name="stringField" column="STRINGVAL"/>
</embedded>
</key>
<value column="VALUEVAL"/>
</field>

That certainly doesnt agree with my definition of a PK specification  
for the join table. It should be (IMHO) like this

<join>
<primary-key>
<column name="IDENTIFIER"/>
<column name="ID"/>
</primary-key>
</join>
which nicely defines the colums to be used for the join table. Using  
fields of an embedded object and specifying primary-key there is not  
logical to me, and not supported by JPOX either. The JPOX online docs
http://www.jpox.org/docs/1_1/constraints.html
provides what we support for PK definition, which is that above.

Comment by Craig Russell [21/Nov/05 08:16 PM]
[ Permlink ]

Combining the metadata for the field, join, key, and value, I come up 
with this:

<field name="HashMapOfObject_String3" table="HASHMAP_OBJECT3_STR">
<join column="IDENTIFIER"/>
<primary-key>
<column name="IDENTIFIER"/>
<column name="ID"/>
</primary-key>
</join>
<key>
<embedded>
<field name="id" column="ID"/>
<field name="intField" column="INTVAL"/>
<field name="stringField" column="STRINGVAL"/>
</embedded>
</key>
<value column="VALUEVAL"/>
</field>

This should be a standard, portable mapping for the map<embedded  
SimpleClass, embedded String> case.

I will propose to the expert group that a portable mapping for  
arrays, collections, and maps will include a primary key on the join  
table.

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!

Attachment:
This has been downloaded 0 time(s).

[ Post a follow-up to this message ]

Abe,

Check the below example

Owner owner = new Owner("Ted");
owner.add(new Dog("one"));
owner.add(new Dog("two"));

TABLE OWNER
ID NAME
1 Ted

TABLE OWNER_DOG
OWNER_ID NAME
1 one
1 two

If owner_id is primary key and nothing else, we will not be able to store mo
re
than one instance of Dog for each owner. Thus, we need a new column to be pa
rt
of the primary key.

TABLE OWNER_DOG
OWNER_ID UNIQUENESS_ID NAME
1 1 one
1 2 two


Adding one more column will help if we dont have duplicates, but with duplic
ates
we need 3 columns.

Example

Owner owner = new Owner("Ted");
Dog one = new Dog("one");
Dog two = new Dog("two");

owner.add(one);
owner.add(one);
owner.add(two);

The only way I can think it to work is with the following schema

TABLE OWNER
ID NAME
1 Ted

TABLE OWNER_DOG
OWNER_ID UNIQUENESS_ID EMBEDDED_ID NAME
1 1 1 one
1 2 1 one
1 3 2 two



Quoting Abe White <awhite@bea.com>:
 
>
> Why?  I don't understand this at all.  "Datastore identity" is
> meaningless in this context, as embedded objects have no identity
> according to the spec.
>

[ Post a follow-up to this message ]

Hi Abe,

On Nov 21, 2005, at 12:59 PM, Abe White wrote:
 
>
> Why?  I don't understand this at all.  "Datastore identity" is
> meaningless in this context, as embedded objects have no identity
> according to the spec.

Right. They have no persistent identity. Therefore, if you want to
use an embedded PersistenceCapable as a key, you need to identify one
of the embedded fields as a key for the purpose of the map. The
alternative is to simply disallow using an embedded
PersistenceCapable as a key in a map if  the class is defined as
datastore identity. But this seems pretty arbitrary.

Craig

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!

[ Post a follow-up to this message ]

Just one more comment. The fact that a PersistenceCapable should use
datastore identity is orthogonal to whether it contains a field that
can uniquely identify the instance. It is considered best practice in
some circles to use a surrogate key for identifying instances in the
datastore regardless of whether there is a field that is unique. So I
don't see the need for arbitrarily restricting the use case simply
because there is no persistent identity for the key instances.

Craig

On Nov 21, 2005, at 2:43 PM, Craig L Russell wrote:

> Hi Abe,
>
> On Nov 21, 2005, at 12:59 PM, Abe White wrote:
> 
>
> Right. They have no persistent identity. Therefore, if you want to
> use an embedded PersistenceCapable as a key, you need to identify
> one of the embedded fields as a key for the purpose of the map. The
> alternative is to simply disallow using an embedded
> PersistenceCapable as a key in a map if  the class is defined as
> datastore identity. But this seems pretty arbitrary.
>
> Craig
>
> Craig Russell
> Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
> 408 276-5638 mailto:Craig.Russell@sun.com
> P.S. A good JDO? O, Gasp!
>

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!

[ Post a follow-up to this message ]

> The issue came up with the TCK where we were trying to get JPOX to
> recognize that an embedded PersistenceCapable in a join table can
> have a unique identifying field. In the case of datastore identity,
> this doesn't work because there is no primary key field in the
> PersistenceCapable.
>
> <proposed 18.14>
> A portable mapping for arrays, collections, and maps will include a
> primary key on the join table.
> </proposed 18.14>

Why?  I don't understand this at all.  "Datastore identity" is
meaningless in this context, as embedded objects have no identity
according to the spec.

[ Post a follow-up to this message ]

Hi Abe,

I think I now understand your point, which I missed earlier. You
don't see a need for a primary key to be defined on a join table.

The issue is whether a compliant JDO implementation can require that
a join table must have a primary key if it is used to implement a
multi-valued field (collection or map).

Apparently, JPOX requires this, and there is nothing in the spec
currently that addresses the issue at all. Unless we change the spec
to disallow the JPOX implementation, it seems that it is de facto a
portability issue.

What are the implications of not having a primary key in the join
table? The constraint that the Map keys (or Set elements) must be
unique must be enforced by the application (the JDO implementation).
This implies that the entire Map or Set must be instantiated in
memory in order to prevent duplicates. [I don't see how to implement
a lazy loading strategy for such Maps or Sets.]

What else?

Craig

On Nov 21, 2005, at 6:42 PM, Abe White wrote: 
>
> Or the table could have *no primary key at all*, which is what I'm
> arguing should be perfectly portable.
> 
>
> The only requirements for using objects in a Map in Java are that
> they implement hashCode and equals properly.  How do you go from
> that contract to "needs a primary key in the database map table"?

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!

[ Post a follow-up to this message ]

> If owner_id is primary key and nothing else, we will not be able to
> store more
> than one instance of Dog for each owner. Thus, we need a new column
> to be part
> of the primary key.

Or the table could have *no primary key at all*, which is what I'm
arguing should be perfectly portable.

[ Post a follow-up to this message ]

> I think I now understand your point, which I missed earlier. You
> don't see a need for a primary key to be defined on a join table.
> Apparently, JPOX requires this, and there is nothing in the spec
> currently that addresses the issue at all. Unless we change the spec
> to disallow the JPOX implementation, it seems that it is de facto a
> portability issue.

Hi Craig,

I mentioned this in one of the original JIRA issues on this subject. Either 
we
define a <primary-key> (to fully-specify the columns to be used, as you say
in your proposed wording) or we have a way of defining that no PK is to be
defined for that join table. I've no problem with allowing both, which is
what Abe is asking for.

JPOX can implement both, but the spec doesn't define how to say that you hav
e
no PK, so consequently how can we ?

We could interpret no <primary-key> element as saying no PK, but I'm of the
opinion that having a PK in a join table is a "best practice" and should be
the default behaviour. Consequently I want a different way to say that there
is no PK. Maybe something like

<join primary-key="false">


--
Andy
Java Persistent Objects - JPOX

[ Post a follow-up to this message ]

> Right. They have no persistent identity. Therefore, if you want to
> use an embedded PersistenceCapable as a key, you need to identify
> one of the embedded fields as a key for the purpose of the map.
> The alternative is to simply disallow using an embedded
> PersistenceCapable as a key in a map if  the class is defined as
> datastore identity. But this seems pretty arbitrary.

The only requirements for using objects in a Map in Java are that
they implement hashCode and equals properly.  How do you go from that
contract to "needs a primary key in the database map table"?

[ Post a follow-up to this message ]

Hi Andy,
 
>
>
> Hi Craig,
>
> I mentioned this in one of the original JIRA issues on this subject. Eithe
r we
> define a <primary-key> (to fully-specify the columns to be used, as you sa
y
> in your proposed wording) or we have a way of defining that no PK is to be
> defined for that join table. I've no problem with allowing both, which is
> what Abe is asking for.

I agree JDO should allow both.

>
> JPOX can implement both, but the spec doesn't define how to say that you h
ave
> no PK, so consequently how can we ?
>
> We could interpret no <primary-key> element as saying no PK, but I'm of th
e
> opinion that having a PK in a join table is a "best practice" and should b
e
> the default behaviour. Consequently I want a different way to say that the
re
> is no PK. Maybe something like
>
> <join primary-key="false">

I'm not sure whether having a PK in a join table is "best practice" in
general. I think this depends on the data structure. I agree in case of
a map, because the key needs to be unique and the same holds true for
sets. But for others the PK on the join table does not add much value,
does it?

I'm in favor of interpreting a missing <primary-key> element as there is
no primary key on the join table.

Regards Michael
--
Michael Bouschen		Tech@Spree Engineering GmbH
mailto:mbo.tech@spree.de	http://www.tech.spree.de/
Tel.:++49/30/235 520-33		Buelowstr. 66
Fax.:++49/30/2175 2012		D-10783 Berlin

[ Post a follow-up to this message ]