|
Home > Archive > Apache JDO Project > July 2005 > Embedded collections of non-PC objects
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Embedded collections of non-PC objects
|
|
| Andy Jefferson 2005-07-12, 5:45 pm |
| Hi Michelle,
> Now we get an error with collection types where the field is embedded
> but jpox is looking for a table for the field:
> Iteration request failed : SELECT THIS.BIGDECIMAL_ELE FROM
> applicationidentity0.COLLECTION_OF_BIG_DECIMAL19 THIS WHERE 1 =
> THIS.IDENTIFIER_OID
> NestedThrowablesStackTrace:
> ERROR 42X05: Table 'APPLICATIONIDENTITY0.COLLECTION_OF_BIG_DECIMAL19'
> does not exist.
JPOX persists collections of non-PC's in a join table. Is there a join table
here ? because you have none in your metadata.
Are you expecting us to create some massive BLOB column and dump all of the
collection into that ? because we don't support that currently, and I don't
have any requests from users to do such a thing (since querying of the
elements would be getting close to impossible).
--
Andy
Java Persistent Objects - JPOX
| |
| Michelle Caisse 2005-07-12, 5:45 pm |
| Andy Jefferson wrote:
>JPOX persists collections of non-PC's in a join table. Is there a join table
>here ? because you have none in your metadata.
>
>Are you expecting us to create some massive BLOB column and dump all of the
>collection into that ? because we don't support that currently, and I don't
>have any requests from users to do such a thing (since querying of the
>elements would be getting close to impossible).
>
>
>
Hi Andy,
Yup, that's the general idea. We create the BLOB column in the schema.
We are testing cases where these collections are embedded-element=true,
embedded-element=false, and default (also not embedded). We have join
tables for the non-embedded fields and columns in a table for the
embedded fields.
-- Michelle
| |
| Craig Russell 2005-07-12, 5:45 pm |
| | |
| Andy Jefferson 2005-07-13, 2:45 am |
| > JDO2 defines an attribute "serialized" that specifies whether the
> contents of a field should be serialized and dumped into a single
> column, and therefore not queryable. It does allow for any
> serializable data to be stored, regardless of whether the data is
> persistence-capable or not.
>
> So I think it is a requirement for the Reference Implementation...
Of course but the "serialized" attribute is not yet supported, so I suggest
you look at other tests until someone has the time to implement it in JPOX.
--
Andy
Java Persistent Objects - JPOX
| |
| Andy Jefferson 2005-07-14, 5:45 pm |
| > So I think it is a requirement for the Reference Implementation...
> To help clarify this for the TCK, we can/should add the "serialized"
> attribute to the columns in the metadata that are embedded.
Next nightly build of JPOX will have basic support for fields of
collections/maps declared as "serialized". This should get around the initial
schema issues about join tables/columns etc.
--
Andy
Java Persistent Objects - JPOX
| |
| Michelle Caisse 2005-07-14, 5:45 pm |
| That's great, Andy. I will add the serialized declarations to the metadata.
Thanks,
Michelle
Andy Jefferson wrote:
>
>Next nightly build of JPOX will have basic support for fields of
>collections/maps declared as "serialized". This should get around the initial
>schema issues about join tables/columns etc.
>
>
>
>
| |
| Craig Russell 2005-07-14, 5:45 pm |
| | |
| Andy Jefferson 2005-07-14, 5:45 pm |
| Hi Craig,
> Is it possible to default the value of the <field> serialized
> attribute to "true" for embedded non-PC types? That would immediately
> solve our current Collection, Set, and Map issues, without having
> extra stuff in the metadata file.
Anything is possible ...
> The .jdo file specifies the embedded-element, embedded-key, and
> embedded-value attributes for the relevant fields, so simply changing
> the defaults would make it all work.
>
> I looked at the spec again and it doesn't look like the serialized
> attribute should be required, so it would be better if it were
> defaulted
> <spec>
> The embedded attribute specifies whether the field should be stored
> as part of the containing instance instead of as its own instance in
> the datastore. It must be specified or default to "true" for fields
> of primitive types, wrappers, java.lang, java.math, java.util,
> collection, map, and array types specified above; and "false"
> otherwise. While a compliant implementation is permitted to support
> these types as first class instances in the datastore, the semantics
> of embedded=3D=E2=80=9Dtrue=E2=80=9D imply containment. That is, the embe=
dded instances
> have no independent existence in the datastore and have no Extent
> representation.
> The semantics of embedded applied to collection, map, and array types
> applies to the structure of the type, not to the elements, keys, and
> values. That is, the collection itself is considered separate from
> its contents. These may separately be specified to be embedded or
> not, using embedded-element, embedded-key, and embedded-value
> attributes of the collection, array, and map elements.
> The embedded attribute applied to a field of a persistence-capable
> type is a hint to the implementation to treat the field as if it were
> a Second Class Object. But this behavior is not further specified and
> is not portable.
> </spec>
Well this does beg the question, where is the difference between "serialize=
d"=20
and "embedded" at <field> level then ?
If I specify <field embedded=3D"true"> for a collection, is this the same =
as=20
<field serialized=3D"true"> for the collection ? "serialized" doesn't have=
a=20
description in 18.13 btw.
Personally I would expect a Collection<String> or Map<String, String> to=20
default to having the elements queryable, and embedding them (by default) i=
s=20
not going to give me that. Perhaps I just disagree with the spec on this=20
issue :-)
Perhaps some examples would be beneficial ? (in the spec). Not that I'm try=
ing=20
to increase your workload :-0. I see one example of "embedded" and that is=
=20
<embedded> defining fine details of how to embed an object (which is what I=
=20
would expect to use this keyword for anyway), and which seems a long way fr=
om=20
what we're talking about here. If I'm not understanding this then you can b=
et=20
SUN's vast bank balance that users will have a hard time working it out, an=
d=20
I'm not going to spend my spare time explaining it to them
=2D-=20
Andy
Java Persistent Objects - JPOX
| |
| Andy Jefferson 2005-07-15, 2:45 am |
| > > Is it possible to default the value of the <field> serialized
Hi Craig,
Having gone back to the original problem report, I'll give my interpretation
of what the various embedded attributes means (and what I've implemented),
and you can tell me where I've gone wrong :-)
Example 1 : Collection of BigDecimal
1. Basic collection
<field name="myfield">
<collection element-type="java.math.BigDecimal"/>
<join/>
</field>
This creates 2 tables - 1 for the class owning "myfield", and 1 join table to
contain the elements. If <join> is omitted then an error should be thrown
(though i'm not sure if JPOX currently flags this up)
2. Serialised collection
<field name="myfield" serialized="true">
<collection element-type="java.math.BigDecimal"/>
<join/>
</field>
This creates 1 tables, with a BLOB column for "myfield" containing all
elements.
embedded-element has no effect with this example because the element
(BigDecimal) is already embedded (in the join table), and has no way of not
being embedded.
================================
Example 2 : Collection of PC
1. Basic collection with join
<field name="myfield">
<collection element-type="MyElement"/>
<join/>
</field>
This creates 3 tables - 1 for the class owning "myfield", 1 join table and 1
for the element class.
2. Basic collection with FK
<field name="myfield">
<collection element-type="MyElement"/>
</field>
This creates 3 tables - 1 for the class owning "myfield", 1 for the element
class (with FK back to the owner table).
3. Embedded element
<field name="myfield">
<collection element-type="MyElement" embedded-element="true"/>
<join/>
</field>
This creates 3 tables - 1 for the class owning "myfield", and 1 join table
containing the elements (columns aligned with the fields of the PC element).
4. Embedded element (fully specified)
<field name="myfield">
<collection element-type="MyElement" embedded-element="true"/>
<join/>
<element>
<embedded>
... (full spec of field mappings)
</embedded>
</element>
</field>
This creates 3 tables - 1 for the class owning "myfield", and 1 join table
containing the (embedded) elements (columns aligned with the mappings
specified in the <embedded> section above.
5. Embedded collection
<field name="myfield" embedded="true">
<collection element-type="MyElement"/>
</field>
No idea what is expected here .. maybe the same as "serialised" below ?
6. Serialised collection
<field name="myfield" serialized="true">
<collection element-type="MyElement"/>
</field>
This creates 1 table, with 1 BLOB column for "myfield" to include the
serialised collection.
================================
Example 3 : PC with PC field
1. Basic 1-1
<field name="myfield" persistence-modifier="persistent"/>
This creates 2 tables, one for each class, with a FK
2. Embedded PC
<field name="myfield" persistence-modifier="persistent" embedded="true">
</field>
This creates 1 table, with all fields of the PC having their own columns in
the table of the owner PC. Not actually implemented in JPOX (though it will
be fast to do)
3. Embedded PC (fully specified)
<field name="myfield" persistence-modifier="persistent">
<embedded>
.... (mappings for other PC class)
</embedded>
</field>
This creates 1 table, with the fields of the other PC having their own columns
in the table of the owner PC
4. Serialised PC
<field name="myfield" persistence-modifier="persistent" serialised="true"/>
Not yet implemented. I'd assume we just dump the whole of the PC into a BLOB
column, but don't see much use for this :-)
================================
There are complete details of all of these in the JPOX online docs, showing
the classes, the MetaData, and the resultant DB schema.
http://www.jpox.org/docs/1_1/serialised.html
http://www.jpox.org/docs/1_1/embedded.html
http://www.jpox.org/docs/1_1/relati...collection.html
http://www.jpox.org/docs/1_1/relationships_1_1.html
If there's some misunderstanding above of what the spec wants, I'd rather know
it now :-)
[vbcol=seagreen]
> <spec>
> The embedded attribute specifies whether the field should be stored
> as part of the containing instance instead of as its own instance in
> the datastore. It must be specified or default to "true" for fields
> of primitive types, wrappers, java.lang, java.math, java.util,
> collection, map, and array types specified above; and "false"
> otherwise.
> </spec>
I'm ok with this statement for all types *except* Collections/Maps.
Collections/Maps have methods therefore I can't see why they should be
"embedded" by default - they have methods (that we can mimic in JDOQL)
therefore we want to query them. Arrays don't have methods (that we can mimic
in JDOQL) therefore they can be embedded (in a BLOB column). Really depends
on what "embedded" means at <field> level ... is it serialised or something
else ?
--
Andy
| |
| Andy Jefferson 2005-07-15, 7:45 am |
| Some corrections to the previous attempt (due to excessive use of the=20
Ctrl-C-Ctrl-V design pattern) :-
> Example 1 : Collection of BigDecimal
I accidentally omitted this option
3. Embedded collection
<field name=3D"myfield" embedded=3D"true">
=C2=A0 =C2=A0 <collection element-type=3D"java.math.BigInteger"/>
</field>
No idea what is expected here .. probably the same as "serialised" option ?=
=20
Not currently implemented by JPOX.
> Example 2 : Collection of PC
> 2. Basic collection with FK
> <field name=3D"myfield">
> <collection element-type=3D"MyElement"/>
> </field>
This creates *2* tables - 1 for the class owning "myfield", 1 for the eleme=
nt=20
class (with FK back to the owner table).
> 3. Embedded element
> <field name=3D"myfield">
> <collection element-type=3D"MyElement" embedded-element=3D"true"/>
> <join/>
> </field>
This creates *2* tables - 1 for the class owning "myfield", and 1 join table
containing the elements (columns aligned with the fields of the PC element).
> 4. Embedded element (fully specified)
> <field name=3D"myfield">
> <collection element-type=3D"MyElement" embedded-element=3D"true"/>
> <join/>
> <element>
> <embedded>
> ... (full spec of field mappings)
> </embedded>
> </element>
> </field>
This creates *2* tables - 1 for the class owning "myfield", and 1 join tabl=
e=20
containing the (embedded) elements (columns aligned with the mappings=20
specified in the <embedded> section above.
=2D-=20
Andy
| |
| Craig Russell 2005-07-16, 8:45 pm |
| | |
| Andy Jefferson 2005-07-17, 7:45 am |
| Hi Craig,
thanks for your reply and your insights.
>
> The join element has no defaults, so this is not sufficient to
> describe the mapping. You need at least a column attribute naming the
> join column. And you need to name the column in the join table to map
> the BigDecimal values to. So,
>
> <field name="myfield" column="VALUES" table="MYFIELD_TABLE">
> <collection element-type="java.math.BigDecimal"/>
> <join column="JOIN_COLUMN"/>
> </field>
I don't necessarily agree here. We have to qualify the statement with the
following
New schema : The JDO impl is perfectly capable of providing default namings
for columns and perfectly capable of choosing the join columns ... since it
has the (PK) columns in the main table. It provides default namings for
columns in other situations.
Existing schema : The user should specify the columns and table as you stated.
>
> I'd say that serialized implies embedded-element (and vice-versa,
> which is why I'm now questioning the value of serialized as an
> attribute).
OK. That wasn't my interpretation. I see 2 levels of embedded. We have
"embedded" at field level, and "embedded-element" (or -key, -value) at
collection/map level. I see embedded-element/key/value as saying that we want
to embed the elements/keys/values in a join table (like in the example 9 in
the spec), and embedded (at field level) saying that we want to embed the
whole collection/map into the main table.
>
> Since it's embedded, I think there is only one table that contains
> all the fields in the class, including the Collection of MyElement.
>
> You can't map the columns of an embedded Collection of PC elements
> because you would need one column for each field in each PC, which is
> a variable number of columns. And tables have a fixed number of
> columns. So the mapping has to either serialize the Collection and
> store it into a BLOB column or use another table. For embedded
> collection,
As my comment above, I didn't interpret (embedded-element/key/value) like
this. I'll try to justify this, with a map this time :-) ...
1. We have a map with embedded-key=false, embedded-value=false. This ends up
with a main table, and a join table, and optionally (if the keys/values are
PC), tables for key and value. No disagreement there.
2. We have a map with embedded-key=true, embedded-value=false. How would you
store these ? Would you have a BLOB column for the map keys, and have the map
values stored off in their own table (since they aren't embedded) ? This
would make managing the map a bit tricky for the JDO impl (to say the
least!). I would store the keys as embedded into the join table (as per
example 9 in the spec - multiple columns in the join table lining up with the
fields in the key), and have the values in their own table (if PC) with a FK
from the join table. This makes it simple for the JDO impl to manage the map
since the keys and values are stored in the join table.
3. We have a map with embedded-key=true, embedded-value=true. How would you
store this ? Would you store them as a single BLOB column in the main table.
I would store the key AND value in the join table (as per example 9 in the
spec - so we gain columns for key, and columns for value in the join table).
As for your point above about variable number of columns, well example 9 in
the spec is just this case. It embeds the element of a collection into a join
table. This is "embedded" because the elements are not stored as FCO's - they
are embedded into the join table (which is effectively a secondary table
owned by the main table - and which represents the collection).
Now if the user specifies <field embedded="true"> then I would expect to have
to store the whole collection/map as a single BLOB column (like serialized,
so why do we have a serialized attribute too ?)
I'd welcome any clarification from the people in the know who spent a lot of
time designing these levels of specification, and from the JDO vendors that
have supported such embedding for some time on what we interpret these
attributes as.
> It's been on my to-do list for the specification for a while to add
> mapping for arrays, lists, sets, and maps to Chapter 15. This might
> be the time to actually do it.
That would be great. The spec covers many many situations and the metadata is
largely intuitive as to what people specify, but I feel we're missing
clarification on this one part.
Thanks!
--
Andy
Java Persistent Objects - JPOX
| |
| Geoff hendrey 2005-07-17, 5:45 pm |
| --- Andy Jefferson <andy@jpox.org> wrote:
> Hi Craig,
>
> thanks for your reply and your insights.
>
> element-type="java.math.BigDecimal"/>
> "myfield", and 1
> an error should be
> this up)
> sufficient to
> attribute naming the
> the join table to map
> table="MYFIELD_TABLE">
> element-type="java.math.BigDecimal"/>
>
> I don't necessarily agree here. We have to qualify
> the statement with the
> following
> New schema : The JDO impl is perfectly capable of
> providing default namings
> for columns and perfectly capable of choosing the
> join columns ... since it
> has the (PK) columns in the main table. It provides
> default namings for
> columns in other situations.
> Existing schema : The user should specify the
> columns and table as you stated.
Andy, I totally appreciate that you are trying to
provide your users with good forward mapping tools,
but I agree with Craig.
Personally I think the approach to forward schema
genration should be to allow the user to start with a
plain .jdo file (no ORM elements). The tool should
then produce a default ORM file that looks as Craig
describes. An empty <join/> element is going to be a
source of confusion.
I've looked at all the examples in your email, and
lined them up with the spec and I agree with Craig's
interpretations. Keep up the good work, and Go JPOX!
-geoff
________________________________________
____________
Start your day with Yahoo! - make it your home page
http://www.yahoo.com/r/hs
| |
| Craig Russell 2005-07-18, 5:45 pm |
| |
|
|
|
|