The issue about "A truncation error was encountered trying to shrink CHAR
'65535' to length 1" should now be fixed with latest JPOX. JPOX CVS now uses
default "jdbc-type" as per the most recent JDO spec. In particular for this
error, when the Java type is Character/char the jdbc-type chosen will be
CHAR(1) (where the RDBMS supports it, which Derby does).

Test errors/failures previously were
61 for application-id, 60 for datastore-id.

Now we have
51 for application-id, 46 for datastore-id.

and moreover, of the remaining errors there are 15 for application-id, and 1
1
for datastore-id giving "SQL Exception: Invalid character string format for
type INTEGER". This occurs where the schema has been defined with an INTEGER
column to represent a Character/char(/String) Java type (maybe to allow for
previous JPOX behaviour) and implies that the schema needs some updates to
match the default jdbc-type expected by the JDO2 spec (and now also by JPOX)
.

Schema where column is INTEGER but should be CHAR(1)
FieldsOfCharacter : table FIELDSOFCHARACTER has INTEGER columns
FieldsOfPrimitivechar : table FIELDSOFPRIMITIVECHAR has INTEGER columns

Schema where column is INTEGER but should be VARCHAR(...)
HashMapStringKeyCollections, HashMapStringValueCollections,
MapStringKeyCollections, MapStringValueCollections,
TreeMapStringKeyCollections, TreeMapStringValueCollections : join tables hav
e
the key as INTEGER instead of VARCHAR

There may be others of the same nature that I've not listed, just wanted to
give a head start on fixing the schema.


Also, this change may affect other JIRA issues, but sadly
"http://issues.apache.org" is about as unresponsive a server as I've ever
come across, so we'll have to leave that for others to work out ;-)

--
Andy

[ Post a follow-up to this message ]

Hi Michelle, Andy,

sorry I did not reply earlier.

I talked to Craig about this issue yesterday. We agreed on leaving
tables "FIELDSOFCHARACTER", "FIELDSOFPRIMITIVECHAR", and
"PCPointSingleFieldChar" as they are keeping the INTEGER columns.

The rationale is that pc class instances mapped to those tables have
character values which are initialized based on numerical values rather
than character values, e.g. "Character.MAX_VALUE",
"Character.MIN_VALUE", or "System.currentTimeMillis()%Character.MAX_VALUE".

For this reason, we decided to adapt the ORM metadata rather than to
adapt the schema: <field> elements corresponding to fields of type
"char" or "java.lang.Character" will have "jdbc-type" attributes, e.g.

<class name="PCPointSingleFieldPrimitivechar"
table="PCPointSingleFieldChar">
<field name="id" primary-key="true">
<column name="ID" jdbc-type="INTEGER"/>
</field>
..
</class>

For pc classes like "HashtableStringValueCollections" (etc.) the story
is slightly different:

They have several fields of type "java.util.Map" where (sometimes) keys
are FCOs (e.g. SimpleClass instances) and values are strings. The types
of keys and values differ and can be derived by the classname.

Each of these fields is mapped to a separate (join) table having join
columns, key columns, value columns, and index columns. In the current
schema, all of these columns have type "INTEGER".

This is a bug: Keys and values of type "java.lang.String" must be mapped
to columns of type "VARCHAR". Furthermore, join tables which are mapped
by fields of maps or sets do not have index columns. Question: Do we
have to distinguish bewteen sorted maps/sets vs. non-sorted maps/sets?

I'll prepare two patches today fixing these issues - one for each
identity type.

Regards,
Michael

> Hi Andy,
>
> This is really good, a big improvement in the error count.
>
> Andy Jefferson wrote:
> 
> I am fixing these.
> 
> I agree that the KEYVAL columns in *StringKeyCollections and the
> VALUEVAL columns of *StringValueCollections should be VARCHAR.  We have
> some other fixes we need to make to the mapping for these, so I'll make
> these changes with those.
> 
> The server seems to have come to life again.  I will update the issues
> soon.
>
> -- Michelle


--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]

Michael,

Where are the numerical value initializations that you speak of for
FieldsOfCharacter and FieldsOfPrimitivechar?  I don't see it.

-- Michelle

Michael Watzek wrote:

> Hi Michelle, Andy,
>
> sorry I did not reply earlier.
>
> I talked to Craig about this issue yesterday. We agreed on leaving
> tables "FIELDSOFCHARACTER", "FIELDSOFPRIMITIVECHAR", and
> "PCPointSingleFieldChar" as they are keeping the INTEGER columns.
>
> The rationale is that pc class instances mapped to those tables have
> character values which are initialized based on numerical values
> rather than character values, e.g. "Character.MAX_VALUE",
> "Character.MIN_VALUE", or
> "System.currentTimeMillis()%Character.MAX_VALUE".
>
> For this reason, we decided to adapt the ORM metadata rather than to
> adapt the schema: <field> elements corresponding to fields of type
> "char" or "java.lang.Character" will have "jdbc-type" attributes, e.g.
>
>     <class name="PCPointSingleFieldPrimitivechar"
> table="PCPointSingleFieldChar">
>       <field name="id" primary-key="true">
>         <column name="ID" jdbc-type="INTEGER"/>
>       </field>
>       ...
>     </class>
>
> For pc classes like "HashtableStringValueCollections" (etc.) the story
> is slightly different:
>
> They have several fields of type "java.util.Map" where (sometimes)
> keys are FCOs (e.g. SimpleClass instances) and values are strings. The
> types of keys and values differ and can be derived by the classname.
>
> Each of these fields is mapped to a separate (join) table having join
> columns, key columns, value columns, and index columns. In the current
> schema, all of these columns have type "INTEGER".
>
> This is a bug: Keys and values of type "java.lang.String" must be
> mapped to columns of type "VARCHAR". Furthermore, join tables which
> are mapped by fields of maps or sets do not have index columns.
> Question: Do we have to distinguish bewteen sorted maps/sets vs.
> non-sorted maps/sets?
>
> I'll prepare two patches today fixing these issues - one for each
> identity type.
>
> Regards,
> Michael
> 
>
>
>

[ Post a follow-up to this message ]

Hi again Michael,

To further clarify, I agree that we want to leave the fld_char and
fld_Character fields of AllTypes as INTEGER and change the orm to
specify the jdbc-type.  However, I think that Andy was correct in saying
that the tables FIELDSOFCHARACTER and FIELDSOFPRIMITIVECHAR should have
CHAR(1) columns.  I already made that change.  I haven't looked at
PCPointSingleFieldChar, so I have no opinion about that one.

-- Michelle

Michael Watzek wrote:

> Hi Michelle, Andy,
>
> sorry I did not reply earlier.
>
> I talked to Craig about this issue yesterday. We agreed on leaving
> tables "FIELDSOFCHARACTER", "FIELDSOFPRIMITIVECHAR", and
> "PCPointSingleFieldChar" as they are keeping the INTEGER columns.
>
> The rationale is that pc class instances mapped to those tables have
> character values which are initialized based on numerical values
> rather than character values, e.g. "Character.MAX_VALUE",
> "Character.MIN_VALUE", or
> "System.currentTimeMillis()%Character.MAX_VALUE".
>
> For this reason, we decided to adapt the ORM metadata rather than to
> adapt the schema: <field> elements corresponding to fields of type
> "char" or "java.lang.Character" will have "jdbc-type" attributes, e.g.
>
>     <class name="PCPointSingleFieldPrimitivechar"
> table="PCPointSingleFieldChar">
>       <field name="id" primary-key="true">
>         <column name="ID" jdbc-type="INTEGER"/>
>       </field>
>       ...
>     </class>
>
> For pc classes like "HashtableStringValueCollections" (etc.) the story
> is slightly different:
>
> They have several fields of type "java.util.Map" where (sometimes)
> keys are FCOs (e.g. SimpleClass instances) and values are strings. The
> types of keys and values differ and can be derived by the classname.
>
> Each of these fields is mapped to a separate (join) table having join
> columns, key columns, value columns, and index columns. In the current
> schema, all of these columns have type "INTEGER".
>
> This is a bug: Keys and values of type "java.lang.String" must be
> mapped to columns of type "VARCHAR". Furthermore, join tables which
> are mapped by fields of maps or sets do not have index columns.
> Question: Do we have to distinguish bewteen sorted maps/sets vs.
> non-sorted maps/sets?
>
> I'll prepare two patches today fixing these issues - one for each
> identity type.
>
> Regards,
> Michael
> 
>
>
>

[ Post a follow-up to this message ]

Hi Andy,

I changed the column types of table "FIELDSOFCHARACTER" back to INTEGER.
I also changed the ORM metadata for class "FieldsOfCharacter", e.g.

<class name="FieldsOfPrimitivechar" table="FIELDSOFPRIMITIVECHAR">
..
<field name="char0">
<column name="CHAR0" jdbc-type="INTEGER"/>
</field>
..
</class>

Running test "TestFieldsOfCharacter" I receive an exception at
pm.makePeristent (see below). Looking at the stacktrace it seems that
JPOX calls PreparedStatement.setString though I specified "INTEGER" above.

The bound value is "java.lang.Character.MIN_VALUE". The same happens for
"java.lang.Character.MAX_VALUE". I do not receive an exception when I
bind a char value like '1' and '2'.

I'm using JPOX nightly build 2005-08-01. Can you please check and verify
if JPOX calls "PrepraredStatement.setString" for the case above?

Regards,
MIchael

[java] ERROR 22018: Invalid character string format for type INTEGER.
[java]      at
org.apache.derby.iapi.error.StandardException.newException(StandardException
.java)
[java]      at
org.apache.derby.iapi.types.DataType.invalidFormat(DataType.java)
[java]      at
org.apache.derby.iapi.types.SQLInteger.setValue(SQLInteger.java)
[java]      at
org.apache.derby.impl.jdbc.EmbedPreparedStatement.setString(EmbedPreparedSta
tement.java)
[java]      at
org.jpox.store.rdbms.mapping.CharRDBMSMapping.setObject(CharRDBMSMapping.jav
a:531)
[java]      at
org.jpox.store.mapping.SingleFieldMapping.setObject(SingleFieldMapping.java:
235)
[java]      at
org.jpox.store.rdbms.fieldmanager.ParameterSetter.storeObjectField(Parameter
Setter.java:129)
[java]      at
org.jpox.state.StateManagerImpl.providedObjectField(StateManagerImpl.java:24
02)
[java]      at
org.apache.jdo.tck.pc.fieldtypes.FieldsOfCharacter.jdoProvideField(FieldsOfC
haracter.java)
[java]      at
org.apache.jdo.tck.pc.fieldtypes.FieldsOfCharacter.jdoProvideFields(FieldsOf
Character.java)
[java]      at
org.jpox.state.StateManagerImpl.provideFields(StateManagerImpl.java:2740)
[java]      at
org.jpox.store.rdbms.request.InsertRequest.execute(InsertRequest.java:241)
[java]      at
org.jpox.store.rdbms.table.ClassTable.insert(ClassTable.java:1750)
[java]      at
org.jpox.store.StoreManager.insert(StoreManager.java:721)
[java]      at
org.jpox.state.StateManagerImpl.internalMakePersistent(StateManagerImpl.java
:3092)
[java]      at
org.jpox.state.StateManagerImpl.makePersistent(StateManagerImpl.java:3065)
[java]      at
org.jpox.AbstractPersistenceManager. internalMakePersistent(AbstractPersisten

ceManager.java:1014)
[java]      at
org.jpox.AbstractPersistenceManager. makePersistent(AbstractPersistenceManage

r.java:1050)
[java]      at
org.apache.jdo.tck.models.fieldtypes.TestFieldsOfCharacter.runTest(TestField
sOfCharacter.java:90)
[java]      at
org.apache.jdo.tck.models.fieldtypes.TestFieldsOfCharacter.test(TestFieldsOf
Character.java:66)
[java]      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
[java]      at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
)
[java]      at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
.java:25)
[java]      at org.apache.jdo.tck.JDO_Test.runBare(JDO_Test.java:197)
[java]      at
org.apache.jdo.tck.util.BatchTestRunner.start(BatchTestRunner.java:128)
[java]      at
org.apache.jdo.tck.util.BatchTestRunner.main(BatchTestRunner.java:106)

> Hi Michelle, Andy,
>
> sorry I did not reply earlier.
>
> I talked to Craig about this issue yesterday. We agreed on leaving
> tables "FIELDSOFCHARACTER", "FIELDSOFPRIMITIVECHAR", and
> "PCPointSingleFieldChar" as they are keeping the INTEGER columns.
>
> The rationale is that pc class instances mapped to those tables have
> character values which are initialized based on numerical values rather
> than character values, e.g. "Character.MAX_VALUE",
> "Character.MIN_VALUE", or "System.currentTimeMillis()%Character.MAX_VALUE"
.
>
> For this reason, we decided to adapt the ORM metadata rather than to
> adapt the schema: <field> elements corresponding to fields of type
> "char" or "java.lang.Character" will have "jdbc-type" attributes, e.g.
>
>     <class name="PCPointSingleFieldPrimitivechar"
> table="PCPointSingleFieldChar">
>       <field name="id" primary-key="true">
>         <column name="ID" jdbc-type="INTEGER"/>
>       </field>
>       ...
>     </class>
>
> For pc classes like "HashtableStringValueCollections" (etc.) the story
> is slightly different:
>
> They have several fields of type "java.util.Map" where (sometimes) keys
> are FCOs (e.g. SimpleClass instances) and values are strings. The types
> of keys and values differ and can be derived by the classname.
>
> Each of these fields is mapped to a separate (join) table having join
> columns, key columns, value columns, and index columns. In the current
> schema, all of these columns have type "INTEGER".
>
> This is a bug: Keys and values of type "java.lang.String" must be mapped
> to columns of type "VARCHAR". Furthermore, join tables which are mapped
> by fields of maps or sets do not have index columns. Question: Do we
> have to distinguish bewteen sorted maps/sets vs. non-sorted maps/sets?
>
> I'll prepare two patches today fixing these issues - one for each
> identity type.
>
> Regards,
> Michael
> 
>
>
>


--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]

Hi Michelle,

please have a look at test "TestFieldsOfCharacter". It uses
"java.lang.Charcater.MAX_VALUE" and "java.lang.Character.MIN_VALUE".

Concerning test "TestFieldsOfPrimitivechar" I am wrong. This test
initializes character fields using real characters like 'a'. For this
reason, I agree to change the column types to "CHAR(1)" for table
"FIELDSOFPRIMITIVECHAR".

Regards,
Michael
[vbcol=seagreen]
> Michael,
>
> Where are the numerical value initializations that you speak of for
> FieldsOfCharacter and FieldsOfPrimitivechar?  I don't see it.
>
> -- Michelle
>
> Michael Watzek wrote:
> 


--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]

> I changed the column types of table "FIELDSOFCHARACTER" back to INTEGER.
> I also changed the ORM metadata for class "FieldsOfCharacter", e.g.

> I'm using JPOX nightly build 2005-08-01. Can you please check and verify
> if JPOX calls "PrepraredStatement.setString" for the case above?

Hi Michael,

It's not in SVN. When it's in SVN then I'll have a look.

For the record on the issue of CHAR(1) or INTEGER, I'm not too fussed whethe
r
we have CHAR(1), or INTEGER or whatever - all I want is consistent metadata
and schema so it's a fair test.

--
Andy

[ Post a follow-up to this message ]

Okay, agreed.  -- Michelle

Michael Watzek wrote:

> Hi Michelle,
>
> please have a look at test "TestFieldsOfCharacter". It uses
> "java.lang.Charcater.MAX_VALUE" and "java.lang.Character.MIN_VALUE".
>
> Concerning test "TestFieldsOfPrimitivechar" I am wrong. This test
> initializes character fields using real characters like 'a'. For this
> reason, I agree to change the column types to "CHAR(1)" for table
> "FIELDSOFPRIMITIVECHAR".
>
> Regards,
> Michael
> 
>
>

[ Post a follow-up to this message ]

Hi Andy,

This is really good, a big improvement in the error count.

Andy Jefferson wrote:

>...
>Schema where column is INTEGER but should be CHAR(1)
>FieldsOfCharacter : table FIELDSOFCHARACTER has INTEGER columns
>FieldsOfPrimitivechar : table FIELDSOFPRIMITIVECHAR has INTEGER columns
>
>
I am fixing these.

>Schema where column is INTEGER but should be VARCHAR(...)
>HashMapStringKeyCollections, HashMapStringValueCollections,
>MapStringKeyCollections, MapStringValueCollections,
>TreeMapStringKeyCollections, TreeMapStringValueCollections : join tables ha
ve
>the key as INTEGER instead of VARCHAR
>
>
I agree that the KEYVAL columns in *StringKeyCollections and the
VALUEVAL columns of *StringValueCollections should be VARCHAR.  We have
some other fixes we need to make to the mapping for these, so I'll make
these changes with those.

>Also, this change may affect other JIRA issues, but sadly
>"http://issues.apache.org" is about as unresponsive a server as I've ever
>come across, so we'll have to leave that for others to work out ;-)
>
>
>
The server seems to have come to life again.  I will update the issues soon.

-- Michelle

[ Post a follow-up to this message ]

Hi Andy,

I found the problem:

I had prepared the ORM metadata for class "FieldsOfCharacter" in another
local workspace. Mistakenly, I made a stupid copy/paste mistake copying
wrong metadata for class "FieldsOfCharacter" into my current workspace:
I copied the contents of file "FieldsOfPrimitivechar-derby.orm" to file
"FieldsOfCharacter-derby.orm".

Fixing this, test "TestFieldsOfCharacter" successfully passes.

One question: JPOX does not complain if the ORM metadata for a class
specifies fields which do not exist in that class. Would it be an option
to throw a JDOFatalUserException("ORM metadata of class XXX specifies
column mapping for field YYY. This field does not exist in the class.")
in this case?

Regards,
Michael
 
>
> 
>
>
> Hi Michael,
>
> It's not in SVN. When it's in SVN then I'll have a look.
>
> For the record on the issue of CHAR(1) or INTEGER, I'm not too fussed whet
her
> we have CHAR(1), or INTEGER or whatever - all I want is consistent metadat
a
> and schema so it's a fair test.
>


--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]