Hi,

JIRA JDO-13 was filed because a TCK test hangs
(org.apache.jdo.tck.lifecycle.PMsCanSharePCClassesButNotPCInstances).

That TCK test starts 5 concurrent threads. Each thread makes an object
of class "org.apache.jdo.tck.pc.lifecycle.StateTransitionObj" persistent
in a separate transaction. Afterwards, each thread waits for the other
threads to finish their transactions. After each threads has finished
its transaction, all threads continue execution.

When such a thread makes an object persistent, it increases a counter.
After its transaction has finished, it checks that counter and sleeps
until the counter equals 5.

Unfortunately, the code is not prepared for exceptions thrown while
transactions are executing. In such cases, the counter is not increased
and all threads wait forever.

After I fixed this in my workspace, the exception points to the problem:

ERROR 42X05: Table 'STATETRANSITIONOBJ' does not exist.

The files schema1.sql and schema2.sql do not contain that table.
Furthermore, I cannot find a .orm file in
org/apache/jdo/tck/pc/lifecycle package for that class - neither for
datastore identity nor for application identity.

There are .jdo files for both identities. Confusingly, the .jdo file for
application identity says datastore identity. I'm not sure if this is a
bug, as class "org.apache.jdo.tck.pc.lifecycle.StateTransitionObj" has
only two fields and, because of the field names, only one of them seem
to be persistent:

private int           int_field;
private transient int nonmanaged_field;

However, the .jdo files say that both are persistent. Probably, these
files do not comply to TCK20 conventions, as they are no package jdo files?

If it is only one field which is persistent, do we have a problem with
application identity? In that case, the field must be the PK.

Summerizing, in addition to the fix in the test code we need to

1) fix the SQL schema
2) add the missing .orm files
3) fix the .jdo files

Eric, Andy, Michelle:

I'm not sure, if the schema is autogenerated or if schema1.sql and/or
schema2.sql is used. Can one of you fix the schema issue?

Craig:

I'm not sure which fields of class
"org.apache.jdo.tck.pc.lifecycle.StateTransitionObj" are persistent. Can
you also help on the application/datastore issue? Is it reasonable to
define a second persistent field?

Thanks,
Michael
--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]

Hi, Michael,

Comments below.

Michael Watzek wrote:

> Hi,
>
> JIRA JDO-13 was filed because a TCK test hangs
> (org.apache.jdo.tck.lifecycle.PMsCanSharePCClassesButNotPCInstances).
>
> That TCK test starts 5 concurrent threads. Each thread makes an object
> of class "org.apache.jdo.tck.pc.lifecycle.StateTransitionObj"
> persistent in a separate transaction. Afterwards, each thread waits
> for the other threads to finish their transactions. After each threads
> has finished its transaction, all threads continue execution.
>
> When such a thread makes an object persistent, it increases a counter.
> After its transaction has finished, it checks that counter and sleeps
> until the counter equals 5.
>
> Unfortunately, the code is not prepared for exceptions thrown while
> transactions are executing. In such cases, the counter is not
> increased and all threads wait forever.
>
> After I fixed this in my workspace, the exception points to the problem:
>
> ERROR 42X05: Table 'STATETRANSITIONOBJ' does not exist.
>
> The files schema1.sql and schema2.sql do not contain that table.
> Furthermore, I cannot find a .orm file in
> org/apache/jdo/tck/pc/lifecycle package for that class - neither for
> datastore identity nor for application identity.

The missing schema and metadata are the subject of JDO-47.

>
> There are .jdo files for both identities. Confusingly, the .jdo file
> for application identity says datastore identity. I'm not sure if this
> is a bug, as class
> "org.apache.jdo.tck.pc.lifecycle.StateTransitionObj" has only two
> fields and, because of the field names, only one of them seem to be
> persistent:
>
>     private int           int_field;
>     private transient int nonmanaged_field;
>
> However, the .jdo files say that both are persistent. Probably, these
> files do not comply to TCK20 conventions, as they are no package jdo
> files?
>
> If it is only one field which is persistent, do we have a problem with
> application identity? In that case, the field must be the PK.

The jdo files were never modified properly for tck20 because we were
planning not to use the tests that require this pc class, but then we
decided not to remove the tests.  So this is simply unfinished work.

I may as well fix this, because I am changing the schema now anyway. If
you fixed it, we would have to merge.

The int_field should be persisistent and is the pk for application
identity. We have decided to generally add a column called
DATASTORE_IDENTITY to the tables as the pk for datastore identity:

CREATE TABLE <tablename> (
DATASTORE_IDENTITY BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY,
..
CONSTRAINT PT_CONST PRIMARY KEY (DATASTORE_IDENTITY)
);

I am making this change to the metadata and schema right now.

-- Michelle

>
>
> Summerizing, in addition to the fix in the test code we need to
>
> 1) fix the SQL schema
> 2) add the missing .orm files
> 3) fix the .jdo files
>
> Eric, Andy, Michelle:
>
> I'm not sure, if the schema is autogenerated or if schema1.sql and/or
> schema2.sql is used. Can one of you fix the schema issue?
>
> Craig:
>
> I'm not sure which fields of class
> "org.apache.jdo.tck.pc.lifecycle.StateTransitionObj" are persistent.
> Can you also help on the application/datastore issue? Is it reasonable
> to define a second persistent field?
>
> Thanks,
> Michael

[ Post a follow-up to this message ]

[ Post a follow-up to this message ]

Hi Michael,

I will fix the schema and metadata, assuming that you will add a field
called id to the StateTransitionObj class that will be the primary key.
I will check in the schema and metadata today so you can test with it.
Is this okay with you?

-- Michelle


Craig Russell wrote:
 
>
>
> This is true, and the problem with it is that since it would be the
> PK, it cannot generally be written. Additionally there is no
> application identity class.
> 
> 0.0 Fix the test case to catch exceptions and complete.
> 0.1 Fix the StateTransitionObject to add an int id field with
> application-created values, and add an embedded objectid class using
> the new field as primary key. (The default constructor should set the
> id field and increment the value to make it unique; the objectid class
> should be embedded and have the same int id field)
> 
>
>
> Yes, see above.
>
> So, Michael,
>
> If you have the JPOX environment set up, you should be able to fix the
> StateTransitionObj.java and
> the PMsCanSharePCClassesButNotPCInstances.java, the .jdo and .orm files.
>
> Just coordinate with Michelle so you don't conflict on the schema1.sql
> and schema2.sql files. Since it's just adding a table, there should be
> minimal conflicts.
>
> Thanks,
>
> Craig

[ Post a follow-up to this message ]

[ Post a follow-up to this message ]

Hi Michelle,

thanks for the checkin!

I have added an object id class in "StateTransitionObj.jdo" for
application identity. I'll run the TCK for application identity and
datastore identity overnight and will send the patch for JIRA JDO-13
tomorrow.

Regards,
Michael
[vbcol=seagreen]
> Hi, Michael,
>
> I just checked in the metadata/schema portion of the fix for JDO-47
> (lifecycle metadata).  The class still needs to add an id field.  You
> won't get a commit notice because it bounced because it was too large!
>
> -- Michelle
>
> Michelle Caisse wrote:
> 


--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]

Hi, Michael,

Sounds good, thanks.  I found that the 6/6 JPOX build ran 10x slower
than the 6/3 build -- 10 hours to run the TCK.  I am running the 6/7
build right now, and I'm not sure what the final time will be.  At first
it seemed to be running pretty quickly, but now that it's doing the
fieldtypes tests, it is very slow.  So you may want to use an older JPOX
build if you hope to get both application and datastore identity done in
one night!

-- Michelle

Michael Watzek wrote:

> Hi Michelle,
>
> thanks for the checkin!
>
> I have added an object id class in "StateTransitionObj.jdo" for
> application identity. I'll run the TCK for application identity and
> datastore identity overnight and will send the patch for JIRA JDO-13
> tomorrow.
>
> Regards,
> Michael
> 
>
>

[ Post a follow-up to this message ]

> Sounds good, thanks.  I found that the 6/6 JPOX build ran 10x slower
> than the 6/3 build -- 10 hours to run the TCK.  I am running the 6/7
> build right now, and I'm not sure what the final time will be.  At first
> it seemed to be running pretty quickly, but now that it's doing the
> fieldtypes tests, it is very slow.  So you may want to use an older JPOX
> build if you hope to get both application and datastore identity done in
> one night!

Hi Michelle,

have you got "org.jpox.autoStartMechanism" set to anything in particular ?
The fact that things slow down is symptomatic of JPOX progressively loading 
up
all of the classes that it has ever encountered during its lifetime with the
DB schema (JPOX_TABLES table) that it connects to. So with each test, it
finds more classes and so loads up the MetaData of the previous ones, plus
the new ones. If you set this property to "None" it will start from scratch
each time (and so not go off an load up other classes not used in that test)
.
The log would tell you what is taking the time

Nothing significant has changed in the period you mention (AFAIK) and I see 
no
change in runtime speed here on our unit tests.


--
Andy
Java Persistent Objects JDO - JPOX

[ Post a follow-up to this message ]

Hi Michelle,

I still use JPOX 1.1.0-beta-3. One TCK run takes about 2 hours. So I'm
confident that the two runs will finish overnight :-).

Does it make sense to use mysql instead of Derby - just to make sure
it's not Derby slowing down the performance? We have to adapt the schema
files, the URL and the driver name in jdori.properties, and the classpath.

Regards,
Michael


Andy Jefferson wrote: 
>
>
> Hi Michelle,
>
> have you got "org.jpox.autoStartMechanism" set to anything in particular ?
> The fact that things slow down is symptomatic of JPOX progressively loadin
g up
> all of the classes that it has ever encountered during its lifetime with t
he
> DB schema (JPOX_TABLES table) that it connects to. So with each test, it
> finds more classes and so loads up the MetaData of the previous ones, plus
> the new ones. If you set this property to "None" it will start from scratc
h
> each time (and so not go off an load up other classes not used in that tes
t).
> The log would tell you what is taking the time
>
> Nothing significant has changed in the period you mention (AFAIK) and I se
e no
> change in runtime speed here on our unit tests.
>
>


--
-------------------------------------------------------------------
Michael Watzek                  Tech@Spree Engineering GmbH
mailto:mwa.tech@spree.de        Buelowstr. 66
Tel.:  ++49/30/235 520 36       10783 Berlin - Germany
Fax.:  ++49/30/217 520 12       http://www.spree.de/
-------------------------------------------------------------------

[ Post a follow-up to this message ]

[ Post a follow-up to this message ]