Hey Guys,

When defining the Type ObjectClass
entry I use
m-may to define simple properties that are optional
and m-must to define simple properties that
are required.

Then I need to define the ObjectClass's
Complex Properties.

So I need something like
m-complexMay and m-complexMust.

Looking at current meta ObjectClasses,
I would think it makes sense to create another
meta ObjectClass that has these attributes,
possibly called metaSDOObjectClass?

Thoughts?

Thanks,
- Ole

[ Post a follow-up to this message ]

Hi Ole,

Ole Ersoy schrieb:
> Hey Guys,
>
> When defining the Type ObjectClass
> entry I use
> m-may to define simple properties that are optional
> and m-must to define simple properties that
> are required.
>
> Then I need to define the ObjectClass's
> Complex Properties.
>

What do you mean with complex properties? Is that something like XSD
complex types?

Regards,
Stefan Seelmann

[ Post a follow-up to this message ]

Hi Stefan,

SNIP
> What do you mean with complex properties? Is that something like XSD
> complex types?

Yes - you hit the nail right on the head.

So simple properties are things with primitive types
like int, float, boolean, String, etc. and
complex properties are references to a user
defined type.

The Java analogy would be
variables like

private String name;
...is a simple property

and

private Account account;
... is a complex property

Cheers,
- Ole

[ Post a follow-up to this message ]

Ole Ersoy a écrit :

> Hey Guys,
>
> When defining the Type ObjectClass
> entry I use
> m-may to define simple properties that are optional
> and m-must to define simple properties that
> are required.
>
> Then I need to define the ObjectClass's
> Complex Properties.
>
> So I need something like
> m-complexMay and m-complexMust.
>
> Looking at current meta ObjectClasses,
> I would think it makes sense to create another
> meta ObjectClass that has these attributes,
> possibly called metaSDOObjectClass?
>
> Thoughts?

Meta attributes are by no means intended to be used to describe user's
defined elements. They are used to describe elements used internally by
the server to deal with the schema itself. Think about meta-schema as
metaData in a RDBMS

We won't add a m-xxx element. Just add the attribute/objectclass you
need, it's not a problem, you will have to create a new schema (a little
bit like the java.schema schema). This is the way to go.

Emmanuel

[ Post a follow-up to this message ]

Hi Ole,

as Emmanuel I would recommend to create objectClasses and attributeTypes
for each user defined class.

Do you already have a plan how to store references between objects in
LDAP? There are different strategies.

For example you could use distinguished names to implement references.

Say, there are two POJOs Employee and Department:
----------------------------------------
class Employee
{
String firstName;
String lastName;
Department department;
}

class Department
{
String departmentName
List<Employee> employees;
}
----------------------------------------

Then two concrete objects could be stored like that in LDAP:
----------------------------------------
 cn=12345678,ou=Employee,ou=das,dc=exampl
e,dc=com
cn: 12345678
givenName: John
sn: Doe
department:  cn=23456789,ou=Department,ou=das,dc=exam
ple,dc=com

 cn=23456789,ou=Department,ou=das,dc=exam
ple,dc=com
cn: 23456789
departmentName: Development
employee:  cn=12345678,ou=Employee,ou=das,dc=exampl
e,dc=com
employee:  cn=12345679,ou=Employee,ou=das,dc=exampl
e,dc=com
employee:  cn=12345680,ou=Employee,ou=das,dc=exampl
e,dc=com
employee: .....
----------------------------------------

That is just one alternative. But it is really important to design this
properly. I think Alex and David Jencks discussed such strategies some
time ago.

Kind Regards,
Stefan Seelmann


Ole Ersoy schrieb:
> Hi Stefan,
>
> SNIP 
>
> Yes - you hit the nail right on the head.
>
> So simple properties are things with primitive types
> like int, float, boolean, String, etc. and
> complex properties are references to a user
> defined type.
>
> The Java analogy would be
> variables like
>
> private String name;
> ...is a simple property
>
> and
>
> private Account account;
> ... is a complex property
>
> Cheers,
> - Ole

[ Post a follow-up to this message ]

Hi Emmanuel and Stefan,

Stefan Seelmann wrote:
> Hi Ole,
>
> as Emmanuel I would recommend to create objectClasses and attributeTypes
> for each user defined class.

Yes we are doing that.

> Do you already have a plan how to store references between objects in
> LDAP?

I have one plan, but Emmanuel and yourself are goooddd, so I'll
gladly here more.

> There are different strategies.
>
> For example you could use distinguished names to implement references.
>
> Say, there are two POJOs Employee and Department:
> ----------------------------------------
> class Employee
> {
>   String firstName;
>   String lastName;
>   Department department;
> }
>
> class Department
> {
>   String departmentName
>   List<Employee> employees;
> }
> ----------------------------------------
>
> Then two concrete objects could be stored like that in LDAP:
> ----------------------------------------
>  cn=12345678,ou=Employee,ou=das,dc=exampl
e,dc=com
> cn: 12345678
> givenName: John
> sn: Doe
> department:  cn=23456789,ou=Department,ou=das,dc=exam
ple,dc=com
>
>  cn=23456789,ou=Department,ou=das,dc=exam
ple,dc=com
> cn: 23456789
> departmentName: Development
> employee:  cn=12345678,ou=Employee,ou=das,dc=exampl
e,dc=com
> employee:  cn=12345679,ou=Employee,ou=das,dc=exampl
e,dc=com
> employee:  cn=12345680,ou=Employee,ou=das,dc=exampl
e,dc=com
> employee: .....
> ----------------------------------------
>
> That is just one alternative. But it is really important to design this
> properly. I think Alex and David Jencks discussed such strategies some
> time ago.

Yes this is precisely how we are doing it.  Great to know we are all on
the same page.


============
WRITE OPERATION
============

The part where I need the additional "metaObjectClass"
is during the writing and restoration of
metadata.

So before the DAS writes DataObject instances
it writes each instances Type, corresponding a ObjectClass
entry in the DAS's schema area.

So Employee would have a unique ObjectClass and
Department would have a unique ObjectClass.

Then once these are written, it starts to write
the DataObject instances as you showed above.

============
READ OPERATION
============

Then upon a restore of a DataGraph, the first
thing the DAS does is restore the Type system.

This includes all user defined types like Employee
and Department.

When restoring Employee it creates
EAttributes (Which are models of simple xsd properties
corresponding to AttributeType entries)
and EReferences which are models of complex xsd properties
that have a corresponding ObjectClass entry.

For example when the DAS restores the
the Employee type it would do something like this:

EClass eClass   = EcoreFactory.eINSTANCE.createEClass("Employee");

EAttribute firstName EcoreFactory.eINSTANCE.createEAttribute("firstName");
...
EReference department =
EcoreFactory.eINSTANCE.createEReference("department");
....
eClass.getEAttributes().add(firstName);
eClass.getEReferences().add(department);
..............

Now it can create an instance of an employee
like this:

EObject employee = EmployeeFactory.eINSTANCE.create("Employee");
Then it starts restoring the employee state using the
employee's entry.

So during this Type restore process,
it gets the EAttribute instances it needs
to create from the
com.example.Employee ObjectClass's "m-may" and "m-must"
attributes, which would be named:

com-example-Employee-firstName
com-example-Employee-lastName

And it would get the "department"
EReference (named com-example-Employee-department)
name from the "m-complexMay"
attribute if an instance of the EReference
is required, and "m-complexMust" attribute
otherwise.

Does that make sense?

Thanks,
- Ole

[ Post a follow-up to this message ]

Hi Ole,

On 4/22/07, Ole Ersoy <ole.ersoy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wr
ote:
SNIP ...

Hi Emmanuel and Stefan,
>
> Stefan Seelmann wrote: 
>
> Yes we are doing that.
> 
>
> I have one plan, but Emmanuel and yourself are goooddd, so I'll
> gladly here more.
> 


Yes a number of different approaches can be defined to do this.  I have
experimented with a few things like using DN, relative names, subordinate
objects or even subordinate aliases/referrals.  However I have not come to
any solid conclusions on which technique is best.

Oh btw another approach is to define yet another objectClass that acts to
link one entity to another sort of like a relationship OC.

Yet another approach is to create a unique ID for a type like departmentID
and use that in the Employee entry along with in the Department entries.
This way the reference is implicit and a search can easily locate the
departmentID.  Sometimes people put an LDAP referral into a attribute with
search terms.  NOTE: an LDAP referral can contain a filter in it with base
etc to locate zero or more entries based on their attribute values.  Let me
give you an example:

First the syntax:

ldap://host:port/dn?attributes?scope?filter?extensions

Now an example:

ldap://localhost:10389/ou=das??sub?(departmentId=engineering)

This referral returns all entries that have departmentId equal to
engineering.  If departmentId is unique you only get zero or one entry.  If
you make a departmentRef attribute and make it extend the ref attribute you
can stuff this referral into it.  That way you associate the Employee with a
department.

<OT>
It sure would be nice to have a control or something that could
automatically
dereferrence this referral and return all the objects in the object graph
for you
within the same search.  So you search looking for an employ with the deref
control and you'll get the employee and all the referrenced entries with it.
</OT>

Yes this is precisely how we are doing it.  Great to know we are all on[vbcol=seagreen]
> the same page.


Just to warn you this is a tough subject.  Make sure you think hard about
how to model relationships and consult existing material on this subject.
If I had the time I would research and experiment with this more since it's
a significant issue.

Just remember you have to think not only about how this information is
encoded in entries but also how you efficiently conduct searches to build
your object graph.

============
> WRITE OPERATION
> ============
>
> The part where I need the additional "metaObjectClass"
> is during the writing and restoration of
> metadata.


So when you use meta you mean meta in the DAS meta data sense and not the
meta schema sense in ApacheDS right?  We have to be careful I guess with the
jargon we use since there could be some confusion here.

So before the DAS writes DataObject instances
> it writes each instances Type, corresponding a ObjectClass
> entry in the DAS's schema area.


Makes sense this is like the DDL you need to apply to add tables in an RDBMS
before adding the records.

So Employee would have a unique ObjectClass and
> Department would have a unique ObjectClass.
>
> Then once these are written, it starts to write
> the DataObject instances as you showed above.
>
> ============
> READ OPERATION
> ============
>
> Then upon a restore of a DataGraph, the first
> thing the DAS does is restore the Type system.
>
> This includes all user defined types like Employee
> and Department.
>
> When restoring Employee it creates
> EAttributes (Which are models of simple xsd properties
> corresponding to AttributeType entries)
> and EReferences which are models of complex xsd properties
> that have a corresponding ObjectClass entry.
>
> For example when the DAS restores the
> the Employee type it would do something like this:
>
> EClass eClass   = EcoreFactory.eINSTANCE.createEClass("Employee");
>
> EAttribute firstName EcoreFactory.eINSTANCE.createEAttribute("firstName");
> ...
> EReference department =
> EcoreFactory.eINSTANCE.createEReference("department");
> ....
> eClass.getEAttributes().add(firstName);
> eClass.getEReferences().add(department);
> ..............


Ok so you're reconstructing the ecore model from the LDAP schema here.  It's
finally making sense to me now.  Let me know if I am incorrect.

Now it can create an instance of an employee
> like this:
>
> EObject employee = EmployeeFactory.eINSTANCE.create("Employee");
> Then it starts restoring the employee state using the
> employee's entry.
>
> So during this Type restore process,
> it gets the EAttribute instances it needs
> to create from the
> com.example.Employee ObjectClass's "m-may" and "m-must"
> attributes, which would be named:
>
> com-example-Employee-firstName
> com-example-Employee-lastName
>
> And it would get the "department"
> EReference (named com-example-Employee-department)
> name from the "m-complexMay"
> attribute if an instance of the EReference
> is required, and "m-complexMust" attribute
> otherwise.
>
> Does that make sense?


Yeah it makes sense to me but I would not start mixing our meta schema stuff
with your mechanism for modeling required/optional references to complex
objects.  Use something domain specific.  For example create an attribute in
Employee called "department" or "departmentId" and use that to hold the
reference information encoded however you like (i.e. DN, custom syntax,
value of unique id for the department).

See what I'm saying? Just stop trying to use the meta.schema terms for
this.  It's not worth it first of all and secondly it's not going to work
because of collisions and typing issues.  Meaning if you have a Address
complex type now for the Employee then you get a collision where you cannot
use m-complexMust for it.

HTH,
Alex

[ Post a follow-up to this message ]

Hi Alex,

Thanks for all the additional material on search
related considerations.  Good stuff.
I'm keeping it in my parking lot, so I can
come back to it once I have the ability to
write and restore datagraphs.

SNIP

>
>     ============
>     WRITE OPERATION
>     ============
>
>     The part where I need the additional "metaObjectClass"
>     is during the writing and restoration of
>     metadata.
>
>
> So when you use meta you mean meta in the DAS meta data sense and not
> the meta schema sense in ApacheDS right?

Well - let me approach it like this.  I'll tell you
what I was planning to do, and then we can discuss
the proper terminology for it.

When I define the ObjectClass that corresponds
to each DataObject instances type (Class), I need
to define the Type's simple properties (Which use
the already available "m-must" and "m-may" meta AttributeType
entries.

Then I also need to define members that correspond to
ComplextProperties.  For instance the employee has a
a complex property reference to a Department.

Does that make sense.  However ApacheDS's current metaObjectClasses
(the ones that are used to define other ObjectClass entries like
Department and Employee) don't have anything like
"m-complexMust" or "m-complexMay" so I was going to define these
on something in an ObjectClass entry maybe called metaSDOObjectClass
and then add this as an "objectClass" attribute when I'm defining
the ObjectClasses for Department and Employee.

I don't have to put this ObjectClass with ADS's metaObjectClasses.
I can make a new schema area for the DAS and add it there.  I think
that's what you are wondering about?

SNIP

> Makes sense this is like the DDL you need to apply to add tables in an
> RDBMS before adding the records.

Yes
SNIP

>
>
> Ok so you're reconstructing the ecore model from the LDAP schema here.
> It's finally making sense to me now.  Let me know if I am incorrect.

Right On!

SNIP

>
>
> Yeah it makes sense to me but I would not start mixing our meta schema
> stuff with your mechanism for modeling required/optional references to
> complex objects.  Use something domain specific.  For example create an
> attribute in Employee called "department" or "departmentId"

Well - we don't have to mix ADS metaObjectClasses (metaTop,
metaObjectClass, etc.) with the DAS one.  I can put it in a custom
schema area for the DAS.  Sound ok?

So when defining the ObjectClass for Employee, I would
add all simple properties using the metaObjectClass's
"m-must" and "m-may" attributes.

However, I would also add the ObjectClass metaSDOObjectClass,
enabling me to use "m-complexMust" and "m-ComplexMay" for the
complex properties.  Make sense?  Before you answer,
you may want to have a look
at the example I gave at the bottom.


SNIP

> See what I'm saying? Just stop trying to use the meta.schema terms for
> this.  It's not worth it first of all and secondly it's not going to
> work because of collisions and typing issues.  Meaning if you have a
> Address complex type now for the Employee then you get a collision where
> you cannot use m-complexMust for it.

I'm sorry Alex - Maybe I'm missing something.  As far as I can see the
semantics are the same as for simple properties / attributes that use
"m-must" and "m-may".

So for instance the Employee could have "m-complexMust" attributes

- Address
- Department
- Parking
- etc.

When constructing the ObjectClass for Employee these would be added
using Attributes like this:

new BasicAttribute("m-complexMust", "com-example-Employee-address".

new BasicAttribute("m-complexMust", "com-example-Employee-department".

new BasicAttribute("m-complexMust", "com-example-Employee-parking".

And the simple properties like name, etc. would be added like this:

new BasicAttribute("m-must", "com-example-Employee-name".

etc.

Does that make sense?

Thanks,
- Ole

[ Post a follow-up to this message ]

Ole,

On 4/22/07, Ole Ersoy <ole.ersoy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> Hi Alex,
>
> Thanks for all the additional material on search
> related considerations.  Good stuff.
> I'm keeping it in my parking lot, so I can
> come back to it once I have the ability to
> write and restore datagraphs.


My pleasure.

SNIP
> 
>
> Well - let me approach it like this.  I'll tell you
> what I was planning to do, and then we can discuss
> the proper terminology for it.



Buddy I can't be wasting cycles these days dancing around these things.
Otherwise I would write the DAS me self .  Basically let's stop mixing two
separate aspects of this problem.  I don't want to waste cycles this way but
to spend my time giving you exactly what you need to succeed.


> When I define the ObjectClass that corresponds
> to each DataObject instances type (Class), I need
> to define the Type's simple properties (Which use
> the already available "m-must" and "m-may" meta AttributeType
> entries.
>
> Then I also need to define members that correspond to
> ComplextProperties.  For instance the employee has a
> a complex property reference to a Department.
>
> Does that make sense.  However ApacheDS's current metaObjectClasses
> (the ones that are used to define other ObjectClass entries like
> Department and Employee) don't have anything like
> "m-complexMust" or "m-complexMay" so I was going to define these
> on something in an ObjectClass entry maybe called metaSDOObjectClass
> and then add this as an "objectClass" attribute when I'm defining
> the ObjectClasses for Department and Employee.


This is exactly what I mean.  Why are you creating a metaSDOObjectClass?  Is
this a meta schema object?  Man I am not this mentally nimble.

I don't have to put this ObjectClass with ADS's metaObjectClasses.
> I can make a new schema area for the DAS and add it there.  I think
> that's what you are wondering about?


No I just don't want to be mixing the jargon here because it's totally
confusing me and is unnecessary.  Sorry if this sounds harsh but I have very
little time these days due to various issues.  Trying to make time spent in
from of gmail.com count .

 
>
> Well - we don't have to mix ADS metaObjectClasses (metaTop,
> metaObjectClass, etc.) with the DAS one.  I can put it in a custom
> schema area for the DAS.  Sound ok?


I still think you're mixing stuff here together in terms of the concepts.  I
think you're trying to create general objectClasses for representing any DAS
object.  I'm trying to tell you to stay away from that since you can create
custom domain specific objects since you're generating this stuff anyway.
So let's not create any objectClasses with metaXYZ in it or any m-
attributes in it.   Maybe I am missing something you're saying all together
thoh.

So when defining the ObjectClass for Employee, I would
> add all simple properties using the metaObjectClass's
> "m-must" and "m-may" attributes.
>
> However, I would also add the ObjectClass metaSDOObjectClass,
> enabling me to use "m-complexMust" and "m-ComplexMay" for the
> complex properties.  Make sense?  Before you answer,
> you may want to have a look
> at the example I gave at the bottom.




SNIP
> 
>
> I'm sorry Alex - Maybe I'm missing something.


Yes you are missing a lot.

As far as I can see the
> semantics are the same as for simple properties / attributes that use
> "m-must" and "m-may".
>
> So for instance the Employee could have "m-complexMust" attributes
>
> - Address
> - Department
> - Parking
> - etc.
>
> When constructing the ObjectClass for Employee these would be added
> using Attributes like this:
>
> new BasicAttribute("m-complexMust", "com-example-Employee-address".
>
> new BasicAttribute("m-complexMust", "com-example-Employee-department".
>
> new BasicAttribute("m-complexMust", "com-example-Employee-parking".
>
> And the simple properties like name, etc. would be added like this:
>
> new BasicAttribute("m-must", "com-example-Employee-name".
>
> etc.
>
> Does that make sense?



What I am saying is you don't need to do this at all.  I think you're
missing some big concepts.   When you define the employee object class and
need an attribute to reference the department complex object just use the
m-must or m-may attributes.   Say this attribute is called
com-example-Department.  If you create the attributeType definition for this
then you can define the syntax for this complex object reference to be
anything you want.  So you control the type of the object that way not by
defining a m-mayComplex attribute type at the meta level.

That's it. These m-may and m-must attributes work for any kind of attribute
with any kind of syntax you may define.  This m-mustComplex m-mayCOmplex
stuff is absolutely useless.

Sorry about my frustration but I feel like I'm spending a lot of time and my
words are not describing things well enough for you to understand.

Alex

[ Post a follow-up to this message ]

Hey Alex,

(BTW - If you are pressed for time, just say "Ole I'll review
this later - Worst case I'll bump into a constraint, and have to reroute
:-) )  No big deal.

Hopefully this clears the fog a little

> What I am saying is you don't need to do this at all.  I think you're
> missing some big concepts.   When you define the employee object class
> and need an attribute to reference the department complex object just
> use the m-must or m-may attributes.

The reason I want "m-complexMay" or "m-complexMust" is because
these tell the DAS to create an EReference when restoring metadata.

"m-may" and "m-must" tell the DAS to create an EAttribute when
restoring metadata.

If I use the approach you are suggesting I have to do additional
processing to figure out whether it's an EReference or EAttribute.
Not that it is a huge deal, but I think "m-ComplexMay" and
"m-ComplexMust" is cleaner, and will result in better performance.

> Sorry about my frustration but I feel like I'm spending a lot of time
> and my words are not describing things well enough for you to understand.

No pb.  Hopefully I come across as if I understand what you are saying
now.  I think the two are minor differences in approach, unless I'm
still missing something.

How about this.  I'll try it out.  If I'm missing something.  I'll
get smacked.  Then I'll rework it your way.  I just think it's a simpler
design with "m-complexMay" and "m-complexMust".

Incidentally the SDO JSR takes your approach.  They define everything as
a Type.  So no EAttribute and EReference.  They are both just Type.  I
still like the additional semantics behind EAttribute and EReference, so
I'll give it a shot.  Maybe I'll get burnt, and then you can tell me "I
told you so!"  :-)

Thanks,
- Ole

[ Post a follow-up to this message ]