Solved

OData DateTimeOffset (C#) throws ODP_DESERIALIZATION error

Manulak
Rank
Userlevel 4
Badge +8

When creating a .NET interface (C#) using the standard OData connected service library, the standard way to set the data time is to use the very own type DateTimeOffset. 

DateTimeOffset now_ = DateTimeOffset.Now;

hu.Loaded = now_;

But, this ends the request up with ODP_DESERIALIZATION_ERROR. 

The connector we use: OData Connected Service - Visual Studio Marketplace 
If that’s not supported by IFS, what is IFS’s standard and supported OData connecter for .NET applications?

If that’s supported, is there a bug fix from IFS to overcome this issue?

Thank you

 

Sample code:

 

icon

Best answer by Jonas Feigl 1 October 2021, 16:42

View original
Manulak Dissanayake | https://rb.gy/v88r80
This topic has been closed for comments

12 replies

M
Rank
Userlevel 3
Badge +10

We don’t use the OData connection service where I work.  We connect directly to the database.  That means I don’t know what OData requirements are.  I can say that there are two other posts in the community for a similar question.  They were resolved by changing data types. 

A deserialization error usually means a problem creating the XML from the class data.  Are you sure that the LoadHandlingUnitStruct.Loaded is the same data/class type as DateTimeOffset.Now() function returns?

Manulak
Rank
Userlevel 4
Badge +8

We don’t use the OData connection service where I work.  We connect directly to the database.  That means I don’t know what OData requirements are.  I can say that there are two other posts in the community for a similar question.  They were resolved by changing data types. 

A deserialization error usually means a problem creating the XML from the class data.  Are you sure that the LoadHandlingUnitStruct.Loaded is the same data/class type as DateTimeOffset.Now() function returns?

Thanks for the reply.

  • So your applications directly connect to Oracle without using the REST endpoints? Then I wonder how the connection is made, how far the app supports the platform independence and how far it complies with the security standards. However, here the concern is with the IFS OData support vs DateTimeOffset format.
  • The situation is same with Postman too, hence basically there’s an issue with IFS’s REST datetime format with the .NET OData Connected Service.
  • I’ve seen those two cases of the community but they suggest to either check the formats, or to add the additional request headers.
    • The DataTime formats cannot be casted or there are no varchar operations possible, i.e. the datetime should be set via the data type DateTimeOffset when using the .NET OData Connected Service. But, the default OData format should be supported by IFS.
    • Adding additional request parameters to overcome deserialization issue is a workaround which we do not like to accept as a long-term solution, hence this question..!
  • We are not worried about the DateTimeOffset.Now format, because that’s the standard OData supported format, hence we expect IFS should do additional investigations to find why it’s ended up in deserialization error.

Thanks again.

Manulak Dissanayake | https://rb.gy/v88r80
M
Rank
Userlevel 3
Badge +10

Referring to your last bullet point:  if that is the case then why post it on the community page instead of creating a case?

Manulak
Rank
Userlevel 4
Badge +8

Referring to your last bullet point:  if that is the case then why post it on the community page instead of creating a case?

The case is created way before this community question.

This community question brings in several benefits, including

  • knowing if someone faced the same issue and found a way to solve it maybe using different libraries, datatypes, etc
  • knowing if there’s already a solution or a fix in progress, etc
  • sharing the solution with the community if I were to find the solution somehow, which is the whole point of the communities like these..
Manulak Dissanayake | https://rb.gy/v88r80
Jonas Feigl
Rank
Userlevel 5
Badge +17

@Manulak 

I'm fairly certain that this is working as expected and in line with the OData specification. When you use DateTimeOffset.Now you end up with a value with fractional seconds. This is not allowed for attribute WantedDeliveryDate.

Reason: 

The OData v4.0 specification contains the following two statements in regards to Precision in DateTimeOffsets:

1. For a temporal property the value of this attribute specifies the number of decimal places allowed in the seconds portion of the property’s value

2. If no value is specified, the temporal property has a precision of zero.

When you have a look at the OData $metadata document of this service you will find that the WantedDeliveryDate property has no Precision. As such a precision of "0" is assumed - in line with the OData 4.0 specification. As a result any values passed in require the fractional seconds to be either completely absent or zero.

Where things get confusing is that this part of the OData specification has actually been changed with OData 4.01. The above second statement has been removed and it now states the following instead: 

- Absence of $Precision means arbitrary precision.

While I think IFS is working according to spec (as we have implemented v4.0 and not v4.01) I agree that it might be beneficial for IFS to explicitly set the precision in the metadata document to "0" to avoid this confusion.

 

In regards to your specific use case the following statement should provide you with a DateTimeOffset without fractional seconds, i.e. with precision "0":

new DateTimeOffset(DateTimeOffset.Now.Ticks/10000000*10000000, DateTimeOffset.Now.Offset);
 

Jonas Feigl
Rank
Userlevel 5
Badge +17

One minor correction to my previous statement: OData v4.0 and OData v4.01 actually both specify that “If no value is specified, the temporal property has a precision of zero.” so IFS is working correctly according to both versions but there is some ambiguity in the OData v4.01 specification that you shoudl be aware of:

 

This part of the specification states

If no value is specified, the temporal property has a precision of zero.

 

Whereas this part states 

Absence of $Precision means arbitrary precision.

without clarifying that this only applies to numeric fields.In particular the fact that after this statement it shows an example with a DateTimeOffset makes this problematic.

 

I have raised this inconsistency with the OData Mailing Group to get this reviewed: https://markmail.org/search/?q=&q=list%3Aorg.oasis-open.lists.odata-comment#query:%20list%3Aorg.oasis-open.lists.odata-comment+page:1+mid:m24js4tpvy2d374k+state:results

Manulak
Rank
Userlevel 4
Badge +8

One minor correction to my previous statement: OData v4.0 and OData v4.01 actually both specify that “If no value is specified, the temporal property has a precision of zero.” so IFS is working correctly according to both versions but there is some ambiguity in the OData v4.01 specification that you shoudl be aware of:

 

This part of the specification states

If no value is specified, the temporal property has a precision of zero.

 

Whereas this part states 

Absence of $Precision means arbitrary precision.

without clarifying that this only applies to numeric fields.In particular the fact that after this statement it shows an example with a DateTimeOffset makes this problematic.

 

I have raised this inconsistency with the OData Mailing Group to get this reviewed: https://markmail.org/search/?q=&q=list%3Aorg.oasis-open.lists.odata-comment#query:%20list%3Aorg.oasis-open.lists.odata-comment+page:1+mid:m24js4tpvy2d374k+state:results

 

Thanks @Jonas Feigl for the response, but I’m not yet convinced.

This is clearly an IFS limitation due to IFS’s standard (default) database design concept with using DATE as the datatype for related table columns. The DATE field cannot handle these fractions of the input, hence it emits a deserialization error. 

In OData context too, it’s an optional attribute, hence implementation of IFS by forcefully setting it to 0 is not enough. If a value is received, IFS should still handle it (by truncating or ignoring) so that the standard requests with precision fraction values will not be failed. 

Check the following quote…

…Client developers MUST be aware of the potential for data loss when round-tripping values of greater precision…

It of course says there could be data losses, but it never says the receiver should or will need to fail the request or whatsoever.

If IFS database design does not support these fractions, but still the projection materials should support OData standards, they should of course handle it, and not to fail the request..!

Manulak Dissanayake | https://rb.gy/v88r80
Jonas Feigl
Rank
Userlevel 5
Badge +17

Regarding

This is clearly an IFS limitation due to IFS’s standard (default) database design concept with using DATE as the datatype for related table columns.

You are indeed correct that IFS mainly uses the Oracle DATE datatype and this can indeed not handle fractional seconds. This is the specific reason why IFS does not define a precision for the DateTimeOffset attributes in the OData metadata. As is documented in the OData specification this then defaults to a precision of 0.

The Deserialization error you are receiving is not raised by the IFS implementation though but rather by the Apache Olingo library which performs this validation based on the metadata provided.

So yes this behaviour is intended by IFS but this follows the OData specification and this validation is not something IFS-specific.

If an IFS attribute is defined as an Oracle TIMESTAMP (which is rarely used - but the support is there!) you are also able to provide/receive higher precision in the OData request/receive. An example of such an attribute is in projection “MobileTransTrace” where the Timestamp attribute has a higher precision:

OData $metadata for MobileTransTrace projection

 

Regarding

IFS should still handle it (by truncating or ignoring) so that the standard requests with precision fraction values will not be failed. 

No. The OData documentation about the Precision is very clear about this by stating “number of decimal places allowed in the seconds portion of the property’s value”. You are calling the endpoint with a value with a higher precision which is simply not allowed and as such is rightly rejected with an error. This is working in line with the OData specification and the Apache Olingo implementation of this specification.

 

…Client developers MUST be aware of the potential for data loss when round-tripping values of greater precision…

This statement is not related to the issue we are discussing here at all:

  • We are discussing the client sending values to the server with higher precision than the server allows.
  • This statement in the specification is about the opposite scenario: A server returning values with higher precision than the client can handle

If an attribute in IFS is defined as an Oracle type “TIMESTAMP(9)” it will have the IFS Projection Type “Timestamp(9)” and as a result the OData type “DateTimeOffset” with a Precision “9”.

The client developers (in this scenario you) has to be aware that C# only supports 7 digits so after reading such a value from the server 2 decimals of precision will be lost and when writing this value back to the server the precision will then also be lost on the server.

The specification does not talk about a precision loss when sending higher precision to the server as this is not allowed in the first place as mentioned above.

 

Manulak
Rank
Userlevel 4
Badge +8

Hello @Jonas Feigl 

Thanks for the explanation, but it doesn’t help much I’m afraid.

I think it’ll be productive if you come out of the IFS mindset and start looking into the issue as an external party who utilizes IFS’s RESTful integration support to integrate a C# webservice.

We don’t need to know, rather it doesn’t make sense to explain or discuss how IFS uses libraries, their limitations, etc (Apache Olingo library, etc), which external parties don’t focus on, and don’t care about. What matters is that the IFS projections fails the requests made with the OData primitive data types.

 

Just FYI : We use MS OData connected service libraries from Marketplace (which is standard OData library for development) and use DateTimeOffset which is a primitive data type of OData. It doesn’t work with IFS, i.e. IFS supports OData standards within the projections but looks like “not all” :-) 

However, none of above C# materials are needed, even the projections fail with the Postman with precisions ;-) 

However, we found few ugly workarounds but we wait till IFS solves this for sure. When this is solved, I’ll inform via this same thread. Thanks for sharing your view on this.

/Manulak

 

Manulak Dissanayake | https://rb.gy/v88r80
Jonas Feigl
Rank
Userlevel 5
Badge +17

 @Manulak 

I’m not looking at this from an IFS mind set. I’m looking at this strictly from an OData documentation perspective. Yes it does fail and so it should as this is what the OData specification defines.

Look at it this way:

If the OData endpoint defines a String attribute with a MaxLength=200 and you send a String with 205 characters you would not expect IFS to cut off the last 5 characters just because “String is a primitive data type of OData” either. Why would you expect this for DateTimeOffset?

 

The OData specification defines Datatypes (like String and DateTimeOffset) but then also describes Facets which put further restrictions on top of these base types. For String there is the “MaxLength” Facet which limits the maximum length and for DateTimeOffset there is the “Precision” Facet which limits the maximum precision of fractional seconds. 

 

You are simply using the DateTimeOffset type with too much precision in C# and that is not a fault within the IFS REST APIs but with the values you provide.

If you replace

DateTimeOffset.Now

with something like the following (which removes milli, micro and nano seconds from the DateTimeOffSet)

new DateTimeOffset(DateTimeOffset.Now.Ticks/10000000*10000000, DateTimeOffset.Now.Offset);

it will work.

 

There is nothing for IFS to solve as IFS strictly follows the OData specification.

Manulak
Rank
Userlevel 4
Badge +8

Good points, @Jonas Feigl 

Thanks for the responses. I’ll try your suggestions.. 

Manulak Dissanayake | https://rb.gy/v88r80
Manulak
Rank
Userlevel 4
Badge +8

Hello @Jonas Feigl 

Because of the standard IFS design of using DATE as the common format for date time, providing a technical solution for the business need of keeping the fractions of seconds saved is nearly impossible rather hard and will take quite a bit of time, hence, after considering the tight time frame of the project, etc., we decided to stick to the bypass solution to truncate the fractions and send it IFS via the payload.

IFS still will work on finding a way to support the fractions of seconds, and I’ll share if I get any update from IFS as promised.

For the moment, this workaround (or something else similar) will be the the solution for us.

Manulak Dissanayake | https://rb.gy/v88r80
Terms & ConditionsCookie settings