Question

Error when importing Document via REST API with Chinese characters in document title

Thilini Kumarasinghe
Rank
Userlevel 7
Badge +18

Test scenario provided by customer:

 

1. Create Document Number

 

POST https://sbm-dev.ifsworld.com/int/ifsapplications/projection/v1/CreateAndImportDocument.svc/CreateDocument

BODY:

{

"DocClass": "2000",

"DocNo": "MAWOUK-CHI-TEST002",

"Title": "P3+SS-G040200001+D版+管理体系手册【三体系】",

"BookingList": "null",

"Id1": "null",

"Id2": "null",

"CreateFileRef": "YES",

"DocRev": "R1",

"DocSheet": "1"

}

 

2. Upload a file and link it to a document revision

PATCH https://sbm-dev.ifsworld.com/int/ifsapplications/projection/v1/CreateAndImportDocument.svc/EdmFileSet(DocClass='2000',DocNo='MAWOUK-CHI-TEST001',DocSheet='1',DocRev='R1',DocType='ORIGINAL',FileNo=1)/FileData

HEADERS to add to request:

Content-Type: application/octet-stream

If-Match: *

X-IFS-Content-Disposition: filename=UDMrU1MtRzA0MDIwMDAwMStE54mIK+euoeeQhuS9k+ezu+aJi+WGjOOAkOS4ieS9k+ezu+OAkS5wZGY=

 

BODY:

Binary, file to upload data from (P3+SS-G040200001+D版+管理体系手册【三体系.pdf) attached to this case.

 

 

 

{

    "error": {

        "code": "UNEXPECTED",

        "message": "Unexpected internal server error occurred."

    }

}

 

 

 

If you perform the same actions, but with a  non-Chinese named file and therefore a different X-IFS-Content-Disposition header value, then the process works

 

Support has tried to use the same document from the arena application, and it works fine. Try to capture the data from fiddler and could not find anything related.

 

please note that the REST API calls used by the Aurena client, and by the integration are different.

For the integration of documents, there is a specific CreateAndImportDocument Projection provided by PD using the 'integration' path, or 'Compatibility' as it is called in the API explorer.

 

The exact call has been provided and is similar in content to the Aurena CLient call, however it appears that the Integration call does not handle Chinese characters correctly

 

Question:

 

Please can you review any formatting related to the document title that is required or any encoding restrictions. If there’s any conversion or character set needs to be passed,  can anyone inform us please ?

www.linkedin.com/in/thilini-kumarasinghe-7ab61b78

15 replies

Mathias Dahl
Rank
Userlevel 7
Badge +30

Hi,

You need to Base64-encode the file name. You can see this easily in Chrome. Here is my uploading a file with the same name using the Upload Document assistant:

 

FYI, unless you really want the original file name saved in the database, you don’t need it. You only need to ensure the file extension is the correct one, since we use that to know what file type to save. Still the filename needs to be Base64-encoded. 

Good luck!

/Mathias

/Mathias
M
Rank
Userlevel 3
Badge +6

Hi Mathias,

I can see in your comment and in the original question that the filename is being BASE64 encoded:

X-IFS-Content-Disposition: filename=UDMrU1MtRzA0MDIwMDAwMStE54mIK+euoeeQhuS9k+ezu+aJi+WGjOOAkOS4ieS9k+ezu+OAkS5wZGY=

 

When performing the action in the Aurena client the process completes successfully (as you have also mentioned).
When uploading from an integration we are using the ‘CreateAndImportDocument’ projection where the documentation provided instructs us to use the following call to patch the file data:

PATCH {baseURL}/int/ifsapplications/projection/v1/CreateAndImportDocument.svc/EdmFileSet(DocClass='2000',DocNo='MAWOUK-CHI-TEST001',DocSheet='1',DocRev='R1',DocType='ORIGINAL',FileNo=1)/FileData

HEADERS added to the request (including base64 encoded filename):

Content-Type: application/octet-stream

If-Match: *

X-IFS-Content-Disposition: filename=UDMrU1MtRzA0MDIwMDAwMStE54mIK+euoeeQhuS9k+ezu+aJi+WGjOOAkOS4ieS9k+ezu+OAkS5wZGY=

 

BODY:

Binary, file to upload data from (P3+SS-G040200001+D版+管理体系手册【三体系.pdf) 

 

This has been tested using POSTMAN and it fails.

This has been tested using Dell Boomi and it fails

 

When tested with a file with non-chinese characters in the name; it succeeds.

It would appear that the CreateAndImportDocument projection and REST API are handling the name differently to the Aurena client projection (even though the same X-IFS-Content-Disposition: filename=UDMrU1MtRzA0MDIwMDAwMStE54mIK+euoeeQhuS9k+ezu+aJi+WGjOOAkOS4ieS9k+ezu+OAkS5wZGY= can be seen when debugging)

Mathias Dahl
Rank
Userlevel 7
Badge +30

The If-Match header is wrong, I think.

It should look like this:

 

/Mathias
M
Rank
Userlevel 3
Badge +6

Hi Mathias,

The If-Match: * header operates correctly for all other PATCH calls for uploading document filedata.
This is also as designed using the UPSERT functionality of the system.

To validate this I have also tested with a document that was created and the ETag retrieved prior to the PATCH call so that I could test it.

The same error was observed.

When attempting this with a base64 encoded filename in the ‘X-IFS-Content-Disposition’ header that was not originally containing Chinese characters, then this executes without error.

Could there be something that is decoding the Chinese characters in an incorrect character set?

/Matt 

Mathias Dahl
Rank
Userlevel 7
Badge +30

Hi Mathias,

The If-Match: * header operates correctly for all other PATCH calls for uploading document filedata.
This is also as designed using the UPSERT functionality of the system.

To validate this I have also tested with a document that was created and the ETag retrieved prior to the PATCH call so that I could test it.

The same error was observed.

When attempting this with a base64 encoded filename in the ‘X-IFS-Content-Disposition’ header that was not originally containing Chinese characters, then this executes without error.

 

Ah, yes, you are right. If the ETag was wrong all files should have failed, sorry. I just remember that uploads have failed with the wrong ETag earlier.

Could there be something that is decoding the Chinese characters in an incorrect character set?

Perhaps. I had a look at the code in Docman and the decoding does not happen there. So it must happen in the OData Provider. Which character encoding did you use in the string before you did the Base64 encoding? My guess is you should use UTF-8.

@chanaka-shanil Do you know who might know the answer to this?

Just to rule out any error in this particular REST API, did you also try any of the other projecttions we have that allows file upload? For example the CreateDocuments or the DocumentHome/DocReferenceObjectAttachmentHandling.

 

 

/Mathias
M
Rank
Userlevel 3
Badge +6

Hi @Mathias Dahl , @chanaka-shanil,

Do we have any further information on this?

The issue only arises when the Chinese filename is base64 encoded; this is a business requirement to keep the full and correct filename.

Thanks,
Matt

Mathias Dahl
Rank
Userlevel 7
Badge +30

Hi Matt,

Could you have a look at the question and suggestion I added in my previous reply?

Perhaps you should also file a proper support case for this such that it get the attention it deserves. I am not sure if Chanaka have the bandwidth to read and comment here.

By the way, about the business requirement: do you know how the customer plan to use the file name once the document has been imported? Do they really need it under the File Refs tab (Original File Name)? If yes, for what purpose? Not saying it is not useful (we do support keeping the original file name after all), just trying to understand how they use the system.

Thanks!

 

/Mathias
M
Rank
Userlevel 3
Badge +6

Hi Mathias,

A support case (G2275908) has already been raised and it was the case owner @Thilini Kumarasinghe who created this community item.

@Thilini Kumarasinghe, can you please investigate this issue further.

Mathias Dahl
Rank
Userlevel 7
Badge +30

Regardless if we discuss here or on the support case, I think it would help us help you if we could get some more information as per what I have asked above.

 

/Mathias
M
Rank
Userlevel 3
Badge +6

In case anyone else comes across this issue or similar; the solution was the ‘type’ of BASE64 in use…

There are 2 subtypes of BASE64… ‘std’ and ‘url compatible’.
Always ensure that the URL compatible version is used.

Mathias Dahl
Rank
Userlevel 7
Badge +30

Hi,

Thanks for reporting back here.

I did not think there were more than one way to do Base64 encoding. Did you perhaps encode the output of the Base64 encoding using URL encoding?

I wonder why this is needed though… Might even be a bug someplace...

 

 

/Mathias
M
Rank
Userlevel 3
Badge +6

I too thought that Base64 was  Base64, but there is also Base64url:
 

This is needed when using URL’s such as in REST API calls to stop certian characters effecting the URL string/data

Mathias Dahl
Rank
Userlevel 7
Badge +30

Thanks! What I don’t get is why it is used in this context, when we are PATCHing data. The Base64 encoded file name is not part of a URL.

I’ll see if I can dig more into this…

 

/Mathias
Mathias Dahl
Rank
Userlevel 7
Badge +30

Got some answers on this one. Firstly, it is done on purpose, and secondly it is documented here:

http://techdocs/apps10/Foundation1/050_development/024_integration/010_restful_odata/default.htm#Common_issues_errors

That’s an internal documentation site for Apps 10. I tried to find the same document in the public site for IFS Cloud but the same information does not seem to be there:

https://docs.ifs.com/techdocs/21r2/010_overview/240_integration/400_rest_apis/

I will ask if the same can be added there. For sure this should be added to the API documentation for the premium Docman API, but it should also be in some more central or common place.

 

/Mathias
Peter Sketch Boomi
Rank
Userlevel 3
Badge +7

Hi @Thilini Kumarasinghe , @Mathias Dahl and @Matt Woonton 

I’ve come across this base64 thing previously as well.  I believe it’s linked to the historic use of base64 to encode mime content -it’s basically wrapping the text and adding cr characters in there, which breaks the url of course.

@Thilini Kumarasinghe , if you’re actually trying to do this from Boomi, I’ve built a process to do all of this for you - including doing things like check the extension is registered before uploading etc..  Let me know if you’d like it - happy to share it.

 

 

 

In the operation “Encode Filename to Base64 and strip CR”, I do a Base64 encode and then do a search/replace on \R

 

 

Peter

ISV Lead @ Boomi

Reply


Terms & ConditionsCookie settings