Question

IFS Cloud - Document Management

  • 17 January 2022
  • 39 replies
  • 1100 views

Userlevel 1
Badge +3

Hello,

On IFS Cloud - We can retrieve documents stored on FTP path (Stored in pervious versions of IFS). But are not able to add any new document. there is no error while uploading. But where retrieving it gives error.  

https://ifsuat.<aaa>/main/ifsapplications/projection/v1/ManageDocuments.svc/DocIssueSet(DocClass='BRIEF',DocNo='BRIEF008664',DocSheet='1',DocRev='1')/EdmFileRefArray(DocClass='BRIEF',DocNo='BRIEF008664',DocSheet='1',DocRev='1',DocType='ORIGINAL',FileNo=1)/FileData might be having issues, or it may have moved permanently to a new web address.


39 replies

Userlevel 1
Badge +3

Hello Dhananji,

Its working fine after SU3. 

Regards

Pankaj

Userlevel 3
Badge +4

Hi,

I m having a IFS cloud customer with IFS Cloud 21.1, SU4 claiming that the issue  he had  in SU2 is still occurring for ftp repository.

Quoting from the customer: 

“When viewing an existing document which was stored in the ftp repository long time ago, the error message is:
{"error":{"code":"ODATA_PROVIDER_ERROR","message":"Ein interner Serverfehler ist aufgetreten. Wenden Sie sich an den Administrator.","details":[{"code":"PROJECTION_IMPL_EXCEPTION","message":"Error when reading the file: Error while downloading the file. Failed to connect to FTP server: IFS-FTP-REP-PROD: System error"}]}}

If you create a new document, no error is shown but the file is not stored in the repository.”

When he first reported at SU2, we have asked him to wait till SU4. 

Any views on this? 

Thank you all in advance. :) 
Dhananji

Userlevel 7
Badge +30

@ORFJAN 

Very glad to hear that, thanks for reporting back here as well!

 

Userlevel 3
Badge +7

Dear all,
just want to share for others here, the issue was found in K8s cluster and fix is coming with 21R2SU3. Having SU3 installed, it works fine then.
Thank you @Mathias Dahl for all your support.
Jan

Userlevel 7
Badge +30

Got some questions from a colleague. I copied them to the case.

 

Userlevel 3
Badge +7

Hi Mathias,
thanks for update, I will post the update to the case too.
No new info/findings have arrived on this via case, so I appreciate to get any.
You are most probably right with the FTP server. So let’s see the finding you have.
Thanks Jan

Userlevel 7
Badge +30

Thanks for the update. If you can, add the same information on the case/ticket so we have the full information there. I can see that we have now got a comment from the other R&D team, in Jira. Not sure if it has yet reached you via ServiceNow. I will have a look at it now.

I would not expect the FTP server being the culprit here, given the tests that you have done, rather it is the container. It works “outside” the container, from the Linux machine that hosts the containers. It does not make sense to me that the FTP server should be causing the problems.

At the same time, it’s worth trying. I am often wrong :)

 

Userlevel 3
Badge +7

Hi Mathias,
yesterday we were trying to monitor the network traffic from linux box to FTP server with using tcpdump  and we were trying to see the difference between upload and download.
We recognized there is no traffic initiated to FTP server on the port where the download is supposed to happen.
While you are investigating we can also try to use another FTP server (with newer OS then WS2012) whether we see the same behavior.  
BR Jan
 

Userlevel 7
Badge +30

I’m out of ideas. “It’s something with the network” :-)

I hope my colleagues who knows the ins and outs of containers will join in soon with some fresh ideas of things to try.

They should be in touch anytime soon.

 

Userlevel 3
Badge +7

Hi Mathias,
pls. find new document attachment to the case with my findings. I used curl to retrieve the pdf file from FTP. I run the command from:
1. MTS Ubuntu OS - works fine, ending with status 226
2. oData container in K8s - does not work, ending with status 550, despite K8s firewall disabled

I would like to highlight also screenshot with FTP log for both scenarios, for scenario #2 it seems the Data Channel is closed with status 258. And I noticed that  the IP address of the client is missing on these lines. It seems the Data Channel communication is not somehow routed back to K8s cluster. Or any other idea?
Thanks Jan

Userlevel 7
Badge +30

Thanks for the update. Yes, I think the firewall rules could definitely be the cause here, even if upload works but download does not. Because of the sometimes hard to understand handling of the different ports when using FTP.

Looking forward to hear the results of further experimentation. I hope we can all learn from this.

 

Userlevel 3
Badge +7

@Mathias Dahl 
just a quick update, I did not manage to run FTP from odata container, but I was testing from linux box and I think I found the issue. The FTP server is by default using quite wide range of ports for passive mode and from security reasons not all of them are opened from the linux box. I will share more details with findings and testing results later today/tomorrow.
 

 

Userlevel 3
Badge +7

Hi @Mathias Dahl
good finding:point_up_2_tone2:. We will try to use FTP client from linux box to make sure upload/downloads work from there.
Thanks a lot for checking the logs! Jan
   

Userlevel 7
Badge +30

Thanks alot! Although it might not look like it, I think there are errors in the FTP log file 🙃 Have a look at my latest comment on the case in Service Now.

 

Userlevel 3
Badge +7

@Mathias Dahl I found some FTP related ERROR entries in the APP dump, attached to the case.

Userlevel 3
Badge +7

Hi @Mathias Dahl ,
I have uploaded following to the case.

  • document with screenshots from my further testing
  • FTP log - but no errors found with there
  • dump from IFS application server

Unf. not too much error messages around, if needed we can have a screen sharing session to look at this together.
Thanks Jan

Userlevel 7
Badge +30

When you try to download the document, do you see any server errors? Please see what I wrote above on how you might be able to see some server errors, using the Aurena Agent and the Document Revision page.

Also, do you have any log information from the download that Docman does?

 

Userlevel 3
Badge +7

I am sometimes very paranoid when it comes to what information I get using technical investigations like these :-) so I must ask: how do you see that the file has been uploaded correctly? If you look “in the FTP server”, then it’s fine. If you trusted Docman that the upload went well, because there were no errors, well, then it is not always the case that it went well :-p

Yep, I used WinSCP FTP client to connect to FTP server and I can see the file there. Pls. see screenshot in the last attachment. 

Userlevel 7
Badge +30

Thanks, @ORFJAN Looks good, I think.

About this that you wrote:

I would like to highlight that on 21R2SU2 there is no issue to upload the file to FTP repository, the file is there

I am sometimes very paranoid when it comes to what information I get using technical investigations like these :-) so I must ask: how do you see that the file has been uploaded correctly? If you look “in the FTP server”, then it’s fine. If you trusted Docman that the upload went well, because there were no errors, well, then it is not always the case that it went well :-p

So, assuming the upload works, it’s strange that the download does not, and I cannot explain it. My only theory is that there is a “communication problem” between the FTP library we use in Java inside the IFS Odata container, and the FTP server. What would be good to try is to run an FTP session from the same machine where the containers run, to the FTP server. And try both upload and download, using passive FTP. EVEN better is to try the same thing from INSIDE the OData container. We’ll see if it will come to that. We might need to test that because the container itself “has a network”, with settings and even a firewall I think. So even if the communication works on the machine where the container runs, it could still be that the communication does not work from inside the container.

With all that said, I will now await some more information from you :)

 

Userlevel 3
Badge +7

Hi @Mathias Dahl , let me reply on question #1 n the attached document, I will be coming with the rest of the points during the day. Jan

Ref. 1. we use FTP with IIS on Windows Server 2012, passive mode supported, proved with FTP client WinSCP in passive mode. Yes we use IP address for the repository.

Document with screenshots was attached to the case.
Ticket Form for Case - IFS Customer Support

Userlevel 7
Badge +30

Also, do you use the IP address or machine name in the repository set up?

 

Userlevel 7
Badge +30

@ORFJAN 

Hi again, I have tested FTP repositories now, in our internal environment:

  • Latest IFS Cloud 21R2 (this is in our development environment)

  • FileZilla FTP server (64 bit) installed on a Windows Server 2012 R2, username = guest, password = guest, virtual path = / mapped to c:\docman\ftp, with full read and write access

  • I used the machine name and not the IP in the repository setup in Docman. Set / as path there. No port set (will default to 21 on the server).

Upload AND download works. In the FTP server log we can see both the upload and download that is done from the IFS Odata container working well.

The only problem is after the download is finished, the client aborts the connection and this shows as an error in the FTP server. The file downloads though, without any visible error on the client side.

If you try to download with the Aurena Agent enabled, do you get any detailed error message showing up in the toast in the lower right corner? By using the Document Revision page you should be able to see any server error. The client errors pasted earlier in this post does not give any clues unfortunately (this is often the case, that we want to see the server errors since the client error is sometimes just a side effect of the server error).

Some questions as well:

  1. Which FTP server do you use?
  2. Are there any logs you can look at there?
  3. Have you enabled support for passive FTP on the server and in your firewall? IFS uses passive nowadays, while we used active before (a change we had to make due to moving into containers).

Thanks!

 

Userlevel 7
Badge +30

we will check the option further however seeing how many issues/gaps/redesigns is connected with the upgrade from IFS10 to IFS Cloud it’s another effort and complexity we adding here and not sure we have capacity for. If you see feasible to have FTP issue fixed with SU4 I would prefer to go that way.

I hear you… Let’s hope that we can recreate the FTP issue internally first, then try to find what the underlying problem can be.

As mentioned above, a fix was done in the container/POD area which fixed at least one specific FTP issue. What surprises me about your problem is that the upload goes well, that is something I don’t remember seeing when we have had FTP problems of our own.

 

Userlevel 3
Badge +7

Hi Mathias,
thanks for sharing, we will check the option further however seeing how many issues/gaps/redesigns is connected with the upgrade from IFS10 to IFS Cloud it’s another effort and complexity we adding here and not sure we have capacity for. If you see feasible to have FTP issue fixed with SU4 I would prefer to go that way.

Thanks Jan

Userlevel 7
Badge +30

Thanks!

Regarding the Shared repository workaround proposal, I can not see that feasible. We have thousands of docs stored on the FTP related to Product Development.

I see. Again, it could just be a workaround until the FTP problem has been sorted out.

You might not know or realize this, but it is also possible to “relocate” your current FTP files to a Shared repo with some file copying plus database table changes.

You “only” need to understand how the (very few) underlying tables are structured and you can with quite little effort copy the FTP files to a share and then update “the file and repository tables” to point to the share instead. We are talking directly updating the database tables here, which may sound scary but is not very much scary :) I recently replied to another post which describes the different tables that are involved:

Again, could be a workaround or a permanent solution. Good to be prepared for the worst, right? :)

And the above would assume you can get a Shared repo working first. We know it works but it can take time finding the correct set up.

 

Reply