Deprecation of the FTP and Shared repository types in Docman in 24R1 - a heads-up

  • 5 October 2023
  • 57 replies
  • 2074 views
Show first post

57 replies

Mathias Dahl
Rank
Userlevel 7
Badge +30

A more technical question related to the subject. It is specific for Remote Deployment but also interested for Cloud Deployment.

The document files are temporarily stored in the Application Server when working with IFS Applications 10 and older (at least IEE version).

How does it work in IFS Cloud?
Will the documents be temporarily stored somewhere during the process or will they “only” pass by the Kubernetes Cluster on the way to the client? 

Hi Stefan, thanks for asking here.

The file content is streamed all the way, we are not creating any temporary files. All parts of the flow that's controlled by IFS is written in Java and there, instead of working with files, we are working with input and output streams. While it was possible in Apps 10, it's much harder to handle temporary files in Kubernetes, especially if they are large, since each container has a fixed amount of storage space.

 

/Mathias
M
Rank
Badge +1

Hi Mathias Dahl

We are a larger company that is currently in the process of porting to ifs cloud.
(remote managed solution, servers are on premise)

We are using DOCMAN heavily and today there are more than 4 million files (Size over 3 TB) increasing daily.
Our today Solution is relaying on Shared Folder (Shared repository). The main problems of changing to "IFS Cloud File Storage" would be:

a) no file server likes to have more than some 100k files in the same folder
b) we see high risk to be non-compliant with government orders and regulations, if we are no longer able to separate Files in different folders and locations,  which could cause in massive negative impact to our business
c) we have documents on some different (older) ifs systems where we can use the "old" servers without need to duplicate the physical files
d) we have a long-term archive solution that represents the files as UNC path\file

So basically, if the possibility to use shared folders (Shared repository) is removed with 24/1 we will run into a deep problem that will seriously impact our project in a most critical way.

what are the chances, that we still can use shared folders and what are the required measures on IFS side to ensure this?

Regards

Stefan

B
Rank
Badge +2

@Mathias Dahl

 

I've just come from a meeting where we discussed exactly this topic. 
Concentrating on repository type "file storage" may well be the right way from a technical point of view. But the limitations and restrictions that this decision entails are significant for our customers. I fully agree with the existing opinions that i have read.
But this change is only half of the truth. Our customer projects always have the same requirements.
It is about internal order documents created by IFS, e.g. order confirmation. This document (PDF) should be automatically attached to the object (customer order). IFS offers "Report Rules" to use for this purpose. This also works well as a standalone solution. But Report Rules do not work together with repository type "File Storage". It is currently not possible in IFS CLOUD to implement a continuous process flow with standard functionality. Creation and printing of an order document incl. attachment to an object and then a transfer to a repository (file storage). We practised for a long time in our last customer project and then we had to go back to "Shared Repository". You may can imagine that our customer wasen’t really happy.
My expectation is, that IFS will deliver a standard solution for this requirement in a short time.

Thanks and best regards,
Beat

 

Mathias Dahl
Rank
Userlevel 7
Badge +30

Thanks @BytBeatbW 

We are aware of the fact that not all functionality that worked with a Database repository in Docman works with File Storage. I think there is a plan to handle each gap there.

As for the problem you mention (archiving reports in Docman), depending on your usage, it should not be a problem to keep those types of documents in the database, even if the bulk of your documentation reside in File Storage. I say "depending on your usage" because it might be that you are archiving so many documents that it becomes a problem with the database size growing too much. Archiving reports doesn't strike me as a process that should fill the database too much though, but then again I don't know how many reports your customers are archiving…

This issue has little to do with the deprecation of FTP and Shared repository types though, and much more to do with the long term focus on File Storage.

Although it's related, it's not the same thing. We have not decided to remove the Database repository type yet.

 

/Mathias
Mathias Dahl
Rank
Userlevel 7
Badge +30

@maxonbs 

 

> a) no file server likes to have more than some 100k files in the same folder

I have been here a long time and I have seen that problem, at least in the old days.

These seems to be the limits of file systems in use today:

File System    Maximum Files per Folder

NTFS           4,294,967,295 (2^32 - 1) files

ext4           Variable (Practically unlimited)

XFS            Variable (Practically unlimited)

Btrfs          Variable (Practically unlimited)

ext3           Around 32,000 to 64,000 files (depends on block size)

Seems we should avoid ext3 nowadays... Apart from that, we should be fine.

It's also something you as a customer should not need to think about. If IFS Cloud File Storage supports "local" file storage in Remote, it's its job to make sure it works for a very large number of files. If there turns out to be problems with too many files, it's something IFS needs to figure out a solution for.

Agreed?

> b) we see high risk to be non-compliant with government orders and regulations, if we are no longer able to separate Files in different folders and locations,  which could cause in massive negative impact to our business

We would like to challenge this, and/or understand more. It seems to be similar to what I'm commenting on here, even if there might be other underlying reasons:

https://community.ifs.com/document-management-docman-248/reinstate-support-for-external-document-repositories-and-mulitple-repositories-42349?postid=158085#post158085

> c) we have documents on some different (older) ifs systems where we can use the "old" servers without need to duplicate the physical files

Are you talking about backup and recovery here? If so, it's an interesting point and it's something that should be directed against the IFS Cloud File Storage team (a way to put away files to slower/cheaper hardware/storage).

> d) we have a long-term archive solution that represents the files as UNC path\file

It should be possible to make that work with the SMB storage support in IFS Cloud File Storage, don't you think?

> what are the chances, that we still can use shared folders and what are the required measures on IFS side to ensure this?

The idea is to remove the current support for shared folders “soon” and replace it with support for multiple storage locations (also shares) in IFS Cloud File Storage. 
 

/Mathias
Tor
Rank
Userlevel 3
Badge +4

Very useful heads-up @Mathias Dahl . May I ask a somewhat related question? My customer has got classified data in their files that they don’t want to leave their network, but IFS is running at an IFS partner data center. If using File Storage would the files ever be stored (temporarily) on IFS servers using this option? 

Mathias Dahl
Rank
Userlevel 7
Badge +30

Very useful heads-up @Mathias Dahl . May I ask a somewhat related question? My customer has got classified data in their files that they don’t want to leave their network, but IFS is running at an IFS partner data center. If using File Storage would the files ever be stored (temporarily) on IFS servers using this option? 

The data is never stored on disk in the "middle tier" (a Kubernetes cluster with multiple containers), but the data will of course go through it, via the network calls that streams the files from and to the backend storage.

 

/Mathias

Reply


Terms & ConditionsCookie settings