Deprecation of the FTP and Shared repository types in Docman in 24R1 - a heads-up

The data is never stored on disk in the "middle tier" (a Kubernetes cluster with multiple containers), but the data will of course go through it, via the network calls that streams the files from and to the backend storage.

@maxonbs 

> a) no file server likes to have more than some 100k files in the same folder

I have been here a long time and I have seen that problem, at least in the old days.

These seems to be the limits of file systems in use today:

File System    Maximum Files per Folder

NTFS           4,294,967,295 (2^32 - 1) files

ext4           Variable (Practically unlimited)

XFS            Variable (Practically unlimited)

Btrfs          Variable (Practically unlimited)

ext3           Around 32,000 to 64,000 files (depends on block size)

Seems we should avoid ext3 nowadays... Apart from that, we should be fine.

It's also something you as a customer should not need to think about. If IFS Cloud File Storage supports "local" file storage in Remote, it's its job to make sure it works for a very large number of files. If there turns out to be problems with too many files, it's something IFS needs to figure out a solution for.

Agreed?

> b) we see high risk to be non-compliant with government orders and regulations, if we are no longer able to separate Files in different folders and locations,  which could cause in massive negative impact to our business

We would like to challenge this, and/or understand more. It seems to be similar to what I'm commenting on here, even if there might be other underlying reasons:

https://community.ifs.com/document-management-docman-248/reinstate-support-for-external-document-repositories-and-mulitple-repositories-42349?postid=158085#post158085

> c) we have documents on some different (older) ifs systems where we can use the "old" servers without need to duplicate the physical files

Are you talking about backup and recovery here? If so, it's an interesting point and it's something that should be directed against the IFS Cloud File Storage team (a way to put away files to slower/cheaper hardware/storage).

> d) we have a long-term archive solution that represents the files as UNC path\file

It should be possible to make that work with the SMB storage support in IFS Cloud File Storage, don't you think?

> what are the chances, that we still can use shared folders and what are the required measures on IFS side to ensure this?

The idea is to remove the current support for shared folders “soon” and replace it with support for multiple storage locations (also shares) in IFS Cloud File Storage. 
 

@Mathias Dahl

I've just come from a meeting where we discussed exactly this topic. 
Concentrating on repository type "file storage" may well be the right way from a technical point of view. But the limitations and restrictions that this decision entails are significant for our customers. I fully agree with the existing opinions that i have read.
But this change is only half of the truth. Our customer projects always have the same requirements.
It is about internal order documents created by IFS, e.g. order confirmation. This document (PDF) should be automatically attached to the object (customer order). IFS offers "Report Rules" to use for this purpose. This also works well as a standalone solution. But Report Rules do not work together with repository type "File Storage". It is currently not possible in IFS CLOUD to implement a continuous process flow with standard functionality. Creation and printing of an order document incl. attachment to an object and then a transfer to a repository (file storage). We practised for a long time in our last customer project and then we had to go back to "Shared Repository". You may can imagine that our customer wasen’t really happy.
My expectation is, that IFS will deliver a standard solution for this requirement in a short time.

Thanks and best regards,
Beat

Hi Stefan, thanks for asking here.

The file content is streamed all the way, we are not creating any temporary files. All parts of the flow that's controlled by IFS is written in Java and there, instead of working with files, we are working with input and output streams. While it was possible in Apps 10, it's much harder to handle temporary files in Kubernetes, especially if they are large, since each container has a fixed amount of storage space.

Hi @chanaka-shanil,

May I get your advice on the matter, please. 

Your support is much appreciated.

Thanks

Thanks @Kelum-WIA 

Then I think they should migrate to FS now. We have a built-in assistant for that. As I said, I don't think they will be able to use another Azure Storage account, so they will not save money that way, but they should save money on not using the more expensive database storage, so it should be a win anyway.

Hi @Kelum-WIA 

 > currently uses the default Database ... They won’t move from Database to File Storage

Do you know why? File Storage is cheaper and backup and restore operations quicker.

> the File Storage uses an Azure Blob Storage Account to store files in the Cloud deployment model. Is it possible to have a completely different Azure subscription for this Azure Blob Storage?

This is really a question about the IFS Cloud File Storage feature and not Document Management, but no, not as far as I know.

Any comments, @chanaka-shanil ?

Hi Mikael,

What I have understood is that it is only possible to define one File Storage location in Cloud version.

Correct. In Cloud Deployment, all files are stored in Azure Storage and in Remote Deployment all files are stored under one SMB/Samba/Windows network share.

This will cause problems for e.g. Defence industry when classified document shall be stored. The demand is that they must be stored in the country where the XC license is valid.

For an installation where there are sites in different countries it must be possible to define more than one location as each country have their own XC licenses.

It's a problem but it's not a new problem and we haven't supported that since Apps 7 when our "fat Windows client" directly communicated with the document file repository. The change/deprecation we intended to communicate through this post does not change anything related to what you wrote there.

How is this demand addressed in the change of the functionality?

It's not addressed now and it was not addressed since Apps 7 (which had it's own problems, mostly from a security point of view).

To summarize, we argue that the change/deprecation neither makes the situation better or worse with regards to the requirement you describe (which is about ITAR, Export Control, whatever you want to call it).

Is it clear?

Thanks for reporting that. Seems the document moved here:

https://docs.ifs.com/techdocs/23r1/030_administration/210_cloud_file_storage/#introduction

I updated the link in my original post.

@kjro That’s not in the plans as I understand things, no.

@chanaka-shanil - FYI

@dean

This is the problem.  IFS wants all customers to move to the hosted cloud, but is removing functionality that would enable them to do so.  

IFS probably want that, yes, but still see that there is a need for “on premise” in some cases. Here, the needs of cloud operations, and the certifications they work hard to adhere to, conflicts with the need of some customers, and then another option is needed (“Remote”). It makes sense to me.

IFS wants to promote that the IFS applications is one product and is the same whether implemented in the IFS Cloud or Remote,  a key selling point that differentiates us from our competition.  and what you are saying here is that it is different and functions different.

Let’s sat that IFS Cloud consists of, say, one million “features” (~8000 different object types, where each object type surely have hundreds of behaviors or features, on average).

I don't think it's unreasonable that, due to the very different "execution context" (public cloud, with all that entails when it comes to security, for example vs on premise hardware controlled by customer), a few of those features will not work the same, or cannot be implemented in either of the deployment options.

Based on your comment regarding performance,  I could be wrong but in earlier versions of IFS the ftp was implemented in the thick client, not the application server.

We left the thick client when we released Apps 7.5, over 10 years ago. Since then, all file transfers have been going via the application server.

...in either case if find it interesting that you would not implement across multiple regions for performance, especially with global companies.

I think it's not impossible to get there, but we need more features to get there, each one needs to be prioritized and investigated and I think our cloud operations would also need to evolve to support that.

We have print servers and other services that connect IFS to on-prem solutions

I didn't know that and since that's news to me I cannot comment on if it's relevant or not to the discussion.

Regardless, was I able to explain to you clearly what the idea is that you need to register, about IFS Cloud File Storage supporting multiple shares (and Docman needs to be adjusted to use that feature)?

@Mathias Dahl  “As you know, IFS Cloud in Cloud Deployment does not support storing documents on hardware owned by the customer. There we only support database or IFS Cloud File Storage with Azure Storage as the backend. That's a business decision made by IFS (not "R&D") when IFS Cloud was first released.”

This is the problem.  IFS wants all customers to move to the hosted cloud, but is removing functionality that would enable them to do so.  IFS wants to promote that the IFS applications is one product and is the same whether implemented in the IFS Cloud or Remote,  a key selling point that differentiates us from our competition.  and what you are saying here is that it is different and functions different.

Multiple shares/repositories in the cloud does address the issue of segregation of files, and that is a good thing and will address one of the use cases of multi company and multi country.

Based on your comment regarding performance,  I could be wrong but in earlier versions of IFS the ftp was implemented in the thick client, not the application server.  in either case if find it interesting that you would not implement across multiple regions for performance, especially with global companies.

Regarding “all network traffic anyway goes through a single point. This is why our customer SAAB, which need to follow export control regulations, cannot fully use Docman for all their document management.”    Explain your reasoning behind this,   

We have print servers and other services that connect IFS to on-prem solutions, seems like it would be better to resolve the issue so our customers can get the full value of IFS Docman rather than forcing them to either not use the solution or be limited.

No what I want is the solution to support multiple repositories and external non-cloud repositories as the current functionality in IFS 10 supports and every previous version since IFS 98. Just having multiple shares in the cloud is not enough.