Hi Mathias,

One scenario comes to my mind is report rules only work for database option. Therefore  need to consider that before deprecating db option.

Regards,

Sahan  

Who benefits from this and how do they benefit?  This seems rather capricious and difficult for customers who rely on currently available options.  Aren’t there more important things we could do to help our customers instead of making their lives more difficult?

Hi Mathias,

Regarding possible deprecation of the db option I am for the moment maybe too stuck in “old” pros we’ve been communication for a while now, to be able to see this is a good way to go. There are of course cons with db storage as well (like db size and recovery issues) but the pros I’m thinking of are;
- Easy copying of a prod environment to test retaining documents and connections,
- SOX (Sarbanes Oxley) support for tracking read and write activities on files,
- Strong security (Oracle as a container also for the documents).

Are there any means to mitigate or meet those arguments in a solution with only “external” storage?

Regards,
Erik

My comments on your valid comments on database storage:

Easy copying

This is true. In our managed cloud offering, I think they are working on a way to clone the files in the File Storage repository (which use Azure Storage). As for Remote deployment, I think you are on your own, like with FTP and Shared repositories.

SOX support

With IEE out of the picture, there is no longer a need to be able to read data directly from the database view, which means we can control all access to document file data with code, which in turn mean we can log every file operations. So I would argue that the SOX support will be as strong as before.

Security

I think it's hard to beat the security we have in Oracle, and keeping the files in a separate place introduces another attack vector for sure, but I think with the right setup (only the middle tier/Kubernetes should be able to access the file server) you can get strong security even with external storage.

By having fewer repository options we will save time spent on maintaining them, and we can spend that time on "fun" things which will benefit customers. Also, FTP support is getting rather long in the tooth and does not provide much benefit over IFS Cloud File Storage, provided by the platform.

Can a customer access documents stored on Sharepoint if set to External?

There is no direct support for using SharePoint as a repository in IFS Document Management.

Seems these options are there for ages. Thus, I’m a bit skeptic about the workload spent in support. We’ve been smart enough to let our customers choose between remote and Cloud hosting, which a differenciator from the competition. Even if Cloud hosting on Azure is in our favor from a business perspective of course. It’s like we want our customers choose the most expensive option with file storage or premium database.

I think there is a misunderstanding here.

IFS don't support FTP and shared repositories in Cloud deployment and in Remote customers will have two options as well, File Storage and Database, both under their complete control. In Cloud the cheapest option is for sure File Storage.

It's also not the case that, once implemented, something like FTP support is free of maintenance. Platform and other technological changes requires reimplantation. I think we have had three or four implementations of FTP support already now. There is a cost with that.

@Mathias Dahl ,

We currently use a hybrid approach with document management between database and shared folders.  We typically keep our large volume of document attachments in shared folders to prevent database bloat and lower volume or more sensitive documents in the database.  With IFS decision to deprecate shared folder in 23R2 and eliminate shared folders completely in 24R1 we have decided to migrate all documents into our database but this is not ideal for us.  Moving into the database or to a Cloud File Storage adds cost for us we just hadn’t planned on.  Clustered NAS storage isn’t to expensive in our data center and it kept all our documents within our facilities.  Yes moving them into the database will keep them within our data center but database storage is more expensive for us as its on our SAN’s using SSD which we really don’t need for documents that are access irregularly.

We seem to be incurring more and more costs with IFS Cloud and functionality we’ve come to rely on is being removed.  While we like the IFS Aurena interface we are concerned with what we will have to redo just to get back to where we are with IFS Application 10 and IEE.

Regards,

William Klotz

@william.klotz 

I think there is a misunderstanding here.

The ability for cheap external (outside of the database) storage in Remote deployment will not go away. Can you read my original post again?

Hi @Mathias Dahl ,

I must have read your initial email 4 or 5 times and I didn’t get it until your last response where I decided to reference the online documentation and go through the IFS Cloud File Storage information.  Then I read your initial email again and I understood what the second paragraph under TODAY was saying.  

I would recommend you include this reference to the documentation will visually really shows what you are explaining.  For me it really did the trick.

https://docs.ifs.com/techdocs/23r1/020_lifecycle/100_file_storage_service/#introduction

Regards,

William Klotz

Thanks, @william.klotz, I added a link to the technical documentation on IFS Cloud File Storage now. So, you are all good now, right? :-) 

Hi @Mathias Dahl ,
Currently, we are also using a hybrid approach with repository types Database and Shared.

Database is used for frequently used documents that require quick access. 

Repository type Shared is used for 2 different types of documents. These 2 different types of documents are assigned to several different document classes.
a) Documents with legal requirements to electronic archiving. This applies for all types of outgoing and incoming invoices and contracts. 
legal requirement: For (tax) audit security, only data carriers that can be written once (CD-R, DVD-R, Blue-ray) or special, hard-drive-based archive software come into consideration. This repository is special WORM, relatively expensive.

b) Documents that are generated by different IFS transactions (eg. purchase orders, sales quotations, customer orders, delivery notes, ...) without legal requirments. These repositories are on cheaper hardware.

Based on your description and the information in About Document file repositories , I understand it is not possible to use different physical storage devices for different document classes, because this would require the possibility to use multiple existing shared network folders. My understanding is, that IFS cloud file storage only supports the configuration of one single SMB share path in custom_values.yaml

I understand, that there are pros and cons of the different repository types. Anyways, with the move to IFS Cloud, migrating our document repositories will require a considerable effort, even with the migration tools IFS will provide for this.

In addition to the above, reality is, that documents are also created and stored in other source systems like MS-Teams/Sharepoint environments or PLM-systems (eg. drawings in Siemens Teamcenter) just to name some examples. As processes span across different systems and integration gets thighter, my question: Is there any intention for data virtualization of documents? Meaning, to have the possibility to connect documents from other source systems / source repositories to IFS objects, without the need of physical duplication of document file data and maybe even document meta data.

Regards

Johannes Wittwer

@JohannesWittwer 

Hi, thanks for your comments and sharing the setup you use. I will comment on some of what you wrote, below.

Database is used for frequently used documents that require quick access.

Can you explain what "quick" means, here? Is it noticeable faster for your users to view a document that's stored in the database compared to being stored externally?

Based on your description and the information in About Document file repositories , I understand it is not possible to use different physical storage devices for different document classes, because this would require the possibility to use multiple existing shared network folders. My understanding is, that IFS cloud file storage only supports the configuration of one single SMB share path in custom_values.yaml

That's correct. Since we will rely on the capabilities of IFS Cloud File Storage, we will be limited with what it provides. I would recommend you file an idea here on IFS Community to have our platform team consider more flexibility in this area.

Is there any intention for data virtualization of documents? Meaning, to have the possibility to connect documents from other source systems / source repositories to IFS objects, without the need of physical duplication of document file data and maybe even document meta data.

The only thing I can think of is that we discussed how we could integrate with Microsoft Teams. But I think the main intent there is to, in some way, collaborate on documents within Teams and, at a later time, check in the final documents to IFS Document Management.

In general, a simple way to keep track of documents in other systems is to check in "link files" to them in Docman. It can be a .lnk or .url file as per what Windows supports, or you can check in a simple text file with a link in it and a macro could follow/open the link. Or the file could be an empty dummy file and the link can be stored in some field on the document revision and the macro could open that one. This solution is easy to setup but has limits when it comes to editing documents and when it comes to authentication (the user will have to login to that other system).

Thanks!

This is a huge mistake and will cause many customers to not opt for IFS in the cloud.  Many of our customers in the Utilities Industries and those that serve Aerospace and Defense industries are dependent on the capability to have external document storage for certain classes of documents. these documents need to be stored on prem or on secure non cloud vaults by regulation.   The ability to link these documents to metadata and processes is one of the key features of IFS.

This lack of capability will force many companies in regulated industries to opt for an on prem solution or potentially a competitor solution that is cloud based that allows this capability.    Not having these capabilities is taking away key functionality existing customers are using and will hurt us in the marketplace.

You mentioned checking in “Link” documents in a previous message however I have found that doing that is somewhat inconsistent in its behavior and doesnt work very well. 

Hi @DeanBurell ,

About this:

Many of our customers in the Utilities Industries and those that serve Aerospace and Defense industries are dependent on the capability to have external document storage for certain classes of documents.

External storage (storage outside the database) is not going away. What's going away is Docman's explicit/native support for FTP and Shared repositories. It's replaced by Docman using IFS Cloud File Storage which now has the option for external storage in Remote Deployment. And, like before, you can control what documents classes are stored in the database and what documents are stored externally. Both in Cloud and Remote Deployment.

This is a point that others seem to have missed too and I can only apologize for not expressing this point clearly enough.

I hope this helps.

Hi @Mathias Dahl ,

As i understand the requirement mentioned by @DeanBurell, my question for clarification is:

Will the future solution support the capability to assign document classes to different external file storages. 

/Johannes

@Mathias Dahl  @JohannesWittwer Correct.  Cloud storage for documents is not allowed for certain types of documentation in the US utility industry (sensitive documents that expose cyber security risks).  Also for companies that serve both commercial and defense industries that do not have a full requirement to be in gov cloud but do have some classes of documents that are ITAR (drawings) or have similar requirements.

The ability to have an on prem/noncloud  document vault/repository is a critical requirement.

It sounds like from your description that you will not support a cloud deployment with a non cloud external repository.  This is a problem not only with new cloud prospects, but preventing existing customers from moving to the IFS Cloud.

Dean

@DeanBurell 

It sounds like from your description that you will not support a cloud deployment with a non cloud external repository. 

That's where we are today, yes.

If you need storage outside the database and cannot use Azure Storage, then Remote Deployment is your only option, and has been for a number of releases now. What I wrote in my original post here doesn't change this fact.

What changes is the number of storage options outside the database in Remote Deployment, from 3 to 1. And, at least initially, the setup will be less flexible since you can only configure one external location.

I recommend everyone who want more options and capabilities to create an idea in the Ideas section here on IFS Community.

Thanks!

@Mathias Dahl  by number of releases you mean IFS Cloud.  because  this was supported up through IFS 10 and had been since I started working with IFS in 1997.    What you have essentially done is remove functionality that customers have been using for 20+ years and have now prevented them from using IFS in the Cloud.   

You have also hampered any efforts to sell IFS in the Cloud to any new customers that have regulatory requirements or the desire to store documents externally for other reasons i.e. size, network distribution, etc.  We will now need to either get them to pay the 30% premium to move into Gov Cloud or recommend they go on prem, which will probably get us eliminated from opportunities that are looking for cloud solutions.

If your goal is to get all of the customers into the IFS hosted cloud, this just worked against that.

This isnt about people who want more options and capabilities, it is about you taking away options and capabilities they already have with out asking.  and now you are also only enabling 1 external location?   You do realize that IFS is a multi country, multi company application?   we utilize the multiple location capabilities to segregate documents across companies in IFS as well.   

I am getting the feeling that R&D is not thinking through these changes or asking any of their customers or partner about the impact of removing functionality prior to doing so.

Dean

@DeanBurell 

Thanks for the input. 

I recommend you file an idea here on IFS Community to get support for multiple storage locations in IFS Cloud File Storage in Remote deployment. If that can be implemented, Docman can also support it, and it will then be very similar to how it works today with our support for FTP and Shared repositories.

@Mathias Dahl Seems ridiculous to have to create an idea for this as if it were new functionality rather than functionality that already exists that you are taking away.  Perhaps it would be better if you surveyed your customer base before removing functionality.

As I do not have access to all of the customer base, that is using this functionality and I know many people do not frequent the community., the only way I can effectively draw attention to this would be through public social media.  And I would rather not air IFS shortcomings to the world.

I have created an Idea to keep the support of external and multiple repositories. Please promote this to everyone that will be affected…

There will be many people affected... here are just some use cases

Use case 1:  Customer manufactures dual use products and technologies and sells to both  commercial and defense markets. They have some ITAR /defense requirements for some items only, and do not need a full Gov Cloud solution. In this scenario with the capability to have external repositories, we can implement this customer in IFS Cloud, with IFS Cloud hosting and manage any sensitive documents in an external repository, but still have them linked to the appropriate functionality and control their access through security. Their storage in an ITAR controlled repository gives them the coverage they need.

Use case 2: utility provider implementing in IFS Cloud– by law / regulation power and utility companies in NA cannot store potentially sensitive documents on the cloud. Any documents that may contain any information that could expose any information of vulnerabilities cannot be stored in a cloud.  Having these sensitive documents in an on-prem location, but linked to metadata in IFS Cloud enables them to maintain an extra layer of security.

Use case 3: utility provider – utility provider is implemented on prem, but as many of our customers are, they are comprised of multiple legal and operating entities each with their own set of users and documentation.  Because IFS implementation of document management is global, i.e. is not company specific, we implemented multiple repositories and classes for each company  to ensure that we physically kept each companies documents separate. This prevented anyone from being able to gain access to all of the documents even in the event of a breach.

Use case 4: Multi-Company, Multi-National customers – in many cases we would want to implement document repositories in the region they are located and or segregated by country/company for performance and security reasons.   i.e.  companies with Israeli locations or entities, and also locations or entities in the middle east.   Or companies with legal entities in France or Canada which have regulations that require data reside in country.

@ksander @JohannesWittwer @Mike The FSM TechnoGeek  @william.klotz  @Sahan Udana