Cloud File Storage Migration Tool Support on Remote Deployment

I don't have the time to answer in detail but, if you can spend some time trying to learn about how IFS stores data about the checked in files, in Docman and File Storage, you can probably figure out a way to do the migration much faster than going through the tool. By getting an understanding of how things works under the cover, and it's NOT hard, you will be in control and can possibly save time on troubleshooting any errors from the tool. I have outlined this in other posts but you want to look at all the "edm_XYZ" tables, plus fss_file_tab, to get the full picture. And also, the file names in file storage needs to be base 64 encoded...

Thanks @Mathias Dahl  for your reply, appreciated

@AmilaF We are upgrading from Apps 10 Update 18 to IFS Cloud 23R2. Currently our documents are held in an FTP repository in Apps 10 and like you, we are having to migrate to FileStorage using a SMB Share on a Windows server.

IT have copied the files from the FTP to the SMB Share, and then I have written some PL/SQL scripts to correct the setup within IFS Cloud. 

This seemed to work, and we can view existing and upload new documents.

If you are allowed to share those scripts I'm sure others here would be grateful to have them as inspiration for their own migrations.

Dear @SHAUN_KERSLAKE 

Many thanks for your reply, much appreciated. As indicated by @Mathias Dahl, is it possible to share those scripts for review to see if they could be useful for us?
 

kr

Amila

​@Mathias Dahl 

I thought I'd share an update, as it might be helpful to someone. I tested the Cloud File Storage Migration Tool, which is used to migrate files from an FTP server to IFS Cloud (SMB share). To simplify the process and avoid potential infrastructure issues, I deliberately avoided complex scenarios involving network shares requiring authentication, firewall restrictions, and similar configurations. These issues will need to be addressed based on each customer's situation.

Since I do not have access to an IFS 10 environment for testing, I enabled the deprecated FTP repository in the IFS 24 R1 version and successfully moved the documents from the FTP server to file storage.

As stated in the documentation, the process works as expected when the instructions are strictly followed.

Kind Regards

Amila

​@AmilaF 

Thanks for testing and the feedback. Good to know it does not need to be hard if you try to simplify things, in particular access setup and firewalls.

On this, though:

> I enabled the deprecated FTP repository in the IFS 24 R1 version and successfully moved the documents from the FTP server to file storage.

The deprecation of FTP and Shared repositories has to do with the IFS Cloud. That is, the target system in a migration scenario. So the setting you changed has no impact on what you did 🙂 It would have worked regardless.

Morning,

Sorry didn’t respond to previous posts...we have successfully upgraded to IFS Cloud from Apps 10 as of the start of December!

One of the steps for migrating to Cloud, was obviously around the document management setup. In Apps 10 we had an Azure FTP repository whilst IFS Cloud we went with File share on a Windows server. 

The steps we followed were:

1. IT copied the files across from FTP→ Windows Share. 
2. Once copy completed, IT produced a file listing the file name along with the size of file (This is required for input into IFS Cloud)
3. Create Document Management Repository in IFS Cloud (Document Management->Basic Data->Edm Basic->Repositories) and set to Generating. 
4. Created a data migration job to Import the file lengths for documents into a temp table
5. Executed a custom Oracle PL/SQL Script to Change document Links and add to fileshare (Please message me if you would like a copy of the script)

One of the issues that we experienced with a couple of documents was around the legacy user file name being too long which Cloud didn’t allow.

We were unable to migrate these into Cloud directly, so we took the decision to omit these. 

Hope the above helps others migrating from Apps 10 to Cloud

Regards

Shaun

Thanks ​@SHAUN_KERSLAKE for sharing that. I hope it can inspire others to try a more "custom" migration approach.

The File Storage migration tool, trying to support many scenarios and as many edge cases as we can think of, cannot take the kind of shortcuts that we can do in a customer-specific migration scenario where we know much more about the setup and the data. Doing a custom migration can potentially speed up the actual migration process a lot.

About the long user file name. The user file name is not important, neither in Docman, nor in File Storage. We do migrate it, and use it in File Storage, but more from a perspective of having a way to identify files if, for some reason, other pieces of information get lost. You could just have truncated it, but perhaps you realized those documents was not that important? You can always bring them in manually if they are not too many…

I'm sure others here would appreciate the PL/SQL script 🙂 

Does anyone that was involved in this happen to have the script still for the PL/SQL? 

Hi ​@SHAUN_KERSLAKE  No, at least I haven’t received that PL/SQL code.

Hi @AmilaF, all,

We went through a similar exercise, so I'm sharing our experience. Besides the Cloud File Storage Migration Tool or a fully custom script, there's another option: do the migration in 3 steps, as we did.

1. Clean up the files on FTP first. Agree with the business which documents you really need, and remove duplicates, obsolete revisions and orphaned files. Every GB you don't migrate saves time in all the following steps.
2. Migrate the documents from FTP to the DB repository. Once the files are in the database, they move together with the DB during the upgrade. You don't need to worry about FTP connectivity, firewalls or authentication on the IFS Cloud side, and FTP is no longer supported as a repository there anyway.
3. Migrate from the DB to Azure file share in IFS Cloud using the standard "Transfer Documents" functionality.

Our good practice is to keep the newest documents in the DB (created within the last half a year), because they are used most often. A scheduled job then keeps moving the rest to Azure file share, running in batches outside business hours. With millions of documents, it's impossible to move everything in one go.

One limitation: "Transfer Documents" can only filter by document class, which isn't enough for large volumes. That's why we proposed adding a period filter and a batch size. Pls. vote if you find it useful:
https://community.ifs.com/ideas/ifs-docman-enhancement-of-transfer-documents-functionality-52187

As @Mathias Dahl mentioned, file names on Azure file storage are base64 encoded. If you need to trace a document to its file on the share, the SQL we use is here:
https://community.ifs.com/ideas/ifs-doc-mgt-azure-fss-base64-file-name-in-ifs-52182

Hope this helps.

BR Jan