Best Practices for Data Migration to IFS Cloud (24.1.3)

Hello Markus,

I’m working on solutioning Legacy to Cloud migration. The IFS recommended tool today for such a task is called Data Migration Manager (used to be called “Smart Data Manager” before). If you are going to be working on a Legacy to IFS Cloud migration, I would recommend you to start with the IFS Academy training on Smart Data Manager. 

The training is 8 hours and 15 minutes, it costs about 600 bucks but is worth the investment if you will be doing that kind for job.

There is also the Data Migration Manager documentation here: https://docs.ifs.com/techdocs/24r1/030_administration/050_data_management/080_data_migration_manager/

The documentation is a bit difficult to follow if you don’t have “on-hands” experience. I think the documentation is good but the purpose is to be more of a reference material rather than a learning tool. Once you can get your hands on the Data Migration Manager (I think that is a module you need access to and your company must have purchased to component in order for it to be visible on your navigator) the best way to learn is to try to do things there. Create a small file with one or two rows (like one or two master parts) and try to understand how data flow (from your legacy files, to your input containers, to the output containers), how to do transformations, how to configure your sources and all the work that is required to master that tool.

Talk to the sales rep serving the company you work for, they should be able to do a quick presentation on the tool where you can see what the tool does in a nutshell.

The Data Migration Jobs (who also exist under the Data Management) are a different kind of animal, do not confuse the Data Migration Group with the Data Migration Manager tool (IFS has not done a great job of separating these things, they even exist under the same menu).

Where as the Data Migration Jobs serve the purpose of migrating/modifying a particular Logical Unit or a specific table, the Data Migration Manager is an end-to-end migration tool to move ALL your data using complex rules, validations, and transformations.

I know this is not all that can be said, a full book can be written about that stuff, but hopefully these ideas help you to get you on track to continue with your own research.

Hi Markus,

as this is a huge task, it’s not that easy to answer this in short.

However - here are some tips:

• I’d rather use the FNDMIG migration tool, as you already stated.
• The IFS Docs are pretty good: https://docs.ifs.com/techdocs/24r1/030_administration/050_data_management/050_data_migration/#contents
• Use CREATE_TABE_FROM_FILE (Procedure Name) for importing legacy data from a flat file format like .csv
• Use MIGRATE_FROM_SOURCE_DATE (Procedure Name) to migrate this data from the then existing table
• Keep an eye on the correct char set and data formats like Number (decimals) and Date (Format Mask)
• If there are two possible options for the same coumn (<COLUMN> and <COLUMN>_DB) in source mapping, always use _DB
• Try to keep your migrations jobs clean and cleary arranged.
• Use user IFSAPP for migration tasks
• Test small data sets first and as well update and inserts.
• …

Hope this helps for a start.

BR

Florian 

/FLEXiCODE GmbH

Hi Markus 

I don’t mean to get into one of this an endless philosophical discussion as to whether the FNDMIG or Data Migration Manager would deliver in this case as likely with some elbow grease you can make both of them do what you need to accomplish.

What I would like to understand is why you would recommend FNDMIG since in my recent experience, IFS seems to be moving their R&D into the Data Migration Manager. The DMM can do all FNDMIG does and much more, and if you look at the lasts Release Updates for IFS, it is clearly where they are going as far as Data Migration tooling. I’m not saying FNDMIG will be unsupported in the near future, but I think this is basically an APPS10 feature that they moved to the cloud “as-is” and will not see any investment in enhancements or improvements in the future at all.

I’m curious if your recommendation is based on your experience with FNDMIG and that you are comfortable with the tool because you have used it so much (we all prefer what we already know) and maybe you have not tried the Data Migration Manager yet. 

I have used both tools and so far found the FNDMIG not to be a great tool for an end-to-end ERP to IFS Cloud migrations (for anything but very small implementations). What I mean to say is if, let’s say, your master part migration will consist of more than 10,000 rows (I know this an arbitrary threshold, consider it a “ballkpark”) if you are migrating more than 50 containers (again, just a ballpark) handling all that orchestration with FNDMIG becomes slow and cumbersome. The tool doesn’t work well when you are working with LOTS of data.

Don’t get me wrong: we have used FNDMIG (and we still use it today) for things such as mass modifications using the Modify__ method (some property that needs to change for thousands of entries and doing it manually would be extremely time consuming) for example, recently we implemented an FNDMIG solution for mass update of Safety Lead Time based on some pre-defined criteria affecting thousands of parts).

As a side note, you said “Use IFSAPP for migration tasks”. I’m also curious whether you are in the Cloud as my experience in IFS Cloud is that IFS will not provide IFSAPP credentials to customers for Use Place environments (only IFSADMIN).

Like I say, I don’t mean to start a long discussion or question other answers validity, just learn from other community member’s experiences :).

Thanks!

@msurasky-price Thanks for your reply. I’ll come back to this, as this is an interesting topic and, as you’ve mentioned, IFS is investing only in the Data Migration Manager.

Hello, 
thank you very much for your answers with tips and hints.

I will ask our consulting team about the Data Migration Manager.

It is now also clear to me that the procedure via LOAD and MIG jobs is perhaps not the most convenient but a common way of migration.

However, I also think that this topic is interesting and important for several people. 

Perhaps the original “question” can be changed to a “conversation” and experienced and newcomers can talk to each other.
 

Thank you all

Greetings

Markus

@Markus S. If I was you, going the data migration route using migration job is the initial way to go. 

The moment you understand how these migration jobs work you will have way more feeling about the datamigration manager works. 

To understand it you need to have a very solid understanding of IFS, how the Logical units work, how they are interlinked etc. 

So I would suggest to get anyway a good understanding of the datamigration jobs and then decide if its worth it to invest in using the Data migration manager. High level I was very excited as well but in the end you want to be flexible and then the migration jobs are where you can do everything. 

Hello, 
Thank you very much for your message.
I will now continue to work on LOAD and MIG jobs via data migration. 

I have already been able to celebrate some successes by practicing. 
It's starting to be fun. And my colleagues are also happy when more and more data is imported into the test database.
:)

Hi ​@msurasky-price,

sorry for my delayed answer. End of year is the high season for migration tasks ;-).

Brief answers to your comments: 

1. “IFS seems to be moving their R&D into the Data Migration Manager”

Yes. From my understanding, this is definitely the way-to-go and product strategy of IFS. And this is a factor we should consider.

2. What I would like to understand is why you would recommend FNDMIG (nevertheless)

As ​@kvbe noted, an understanding of the data migration techniques and mechanism in IFS as well as the data structure is the foundation for further progressing. As soon as you have overcome the first hurdles and have at least a bit of knowledge of SQL the FNDMIG ist just more simple and flexible, yet powerful. If users don’t have any knowledge in SQL and don’t have the aim to dig a little bit deeper, the Data Migration Manager is the tool to use.

100. “The tool doesn’t work well when you are working with LOTS of data.”

I agree that the administrative effort outside IFS increases with extensive data migration, due to the simplicity of FNDMIG. However, the decisive factor is not the number of data records, but rather the number of source systems IMHO. In my experience, the processing speed of FNDMIG has increased significantly in IFS Cloud.

500. “As a side note, you said “Use IFSAPP for migration tasks”.”

Tbh: Most of our new customers use a private / managed cloud outside the IFS ecosystem to run ther IFS Cloud instances. So we’re having the IFSAPP credentials. The point here is: Use a administration user / power user rather than a normal end user. Some customers also create a separate FNDUSER for migration purposes. Then you have to keep in mind, that missing permissions can lead to data issues.

To sum it up:

I have to admit that I have much more experience with the FNDMIG and it probably plays a role. My experience with the DMM is that it is much more rigid and complex to use, a lot has to be configured in advance and the dependencies are much more interlinked. This leads to a comparatively high time expenditure for operating the tool, which I am spared when using the FNDMIG. Nevertheless, I have to say that we are also continuing to monitor the Data Migration Manager and its development and that it certainly depends on the parameters in the course of the projects / data migrations. If it is a complex initiative in which, for example, a lot of key information such as numbering systems need to be converted, data transformed or areas outside the typical master data need to be migrated, you won't get very far with the Data Migration Manager. For simpler migrations, in which the legacy data or source data is already well prepared in advance, I can imagine an advantage of the DMM. It would be interesting to run a project in parallel with both tools and see how it actually goes.
With the DMM, IFS is creating another option that will certainly become more interesting in the future. At the same time, layering with SVC and REST-APIs also offer further possibilities. It will be interesting to see where the journey takes us. I will probably work with both tools in the future and decide depending on the area of application.

BR

Florian

FLEXiCODE

Hi ​@FlorianTauber 

This is my time to say “sorry for the long delay”. I guess we are all busy delivering on commitments and is hard to find the time to monitor these conversations and keep them going.

I agree with what you have said. Prior experience is key when deciding on Tooling/Implementation. Sometimes you may decide to implement with a tool/technology for the simple reason you know how to do it and know it delivers. I would say after a couple of months working with the DMM, some of your concerns are real. There is a lot of overengineering on baked in the tool that sometimes makes a simple migration a multi-step process where the tool gets in the way.

Having said that, I think with time and experience the DMM delivers as well. And since IFS is putting his R&D efforts there, I think the logical thing to do (especially in the long run) is to learn it and use it (on top of FNDMIG).

Thanks for your insights, we are hosted in the cloud so IFSAPP is a no-no for us, which I really miss (from the APPS10 days). I can see why they decided to tighten things up but if I start mentioning the amount of times I had to deal with issues stemming from the fact the IFSAPP account is no longer available I could easily write a 10 page blog :).

Thanks!

Hello!

I come from the A&D domain, and I have a slightly different perspective on the data migration (DM) tools for IFS Cloud. There are two points in the conversation thus far on which I would like to comment.

1. “IFS seems to be moving their R&D into the Data Migration Manager”

This isn’t entirely true. Yes, I agree that the Data Migration Manager (DMM) takes the lion’s share of the R&D roadmap for data migration, but as IFS Maintenix is being developed into the new IFS Cloud for Aviation Maintenance (ICAM) solution, the FNDMIG tool is getting some love too.

2. “[FNDMIG] doesn’t work well when you are working with LOTS of data.”

I would argue that neither does the DMM. Furthermore, with 24R2, R&D is introducing a new migration job type/procedure for FNDMIG, called MIGRATE_HIGH_VOLUME_DATA, which is specifically intended to improve performance with a high volume of records. This is explained in the IFS Academy course, 24R2 Data Management – Data Migration : New Features (ID: 24R2CLOAINSDTADM).

In A&D implementations—due in part to the nature of the aerospace industry being highly regulated, where operators having very small margins for errors in their business critical data; and due in part to the typical volume of records that need to be migrated—tend to have migration processes that are developed and tested iteratively over many cycles. Operators also can’t afford long interruptions to their business processes, so the cutover window needs to be as short as possible, and they need lots of transactional data to be as current as possible to avoid a lengthy catch-up period. The DM scope gets built up incrementally, processes get refined until stringent data quality tolerances are satisfied, and the processes get practiced and optimized until they can be executed proficiently in the shortest time possible. Therefore, the DM processes need to be repeatable, automated, and highly performant.

The DMM has many helpful features for working with the data that’s being migrated—to validate it, flag duplicates, and prepare it for deployment—but (in my opinion) it is better for curating and maintaining a dataset that is mostly static, which may then be deployed to multiple environments as needed. A&D needs processes that work more like a pipeline, moving millions of records all the way from legacy sources/extracts through data mappings and transformations to loading into the target environment, and this favors FNDMIG. With the enhancements to FNDMIG starting in 24R2, the concerns about performance of the tool with “LOTS of data” will start to be addressed.