IFS Cloud 23R2 - Setting up SMB storage

Hi @chanaka-shanil , 
Thanks for the pointers. With your & Pasindu’s help, we found the root cause and managed to fix it.

Issue was: I setup the 23R2 middleware infrastructure using the “Advanced/Manual” mode. Apart from usual commands (eg: ‘KEY’, ‘KUBERENETES’ etc), I had to run the ‘STORAGE’ as well for this environment which I have missed.

ps> .\main.ps1 -resource 'STORAGE'

https://docs.ifs.com/techdocs/23r2/070_remote_deploy/010_installing_fresh_system/030_preparing_server/50_windows_managementserver/#install_ifs-storage_helm_chart

Once this is executed successfully, I can see a new namespace & 2 pods within it.

 
Then the ifs-storage pod started to work fine.
 

Thanks,
/Nishanth

Hi @Mathias Dahl,

thanks for sharing, then it’s little bit odd to use the screen, since the selection can be limited by Doc Class only. See the count of the documents behind on the screenshot (sorted in descending) 😉 

@Mathias Dahl 
of course for test I selected a doc class with approx. 250 documents behind and it went through within approx. 5 mins, however seeing the counts of the most heavy doc classes I don’t think we can run it for ¾ million of documents in 1 go, I guess we will have to split the transfer into some batches (probably by date) which is unf. not possible in the screen so most probably we will have to do it from backend. More over some of the documents have been archived in backup table in different DB schema, so not all the docs are really having file behind.

 Hi @Mathias Dahl,
we have discussed the new way of the file saving further internally and good news is that we could survive with 1 subfolder and with the new file naming. 
However we have identified further business scenarios where we access files directly or via a http link and unfortunately we will have to invest some effort to migrate these cases to restAPI.
Since only 1 subfolder is allowed, there is also a question how to separate documents from PROD and TEST/UAT instances.

Regarding the reference between IFS doc record and the file, it seems it’s not only me who is convinced it’s not a big deal to see the reference on IFS screen and/or Azure file (on IFS screen or on Azure file) and it would bring more confident and transparency, some screenshots below to inspire;-) 
Have a nice weekend!
Jan
 

Hi @chanaka-shanil,
Below is the last segment of describe which shows the error message. let me know if you need the full describe note

Events:
  Type     Reason              Age                  From                     Message
  ----     ------              ----                 ----                     -------
  Warning  FailedMount         5m7s (x61 over 17h)  kubelet                  Unable to attach or mount volumes: unmounted volumes=[fss-volume], unattached volumes=[kube-api-access-rctjv linkerd-identity-end-entity linkerd-identity-token labelinfo secrets fss-volume linkerd-proxy-init-xtables-lock]: timed out waiting for the condition
  Warning  FailedAttachVolume  17s (x271 over 17h)  attachdetach-controller  AttachVolume.Attach failed for volume "ifs-fss-pv-smb-aspiit-448281875b94fdf905e063d09a09d46d5303ff0273703ff1e6486d2fc2e984ed" : timed out waiting for external-attacher of smb.csi.k8s.io CSI driver to attach volume ifs-fss-pv-smb-aspiit-448281875b94fdf905e063d09a09d46d5303ff0273703ff1e6486d2fc2e984ed


/Nishanth

Hi @chanaka-shanil ,
Thanks for the input. I confirmed the connection from the k8s server to fileshare which is working fine. I couldnt test exactly with the file-share pod though since the pod is not started yet. I tried via another pod (via shell login), the deployment identifies the host/IP successfully. Any other way we could verify this?

I will do a reconfig using the IP instead of host and get back to you

Thanks,
/Nishanth

Hi @Mathias Dahl 
 

The safest way is to use two separate shares for different environments. You don't want to risk overwriting PROD, at least, with files people are playing with in TEST/DEV…

Yes, indeed. However, without a possibility to open existing documents from PROD in TEST/UAT since only 1 file share is allowed for an instance or? 

I think you missed the word "not" in there somewhere, but I think I get the message anyway... 🙂

Could be;-)

This transparency you talk about, cannot be something a regular user need to care about, right? It's "an admin thing". And an admin can, once, to feel safe, use a tool or site to decode the Base64 encoding, to be convinced the file is actually where it should be. Once people understand how this works, do you think it's a big problem that you cannot look at the file names on the screen, in IFS and the Azure/SMB thing, and visually match them?

Let me illustrate this on a case I faced on Friday. I was searching for a document in IFS and I wanted to get a preview of the file from there. When I pushed button “preview” the document was downloaded but with 0 size. Then I started to explore further for a reason, the first thought was to check, whether the file is on share, but looking at folder with the files I can just guess based on date/time creation. Do the one really need to go to PL/SQL and write the script to check to be 100% sure?

@Mathias Dahl ,
 

I understand how it can be convenient to be logged in to TEST and be able to see documents from PROD, but it feels dangerous trying to do that by "reusing" the share. How about just copying a small sub set from PROD to TEST to be used for testing and verification purposes, if they are needed at all? (I would just create new ones, it takes a minute to upload a bunch of files to test with)

I have an attempt to answer, which subset, since I don’t know what documents the files are connected too;-). Well, to keep additional share and copy means additional costs too. We will need to test and think further, thanks for confirmation of the limitations.

Thanks for the scenario! One way to do it is to take the file name as seen in Docman (say, TE_DOC-4657860-1-1-1.JPG) and running them through an online Base64 encoder (like https://www.base64encode.org/)

Encoding the file name (minus period and file extension) above will get you this:

VEVfRE9DLTQ2NTc4NjAtMS0xLTE=

Then look for that file on the share.

No PL/SQL scripts needed 🙂

I see, but still in case of particular files one need to go to a page do a copy/paste and check, however I still don’t understand why this information can not be available on IFS screen directly, is there any technical challenge I can not see? From my perspective it’s an easy cherry a lot of other customers will appreciate and on top of that you will save time on similar discussions with other stakeholders once this is used broadly;-)
Thanks

Hi @Mathias Dahl / @chanaka-shanil,
We are actually facing the same issue, file storage pod is remaining in the init status,

Any idea what might be the issue?

I’m also little bit confused with highlighted statement in technical doc below, can you pls. explain?

 

Hi @Mathias Dahl  then it’s ok, however I expect Azure file share will work too.

Hi @chanaka-shanil ,
I added the IP instead of the host and the result is unfortunately same. I described the pod & the error note is also similar

  Type     Reason              Age                 From                     Message
  ----     ------              ----                ----                     -------
  Warning  FailedMount         95s (x3 over 10m)   kubelet                  Unable to attach or mount volumes: unmounted volumes=[fss-volume], unattached volumes=[labelinfo secrets fss-volume linkerd-proxy-init-xtables-lock kube-api-access-f6mkl linkerd-identity-end-entity linkerd-identity-token]: timed out waiting for the condition
  Warning  FailedAttachVolume  56s (x12 over 31m)  attachdetach-controller  AttachVolume.Attach failed for volume "ifs-fss-pv-smb-aspiit-4249944e4445c893f4a70ac42f5f82effb23163d969aa7bdeafe6bc0ad858420" : timed out waiting for external-attacher of smb.csi.k8s.io CSI driver to attach volume ifs-fss-pv-smb-aspiit-4249944e4445c893f4a70ac42f5f82effb23163d969aa7bdeafe6bc0ad858420

Hi @Mathias Dahl 
yes, we have made Azure share working and we also did first testing, with few questions around:

1. it seems all the documents are stored into 1 subdirectory called “docman” on the Azure share, and we were not able to force it to store them into another subdirectory (subdirectory IFSOCTEST on the screenshot below was created manually), despite the repository setting below. So it’s not possible to store particular doc classes into their subdirectories as we do today with FTP?

2. I cannot see any connection between document in IFS and file on share, since I did not find any properties/etag or metada stored with the file on the share. 
 

@ORFJAN 

The built-in assistant to move document files between repositories is not very performant, partly because it goes via IFS Connect, which in turn calls the projection that does the heavy lifting.

Make sure to try it out with hundreds or so documents to see how fast it is for you (I think IFS Connect can be tweaked in various ways but I don't know much about it)

For documents stored in database, the IFS Cloud File Storage migration tool might be an option. It's really meant for Apps 8/9/10 -> IFS Cloud migrations, but it does actually work between IFS Cloud versions as well...

You might want to develop something "custom" as well, that takes shortcuts that works in your scenario/setup. Moving the files is the thing that takes time, the rest is "changing the file pointers".