Question

Docman, handling files larger than 10 GB


Userlevel 6
Badge +14

Hi all,

We have files as large as 10 GB. When we try to create them in IFS,  we get this error after 25 min.

It happens on both database and ftp repositories.

Any ideas on how to handle this?


10 replies

Userlevel 7
Badge +21

Hans,

Have you tried to import these type of files using path: Document Management\File Import?

Steve

Userlevel 6
Badge +14

@eqbstal 

Unfortunate, same result.

Userlevel 7
Badge +21

No idea if this really is going to work. Are you able to save the file on the FTP and only create (manual) the link between the object and the URL towards the FTP location?

Userlevel 7
Badge +21

Hi,

Given that this error occurs after 25 or so minutes after the initiation it’s possible either a network timeout or the default statement execution timeout which is set to 30 minutes is kicking in and disconnecting the running process. Probably worth having a look at the MWS logs to see what is going on there.

 

 

cheers!

Userlevel 7
Badge +30

Like @Sajith D, I also suspect some timeout happening. In theory, as long as Oracle can handle it, or the FTP server can handle it, there is no limit on the size of files you can upload.

That being said, I am not sure if using Docman for 10 GB files is the best option. For example, when a user is “viewing” these files (what are they, CAD models?), the client will freeze for many minutes. Sure, for some processes this is acceptable, but not for others. If you like, please tell us more about this scenario.

If you would use video files, which when using more appropriate storage means can be streamed (the whole files is not needed to be downloaded before the user can start watching), that will not work in IFS (at least not in IEE, but perhaps in Aurena...).

 

Userlevel 6
Badge +14

@Mathias Dahl ,

These files are made with Adobe Illustrator (.ai) for digital printing. Basically a 2x3 meter, high resolution bitmap file.

I would like files to be handled by docman, because of the features like revision control, object connection etc.

 

Userlevel 2
Badge +6

Hi Hans,

 

We have an external process that imports documents into docman. We are using a file share. The same principal should apply to FTP though. We use the API to create a new document and a new file reference (EDM_FILE). The EDM_FILE record points to the file, it contains the location and the filename. We then move the file to this location outside IFS. To test this, you can import a small document into docman. Use the EDM_FILE record to locate it on the disk. Then replace that file with the large 10GB file. Personally, I would not use docman for illustrator files. If you generated a PDF what size would this be? I would use the PDF in document control as it is possible linked to parts or customer orders or something. There are probably better tools out there for illustrator revision control. This is a matter of personal choice though.

 

Userlevel 6
Badge +14

Hi all,

Thank you for the suggestions. Unfortunate neither seems to be a quick fix to a scenario not that common.

I thing we have to do 2 things:

  1. When the user tries to checkin a file larger than x, block the action with a warning on what to do instead. In IEE, something like this. (With Aurena Agent, it is not working directly).
  2. Do the checkin manually as you describe above. If that turns out to be too troublesome on daily basic, do some programming.   
Badge +2

Hi, we are getting a bit of a different error (“The Operation FinishCheckIn is not allowed for document XXXXX in state Checked In") and have been having this issue with checking in large documents into IFS (~2GB) for years. We had this issue in IFS 7.5 and recently upgraded to Apps10 and the issue remains. Ours fails after a few minutes. We have tried updating the middleware timeout settings but that did not work. We also rely on an FTP server for these documents and have done testing to confirm that our documents properly make it to the FTP but they do not make it into IFS in terms of the check-in finishing.

Userlevel 6
Badge +14

Hi @SKRSGREEN 

We have no problems with 2GB files. 

 

 

Reply