This topic contains 2 replies, has 2 voices, and was last updated by  Jeff Drumm 2 years, 2 months ago.

Move Files Across Network

  • I was just wondering if anyone has every had to move files across a network. I cannot seem to get the translator to move a file across either a mapped network drive or a general UNC path (e.g \\192.168.11.11\c$) etc.
    Has anyone run across this and if so how did you solve it.

    Thanks,
    tony

    If you’re running Iguana on Windows, you’ll need to modify the service to run under a regular user account rather than the LocalSystem account. LocalSystem doesn’t have access to network shares. Once you make the change, stop/start the service.

    And of course the account under which you run Iguana will need access granted to the shares you wish to move files from/to.

    Jeff Drumm ◊ VP and COO ◊ HICG, LLC. ◊ http://www.hicgrp.com

    Since we’re talking about moving files to network shares here, I figured this would be a reasonable place to mention a “gotcha.” It’s not specific to Iguana, but Iguana’s high performance may have helped to create the issue.

    A recent project I worked on has Iguana picking up PDF files from a directory hierarchy on a secure FTP server (via CIFS rather than sftp, but that’s not particularly relevant to this issue), then deposit them on a network share via a UNC path. Once the file is successfully written, an XML meta-data companion file is created in the same path (first as a temporary file, then renamed to its target name with .xml extension). The existence of the XML file in the target path is the “trigger” to the receiving application that the PDF file is available for import. Particular care is taken to make sure the file handles are flushed and closed after each file is completely written.

    The target application, though, would complain regularly that it could not find the PDF file after reading the related XML file. This was a bit confounding because the logic in Iguana should guarantee that the PDF file is already in place before the XML file is even written to its temporary filename.

    Finding no logical reason for this behavior in the Iguana code, we reached out to the infrastructure team for their input, and learned that the target location is part of a DFS (Distributed File System) cluster. Files created on that filesystem aren’t being synced to distributed storage in the same order as they’re written, and this would regularly cause the receiving application to “see” the XML file before the PDF file would appear.

    Due to the fact that this application was already in production and the the changes needed to recode the receiving application to support some sort of “retry” functionality could take days or weeks for approvals, development and testing, we decided to introduce an arbitrary delay of 10 seconds between the depositing of the PDF file and the creation of the XML file. This should give DFS enough time to sync the file across member drives, and therefore have it already visible/accessible when the XML file is created.

    One might wonder why we didn’t have Iguana check for the existence of the PDF file in the target location before writing the XML file. I’ll leave the reason for not doing so as an exercise for the reader 😀

    As I mentioned at the beginning, this issue isn’t specifically an Iguana issue, but Iguana is a speedy creature and the problem we experienced was a result of things just happening too quickly.

    Jeff Drumm ◊ VP and COO ◊ HICG, LLC. ◊ http://www.hicgrp.com

You must be logged in to reply to this topic.