Check File Lock

Hi Guys,

I have a feature request (if it doesn’t already exist somewhere). Basically, I know os.fs.access(k) checks the permissions on a file, but as far as I can tell, it can’t check if a file has a lock on it which would mean that it can be accessed but not renamed/deleted/moved etc. Currently, I attempt to rename the file to itself ( if os.rename(k,k) == true then … end ), but it would be nice if there was a way to check this using the os.fs API. Even if it were included in os.fs.stat() it would be great!

Thanks,
Alan.

I can certainly see the value of this. There are a couple of file lock implementations mentioned in the Lua-users wiki. The one that seems the most complete is ExtensionProposal.

One other reason I like ExtensionProposal is that it includes os.dir(), an iterator for directories that doesn’t rely on OS-dependent globbing. This would allow you to write translator code that is independent of OS file pattern matching.

Jeff Drumm ◊ VP and COO ◊ HICG, LLC. ◊ http://www.hicgrp.com

There is value in this. But rather small value in my opinion.

If there was an API to check if a file is locked to make the code robust it would still be necessary to check os.rename succeeds or not. Why? Because there is a race condition that means there is a finite chance of another process locking the file in between testing for it and the rename operation. What would be really irritating about it is that would be a race condition that would be highly unlikely to show up in testing and would be really hard to reproduce 🙁 It might show up once in 100,000k transactions and stop one transaction for someone’s critical medical procedure.

So either way you need a lua wrapper to check for the error, so not much of a win in my opinion.

Ditto for the leaky abstraction on the file APIs. It’s a topic I explored a little myself with a Lua ‘file’ module:

http://help.interfaceware.com/kb/the-anatomy-of-an-iguana-app/5

The good news is that if you disagree with me, because we do document how to compile C extensions to bind to Iguana to incorporate C extension DLLs there is nothing to stop some collaboration and working together to port over the ExtensionProposal. In fact if you do we’re happy to give some help on our side to help you do that.

http://help.interfaceware.com/kb/1468

Or maybe just collaborate to flesh out the file module to make it more complete. Chances are though that you both have other more pressing things to concentrate on that are more impactful for your companies. If I am wrong though we’re happy to front up some cycles on our end to help you. I’d suggest putting the code into a GIT repo and we can help with that process.

For the most part though the conclusions I have have come to is that everyone’s very busy and so on our end we need to be thinking more carefully and focusing on how we can generate the most value for our user community.

So Iguana 6.0 is really going to be about focusing full out on all things to do with configuration management and source control. I think if we focus on that exclusively and put together all the ideas we have in this area then we have produce a level of change that will have real impact – something that should change the day to day work (in a positive way!) of everyone who works with Iguana then it is something that should help all our customers be more efficient, do more and help the businesses we all work in be more successful.

To do it means we’re probably going to need to flexible on release dates – the overall feedback we’re getting is that because we now have so many OEMs with such large installed numbers of Iguana instances that it’s costly expensive process to upgrade those instances – so it really needs to be worth it in terms of added value with each major release – rather than having many point releases which are hard to keep up with.

So after Iguana 6 will be focusing on another big area where we can make a big impact by fully replacing and rethinking what we have etc.

I don’t think Alan really wants a function to check the file for a lock before opening it; he fundamentally wants a way to ensure that the file is his exclusively to modify once it’s opened.

So (assuming lock functionality becomes available), open the file, check for success, lock the file, check for success, do the file manipulation, then close the file (which would clear the lock).

A little different in process, but gets the job done without a race condition. Of course, the other application accessing the file needs to gracefully handle locks as well.

Jeff Drumm ◊ VP and COO ◊ HICG, LLC. ◊ http://www.hicgrp.com

Thanks for the responses guys!

I agree with both of you to be honest; it would be a handy feature to have, but for my purposes and can do a workaround by just doing a check with os.rename() before proceeding with the logic for parsing and moving the file. The problem is that if a user has a lock on a file I am trying to process, I cannot move it when done to an archive folder. This means that the file is processed multiple times while it is open and is raising some issues. My workaround is working fine though so I understand it’s a low priority one, but thought I’d give a shout out in case you saw some value in it.

Kind regards,
Alan.

Basically need to have some periodic clean up that can pick up those locked files.

Of course if you put it on to Linux or Unix server you have less headaches of this nature… 🙂 One of the things that these operating systems got right is that you is you can rename files which are in use. It seems counter intuitive when one is used to Windows but it’s really handy for making installers that just work etc. Not an option I know but worth knowing that the universe of other operating systems isn’t so impacted by this issue.

One suggestion on the clean up process – have your script write out some file with a known extension like “.processed” that your script can check for and try to clean up the processed files – the existence of the file will make it clear that the file has been processed already and just needs to be cleaned up.

Occurs to me even with the ability to lock files and do operations to them it’s not a big win. If a script couldn’t get a lock on a file it would either have to wait until it make a lock – i.e. go into a slow blocking operation or loop round and come back to it later – which much the same algorithm as what one can do with using pcall around os.rename.

Anyways just thought I would add this in case anyone is reading this thread later thinking about the same type of problem.