Use file locks to synchronize access to ModuleCache.

[ovyalov]

Summary

Concurrent remote test runs may attempt to read and write from module cache simultaneously.
This CL intends to fix such races by using file-based locks - module id is used as a lock file name providing synchronization on per-module basis.

[zturner]

Would a mutex not work? At least on Windows we can have mutexes that can synchronize among different processes. LockFile will lock a file for exclusive access by a process, but if another process fails to acquire the lock, we won't be able to distinguish the failure from an a different kind of error, like an I/O error.

Also I've never really liked the concept of lock files to begin with, because it requires a cleanup step which won't happen if the program crashes. If there's a way to make this work with a true synchronization primitive I think that would be better. On Windows you do this by just calling CreateMutex and passing a non-null value for the name. Not sure on other platforms.

[ovyalov]

In D9056#157199, @zturner wrote:

Would a mutex not work? At least on Windows we can have mutexes that can synchronize among different processes. LockFile will lock a file for exclusive access by a process, but if another process fails to acquire the lock, we won't be able to distinguish the failure from an a different kind of error, like an I/O error.

Also I've never really liked the concept of lock files to begin with, because it requires a cleanup step which won't happen if the program crashes. If there's a way to make this work with a true synchronization primitive I think that would be better. On Windows you do this by just calling CreateMutex and passing a non-null value for the name. Not sure on other platforms.

Thanks for suggestion - mutex sounds like a good alternative. On the other hand, AFAIK file locks are capable to handle program crash properly and to revoke ownership from terminated process.
There are POSIX named semaphores which might be used as system-wide mutexes but I don't know for sure how widely they are supported on all platforms - file-based locks seems to be pretty widely adopted.

If you feel strongly against using file locks on Windows I can rewrite CL with mutexes - otherwise, we can do it iteratively:

1. Make initial version of this CL based on file locks to enable concurrent test runs.
2. I will migrate solution to use mutex soon (within a month) .

[zturner]

If it's able to handle process crashing without leaving lock files around, maybe it's fine as is. I've added some additional suggestions. I think there's still one more issue that stands out, which is that it's probably better to use the existing file hadnle that is already contained in the Module rather than calling CreateFile(..OPEN_EXISTING..) but I think that will be difficult to fix, as it's not easy to access the HANDLE being used internally by the Module. I don't think it will be broken as is, so I'm mostly just adding this as a note. Eventually one of us on the Windows side can fix this when it becomes an issue (although you're more than welcome to try if you want). I think it's the DataBuffer in the ObjectFile that has a file handle open already, because it's mmap'ed part of the object file. So the best fix would be to somehow get that HANDLE that was used to create the memory map, and use that for the call to LockFileEx.

source/Host/windows/LockFileWindows.cpp

27–29 ↗

(On Diff #23869)

You can get by without creating an event handle at all if you use GetOverlappedResult() instead of WaitForSingleObject. I slightly prefer that to using the event handle, especially if this is going to be happening very frequently. If you decide to do that, just leave the hEvent initialized to 0 and change the WaitForSingleObject to GetOverlappedResult.

53 ↗

(On Diff #23869)

Remove this call to _close, as as m_file should refer to the original module file, not a separate "lock file" (see later where I elaborate on this).

91–93 ↗

(On Diff #23869)

Same as before, we can remove the event handle and just use GetOverlappedResult

source/Utility/ModuleCache.cpp

150–153

Per the previous comments, can move all of this code into LockFilePosix.

source/Utility/ModuleCache.h

75–76 ↗

(On Diff #23869)

So the way locking a file works on windows, is that if you want to lock "foo.exe", you get a handle to foo.exe and you lock that. You don't make ".lock/<lock file name>" and then lock that, which is what it looks like you're doing. Because if you do that, you've got a totally new file, and then the ranges that you're passing to LockFileEx and UnlockFileEx don't make sense in the context of this new file.

Does this work on Linux? How does it know that even though your fd refers to .lock/<lock file name>, the range you're specifying refers to the original file?

In any case, maybe the best thing to do is have all this logic about building up the lock file name be internal to the LockFilePosix / LockFileWindows implementation. Somehow you will have to pass a HANDLE to the actual module file. If there's already a HANDLE deep down inside of the ModuleSP somewhere, best to use that. Otherwise you can probably just call CreateFile(... OPEN_EXISTING ...) on the module spec. But all of this can be internal to the FileLock class. Just have a constructor that takes a FileSpec to the thing you want to lock. On the linux side do the subdirectory thing, and on the Windows side just call CreateFile() on that path directly.

[zturner]

source/Host/windows/LockFileWindows.cpp

53 ↗

(On Diff #23869)

Sorry, a little clarification on this. If you call CreateFile(...OPEN_EXISTING...) then change this to CloseHandle(). If you end up figuring out a way to get the existing handle that the Module is using, then don't call CloseHandle when you release the lock.

[clayborg]

This doesn't seem to take care of the case:

process 1: calls ModuleCache::GetAndPut() on line 134 and then gets a path to the file lock file and creates and locks it
process 1: Starts getting the file and writing it to disk by creating it and writes a half of the file's bytes to the cache location "/tmp/usr/lib/libfoo.so"
process 2: calls ModuleCache::GetAndPut() on line 134 and notices there is a "/tmp/usr/lib/libfoo.so" in the cache and it returns the incomplete copy of the file

Don't we need to take the lock before the first call to "auto error = Get (root_dir_spec" on line 134?

If this is an issue we could read to a temp file first ("/tmp/usr/lib/libfoo.so.<UUID>") and then move it back to "/tmp/usr/lib/libfoo.so" atomically after we are done writing the file contents, and then unlock the file lock. This would prevent people from getting an incomplete copy in process 2?

[ovyalov]

In D9056#157301, @clayborg wrote:

This doesn't seem to take care of the case:

process 1: calls ModuleCache::GetAndPut() on line 134 and then gets a path to the file lock file and creates and locks it
process 1: Starts getting the file and writing it to disk by creating it and writes a half of the file's bytes to the cache location "/tmp/usr/lib/libfoo.so"
process 2: calls ModuleCache::GetAndPut() on line 134 and notices there is a "/tmp/usr/lib/libfoo.so" in the cache and it returns the incomplete copy of the file

Don't we need to take the lock before the first call to "auto error = Get (root_dir_spec" on line 134?

If this is an issue we could read to a temp file first ("/tmp/usr/lib/libfoo.so.<UUID>") and then move it back to "/tmp/usr/lib/libfoo.so" atomically after we are done writing the file contents, and then unlock the file lock. This would prevent people from getting an incomplete copy in process 2?

Thanks for catching this - I missed some race conditions:(
Since there is additional race maybe introduced by lock file removal (when one process is about to delete lock file and another process is going to open it) I decided to follow Zach advise and use mutexes.

[ovyalov]

1. Converted solution to use named mutexes instead of file locks.
2. Fixed race condition in GetAndPut by moving lock up to the beginning of the method.

[zturner]

include/lldb/Host/Mutex.h

36

I think we should change from returning int to either returning or taking by reference an lldb_private::Error. The same applies for the rest of the functions.

The same applies for all the rest of the functions. I know you are just copying the old API format, but maybe if it's not too much additional churn this would be a good time to convert this API to use the Error class.

50

If we change from int to lldb_private::Error, then this failure_message can go away. It's kind of a bad idea to have default parameters on virtual methods anwyay, because depending on who derives from it, you might get different default values depending on what type of pointer you call through, which is confusing.

260–261

Just curious, what is the purpose of this Global prefix? Windows doesn't require it, does posix require this "/" at the beginning? It would be nice if the names were at least somewhat consistent. Also it would be good if the mutexes could be "namespaced" so to speak, so that if you run some shell command or tool to display all the mutexes on the system, we can easily identify an LLDB mutex from some other mutex. With that in mind, what about a template like "lldb-$pid-$mutex_name

source/Host/windows/Mutex.cpp

118–121

HANDLE is already a typedef of void*, is this necessary?

137

Might as well use nullptr instead of NULL for consistency.

[ovyalov]

include/lldb/Host/Mutex.h

36

I support this idea of using Error instead of int but it seems too many things will be involved in change - ModuleList, Debugger, ConnectionFileDescriptor, GDBRemote* classes,...
Let's make it as a separate redesign CL.

260–261

Global means that mutex will be visible within all user's Terminal sessions - it's rather optional thing.

I agree that templated name convention makes sense but I'm not sure about $pid - mutex/semaphore ownership is transferred from process to process and with such naming scheme $pid will reflect the pid of creator which might be already dead.
With $pid as a part of mutex template you can use this mutex only to synchronize threads within the same process but not different processes.

source/Host/windows/Mutex.cpp

118–121

Removed

137

Done

[ovyalov]

• Added mutex name lldb prefix
• Minor fixes in windows::NamedMutex

[zturner]

Windows stuff looks good, only minor comments about the namespace thing.

include/lldb/Host/Mutex.h

260–261

Makes sense about the pid. I didn't know about this Global\ Local\ thing before, but I looked it up and seems you're right. If I understand this correctly, then suppose 2 users are connected via Remote Desktop to a Windows machine, and you run LLDB. If you put the mutex in the global namespace, then both users will see the same mutex. If you just use the default though, each will have their own mutex even though the names are the same.

It seems like the latter is what we want. You're trying to solve the race condition in the case of all these different processes running in the same terminal session, so I don't see a need for cross-session synchronization. If I have it correct, then it seems we should remove the Global\.

[zturner]

Ahh, actually maybe we actually do want global in this case. I was thinking that different users would be trying to synchronize among different files, but it would probably be the same file.

[ovyalov]

In D9056#157906, @zturner wrote:

Ahh, actually maybe we actually do want global in this case. I was thinking that different users would be trying to synchronize among different files, but it would probably be the same file.

I think we need to have Global prefix because if an user is logged in simultaneously via RDP and in interactive session he/she uses the same profile directory - so, in case of LLDB temp directory will be %HOMEPATH%\AppData\Temp\.... To make sure that both of them are synchronized on the same mutex to access temp directory Global prefix is required.

[ovyalov]

Let me put on hold this CL since there are some serious issues regarding using POSIX semaphores since they don't recover after process crash - will think more about it and will send a patch later.

[ovyalov]

Decided to stick to file locks since they support process crash recovery both in Posix and Windows implementations.
Main changes:

• Use %UUID%/.lock file for synchronization purposes.
• Download a module to %UUID%/.temp file so iut can be moved to a final file in the same directory without need to copy it.
• Obtain a file lock as a very first step in GetAndPut - just as a safety precaution if rename operation isn't atomic.

[clayborg]

Looks good.

[ovyalov]

AFFECTED FILES

/lldb/trunk/include/lldb/Host/LockFile.h
/lldb/trunk/include/lldb/Host/LockFileBase.h
/lldb/trunk/include/lldb/Host/posix/LockFilePosix.h
/lldb/trunk/include/lldb/Host/windows/LockFileWindows.h
/lldb/trunk/lldb.xcodeproj/project.pbxproj
/lldb/trunk/source/Host/CMakeLists.txt
/lldb/trunk/source/Host/common/LockFileBase.cpp
/lldb/trunk/source/Host/posix/LockFilePosix.cpp
/lldb/trunk/source/Host/windows/LockFileWindows.cpp
/lldb/trunk/source/Target/Platform.cpp
/lldb/trunk/source/Utility/ModuleCache.cpp
/lldb/trunk/source/Utility/ModuleCache.h

USERS

ovyalov (Author)

http://reviews.llvm.org/rL236736

include/lldb/Host/Mutex.h

 //===-- Mutex.h -------------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef liblldb_Mutex_h_
 #define liblldb_Mutex_h_
 #if defined(__cplusplus)
 
 #include "lldb/lldb-types.h"
 #include <assert.h>
 
-#ifdef LLDB_CONFIGURATION_DEBUG
 #include <string>
-#endif
 
 namespace lldb_private {
 
-//----------------------------------------------------------------------
-/// @class Mutex Mutex.h "lldb/Host/Mutex.h"
-/// @brief A C++ wrapper class for pthread mutexes.
-//----------------------------------------------------------------------
-class Mutex
+class MutexBase
 {
 public:
-    friend class Locker;
-    friend class Condition;
-    
-    enum Type
-    {
-        eMutexTypeNormal,       ///< Mutex that can't recursively entered by the same thread
-        eMutexTypeRecursive     ///< Mutex can be recursively entered by the same thread
-    };
+    virtual ~MutexBase() = default;
+    //------------------------------------------------------------------
+    /// Lock the mutex.
+    ///
+    /// Locks the mutex owned by this object. If the mutex is already
+    /// locked, the calling thread will block until the mutex becomes
+    /// available.
+    ///
+    /// @return
+    ///     The non-zero error code in case of error.
+    //------------------------------------------------------------------
+    virtual int
+    Lock() = 0;
 
     //------------------------------------------------------------------
-    /// @class Mutex::Locker
+    /// Try to lock the mutex.
+    ///
+    /// Attempts to lock the mutex owned by this object without blocking.
+    /// If the mutex is already locked, TryLock() will not block waiting
+    /// for the mutex, but will return an error condition.
+    ///
+    /// @return
+    ///     The non-zero error code in case of error.
+    //------------------------------------------------------------------
+    virtual int
+    TryLock(const char *failure_message = NULL) = 0;
+
+    //------------------------------------------------------------------
+    /// Unlock the mutex.
+    ///
+    /// If the current thread holds the lock on the owned mutex, then
+    /// Unlock() will unlock the mutex. Calling Unlock() on this object
+    /// when the calling thread does not hold the lock will result in
+    /// undefined behavior.
+    ///
+    /// @return
+    ///     The non-zero error code in case of error.
+    //------------------------------------------------------------------
+    virtual int
+    Unlock() = 0 ;
+
+    //------------------------------------------------------------------
+    /// @class MutexBase::Locker
     ///
     /// A scoped locking class that allows a variety of pthread mutex
-    /// objects to have a mutex locked when an Mutex::Locker
+    /// objects to have a mutex locked when an MutexBase::Locker
     /// object is created, and unlocked when it goes out of scope or
-    /// when the Mutex::Locker::Reset(pthread_mutex_t *)
+    /// when the MutexBase::Locker::Reset(pthread_mutex_t *)
     /// is called. This provides an exception safe way to lock a mutex
     /// in a scope.
     //------------------------------------------------------------------
     class Locker
     {
     public:
         //--------------------------------------------------------------
         /// Default constructor.
         ///
         /// This will create a scoped mutex locking object that doesn't
         /// have a mutex to lock. One will need to be provided using the
-        /// Mutex::Locker::Reset(pthread_mutex_t *) method.
+        /// MutexBase::Locker::Reset(pthread_mutex_t *) method.
         ///
-        /// @see Mutex::Locker::Reset(pthread_mutex_t *)
+        /// @see MutexBase::Locker::Reset(pthread_mutex_t *)
         //--------------------------------------------------------------
         Locker();
 
         //--------------------------------------------------------------
-        /// Constructor with a Mutex object.
+        /// Constructor with a MutexBase object.
         ///
         /// This will create a scoped mutex locking object that extracts
         /// the mutex owned by \a m and locks it.
         ///
         /// @param[in] m
-        ///     An instance of a Mutex object that contains a
+        ///     An instance of a MutexBase object that contains a
         ///     valid mutex object.
         //--------------------------------------------------------------
-        Locker(Mutex& m);
+        Locker(MutexBase& m);
 
         //--------------------------------------------------------------
-        /// Constructor with a Mutex object pointer.
+        /// Constructor with a MutexBase object pointer.
         ///
         /// This will create a scoped mutex locking object that extracts
         /// the mutex owned by a m and locks it.
         ///
         /// @param[in] m
-        ///     A pointer to instance of a Mutex object that
+        ///     A pointer to instance of a MutexBase object that
         ///     contains a valid mutex object.
         //--------------------------------------------------------------
-        Locker(Mutex* m);
+        Locker(MutexBase* m);
 
         //--------------------------------------------------------------
-        /// Desstructor
+        /// Destructor
         ///
         /// Unlocks any valid pthread_mutex_t that this object may
         /// contain.
         //--------------------------------------------------------------
         ~Locker();
 
         //--------------------------------------------------------------
         /// Change the contained mutex.
         ///
         /// Unlock the current mutex in this object (if it contains a
         /// valid mutex) and lock the new \a mutex object if it is
         /// non-NULL.
         //--------------------------------------------------------------
         void
-        Lock (Mutex &mutex);
+        Lock (MutexBase &mutex);
         
         void
-        Lock (Mutex *mutex)
+        Lock (MutexBase *mutex)
         {
             if (mutex)
                 Lock(*mutex);
         }
 
         //--------------------------------------------------------------
         /// Change the contained mutex only if the mutex can be locked.
         ///
         /// Unlock the current mutex in this object (if it contains a
         /// valid mutex) and try to lock \a mutex. If \a mutex can be 
         /// locked this object will take ownership of the lock and will
         /// unlock it when it goes out of scope or Reset or TryLock are
         /// called again. If the mutex is already locked, this object
         /// will not take ownership of the mutex.
         ///
         /// @return
         ///     Returns \b true if the lock was aquired and the this 
         ///     object will unlock the mutex when it goes out of scope,
         ///     returns \b false otherwise.
         //--------------------------------------------------------------
         bool
-        TryLock (Mutex &mutex, const char *failure_message = NULL);
+        TryLock (MutexBase &mutex, const char *failure_message = NULL);
         
         bool
-        TryLock (Mutex *mutex, const char *failure_message = NULL)
+        TryLock (MutexBase *mutex, const char *failure_message = NULL)
         {
             if (mutex)
                 return TryLock(*mutex, failure_message);
             else
                 return false;
         }
 
         void
         Unlock ();
 
     protected:
         //--------------------------------------------------------------
         /// Member variables
         //--------------------------------------------------------------
-        Mutex *m_mutex_ptr;
+        MutexBase *m_mutex_ptr;
 
     private:
         Locker(const Locker&);
         const Locker& operator=(const Locker&);
     };
+};
 
+//----------------------------------------------------------------------
+/// @class Mutex Mutex.h "lldb/Host/Mutex.h"
+/// @brief A C++ wrapper class for pthread mutexes.
+//----------------------------------------------------------------------
+class Mutex : public MutexBase
+{
+public:
+    friend class Condition;
+
+    enum Type
+    {
+        eMutexTypeNormal,       ///< Mutex that can't recursively entered by the same thread
+        eMutexTypeRecursive     ///< Mutex can be recursively entered by the same thread
+    };
 
     //------------------------------------------------------------------
     /// Default constructor.
     ///
     /// Creates a pthread mutex with no attributes.
     //------------------------------------------------------------------
     Mutex();
 
public:
     //------------------------------------------------------------------
     Mutex(Mutex::Type type);
 
     //------------------------------------------------------------------
     /// Destructor.
     ///
     /// Destroys the mutex owned by this object.
     //------------------------------------------------------------------
-#ifdef LLDB_CONFIGURATION_DEBUG
-    virtual
-#endif
     ~Mutex();
 
-    //------------------------------------------------------------------
-    /// Lock the mutex.
-    ///
-    /// Locks the mutex owned by this object. If the mutex is already
-    /// locked, the calling thread will block until the mutex becomes
-    /// available.
-    ///
-    /// @return
-    ///     The error code from \c pthread_mutex_lock().
-    //------------------------------------------------------------------
-#ifdef LLDB_CONFIGURATION_DEBUG
-    virtual
-#endif
     int
-    Lock();
+    Lock() override;
 
-    //------------------------------------------------------------------
-    /// Try to lock the mutex.
-    ///
-    /// Attempts to lock the mutex owned by this object without blocking.
-    /// If the mutex is already locked, TryLock() will not block waiting
-    /// for the mutex, but will return an error condition.
-    ///
-    /// @return
-    ///     The error code from \c pthread_mutex_trylock().
-    //------------------------------------------------------------------
-#ifdef LLDB_CONFIGURATION_DEBUG
-    virtual
-#endif
     int
-    TryLock(const char *failure_message = NULL);
+    TryLock(const char *failure_message = NULL) override;
 
-    //------------------------------------------------------------------
-    /// Unlock the mutex.
-    ///
-    /// If the current thread holds the lock on the owned mutex, then
-    /// Unlock() will unlock the mutex. Calling Unlock() on this object
-    /// when the calling thread does not hold the lock will result in
-    /// undefined behavior.
-    ///
-    /// @return
-    ///     The error code from \c pthread_mutex_unlock().
-    //------------------------------------------------------------------
-#ifdef LLDB_CONFIGURATION_DEBUG
-    virtual
-#endif
     int
-    Unlock();
+    Unlock() override;
 
 protected:
     //------------------------------------------------------------------
     // Member variables
     //------------------------------------------------------------------
     // TODO: Hide the mutex in the implementation file in case we ever need to port to an
     // architecture that doesn't have pthread mutexes.
     lldb::mutex_t m_mutex; ///< The OS mutex object.
 
 private:
     //------------------------------------------------------------------
     /// Mutex get accessor.
     ///
     /// @return
     ///     A pointer to the pthread mutex object owned by this object.
     //------------------------------------------------------------------
     lldb::mutex_t *
     GetMutex();
 
     Mutex(const Mutex&);
     const Mutex& operator=(const Mutex&);
 };
 
+class NamedMutex : public MutexBase
+{
+public:
+    // Full mutex name contains system-specific prefix and base_name:
+    // POSIX  - "/$base_name"
+    // Windows - "Global\$base_name"
+    explicit NamedMutex(const char* base_name);
+    ~NamedMutex();
+
+    int
+    Lock() override;
+
+    int
+    TryLock(const char *failure_message = NULL) override;
+
+    int
+    Unlock() override;
+
+    const std::string&
+    GetName() const
+    {
+        return m_name;
+    }
+
+    // Returns max length of mutex name, -1 for unlimited.
+    static int
+    GetMaxNameLength();
+
+protected:
+    //------------------------------------------------------------------
+    // Member variables
+    //------------------------------------------------------------------
+    const std::string m_name;
+    void* m_mutex; ///< The OS mutex object.
+
+private:
+    NamedMutex(const NamedMutex&);
+    const NamedMutex& operator=(const NamedMutex&);
+
+};
+
 #ifdef LLDB_CONFIGURATION_DEBUG
 class TrackingMutex : public Mutex
 {
 public:
     TrackingMutex() : Mutex()  {}
     TrackingMutex(Mutex::Type type) : Mutex (type) {}
     
     virtual
     ~TrackingMutex() {}
     
-    virtual int
-    Unlock ();
+    int
+    Unlock () override;
 
-    virtual int
-    TryLock (const char *failure_message = NULL)
+    int
+    TryLock (const char *failure_message = NULL) override
     {
         int return_value = Mutex::TryLock();
         if (return_value != 0 && failure_message != NULL)
         {
             m_failure_message.assign(failure_message);
             m_thread_that_tried = pthread_self();
         }
         return return_value;
     }
     
 protected:
     pthread_t m_thread_that_tried;
     std::string m_failure_message;
 };
 
 class LoggingMutex : public Mutex
 {
 public:
     LoggingMutex() : Mutex(),m_locked(false)  {}
     LoggingMutex(Mutex::Type type) : Mutex (type),m_locked(false) {}
     
     virtual
     ~LoggingMutex() {}
     
-    virtual int
-    Lock ();
+    int
+    Lock () override;
     
-    virtual int
-    Unlock ();
+    int
+    Unlock () override;
     
-    virtual int
-    TryLock (const char *failure_message = NULL);
+    int
+    TryLock (const char *failure_message = NULL) override;
 protected:
     bool m_locked;
 };
 #endif
 
 } // namespace lldb_private
 
 #endif  // #if defined(__cplusplus)
 #endif

lldb.xcodeproj/project.pbxproj

/* Begin PBXBuildFile section */
     256CBDBA1ADD107200BC6CDC /* RegisterContextLinux_arm.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 256CBDB61ADD107200BC6CDC /* RegisterContextLinux_arm.cpp */; };
     256CBDBB1ADD107200BC6CDC /* RegisterContextLinux_arm.h in Headers */ = {isa = PBXBuildFile; fileRef = 256CBDB71ADD107200BC6CDC /* RegisterContextLinux_arm.h */; };
     256CBDBC1ADD107200BC6CDC /* RegisterContextLinux_mips64.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 256CBDB81ADD107200BC6CDC /* RegisterContextLinux_mips64.cpp */; };
     256CBDBD1ADD107200BC6CDC /* RegisterContextLinux_mips64.h in Headers */ = {isa = PBXBuildFile; fileRef = 256CBDB91ADD107200BC6CDC /* RegisterContextLinux_mips64.h */; };
     256CBDC01ADD11C000BC6CDC /* RegisterContextPOSIX_arm.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 256CBDBE1ADD11C000BC6CDC /* RegisterContextPOSIX_arm.cpp */; };
     256CBDC11ADD11C000BC6CDC /* RegisterContextPOSIX_arm.h in Headers */ = {isa = PBXBuildFile; fileRef = 256CBDBF1ADD11C000BC6CDC /* RegisterContextPOSIX_arm.h */; };
     257E47171AA56C2000A62F81 /* ModuleCache.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 257E47151AA56C2000A62F81 /* ModuleCache.cpp */; };
     257E47181AA56C2000A62F81 /* ModuleCache.h in Headers */ = {isa = PBXBuildFile; fileRef = 257E47161AA56C2000A62F81 /* ModuleCache.h */; };
+    25B67C351AE148020000D19B /* Mutex.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 25B67C341AE148020000D19B /* Mutex.cpp */; };
     25EF23781AC09B3700908DF0 /* AdbClient.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 25EF23751AC09AD800908DF0 /* AdbClient.cpp */; };
     25EF23791AC09B4200908DF0 /* AdbClient.h in Headers */ = {isa = PBXBuildFile; fileRef = 25EF23761AC09AD800908DF0 /* AdbClient.h */; };
     260157C61885F51C00F875CF /* libpanel.dylib in Frameworks */ = {isa = PBXBuildFile; fileRef = 260157C41885F4FF00F875CF /* libpanel.dylib */; };
     260157C81885F53100F875CF /* libpanel.dylib in Frameworks */ = {isa = PBXBuildFile; fileRef = 260157C41885F4FF00F875CF /* libpanel.dylib */; };
     2606EDDF184E68A10034641B /* liblldb-core.a in Frameworks */ = {isa = PBXBuildFile; fileRef = 2689FFCA13353D7A00698AC0 /* liblldb-core.a */; };
     260A248E15D06C50009981B0 /* OptionValues.h in Headers */ = {isa = PBXBuildFile; fileRef = 260A248D15D06C4F009981B0 /* OptionValues.h */; };
     260A39A619647A3A004B4130 /* Pipe.h in Headers */ = {isa = PBXBuildFile; fileRef = 260A39A519647A3A004B4130 /* Pipe.h */; };
     260A63171861008E00FECF8E /* IOHandler.h in Headers */ = {isa = PBXBuildFile; fileRef = 260A63161861008E00FECF8E /* IOHandler.h */; };
/* Begin PBXFileReference section */
     256CBDB61ADD107200BC6CDC /* RegisterContextLinux_arm.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = RegisterContextLinux_arm.cpp; path = Utility/RegisterContextLinux_arm.cpp; sourceTree = "<group>"; };
     256CBDB71ADD107200BC6CDC /* RegisterContextLinux_arm.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = RegisterContextLinux_arm.h; path = Utility/RegisterContextLinux_arm.h; sourceTree = "<group>"; };
     256CBDB81ADD107200BC6CDC /* RegisterContextLinux_mips64.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = RegisterContextLinux_mips64.cpp; path = Utility/RegisterContextLinux_mips64.cpp; sourceTree = "<group>"; };
     256CBDB91ADD107200BC6CDC /* RegisterContextLinux_mips64.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = RegisterContextLinux_mips64.h; path = Utility/RegisterContextLinux_mips64.h; sourceTree = "<group>"; };
     256CBDBE1ADD11C000BC6CDC /* RegisterContextPOSIX_arm.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = RegisterContextPOSIX_arm.cpp; path = Utility/RegisterContextPOSIX_arm.cpp; sourceTree = "<group>"; };
     256CBDBF1ADD11C000BC6CDC /* RegisterContextPOSIX_arm.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = RegisterContextPOSIX_arm.h; path = Utility/RegisterContextPOSIX_arm.h; sourceTree = "<group>"; };
     257E47151AA56C2000A62F81 /* ModuleCache.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ModuleCache.cpp; path = source/Utility/ModuleCache.cpp; sourceTree = "<group>"; };
     257E47161AA56C2000A62F81 /* ModuleCache.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = ModuleCache.h; path = source/Utility/ModuleCache.h; sourceTree = "<group>"; };
+    25B67C341AE148020000D19B /* Mutex.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = Mutex.cpp; sourceTree = "<group>"; };
     25EF23751AC09AD800908DF0 /* AdbClient.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = AdbClient.cpp; sourceTree = "<group>"; };
     25EF23761AC09AD800908DF0 /* AdbClient.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = AdbClient.h; sourceTree = "<group>"; };
     260157C41885F4FF00F875CF /* libpanel.dylib */ = {isa = PBXFileReference; lastKnownFileType = "compiled.mach-o.dylib"; name = libpanel.dylib; path = /usr/lib/libpanel.dylib; sourceTree = "<absolute>"; };
     260223E7115F06D500A601A2 /* SBCommunication.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = SBCommunication.h; path = include/lldb/API/SBCommunication.h; sourceTree = "<group>"; };
     260223E8115F06E500A601A2 /* SBCommunication.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = SBCommunication.cpp; path = source/API/SBCommunication.cpp; sourceTree = "<group>"; };
     26022531115F27FA00A601A2 /* SBFileSpec.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = SBFileSpec.h; path = include/lldb/API/SBFileSpec.h; sourceTree = "<group>"; };
     26022532115F281400A601A2 /* SBFileSpec.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = SBFileSpec.cpp; path = source/API/SBFileSpec.cpp; sourceTree = "<group>"; };
     260A248D15D06C4F009981B0 /* OptionValues.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = OptionValues.h; path = include/lldb/Interpreter/OptionValues.h; sourceTree = "<group>"; };
3F8169261ABB73C1001DA9DF /* Initialization */ = {
         3F8169301ABB7A6D001DA9DF /* SystemLifetimeManager.cpp */,
       );
       name = Initialization;
       sourceTree = "<group>";
     };
     3FDFDDC4199D37BE009756A7 /* posix */ = {
       isa = PBXGroup;
       children = (
+        25B67C341AE148020000D19B /* Mutex.cpp */,
         AFDFDFD019E34D3400EAE509 /* ConnectionFileDescriptorPosix.cpp */,
         3FDFDDC5199D37ED009756A7 /* FileSystem.cpp */,
         3FDFE53019A292F0009756A7 /* HostInfoPosix.cpp */,
         3FDFE53219A29304009756A7 /* HostInfoPosix.h */,
         3FDFE56A19AF9C44009756A7 /* HostProcessPosix.cpp */,
         3FDFE56E19AF9C5A009756A7 /* HostProcessPosix.h */,
         3FDFE56B19AF9C44009756A7 /* HostThreadPosix.cpp */,
         3FDFE56F19AF9C5A009756A7 /* HostThreadPosix.h */,
2689FFC613353D7A00698AC0 /* Sources */ = {
         26D7E45D13D5E30A007FD12B /* SocketAddress.cpp in Sources */,
         94CD7D0C19A3FBCE00908B7C /* AppleObjCTypeEncodingParser.cpp in Sources */,
         94B6E76213D88365005F417F /* ValueObjectSyntheticFilter.cpp in Sources */,
         26F4A21C13FBA31A0064B613 /* ThreadMemory.cpp in Sources */,
         94EA27CE17DE91750070F505 /* LibCxxUnorderedMap.cpp in Sources */,
         266DFE9713FD656E00D0C574 /* OperatingSystem.cpp in Sources */,
         26954EBE1401EE8B00294D09 /* DynamicRegisterInfo.cpp in Sources */,
         26274FA714030F79006BA130 /* DynamicLoaderDarwinKernel.cpp in Sources */,
+        25B67C351AE148020000D19B /* Mutex.cpp in Sources */,
         94FA3DE01405D50400833217 /* ValueObjectConstResultChild.cpp in Sources */,
         3FDFED2819BA6D96009756A7 /* HostThread.cpp in Sources */,
         949ADF031406F648004833E1 /* ValueObjectConstResultImpl.cpp in Sources */,
         B27318421416AC12006039C8 /* WatchpointList.cpp in Sources */,
         26CFDCA3186163A4000E63E5 /* Editline.cpp in Sources */,
         26E152261419CAD4007967D0 /* ObjectFilePECOFF.cpp in Sources */,
         B2462247141AD37D00F3D409 /* OptionGroupWatchpoint.cpp in Sources */,
         49A71FE7141FFA5C00D59478 /* IRInterpreter.cpp in Sources */,

source/Host/CMakeLists.txt

add_host_subdirectory(windows
     windows/getopt/GetOptInc.cpp
     )
 else()
   add_host_subdirectory(posix
     posix/FileSystem.cpp
     posix/HostInfoPosix.cpp
     posix/HostProcessPosix.cpp
     posix/HostThreadPosix.cpp
+    posix/Mutex.cpp
     posix/PipePosix.cpp
     )
 
   if (NOT __ANDROID_NDK__)
     add_host_subdirectory(posix
       posix/ProcessLauncherPosix.cpp
       )
   endif()

source/Host/common/Mutex.cpp

 #include "lldb/Host/Host.h"
 
 #ifndef _WIN32
 #include <pthread.h>
 #endif
 #include <string.h>
 #include <stdio.h>
 
-#if 0
-// This logging is way too verbose to enable even for a log channel. 
-// This logging can be enabled by changing the "#if 0", but should be
-// reverted prior to checking in.
-#include <cstdio>
-#define DEBUG_LOG(fmt, ...) printf(fmt, ## __VA_ARGS__)
-#else
-#define DEBUG_LOG(fmt, ...)
-#endif
-
-// Enable extra mutex error checking
-#ifdef LLDB_CONFIGURATION_DEBUG
-#define ENABLE_MUTEX_ERROR_CHECKING 1
-#include <inttypes.h>
-#endif
-
-#if ENABLE_MUTEX_ERROR_CHECKING
-#include <set>
-
-enum MutexAction
-{
-    eMutexActionInitialized,
-    eMutexActionDestroyed,
-    eMutexActionAssertInitialized
-};
-
-static bool
-error_check_mutex (pthread_mutex_t *m, MutexAction action)
-{
-    typedef std::set<pthread_mutex_t *> mutex_set;
-    static pthread_mutex_t g_mutex_set_mutex = PTHREAD_MUTEX_INITIALIZER;
-    static mutex_set g_initialized_mutex_set;
-    static mutex_set g_destroyed_mutex_set;
-
-    bool success = true;
-    int err;
-    // Manually call lock so we don't to any of this error checking
-    err = ::pthread_mutex_lock (&g_mutex_set_mutex);
-    assert(err == 0);
-    switch (action)
-    {
-        case eMutexActionInitialized:
-            // Make sure this isn't already in our initialized mutex set...
-            assert (g_initialized_mutex_set.find(m) == g_initialized_mutex_set.end());
-            // Remove this from the destroyed set in case it was ever in there
-            g_destroyed_mutex_set.erase(m);
-            // Add the mutex to the initialized set
-            g_initialized_mutex_set.insert(m);
-            break;
-            
-        case eMutexActionDestroyed:
-            // Make sure this isn't already in our destroyed mutex set...
-            assert (g_destroyed_mutex_set.find(m) == g_destroyed_mutex_set.end());
-            // Remove this from the initialized so we can put it into the destroyed set
-            g_initialized_mutex_set.erase(m);
-            // Add the mutex to the destroyed set
-            g_destroyed_mutex_set.insert(m);
-            break;
-        case eMutexActionAssertInitialized:
-            // This function will return true if "m" is in the initialized mutex set
-            success = g_initialized_mutex_set.find(m) != g_initialized_mutex_set.end();
-            assert (success);
-            break;
-    }
-    // Manually call unlock so we don't to any of this error checking
-    err = ::pthread_mutex_unlock (&g_mutex_set_mutex);
-    assert(err == 0);
-    return success;
-}
-
-#endif
 
 using namespace lldb_private;
 
 //----------------------------------------------------------------------
 // Default constructor.
 //
 // This will create a scoped mutex locking object that doesn't have
 // a mutex to lock. One will need to be provided using the Reset()
 // method.
 //----------------------------------------------------------------------
-Mutex::Locker::Locker () :
+MutexBase::Locker::Locker () :
     m_mutex_ptr(NULL)
 {
 }
 
 //----------------------------------------------------------------------
 // Constructor with a Mutex object.
 //
 // This will create a scoped mutex locking object that extracts the
 // mutex owned by "m" and locks it.
 //----------------------------------------------------------------------
-Mutex::Locker::Locker (Mutex& m) :
+MutexBase::Locker::Locker (MutexBase& m) :
     m_mutex_ptr(NULL)
 {
     Lock (m);
 }
 
 //----------------------------------------------------------------------
-// Constructor with a Mutex object pointer.
+// Constructor with a MutexBase object pointer.
 //
 // This will create a scoped mutex locking object that extracts the
 // mutex owned by "m" and locks it.
 //----------------------------------------------------------------------
-Mutex::Locker::Locker (Mutex* m) :
+MutexBase::Locker::Locker (MutexBase* m) :
     m_mutex_ptr(NULL)
 {
     if (m)
         Lock (m);
 }
 
 //----------------------------------------------------------------------
 // Destructor
 //
 // Unlocks any owned mutex object (if it is valid).
 //----------------------------------------------------------------------
-Mutex::Locker::~Locker ()
+MutexBase::Locker::~Locker ()
 {
     Unlock();
 }
 
 //----------------------------------------------------------------------
 // Unlock the current mutex in this object (if this owns a valid
 // mutex) and lock the new "mutex" object if it is non-NULL.
 //----------------------------------------------------------------------
 void
-Mutex::Locker::Lock (Mutex &mutex)
+MutexBase::Locker::Lock (MutexBase &mutex)
 {
     // We already have this mutex locked or both are NULL...
     if (m_mutex_ptr == &mutex)
         return;
 
     Unlock ();
 
     m_mutex_ptr = &mutex;
     m_mutex_ptr->Lock();
 }
 
 void
-Mutex::Locker::Unlock ()
+MutexBase::Locker::Unlock ()
 {
     if (m_mutex_ptr)
     {
         m_mutex_ptr->Unlock ();
         m_mutex_ptr = NULL;
     }
 }
 
 bool
-Mutex::Locker::TryLock (Mutex &mutex, const char *failure_message)
+MutexBase::Locker::TryLock (MutexBase &mutex, const char *failure_message)
 {
     // We already have this mutex locked!
     if (m_mutex_ptr == &mutex)
         return true;
 
     Unlock ();
 
     if (mutex.TryLock(failure_message) == 0)
         m_mutex_ptr = &mutex;
 
     return m_mutex_ptr != NULL;
 }
 
-#ifndef _WIN32
-
-//----------------------------------------------------------------------
-// Default constructor.
-//
-// Creates a pthread mutex with no attributes.
-//----------------------------------------------------------------------
-Mutex::Mutex () :
-    m_mutex()
-{
-    int err;
-    err = ::pthread_mutex_init (&m_mutex, NULL);
-#if ENABLE_MUTEX_ERROR_CHECKING
-    if (err == 0)
-        error_check_mutex (&m_mutex, eMutexActionInitialized);
-#endif
-    assert(err == 0);
-}
-
-//----------------------------------------------------------------------
-// Default constructor.
-//
-// Creates a pthread mutex with "type" as the mutex type.
-//----------------------------------------------------------------------
-Mutex::Mutex (Mutex::Type type) :
-    m_mutex()
-{
-    int err;
-    ::pthread_mutexattr_t attr;
-    err = ::pthread_mutexattr_init (&attr);
-    assert(err == 0);
-    switch (type)
-    {
-    case eMutexTypeNormal:
-#if ENABLE_MUTEX_ERROR_CHECKING
-        err = ::pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_ERRORCHECK);
-#else
-        err = ::pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_NORMAL);
-#endif
-        break;
-
-    case eMutexTypeRecursive:
-        err = ::pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_RECURSIVE);
-        break;
-    }
-    assert(err == 0);
-    err = ::pthread_mutex_init (&m_mutex, &attr);
-#if ENABLE_MUTEX_ERROR_CHECKING
-    if (err == 0)
-        error_check_mutex (&m_mutex, eMutexActionInitialized);
-#endif
-    assert(err == 0);
-    err = ::pthread_mutexattr_destroy (&attr);
-    assert(err == 0);
-}
-
-//----------------------------------------------------------------------
-// Destructor.
-//
-// Destroys the mutex owned by this object.
-//----------------------------------------------------------------------
-Mutex::~Mutex()
-{
-#if ENABLE_MUTEX_ERROR_CHECKING
-    int err = ::pthread_mutex_destroy (&m_mutex);
-    assert(err == 0);
-    if (err == 0)
-        error_check_mutex (&m_mutex, eMutexActionDestroyed);
-    else
-    {
-        Host::SetCrashDescriptionWithFormat ("%s error: pthread_mutex_destroy() => err = %i (%s)", __PRETTY_FUNCTION__, err, strerror(err));
-        assert(err == 0);
-    }
-    memset (&m_mutex, '\xba', sizeof(m_mutex));
-#else
-    ::pthread_mutex_destroy (&m_mutex);
-#endif
-}
-
-//----------------------------------------------------------------------
-// Locks the mutex owned by this object, if the mutex is already
-// locked, the calling thread will block until the mutex becomes
-// available.
-//
-// RETURNS
-//  The error code from the pthread_mutex_lock() function call.
-//----------------------------------------------------------------------
-int
-Mutex::Lock()
-{
-    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_lock (%p)...\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex);
-
-#if ENABLE_MUTEX_ERROR_CHECKING
-    error_check_mutex (&m_mutex, eMutexActionAssertInitialized);
-#endif
-
-    int err = ::pthread_mutex_lock (&m_mutex);
-    
-
-#if ENABLE_MUTEX_ERROR_CHECKING
-    if (err)
-    {
-        Host::SetCrashDescriptionWithFormat ("%s error: pthread_mutex_lock(%p) => err = %i (%s)", __PRETTY_FUNCTION__, &m_mutex, err, strerror(err));
-        assert(err == 0);
-    }
-#endif
-    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_lock (%p) => %i\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex, err);
-    return err;
-}
-
-//----------------------------------------------------------------------
-// Attempts to lock the mutex owned by this object without blocking.
-// If the mutex is already locked, TryLock() will not block waiting
-// for the mutex, but will return an error condition.
-//
-// RETURNS
-//  The error code from the pthread_mutex_trylock() function call.
-//----------------------------------------------------------------------
-int
-Mutex::TryLock(const char *failure_message)
-{
-#if ENABLE_MUTEX_ERROR_CHECKING
-    error_check_mutex (&m_mutex, eMutexActionAssertInitialized);
-#endif
-
-    int err = ::pthread_mutex_trylock (&m_mutex);
-    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_trylock (%p) => %i\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex, err);
-    return err;
-}
-
-//----------------------------------------------------------------------
-// If the current thread holds the lock on the owned mutex, then
-// Unlock() will unlock the mutex. Calling Unlock() on this object
-// that the calling thread does not hold will result in undefined
-// behavior.
-//
-// RETURNS
-//  The error code from the pthread_mutex_unlock() function call.
-//----------------------------------------------------------------------
-int
-Mutex::Unlock()
-{
-#if ENABLE_MUTEX_ERROR_CHECKING
-    error_check_mutex (&m_mutex, eMutexActionAssertInitialized);
-#endif
-
-    int err = ::pthread_mutex_unlock (&m_mutex);
-
-#if ENABLE_MUTEX_ERROR_CHECKING
-    if (err)
-    {
-        Host::SetCrashDescriptionWithFormat ("%s error: pthread_mutex_unlock(%p) => err = %i (%s)", __PRETTY_FUNCTION__, &m_mutex, err, strerror(err));
-        assert(err == 0);
-    }
-#endif
-    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_unlock (%p) => %i\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex, err);
-    return err;
-}
-
-#endif
-
 //----------------------------------------------------------------------
 // Mutex get accessor.
 //----------------------------------------------------------------------
 lldb::mutex_t *
 Mutex::GetMutex()
 {
     return &m_mutex;
 }

source/Host/posix/Mutex.cpp

+//===-- Mutex.cpp -----------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Host/Mutex.h"
+#include "lldb/Host/Host.h"
+#include "lldb/Core/Log.h"
+
+#include <sstream>
+
+#include <fcntl.h>
+#include <pthread.h>
+#include <semaphore.h>
+
+using namespace lldb_private;
+
+#if 0
+// This logging is way too verbose to enable even for a log channel.
+// This logging can be enabled by changing the "#if 0", but should be
+// reverted prior to checking in.
+#include <cstdio>
+#define DEBUG_LOG(fmt, ...) printf(fmt, ## __VA_ARGS__)
+#else
+#define DEBUG_LOG(fmt, ...)
+#endif
+
+// Enable extra mutex error checking
+#ifdef LLDB_CONFIGURATION_DEBUG
+#define ENABLE_MUTEX_ERROR_CHECKING 1
+#include <inttypes.h>
+#endif
+
+#if ENABLE_MUTEX_ERROR_CHECKING
+#include <set>
+
+enum MutexAction
+{
+    eMutexActionInitialized,
+    eMutexActionDestroyed,
+    eMutexActionAssertInitialized
+};
+
+static bool
+error_check_mutex (pthread_mutex_t *m, MutexAction action)
+{
+    typedef std::set<pthread_mutex_t *> mutex_set;
+    static pthread_mutex_t g_mutex_set_mutex = PTHREAD_MUTEX_INITIALIZER;
+    static mutex_set g_initialized_mutex_set;
+    static mutex_set g_destroyed_mutex_set;
+
+    bool success = true;
+    int err;
+    // Manually call lock so we don't to any of this error checking
+    err = ::pthread_mutex_lock (&g_mutex_set_mutex);
+    assert(err == 0);
+    switch (action)
+    {
+        case eMutexActionInitialized:
+            // Make sure this isn't already in our initialized mutex set...
+            assert (g_initialized_mutex_set.find(m) == g_initialized_mutex_set.end());
+            // Remove this from the destroyed set in case it was ever in there
+            g_destroyed_mutex_set.erase(m);
+            // Add the mutex to the initialized set
+            g_initialized_mutex_set.insert(m);
+            break;
+
+        case eMutexActionDestroyed:
+            // Make sure this isn't already in our destroyed mutex set...
+            assert (g_destroyed_mutex_set.find(m) == g_destroyed_mutex_set.end());
+            // Remove this from the initialized so we can put it into the destroyed set
+            g_initialized_mutex_set.erase(m);
+            // Add the mutex to the destroyed set
+            g_destroyed_mutex_set.insert(m);
+            break;
+        case eMutexActionAssertInitialized:
+            // This function will return true if "m" is in the initialized mutex set
+            success = g_initialized_mutex_set.find(m) != g_initialized_mutex_set.end();
+            assert (success);
+            break;
+    }
+    // Manually call unlock so we don't to any of this error checking
+    err = ::pthread_mutex_unlock (&g_mutex_set_mutex);
+    assert(err == 0);
+    return success;
+}
+
+#endif
+
+//----------------------------------------------------------------------
+// Default constructor.
+//
+// Creates a pthread mutex with no attributes.
+//----------------------------------------------------------------------
+Mutex::Mutex () :
+    m_mutex()
+{
+    int err;
+    err = ::pthread_mutex_init (&m_mutex, NULL);
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err == 0)
+        error_check_mutex (&m_mutex, eMutexActionInitialized);
+#endif
+    assert(err == 0);
+}
+
+//----------------------------------------------------------------------
+// Default constructor.
+//
+// Creates a pthread mutex with "type" as the mutex type.
+//----------------------------------------------------------------------
+Mutex::Mutex (Mutex::Type type) :
+    m_mutex()
+{
+    int err;
+    ::pthread_mutexattr_t attr;
+    err = ::pthread_mutexattr_init (&attr);
+    assert(err == 0);
+    switch (type)
+    {
+    case eMutexTypeNormal:
+#if ENABLE_MUTEX_ERROR_CHECKING
+        err = ::pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_ERRORCHECK);
+#else
+        err = ::pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_NORMAL);
+#endif
+        break;
+
+    case eMutexTypeRecursive:
+        err = ::pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_RECURSIVE);
+        break;
+    }
+    assert(err == 0);
+    err = ::pthread_mutex_init (&m_mutex, &attr);
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err == 0)
+        error_check_mutex (&m_mutex, eMutexActionInitialized);
+#endif
+    assert(err == 0);
+    err = ::pthread_mutexattr_destroy (&attr);
+    assert(err == 0);
+}
+
+//----------------------------------------------------------------------
+// Destructor.
+//
+// Destroys the mutex owned by this object.
+//----------------------------------------------------------------------
+Mutex::~Mutex()
+{
+#if ENABLE_MUTEX_ERROR_CHECKING
+    int err = ::pthread_mutex_destroy (&m_mutex);
+    assert(err == 0);
+    if (err == 0)
+        error_check_mutex (&m_mutex, eMutexActionDestroyed);
+    else
+    {
+        Host::SetCrashDescriptionWithFormat ("%s error: pthread_mutex_destroy() => err = %i (%s)", __PRETTY_FUNCTION__, err, strerror(err));
+        assert(err == 0);
+    }
+    memset (&m_mutex, '\xba', sizeof(m_mutex));
+#else
+    ::pthread_mutex_destroy (&m_mutex);
+#endif
+}
+
+//----------------------------------------------------------------------
+// Locks the mutex owned by this object, if the mutex is already
+// locked, the calling thread will block until the mutex becomes
+// available.
+//
+// RETURNS
+//  The error code from the pthread_mutex_lock() function call.
+//----------------------------------------------------------------------
+int
+Mutex::Lock()
+{
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_lock (%p)...\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex);
+
+#if ENABLE_MUTEX_ERROR_CHECKING
+    error_check_mutex (&m_mutex, eMutexActionAssertInitialized);
+#endif
+
+    int err = ::pthread_mutex_lock (&m_mutex);
+
+
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err)
+    {
+        Host::SetCrashDescriptionWithFormat ("%s error: pthread_mutex_lock(%p) => err = %i (%s)", __PRETTY_FUNCTION__, &m_mutex, err, strerror(err));
+        assert(err == 0);
+    }
+#endif
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_lock (%p) => %i\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex, err);
+    return err;
+}
+
+//----------------------------------------------------------------------
+// Attempts to lock the mutex owned by this object without blocking.
+// If the mutex is already locked, TryLock() will not block waiting
+// for the mutex, but will return an error condition.
+//
+// RETURNS
+//  The error code from the pthread_mutex_trylock() function call.
+//----------------------------------------------------------------------
+int
+Mutex::TryLock(const char *failure_message)
+{
+#if ENABLE_MUTEX_ERROR_CHECKING
+    error_check_mutex (&m_mutex, eMutexActionAssertInitialized);
+#endif
+
+    int err = ::pthread_mutex_trylock (&m_mutex);
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_trylock (%p) => %i\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex, err);
+    return err;
+}
+
+//----------------------------------------------------------------------
+// If the current thread holds the lock on the owned mutex, then
+// Unlock() will unlock the mutex. Calling Unlock() on this object
+// that the calling thread does not hold will result in undefined
+// behavior.
+//
+// RETURNS
+//  The error code from the pthread_mutex_unlock() function call.
+//----------------------------------------------------------------------
+int
+Mutex::Unlock()
+{
+#if ENABLE_MUTEX_ERROR_CHECKING
+    error_check_mutex (&m_mutex, eMutexActionAssertInitialized);
+#endif
+
+    int err = ::pthread_mutex_unlock (&m_mutex);
+
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err)
+    {
+        Host::SetCrashDescriptionWithFormat ("%s error: pthread_mutex_unlock(%p) => err = %i (%s)", __PRETTY_FUNCTION__, &m_mutex, err, strerror(err));
+        assert(err == 0);
+    }
+#endif
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] pthread_mutex_unlock (%p) => %i\n", Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), &m_mutex, err);
+    return err;
+}
+
+namespace
+{
+
+const int kInvalidMutex = -1;
+const int kDefaultMaxNameLen = 30;
+const char* kNamePrefix = "/lldb-";
+
+sem_t* getSemaphore(void* sem_handle)
+{
+    return reinterpret_cast<sem_t*>(sem_handle);
+}
+
+std::string createSemaphoreName(const char* base_name)
+{
+    std::ostringstream name;
+    name << kNamePrefix << base_name;
+    return name.str();
+}
+
+}  // namespace
+
+NamedMutex::NamedMutex(const char* base_name):
+    m_name(createSemaphoreName(base_name)),
+    m_mutex(nullptr)
+{
+    sem_t* sem = ::sem_open(m_name.c_str(), O_CREAT, 0660, 1);
+    if (sem == SEM_FAILED)
+    {
+#if ENABLE_MUTEX_ERROR_CHECKING
+        Host::SetCrashDescriptionWithFormat ("%s error: sem_open(%s) => err = %s", __PRETTY_FUNCTION__, m_name.c_str(), ::strerror(errno));
+        assert(sem != SEM_FAILED);
+#endif
+    }
+    else
+        m_mutex = sem;
+
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] sem_open (%s) => %p\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_name.c_str(), m_mutex);
+}
+
+NamedMutex::~NamedMutex()
+{
+    if (m_mutex != nullptr)
+    {
+        int err = ::sem_close(getSemaphore(m_mutex));
+#if ENABLE_MUTEX_ERROR_CHECKING
+        if (err != 0)
+        {
+            Host::SetCrashDescriptionWithFormat ("%s error: sem_close(%p) => err = %s", __PRETTY_FUNCTION__, m_mutex, ::strerror(errno));
+            assert(err == 0);
+        }
+#endif
+        DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] sem_close (%p) => %i\n",
+                   Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+        m_mutex = nullptr;
+        err = ::sem_unlink(m_name.c_str());
+#if ENABLE_MUTEX_ERROR_CHECKING
+        if (err != 0)
+        {
+            Host::SetCrashDescriptionWithFormat ("%s error: sem_unlink(%s) => err = %s", __PRETTY_FUNCTION__, m_name.c_str(), ::strerror(errno));
+            assert(err == 0);
+        }
+#endif
+        DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] sem_unlink (%s) => %i\n",
+                   Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_name.c_str(), err);
+    }
+}
+
+int
+NamedMutex::Lock()
+{
+    if (m_mutex == nullptr)
+        return kInvalidMutex;
+
+    int err = ::sem_wait(getSemaphore(m_mutex));
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err != 0)
+    {
+        Host::SetCrashDescriptionWithFormat ("%s error: sem_wait(%p) => err = %s", __PRETTY_FUNCTION__, m_mutex, ::strerror(errno));
+        assert(err == 0);
+    }
+#endif
+
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] sem_wait (%p) => %i\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+    return err;
+}
+
+int
+NamedMutex::TryLock(const char *failure_message)
+{
+    if (m_mutex == nullptr)
+        return kInvalidMutex;
+
+    int err = ::sem_trywait(getSemaphore(m_mutex));
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err != 0)
+    {
+        Host::SetCrashDescriptionWithFormat ("%s error: sem_wait(%p) => err = %s", __PRETTY_FUNCTION__, m_mutex, ::strerror(errno));
+        assert(err == 0);
+    }
+#endif
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] sem_trywait (%p) => %i\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+    return err;
+}
+
+int
+NamedMutex::Unlock()
+{
+    if (m_mutex == nullptr)
+        return kInvalidMutex;
+
+    int err = ::sem_post(getSemaphore(m_mutex));
+#if ENABLE_MUTEX_ERROR_CHECKING
+    if (err != 0)
+    {
+        Host::SetCrashDescriptionWithFormat ("%s error: sem_post(%p) => err = %s", __PRETTY_FUNCTION__, m_mutex, ::strerror(errno));
+        assert(err == 0);
+    }
+#endif
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] sem_post (%p) => %i\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+    return err;
+}
+
+int
+NamedMutex::GetMaxNameLength()
+{
+    return kDefaultMaxNameLen - strlen(kNamePrefix);
+}

source/Host/windows/Mutex.cpp

 //===-- Mutex.cpp -----------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Host/Mutex.h"
 #include "lldb/Host/Host.h"
 #include "lldb/Host/windows/windows.h"
 
+#include <sstream>
 #include <string.h>
 #include <stdio.h>
 
 #if 0
 // This logging is way too verbose to enable even for a log channel.
 // This logging can be enabled by changing the "#if 0", but should be
 // reverted prior to checking in.
 #include <cstdio>

 //  The error code from the pthread_mutex_unlock() function call.
 //----------------------------------------------------------------------
 int
 Mutex::Unlock()
 {
     LeaveCriticalSection(static_cast<PCRITICAL_SECTION>(m_mutex));
     return 0;
 }
+
+namespace
+{
+
+const int kInvalidMutex = -1;
+const char* kNamePrefix = "Global\\lldb-";
+
+std::string createMutexName(const char* base_name)
+{
+    std::ostringstream name;
+    name << kNamePrefix << base_name;
+    return name.str();
+}
+
+}  // namespace
+
+NamedMutex::NamedMutex(const char* base_name):
+    m_name(createMutexName(base_name)),
+    m_mutex(nullptr)
+{
+    HANDLE h_mutex = ::CreateMutex(nullptr, TRUE, m_name.c_str());
+    if (h_mutex != nullptr)
+        m_mutex = h_mutex;
+
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] CreateMutex (%s) => %p\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_name.c_str(), m_mutex);
+}
+
+NamedMutex::~NamedMutex()
+{
+    if (m_mutex != nullptr)
+    {
+        DWORD err = 0;
+        if (!::CloseHandle(m_mutex))
+            err = ::GetLastError();
+        DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] CloseHandle (%p) => %d\n",
+                   Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+        m_mutex = nullptr;
+    }
+}
+
+int
+NamedMutex::Lock()
+{
+    if (m_mutex == nullptr)
+        return kInvalidMutex;
+
+    int err = 0;
+    if (::WaitForSingleObject(m_mutex, INFINITE) != WAIT_OBJECT_0)
+        err = ::GetLastError();
+
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] WaitForSingleObject (%p, INFINITE) => %d\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+    return err;
+}
+
+int
+NamedMutex::TryLock(const char *failure_message)
+{
+    if (m_mutex == nullptr)
+        return kInvalidMutex;
+
+    int err = 0;
+    if (::WaitForSingleObject(m_mutex, 0) != WAIT_OBJECT_0)
+        err = ::GetLastError();
+
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] WaitForSingleObject (%p, 0) => %d\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+    return err;
+}
+
+int
+NamedMutex::Unlock()
+{
+    if (m_mutex == nullptr)
+        return kInvalidMutex;
+
+    DWORD err = 0;
+    if (!::ReleaseMutex(m_mutex))
+        err = ::GetLastError();
+    DEBUG_LOG ("[%4.4" PRIx64 "/%4.4" PRIx64 "] ReleaseMutex (%p) => %d\n",
+               Host::GetCurrentProcessID(), Host::GetCurrentThreadID(), m_mutex, err);
+
+    return err;
+}
+
+int
+NamedMutex::GetMaxNameLength()
+{
+    return MAX_PATH - strlen(kNamePrefix);
+}

source/Utility/ModuleCache.cpp

 //===--------------------- ModuleCache.cpp ----------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #include "ModuleCache.h"
 
 #include "lldb/Core/Module.h"
 #include "lldb/Core/ModuleSpec.h"
 #include "lldb/Host/FileSystem.h"
+#include "lldb/Host/Mutex.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/FileUtilities.h"
 
 #include <assert.h>
 
 #include <cstdio>
 #
 using namespace lldb;

 Error
 ModuleCache::GetAndPut (const FileSpec &root_dir_spec,
                         const char *hostname,
                         const ModuleSpec &module_spec,
                         const Downloader &downloader,
                         lldb::ModuleSP &cached_module_sp,
                         bool *did_create_ptr)
 {
+    // Get named mutex lock by module UUID.
+    std::string mutex_name = module_spec.GetUUID ().GetAsString ("");
+    // Named mutex name may have length limitations - prune a name if it exceeds max size.
+    const auto mutex_max_len = NamedMutex::GetMaxNameLength();
+    if (mutex_max_len > 0 && mutex_max_len < mutex_name.size())
+        mutex_name = mutex_name.substr(0, mutex_max_len);
+
+    NamedMutex module_mutex(mutex_name.c_str ());
+    NamedMutex::Locker lock(module_mutex);
+
     // Check local cache for a module.
     auto error = Get (root_dir_spec,
                       hostname,
                       module_spec,
                       cached_module_sp,
                       did_create_ptr);
     if (error.Success ())
         return error;
 
     FileSpec tmp_download_file_spec;
     error = downloader (module_spec, tmp_download_file_spec);
     llvm::FileRemover tmp_file_remover (tmp_download_file_spec.GetPath ().c_str ());
     if (error.Fail ())
         return Error("Failed to download module: %s", error.AsCString ());
 
     // Put downloaded file into local module cache.
     error = Put (root_dir_spec,
                  hostname,
                  module_spec,
                  tmp_download_file_spec);
     if (error.Fail ())