This is an archive of the discontinued LLVM Phabricator instance.

Differential D37930

Use ThreadLauncher to launch TaskPool threads
ClosedPublic

Authored by fjricci on Sep 15 2017, 1:55 PM.

Details

Reviewers
labath
tberghammer
clayborg
Commits
rG7ddfe8ef7556: Use ThreadLauncher to launch TaskPool threads
rG52ca3286fbf9: Use ThreadLauncher to launch TaskPool threads
rLLDB313637: Use ThreadLauncher to launch TaskPool threads
rLLDB313537: Use ThreadLauncher to launch TaskPool threads
rL313637: Use ThreadLauncher to launch TaskPool threads
rL313537: Use ThreadLauncher to launch TaskPool threads
Summary

This allows for the stack size to be configured, which isn't
possible with std::thread. Prevents overflowing the stack when
performing complex operations in the task pool on darwin,
where the default pthread stack size is only 512kb.

Diff Detail

Repository
rL LLVM

Event Timeline

fjricci created this revision.Sep 15 2017, 1:55 PM
clayborg accepted this revision.Sep 15 2017, 2:10 PM
This revision is now accepted and ready to land.Sep 15 2017, 2:10 PM
tberghammer accepted this revision.Sep 18 2017, 3:10 AM

Looks good

source/Utility/TaskPool.cpp
61 ↗(On Diff #115484)

(nit): Can you create a named constant for the min stack size as just based on this line I would have no idea what does that number do?

Closed by commit rL313537: Use ThreadLauncher to launch TaskPool threads (authored by fjricci). · Explain WhySep 18 2017, 8:20 AM
This revision was automatically updated to reflect the committed changes.
fjricci reopened this revision.Sep 18 2017, 8:46 AM

Is ThreadLauncher unavailable in this code for some reason? The link failed on linux buildbots (building lldb on Darwin was fine locally): http://lab.llvm.org:8011/builders/lldb-x86_64-ubuntu-14.04-cmake/builds/13311/steps/run%20unit%20tests/logs/stdio

This revision is now accepted and ready to land.Sep 18 2017, 8:46 AM
fjricci updated this revision to Diff 115691.Sep 18 2017, 11:30 AM

Move TaskPool from Utility to Host

Harbormaster completed remote builds in B10368: Diff 115691.Sep 18 2017, 11:30 AM
Herald added subscribers: JDevlieghere, mgorny. · View Herald TranscriptSep 18 2017, 11:30 AM
fjricci requested review of this revision.Sep 18 2017, 11:32 AM
fjricci edited edge metadata.
tberghammer accepted this revision.Sep 19 2017, 6:59 AM

I don't really have an opinion if moving TaskPool to Host is a good or bad idea (haven't followed the recent layering efforts) but other then that looks good. I also tested it on Linux and it works fine.

This revision is now accepted and ready to land.Sep 19 2017, 6:59 AM
clayborg accepted this revision.Sep 19 2017, 8:34 AM
Closed by commit rL313637: Use ThreadLauncher to launch TaskPool threads (authored by fjricci). · Explain WhySep 19 2017, 8:40 AM
This revision was automatically updated to reflect the committed changes.
labath mentioned this in D38938: Logging: provide a way to safely disable logging in a forked process.Oct 15 2017, 2:42 PM
labath mentioned this in rL316173: Logging: Make sure logging machinery is in a consistent state after forking.Oct 19 2017, 10:43 AM
labath mentioned this in rL316368: Logging: Disable logging after fork().Oct 23 2017, 12:42 PM

Revision Contents

PathSize
lldb/
trunk/
include/
lldb/
Host/
92 lines
Utility/
92 lines
lldb.xcodeproj/
8 lines
source/
Host/
1 line
common/
109 lines
Plugins/
SymbolFile/
DWARF/
2 lines
Utility/
1 line
98 lines
unittests/
Host/
1 line
43 lines
Utility/
1 line
43 lines

Diff 115844

lldb/trunk/include/lldb/Host/TaskPool.h

//===--------------------- TaskPool.h ---------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#ifndef utility_TaskPool_h_
#define utility_TaskPool_h_
#include "llvm/ADT/STLExtras.h"
#include <functional> // for bind, function
#include <future>
#include <list>
#include <memory> // for make_shared
#include <mutex> // for mutex, unique_lock, condition_variable
#include <type_traits> // for forward, result_of, move
// Global TaskPool class for running tasks in parallel on a set of worker thread
// created the first
// time the task pool is used. The TaskPool provide no guarantee about the order
// the task will be run
// and about what tasks will run in parallel. None of the task added to the task
// pool should block
// on something (mutex, future, condition variable) what will be set only by the
// completion of an
// other task on the task pool as they may run on the same thread sequentally.
class TaskPool {
public:
// Add a new task to the task pool and return a std::future belonging to the
// newly created task.
// The caller of this function has to wait on the future for this task to
// complete.
template <typename F, typename... Args>
static std::future<typename std::result_of<F(Args...)>::type>
AddTask(F &&f, Args &&... args);
// Run all of the specified tasks on the task pool and wait until all of them
// are finished
// before returning. This method is intended to be used for small number tasks
// where listing
// them as function arguments is acceptable. For running large number of tasks
// you should use
// AddTask for each task and then call wait() on each returned future.
template <typename... T> static void RunTasks(T &&... tasks);
private:
TaskPool() = delete;
template <typename... T> struct RunTaskImpl;
static void AddTaskImpl(std::function<void()> &&task_fn);
};
template <typename F, typename... Args>
std::future<typename std::result_of<F(Args...)>::type>
TaskPool::AddTask(F &&f, Args &&... args) {
auto task_sp = std::make_shared<
std::packaged_task<typename std::result_of<F(Args...)>::type()>>(
std::bind(std::forward<F>(f), std::forward<Args>(args)...));
AddTaskImpl([task_sp]() { (*task_sp)(); });
return task_sp->get_future();
}
template <typename... T> void TaskPool::RunTasks(T &&... tasks) {
RunTaskImpl<T...>::Run(std::forward<T>(tasks)...);
}
template <typename Head, typename... Tail>
struct TaskPool::RunTaskImpl<Head, Tail...> {
static void Run(Head &&h, Tail &&... t) {
auto f = AddTask(std::forward<Head>(h));
RunTaskImpl<Tail...>::Run(std::forward<Tail>(t)...);
f.wait();
}
};
template <> struct TaskPool::RunTaskImpl<> {
static void Run() {}
};
// Run 'func' on every value from begin .. end-1. Each worker will grab
// 'batch_size' numbers at a time to work on, so for very fast functions, batch
// should be large enough to avoid too much cache line contention.
void TaskMapOverInt(size_t begin, size_t end,
const llvm::function_ref<void(size_t)> &func);
#endif // #ifndef utility_TaskPool_h_

lldb/trunk/include/lldb/Utility/TaskPool.h

//===--------------------- TaskPool.h ---------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#ifndef utility_TaskPool_h_
#define utility_TaskPool_h_
#include "llvm/ADT/STLExtras.h"
#include <functional> // for bind, function
#include <future>
#include <list>
#include <memory> // for make_shared
#include <mutex> // for mutex, unique_lock, condition_variable
#include <type_traits> // for forward, result_of, move
// Global TaskPool class for running tasks in parallel on a set of worker thread
// created the first
// time the task pool is used. The TaskPool provide no guarantee about the order
// the task will be run
// and about what tasks will run in parallel. None of the task added to the task
// pool should block
// on something (mutex, future, condition variable) what will be set only by the
// completion of an
// other task on the task pool as they may run on the same thread sequentally.
class TaskPool {
public:
// Add a new task to the task pool and return a std::future belonging to the
// newly created task.
// The caller of this function has to wait on the future for this task to
// complete.
template <typename F, typename... Args>
static std::future<typename std::result_of<F(Args...)>::type>
AddTask(F &&f, Args &&... args);
// Run all of the specified tasks on the task pool and wait until all of them
// are finished
// before returning. This method is intended to be used for small number tasks
// where listing
// them as function arguments is acceptable. For running large number of tasks
// you should use
// AddTask for each task and then call wait() on each returned future.
template <typename... T> static void RunTasks(T &&... tasks);
private:
TaskPool() = delete;
template <typename... T> struct RunTaskImpl;
static void AddTaskImpl(std::function<void()> &&task_fn);
};
template <typename F, typename... Args>
std::future<typename std::result_of<F(Args...)>::type>
TaskPool::AddTask(F &&f, Args &&... args) {
auto task_sp = std::make_shared<
std::packaged_task<typename std::result_of<F(Args...)>::type()>>(
std::bind(std::forward<F>(f), std::forward<Args>(args)...));
AddTaskImpl([task_sp]() { (*task_sp)(); });
return task_sp->get_future();
}
template <typename... T> void TaskPool::RunTasks(T &&... tasks) {
RunTaskImpl<T...>::Run(std::forward<T>(tasks)...);
}
template <typename Head, typename... Tail>
struct TaskPool::RunTaskImpl<Head, Tail...> {
static void Run(Head &&h, Tail &&... t) {
auto f = AddTask(std::forward<Head>(h));
RunTaskImpl<Tail...>::Run(std::forward<Tail>(t)...);
f.wait();
}
};
template <> struct TaskPool::RunTaskImpl<> {
static void Run() {}
};
// Run 'func' on every value from begin .. end-1. Each worker will grab
// 'batch_size' numbers at a time to work on, so for very fast functions, batch
// should be large enough to avoid too much cache line contention.
void TaskMapOverInt(size_t begin, size_t end,
const llvm::function_ref<void(size_t)> &func);
#endif // #ifndef utility_TaskPool_h_

lldb/trunk/lldb.xcodeproj/project.pbxproj

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 2,658 Lines • ▼ Show 20 Lines
6D95DBFE1B9DC057000E318A /* HashedNameToDIE.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HashedNameToDIE.cpp; sourceTree = "<group>"; }; 6D95DBFE1B9DC057000E318A /* HashedNameToDIE.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = HashedNameToDIE.cpp; sourceTree = "<group>"; };
6D95DBFF1B9DC057000E318A /* SymbolFileDWARFDwo.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SymbolFileDWARFDwo.cpp; sourceTree = "<group>"; }; 6D95DBFF1B9DC057000E318A /* SymbolFileDWARFDwo.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = SymbolFileDWARFDwo.cpp; sourceTree = "<group>"; };
6D95DC031B9DC06F000E318A /* DIERef.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DIERef.h; sourceTree = "<group>"; }; 6D95DC031B9DC06F000E318A /* DIERef.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = DIERef.h; sourceTree = "<group>"; };
6D95DC041B9DC06F000E318A /* SymbolFileDWARFDwo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SymbolFileDWARFDwo.h; sourceTree = "<group>"; }; 6D95DC041B9DC06F000E318A /* SymbolFileDWARFDwo.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = SymbolFileDWARFDwo.h; sourceTree = "<group>"; };
6D99A3611BBC2F1600979793 /* ArmUnwindInfo.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = ArmUnwindInfo.h; path = include/lldb/Symbol/ArmUnwindInfo.h; sourceTree = "<group>"; }; 6D99A3611BBC2F1600979793 /* ArmUnwindInfo.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = ArmUnwindInfo.h; path = include/lldb/Symbol/ArmUnwindInfo.h; sourceTree = "<group>"; };
6D99A3621BBC2F3200979793 /* ArmUnwindInfo.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ArmUnwindInfo.cpp; path = source/Symbol/ArmUnwindInfo.cpp; sourceTree = "<group>"; }; 6D99A3621BBC2F3200979793 /* ArmUnwindInfo.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = ArmUnwindInfo.cpp; path = source/Symbol/ArmUnwindInfo.cpp; sourceTree = "<group>"; };
6D9AB3DC1BB2B74E003F2289 /* TypeMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = TypeMap.cpp; path = source/Symbol/TypeMap.cpp; sourceTree = "<group>"; }; 6D9AB3DC1BB2B74E003F2289 /* TypeMap.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = TypeMap.cpp; path = source/Symbol/TypeMap.cpp; sourceTree = "<group>"; };
6D9AB3DE1BB2B76B003F2289 /* TypeMap.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = TypeMap.h; path = include/lldb/Symbol/TypeMap.h; sourceTree = "<group>"; }; 6D9AB3DE1BB2B76B003F2289 /* TypeMap.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = TypeMap.h; path = include/lldb/Symbol/TypeMap.h; sourceTree = "<group>"; };
6DEC6F381BD66D750091ABA6 /* TaskPool.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = TaskPool.cpp; path = source/Utility/TaskPool.cpp; sourceTree = "<group>"; }; 6DEC6F381BD66D750091ABA6 /* TaskPool.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = TaskPool.cpp; path = source/Host/common/TaskPool.cpp; sourceTree = "<group>"; };
6DEC6F3A1BD66D950091ABA6 /* TaskPool.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = TaskPool.h; path = include/lldb/Utility/TaskPool.h; sourceTree = "<group>"; }; 6DEC6F3A1BD66D950091ABA6 /* TaskPool.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; name = TaskPool.h; path = include/lldb/Host/TaskPool.h; sourceTree = "<group>"; };
8C26C4241C3EA4340031DF7C /* TSanRuntime.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = TSanRuntime.cpp; path = TSan/TSanRuntime.cpp; sourceTree = "<group>"; }; 8C26C4241C3EA4340031DF7C /* TSanRuntime.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; name = TSanRuntime.cpp; path = TSan/TSanRuntime.cpp; sourceTree = "<group>"; };
8C26C4251C3EA4340031DF7C /* TSanRuntime.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = TSanRuntime.h; path = TSan/TSanRuntime.h; sourceTree = "<group>"; }; 8C26C4251C3EA4340031DF7C /* TSanRuntime.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = TSanRuntime.h; path = TSan/TSanRuntime.h; sourceTree = "<group>"; };
8C2D6A52197A1EAF006989C9 /* MemoryHistory.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = MemoryHistory.cpp; path = source/Target/MemoryHistory.cpp; sourceTree = "<group>"; }; 8C2D6A52197A1EAF006989C9 /* MemoryHistory.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; name = MemoryHistory.cpp; path = source/Target/MemoryHistory.cpp; sourceTree = "<group>"; };
8C2D6A54197A1EBE006989C9 /* MemoryHistory.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = MemoryHistory.h; path = include/lldb/Target/MemoryHistory.h; sourceTree = "<group>"; }; 8C2D6A54197A1EBE006989C9 /* MemoryHistory.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; name = MemoryHistory.h; path = include/lldb/Target/MemoryHistory.h; sourceTree = "<group>"; };
8C2D6A5A197A1FDC006989C9 /* MemoryHistoryASan.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MemoryHistoryASan.cpp; sourceTree = "<group>"; }; 8C2D6A5A197A1FDC006989C9 /* MemoryHistoryASan.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = MemoryHistoryASan.cpp; sourceTree = "<group>"; };
8C2D6A5B197A1FDC006989C9 /* MemoryHistoryASan.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MemoryHistoryASan.h; sourceTree = "<group>"; }; 8C2D6A5B197A1FDC006989C9 /* MemoryHistoryASan.h */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.c.h; path = MemoryHistoryASan.h; sourceTree = "<group>"; };
8C3BD9931EF45D9B0016C343 /* MainThreadCheckerRuntime.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = MainThreadCheckerRuntime.h; sourceTree = "<group>"; }; 8C3BD9931EF45D9B0016C343 /* MainThreadCheckerRuntime.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = MainThreadCheckerRuntime.h; sourceTree = "<group>"; };
8C3BD9951EF45D9B0016C343 /* MainThreadCheckerRuntime.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = MainThreadCheckerRuntime.cpp; sourceTree = "<group>"; }; 8C3BD9951EF45D9B0016C343 /* MainThreadCheckerRuntime.cpp */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.cpp.cpp; path = MainThreadCheckerRuntime.cpp; sourceTree = "<group>"; };
▲ Show 20 Lines • Show All 1,731 Lines • ▼ Show 20 Lines 2682F168115ED9C800CCFF99 /* Utility */ = {
94380B8019940B0300BFE4A8 /* StringLexer.h */, 94380B8019940B0300BFE4A8 /* StringLexer.h */,
94380B8119940B0A00BFE4A8 /* StringLexer.cpp */, 94380B8119940B0A00BFE4A8 /* StringLexer.cpp */,
9A35765E116E76A700E8ED2F /* StringList.h */, 9A35765E116E76A700E8ED2F /* StringList.h */,
9A35765F116E76B900E8ED2F /* StringList.cpp */, 9A35765F116E76B900E8ED2F /* StringList.cpp */,
26F2F8FD1B156678007857DE /* StructuredData.h */, 26F2F8FD1B156678007857DE /* StructuredData.h */,
AFEC3361194A8ABA00FF05C6 /* StructuredData.cpp */, AFEC3361194A8ABA00FF05C6 /* StructuredData.cpp */,
94BA8B6E176F8CA0005A91B5 /* Range.h */, 94BA8B6E176F8CA0005A91B5 /* Range.h */,
94BA8B6C176F8C9B005A91B5 /* Range.cpp */, 94BA8B6C176F8C9B005A91B5 /* Range.cpp */,
6DEC6F3A1BD66D950091ABA6 /* TaskPool.h */,
6DEC6F381BD66D750091ABA6 /* TaskPool.cpp */,
AFF8FF0B1E779D4B003830EF /* TildeExpressionResolver.cpp */, AFF8FF0B1E779D4B003830EF /* TildeExpressionResolver.cpp */,
AFF8FF0D1E779D51003830EF /* TildeExpressionResolver.h */, AFF8FF0D1E779D51003830EF /* TildeExpressionResolver.h */,
26BC7D7E10F1B77400F91463 /* Timer.h */, 26BC7D7E10F1B77400F91463 /* Timer.h */,
26BC7E9610F1B85900F91463 /* Timer.cpp */, 26BC7E9610F1B85900F91463 /* Timer.cpp */,
2654A6911E552F3C00DA1013 /* UriParser.h */, 2654A6911E552F3C00DA1013 /* UriParser.h */,
33064C991A5C7A330033D415 /* UriParser.cpp */, 33064C991A5C7A330033D415 /* UriParser.cpp */,
AFC2DCEA1E6E2F7D00283714 /* UserID.cpp */, AFC2DCEA1E6E2F7D00283714 /* UserID.cpp */,
AFC2DCEC1E6E2F8C00283714 /* UserID.h */, AFC2DCEC1E6E2F8C00283714 /* UserID.h */,
▲ Show 20 Lines • Show All 817 Lines • ▼ Show 20 Lines 26BC7DD010F1B7C100F91463 /* Host */ = {
2654A68E1E552D2400DA1013 /* PseudoTerminal.h */, 2654A68E1E552D2400DA1013 /* PseudoTerminal.h */,
2654A68C1E552D1500DA1013 /* PseudoTerminal.cpp */, 2654A68C1E552D1500DA1013 /* PseudoTerminal.cpp */,
236124A71986B50E004EFC37 /* Socket.h */, 236124A71986B50E004EFC37 /* Socket.h */,
26D7E45B13D5E2F9007FD12B /* SocketAddress.h */, 26D7E45B13D5E2F9007FD12B /* SocketAddress.h */,
26D7E45C13D5E30A007FD12B /* SocketAddress.cpp */, 26D7E45C13D5E30A007FD12B /* SocketAddress.cpp */,
267A47F21B14115A0021A5BC /* SoftwareBreakpoint.h */, 267A47F21B14115A0021A5BC /* SoftwareBreakpoint.h */,
232CB613191E00CC00EF39FC /* SoftwareBreakpoint.cpp */, 232CB613191E00CC00EF39FC /* SoftwareBreakpoint.cpp */,
2689B0A4113EE3CD00A4AEDB /* Symbols.h */, 2689B0A4113EE3CD00A4AEDB /* Symbols.h */,
6DEC6F3A1BD66D950091ABA6 /* TaskPool.h */,
6DEC6F381BD66D750091ABA6 /* TaskPool.cpp */,
268DA871130095D000C9483A /* Terminal.h */, 268DA871130095D000C9483A /* Terminal.h */,
3FDFED2319BA6D55009756A7 /* ThreadLauncher.h */, 3FDFED2319BA6D55009756A7 /* ThreadLauncher.h */,
267A48031B1416080021A5BC /* XML.h */, 267A48031B1416080021A5BC /* XML.h */,
267A48001B1411E40021A5BC /* XML.cpp */, 267A48001B1411E40021A5BC /* XML.cpp */,
); );
name = Host; name = Host;
sourceTree = "<group>"; sourceTree = "<group>";
}; };
▲ Show 20 Lines • Show All 5,522 Lines • Show Last 20 Lines

lldb/trunk/source/Host/CMakeLists.txt

Show All 25 Linesadd_host_subdirectory(common
common/PipeBase.cpp common/PipeBase.cpp
common/ProcessRunLock.cpp common/ProcessRunLock.cpp
common/PseudoTerminal.cpp common/PseudoTerminal.cpp
common/Socket.cpp common/Socket.cpp
common/SocketAddress.cpp common/SocketAddress.cpp
common/SoftwareBreakpoint.cpp common/SoftwareBreakpoint.cpp
common/StringConvert.cpp common/StringConvert.cpp
common/Symbols.cpp common/Symbols.cpp
common/TaskPool.cpp
common/TCPSocket.cpp common/TCPSocket.cpp
common/Terminal.cpp common/Terminal.cpp
common/ThreadLauncher.cpp common/ThreadLauncher.cpp
common/XML.cpp common/XML.cpp
common/UDPSocket.cpp common/UDPSocket.cpp
) )
# Keep track of whether we want to provide a define for the # Keep track of whether we want to provide a define for the
▲ Show 20 Lines • Show All 150 Lines • Show Last 20 Lines

lldb/trunk/source/Host/common/TaskPool.cpp

//===--------------------- TaskPool.cpp -------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#include "lldb/Host/TaskPool.h"
#include "lldb/Host/ThreadLauncher.h"
#include <cstdint> // for uint32_t
#include <queue> // for queue
#include <thread> // for thread
namespace {
class TaskPoolImpl {
public:
static TaskPoolImpl &GetInstance();
void AddTask(std::function<void()> &&task_fn);
private:
TaskPoolImpl();
static lldb::thread_result_t WorkerPtr(void *pool);
static void Worker(TaskPoolImpl *pool);
std::queue<std::function<void()>> m_tasks;
std::mutex m_tasks_mutex;
uint32_t m_thread_count;
};
} // end of anonymous namespace
TaskPoolImpl &TaskPoolImpl::GetInstance() {
static TaskPoolImpl g_task_pool_impl;
return g_task_pool_impl;
}
void TaskPool::AddTaskImpl(std::function<void()> &&task_fn) {
TaskPoolImpl::GetInstance().AddTask(std::move(task_fn));
}
TaskPoolImpl::TaskPoolImpl() : m_thread_count(0) {}
void TaskPoolImpl::AddTask(std::function<void()> &&task_fn) {
static const uint32_t max_threads = std::thread::hardware_concurrency();
const size_t min_stack_size = 8 * 1024 * 1024;
std::unique_lock<std::mutex> lock(m_tasks_mutex);
m_tasks.emplace(std::move(task_fn));
if (m_thread_count < max_threads) {
m_thread_count++;
// Note that this detach call needs to happen with the m_tasks_mutex held.
// This prevents the thread
// from exiting prematurely and triggering a linux libc bug
// (https://sourceware.org/bugzilla/show_bug.cgi?id=19951).
lldb_private::ThreadLauncher::LaunchThread("task-pool.worker", WorkerPtr,
this, nullptr, min_stack_size)
.Release();
}
}
lldb::thread_result_t TaskPoolImpl::WorkerPtr(void *pool) {
Worker((TaskPoolImpl *)pool);
return 0;
}
void TaskPoolImpl::Worker(TaskPoolImpl *pool) {
while (true) {
std::unique_lock<std::mutex> lock(pool->m_tasks_mutex);
if (pool->m_tasks.empty()) {
pool->m_thread_count--;
break;
}
std::function<void()> f = pool->m_tasks.front();
pool->m_tasks.pop();
lock.unlock();
f();
}
}
void TaskMapOverInt(size_t begin, size_t end,
const llvm::function_ref<void(size_t)> &func) {
std::atomic<size_t> idx{begin};
size_t num_workers =
std::min<size_t>(end, std::thread::hardware_concurrency());
auto wrapper = [&idx, end, &func]() {
while (true) {
size_t i = idx.fetch_add(1);
if (i >= end)
break;
func(i);
}
};
std::vector<std::future<void>> futures;
futures.reserve(num_workers);
for (size_t i = 0; i < num_workers; i++)
futures.push_back(TaskPool::AddTask(wrapper));
for (size_t i = 0; i < num_workers; i++)
futures[i].wait();
}

lldb/trunk/source/Plugins/SymbolFile/DWARF/SymbolFileDWARF.cpp

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Lines
#include "lldb/Symbol/TypeSystem.h" #include "lldb/Symbol/TypeSystem.h"
#include "lldb/Symbol/VariableList.h" #include "lldb/Symbol/VariableList.h"
#include "Plugins/Language/CPlusPlus/CPlusPlusLanguage.h" #include "Plugins/Language/CPlusPlus/CPlusPlusLanguage.h"
#include "Plugins/Language/ObjC/ObjCLanguage.h" #include "Plugins/Language/ObjC/ObjCLanguage.h"
#include "lldb/Target/Language.h" #include "lldb/Target/Language.h"
#include "lldb/Utility/TaskPool.h" #include "lldb/Host/TaskPool.h"
#include "DWARFASTParser.h" #include "DWARFASTParser.h"
#include "DWARFASTParserClang.h" #include "DWARFASTParserClang.h"
#include "DWARFCompileUnit.h" #include "DWARFCompileUnit.h"
#include "DWARFDIECollection.h" #include "DWARFDIECollection.h"
#include "DWARFDebugAbbrev.h" #include "DWARFDebugAbbrev.h"
#include "DWARFDebugAranges.h" #include "DWARFDebugAranges.h"
#include "DWARFDebugInfo.h" #include "DWARFDebugInfo.h"
▲ Show 20 Lines • Show All 4,298 Lines • Show Last 20 Lines

lldb/trunk/source/Utility/CMakeLists.txt

Show All 23 Linesadd_lldb_library(lldbUtility
StreamCallback.cpp StreamCallback.cpp
StreamGDBRemote.cpp StreamGDBRemote.cpp
StreamString.cpp StreamString.cpp
StringExtractor.cpp StringExtractor.cpp
StringExtractorGDBRemote.cpp StringExtractorGDBRemote.cpp
StringLexer.cpp StringLexer.cpp
StringList.cpp StringList.cpp
StructuredData.cpp StructuredData.cpp
TaskPool.cpp
TildeExpressionResolver.cpp TildeExpressionResolver.cpp
Timer.cpp Timer.cpp
UserID.cpp UserID.cpp
UriParser.cpp UriParser.cpp
UUID.cpp UUID.cpp
VASprintf.cpp VASprintf.cpp
VMRange.cpp VMRange.cpp
LINK_LIBS LINK_LIBS
# lldbUtility cannot have any dependencies # lldbUtility cannot have any dependencies
LINK_COMPONENTS LINK_COMPONENTS
BinaryFormat BinaryFormat
Support Support
) )

lldb/trunk/source/Utility/TaskPool.cpp

//===--------------------- TaskPool.cpp -------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
#include "lldb/Utility/TaskPool.h"
#include <cstdint> // for uint32_t
#include <queue> // for queue
#include <thread> // for thread
namespace {
class TaskPoolImpl {
public:
static TaskPoolImpl &GetInstance();
void AddTask(std::function<void()> &&task_fn);
private:
TaskPoolImpl();
static void Worker(TaskPoolImpl *pool);
std::queue<std::function<void()>> m_tasks;
std::mutex m_tasks_mutex;
uint32_t m_thread_count;
};
} // end of anonymous namespace
TaskPoolImpl &TaskPoolImpl::GetInstance() {
static TaskPoolImpl g_task_pool_impl;
return g_task_pool_impl;
}
void TaskPool::AddTaskImpl(std::function<void()> &&task_fn) {
TaskPoolImpl::GetInstance().AddTask(std::move(task_fn));
}
TaskPoolImpl::TaskPoolImpl() : m_thread_count(0) {}
void TaskPoolImpl::AddTask(std::function<void()> &&task_fn) {
static const uint32_t max_threads = std::thread::hardware_concurrency();
std::unique_lock<std::mutex> lock(m_tasks_mutex);
m_tasks.emplace(std::move(task_fn));
if (m_thread_count < max_threads) {
m_thread_count++;
// Note that this detach call needs to happen with the m_tasks_mutex held.
// This prevents the thread
// from exiting prematurely and triggering a linux libc bug
// (https://sourceware.org/bugzilla/show_bug.cgi?id=19951).
std::thread(Worker, this).detach();
}
}
void TaskPoolImpl::Worker(TaskPoolImpl *pool) {
while (true) {
std::unique_lock<std::mutex> lock(pool->m_tasks_mutex);
if (pool->m_tasks.empty()) {
pool->m_thread_count--;
break;
}
std::function<void()> f = pool->m_tasks.front();
pool->m_tasks.pop();
lock.unlock();
f();
}
}
void TaskMapOverInt(size_t begin, size_t end,
const llvm::function_ref<void(size_t)> &func) {
std::atomic<size_t> idx{begin};
size_t num_workers =
std::min<size_t>(end, std::thread::hardware_concurrency());
auto wrapper = [&idx, end, &func]() {
while (true) {
size_t i = idx.fetch_add(1);
if (i >= end)
break;
func(i);
}
};
std::vector<std::future<void>> futures;
futures.reserve(num_workers);
for (size_t i = 0; i < num_workers; i++)
futures.push_back(TaskPool::AddTask(wrapper));
for (size_t i = 0; i < num_workers; i++)
futures[i].wait();
}

lldb/trunk/unittests/Host/CMakeLists.txt

set (FILES set (FILES
FileSpecTest.cpp FileSpecTest.cpp
FileSystemTest.cpp FileSystemTest.cpp
HostTest.cpp HostTest.cpp
MainLoopTest.cpp MainLoopTest.cpp
SocketAddressTest.cpp SocketAddressTest.cpp
SocketTest.cpp SocketTest.cpp
SymbolsTest.cpp SymbolsTest.cpp
TaskPoolTest.cpp
) )
if (CMAKE_SYSTEM_NAME MATCHES "Linux|Android") if (CMAKE_SYSTEM_NAME MATCHES "Linux|Android")
list(APPEND FILES list(APPEND FILES
linux/HostTest.cpp linux/HostTest.cpp
linux/SupportTest.cpp linux/SupportTest.cpp
) )
endif() endif()
add_lldb_unittest(HostTests add_lldb_unittest(HostTests
${FILES} ${FILES}
LINK_LIBS LINK_LIBS
lldbCore lldbCore
lldbHost lldbHost
) )

lldb/trunk/unittests/Host/TaskPoolTest.cpp

#include "gtest/gtest.h"
#include "lldb/Host/TaskPool.h"
TEST(TaskPoolTest, AddTask) {
auto fn = [](int x) { return x * x + 1; };
auto f1 = TaskPool::AddTask(fn, 1);
auto f2 = TaskPool::AddTask(fn, 2);
auto f3 = TaskPool::AddTask(fn, 3);
auto f4 = TaskPool::AddTask(fn, 4);
ASSERT_EQ(10, f3.get());
ASSERT_EQ(2, f1.get());
ASSERT_EQ(17, f4.get());
ASSERT_EQ(5, f2.get());
}
TEST(TaskPoolTest, RunTasks) {
std::vector<int> r(4);
auto fn = [](int x, int &y) { y = x * x + 1; };
TaskPool::RunTasks([fn, &r]() { fn(1, r[0]); }, [fn, &r]() { fn(2, r[1]); },
[fn, &r]() { fn(3, r[2]); }, [fn, &r]() { fn(4, r[3]); });
ASSERT_EQ(2, r[0]);
ASSERT_EQ(5, r[1]);
ASSERT_EQ(10, r[2]);
ASSERT_EQ(17, r[3]);
}
TEST(TaskPoolTest, TaskMap) {
int data[4];
auto fn = [&data](int x) { data[x] = x * x; };
TaskMapOverInt(0, 4, fn);
ASSERT_EQ(data[0], 0);
ASSERT_EQ(data[1], 1);
ASSERT_EQ(data[2], 4);
ASSERT_EQ(data[3], 9);
}

lldb/trunk/unittests/Utility/CMakeLists.txt

add_subdirectory(Helpers) add_subdirectory(Helpers)
add_lldb_unittest(UtilityTests add_lldb_unittest(UtilityTests
ConstStringTest.cpp ConstStringTest.cpp
JSONTest.cpp JSONTest.cpp
LogTest.cpp LogTest.cpp
NameMatchesTest.cpp NameMatchesTest.cpp
StatusTest.cpp StatusTest.cpp
StringExtractorTest.cpp StringExtractorTest.cpp
StructuredDataTest.cpp StructuredDataTest.cpp
TaskPoolTest.cpp
TildeExpressionResolverTest.cpp TildeExpressionResolverTest.cpp
TimeoutTest.cpp TimeoutTest.cpp
TimerTest.cpp TimerTest.cpp
UriParserTest.cpp UriParserTest.cpp
VASprintfTest.cpp VASprintfTest.cpp
LINK_LIBS LINK_LIBS
lldbUtility lldbUtility
lldbUtilityHelpers lldbUtilityHelpers
LINK_COMPONENTS LINK_COMPONENTS
Support Support
) )
add_unittest_inputs(UtilityTests add_unittest_inputs(UtilityTests
StructuredData-basic.json StructuredData-basic.json
) )

lldb/trunk/unittests/Utility/TaskPoolTest.cpp

#include "gtest/gtest.h"
#include "lldb/Utility/TaskPool.h"
TEST(TaskPoolTest, AddTask) {
auto fn = [](int x) { return x * x + 1; };
auto f1 = TaskPool::AddTask(fn, 1);
auto f2 = TaskPool::AddTask(fn, 2);
auto f3 = TaskPool::AddTask(fn, 3);
auto f4 = TaskPool::AddTask(fn, 4);
ASSERT_EQ(10, f3.get());
ASSERT_EQ(2, f1.get());
ASSERT_EQ(17, f4.get());
ASSERT_EQ(5, f2.get());
}
TEST(TaskPoolTest, RunTasks) {
std::vector<int> r(4);
auto fn = [](int x, int &y) { y = x * x + 1; };
TaskPool::RunTasks([fn, &r]() { fn(1, r[0]); }, [fn, &r]() { fn(2, r[1]); },
[fn, &r]() { fn(3, r[2]); }, [fn, &r]() { fn(4, r[3]); });
ASSERT_EQ(2, r[0]);
ASSERT_EQ(5, r[1]);
ASSERT_EQ(10, r[2]);
ASSERT_EQ(17, r[3]);
}
TEST(TaskPoolTest, TaskMap) {
int data[4];
auto fn = [&data](int x) { data[x] = x * x; };
TaskMapOverInt(0, 4, fn);
ASSERT_EQ(data[0], 0);
ASSERT_EQ(data[1], 1);
ASSERT_EQ(data[2], 4);
ASSERT_EQ(data[3], 9);
}
lldb/trunk/unittests/Host/TaskPoolTest.cpp