DarkHelp  v1.8.5-1
C++ API for the neural network framework Darknet
Looking for a C++ dev who knows OpenCV?
I'm looking for work. Hire me!
DarkHelp::DHThreads Class Referencefinal

This class allows you to easily run multiple identical copies of a neural network to process many files at once. More...

#include "DarkHelpThreads.hpp"

Collaboration diagram for DarkHelp::DHThreads:

Public Types

using ResultsMap = std::map< std::string, DarkHelp::PredictionResults >
 The key is the input image filename, while the value contains a copy of the prediction results. More...
 

Public Member Functions

 DHThreads ()
 Constructor. More...
 
 DHThreads (const DarkHelp::Config &c, const size_t workers, const std::filesystem::path &output_directory=".")
 Constructor. More...
 
 DHThreads (const std::filesystem::path &filename, const std::string &key, const size_t workers, const std::filesystem::path &output_directory=".", const DarkHelp::EDriver &driver=DarkHelp::EDriver::kDarknet)
 Constructor. More...
 
 ~DHThreads ()
 Destructor. This will immediately stop all the worker threads and discard any results. More...
 
std::string add_image (cv::Mat image)
 Add a single image to be processed. More...
 
DHThreadsadd_images (const std::filesystem::path &dir)
 Can be used to add a single image, or a subdirectory. More...
 
size_t files_remaining () const
 Get the number of images and files that have not yet been processed. More...
 
DarkHelp::NNget_nn (const size_t idx)
 Gain access to the neural network for the given worker thread. More...
 
ResultsMap get_results ()
 Get all of the available prediction results. More...
 
DHThreadsinit (const DarkHelp::Config &c, const size_t workers, const std::filesystem::path &output_directory=".")
 Mostly for use with the default constructor. More...
 
DHThreadsinit (const std::filesystem::path &filename, const std::string &key, const size_t workers, const std::filesystem::path &output_directory=".", const DarkHelp::EDriver &driver=DarkHelp::EDriver::kDarknet)
 Similar to the other init() call, but uses the name of the "bundle" file introduced in DarkHelp v1.8. More...
 
size_t networks_loaded () const
 Get the number of worker threads which have loaded a copy of the neural network. More...
 
DHThreadspurge ()
 Removes all input files, waits for all worker threads to finish processing, clears out any results, and resets the image index (similar to reset_image_index()). More...
 
DHThreadsreset_image_index ()
 Reset to zero the image index used by add_image() to generate the image filename. More...
 
DHThreadsrestart ()
 Starts all of the processing threads. More...
 
DHThreadsstop ()
 Causes the threads to stop processing and exit. More...
 
ResultsMap wait_for_results ()
 Returns when there are zero input files remaining and all worker threads have finished processing images. More...
 

Public Attributes

std::atomic< bool > annotate_output_images
 Determines if annotated output images are created. More...
 
DarkHelp::Config cfg
 A copy of the configuration to use when instantiating each of the DarkHelp::NN objects. More...
 
std::atomic< bool > detele_input_file_after_processing
 Determines whether the input images are deleted once they are processed. More...
 

Private Member Functions

void run (const size_t id)
 The method that each worker thread runs to process images. More...
 

Private Attributes

std::atomic< size_t > files_processing
 The number of worker threads which are currently processing an image. More...
 
std::deque< std::string > input_files
 Used to keep track of all the input files remaining to be processed. More...
 
std::mutex input_image_and_file_lock
 Lock used to protect access to both input_files and input_images. More...
 
std::atomic< size_t > input_image_index
 Used by add_image() to generate an image filename. More...
 
std::map< std::string, cv::Mat > input_images
 Used to keep track of all the input images remaining to be processed. More...
 
std::vector< DarkHelp::NN * > networks
 Address to the worker thread neural networks. More...
 
std::filesystem::path output_dir
 The directory where the output will be saved. More...
 
std::atomic< bool > stop_requested
 If the threads need to stop, set this variable to true. More...
 
std::vector< std::thread > threads
 The threads that were started. More...
 
std::atomic< size_t > threads_ready
 Track the number of worker threads which have loaded the neural network. More...
 
size_t worker_threads_to_start
 The number of threads to start. More...
 
std::condition_variable trigger
 Used to signal the worker threads when more work becomes available. More...
 
std::mutex trigger_lock
 Used to signal the worker threads when more work becomes available. More...
 
ResultsMap all_results
 The prediction results for all the image file which have been processed. More...
 
std::mutex results_lock
 The prediction results for all the image file which have been processed. More...
 

Detailed Description

This class allows you to easily run multiple identical copies of a neural network to process many files at once.

If you have enough vram, or if you're running the CPU-only version of Darknet on a device with many cores, then you can use this class to load multiple copies of a neural network. Each copy will run on a different thread, and can process images independently of each other. This works especially well when the network is loaded once and re-used many times to process a large set of images.

For example, on a RTX 3050 with 4 GiB of vram, a YOLOv4-tiny network measuring 768x576 takes 29.4 seconds to process 1540 image files.

With this class, the same set of 1540 images only takes 8.8 seconds to process using 13 identical instances of the network loaded on the GPU.

(Each instance of the network consumes 289 MiB of vram, which is why 13 copies can be loaded at once on a GPU with 4 GiB of vram.)

Note this header file is not included by DarkHelp.hpp. To use this functionality you'll need to explicitely include this header file.

Since
2024-03-26

Member Typedef Documentation

◆ ResultsMap

The key is the input image filename, while the value contains a copy of the prediction results.

See also
DarkHelp::PredictionResults (a vector of results)
DarkHelp::PredictionResult (an individual result)
Since
2024-03-26

Constructor & Destructor Documentation

◆ DHThreads() [1/3]

DarkHelp::DHThreads::DHThreads ( )

Constructor.

No worker threads are started with this constructor. You'll need to manually call init().

Since
2024-03-26

◆ DHThreads() [2/3]

DarkHelp::DHThreads::DHThreads ( const DarkHelp::Config c,
const size_t  workers,
const std::filesystem::path &  output_directory = "." 
)

Constructor.

Parameters
[in]cThe DarkHelp configuration that each thread should use to load the neural network.
[in]workersThe number of worker threads to create. Each of these threads will also load a copy of the neural network, so the number you use here should depend on the amount of vram available and the neural network dimensions.
[in]output_directoryThe directory where the output files will be saved when annotate_output_images is enabled. If the directory does not exist, then it will be created. If it already exists, then it is left as-is, meaning existing files will remain.

The DHThreads constructor will automatically call init() to ensure all the threads and the neural networks are running.

Call add_image() or add_images() to get the threads to start processing images. Then call wait_for_results() or get_results().

Since
2024-03-26

References init().

Here is the call graph for this function:

◆ DHThreads() [3/3]

DarkHelp::DHThreads::DHThreads ( const std::filesystem::path &  filename,
const std::string &  key,
const size_t  workers,
const std::filesystem::path &  output_directory = ".",
const DarkHelp::EDriver driver = DarkHelp::EDriver::kDarknet 
)

Constructor.

This is similar to the other constructor, but uses the name of the "bundle" file introduced in DarkHelp v1.8. Run the CLI tool DarkHelpCombine to create a .dh bundle file.

Unlike the other DHThreads constructor, this one will wait until all the worker threads have fully loaded the neural network before returning to the caller. (This is required due to the use of the obfuscated bundle file.)

See also
DarkHelp::combine()
Since
2024-04-16

References init().

Here is the call graph for this function:

◆ ~DHThreads()

DarkHelp::DHThreads::~DHThreads ( )

Destructor. This will immediately stop all the worker threads and discard any results.

Member Function Documentation

◆ add_image()

std::string DarkHelp::DHThreads::add_image ( cv::Mat  image)

Add a single image to be processed.

Unlike add_images(), this does not require the image to be written to disk. The cv::Mat must be in standard OpenCV BGR format, and the lifetime of the image data must extend to when the image will be picked up and processed by one of the worker threads executing run().

Returns
A "virtual" filename will be created and returned to represent the OpenCV image. This filename is needed to correctly interpret the results from wait_for_results(). The filename generated contains a numerical value which is assigned in increasing sequential order, until one of purge(), restart() or reset_image_index() are called.
Note
Images added via add_image() will be processed before filenames added via add_images(). This is done to ensure that memory is freed up as quickly as possible (filenames barely take any memory).
See also
add_images()
reset_image_index()
Since
2024-04-01

◆ add_images()

DarkHelp::DHThreads & DarkHelp::DHThreads::add_images ( const std::filesystem::path &  dir)

Can be used to add a single image, or a subdirectory.

If a subdirectory, then recurse looking for all images. Call this as many times as necessary until all images have been added. Image processing by the worker threads will start immediately. Additional images can be added at any time, even while the worker threads have already started processing the first set of images.

Note
Images added via add_image() will be processed before filenames added via add_images(). This is done to ensure that memory is freed up as quickly as possible.
See also
add_image()
Since
2024-03-26

◆ files_remaining()

size_t DarkHelp::DHThreads::files_remaining ( ) const
inline

Get the number of images and files that have not yet been processed.

This is the number of unprocessed images and files combined, plus the images which are in the middle of being processed.

Since
2024-03-26

References files_processing, input_files, and input_images.

◆ get_nn()

DarkHelp::NN * DarkHelp::DHThreads::get_nn ( const size_t  idx)

Gain access to the neural network for the given worker thread.

This will return nullptr if the given worker thread has not loaded a neural network.

Since
2024-03-26

◆ get_results()

DarkHelp::DHThreads::ResultsMap DarkHelp::DHThreads::get_results ( )

Get all of the available prediction results.

You may call this as often you like to get intermediate results (if any are available), or you can indirectly call this via wait_for_results().

Note
Once the results are read and returned, the structure in which the results are stored internally is cleared.

The difference between wait_for_results() and get_results() is that wait_for_results() will wait until all the results are available, while get_results() will immediately return with whatever results are available, even if the results are empty.

Since
2024-03-26

◆ init() [1/2]

DarkHelp::DHThreads & DarkHelp::DHThreads::init ( const DarkHelp::Config c,
const size_t  workers,
const std::filesystem::path &  output_directory = "." 
)

Mostly for use with the default constructor.

Parameters are the same as the other constructor.

Since
2024-03-26

References DarkHelp::edit_cfg_file(), and DarkHelp::verify_cfg_and_weights().

Referenced by DHThreads().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ init() [2/2]

DarkHelp::DHThreads & DarkHelp::DHThreads::init ( const std::filesystem::path &  filename,
const std::string &  key,
const size_t  workers,
const std::filesystem::path &  output_directory = ".",
const DarkHelp::EDriver driver = DarkHelp::EDriver::kDarknet 
)

Similar to the other init() call, but uses the name of the "bundle" file introduced in DarkHelp v1.8.

Note
This will not delete the obfuscated bundle file from disk.
Since
2024-04-16

References DarkHelp::extract().

Here is the call graph for this function:

◆ networks_loaded()

size_t DarkHelp::DHThreads::networks_loaded ( ) const
inline

Get the number of worker threads which have loaded a copy of the neural network.

Since
2024-03-26

References threads_ready.

◆ purge()

DarkHelp::DHThreads & DarkHelp::DHThreads::purge ( )

Removes all input files, waits for all worker threads to finish processing, clears out any results, and resets the image index (similar to reset_image_index()).

Unlike the call to stop(), calling purge() does not terminate the worker threads or unload the neural networks, so you can immediately call add_image() or add_images() without needing to restart().

Since
2024-03-26

◆ reset_image_index()

DarkHelp::DHThreads & DarkHelp::DHThreads::reset_image_index ( )

Reset to zero the image index used by add_image() to generate the image filename.

Since
2024-04-01

◆ restart()

DarkHelp::DHThreads & DarkHelp::DHThreads::restart ( )

Starts all of the processing threads.

This is automatically called by init(), but may also be called manually if stop() was called. Calling restart() when the threads are already running will cause the existing threads to stop, input files to be cleared, and existing results to be reset.

Warning
If a "bundle" file was used when loading the neural network, then the .cfg, .names, and .weights files do not exist on disk and the call to restart() will fail since the neural network cannot be loaded.
See also
init()
Since
2024-03-26

References run().

Here is the call graph for this function:

◆ run()

void DarkHelp::DHThreads::run ( const size_t  id)
private

The method that each worker thread runs to process images.

See also
restart()

References DarkHelp::NN::annotate(), and DarkHelp::NN::predict().

Referenced by restart().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ stop()

DarkHelp::DHThreads & DarkHelp::DHThreads::stop ( )

Causes the threads to stop processing and exit.

All results and input files are cleared. Does not return until all threads have joined. Worker threads can be restarted by calling either init() or restart().

See also
purge()
Since
2024-03-26

◆ wait_for_results()

DarkHelp::DHThreads::ResultsMap DarkHelp::DHThreads::wait_for_results ( )

Returns when there are zero input files remaining and all worker threads have finished processing images.

Note this will clear out the results since it internally calls get_results().

The difference between wait_for_results() and get_results() is that wait_for_results() will wait until all the results are available, while get_results() will immediately return with whatever results are available at this point in time.

Since
2024-03-26

Member Data Documentation

◆ all_results

ResultsMap DarkHelp::DHThreads::all_results
private

The prediction results for all the image file which have been processed.

See also
get_results()

◆ annotate_output_images

std::atomic<bool> DarkHelp::DHThreads::annotate_output_images

Determines if annotated output images are created.

Default value is false.

Since
2024-03-26

◆ cfg

DarkHelp::Config DarkHelp::DHThreads::cfg

A copy of the configuration to use when instantiating each of the DarkHelp::NN objects.

This is only referenced by restart(). Meaning if you change cfg after the DHThreads() consructor or init(), you'll need to call stop() or restart().

Since
2024-03-26

◆ detele_input_file_after_processing

std::atomic<bool> DarkHelp::DHThreads::detele_input_file_after_processing

Determines whether the input images are deleted once they are processed.

Default value is false, meaning files are not deleted.

Since
2024-03-26

◆ files_processing

std::atomic<size_t> DarkHelp::DHThreads::files_processing
private

The number of worker threads which are currently processing an image.

Referenced by files_remaining().

◆ input_files

std::deque<std::string> DarkHelp::DHThreads::input_files
private

Used to keep track of all the input files remaining to be processed.

See also
add_image()
add_images()
input_image_and_file_lock

Referenced by files_remaining().

◆ input_image_and_file_lock

std::mutex DarkHelp::DHThreads::input_image_and_file_lock
private

Lock used to protect access to both input_files and input_images.

◆ input_image_index

std::atomic<size_t> DarkHelp::DHThreads::input_image_index
private

Used by add_image() to generate an image filename.

◆ input_images

std::map<std::string, cv::Mat> DarkHelp::DHThreads::input_images
private

Used to keep track of all the input images remaining to be processed.

See also
add_image()
add_images()
input_image_and_file_lock

Referenced by files_remaining().

◆ networks

std::vector<DarkHelp::NN *> DarkHelp::DHThreads::networks
private

Address to the worker thread neural networks.

See also
get_nn()

◆ output_dir

std::filesystem::path DarkHelp::DHThreads::output_dir
private

The directory where the output will be saved.

◆ results_lock

std::mutex DarkHelp::DHThreads::results_lock
private

The prediction results for all the image file which have been processed.

See also
get_results()

◆ stop_requested

std::atomic<bool> DarkHelp::DHThreads::stop_requested
private

If the threads need to stop, set this variable to true.

See also
stop()

◆ threads

std::vector<std::thread> DarkHelp::DHThreads::threads
private

The threads that were started.

See also
worker_threads_to_start

◆ threads_ready

std::atomic<size_t> DarkHelp::DHThreads::threads_ready
private

Track the number of worker threads which have loaded the neural network.

Referenced by networks_loaded().

◆ trigger

std::condition_variable DarkHelp::DHThreads::trigger
private

Used to signal the worker threads when more work becomes available.

◆ trigger_lock

std::mutex DarkHelp::DHThreads::trigger_lock
private

Used to signal the worker threads when more work becomes available.

◆ worker_threads_to_start

size_t DarkHelp::DHThreads::worker_threads_to_start
private

The number of threads to start.


The documentation for this class was generated from the following files: