DarkHelp  v1.4.30-1
C++ API for the neural network framework Darknet
Looking for a C++ dev who knows OpenCV?
I'm looking for work. Hire me!
DarkHelp::NN Class Referencefinal

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain. More...

Collaboration diagram for DarkHelp::NN:

Public Member Functions

 ~NN ()
 Destructor. This automatically calls reset() to release memory allocated by the neural network. More...
 
 NN ()
 Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called. More...
 
 NN (const Config &cfg)
 Constructor. More...
 
 NN (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet)
 Constructor. More...
 
NNinit (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet)
 Initialize ("load") the darknet neural network. More...
 
NNinit ()
 Initialize ("load") the darknet neural network. This uses the values within DarkHelp::NN::config. More...
 
NNreset ()
 The opposite of DarkHelp::NN::init(). More...
 
NNclear ()
 Clear out the images and the predictions stored internally within this object. More...
 
bool is_initialized () const
 Determines if a neural network has been loaded. More...
 
bool is_loaded () const
 Alias for is_initialized(). Returns true if the neural network has been loaded. More...
 
bool empty () const
 Only returns true if both original_image and prediction_results are both empty. More...
 
PredictionResults predict (const std::string &image_filename, const float new_threshold=-1.0f)
 Use the neural network to predict what is contained in this image. More...
 
PredictionResults predict (cv::Mat mat, const float new_threshold=-1.0f)
 Use the neural network to predict what is contained in this image. More...
 
PredictionResults predict_tile (cv::Mat mat, const float new_threshold=-1.0f)
 Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions. More...
 
cv::Mat annotate (const float new_threshold=-1.0f)
 Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image. More...
 
std::string duration_string ()
 Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation. More...
 
cv::Size network_size ()
 Determine the size of the network. For example, 416x416, or 608x480. More...
 
NNsnap_annotations ()
 Snap all the annotations. More...
 
NNsnap_annotation (PredictionResult &pred)
 Snap only the given annotation. More...
 
 NN (const NN &)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 
 NN (NN &&)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 
NNoperator= (const NN &)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 
NNoperator= (NN &&)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 

Static Public Member Functions

static std::string version ()
 Get a version string for the DarkHelp library. E.g., could be 1.0.0-123. More...
 

Public Attributes

void * darknet_net
 The Darknet network, but stored as a void* pointer so we don't have to include darknet.h. More...
 
VStr names
 A vector of names corresponding to the identified classes. More...
 
std::chrono::high_resolution_clock::duration duration
 The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed. More...
 
PredictionResults prediction_results
 A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict(). More...
 
cv::Mat original_image
 The most recent image handled by DarkHelp::NN::predict(). More...
 
cv::Mat annotated_image
 The most recent output produced by DarkHelp::NN::annotate(). More...
 
cv::Mat binary_inverted_image
 Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled. More...
 
size_t horizontal_tiles
 The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More...
 
size_t vertical_tiles
 The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More...
 
cv::Size tile_size
 The size that was used for each individual tile by DarkHelp::NN::predict_tile(). More...
 
Config config
 Configuratin for the neural network. More...
 

Protected Member Functions

PredictionResults predict_internal (cv::Mat mat, const float new_threshold=-1.0f)
 Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction. More...
 
void predict_internal_darknet ()
 Called from DarkHelp::NN::predict_internal(). More...
 
void predict_internal_opencv ()
 Called from DarkHelp::NN::predict_internal(). More...
 
NNname_prediction (PredictionResult &pred)
 Give a consistent name to the given production result. More...
 

Protected Attributes

cv::Size network_dimensions
 Size of the neural network, e.g., 416x416 or 608x608. More...
 

Detailed Description

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain.

For example:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
for (const auto & filename : image_filenames)
{
// these next two lines is where DarkHelp calls into Darknet to do all the hard work
nn.predict(filename);
cv::Mat mat = nn.annotate(); // annotates the most recent image seen by predict()
// use standard OpenCV calls to show the image results in a window
cv::imshow("prediction", mat);
cv::waitKey();
}

Instead of calling DarkHelp::NN::annotate(), you can get the detection results and iterate through them:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto results = nn.predict("test_image_01.jpg");
for (const auto & det : results)
{
std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}

Instead of writing your own loop, you can also use the std::ostream operator<<() like this:

const auto results = nn.predict("test_image_01.jpg");
std::cout << results << std::endl;

Constructor & Destructor Documentation

◆ NN() [1/5]

DarkHelp::NN::NN ( const NN )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ NN() [2/5]

DarkHelp::NN::NN ( NN &&  )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ ~NN()

DarkHelp::NN::~NN ( )

Destructor. This automatically calls reset() to release memory allocated by the neural network.

Here is the call graph for this function:

◆ NN() [3/5]

DarkHelp::NN::NN ( )

Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called.

Here is the call graph for this function:

◆ NN() [4/5]

DarkHelp::NN::NN ( const Config cfg)

Constructor.

This constructor automatically calls DarkHelp::NN::init() to load the neural network. The neural network filenames within the cfg object must not be empty otherwise the call to DarkHelp::NN::init() will fail.

Here is the call graph for this function:

◆ NN() [5/5]

DarkHelp::NN::NN ( const std::string &  cfg_filename,
const std::string &  weights_filename,
const std::string &  names_filename = "",
const bool  verify_files_first = true,
const EDriver  driver = EDriver::kDarknet 
)

Constructor.

This constructor automatically calls DarkHelp::NN::init() with the given parameters.

Note
The order in which you pass the various filenames is not important if verify_files_first is set to true (the default value). This is because DarkHelp::NN::init() will call DarkHelp::verify_cfg_and_weights() to correctly determine which is the .cfg, .weights, and .names file, and swap the names around as necessary so Darknet is given the correct filenames.
Here is the call graph for this function:

Member Function Documentation

◆ operator=() [1/2]

NN& DarkHelp::NN::operator= ( const NN )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ operator=() [2/2]

NN& DarkHelp::NN::operator= ( NN &&  )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ version()

static std::string DarkHelp::NN::version ( )
static

Get a version string for the DarkHelp library. E.g., could be 1.0.0-123.

◆ init() [1/2]

DarkHelp::NN & DarkHelp::NN::init ( const std::string &  cfg_filename,
const std::string &  weights_filename,
const std::string &  names_filename = "",
const bool  verify_files_first = true,
const EDriver  driver = EDriver::kDarknet 
)

Initialize ("load") the darknet neural network.

If verify_files_first has been enabled (the default) then this method will also call the static method DarkHelp::verify_cfg_and_weights() to perform some last-minute validation prior to darknet loading the neural network.

Here is the call graph for this function:

◆ init() [2/2]

DarkHelp::NN & DarkHelp::NN::init ( )

Initialize ("load") the darknet neural network. This uses the values within DarkHelp::NN::config.

Exceptions
std::invalid_argumentif the .cfg or .weights filenames have not been set.
std::runtime_errorif the call to darknet's load_network_custom() has failed.
std::invalid_argumentif there is a blank line in the .names file.
std::invalid_argumentif the network dimensions cannot be read from the .cfg file
Here is the call graph for this function:
Here is the caller graph for this function:

◆ reset()

DarkHelp::NN & DarkHelp::NN::reset ( )

The opposite of DarkHelp::NN::init().

This is automatically called by the destructor.

See also
clear()
Here is the caller graph for this function:

◆ clear()

DarkHelp::NN & DarkHelp::NN::clear ( )

Clear out the images and the predictions stored internally within this object.

This makes it seem as if the NN object has not yet been used to process any images. Unlike reset(), this does not change any settings, it only clears the images and the predictions. If a neural network has been loaded, calling clear() does not unload that neural network, and is_initialized() will continue to return true.

Calling this method between images is not necessary, but is included for completeness.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ is_initialized()

bool DarkHelp::NN::is_initialized ( ) const

Determines if a neural network has been loaded.

For example:

See also
empty()
clear()
Since
2022-07-12
Here is the caller graph for this function:

◆ is_loaded()

bool DarkHelp::NN::is_loaded ( ) const
inline

Alias for is_initialized(). Returns true if the neural network has been loaded.

Here is the call graph for this function:

◆ empty()

bool DarkHelp::NN::empty ( ) const

Only returns true if both original_image and prediction_results are both empty.

This will only happen in the following situations:

  • when a NN object has been instantiated but init() has not yet been called and no neural network is loaded
  • when a NN object is initialized but no image has been processed
  • after clear() has been called which results in the removal of internal images and predictions
See also
is_initialized()
clear()

◆ predict() [1/2]

DarkHelp::PredictionResults DarkHelp::NN::predict ( const std::string &  image_filename,
const float  new_threshold = -1.0f 
)

Use the neural network to predict what is contained in this image.

This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.

Parameters
[in]image_filenameThe name of the image file to load from disk and analyze. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio.
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.
Returns
A vector of DarkHelp::PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.
See also
Image Tiling
DarkHelp::NN::predict_tile()
DarkHelp::PredictionResult
DarkHelp::Config::enable_tiles
DarkHelp::Config::sort_predictions
DarkHelp::NN::duration
Exceptions
std::invalid_argumentif the image failed to load.

◆ predict() [2/2]

DarkHelp::PredictionResults DarkHelp::NN::predict ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)

Use the neural network to predict what is contained in this image.

This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.

Parameters
[in]matA OpenCV2 image which has already been loaded and which needs to be analyzed. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio.
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.
Returns
A vector of DarkHelp::PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.
See also
Image Tiling
DarkHelp::NN::predict_tile()
DarkHelp::PredictionResult
DarkHelp::Config::enable_tiles
DarkHelp::Config::sort_predictions
DarkHelp::NN::duration
Exceptions
std::invalid_argumentif the image is empty.

◆ predict_tile()

DarkHelp::PredictionResults DarkHelp::NN::predict_tile ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)

Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions.

This is explained in details in Image Tiling.

Note
The method DarkHelp::NN::predict() will automatically call DarkHelp::NN::predict_tile() if necessary when DarkHelp::Config::enable_tiles has been enabled. If you don't want to use image tiling, then DarkHelp::Config::enable_tiles must be set to false (which is the default).

Here is a visual representation of a large image broken into 4 tiles for processing by Darknet. It is important to understand that neither the individual image tiles nor their results are returned to the caller. DarkHelp only returns the final results once each tile has been processed and the vectors have been merged together.

See also
Image Tiling
DarkHelp::NN::predict()
Exceptions
std::invalid_argumentif the image is empty.

◆ annotate()

cv::Mat DarkHelp::NN::annotate ( const float  new_threshold = -1.0f)

Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image.

The output annotated image is stored in DarkHelp::NN::annotated_image as well as returned to the caller.

This is an example of what an annotated image looks like:

Parameters
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value.

Turning down the threshold in DarkHelp::NN::annotate() wont bring back predictions that were excluded due to a higher threshold originally used with DarkHelp::NN::predict(). Here is an example:

nn.predict("image.jpg", 0.75); // note the threshold is set to 75% for prediction
nn.annotate(0.25); // note the threshold is now modified to be 25%

In the previous example, when annotate() is called with the lower threshold of 25%, the predictions had already been capped at 75%. This means any prediction between >= 25% and < 75% were excluded from the prediction results. The only way to get those predictions is to re-run predict() with a value of 0.25.

Note
Annotations wont be drawn if DarkHelp::Config::annotation_line_thickness is less than 1.
See also
DarkHelp::Config::annotation_colours
DarkHelp::Config::annotation_font_scale
DarkHelp::Config::annotation_font_thickness
DarkHelp::Config::annotation_line_thickness
DarkHelp::Config::annotation_include_duration
DarkHelp::Config::annotation_include_timestamp
DarkHelp::Config::annotation_auto_hide_labels
DarkHelp::Config::annotation_shade_predictions
DarkHelp::Config::annotation_suppress_classes
Exceptions
std::logic_errorif an attempt is made to annotate an empty image
Here is the call graph for this function:

◆ duration_string()

std::string DarkHelp::NN::duration_string ( )

Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation.

For example, this might return "912 microseconds" or "375 milliseconds".

See also
DarkHelp::NN::annotate()
Here is the call graph for this function:

◆ network_size()

cv::Size DarkHelp::NN::network_size ( )

Determine the size of the network. For example, 416x416, or 608x480.

◆ snap_annotations()

DarkHelp::NN & DarkHelp::NN::snap_annotations ( )

Snap all the annotations.

This is automatically called from predict() when DarkHelp::Config::snapping_enabled is set to true. When set to false, you can manually invoke this method to get the annotations to snap, or you can also manually call DarkHelp::NN::snap_annotation() on specific annotations as needed.

Note
This can be expensive to run depending on the image dimensions, the image threshold limits, and the amount of "snapping" required for each annotation since the process of "snapping" is iterative and requires looking for blank spaces within the image.
See also
DarkHelp::Config::snapping_enabled
DarkHelp::Config::snapping_horizontal_tolerance
Image Setting
snapping_enabled=false
snapping_enabled=true

◆ snap_annotation()

◆ predict_internal()

DarkHelp::PredictionResults DarkHelp::NN::predict_internal ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)
protected

Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction.

This uses the image stored in DarkHelp::NN::original_image.

Exceptions
std::logic_errorif the DarkHelp object has not been initialized.
std::logic_errorif the network is invalid.
std::logic_errorif the image is invalid.

◆ predict_internal_darknet()

void DarkHelp::NN::predict_internal_darknet ( )
protected

Called from DarkHelp::NN::predict_internal().

See also
DarkHelp::NN::predict()
Here is the call graph for this function:

◆ predict_internal_opencv()

void DarkHelp::NN::predict_internal_opencv ( )
protected

Called from DarkHelp::NN::predict_internal().

See also
DarkHelp::NN::predict()
Here is the call graph for this function:

◆ name_prediction()

DarkHelp::NN & DarkHelp::NN::name_prediction ( PredictionResult pred)
protected

Give a consistent name to the given production result.

This gets called by both DarkHelp::NN::predict_internal() and DarkHelp::NN::predict_tile() and is intended for internal use only.

Member Data Documentation

◆ darknet_net

void* DarkHelp::NN::darknet_net

The Darknet network, but stored as a void* pointer so we don't have to include darknet.h.

This will only be set when the driver is DarkHelp::EDriver::kDarknet in DarkHelp::NN::init().

◆ names

VStr DarkHelp::NN::names

A vector of names corresponding to the identified classes.

This is typically setup in the constructor, but can be manually set afterwards.

◆ duration

std::chrono::high_resolution_clock::duration DarkHelp::NN::duration

The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed.

If using DarkHelp::NN::predict_tile(), then this will store the sum of all durations across the entire set of tiles.

See also
DarkHelp::NN::duration_string()

◆ prediction_results

PredictionResults DarkHelp::NN::prediction_results

A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict().

◆ original_image

cv::Mat DarkHelp::NN::original_image

The most recent image handled by DarkHelp::NN::predict().

◆ annotated_image

cv::Mat DarkHelp::NN::annotated_image

The most recent output produced by DarkHelp::NN::annotate().

◆ binary_inverted_image

cv::Mat DarkHelp::NN::binary_inverted_image

Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled.

◆ horizontal_tiles

size_t DarkHelp::NN::horizontal_tiles

The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().

This is set to 1 if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.

See also
DarkHelp::NN::vertical_tiles
Image Tiling
DarkHelp::Config::enable_tiles

◆ vertical_tiles

size_t DarkHelp::NN::vertical_tiles

The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().

This is set to 1 if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.

See also
DarkHelp::NN::horizontal_tiles
Image Tiling
DarkHelp::Config::enable_tiles

◆ tile_size

cv::Size DarkHelp::NN::tile_size

The size that was used for each individual tile by DarkHelp::NN::predict_tile().

This will be the size of the network when calling DarkHelp::NN::predict().

For example, if the network is 416x416, and the image used with DarkHelp::NN::predict_tile() measures 1280x960, then:

See also
Image Tiling
DarkHelp::Config::enable_tiles

◆ config

Config DarkHelp::NN::config

Configuratin for the neural network.

This includes both settings for the neural network itself and everything needed to annotate images/frames.

◆ network_dimensions

cv::Size DarkHelp::NN::network_dimensions
protected

Size of the neural network, e.g., 416x416 or 608x608.

See also
DarkHelp::NN::network_size()

The documentation for this class was generated from the following files:
DarkHelp::NN::predict
PredictionResults predict(const std::string &image_filename, const float new_threshold=-1.0f)
Use the neural network to predict what is contained in this image.
Definition: DarkHelpNN.cpp:365
DarkHelp::NN
Instantiate one of these objects by giving it the name of the .cfg and .weights file,...
Definition: DarkHelpNN.hpp:60
DarkHelp::NN::annotate
cv::Mat annotate(const float new_threshold=-1.0f)
Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp:...
Definition: DarkHelpNN.cpp:666