DarkHelp  v1.7.11-3
C++ API for the neural network framework Darknet
Looking for a C++ dev who knows OpenCV?
I'm looking for work. Hire me!
DarkHelp::NN Class Referencefinal

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain. More...

#include "DarkHelpNN.hpp"

Collaboration diagram for DarkHelp::NN:

Public Member Functions

 NN ()
 Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called. More...
 
 NN (const Config &cfg)
 Constructor. More...
 
 NN (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet)
 Constructor. More...
 
 ~NN ()
 Destructor. This automatically calls reset() to release memory allocated by the neural network. More...
 
cv::Mat annotate (const float new_threshold=-1.0f)
 Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image. More...
 
NNclear ()
 Clear out the images and the predictions stored internally within this object. More...
 
std::string duration_string ()
 Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation. More...
 
bool empty () const
 Only returns true if both original_image and prediction_results are both empty. More...
 
int image_channels ()
 Return the number of channels defined in the .cfg file. Usually, this will be 3. More...
 
NNinit ()
 Initialize ("load") the darknet neural network. More...
 
NNinit (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet)
 Initialize ("load") the darknet neural network. More...
 
bool is_initialized () const
 Determines if a neural network has been loaded. More...
 
bool is_loaded () const
 Alias for is_initialized(). Returns true if the neural network has been loaded. More...
 
cv::Size network_size ()
 Determine the size of the network. For example, 416x416, or 608x480. More...
 
PredictionResults predict (const std::string &image_filename, const float new_threshold=-1.0f)
 Use the neural network to predict what is contained in this image. More...
 
PredictionResults predict (cv::Mat mat, const float new_threshold=-1.0f)
 Use the neural network to predict what is contained in this image. More...
 
PredictionResults predict_tile (cv::Mat mat, const float new_threshold=-1.0f)
 Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions. More...
 
NNreset ()
 The opposite of DarkHelp::NN::init(). More...
 
NNsnap_annotation (PredictionResult &pred)
 Snap only the given annotation. More...
 
NNsnap_annotations ()
 Snap all the annotations. More...
 
 NN (const NN &)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 
 NN (NN &&)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 
NNoperator= (const NN &)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 
NNoperator= (NN &&)=delete
 We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...
 

Static Public Member Functions

static std::string version ()
 Get a version string for the DarkHelp library. E.g., could be 1.5.13-1. More...
 

Public Attributes

cv::Mat annotated_image
 The most recent output produced by DarkHelp::NN::annotate(). More...
 
cv::Mat binary_inverted_image
 Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled. More...
 
Config config
 Configuratin for the neural network. More...
 
void * darknet_net
 The Darknet network, but stored as a void* pointer so we don't have to include darknet.h. More...
 
std::chrono::high_resolution_clock::duration duration
 The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed. More...
 
size_t horizontal_tiles
 The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More...
 
VStr names
 A vector of names corresponding to the identified classes. More...
 
cv::Mat original_image
 The most recent image handled by DarkHelp::NN::predict(). More...
 
PredictionResults prediction_results
 A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict(). More...
 
cv::Size tile_size
 The size that was used for each individual tile by DarkHelp::NN::predict_tile(). More...
 
size_t vertical_tiles
 The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More...
 

Protected Member Functions

NNname_prediction (PredictionResult &pred)
 Give a consistent name to the given production result. More...
 
PredictionResults predict_internal (cv::Mat mat, const float new_threshold=-1.0f)
 Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction. More...
 
void predict_internal_darknet ()
 Called from DarkHelp::NN::predict_internal(). More...
 
void predict_internal_opencv ()
 Called from DarkHelp::NN::predict_internal(). More...
 

Protected Attributes

cv::Size network_dimensions
 Size of the neural network, e.g., 416x416 or 608x608. More...
 
int number_of_channels
 The number of channels defined in the .cfg file. This is normally set to 3. More...
 

Detailed Description

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain.

For example:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
for (const auto & filename : image_filenames)
{
// these next two lines is where DarkHelp calls into Darknet to do all the hard work
nn.predict(filename);
cv::Mat mat = nn.annotate(); // annotates the most recent image seen by predict()
// use standard OpenCV calls to show the image results in a window
cv::imshow("prediction", mat);
cv::waitKey();
}
Instantiate one of these objects by giving it the name of the .cfg and .weights file,...
Definition: DarkHelpNN.hpp:61

Instead of calling DarkHelp::NN::annotate(), you can get the detection results and iterate through them:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto results = nn.predict("test_image_01.jpg");
for (const auto & det : results)
{
std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}

Instead of writing your own loop, you can also use the std::ostream operator<<() like this:

const auto results = nn.predict("test_image_01.jpg");
std::cout << results << std::endl;

Constructor & Destructor Documentation

◆ NN() [1/5]

DarkHelp::NN::NN ( const NN )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ NN() [2/5]

DarkHelp::NN::NN ( NN &&  )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ ~NN()

DarkHelp::NN::~NN ( )

Destructor. This automatically calls reset() to release memory allocated by the neural network.

References reset().

Here is the call graph for this function:

◆ NN() [3/5]

DarkHelp::NN::NN ( )

Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called.

References reset().

Here is the call graph for this function:

◆ NN() [4/5]

DarkHelp::NN::NN ( const Config cfg)

Constructor.

This constructor automatically calls DarkHelp::NN::init() to load the neural network. The neural network filenames within the cfg object must not be empty otherwise the call to DarkHelp::NN::init() will fail.

References config, and init().

Here is the call graph for this function:

◆ NN() [5/5]

DarkHelp::NN::NN ( const std::string &  cfg_filename,
const std::string &  weights_filename,
const std::string &  names_filename = "",
const bool  verify_files_first = true,
const EDriver  driver = EDriver::kDarknet 
)

Constructor.

This constructor automatically calls DarkHelp::NN::init() with the given parameters.

Note
The order in which you pass the various filenames is not important if verify_files_first is set to true (the default value). This is because DarkHelp::NN::init() will call DarkHelp::verify_cfg_and_weights() to correctly determine which is the .cfg, .weights, and .names file, and swap the names around as necessary so Darknet is given the correct filenames.

References init().

Here is the call graph for this function:

Member Function Documentation

◆ annotate()

cv::Mat DarkHelp::NN::annotate ( const float  new_threshold = -1.0f)

Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image.

The output annotated image is stored in DarkHelp::NN::annotated_image as well as returned to the caller.

This is an example of what an annotated image looks like:

Parameters
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value.

Turning down the threshold in DarkHelp::NN::annotate() wont bring back predictions that were excluded due to a higher threshold originally used with DarkHelp::NN::predict(). Here is an example:

nn.predict("image.jpg", 0.75); // note the threshold is set to 75% for prediction
nn.annotate(0.25); // note the threshold is now modified to be 25%

In the previous example, when annotate() is called with the lower threshold of 25%, the predictions had already been capped at 75%. This means any prediction between >= 25% and < 75% were excluded from the prediction results. The only way to get those predictions is to re-run predict() with a value of 0.25.

Note
Annotations wont be drawn if DarkHelp::Config::annotation_line_thickness is less than 1.
See also
DarkHelp::Config::annotation_colours
DarkHelp::Config::annotation_font_scale
DarkHelp::Config::annotation_font_thickness
DarkHelp::Config::annotation_line_thickness
DarkHelp::Config::annotation_include_duration
DarkHelp::Config::annotation_include_timestamp
DarkHelp::Config::annotation_auto_hide_labels
DarkHelp::Config::annotation_shade_predictions
DarkHelp::Config::annotation_suppress_classes
Exceptions
std::logic_errorif an attempt is made to annotate an empty image

References DarkHelp::duration_string(), DarkHelp::get_default_annotation_colours(), and DarkHelp::pixelate_rectangles().

Referenced by Annotate(), and DarkHelp::DHThreads::run().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ clear()

DarkHelp::NN & DarkHelp::NN::clear ( )

Clear out the images and the predictions stored internally within this object.

This makes it seem as if the NN object has not yet been used to process any images. Unlike reset(), this does not change any configuration settings, it only clears the images and the predictions. If a neural network has been loaded, calling clear() does not unload that neural network, and is_initialized() will continue to return true.

Calling this method between images is not necessary, but is included for completeness.

References clear().

Referenced by clear().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ duration_string()

std::string DarkHelp::NN::duration_string ( )

Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation.

For example, this might return "912 microseconds" or "375 milliseconds".

See also
DarkHelp::NN::annotate()

References DarkHelp::duration_string().

Referenced by GetPredictionResults().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ empty()

bool DarkHelp::NN::empty ( ) const

Only returns true if both original_image and prediction_results are both empty.

This will only happen in the following situations:

  • when a NN object has been instantiated but init() has not yet been called and no neural network is loaded
  • when a NN object is initialized but no image has been processed
  • after clear() has been called which results in the removal of internal images and predictions
See also
is_initialized()
clear()

◆ image_channels()

int DarkHelp::NN::image_channels ( )

Return the number of channels defined in the .cfg file. Usually, this will be 3.

◆ init() [1/2]

DarkHelp::NN & DarkHelp::NN::init ( )

Initialize ("load") the darknet neural network.

This uses the values within DarkHelp::NN::config and is called automatically if the network files have been specified to the constructor. You only need to manually call init() if the default constructor without filenames is used.

Exceptions
std::invalid_argumentif the .cfg or .weights filenames have not been set.
std::runtime_errorif the call to darknet's load_network_custom() has failed.
std::invalid_argumentif there is a blank line in the .names file.
std::invalid_argumentif the network dimensions cannot be read from the .cfg file
std::invalid_argumentif the channels=... line in the .cfg file is not 1 or 3

References DarkHelp::edit_cfg_file(), DarkHelp::kDarknet, DarkHelp::kMax, DarkHelp::kMin, DarkHelp::kOpenCVCPU, and DarkHelp::toggle_output_redirection().

Referenced by NN().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ init() [2/2]

DarkHelp::NN & DarkHelp::NN::init ( const std::string &  cfg_filename,
const std::string &  weights_filename,
const std::string &  names_filename = "",
const bool  verify_files_first = true,
const EDriver  driver = EDriver::kDarknet 
)

Initialize ("load") the darknet neural network.

If verify_files_first has been enabled (the default) then this method will also call the static method DarkHelp::verify_cfg_and_weights() to perform some last-minute validation prior to darknet loading the neural network.

References DarkHelp::kDarknet, and DarkHelp::verify_cfg_and_weights().

Here is the call graph for this function:

◆ is_initialized()

bool DarkHelp::NN::is_initialized ( ) const

Determines if a neural network has been loaded.

For example:

See also
empty()
clear()
Since
2022-07-12

References DarkHelp::kDarknet.

Referenced by is_loaded().

Here is the caller graph for this function:

◆ is_loaded()

bool DarkHelp::NN::is_loaded ( ) const
inline

Alias for is_initialized(). Returns true if the neural network has been loaded.

References is_initialized().

Here is the call graph for this function:

◆ name_prediction()

DarkHelp::NN & DarkHelp::NN::name_prediction ( PredictionResult pred)
protected

Give a consistent name to the given production result.

This gets called by both DarkHelp::NN::predict_internal() and DarkHelp::NN::predict_tile() and is intended for internal use only.

References DarkHelp::PredictionResult::all_probabilities, DarkHelp::PredictionResult::best_class, DarkHelp::PredictionResult::best_probability, and DarkHelp::PredictionResult::name.

◆ network_size()

cv::Size DarkHelp::NN::network_size ( )

Determine the size of the network. For example, 416x416, or 608x480.

Referenced by GetPredictionResults().

Here is the caller graph for this function:

◆ operator=() [1/2]

NN& DarkHelp::NN::operator= ( const NN )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ operator=() [2/2]

NN& DarkHelp::NN::operator= ( NN &&  )
delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ predict() [1/2]

DarkHelp::PredictionResults DarkHelp::NN::predict ( const std::string &  image_filename,
const float  new_threshold = -1.0f 
)

Use the neural network to predict what is contained in this image.

This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.

Parameters
[in]image_filenameThe name of the image file to load from disk and analyze. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio.
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.
Returns
A vector of DarkHelp::PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.
See also
Image Tiling
DarkHelp::NN::predict_tile()
DarkHelp::PredictionResult
DarkHelp::Config::enable_tiles
DarkHelp::Config::sort_predictions
DarkHelp::NN::duration
Exceptions
std::invalid_argumentif the image failed to load.

Referenced by Predict(), PredictFN(), and DarkHelp::DHThreads::run().

Here is the caller graph for this function:

◆ predict() [2/2]

DarkHelp::PredictionResults DarkHelp::NN::predict ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)

Use the neural network to predict what is contained in this image.

This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.

Parameters
[in]matA OpenCV2 image which has already been loaded and which needs to be analyzed. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio.
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.
Returns
A vector of DarkHelp::PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.
See also
Image Tiling
DarkHelp::NN::predict_tile()
DarkHelp::PredictionResult
DarkHelp::Config::enable_tiles
DarkHelp::Config::sort_predictions
DarkHelp::NN::duration
Exceptions
std::invalid_argumentif the image is empty.

◆ predict_internal()

DarkHelp::PredictionResults DarkHelp::NN::predict_internal ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)
protected

Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction.

This uses the image stored in DarkHelp::NN::original_image.

Exceptions
std::logic_errorif the DarkHelp object has not been initialized.
std::logic_errorif the network is invalid.
std::logic_errorif the image is invalid.

References DarkHelp::kAscending, DarkHelp::kDarknet, DarkHelp::kDescending, DarkHelp::kInvalid, and DarkHelp::kPageOrder.

◆ predict_internal_darknet()

◆ predict_internal_opencv()

void DarkHelp::NN::predict_internal_opencv ( )
protected

◆ predict_tile()

DarkHelp::PredictionResults DarkHelp::NN::predict_tile ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)

Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions.

This is explained in details in Image Tiling.

Note
The method DarkHelp::NN::predict() will automatically call DarkHelp::NN::predict_tile() if necessary when DarkHelp::Config::enable_tiles has been enabled. If you don't want to use image tiling, then DarkHelp::Config::enable_tiles must be set to false (which is the default).

Here is a visual representation of a large image broken into 4 tiles for processing by Darknet. It is important to understand that neither the individual image tiles nor their results are returned to the caller. DarkHelp only returns the final results once each tile has been processed and the vectors have been merged together.

See also
Image Tiling
DarkHelp::NN::predict()
Exceptions
std::invalid_argumentif the image is empty.

◆ reset()

DarkHelp::NN & DarkHelp::NN::reset ( )

The opposite of DarkHelp::NN::init().

This is automatically called by the destructor.

See also
clear()

Referenced by NN(), and ~NN().

Here is the caller graph for this function:

◆ snap_annotation()

◆ snap_annotations()

DarkHelp::NN & DarkHelp::NN::snap_annotations ( )

Snap all the annotations.

This is automatically called from predict() when DarkHelp::Config::snapping_enabled is set to true. When set to false, you can manually invoke this method to get the annotations to snap, or you can also manually call DarkHelp::NN::snap_annotation() on specific annotations as needed.

Note
This can be expensive to run depending on the image dimensions, the image threshold limits, and the amount of "snapping" required for each annotation since the process of "snapping" is iterative and requires looking for blank spaces within the image.
See also
DarkHelp::Config::snapping_enabled
DarkHelp::Config::snapping_horizontal_tolerance
Image Setting
snapping_enabled=false
snapping_enabled=true

◆ version()

static std::string DarkHelp::NN::version ( )
static

Get a version string for the DarkHelp library. E.g., could be 1.5.13-1.

Member Data Documentation

◆ annotated_image

cv::Mat DarkHelp::NN::annotated_image

The most recent output produced by DarkHelp::NN::annotate().

◆ binary_inverted_image

cv::Mat DarkHelp::NN::binary_inverted_image

Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled.

◆ config

◆ darknet_net

void* DarkHelp::NN::darknet_net

The Darknet network, but stored as a void* pointer so we don't have to include darknet.h.

This will only be set when the driver is DarkHelp::EDriver::kDarknet in DarkHelp::NN::init().

◆ duration

std::chrono::high_resolution_clock::duration DarkHelp::NN::duration

The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed.

If using DarkHelp::NN::predict_tile(), then this will store the sum of all durations across the entire set of tiles.

See also
DarkHelp::NN::duration_string()

◆ horizontal_tiles

size_t DarkHelp::NN::horizontal_tiles

The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().

This is set to 1 if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.

See also
DarkHelp::NN::vertical_tiles
Image Tiling
DarkHelp::Config::enable_tiles

Referenced by GetPredictionResults().

◆ names

VStr DarkHelp::NN::names

A vector of names corresponding to the identified classes.

This is typically setup in the constructor, but can be manually set afterwards.

Referenced by GetPredictionResults().

◆ network_dimensions

cv::Size DarkHelp::NN::network_dimensions
protected

Size of the neural network, e.g., 416x416 or 608x608.

See also
DarkHelp::NN::network_size()

◆ number_of_channels

int DarkHelp::NN::number_of_channels
protected

The number of channels defined in the .cfg file. This is normally set to 3.

See also
image_channels()

◆ original_image

cv::Mat DarkHelp::NN::original_image

The most recent image handled by DarkHelp::NN::predict().

Referenced by GetPredictionResults().

◆ prediction_results

PredictionResults DarkHelp::NN::prediction_results

A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict().

Referenced by GetPredictionResults(), Predict(), and PredictFN().

◆ tile_size

cv::Size DarkHelp::NN::tile_size

The size that was used for each individual tile by DarkHelp::NN::predict_tile().

This will be the size of the network when calling DarkHelp::NN::predict().

For example, if the network is 416x416, and the image used with DarkHelp::NN::predict_tile() measures 1280x960, then:

See also
Image Tiling
DarkHelp::Config::enable_tiles

Referenced by GetPredictionResults().

◆ vertical_tiles

size_t DarkHelp::NN::vertical_tiles

The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().

This is set to 1 if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.

See also
DarkHelp::NN::horizontal_tiles
Image Tiling
DarkHelp::Config::enable_tiles

Referenced by GetPredictionResults().


The documentation for this class was generated from the following files: