DarkHelp  v1.0.0-3003
C++ API for the neural network framework Darknet
Looking for a C++ dev who knows OpenCV?
I'm looking for work. Hire me!
DarkHelp Class Reference

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call predict() as often as necessary to determine what the images contain. More...

Collaboration diagram for DarkHelp:

Classes

struct  PredictionResult
 Structure used to store interesting information on predictions. More...
 

Public Types

enum  ESort {
  ESort::kUnsorted,
  ESort::kAscending,
  ESort::kDescending
}
 
typedef std::map< std::string, std::string > MStr
 Map of strings where both the key and the value are std::string. More...
 
typedef std::vector< std::string > VStr
 Vector of text strings. Typically used to store the class names. More...
 
typedef std::vector< cv::Scalar > VColours
 Vector of colours to use by annotate(). More...
 
typedef std::map< int, float > MClassProbabilities
 Map of a class ID to a probability that this object belongs to that class. More...
 
typedef std::vector< PredictionResultPredictionResults
 A vector of predictions for the image analyzed by predict(). More...
 

Public Member Functions

virtual ~DarkHelp ()
 Destructor. This automatically calls reset() to release memory allocated by the neural network. More...
 
 DarkHelp ()
 Constructor. When using this constructor, the neural network remains uninitialized until init() is called. More...
 
 DarkHelp (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true)
 Constructor. More...
 
virtual std::string version () const
 Get a version string for the DarkHelp library. E.g., could be 1.0.0-123. More...
 
virtual DarkHelpinit (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true)
 Initialize ("load") the darknet neural network. More...
 
virtual void reset ()
 The opposite of init(). This is automatically called by the destructor. More...
 
virtual PredictionResults predict (const std::string &image_filename, const float new_threshold=-1.0f)
 Use the neural network to predict what is contained in this image. More...
 
virtual PredictionResults predict (cv::Mat mat, const float new_threshold=-1.0f)
 Use the neural network to predict what is contained in this image. More...
 
virtual cv::Mat annotate (const float new_threshold=-1.0f)
 Takes the most recent prediction_results, and applies them to the most recent original_image. More...
 
virtual std::string duration_string ()
 Return the duration as a text string which can then be added to the image during annotation. More...
 

Static Public Member Functions

static VColours get_default_annotation_colours ()
 Obtain a vector of several bright colours that may be used to annotate images. More...
 
static MStr verify_cfg_and_weights (std::string &cfg_filename, std::string &weights_filename, std::string &names_filename)
 Look at the names and/or the contents of all 3 files and swap the filenames around if necessary so the .cfg, .weights, and .names are assigned where they should be. More...
 

Public Attributes

void * net
 The Darknet network, but stored as a void* pointer so we don't have to include darknet.h. More...
 
VStr names
 A vector of names corresponding to the identified classes. More...
 
std::chrono::high_resolution_clock::duration duration
 The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time predict() took to run on the last image to be processed. More...
 
float threshold
 Image prediction threshold. More...
 
float hierarchy_threshold
 Used during prediction. More...
 
float non_maximal_suppression_threshold
 Non-Maximal Suppression (NMS) threshold suppresses overlapping bounding boxes and only retains the bounding box that has the maximum probability of object detection associated with it. More...
 
PredictionResults prediction_results
 A copy of the most recent results after applying the neural network to an image. This is set by predict(). More...
 
bool names_include_percentage
 Determines if the name given to each prediction includes the percentage. More...
 
bool include_all_names
 Determine if multiple class names are included when labelling an item. More...
 
VColours annotation_colours
 The colours to use in annotate(). More...
 
cv::HersheyFonts annotation_font_face
 Font face to use in annotate(). Defaults to cv::HersheyFonts::FONT_HERSHEY_SIMPLEX. More...
 
double annotation_font_scale
 Scaling factor used for the font in annotate(). Defaults to 0.5. More...
 
int annotation_font_thickness
 Thickness of the font in annotate(). Defaults to 1. More...
 
int annotation_line_thickness
 Thickness of the lines to draw in annotate(). Defaults to 2. More...
 
bool annotation_include_duration
 If set to true then annotate() will call duration_string() and display on the top-left of the image the length of time predict() took to process the image. More...
 
bool annotation_include_timestamp
 If set to true then annotate() will display a timestamp on the bottom-left corner of the image. More...
 
bool fix_out_of_bound_values
 Darknet sometimes will return values that are out-of-bound, especially when working with low thresholds. More...
 
cv::Mat original_image
 The most recent image handled by predict(). More...
 
cv::Mat annotated_image
 The most recent output produced by annotate(). More...
 
ESort sort_predictions
 Determines if the predictions will be sorted the next time predict() is called. More...
 

Protected Member Functions

PredictionResults predict (const float new_threshold=-1.0f)
 Used by all the other predict() calls to do the actual network prediction. More...
 

Detailed Description

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call predict() as often as necessary to determine what the images contain.

For example:

DarkHelp darkhelp("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
for (const auto & filename : image_filenames)
{
// these next two lines is where DarkHelp calls into Darknet to do all the hard work
darkhelp.predict(filename);
cv::Mat mat = darkhelp.annotate(); // annotates the most recent image seen by predict()
// use standard OpenCV calls to show the image results in a window
cv::imshow("prediction", mat);
cv::waitKey();
}

Instead of calling annotate(), you can get the detection results and iterate through them:

DarkHelp darkhelp("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto results = darkhelp.predict("test_image_01.jpg");
for (const auto & det : results)
{
std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}

Instead of writing your own loop, you can also use the std::ostream operator<<() like this:

const auto results = darkhelp.predict("test_image_01.jpg");
std::cout << results << std::endl;

Member Typedef Documentation

◆ MStr

typedef std::map<std::string, std::string> DarkHelp::MStr

Map of strings where both the key and the value are std::string.

◆ VStr

typedef std::vector<std::string> DarkHelp::VStr

Vector of text strings. Typically used to store the class names.

◆ VColours

typedef std::vector<cv::Scalar> DarkHelp::VColours

Vector of colours to use by annotate().

See also
annotation_colours
get_default_annotation_colours()

◆ MClassProbabilities

typedef std::map<int, float> DarkHelp::MClassProbabilities

Map of a class ID to a probability that this object belongs to that class.

The key is the zero-based index of the class, while the value is the probability that the object belongs to that class.

See also
PredictionResult::all_probabilities

◆ PredictionResults

A vector of predictions for the image analyzed by predict().

Each PredictionResult entry in the vector represents a different object in the image.

See also
PredictionResult
prediction_results
sort_predictions

Member Enumeration Documentation

◆ ESort

enum DarkHelp::ESort
strong
See also
sort_predictions
Enumerator
kUnsorted 

Do not sort predictions.

kAscending 

Sort predictions using PredictionResult::best_probability in ascending order (low values first, high values last).

kDescending 

Sort predictions using PredictionResult::best_probability in descending order (high values first, low values last).

Constructor & Destructor Documentation

◆ ~DarkHelp()

DarkHelp::~DarkHelp ( )
virtual

Destructor. This automatically calls reset() to release memory allocated by the neural network.

Here is the call graph for this function:

◆ DarkHelp() [1/2]

DarkHelp::DarkHelp ( )

Constructor. When using this constructor, the neural network remains uninitialized until init() is called.

Here is the call graph for this function:

◆ DarkHelp() [2/2]

DarkHelp::DarkHelp ( const std::string &  cfg_filename,
const std::string &  weights_filename,
const std::string &  names_filename = "",
const bool  verify_files_first = true 
)

Constructor.

This constructor automatically calls init().

Note
The order in which you pass the various filenames is not important if verify_files_first is set to true (the default value). This is because init() will call verify_cfg_and_weights() to correctly determine which is the .cfg, .weights, and .names file, and swap the names around as necessary so Darknet is given the correct filenames.
Here is the call graph for this function:

Member Function Documentation

◆ version()

std::string DarkHelp::version ( ) const
virtual

Get a version string for the DarkHelp library. E.g., could be 1.0.0-123.

◆ init()

DarkHelp & DarkHelp::init ( const std::string &  cfg_filename,
const std::string &  weights_filename,
const std::string &  names_filename = "",
const bool  verify_files_first = true 
)
virtual

Initialize ("load") the darknet neural network.

If verify_files_first has been enabled (the default) then this method will also call the static method verify_cfg_and_weights() to perform some last-minute validation prior to darknet loading the neural network.

Exceptions
std::runtime_errorif the call to darknet's load_network_custom() has failed.
Here is the call graph for this function:
Here is the caller graph for this function:

◆ reset()

void DarkHelp::reset ( )
virtual

The opposite of init(). This is automatically called by the destructor.

Here is the call graph for this function:
Here is the caller graph for this function:

◆ predict() [1/3]

DarkHelp::PredictionResults DarkHelp::predict ( const std::string &  image_filename,
const float  new_threshold = -1.0f 
)
virtual

Use the neural network to predict what is contained in this image.

Parameters
[in]image_filenameThe name of the image file to load from disk and analyze. The member original_image will be set to this image.
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.
Returns
A vector of PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.
See also
PredictionResult
sort_predictions
duration
Exceptions
std::invalid_argumentif the image failed to load.
Here is the caller graph for this function:

◆ predict() [2/3]

DarkHelp::PredictionResults DarkHelp::predict ( cv::Mat  mat,
const float  new_threshold = -1.0f 
)
virtual

Use the neural network to predict what is contained in this image.

Parameters
[in]matA OpenCV2 image which has already been loaded and which needs to be analyzed. The member original_image will be set to this image.
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.
Returns
A vector of PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.
See also
PredictionResult
sort_predictions
duration
Exceptions
std::invalid_argumentif the image is empty.
Here is the call graph for this function:

◆ annotate()

cv::Mat DarkHelp::annotate ( const float  new_threshold = -1.0f)
virtual

Takes the most recent prediction_results, and applies them to the most recent original_image.

The output annotated image is stored in annotated_image as well as returned to the caller.

This is an example of what an annotated image looks like:

Parameters
[in]new_thresholdWhich threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then threshold will be set to this new value.

Turning down the threshold in annotate() wont bring back predictions that were excluded due to a higher threshold originally used with predict(). Here is an example:

darkhelp.predict("image.jpg", 0.75); // note the threshold is set to 75% for prediction
darkhelp.annotate(0.25); // note the threshold is now modified to be 25%

In the previous example, when annotate() is called with the lower threshold of 25%, the predictions had already been capped at 75%. This means any prediction between >= 25% and < 75% were excluded from the prediction results. The only way to get those predictions is to re-run predict() with a value of 0.25.

Note
Annotations wont be drawn if annotation_line_thickness is less than 1.
See also
annotation_colours
annotation_font_scale
annotation_font_thickness
annotation_line_thickness
annotation_include_duration
annotation_include_timestamp
Exceptions
std::logic_errorif an attempt is made to annotate an empty image
Here is the call graph for this function:

◆ duration_string()

std::string DarkHelp::duration_string ( )
virtual

Return the duration as a text string which can then be added to the image during annotation.

For example, this might return "912 microseconds" or "375 milliseconds".

See also
annotate()
Here is the caller graph for this function:

◆ get_default_annotation_colours()

DarkHelp::VColours DarkHelp::get_default_annotation_colours ( )
static

Obtain a vector of several bright colours that may be used to annotate images.

Remember that OpenCV uses BGR, not RGB. So pure red is "{0, 0, 255}". The colours returned by this function are intended to be used by OpenCV, and thus are in BGR format.

See also
annotation_colours
Here is the caller graph for this function:

◆ verify_cfg_and_weights()

DarkHelp::MStr DarkHelp::verify_cfg_and_weights ( std::string &  cfg_filename,
std::string &  weights_filename,
std::string &  names_filename 
)
static

Look at the names and/or the contents of all 3 files and swap the filenames around if necessary so the .cfg, .weights, and .names are assigned where they should be.

This is necessary because darknet tends to segfault if it is given the wrong filename. (For example, if it mistakenly tries to parse the .weights file as a .cfg file.) This function does a bit of sanity checking, determines which file is which, and also returns a map of debug information related to each file.

On input, it doesn't matter which file goes into which parameter. Simply pass in the filenames in any order.

On output, the .cfg, .weights, and .names will be set correctly. If needed for display purposes, some additional information is also passed back using the MStr string map, but most callers should ignore this.

See also
init()
Exceptions
std::invalid_argumentif at least 2 unique filenames have not been provided
std::runtime_errorif the size of the files cannot be determined (one or more file does not exist?)
std::invalid_argumentif the cfg file doesn't exist
std::invalid_argumentif the cfg file doesn't contain "[net]" near the top of the file
std::invalid_argumentif the configuration file does not have a line that says "classes=..."
std::invalid_argumentif the weights file doesn't exist
std::invalid_argumentif weights file has an invalid version number (or weights file is from an extremely old version of darknet?)
std::runtime_errorif the number of lines in the names file doesn't match the number of classes in the configuration file
Here is the caller graph for this function:

◆ predict() [3/3]

DarkHelp::PredictionResults DarkHelp::predict ( const float  new_threshold = -1.0f)
protected

Used by all the other predict() calls to do the actual network prediction.

This uses the image stored in original_image.

Exceptions
std::logic_errorif the network is invalid.
std::logic_errorif the image is invalid.

Member Data Documentation

◆ net

void* DarkHelp::net

The Darknet network, but stored as a void* pointer so we don't have to include darknet.h.

◆ names

VStr DarkHelp::names

A vector of names corresponding to the identified classes.

This is typically setup in the constructor, but can be manually set afterwards.

◆ duration

std::chrono::high_resolution_clock::duration DarkHelp::duration

The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time predict() took to run on the last image to be processed.

See also
duration_string()

◆ threshold

float DarkHelp::threshold

Image prediction threshold.

Defaults to 0.5.

See also
predict()
annotate()

Quote:

[...] threshold is what is used to determine whether or not there is an object in the predicted bounding box. The network predicts an explicit 'objectness' score separate from the class predictions that if above the threshold indicates that a bounding box will be returned. [source]

◆ hierarchy_threshold

float DarkHelp::hierarchy_threshold

Used during prediction.

Defaults to 0.5.

See also
predict()

Quote:

[...] the network traverses the tree of candidate detections and multiples through the conditional probabilities for each item, e.g. object * animal * feline * house cat. The hierarchical threshold is used in this second step, completely after and separate from whether there is an item or not, to decide whether following the tree further to a more specific class is the right action to take. When this threshold is 0, the tree will basically follow the highest probability branch all the way to a leaf node. [source]

Note
This variable used to be named hierchy_threshold. The typo in the name was fixed in December 2019.

◆ non_maximal_suppression_threshold

float DarkHelp::non_maximal_suppression_threshold

Non-Maximal Suppression (NMS) threshold suppresses overlapping bounding boxes and only retains the bounding box that has the maximum probability of object detection associated with it.

Defaults to 0.45.

See also
predict()

Quote:

[...] nms works by looking at all bounding boxes that made it past the 'objectness' threshold and removes the least confident ‚Äčof the boxes that overlap with each other above a certain IOU threshold [source]

(IOU – "intersection over union" – is a ratio that describes how much two areas overlap, where 0.0 means two areas don't overlap at all, and 1.0 means two areas perfectly overlap.)

◆ prediction_results

PredictionResults DarkHelp::prediction_results

A copy of the most recent results after applying the neural network to an image. This is set by predict().

◆ names_include_percentage

bool DarkHelp::names_include_percentage

Determines if the name given to each prediction includes the percentage.

For example, the name for a prediction might be "dog" when this flag is set to false, or it might be "dog 98%" when set to true. Defaults to true.

◆ include_all_names

bool DarkHelp::include_all_names

Determine if multiple class names are included when labelling an item.

For example, if an object is 95% car or 80% truck, then the label could say "car, truck" when this is set to true, and simply "car" when set to false. Defaults to true.

◆ annotation_colours

VColours DarkHelp::annotation_colours

The colours to use in annotate().

Defaults to get_default_annotation_colours().

Remember that OpenCV uses BGR, not RGB. So pure red is "(0, 0, 255)".

◆ annotation_font_face

cv::HersheyFonts DarkHelp::annotation_font_face

Font face to use in annotate(). Defaults to cv::HersheyFonts::FONT_HERSHEY_SIMPLEX.

◆ annotation_font_scale

double DarkHelp::annotation_font_scale

Scaling factor used for the font in annotate(). Defaults to 0.5.

◆ annotation_font_thickness

int DarkHelp::annotation_font_thickness

Thickness of the font in annotate(). Defaults to 1.

◆ annotation_line_thickness

int DarkHelp::annotation_line_thickness

Thickness of the lines to draw in annotate(). Defaults to 2.

◆ annotation_include_duration

bool DarkHelp::annotation_include_duration

If set to true then annotate() will call duration_string() and display on the top-left of the image the length of time predict() took to process the image.

Defaults to true.

When enabed, the duration may look similar to this:

◆ annotation_include_timestamp

bool DarkHelp::annotation_include_timestamp

If set to true then annotate() will display a timestamp on the bottom-left corner of the image.

Defaults to false.

When enabled, the timestamp may look similar to this:

◆ fix_out_of_bound_values

bool DarkHelp::fix_out_of_bound_values

Darknet sometimes will return values that are out-of-bound, especially when working with low thresholds.

For example, the X or Y coordinates might be less than zero, or the width and height might extend beyond the edges of the image. When fix_out_of_bound_values is set to true (the default) then the results (prediction_results) after calling predict() will be capped so all values are positive and do not extend beyond the edges of the image. When set to false, the exact values as returned by darknet will be used. Defaults to true.

◆ original_image

cv::Mat DarkHelp::original_image

The most recent image handled by predict().

◆ annotated_image

cv::Mat DarkHelp::annotated_image

The most recent output produced by annotate().

◆ sort_predictions

ESort DarkHelp::sort_predictions

Determines if the predictions will be sorted the next time predict() is called.

When set to ESort::kUnsorted, the predictions are in the exact same order as they were returned by Darknet. When set to ESort::kAscending or ESort::kDescending, the predictions will be sorted according to PredictionResult::best_probability.

If annotations will be drawn on the image for visual consumption, then it is often preferable to have the higher probability predictions drawn last so they appear "on top". Otherwise, lower probability predictions may overwrite or obscure the more important ones. This means using ESort::kAscending (the default).

If you want to process only the first few predictions instead of drawing annotations, then you may want to sort using ESort::kDescending to ensure you handle the most likely predictions first.

Defaults to ESort::kAscending.


The documentation for this class was generated from the following files:
DarkHelp
Instantiate one of these objects by giving it the name of the .cfg and .weights file,...
Definition: DarkHelp.hpp:73