DarkHelp  v1.4.8-1
C++ API for the neural network framework Darknet
Looking for a C++ dev who knows OpenCV?
I'm looking for work. Hire me!
Summary
Note
This document assumes you already have Darknet installed, and you have a functioning neural network.
If you're not already at that step, you'll want to look up some tutorials like these ones:  

Once you have successfully trained a neural network, the next question becomes: how do you embed it into your C++ application?! Perhaps you've already looked into using Darknet's legacy C API, functions like load_network_custom(), do_nms_sort(), and get_network_boxes(). That API is not easy to work with, and there isn't much documentation nor example code.

(In case it helps, I did put together a blog post with a few details in late August 2019: https://www.ccoderun.ca/programming/2019-08-25_Darknet_C_CPP/.)

DarkHelp lets you skip those C function calls, and simplifies things with an extremely simple-to-use C++ API! The DarkHelp C++ library is available on Linux and Windows.

You load the neural network and the weight files, then call DarkHelp::NN::predict() once per image you'd like analyzed. Each time you get back a new std::vector of predictions.

Since annotating pictures is something that many applications want – especially during debugging – DarkHelp::NN::annotate() is provided to easily mark up images with the detection results. To ease integrating this into larger projects, DarkHelp uses OpenCV's standard cv::Mat images, not Darknet's internal image structure. This is an example of what DarkHelp::NN::annotate() can do with an image and a neural network that detects barcodes:

If you're looking for some sample code to get started, this example loads a network and then loops through several image files:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
for (const auto & filename : image_filenames)
{
// these next two lines is where DarkHelp calls into Darknet to do all the hard work
nn.predict(filename);
cv::Mat mat = nn.annotate(); // annotates the most recent image seen by predict()
// use standard OpenCV calls to show the image results in a window
cv::imshow("prediction", mat);
cv::waitKey();
}

The predictions are stored in a std::vector of structures. (See DarkHelp::PredictionResults.) You can get this vector and iterate through the results like this:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
const auto results = nn.predict("test_image_01.jpg");
for (const auto & det : results)
{
std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}

If you have multiple classes defined in your network, then you may want to look at DarkHelp::PredictionResult::all_probabilities, not only DarkHelp::PredictionResult::best_class and DarkHelp::PredictionResult::best_probability.

The following is the shortest/simplest self-contained example showing how to load a network, run it against a set of images provided on the command-line, and then output the results as a series of coordinates, names, etc:

#include <iostream>
#include <DarkHelp.hpp>
int main(int argc, char *argv[])
{
DarkHelp::NN nn("driving.cfg", "driving_best.weights", "driving.names");
// Loop through all the images specified on the command-line:
for (int idx = 1; idx < argc; idx ++)
{
// get the predictions
const auto results = nn.predict(argv[idx]);
// display the results on the console (meaning coordinates, not displaying the images themselves)
std::cout << results << std::endl; // see the output in the next block below
// to annotate the images, you'd use this instead:
// cv::Mat output = nn.annotate();
// do_something_with_the_image(output);
}
return 0;
}
Note
The order in which you specify the .cfg, .weights, and .names files in the constructor or in DarkHelp::NN::init() is not important due to how the parameters are swapped around by DarkHelp::verify_cfg_and_weights().

Example output from sending the "results" to std::cout like the code in the previous block:

#1/74: loading image "surveillance_frame_000443.jpg"
-> prediction took 4 milliseconds
-> prediction results: 12
-> 1/12: "vehicle 84%" #0 prob=0.838765 x=573 y=223 w=24 h=19 entries=1
-> 2/12: "vehicle 85%" #0 prob=0.845121 x=1034 y=236 w=26 h=19 entries=1
-> 3/12: "motorcycle 93%" #1 prob=0.932856 x=473 y=308 w=24 h=54 entries=1
-> 4/12: "vehicle 98%" #0 prob=0.98197 x=1027 y=242 w=38 h=20 entries=1
...

If you call DarkHelp::NN::annotate() to get back a OpenCV cv::Mat object, you can then display the image with all the annotations, or easily save it as a jpg or png. For example:

nn.predict(argv[idx]);
cv::Mat mat = nn.annotate();
cv::imwrite("output.png", mat, {CV_IMWRITE_PNG_COMPRESSION, 9});

The example call to cv::imwrite() in the previous example might give something similar to this image:

Note that DarkHelp uses OpenCV internally, regardless of whether or not the client code calls DarkHelp::NN::annotate(). This means when you link against the libdarkhelp library you'll also need to link against OpenCV.

DarkHelp::VRect2d
std::vector< cv::Rect2d > VRect2d
Similar to DarkHelp::VRect, but the rectangle uses double instead of int.
Definition: DarkHelp.hpp:79
DarkHelp::Config::~Config
~Config()
Destructor.
Definition: DarkHelpConfig.cpp:9
DarkHelp::Config
All of DarkHelp's configuration is stored within an instance of this class.
Definition: DarkHelpConfig.hpp:30
DarkHelp::Config::annotation_auto_hide_labels
bool annotation_auto_hide_labels
Hide the label if the size of the text exceeds the size of the prediction.
Definition: DarkHelpConfig.hpp:135
DarkHelp::edit_cfg_file
size_t edit_cfg_file(const std::string &cfg_filename, MStr m)
This is used to insert lines into the [net] section of the configuration file.
Definition: DarkHelpUtils.cpp:284
DarkHelp::EDriver::kDarknet
@ kDarknet
Use libdarknet.so.
DarkHelp::NN::horizontal_tiles
size_t horizontal_tiles
The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to call...
Definition: DarkHelpNN.hpp:312
DarkHelp::Config::hierarchy_threshold
float hierarchy_threshold
Used during prediction.
Definition: DarkHelpConfig.hpp:92
DarkHelp::Config::non_maximal_suppression_threshold
float non_maximal_suppression_threshold
Non-Maximal Suppression (NMS) threshold suppresses overlapping bounding boxes and only retains the bo...
Definition: DarkHelpConfig.hpp:106
DarkHelp::Config::binary_threshold_constant
double binary_threshold_constant
Constant to remove from each pixel value when converting image during thresholding.
Definition: DarkHelpConfig.hpp:429
DarkHelp::fix_out_of_bound_normalized_rect
void fix_out_of_bound_normalized_rect(float &cx, float &cy, float &w, float &h)
Automatically called by DarkHelp::NN::predict_internal() when DarkHelp::Config::fix_out_of_bound_valu...
Definition: DarkHelpUtils.cpp:444
DarkHelp::Config::reset
Config & reset()
Reset all config values to their default settings.
Definition: DarkHelpConfig.cpp:45
DarkHelp::version
std::string version()
Get a version string for the DarkHelp library. E.g., could be 1.0.0-123.
Definition: DarkHelpUtils.cpp:39
DarkHelp::NN::~NN
~NN()
Destructor. This automatically calls reset() to release memory allocated by the neural network.
Definition: DarkHelpNN.cpp:35
DarkHelp::Config::names_include_percentage
bool names_include_percentage
Determines if the name given to each prediction includes the percentage.
Definition: DarkHelpConfig.hpp:120
DarkHelp::Config::tile_edge_factor
float tile_edge_factor
This value controls how close to the edge of a tile an object must be to be considered for re-combini...
Definition: DarkHelpConfig.hpp:368
DarkHelp::NN::original_image
cv::Mat original_image
The most recent image handled by DarkHelp::NN::predict().
Definition: DarkHelpNN.hpp:296
DarkHelp::EDriver::kInvalid
@ kInvalid
DarkHelp::VFloat
std::vector< float > VFloat
Vector of float used with OpenCV.
Definition: DarkHelp.hpp:73
DarkHelp::NN::predict
PredictionResults predict(const std::string &image_filename, const float new_threshold=-1.0f)
Use the neural network to predict what is contained in this image.
Definition: DarkHelpNN.cpp:306
DarkHelp::NN::duration
std::chrono::high_resolution_clock::duration duration
The length of time it took to initially load the network and weights (after the DarkHelp object has b...
Definition: DarkHelpNN.hpp:290
DarkHelp::Config::enable_tiles
bool enable_tiles
Determines if calls to DarkHelp::NN::predict() are sent directly to Darknet, or processed first by Da...
Definition: DarkHelpConfig.hpp:253
DarkHelp::Config::threshold
float threshold
Image prediction threshold.
Definition: DarkHelpConfig.hpp:78
DarkHelp::Config::include_all_names
bool include_all_names
Determine if multiple class names are included when labelling an item.
Definition: DarkHelpConfig.hpp:160
DarkHelp::Config::annotation_suppress_classes
std::set< int > annotation_suppress_classes
Determines which classes to suppress during the call to DarkHelp::NN::annotate().
Definition: DarkHelpConfig.hpp:311
DarkHelp::Config::annotation_line_thickness
int annotation_line_thickness
Thickness of the lines to draw in DarkHelp::NN::annotate(). Defaults to 2.
Definition: DarkHelpConfig.hpp:178
DarkHelp::PredictionResult::best_class
int best_class
The class that obtained the highest probability.
Definition: DarkHelpPredictionResult.hpp:114
DarkHelp::NN::predict_internal
PredictionResults predict_internal(cv::Mat mat, const float new_threshold=-1.0f)
Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction.
Definition: DarkHelpNN.cpp:800
DarkHelp::Config::snapping_horizontal_tolerance
int snapping_horizontal_tolerance
Horizontal tolerance (in pixels) used when snapping annotations.
Definition: DarkHelpConfig.hpp:448
DarkHelp::NN::predict_internal_opencv
void predict_internal_opencv()
Called from DarkHelp::NN::predict_internal().
Definition: DarkHelpNN.cpp:980
DarkHelp::get_default_annotation_colours
VColours get_default_annotation_colours()
Obtain a vector of at least 25 different bright colours that may be used to annotate images.
Definition: DarkHelpUtils.cpp:60
DarkHelp::Config::weights_filename
std::string weights_filename
Filename (relative or absolute) for the Darknet/YOLO .weights file.
Definition: DarkHelpConfig.hpp:60
DarkHelp
The DarkHelp namespace contains (almost) everything in the DarkHelp library.
Definition: DarkHelp.hpp:48
DarkHelp::NN
Instantiate one of these objects by giving it the name of the .cfg and .weights file,...
Definition: DarkHelpNN.hpp:60
DarkHelp::NN::name_prediction
NN & name_prediction(PredictionResult &pred)
Give a consistent name to the given production result.
Definition: DarkHelpNN.cpp:1173
DarkHelp::NN::snap_annotation
NN & snap_annotation(PredictionResult &pred)
Snap only the given annotation.
Definition: DarkHelpNN.cpp:1230
DarkHelp::PredictionResult::tile
int tile
The tile number on which this object was found.
Definition: DarkHelpPredictionResult.hpp:133
DarkHelp::MClassProbabilities
std::map< int, float > MClassProbabilities
Map of a class ID to a probability that this object belongs to that class.
Definition: DarkHelpPredictionResult.hpp:26
DarkHelp::Config::combine_tile_predictions
bool combine_tile_predictions
When tiling is enabled, objects may span multiple tiles.
Definition: DarkHelpConfig.hpp:329
DarkHelp::Config::annotation_font_scale
double annotation_font_scale
Scaling factor used for the font in DarkHelp::NN::annotate(). Defaults to 0.5.
Definition: DarkHelpConfig.hpp:172
DarkHelp::duration_string
std::string duration_string(const std::chrono::high_resolution_clock::duration duration)
Format a duration as a text string which is typically added to images or video frames during annotati...
Definition: DarkHelpUtils.cpp:45
DarkHelp::EDriver::kMin
@ kMin
DarkHelp::resize_keeping_aspect_ratio
cv::Mat resize_keeping_aspect_ratio(cv::Mat mat, const cv::Size &desired_size)
Convenience function to resize an image yet retain the exact original aspect ratio.
Definition: DarkHelpUtils.cpp:476
DarkHelp::NN::annotate
cv::Mat annotate(const float new_threshold=-1.0f)
Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp:...
Definition: DarkHelpNN.cpp:607
DarkHelp::NN::init
NN & init()
Initialize ("load") the darknet neural network. This uses the values within DarkHelp::NN::config.
Definition: DarkHelpNN.cpp:123
DarkHelp::Config::snapping_vertical_tolerance
int snapping_vertical_tolerance
Vertical tolerance (in pixels) used when snapping annotations.
Definition: DarkHelpConfig.hpp:459
DarkHelp::Config::binary_threshold_block_size
int binary_threshold_block_size
Block size (in pixels) to use when creating a black-and-white binary image.
Definition: DarkHelpConfig.hpp:421
DarkHelp::NN::vertical_tiles
size_t vertical_tiles
The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to callin...
Definition: DarkHelpNN.hpp:322
DarkHelp::PredictionResult::name
std::string name
A name to use for the object.
Definition: DarkHelpPredictionResult.hpp:128
DarkHelp::Config::enable_debug
bool enable_debug
This enables some non-specific debug functionality within the DarkHelp library.
Definition: DarkHelpConfig.hpp:226
DarkHelp::Config::modify_batch_and_subdivisions
bool modify_batch_and_subdivisions
When training, the "batch=..." and "subdivisions=..." values in the .cfg file are typically set to a ...
Definition: DarkHelpConfig.hpp:278
DarkHelp::Config::names_filename
std::string names_filename
Filename (relative or absolute) for the Darknet/YOLO .names file.
Definition: DarkHelpConfig.hpp:65
DarkHelp::ESort::kAscending
@ kAscending
Sort predictions using DarkHelp::PredictionResult::best_probability in ascending order (low values fi...
DarkHelp::NN::NN
NN()
Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN...
Definition: DarkHelpNN.cpp:43
DarkHelp::NN::snap_annotations
NN & snap_annotations()
Snap all the annotations.
Definition: DarkHelpNN.cpp:1219
DarkHelp::NN::config
Config config
Configuratin for the neural network.
Definition: DarkHelpNN.hpp:341
DarkHelp::PredictionResult::original_size
cv::Size2f original_size
The original normalized width and height returned by darknet.
Definition: DarkHelpPredictionResult.hpp:72
DarkHelp::NN::network_size
cv::Size network_size()
Determine the size of the network. For example, 416x416, or 608x480.
Definition: DarkHelpNN.cpp:792
DarkHelp::Config::snapping_enabled
bool snapping_enabled
Toggle annotation snapping.
Definition: DarkHelpConfig.hpp:413
DarkHelp::VStr
std::vector< std::string > VStr
Vector of text strings. Typically used to store the class names.
Definition: DarkHelp.hpp:64
DarkHelp::Config::annotation_font_face
cv::HersheyFonts annotation_font_face
Font face to use in DarkHelp::NN::annotate(). Defaults to cv::HersheyFonts::FONT_HERSHEY_SIMPLEX.
Definition: DarkHelpConfig.hpp:169
DarkHelp::EDriver::kOpenCVCPU
@ kOpenCVCPU
Use OpenCV's dnn module, but skip CUDA and only use the CPU.
DarkHelp::PredictionResult::rect
cv::Rect rect
OpenCV rectangle which describes where the object is located in the original image.
Definition: DarkHelpPredictionResult.hpp:46
DarkHelp::verify_cfg_and_weights
MStr verify_cfg_and_weights(std::string &cfg_filename, std::string &weights_filename, std::string &names_filename)
Look at the names and/or the contents of all 3 files and swap the filenames around if necessary so th...
Definition: DarkHelpUtils.cpp:96
DarkHelp::NN::network_dimensions
cv::Size network_dimensions
Size of the neural network, e.g., 416x416 or 608x608.
Definition: DarkHelpNN.hpp:362
DarkHelp::EDriver
EDriver
DarkHelp can utilise either libdarknet.so or OpenCV's DNN module to load the neural network and run i...
Definition: DarkHelp.hpp:102
DarkHelp::MStr
std::map< std::string, std::string > MStr
Map of strings where both the key and the value are std::string.
Definition: DarkHelp.hpp:61
DarkHelp::Config::annotation_include_timestamp
bool annotation_include_timestamp
If set to true then DarkHelp::NN::annotate() will display a timestamp on the bottom-left corner of th...
Definition: DarkHelpConfig.hpp:195
DarkHelp::EDriver::kMax
@ kMax
DarkHelp::Config::fix_out_of_bound_values
bool fix_out_of_bound_values
Darknet sometimes will return values that are out-of-bound, especially when working with low threshol...
Definition: DarkHelpConfig.hpp:204
DarkHelp::Config::annotation_shade_predictions
float annotation_shade_predictions
Determines the amount of "shade" used when drawing the prediction rectangles.
Definition: DarkHelpConfig.hpp:153
DarkHelp::ESort
ESort
Definition: DarkHelp.hpp:113
DarkHelp::fast_resize_ignore_aspect_ratio
cv::Mat fast_resize_ignore_aspect_ratio(const cv::Mat &mat, const cv::Size &desired_size)
Resize the given image as quickly as possible to the given dimensions.
Definition: DarkHelpUtils.cpp:518
DarkHelp::NN::version
static std::string version()
Get a version string for the DarkHelp library. E.g., could be 1.0.0-123.
DarkHelp::Config::Config
Config()
Constructor.
Definition: DarkHelpConfig.cpp:14
DarkHelp::Config::cfg_filename
std::string cfg_filename
Filename (relative or absolute) for the Darknet/YOLO .cfg file.
Definition: DarkHelpConfig.hpp:55
DarkHelp::EDriver::kOpenCV
@ kOpenCV
Use OpenCV's dnn module. Attempts to use CUDA, and will automatically revert to CPU if CUDA is not av...
DarkHelp::Config::annotation_include_duration
bool annotation_include_duration
If set to true then DarkHelp::NN::annotate() will call DarkHelp::NN::duration_string() and display on...
Definition: DarkHelpConfig.hpp:187
DarkHelp::PredictionResult::original_point
cv::Point2f original_point
The original normalized X and Y coordinate returned by darknet.
Definition: DarkHelpPredictionResult.hpp:59
DarkHelp::NN::predict_tile
PredictionResults predict_tile(cv::Mat mat, const float new_threshold=-1.0f)
Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if...
Definition: DarkHelpNN.cpp:354
DarkHelp::PredictionResult::best_probability
float best_probability
The probability of the class that obtained the highest value.
Definition: DarkHelpPredictionResult.hpp:121
DarkHelp::NN::duration_string
std::string duration_string()
Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation...
Definition: DarkHelpNN.cpp:786
DarkHelp::NN::names
VStr names
A vector of names corresponding to the identified classes.
Definition: DarkHelpNN.hpp:283
DarkHelp::ESort::kDescending
@ kDescending
Sort predictions using DarkHelp::PredictionResult::best_probability in descending order (high values ...
DarkHelp::VRect
std::vector< cv::Rect > VRect
Vector of OpenCV rectangles used with OpenCV.
Definition: DarkHelp.hpp:76
DarkHelp::Config::only_combine_similar_predictions
bool only_combine_similar_predictions
When combine_tile_predictions is enabled, this determines if an attempt is made to combine prediction...
Definition: DarkHelpConfig.hpp:349
DarkHelp::Config::annotation_font_thickness
int annotation_font_thickness
Thickness of the font in DarkHelp::NN::annotate(). Defaults to 1.
Definition: DarkHelpConfig.hpp:175
DarkHelp::Config::sort_predictions
ESort sort_predictions
Determines if the predictions will be sorted the next time DarkHelp::NN::predict() is called.
Definition: DarkHelpConfig.hpp:220
DarkHelp::NN::tile_size
cv::Size tile_size
The size that was used for each individual tile by DarkHelp::NN::predict_tile().
Definition: DarkHelpNN.hpp:336
DarkHelp::Config::annotation_colours
VColours annotation_colours
The colours to use in DarkHelp::NN::annotate().
Definition: DarkHelpConfig.hpp:166
DarkHelp::PredictionResults
std::vector< PredictionResult > PredictionResults
A vector of predictions for the image analyzed by DarkHelp::NN::predict().
Definition: DarkHelpPredictionResult.hpp:142
DarkHelp::VInt
std::vector< int > VInt
Vector of int used with OpenCV.
Definition: DarkHelp.hpp:70
DarkHelp::NN::binary_inverted_image
cv::Mat binary_inverted_image
Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled.
Definition: DarkHelpNN.hpp:302
DarkHelp::NN::reset
NN & reset()
The opposite of DarkHelp::NN::init(). This is automatically called by the destructor.
Definition: DarkHelpNN.cpp:277
DarkHelp::NN::predict_internal_darknet
void predict_internal_darknet()
Called from DarkHelp::NN::predict_internal().
Definition: DarkHelpNN.cpp:888
DarkHelp::NN::prediction_results
PredictionResults prediction_results
A copy of the most recent results after applying the neural network to an image. This is set by DarkH...
Definition: DarkHelpNN.hpp:293
DarkHelp::Config::driver
EDriver driver
The driver initialization happens in DarkHelp::NN::init().
Definition: DarkHelpConfig.hpp:394
DarkHelp::NN::darknet_net
void * darknet_net
The Darknet network, but stored as a void* pointer so we don't have to include darknet....
Definition: DarkHelpNN.hpp:273
DarkHelp::PredictionResult::all_probabilities
MClassProbabilities all_probabilities
This is only useful if you have multiple classes, and an object may be one of several possible classe...
Definition: DarkHelpPredictionResult.hpp:107
DarkHelp::ESort::kUnsorted
@ kUnsorted
Do not sort predictions.
DarkHelp::Config::tile_rect_factor
float tile_rect_factor
This value controls how close the rectangles needs to line up on two tiles before the predictions are...
Definition: DarkHelpConfig.hpp:388
DarkHelp::NN::operator=
NN & operator=(const NN &)=delete
We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by ...
DarkHelp::PredictionResult
Structure used to store interesting information on predictions.
Definition: DarkHelpPredictionResult.hpp:32
DarkHelp::NN::annotated_image
cv::Mat annotated_image
The most recent output produced by DarkHelp::NN::annotate().
Definition: DarkHelpNN.hpp:299
DarkHelp::VColours
std::vector< cv::Scalar > VColours
Vector of colours to use by DarkHelp::NN::annotate().
Definition: DarkHelp.hpp:67