 |
DarkHelp
v1.1.7-1
C++ API for the neural network framework Darknet
|
Looking for a C++ dev who knows OpenCV? I'm looking for work. Hire me!
|
|
- Note
- This document assumes you already have Darknet installed, and you have a functioning neural network.
If you're not already at that step, you'll want to look up some tutorials like these ones:
Once you have successfully trained a neural network, the next question becomes: how do you embed it into your C++ application?! Perhaps you've already looked into using Darknet's legacy C API, functions like load_network_custom()
, do_nms_sort()
, and get_network_boxes()
. That API is not easy to work with, and there isn't much documentation nor example code.
(In case it helps, I did put together a blog post with a few details in late August 2019: https://www.ccoderun.ca/programming/2019-08-25_Darknet_C_CPP/.)
DarkHelp lets you skip those C function calls, and simplifies things with an extremely simple-to-use C++ API! The DarkHelp C++ library is only available for Linux.
You load the neural network and the weight files, then call DarkHelp::predict() once per image you'd like analyzed. Each time you get back a new std::vector
of predictions.
Since annotating pictures is something that many applications want – especially during debugging – DarkHelp::annotate() is provided to easily mark up images with the detection results. To ease integrating this into larger projects, DarkHelp uses OpenCV's standard cv::Mat
images, not Darknet's internal image structure. This is an example of what DarkHelp::annotate() can do with an image and a neural network that detects barcodes:
If you're looking for some sample code to get started, this example loads a network and then loops through several image files:
DarkHelp darkhelp(
"mynetwork.cfg",
"mynetwork.weights",
"mynetwork.names");
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
for (const auto & filename : image_filenames)
{
darkhelp.predict(filename);
cv::Mat mat = darkhelp.annotate();
cv::imshow("prediction", mat);
cv::waitKey();
}
The predictions are stored in a std::vector
of structures. (See DarkHelp::PredictionResults.) You can get this vector and iterate through the results like this:
DarkHelp darkhelp(
"mynetwork.cfg",
"mynetwork.weights",
"mynetwork.names");
const auto results = darkhelp.predict("test_image_01.jpg");
for (const auto & det : results)
{
std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}
If you have multiple classes defined in your network, then you may want to look at DarkHelp::PredictionResult::all_probabilities, not only DarkHelp::PredictionResult::best_class and DarkHelp::PredictionResult::best_probability.
The following is the shortest/simplest self-contained example showing how to load a network, run it against a set of images provided on the command-line, and then output the results as a series of coordinates, names, etc:
#include <iostream>
#include <DarkHelp.hpp>
int main(int argc, char *argv[])
{
DarkHelp darkhelp(
"driving.cfg",
"driving_best.weights",
"driving.names");
for (int idx = 1; idx < argc; idx ++)
{
const auto results = darkhelp.predict(argv[idx]);
std::cout << results << std::endl;
}
return 0;
}
- Note
- The order in which you specify the
.cfg,
.weights, and
.names files in the constructor or in DarkHelp::init() is not important due to how the parameters are swapped around by DarkHelp::verify_cfg_and_weights().
Example output from sending the "results"
to std::cout
like the code in the previous block:
#1/74: loading image "surveillance_frame_000443.jpg"
-> prediction took 4 milliseconds
-> prediction results: 12
-> 1/12: "vehicle 84%" #0 prob=0.838765 x=573 y=223 w=24 h=19 entries=1
-> 2/12: "vehicle 85%" #0 prob=0.845121 x=1034 y=236 w=26 h=19 entries=1
-> 3/12: "motorcycle 93%" #1 prob=0.932856 x=473 y=308 w=24 h=54 entries=1
-> 4/12: "vehicle 98%" #0 prob=0.98197 x=1027 y=242 w=38 h=20 entries=1
...
If you call DarkHelp::annotate() to get back a OpenCV cv::Mat
object, you can then display the image with all the annotations, or easily save it as a jpg
or png
. For example:
darkhelp.predict(argv[idx]);
cv::Mat mat = darkhelp.annotate();
cv::imwrite("output.png", mat, {CV_IMWRITE_PNG_COMPRESSION, 9});
The example call to cv::imwrite()
in the previous example might give something similar to this image:
Note that DarkHelp uses OpenCV internally, regardless of whether or not the client code calls DarkHelp::annotate(). This means when you link against the libdarkhelp
library you'll also need to link against OpenCV.
DarkHelp()
Constructor. When using this constructor, the neural network remains uninitialized until init() is ca...
Definition: DarkHelp.cpp:58
int annotation_line_thickness
Thickness of the lines to draw in annotate(). Defaults to 2.
Definition: DarkHelp.hpp:568
float hierarchy_threshold
Used during prediction.
Definition: DarkHelp.hpp:481
float tile_rect_factor
This value controls how close the rectangles needs to line up on two tiles before the predictions are...
Definition: DarkHelp.hpp:777
virtual void reset()
The opposite of init(). This is automatically called by the destructor.
Definition: DarkHelp.cpp:179
DarkHelp & name_prediction(PredictionResult &pred)
Give a consistent name to the given production result.
Definition: DarkHelp.cpp:1272
ESort sort_predictions
Determines if the predictions will be sorted the next time predict() is called.
Definition: DarkHelp.hpp:623
cv::Mat original_image
The most recent image handled by predict().
Definition: DarkHelp.hpp:596
cv::Mat annotated_image
The most recent output produced by annotate().
Definition: DarkHelp.hpp:599
float non_maximal_suppression_threshold
Non-Maximal Suppression (NMS) threshold suppresses overlapping bounding boxes and only retains the bo...
Definition: DarkHelp.hpp:495
size_t vertical_tiles
The number of vertical tiles the image was split into by predict_tile() prior to calling predict().
Definition: DarkHelp.hpp:672
float annotation_shade_predictions
Determines the amount of "shade" used when drawing the prediction rectangles.
Definition: DarkHelp.hpp:543
bool enable_tiles
Determines if calls to predict() are sent directly to Darknet, or processed first by predict_tile() t...
Definition: DarkHelp.hpp:654
int best_class
The class that obtained the highest probability.
Definition: DarkHelp.hpp:177
static VColours get_default_annotation_colours()
Obtain a vector of at least 25 different bright colours that may be used to annotate images.
Definition: DarkHelp.cpp:716
virtual cv::Mat annotate(const float new_threshold=-1.0f)
Takes the most recent prediction_results, and applies them to the most recent original_image.
Definition: DarkHelp.cpp:511
bool annotation_auto_hide_labels
Hide the label if the size of the text exceeds the size of the prediction.
Definition: DarkHelp.hpp:526
virtual ~DarkHelp()
Destructor. This automatically calls reset() to release memory allocated by the neural network.
Definition: DarkHelp.cpp:50
bool combine_tile_predictions
When tiling is enabled, objects may span multiple tiles.
Definition: DarkHelp.hpp:752
int tile
The tile number on which this object was found.
Definition: DarkHelp.hpp:195
size_t horizontal_tiles
The number of horizontal tiles the image was split into by predict_tile() prior to calling predict().
Definition: DarkHelp.hpp:663
PredictionResults prediction_results
A copy of the most recent results after applying the neural network to an image. This is set by predi...
Definition: DarkHelp.hpp:498
bool fix_out_of_bound_values
Darknet sometimes will return values that are out-of-bound, especially when working with low threshol...
Definition: DarkHelp.hpp:593
std::string name
A name to use for the object.
Definition: DarkHelp.hpp:190
bool names_include_percentage
Determines if the name given to each prediction includes the percentage.
Definition: DarkHelp.hpp:512
bool annotation_include_timestamp
If set to true then annotate() will display a timestamp on the bottom-left corner of the image.
Definition: DarkHelp.hpp:584
std::set< int > annotation_suppress_classes
Determines which classes to suppress during the call to annotate().
Definition: DarkHelp.hpp:735
cv::HersheyFonts annotation_font_face
Font face to use in annotate(). Defaults to cv::HersheyFonts::FONT_HERSHEY_SIMPLEX.
Definition: DarkHelp.hpp:559
std::vector< std::string > VStr
Vector of text strings. Typically used to store the class names.
Definition: DarkHelp.hpp:81
bool enable_debug
This enables some non-specific debug functionality within the DarkHelp library.
Definition: DarkHelp.hpp:629
cv::Size2f original_size
The original normalized width and height returned by darknet.
Definition: DarkHelp.hpp:135
double annotation_font_scale
Scaling factor used for the font in annotate(). Defaults to 0.5.
Definition: DarkHelp.hpp:562
std::map< int, float > MClassProbabilities
Map of a class ID to a probability that this object belongs to that class.
Definition: DarkHelp.hpp:90
bool include_all_names
Determine if multiple class names are included when labelling an item.
Definition: DarkHelp.hpp:550
cv::Rect rect
OpenCV rectangle which describes where the object is located in the original image.
Definition: DarkHelp.hpp:109
float threshold
Image prediction threshold.
Definition: DarkHelp.hpp:467
virtual PredictionResults predict(const std::string &image_filename, const float new_threshold=-1.0f)
Use the neural network to predict what is contained in this image.
Definition: DarkHelp.cpp:225
float tile_edge_factor
This value controls how close to the edge of a tile an object must be to be considered for re-combini...
Definition: DarkHelp.hpp:764
int annotation_font_thickness
Thickness of the font in annotate(). Defaults to 1.
Definition: DarkHelp.hpp:565
ESort
Definition: DarkHelp.hpp:602
std::chrono::high_resolution_clock::duration duration
The length of time it took to initially load the network and weights (after the DarkHelp object has b...
Definition: DarkHelp.hpp:457
@ kDescending
Sort predictions using PredictionResult::best_probability in descending order (high values first,...
void * net
The Darknet network, but stored as a void* pointer so we don't have to include darknet....
Definition: DarkHelp.hpp:444
bool modify_batch_and_subdivisions
When training, the "batch=..." and "subdivisions=..." values in the .cfg file are typically set to a ...
Definition: DarkHelp.hpp:704
cv::Size tile_size
The size that was used for each individual tile by predict_tile().
Definition: DarkHelp.hpp:685
cv::Point2f original_point
The original normalized X and Y coordinate returned by darknet.
Definition: DarkHelp.hpp:122
std::map< std::string, std::string > MStr
Map of strings where both the key and the value are std::string.
Definition: DarkHelp.hpp:78
float best_probability
The probability of the class that obtained the highest value.
Definition: DarkHelp.hpp:184
PredictionResults predict_internal(cv::Mat mat, const float new_threshold=-1.0f)
Used by all the other predict() calls to do the actual network prediction.
Definition: DarkHelp.cpp:1097
static size_t edit_cfg_file(const std::string &cfg_filename, MStr m)
This is used to insert lines into the [net] section of the configuration file.
Definition: DarkHelp.cpp:940
virtual cv::Size network_size()
Determine the size of the network. For example, 416x416, or 608x480.
Definition: DarkHelp.cpp:702
virtual DarkHelp & init(const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true)
Initialize ("load") the darknet neural network.
Definition: DarkHelp.cpp:109
std::vector< cv::Scalar > VColours
Vector of colours to use by annotate().
Definition: DarkHelp.hpp:84
bool annotation_include_duration
If set to true then annotate() will call duration_string() and display on the top-left of the image t...
Definition: DarkHelp.hpp:576
virtual std::string version() const
Get a version string for the DarkHelp library. E.g., could be 1.0.0-123.
Definition: DarkHelp.cpp:103
Instantiate one of these objects by giving it the name of the .cfg and .weights file,...
Definition: DarkHelp.hpp:73
@ kAscending
Sort predictions using PredictionResult::best_probability in ascending order (low values first,...
std::vector< PredictionResult > PredictionResults
A vector of predictions for the image analyzed by predict().
Definition: DarkHelp.hpp:204
virtual std::string duration_string()
Return the duration as a text string which can then be added to the image during annotation.
Definition: DarkHelp.cpp:690
VColours annotation_colours
The colours to use in annotate().
Definition: DarkHelp.hpp:556
MClassProbabilities all_probabilities
This is only useful if you have multiple classes, and an object may be one of several possible classe...
Definition: DarkHelp.hpp:170
virtual PredictionResults predict_tile(cv::Mat mat, const float new_threshold=-1.0f)
Similar to predict(), but automatically breaks the images down into individual tiles if it is signifi...
Definition: DarkHelp.cpp:273
VStr names
A vector of names corresponding to the identified classes.
Definition: DarkHelp.hpp:450
@ kUnsorted
Do not sort predictions.
Structure used to store interesting information on predictions.
Definition: DarkHelp.hpp:95
static MStr verify_cfg_and_weights(std::string &cfg_filename, std::string &weights_filename, std::string &names_filename)
Look at the names and/or the contents of all 3 files and swap the filenames around if necessary so th...
Definition: DarkHelp.cpp:752