Summary

Note

This document assumes you already have Darknet installed, and you have a functioning neural network.
If you're not already at that step, you'll want to look up some tutorials like these ones:

• https://www.ccoderun.ca/programming/2020-03-07_Darknet/ (newer tutorial, better information)
• https://www.ccoderun.ca/programming/2019-08-17_Darknet_summary/ (older tutorial, different approach)

 

Once you have successfully trained a neural network, the next question becomes: how do you embed it into your C++ application?! Perhaps you've already looked into using Darknet's legacy C API, functions like load_network_custom(), do_nms_sort(), and get_network_boxes(). That API is not easy to work with, and there isn't much documentation nor example code.

(In case it helps, I did put together a blog post with a few details in late August 2019: https://www.ccoderun.ca/programming/2019-08-25_Darknet_C_CPP/.)

DarkHelp lets you skip those C function calls, and simplifies things with an extremely simple-to-use C++ API! The DarkHelp C++ library is available on Linux and Windows.

You load the neural network and the weight files, then call DarkHelp::NN::predict() once per image you'd like analyzed. Each time you get back a new std::vector of predictions.

Since annotating pictures is something that many applications want – especially during debugging – DarkHelp::NN::annotate() is provided to easily mark up images with the detection results. To ease integrating this into larger projects, DarkHelp uses OpenCV's standard cv::Mat images, not Darknet's internal image structure. This is an example of what DarkHelp::NN::annotate() can do with an image and a neural network that detects barcodes:

If you're looking for some sample code to get started, this example loads a network and then loops through several image files:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
 
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
 
for (const auto & filename : image_filenames)
{
    // these next two lines is where DarkHelp calls into Darknet to do all the hard work
    nn.predict(filename);
    cv::Mat mat = nn.annotate(); // annotates the most recent image seen by predict()
 
    // use standard OpenCV calls to show the image results in a window
    cv::imshow("prediction", mat);
    cv::waitKey();
}

The predictions are stored in a std::vector of structures. (See DarkHelp::PredictionResults.) You can get this vector and iterate through the results like this:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
 
const auto results = nn.predict("test_image_01.jpg");
 
for (const auto & det : results)
{
    std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}

If you have multiple classes defined in your network, then you may want to look at DarkHelp::PredictionResult::all_probabilities, not only DarkHelp::PredictionResult::best_class and DarkHelp::PredictionResult::best_probability.

The following is the shortest/simplest self-contained example showing how to load a network, run it against a set of images provided on the command-line, and then output the results as a series of coordinates, names, etc:

#include <iostream>
#include <DarkHelp.hpp>
 
int main(int argc, char *argv[])
{
    DarkHelp::NN nn("driving.cfg", "driving_best.weights", "driving.names");
 
    // Loop through all the images specified on the command-line:
    for (int idx = 1; idx < argc; idx ++)
    {
        // get the predictions
        const auto results = nn.predict(argv[idx]);
 
        // display the results on the console (meaning coordinates, not displaying the images themselves)
        std::cout << results << std::endl; // see the output in the next block below
 
        // to annotate the images, you'd use this instead:
//      cv::Mat output = nn.annotate();
//      do_something_with_the_image(output);
    }
 
    return 0;
}

Note

The order in which you specify the .cfg, .weights, and .names files in the constructor or in DarkHelp::NN::init() is not important due to how the parameters are swapped around by DarkHelp::verify_cfg_and_weights().

Example output from sending the "results" to std::cout like the code in the previous block:

#1/74: loading image "surveillance_frame_000443.jpg"
-> prediction took 4 milliseconds
-> prediction results: 12
-> 1/12: "vehicle 84%" #0 prob=0.838765 x=573 y=223 w=24 h=19 entries=1
-> 2/12: "vehicle 85%" #0 prob=0.845121 x=1034 y=236 w=26 h=19 entries=1
-> 3/12: "motorcycle 93%" #1 prob=0.932856 x=473 y=308 w=24 h=54 entries=1
-> 4/12: "vehicle 98%" #0 prob=0.98197 x=1027 y=242 w=38 h=20 entries=1
...

If you call DarkHelp::NN::annotate() to get back a OpenCV cv::Mat object, you can then display the image with all the annotations, or easily save it as a jpg or png. For example:

nn.predict(argv[idx]);
cv::Mat mat = nn.annotate();
cv::imwrite("output.png", mat, {CV_IMWRITE_PNG_COMPRESSION, 1});

The example call to cv::imwrite() in the previous example might give something similar to this image:

Note that DarkHelp uses OpenCV internally, regardless of whether or not the client code calls DarkHelp::NN::annotate(). This means when you link against the libdarkhelp library you'll also need to link against OpenCV.