DarkHelp C++ API

Table of Contents

• Method #1 (DarkHelp then configure)
• Method #2 (Config then DarkHelp)
• Inference
• Video
• Tracking

The DarkHelp C++ API is a wrapper (not a replacement!) for the libdarknet.so C API.

To use DarkHelp, you must include the project header file within your C++ application:

#include <DarkHelp.hpp>

Method #1 (DarkHelp then configure)

Instantiate a DarkHelp::NN object. These can easily be placed either on the stack, or created dynamically with new. You'll want this object to persist for a long time, as the constructor loads the neural network into memory which takes a (relatively) long time.

const std::string config_file   = argv[1];
const std::string weights_file  = argv[2];
const std::string names_file    = argv[3];
 
DarkHelp::NN nn(config_file, weights_file, names_file);

At this point, the neural network has been fully loaded and is ready to use. But just prior to using it, if you have certain settings you'd like to tweak, now is the time. Several examples:

DarkHelp::NN nn(argv[1], argv[2], argv[3]);
 
nn.config.threshold                     = 0.35;
nn.config.include_all_names             = false;
nn.config.names_include_percentage      = true;
nn.config.annotation_include_duration   = true;
nn.config.annotation_include_timestamp  = false;
nn.config.sort_predictions              = DarkHelp::ESort::kAscending;

Method #2 (Config then DarkHelp)

The alternative method is to instantiate a DarkHelp::Config object first and configuring it as needed. Once it has been setup correctly, then you use it to instantiate the DarkHelp::NN object.

DarkHelp::Config cfg(argv[1], argv[2], argv[3]);
cfg.threshold                       = 0.35;
cfg.include_all_names               = false;
cfg.names_include_percentage        = true;
cfg.annotation_include_duration     = true;
cfg.annotation_include_timestamp    = false;
cfg.sort_predictions                = DarkHelp::ESort::kAscending;
DarkHelp::NN nn(cfg);

This allows the DarkHelp configuration to be passed around if necessary, or have multiple configurations and swapping between them as needed.

Inference

The only thing left is to loop through every image and call DarkHelp::NN::predict(). If you want DarkHelp to annotate the image with the results, you must also call DarkHelp::NN::annotate():

DarkHelp::NN nn(...);
 
for (const auto & filename : get_all_image_names())
{
    // analyze the image and return a vector of structures with all sorts of information
    const auto result = nn.predict(filename);
 
    // get DarkHelp to annotate the image with the most recent results
    cv::Mat output = nn.annotate();
 
    // fictional function to process the results and the annotated image
    handle_image(output, result);
}

Calling any of the DarkHelp::NN::predict() overloads gives back a std::vector of DarkHelp::PredictionResult objects, which should be extremely simple to manage.

Video

While the previous example used image filenames, you can also use cv::Mat objects. Here is an example with cv::Mat images obtained from video frames:

DarkHelp::NN nn(...);
cv::VideoCapture cap("sample.mp4");
while (cap.isOpened())
{
    cv::Mat frame;
    cap >> frame;
    if (frame.empty())
    {
        break;
    }
 
    const auto result = nn.predict(frame);
 
    // display the results as a block of text (not an annotated image)
    std::cout << result << std::endl;
}
cap.release();

For example, the std::cout line in the previous example might results in the following text:

-> prediction results: 2
-> 1/2: "stop sign 100%" #0 prob=0.999795 x=500 y=326 w=253 h=227 entries=1
-> 2/2: "street name 100%" #2 prob=0.999893 x=484 y=231 w=267 h=72 entries=1

But most likely you'll want to handle the result vector yourself, instead of dumping lines of text to std::cout. See DarkHelp::PredictionResult and the various members it provides, such as DarkHelp::PredictionResult::rect and DarkHelp::PredictionResult::all_probabilities.

Tracking

The results from object detection can be passed in to an instance of DarkHelp::PositionTracker. This will attempt to do simple position-based object tracking. The information gathered during object tracking can then be leveraged to draw tails, or uniquely count the objects in a video.

DarkHelp::NN nn("pigs.cfg", "pigs.names", "pigs_best.weights");
DarkHelp::PositionTracker tracker;
 
cv::VideoCapture cap("pigs.mp4");
while (cap.isOpened())
{
    cv::Mat frame;
    cap >> frame;
    if (frame.empty())
    {
        break;
    }
 
    auto results = nn.predict(mat);
    tracker.add(results);
 
    std::cout << results << std::endl;
    std::cout << tracker << std::endl;
 
    for (const auto & prediction : results)
    {
        // get the tracking details for this specific prediction
        const auto & obj = tracker.get(prediction.object_id);
        cv::putText(mat, std::to_string(obj.oid), obj.center(), cv::FONT_HERSHEY_SIMPLEX, 0.75, {0, 0, 0}, 1, cv::LINE_AA);
 
        // use the many rectangles stored in obj.fids_and_rects to draw the tail
        // ...etc...
    }
 
    cv::imshow("output", mat);
    cv::waitKey();
}

See also

DarkHelp::PositionTracker