Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain. More...

Collaboration diagram for DarkHelp::NN:

[legend]

Public Member Functions
	~NN ()
	Destructor. This automatically calls reset() to release memory allocated by the neural network. More...

	NN ()
	Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called. More...

	NN (const Config &cfg)
	Constructor. More...

	NN (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet)
	Constructor. More...

NN &	init (const std::string &cfg_filename, const std::string &weights_filename, const std::string &names_filename="", const bool verify_files_first=true, const EDriver driver=EDriver::kDarknet)
	Initialize ("load") the darknet neural network. More...

NN &	init ()
	Initialize ("load") the darknet neural network. This uses the values within DarkHelp::NN::config. More...

NN &	reset ()
	The opposite of DarkHelp::NN::init(). More...

NN &	clear ()
	Clear out the images and the predictions stored internally within this object. More...

bool	is_initialized () const
	Determines if a neural network has been loaded. More...

bool	is_loaded () const
	Alias for is_initialized(). Returns `true` if the neural network has been loaded. More...

bool	empty () const
	Only returns `true` if both original_image and prediction_results are both empty. More...

PredictionResults	predict (const std::string &image_filename, const float new_threshold=-1.0f)
	Use the neural network to predict what is contained in this image. More...

PredictionResults	predict (cv::Mat mat, const float new_threshold=-1.0f)
	Use the neural network to predict what is contained in this image. More...

PredictionResults	predict_tile (cv::Mat mat, const float new_threshold=-1.0f)
	Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions. More...

cv::Mat	annotate (const float new_threshold=-1.0f)
	Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image. More...

std::string	duration_string ()
	Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation. More...

cv::Size	network_size ()
	Determine the size of the network. For example, 416x416, or 608x480. More...

int	image_channels ()
	Return the number of channels defined in the .cfg file. Usually, this will be `3`. More...

NN &	snap_annotations ()
	Snap all the annotations. More...

NN &	snap_annotation (PredictionResult &pred)
	Snap only the given annotation. More...


	NN (const NN &)=delete
	We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...

	NN (NN &&)=delete
	We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...

NN &	operator= (const NN &)=delete
	We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...

NN &	operator= (NN &&)=delete
	We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU. More...

Static Public Member Functions
static std::string	version ()
	Get a version string for the DarkHelp library. E.g., could be `1.0.0-123`. More...

Public Attributes
void *	darknet_net
	The Darknet network, but stored as a void* pointer so we don't have to include darknet.h. More...

VStr	names
	A vector of names corresponding to the identified classes. More...

std::chrono::high_resolution_clock::duration	duration
	The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed. More...

PredictionResults	prediction_results
	A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict(). More...

cv::Mat	original_image
	The most recent image handled by DarkHelp::NN::predict(). More...

cv::Mat	annotated_image
	The most recent output produced by DarkHelp::NN::annotate(). More...

cv::Mat	binary_inverted_image
	Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled. More...

size_t	horizontal_tiles
	The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More...

size_t	vertical_tiles
	The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict(). More...

cv::Size	tile_size
	The size that was used for each individual tile by DarkHelp::NN::predict_tile(). More...

Config	config
	Configuratin for the neural network. More...

Protected Member Functions
PredictionResults	predict_internal (cv::Mat mat, const float new_threshold=-1.0f)
	Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction. More...

void	predict_internal_darknet ()
	Called from DarkHelp::NN::predict_internal(). More...

void	predict_internal_opencv ()
	Called from DarkHelp::NN::predict_internal(). More...

NN &	name_prediction (PredictionResult &pred)
	Give a consistent name to the given production result. More...

Protected Attributes
cv::Size	network_dimensions
	Size of the neural network, e.g., `416x416` or `608x608`. More...

int	number_of_channels
	The number of channels defined in the .cfg file. This is normally set to `3`. More...

Detailed Description

Instantiate one of these objects by giving it the name of the .cfg and .weights file, then call DarkHelp::NN::predict() as often as necessary to determine what the images contain.

For example:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
 
const auto image_filenames = {"image_0.jpg", "image_1.jpg", "image_2.jpg"};
 
for (const auto & filename : image_filenames)
{
    // these next two lines is where DarkHelp calls into Darknet to do all the hard work
    nn.predict(filename);
    cv::Mat mat = nn.annotate(); // annotates the most recent image seen by predict()
 
    // use standard OpenCV calls to show the image results in a window
    cv::imshow("prediction", mat);
    cv::waitKey();
}

Instead of calling DarkHelp::NN::annotate(), you can get the detection results and iterate through them:

DarkHelp::NN nn("mynetwork.cfg", "mynetwork.weights", "mynetwork.names");
 
const auto results = nn.predict("test_image_01.jpg");
 
for (const auto & det : results)
{
    std::cout << det.name << " (" << 100.0 * det.best_probability << "% chance that this is class #" << det.best_class << ")" << std::endl;
}

Instead of writing your own loop, you can also use the std::ostream operator<<() like this:

const auto results = nn.predict("test_image_01.jpg");

std::cout << results << std::endl;

Constructor & Destructor Documentation

◆ NN() [1/5]

DarkHelp::NN::NN ( const NN & )

delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ NN() [2/5]

DarkHelp::NN::NN ( NN && )

delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ ~NN()

DarkHelp::NN::~NN ( )

Destructor. This automatically calls reset() to release memory allocated by the neural network.

Here is the call graph for this function:

◆ NN() [3/5]

DarkHelp::NN::NN ( )

Constructor. When using this constructor, the neural network remains uninitialized until DarkHelp::NN::init() is called.

Here is the call graph for this function:

◆ NN() [4/5]

DarkHelp::NN::NN ( const Config & cfg )

Constructor.

This constructor automatically calls DarkHelp::NN::init() to load the neural network. The neural network filenames within the cfg object must not be empty otherwise the call to DarkHelp::NN::init() will fail.

Here is the call graph for this function:

◆ NN() [5/5]

DarkHelp::NN::NN	(	const std::string &	cfg_filename,
		const std::string &	weights_filename,
		const std::string &	names_filename = `""`,
		const bool	verify_files_first = `true`,
		const EDriver	driver = `EDriver::kDarknet`
	)

Constructor.

This constructor automatically calls DarkHelp::NN::init() with the given parameters.

Note: The order in which you pass the various filenames is not important if verify_files_first is set to true (the default value). This is because DarkHelp::NN::init() will call DarkHelp::verify_cfg_and_weights() to correctly determine which is the .cfg, .weights, and .names file, and swap the names around as necessary so Darknet is given the correct filenames.

Here is the call graph for this function:

Member Function Documentation

◆ operator=() [1/2]

NN& DarkHelp::NN::operator= ( const NN & )

delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ operator=() [2/2]

NN& DarkHelp::NN::operator= ( NN && )

delete

We know that Darknet and OpenCV DNN does fancy stuff with the GPU and memory allocated to be used by the GPU.

So delete the copying and moving of DarkHelp::NN objects to prevent problems from happening.

◆ version()

static std::string DarkHelp::NN::version ( )

static

Get a version string for the DarkHelp library. E.g., could be 1.0.0-123.

◆ init() [1/2]

DarkHelp::NN & DarkHelp::NN::init	(	const std::string &	cfg_filename,
		const std::string &	weights_filename,
		const std::string &	names_filename = `""`,
		const bool	verify_files_first = `true`,
		const EDriver	driver = `EDriver::kDarknet`
	)

Initialize ("load") the darknet neural network.

If verify_files_first has been enabled (the default) then this method will also call the static method DarkHelp::verify_cfg_and_weights() to perform some last-minute validation prior to darknet loading the neural network.

Here is the call graph for this function:

◆ init() [2/2]

DarkHelp::NN & DarkHelp::NN::init ( )

Initialize ("load") the darknet neural network. This uses the values within DarkHelp::NN::config.

Exceptions

std::invalid_argument	if the .cfg or .weights filenames have not been set.
std::runtime_error	if the call to darknet's `load_network_custom()` has failed.
std::invalid_argument	if there is a blank line in the .names file.
std::invalid_argument	if the network dimensions cannot be read from the .cfg file
std::invalid_argument	if the `channels=`... line in the .cfg file is not 1 or 3

Here is the call graph for this function:

Here is the caller graph for this function:

◆ reset()

DarkHelp::NN & DarkHelp::NN::reset ( )

The opposite of DarkHelp::NN::init().

This is automatically called by the destructor.

See also: clear()

Here is the caller graph for this function:

◆ clear()

DarkHelp::NN & DarkHelp::NN::clear ( )

Clear out the images and the predictions stored internally within this object.

This makes it seem as if the NN object has not yet been used to process any images. Unlike reset(), this does not change any settings, it only clears the images and the predictions. If a neural network has been loaded, calling clear() does not unload that neural network, and is_initialized() will continue to return true.

Calling this method between images is not necessary, but is included for completeness.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ is_initialized()

bool DarkHelp::NN::is_initialized ( ) const

Determines if a neural network has been loaded.

For example:

When a NN object has been created with the default constructor, is_initialized() returns false.
Once init() has been called, is_initialized() returns true.
After clear() has been called, is_initialized() would continue to return true.
After reset() has been called, is_initialized() returns false.

See also: empty(); clear()

Since: 2022-07-12

Here is the caller graph for this function:

◆ is_loaded()

bool DarkHelp::NN::is_loaded ( ) const

inline

Alias for is_initialized(). Returns true if the neural network has been loaded.

Here is the call graph for this function:

◆ empty()

bool DarkHelp::NN::empty ( ) const

Only returns true if both original_image and prediction_results are both empty.

This will only happen in the following situations:

when a NN object has been instantiated but init() has not yet been called and no neural network is loaded
when a NN object is initialized but no image has been processed
after clear() has been called which results in the removal of internal images and predictions

See also: is_initialized(); clear()

◆ predict() [1/2]

DarkHelp::PredictionResults DarkHelp::NN::predict	(	const std::string &	image_filename,
		const float	new_threshold = `-1.0f`
	)

Use the neural network to predict what is contained in this image.

This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.

Parameters

[in]	image_filename	The name of the image file to load from disk and analyze. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio.
[in]	new_threshold	Which threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.

Returns: A vector of DarkHelp::PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.

See also: Image Tiling; DarkHelp::NN::predict_tile(); DarkHelp::PredictionResult; DarkHelp::Config::enable_tiles; DarkHelp::Config::sort_predictions; DarkHelp::NN::duration

Exceptions

std::invalid_argument if the image failed to load.

◆ predict() [2/2]

DarkHelp::PredictionResults DarkHelp::NN::predict	(	cv::Mat	mat,
		const float	new_threshold = `-1.0f`
	)

Use the neural network to predict what is contained in this image.

This results in a call to either DarkHelp::NN::predict_internal() or DarkHelp::NN::predict_tile() depending on how DarkHelp::Config::enable_tiles has been set.

Parameters

[in]	mat	A OpenCV2 image which has already been loaded and which needs to be analyzed. The member DarkHelp::NN::original_image will be set to this image. If the image is larger or smaller than the dimensions of the neural network, then Darknet will stretch the image to match the exact size of the neural network. Stretching the image does not maintain the the aspect ratio.
[in]	new_threshold	Which threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value. The threshold must be either -1, or a value between 0.0 and 1.0 meaning 0% to 100%.

Returns: A vector of DarkHelp::PredictionResult structures, each one representing a different object in the image. The higher the threshold value, the more "certain" the network is that it has correctly identified the object.

See also: Image Tiling; DarkHelp::NN::predict_tile(); DarkHelp::PredictionResult; DarkHelp::Config::enable_tiles; DarkHelp::Config::sort_predictions; DarkHelp::NN::duration

Exceptions

std::invalid_argument if the image is empty.

◆ predict_tile()

DarkHelp::PredictionResults DarkHelp::NN::predict_tile	(	cv::Mat	mat,
		const float	new_threshold = `-1.0f`
	)

Similar to DarkHelp::NN::predict(), but automatically breaks the images down into individual tiles if it is significantly larger than the network dimensions.

This is explained in details in Image Tiling.

Note: The method DarkHelp::NN::predict() will automatically call DarkHelp::NN::predict_tile() if necessary when DarkHelp::Config::enable_tiles has been enabled. If you don't want to use image tiling, then DarkHelp::Config::enable_tiles must be set to false (which is the default).

Here is a visual representation of a large image broken into 4 tiles for processing by Darknet. It is important to understand that neither the individual image tiles nor their results are returned to the caller. DarkHelp only returns the final results once each tile has been processed and the vectors have been merged together.

See also: Image Tiling; DarkHelp::NN::predict()

Exceptions

std::invalid_argument if the image is empty.

◆ annotate()

cv::Mat DarkHelp::NN::annotate ( const float new_threshold = -1.0f )

Takes the most recent DarkHelp::NN::prediction_results, and applies them to the most recent DarkHelp::NN::original_image.

The output annotated image is stored in DarkHelp::NN::annotated_image as well as returned to the caller.

This is an example of what an annotated image looks like:

Parameters

[in] new_threshold Which threshold to use. If less than zero, the previous threshold will be applied. If >= 0, then DarkHelp::Config::threshold will be set to this new value.

Turning down the threshold in DarkHelp::NN::annotate() wont bring back predictions that were excluded due to a higher threshold originally used with DarkHelp::NN::predict(). Here is an example:

nn.predict("image.jpg", 0.75); // note the threshold is set to 75% for prediction

nn.annotate(0.25); // note the threshold is now modified to be 25%

In the previous example, when annotate() is called with the lower threshold of 25%, the predictions had already been capped at 75%. This means any prediction between >= 25% and < 75% were excluded from the prediction results. The only way to get those predictions is to re-run predict() with a value of 0.25.

Note: Annotations wont be drawn if DarkHelp::Config::annotation_line_thickness is less than 1.

See also: DarkHelp::Config::annotation_colours; DarkHelp::Config::annotation_font_scale; DarkHelp::Config::annotation_font_thickness; DarkHelp::Config::annotation_line_thickness; DarkHelp::Config::annotation_include_duration; DarkHelp::Config::annotation_include_timestamp; DarkHelp::Config::annotation_auto_hide_labels; DarkHelp::Config::annotation_shade_predictions; DarkHelp::Config::annotation_suppress_classes

Exceptions

std::logic_error if an attempt is made to annotate an empty image

Here is the call graph for this function:

◆ duration_string()

std::string DarkHelp::NN::duration_string ( )

Return DarkHelp::NN::duration as a text string which can then be added to the image during annotation.

For example, this might return "912 microseconds" or "375 milliseconds".

See also: DarkHelp::NN::annotate()

Here is the call graph for this function:

◆ network_size()

cv::Size DarkHelp::NN::network_size ( )

Determine the size of the network. For example, 416x416, or 608x480.

◆ image_channels()

int DarkHelp::NN::image_channels ( )

Return the number of channels defined in the .cfg file. Usually, this will be 3.

◆ snap_annotations()

DarkHelp::NN & DarkHelp::NN::snap_annotations ( )

Snap all the annotations.

This is automatically called from predict() when DarkHelp::Config::snapping_enabled is set to true. When set to false, you can manually invoke this method to get the annotations to snap, or you can also manually call DarkHelp::NN::snap_annotation() on specific annotations as needed.

Note: This can be expensive to run depending on the image dimensions, the image threshold limits, and the amount of "snapping" required for each annotation since the process of "snapping" is iterative and requires looking for blank spaces within the image.

See also: DarkHelp::Config::snapping_enabled; DarkHelp::Config::snapping_horizontal_tolerance

Image	Setting
	`snapping_enabled=false`
	`snapping_enabled=true`

◆ snap_annotation()

DarkHelp::NN & DarkHelp::NN::snap_annotation ( DarkHelp::PredictionResult & pred )

Snap only the given annotation.

See also: DarkHelp::NN::snap_annotations(); DarkHelp::Config::snapping_enabled; DarkHelp::Config::snapping_horizontal_tolerance

◆ predict_internal()

DarkHelp::PredictionResults DarkHelp::NN::predict_internal	(	cv::Mat	mat,
		const float	new_threshold = `-1.0f`
	)

protected

Used by all the other DarkHelp::NN::predict() calls to do the actual network prediction.

This uses the image stored in DarkHelp::NN::original_image.

Exceptions

std::logic_error	if the DarkHelp object has not been initialized.
std::logic_error	if the network is invalid.
std::logic_error	if the image is invalid.

◆ predict_internal_darknet()

void DarkHelp::NN::predict_internal_darknet ( )

protected

Called from DarkHelp::NN::predict_internal().

See also: DarkHelp::NN::predict()

Here is the call graph for this function:

◆ predict_internal_opencv()

void DarkHelp::NN::predict_internal_opencv ( )

protected

Called from DarkHelp::NN::predict_internal().

See also: DarkHelp::NN::predict()

Here is the call graph for this function:

◆ name_prediction()

DarkHelp::NN & DarkHelp::NN::name_prediction ( PredictionResult & pred )

protected

Give a consistent name to the given production result.

This gets called by both DarkHelp::NN::predict_internal() and DarkHelp::NN::predict_tile() and is intended for internal use only.

Member Data Documentation

◆ darknet_net

void* DarkHelp::NN::darknet_net

The Darknet network, but stored as a void* pointer so we don't have to include darknet.h.

This will only be set when the driver is DarkHelp::EDriver::kDarknet in DarkHelp::NN::init().

◆ names

VStr DarkHelp::NN::names

A vector of names corresponding to the identified classes.

This is typically setup in the constructor, but can be manually set afterwards.

◆ duration

std::chrono::high_resolution_clock::duration DarkHelp::NN::duration

The length of time it took to initially load the network and weights (after the DarkHelp object has been constructed), or the length of time DarkHelp::NN::predict() took to run on the last image to be processed.

If using DarkHelp::NN::predict_tile(), then this will store the sum of all durations across the entire set of tiles.

See also: DarkHelp::NN::duration_string()

◆ prediction_results

PredictionResults DarkHelp::NN::prediction_results

A copy of the most recent results after applying the neural network to an image. This is set by DarkHelp::NN::predict().

◆ original_image

cv::Mat DarkHelp::NN::original_image

The most recent image handled by DarkHelp::NN::predict().

◆ annotated_image

cv::Mat DarkHelp::NN::annotated_image

The most recent output produced by DarkHelp::NN::annotate().

◆ binary_inverted_image

cv::Mat DarkHelp::NN::binary_inverted_image

Intended mostly for internal purpose, this is only useful when annotation "snapping" is enabled.

◆ horizontal_tiles

size_t DarkHelp::NN::horizontal_tiles

The number of horizontal tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().

This is set to 1 if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.

See also: DarkHelp::NN::vertical_tiles; Image Tiling; DarkHelp::Config::enable_tiles

◆ vertical_tiles

size_t DarkHelp::NN::vertical_tiles

The number of vertical tiles the image was split into by DarkHelp::NN::predict_tile() prior to calling DarkHelp::NN::predict().

This is set to 1 if calling DarkHelp::NN::predict(). It may be > 1 if calling DarkHelp::NN::predict_tile() with an image large enough to require multiple tiles.

See also: DarkHelp::NN::horizontal_tiles; Image Tiling; DarkHelp::Config::enable_tiles

◆ tile_size

cv::Size DarkHelp::NN::tile_size

The size that was used for each individual tile by DarkHelp::NN::predict_tile().

This will be the size of the network when calling DarkHelp::NN::predict().

For example, if the network is 416x416, and the image used with DarkHelp::NN::predict_tile() measures 1280x960, then:

DarkHelp::NN::horizontal_tiles will be set to 3
DarkHelp::NN::vertical_tiles will be set to 2
DarkHelp::NN::tile_size will be set to "(427, 480)"

See also: Image Tiling; DarkHelp::Config::enable_tiles

◆ config

Config DarkHelp::NN::config

Configuratin for the neural network.

This includes both settings for the neural network itself and everything needed to annotate images/frames.

◆ network_dimensions

cv::Size DarkHelp::NN::network_dimensions

protected

Size of the neural network, e.g., 416x416 or 608x608.

See also: DarkHelp::NN::network_size()

◆ number_of_channels

int DarkHelp::NN::number_of_channels

protected

The number of channels defined in the .cfg file. This is normally set to 3.

See also: image_channels()

The documentation for this class was generated from the following files:

DarkHelpNN.hpp
DarkHelpNN.cpp

Public Member Functions

Static Public Member Functions

Public Attributes

Protected Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ NN() [1/5]

◆ NN() [2/5]

◆ ~NN()

◆ NN() [3/5]

◆ NN() [4/5]

◆ NN() [5/5]

Member Function Documentation

◆ operator=() [1/2]

◆ operator=() [2/2]

◆ version()

◆ init() [1/2]

◆ init() [2/2]

◆ reset()

◆ clear()

◆ is_initialized()

◆ is_loaded()

◆ empty()

◆ predict() [1/2]

◆ predict() [2/2]

◆ predict_tile()

◆ annotate()

◆ duration_string()

◆ network_size()

◆ image_channels()

◆ snap_annotations()

◆ snap_annotation()

◆ predict_internal()

◆ predict_internal_darknet()

◆ predict_internal_opencv()

◆ name_prediction()

Member Data Documentation

◆ darknet_net

◆ names

◆ duration

◆ prediction_results

◆ original_image

◆ annotated_image

◆ binary_inverted_image

◆ horizontal_tiles

◆ vertical_tiles

◆ tile_size

◆ config

◆ network_dimensions

◆ number_of_channels