TensorRT  7.2.1.6
NVIDIA TensorRT
Looking for a C++ dev who knows TensorRT?
I'm looking for work. Hire me!
nvinfer1 Namespace Reference

The TensorRT API version 1 namespace. More...

Namespaces

 anonymous_namespace{NvInfer.h}
 
 anonymous_namespace{NvInferRuntime.h}
 
 plugin
 
 utility
 
 utils
 

Classes

class  CUDADriverWrapper
 
class  Dims
 Structure to define the dimensions of a tensor. More...
 
class  Dims2
 Descriptor for two-dimensional data. More...
 
class  Dims3
 Descriptor for three-dimensional data. More...
 
class  Dims4
 Descriptor for four-dimensional data. More...
 
class  DimsExprs
 
class  DimsHW
 Descriptor for two-dimensional spatial data. More...
 
class  DynamicPluginTensorDesc
 
class  IActivationLayer
 An Activation layer in a network definition. More...
 
class  IAlgorithm
 Describes a variation of execution of a layer. An algorithm is represented by IAlgorithmVariant and the IAlgorithmIOInfo for each of its inputs and outputs. An algorithm can be selected or reproduced using AlgorithmSelector::selectAlgorithms().". More...
 
class  IAlgorithmContext
 Describes the context and requirements, that could be fulfilled by one or more instances of IAlgorithm. More...
 
class  IAlgorithmIOInfo
 Carries information about input or output of the algorithm. IAlgorithmIOInfo for all the input and output along with IAlgorithmVariant denotes the variation of algorithm and can be used to select or reproduce an algorithm using IAlgorithmSelector::selectAlgorithms(). More...
 
class  IAlgorithmSelector
 Interface implemented by application for selecting and reporting algorithms of a layer provided by the builder. More...
 
class  IAlgorithmVariant
 provides a unique 128-bit identifier, which along with the input and output information denotes the variation of algorithm and can be used to select or reproduce an algorithm, using IAlgorithmSelector::selectAlgorithms() More...
 
class  IBuilder
 Builds an engine from a network definition. More...
 
class  IBuilderConfig
 Holds properties for configuring a builder to produce an engine. More...
 
class  IConcatenationLayer
 A concatenation layer in a network definition. More...
 
class  IConstantLayer
 Layer that represents a constant value. More...
 
class  IConvolutionLayer
 A convolution layer in a network definition. More...
 
class  ICudaEngine
 An engine for executing inference on a built network, with functionally unsafe features. More...
 
class  IDeconvolutionLayer
 A deconvolution layer in a network definition. More...
 
class  IDimensionExpr
 
class  IElementWiseLayer
 A elementwise layer in a network definition. More...
 
class  IErrorRecorder
 Reference counted application-implemented error reporting interface for TensorRT objects. More...
 
class  IExecutionContext
 Context for executing inference using an engine, with functionally unsafe features. More...
 
class  IExprBuilder
 
class  IFillLayer
 Generate an output tensor with specified mode. More...
 
class  IFullyConnectedLayer
 A fully connected layer in a network definition. This layer expects an input tensor of three or more non-batch dimensions. The input is automatically reshaped into an MxV tensor X, where V is a product of the last three dimensions and M is a product of the remaining dimensions (where the product over 0 dimensions is defined as 1). For example: More...
 
class  IGatherLayer
 
class  IGpuAllocator
 Application-implemented class for controlling allocation on the GPU. More...
 
class  IHostMemory
 Class to handle library allocated memory that is accessible to the user. More...
 
class  IIdentityLayer
 A layer that represents the identity function. More...
 
class  IInt8Calibrator
 Application-implemented interface for calibration. More...
 
class  IInt8EntropyCalibrator
 Entropy calibrator. More...
 
class  IInt8EntropyCalibrator2
 Entropy calibrator 2. More...
 
class  IInt8LegacyCalibrator
 Legacy calibrator left for backward compatibility with TensorRT 2.0. More...
 
class  IInt8MinMaxCalibrator
 MinMax Calibrator. More...
 
class  IIteratorLayer
 
class  ILayer
 Base class for all layer classes in a network definition. More...
 
class  ILogger
 Application-implemented logging interface for the builder, engine and runtime. More...
 
class  ILoop
 Helper for creating a recurrent subgraph. More...
 
class  ILoopBoundaryLayer
 
class  ILoopOutputLayer
 An ILoopOutputLayer is the sole way to get output from a loop. More...
 
class  ILRNLayer
 A LRN layer in a network definition. More...
 
class  IMatrixMultiplyLayer
 Layer that represents a Matrix Multiplication. More...
 
class  INetworkDefinition
 A network definition for input to the builder. More...
 
class  IOptimizationProfile
 Optimization profile for dynamic input dimensions and shape tensors. More...
 
class  IPaddingLayer
 Layer that represents a padding operation. More...
 
class  IParametricReLULayer
 Layer that represents a parametric ReLU operation. More...
 
class  IPlugin
 Plugin class for user-implemented layers. More...
 
class  IPluginCreator
 Plugin creator class for user implemented layers. More...
 
class  IPluginExt
 Plugin class for user-implemented layers. More...
 
class  IPluginFactory
 Plugin factory for deserialization. More...
 
class  IPluginRegistry
 Single registration point for all plugins in an application. It is used to find plugin implementations during engine deserialization. Internally, the plugin registry is considered to be a singleton so all plugins in an application are part of the same global registry. Note that the plugin registry is only supported for plugins of type IPluginV2 and should also have a corresponding IPluginCreator implementation. More...
 
class  IPluginV2
 Plugin class for user-implemented layers. More...
 
class  IPluginV2DynamicExt
 
class  IPluginV2Ext
 Plugin class for user-implemented layers. More...
 
class  IPluginV2IOExt
 Plugin class for user-implemented layers. More...
 
class  IPluginV2Layer
 Layer type for pluginV2. More...
 
class  IPoolingLayer
 A Pooling layer in a network definition. More...
 
class  IProfiler
 Application-implemented interface for profiling. More...
 
class  IRaggedSoftMaxLayer
 A RaggedSoftmax layer in a network definition. More...
 
class  IRecurrenceLayer
 
class  IReduceLayer
 Layer that represents a reduction operator across Shape, Int32, Float, and Half tensors. More...
 
class  IRefitter
 Updates weights in an engine. More...
 
class  IResizeLayer
 A resize layer in a network definition. More...
 
class  IRuntime
 Allows a serialized functionally unsafe engine to be deserialized. More...
 
class  IScaleLayer
 A Scale layer in a network definition. More...
 
class  ISelectLayer
 
class  IShapeLayer
 Layer type for getting shape of a tensor. More...
 
class  IShuffleLayer
 Layer type for shuffling data. More...
 
class  ISliceLayer
 Slices an input tensor into an output tensor based on the offset and strides. More...
 
class  ISoftMaxLayer
 A Softmax layer in a network definition. More...
 
class  ITensor
 A tensor in a network definition. More...
 
class  ITopKLayer
 Layer that represents a TopK reduction. More...
 
class  ITripLimitLayer
 
class  IUnaryLayer
 Layer that represents an unary operation. More...
 
struct  Permutation
 
class  PluginField
 Structure containing plugin attribute field names and associated data This information can be parsed to decode necessary plugin metadata. More...
 
struct  PluginFieldCollection
 
class  PluginRegistrar
 Register the plugin creator to the registry The static registry object will be instantiated when the plugin library is loaded. More...
 
struct  PluginTensorDesc
 Fields that a plugin might see for an input or output. More...
 
class  Weights
 An array of weights used as a layer parameter. More...
 

Typedefs

typedef uint32_t QuantizationFlags
 Represents a collection of one or more QuantizationFlag values using binary OR operations. More...
 
typedef uint32_t BuilderFlags
 Represents a collection of one or more QuantizationFlag values using binary OR operations, e.g., 1U << BuilderFlag::kFP16 | 1U << BuilderFlag::kDEBUG. More...
 
using TacticSources = uint32_t
 Represents a collection of one or more TacticSource values combine using bitwise-OR operations. More...
 
typedef uint32_t NetworkDefinitionCreationFlags
 This bitset is capable of representing one or more NetworkDefinitionCreationFlag flags constructed with binary OR operations. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH. More...
 
typedef uint32_t TensorFormats
 It is capable of representing one or more TensorFormat by binary OR operations, e.g., 1U << TensorFormats::kCHW4 | 1U << TensorFormats::kCHW32. More...
 
using PluginFormat = TensorFormat
 PluginFormat is reserved for backward compatibility. More...
 

Enumerations

enum  LayerType : int32_t {
  LayerType::kCONVOLUTION = 0,
  LayerType::kFULLY_CONNECTED = 1,
  LayerType::kACTIVATION = 2,
  LayerType::kPOOLING = 3,
  LayerType::kLRN = 4,
  LayerType::kSCALE = 5,
  LayerType::kSOFTMAX = 6,
  LayerType::kDECONVOLUTION = 7,
  LayerType::kCONCATENATION = 8,
  LayerType::kELEMENTWISE = 9,
  LayerType::kPLUGIN = 10,
  LayerType::kRNN = 11,
  LayerType::kUNARY = 12,
  LayerType::kPADDING = 13,
  LayerType::kSHUFFLE = 14,
  LayerType::kREDUCE = 15,
  LayerType::kTOPK = 16,
  LayerType::kGATHER = 17,
  LayerType::kMATRIX_MULTIPLY = 18,
  LayerType::kRAGGED_SOFTMAX = 19,
  LayerType::kCONSTANT = 20,
  LayerType::kRNN_V2 = 21,
  LayerType::kIDENTITY = 22,
  LayerType::kPLUGIN_V2 = 23,
  LayerType::kSLICE = 24,
  LayerType::kSHAPE = 25,
  LayerType::kPARAMETRIC_RELU = 26,
  LayerType::kRESIZE = 27,
  LayerType::kTRIP_LIMIT = 28,
  LayerType::kRECURRENCE = 29,
  LayerType::kITERATOR = 30,
  LayerType::kLOOP_OUTPUT = 31,
  LayerType::kSELECT = 32,
  LayerType::kFILL = 33
}
 The type values of layer classes. More...
 
enum  PaddingMode : int32_t {
  PaddingMode::kEXPLICIT_ROUND_DOWN = 0,
  PaddingMode::kEXPLICIT_ROUND_UP = 1,
  PaddingMode::kSAME_UPPER = 2,
  PaddingMode::kSAME_LOWER = 3,
  PaddingMode::kCAFFE_ROUND_DOWN = 4,
  PaddingMode::kCAFFE_ROUND_UP = 5
}
 Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode takes precedence if setPaddingMode() and setPrePadding() are also used. More...
 
enum  PoolingType : int32_t {
  PoolingType::kMAX = 0,
  PoolingType::kAVERAGE = 1,
  PoolingType::kMAX_AVERAGE_BLEND = 2
}
 The type of pooling to perform in a pooling layer. More...
 
enum  ScaleMode : int32_t {
  ScaleMode::kUNIFORM = 0,
  ScaleMode::kCHANNEL = 1,
  ScaleMode::kELEMENTWISE = 2
}
 Controls how shift, scale and power are applied in a Scale layer. More...
 
enum  ElementWiseOperation : int32_t {
  ElementWiseOperation::kSUM = 0,
  ElementWiseOperation::kPROD = 1,
  ElementWiseOperation::kMAX = 2,
  ElementWiseOperation::kMIN = 3,
  ElementWiseOperation::kSUB = 4,
  ElementWiseOperation::kDIV = 5,
  ElementWiseOperation::kPOW = 6,
  ElementWiseOperation::kFLOOR_DIV = 7,
  ElementWiseOperation::kAND = 8,
  ElementWiseOperation::kOR = 9,
  ElementWiseOperation::kXOR = 10,
  ElementWiseOperation::kEQUAL = 11,
  ElementWiseOperation::kGREATER = 12,
  ElementWiseOperation::kLESS = 13
}
 Enumerates the binary operations that may be performed by an ElementWise layer. More...
 
enum  RNNOperation : int32_t {
  RNNOperation::kRELU = 0,
  RNNOperation::kTANH = 1,
  RNNOperation::kLSTM = 2,
  RNNOperation::kGRU = 3
}
 Enumerates the RNN operations that may be performed by an RNN layer. More...
 
enum  RNNDirection : int32_t {
  RNNDirection::kUNIDIRECTION = 0,
  RNNDirection::kBIDIRECTION = 1
}
 Enumerates the RNN direction that may be performed by an RNN layer. More...
 
enum  RNNInputMode : int32_t {
  RNNInputMode::kLINEAR = 0,
  RNNInputMode::kSKIP = 1
}
 Enumerates the RNN input modes that may occur with an RNN layer. More...
 
enum  RNNGateType : int32_t {
  RNNGateType::kINPUT = 0,
  RNNGateType::kOUTPUT = 1,
  RNNGateType::kFORGET = 2,
  RNNGateType::kUPDATE = 3,
  RNNGateType::kRESET = 4,
  RNNGateType::kCELL = 5,
  RNNGateType::kHIDDEN = 6
}
 Identifies an individual gate within an RNN cell. More...
 
enum  UnaryOperation : int32_t {
  UnaryOperation::kEXP = 0,
  UnaryOperation::kLOG = 1,
  UnaryOperation::kSQRT = 2,
  UnaryOperation::kRECIP = 3,
  UnaryOperation::kABS = 4,
  UnaryOperation::kNEG = 5,
  UnaryOperation::kSIN = 6,
  UnaryOperation::kCOS = 7,
  UnaryOperation::kTAN = 8,
  UnaryOperation::kSINH = 9,
  UnaryOperation::kCOSH = 10,
  UnaryOperation::kASIN = 11,
  UnaryOperation::kACOS = 12,
  UnaryOperation::kATAN = 13,
  UnaryOperation::kASINH = 14,
  UnaryOperation::kACOSH = 15,
  UnaryOperation::kATANH = 16,
  UnaryOperation::kCEIL = 17,
  UnaryOperation::kFLOOR = 18,
  UnaryOperation::kERF = 19,
  UnaryOperation::kNOT = 20
}
 Enumerates the unary operations that may be performed by a Unary layer. More...
 
enum  ReduceOperation : int32_t {
  ReduceOperation::kSUM = 0,
  ReduceOperation::kPROD = 1,
  ReduceOperation::kMAX = 2,
  ReduceOperation::kMIN = 3,
  ReduceOperation::kAVG = 4
}
 Enumerates the reduce operations that may be performed by a Reduce layer. More...
 
enum  SliceMode : int32_t {
  SliceMode::kDEFAULT = 0,
  SliceMode::kWRAP = 1
}
 Controls how ISliceLayer handles out of bounds coordinates. More...
 
enum  TopKOperation : int32_t {
  TopKOperation::kMAX = 0,
  TopKOperation::kMIN = 1
}
 Enumerates the operations that may be performed by a TopK layer. More...
 
enum  MatrixOperation : int32_t {
  MatrixOperation::kNONE,
  MatrixOperation::kTRANSPOSE,
  MatrixOperation::kVECTOR
}
 Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication. More...
 
enum  ResizeMode : int32_t {
  ResizeMode::kNEAREST = 0,
  ResizeMode::kLINEAR = 1
}
 Enumerates various modes of resize in the resize layer. Resize mode set using setResizeMode(). More...
 
enum  LoopOutput : int32_t {
  LoopOutput::kLAST_VALUE = 0,
  LoopOutput::kCONCATENATE = 1,
  LoopOutput::kREVERSE = 2
}
 Enum that describes kinds of loop outputs. More...
 
enum  TripLimit : int32_t {
  TripLimit::kCOUNT = 0,
  TripLimit::kWHILE = 1
}
 Enum that describes kinds of trip limits. More...
 
enum  FillOperation : int32_t {
  FillOperation::kLINSPACE = 0,
  FillOperation::kRANDOM_UNIFORM = 1
}
 Enumerates the tensor fill operations that may performed by a fill layer. More...
 
enum  CalibrationAlgoType : int32_t {
  CalibrationAlgoType::kLEGACY_CALIBRATION = 0,
  CalibrationAlgoType::kENTROPY_CALIBRATION = 1,
  CalibrationAlgoType::kENTROPY_CALIBRATION_2 = 2,
  CalibrationAlgoType::kMINMAX_CALIBRATION = 3
}
 enum CalibrationAlgoType More...
 
enum  QuantizationFlag : int32_t { QuantizationFlag::kCALIBRATE_BEFORE_FUSION = 0 }
 List of valid flags for quantizing the network to int8. More...
 
enum  BuilderFlag : int32_t {
  BuilderFlag::kFP16 = 0,
  BuilderFlag::kINT8 = 1,
  BuilderFlag::kDEBUG = 2,
  BuilderFlag::kGPU_FALLBACK = 3,
  BuilderFlag::kSTRICT_TYPES = 4,
  BuilderFlag::kREFIT = 5,
  BuilderFlag::kDISABLE_TIMING_CACHE = 6,
  BuilderFlag::kTF32 = 7
}
 List of valid modes that the builder can enable when creating an engine from a network definition. More...
 
enum  ProfilingVerbosity : int32_t {
  ProfilingVerbosity::kDEFAULT = 0,
  ProfilingVerbosity::kNONE = 1,
  ProfilingVerbosity::kVERBOSE = 2
}
 List of verbosity levels of layer information exposed in NVTX annotations. More...
 
enum  TacticSource : int32_t {
  TacticSource::kCUBLAS = 0,
  TacticSource::kCUBLAS_LT = 1
}
 List of tactic sources for TensorRT. More...
 
enum  NetworkDefinitionCreationFlag : int32_t {
  NetworkDefinitionCreationFlag::kEXPLICIT_BATCH = 0,
  NetworkDefinitionCreationFlag::kEXPLICIT_PRECISION = 1
}
 List of immutable network properties expressed at network creation time. NetworkDefinitionCreationFlag is used with createNetworkV2 to specify immutable properties of the network. The createNetwork() function always had an implicit batch dimension being specified by the maxBatchSize builder parameter. createNetworkV2 with kDEFAULT flag mimics that behaviour. More...
 
enum  PluginType : int32_t {
  PluginType::kFASTERRCNN = 0,
  PluginType::kNORMALIZE = 1,
  PluginType::kPERMUTE = 2,
  PluginType::kPRIORBOX = 3,
  PluginType::kSSDDETECTIONOUTPUT = 4,
  PluginType::kCONCAT = 5,
  PluginType::kPRELU = 6,
  PluginType::kYOLOREORG = 7,
  PluginType::kYOLOREGION = 8,
  PluginType::kANCHORGENERATOR = 9
}
 The type values for the various plugins. More...
 
enum  EngineCapability : int32_t {
  EngineCapability::kDEFAULT = 0,
  EngineCapability::kSAFE_GPU = 1,
  EngineCapability::kSAFE_DLA = 2
}
 Forward declaration of IPluginFactory for use by other interfaces. More...
 
enum  DimensionOperation : int32_t {
  DimensionOperation::kSUM = 0,
  DimensionOperation::kPROD = 1,
  DimensionOperation::kMAX = 2,
  DimensionOperation::kMIN = 3,
  DimensionOperation::kSUB = 4,
  DimensionOperation::kEQUAL = 5,
  DimensionOperation::kLESS = 6,
  DimensionOperation::kFLOOR_DIV = 7,
  DimensionOperation::kCEIL_DIV = 8
}
 An operation on two IDimensionExpr, which represent integer expressions used in dimension computations. More...
 
enum  WeightsRole : int32_t {
  WeightsRole::kKERNEL = 0,
  WeightsRole::kBIAS = 1,
  WeightsRole::kSHIFT = 2,
  WeightsRole::kSCALE = 3,
  WeightsRole::kCONSTANT = 4
}
 How a layer uses particular Weights. More...
 
enum  DeviceType : int32_t {
  DeviceType::kGPU,
  DeviceType::kDLA
}
 The device that this layer/network will execute on. More...
 
enum  OptProfileSelector : int32_t {
  OptProfileSelector::kMIN = 0,
  OptProfileSelector::kOPT = 1,
  OptProfileSelector::kMAX = 2
}
 When setting or querying optimization profile parameters (such as shape tensor inputs or dynamic dimensions), select whether we are interested in the minimum, optimum, or maximum values for these parameters. The minimum and maximum specify the permitted range that is supported at runtime, while the optimum value is used for the kernel selection. This should be the "typical" value that is expected to occur at runtime. More...
 
enum  ActivationType : int32_t {
  ActivationType::kRELU = 0,
  ActivationType::kSIGMOID = 1,
  ActivationType::kTANH = 2,
  ActivationType::kLEAKY_RELU = 3,
  ActivationType::kELU = 4,
  ActivationType::kSELU = 5,
  ActivationType::kSOFTSIGN = 6,
  ActivationType::kSOFTPLUS = 7,
  ActivationType::kCLIP = 8,
  ActivationType::kHARD_SIGMOID = 9,
  ActivationType::kSCALED_TANH = 10,
  ActivationType::kTHRESHOLDED_RELU = 11
}
 Enumerates the types of activation to perform in an activation layer. More...
 
enum  DataType : int32_t {
  DataType::kFLOAT = 0,
  DataType::kHALF = 1,
  DataType::kINT8 = 2,
  DataType::kINT32 = 3,
  DataType::kBOOL = 4
}
 The type of weights and tensors. More...
 
enum  DimensionType : int32_t {
  DimensionType::kSPATIAL = 0,
  DimensionType::kCHANNEL = 1,
  DimensionType::kINDEX = 2,
  DimensionType::kSEQUENCE = 3
}
 The type of data encoded across this dimension. More...
 
enum  TensorFormat : int32_t {
  TensorFormat::kLINEAR = 0,
  TensorFormat::kNCHW = kLINEAR,
  TensorFormat::kCHW2 = 1,
  TensorFormat::kNC2HW2 = kCHW2,
  TensorFormat::kHWC8 = 2,
  TensorFormat::kNHWC8 = kHWC8,
  TensorFormat::kCHW4 = 3,
  TensorFormat::kCHW16 = 4,
  TensorFormat::kCHW32 = 5,
  TensorFormat::kDHWC8 = 6,
  TensorFormat::kCDHW32 = 7,
  TensorFormat::kHWC = 8,
  TensorFormat::kDLA_LINEAR = 9,
  TensorFormat::kDLA_HWC4 = 10
}
 Format of the input/output tensors. More...
 
enum  PluginVersion : uint8_t {
  PluginVersion::kV2 = 0,
  PluginVersion::kV2_EXT = 1,
  PluginVersion::kV2_IOEXT = 2,
  PluginVersion::kV2_DYNAMICEXT = 3
}
 
enum  PluginFieldType : int32_t {
  PluginFieldType::kFLOAT16 = 0,
  PluginFieldType::kFLOAT32 = 1,
  PluginFieldType::kFLOAT64 = 2,
  PluginFieldType::kINT8 = 3,
  PluginFieldType::kINT16 = 4,
  PluginFieldType::kINT32 = 5,
  PluginFieldType::kCHAR = 6,
  PluginFieldType::kDIMS = 7,
  PluginFieldType::kUNKNOWN = 8
}
 
enum  TensorLocation : int32_t {
  TensorLocation::kDEVICE = 0,
  TensorLocation::kHOST = 1
}
 The location for tensor data storage, device or host. More...
 
enum  ErrorCode : int32_t {
  ErrorCode::kSUCCESS = 0,
  ErrorCode::kUNSPECIFIED_ERROR = 1,
  ErrorCode::kINTERNAL_ERROR = 2,
  ErrorCode::kINVALID_ARGUMENT = 3,
  ErrorCode::kINVALID_CONFIG = 4,
  ErrorCode::kFAILED_ALLOCATION = 5,
  ErrorCode::kFAILED_INITIALIZATION = 6,
  ErrorCode::kFAILED_EXECUTION = 7,
  ErrorCode::kFAILED_COMPUTATION = 8,
  ErrorCode::kINVALID_STATE = 9,
  ErrorCode::kUNSUPPORTED_STATE = 10
}
 Error codes that can be returned by TensorRT during execution. More...
 

Functions

template<>
constexpr int32_t EnumMax< LayerType > ()
 Maximum number of elements in LayerType enum. More...
 
template<>
constexpr int32_t EnumMax< PaddingMode > ()
 Maximum number of elements in PaddingMode enum. More...
 
template<>
constexpr int32_t EnumMax< PoolingType > ()
 Maximum number of elements in PoolingType enum. More...
 
template<>
constexpr int32_t EnumMax< ScaleMode > ()
 Maximum number of elements in ScaleMode enum. More...
 
template<>
constexpr int32_t EnumMax< ElementWiseOperation > ()
 Maximum number of elements in ElementWiseOperation enum. More...
 
template<>
constexpr int32_t EnumMax< RNNOperation > ()
 Maximum number of elements in RNNOperation enum. More...
 
template<>
constexpr int32_t EnumMax< RNNDirection > ()
 Maximum number of elements in RNNDirection enum. More...
 
template<>
constexpr int32_t EnumMax< RNNInputMode > ()
 Maximum number of elements in RNNInputMode enum. More...
 
template<>
constexpr int32_t EnumMax< RNNGateType > ()
 Maximum number of elements in RNNGateType enum. More...
 
class __attribute__ ((deprecated)) IOutputDimensionsFormula
 
template<>
constexpr int32_t EnumMax< UnaryOperation > ()
 Maximum number of elements in UnaryOperation enum. More...
 
template<>
constexpr int32_t EnumMax< ReduceOperation > ()
 Maximum number of elements in ReduceOperation enum. More...
 
template<>
constexpr int32_t EnumMax< SliceMode > ()
 Maximum number of elements in SliceMode enum. More...
 
template<>
constexpr int32_t EnumMax< TopKOperation > ()
 Maximum number of elements in TopKOperation enum. More...
 
template<>
constexpr int32_t EnumMax< MatrixOperation > ()
 Maximum number of elements in MatrixOperation enum. More...
 
template<>
constexpr int32_t EnumMax< ResizeMode > ()
 Maximum number of elements in ResizeMode enum. More...
 
template<>
constexpr int32_t EnumMax< LoopOutput > ()
 Maximum number of elements in LoopOutput enum. More...
 
template<>
constexpr int32_t EnumMax< TripLimit > ()
 Maximum number of elements in TripLimit enum. More...
 
template<>
constexpr int32_t EnumMax< FillOperation > ()
 Maximum number of elements in FillOperation enum. More...
 
template<>
constexpr int32_t EnumMax< CalibrationAlgoType > ()
 Maximum number of elements in CalibrationAlgoType enum. More...
 
template<>
constexpr int32_t EnumMax< QuantizationFlag > ()
 Maximum number of quantization flags in QuantizationFlag enum. More...
 
template<>
constexpr int32_t EnumMax< BuilderFlag > ()
 Maximum number of builder flags in BuilderFlag enum. More...
 
template<>
constexpr int32_t EnumMax< ProfilingVerbosity > ()
 Maximum number of profile verbosity levels in ProfilingVerbosity enum. More...
 
template<>
constexpr int32_t EnumMax< TacticSource > ()
 Maximum number of tactic sources in TacticSource enum. More...
 
template<>
constexpr int32_t EnumMax< NetworkDefinitionCreationFlag > ()
 Maximum number of elements in NetworkDefinitionCreationFlag enum. More...
 
template<>
constexpr int32_t EnumMax< PluginType > ()
 Maximum number of elements in PluginType enum. More...
 
template<>
constexpr int32_t EnumMax< EngineCapability > ()
 Maximum number of elements in EngineCapability enum. More...
 
template<>
constexpr int32_t EnumMax< DimensionOperation > ()
 Maximum number of elements in DimensionOperation enum. More...
 
template<>
constexpr int32_t EnumMax< WeightsRole > ()
 Maximum number of elements in WeightsRole enum. More...
 
template<>
constexpr int32_t EnumMax< DeviceType > ()
 Maximum number of elements in DeviceType enum. More...
 
template<>
constexpr int32_t EnumMax< OptProfileSelector > ()
 
template<typename T >
constexpr int32_t EnumMax ()
 Forward declare IGpuAllocator for use in other interfaces. More...
 
template<>
constexpr int32_t EnumMax< ActivationType > ()
 Maximum number of elements in ActivationType enum. More...
 
template<>
constexpr int32_t EnumMax< DataType > ()
 Maximum number of elements in DataType enum. More...
 
template<>
constexpr int32_t EnumMax< DimensionType > ()
 Maximum number of elements in DimensionType enum. More...
 
template<>
constexpr int32_t EnumMax< TensorFormat > ()
 Maximum number of elements in TensorFormat enum. More...
 
template<>
constexpr int32_t EnumMax< TensorLocation > ()
 Maximum number of elements in TensorLocation enum. More...
 
template<>
constexpr int32_t EnumMax< ILogger::Severity > ()
 Maximum number of elements in ILogger::Severity enum. More...
 
template<>
constexpr int32_t EnumMax< ErrorCode > ()
 Maximum number of elements in ErrorCode enum. More...
 
void cuErrCheck_ (CUresult stat, const CUDADriverWrapper &wrap, const char *file, int line)
 

Detailed Description

The TensorRT API version 1 namespace.

Typedef Documentation

◆ QuantizationFlags

typedef uint32_t nvinfer1::QuantizationFlags

Represents a collection of one or more QuantizationFlag values using binary OR operations.

See also
IBuilderConfig::getQuantizationFlags(), IBuilderConfig::setQuantizationFlags()

◆ BuilderFlags

typedef uint32_t nvinfer1::BuilderFlags

Represents a collection of one or more QuantizationFlag values using binary OR operations, e.g., 1U << BuilderFlag::kFP16 | 1U << BuilderFlag::kDEBUG.

See also
IBuilderConfig::getFlags(), ITensor::setFlags(),

◆ TacticSources

using nvinfer1::TacticSources = typedef uint32_t

Represents a collection of one or more TacticSource values combine using bitwise-OR operations.

See also
IBuilderConfig::setTacticSources(), IBuilderConfig::getTacticSources()

◆ NetworkDefinitionCreationFlags

This bitset is capable of representing one or more NetworkDefinitionCreationFlag flags constructed with binary OR operations. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH.

See also
IBuilder::createNetworkV2

◆ TensorFormats

typedef uint32_t nvinfer1::TensorFormats

It is capable of representing one or more TensorFormat by binary OR operations, e.g., 1U << TensorFormats::kCHW4 | 1U << TensorFormats::kCHW32.

See also
ITensor::getAllowedFormats(), ITensor::setAllowedFormats(),

◆ PluginFormat

PluginFormat is reserved for backward compatibility.

See also
IPluginExt::getPluginFormats()

Enumeration Type Documentation

◆ LayerType

enum nvinfer1::LayerType : int32_t
strong

The type values of layer classes.

See also
ILayer::getType()
Enumerator
kCONVOLUTION 

Convolution layer.

kFULLY_CONNECTED 

Fully connected layer.

kACTIVATION 

Activation layer.

kPOOLING 

Pooling layer.

kLRN 

LRN layer.

kSCALE 

Scale layer.

kSOFTMAX 

SoftMax layer.

kDECONVOLUTION 

Deconvolution layer.

kCONCATENATION 

Concatenation layer.

kELEMENTWISE 

Elementwise layer.

kPLUGIN 

Plugin layer.

kRNN 

RNN layer.

kUNARY 

UnaryOp operation Layer.

kPADDING 

Padding layer.

kSHUFFLE 

Shuffle layer.

kREDUCE 

Reduce layer.

kTOPK 

TopK layer.

kGATHER 

Gather layer.

kMATRIX_MULTIPLY 

Matrix multiply layer.

kRAGGED_SOFTMAX 

Ragged softmax layer.

kCONSTANT 

Constant layer.

kRNN_V2 

RNNv2 layer.

kIDENTITY 

Identity layer.

kPLUGIN_V2 

PluginV2 layer.

kSLICE 

Slice layer.

kSHAPE 

Shape layer.

kPARAMETRIC_RELU 

Parametric ReLU layer.

kRESIZE 

Resize Layer.

kTRIP_LIMIT 

Loop Trip limit layer.

kRECURRENCE 

Loop Recurrence layer.

kITERATOR 

Loop Iterator layer.

kLOOP_OUTPUT 

Loop output layer.

kSELECT 

Select layer.

kFILL 

Fill layer.

◆ PaddingMode

enum nvinfer1::PaddingMode : int32_t
strong

Enumerates the modes of padding to perform in convolution, deconvolution and pooling layer, padding mode takes precedence if setPaddingMode() and setPrePadding() are also used.

There are three padding styles, EXPLICIT, SAME, and CAFFE, with each style having two variants. The EXPLICIT and CAFFE styles determine if the final sampling location is used or not. The SAME style determine if the asymmetry in the padding is on the pre or post padding.

Shorthand:
I = dimensions of input image.
B = prePadding, before the image data. For deconvolution, prePadding is set before output.
A = postPadding, after the image data. For deconvolution, postPadding is set after output.
P = delta between input and output
S = stride
F = filter
O = output
D = dilation
M = I + B + A ; The image data plus any padding
DK = 1 + D * (F - 1)

Formulas for Convolution:

  • EXPLICIT_ROUND_DOWN:
    O = floor((M - DK) / S) + 1
  • CAFFE_ROUND_DOWN:
    O = floor((I + B * 2 - DK) / S)
  • EXPLICIT_ROUND_UP:
    O = ceil((M - DK) / S) + 1
  • CAFFE_ROUND_UP:
    O = ceil((I + B * 2 - DK) / S)
  • SAME_UPPER:
    O = ceil(I / S)
    P = floor((I - 1) / S) * S + DK - I;
    B = floor(P / 2)
    A = P - B
  • SAME_LOWER:
    O = ceil(I / S)
    P = floor((I - 1) / S) * S + DK - I;
    A = floor(P / 2)
    B = P - A

Formulas for Deconvolution:

  • EXPLICIT_ROUND_DOWN:
  • CAFFE_ROUND_DOWN:
  • EXPLICIT_ROUND_UP:
  • CAFFE_ROUND_UP:
    O = (I - 1) * S + DK - (B + A)
  • SAME_UPPER:
    O = min(I * S, (I - 1) * S + DK)
    P = max(DK - S, 0)
    B = floor(P / 2)
    A = P - B
  • SAME_LOWER:
    O = min(I * S, (I - 1) * S + DK)
    P = max(DK - S, 0)
    A = floor(P / 2)
    B = P - A

Formulas for Pooling:

  • EXPLICIT_ROUND_DOWN:
    O = floor((M - F) / S) + 1
  • EXPLICIT_ROUND_UP:
    O = ceil((M - F) / S) + 1
  • SAME_UPPER:
    O = ceil(I / S)
    P = floor((I - 1) / S) * S + F - I;
    B = floor(P / 2)
    A = P - B
  • SAME_LOWER:
    O = ceil(I / S)
    P = floor((I - 1) / S) * S + F - I;
    A = floor(P / 2)
    B = P - A
  • CAFFE_ROUND_DOWN:
    EXPLICIT_ROUND_DOWN - ((EXPLICIT_ROUND_DOWN - 1) * S >= I + B)
  • CAFFE_ROUND_UP:
    EXPLICIT_ROUND_UP - ((EXPLICIT_ROUND_UP - 1) * S >= I + B)

Pooling Example 1:

Given I = {6, 6}, B = {3, 3}, A = {2, 2}, S = {2, 2}, F = {3, 3}. What is O?
(B, A can be calculated for SAME_UPPER and SAME_LOWER mode)
  • EXPLICIT_ROUND_DOWN:
    Computation:
    M = {6, 6} + {3, 3} + {2, 2} ==> {11, 11}
    O ==> floor((M - F) / S) + 1
    ==> floor(({11, 11} - {3, 3}) / {2, 2}) + {1, 1}
    ==> floor({8, 8} / {2, 2}) + {1, 1}
    ==> {5, 5}
  • EXPLICIT_ROUND_UP:
    Computation:
    M = {6, 6} + {3, 3} + {2, 2} ==> {11, 11}
    O ==> ceil((M - F) / S) + 1
    ==> ceil(({11, 11} - {3, 3}) / {2, 2}) + {1, 1}
    ==> ceil({8, 8} / {2, 2}) + {1, 1}
    ==> {5, 5}
    The sample points are {0, 2, 4, 6, 8} in each dimension.
  • SAME_UPPER:
    Computation:
    I = {6, 6}
    S = {2, 2}
    O = ceil(I / S) = {3, 3}
    P = floor((I - 1) / S) * S + F - I
    ==> floor(({6, 6} - {1, 1}) / {2, 2}) * {2, 2} + {3, 3} - {6, 6}
    ==> {4, 4} + {3, 3} - {6, 6}
    ==> {1, 1}
    B = floor({1, 1} / {2, 2})
    ==> {0, 0}
    A = {1, 1} - {0, 0}
    ==> {1, 1}
  • SAME_LOWER:
    Computation:
    I = {6, 6}
    S = {2, 2}
    O = ceil(I / S) = {3, 3}
    P = floor((I - 1) / S) * S + F - I
    ==> {1, 1}
    A = floor({1, 1} / {2, 2})
    ==> {0, 0}
    B = {1, 1} - {0, 0}
    ==> {1, 1}
    The sample pointers are {0, 2, 4} in each dimension. SAMPLE_UPPER has {O0, O1, O2, pad} in output in each dimension. SAMPLE_LOWER has {pad, O0, O1, O2} in output in each dimension.

Pooling Example 2:

Given I = {6, 6}, B = {3, 3}, A = {3, 3}, S = {2, 2}, F = {3, 3}. What is O?
  • CAFFE_ROUND_DOWN:
    Computation:
    M = {6, 6} + {3, 3} + {3, 3} ==> {12, 12}
    EXPLICIT_ROUND_DOWN ==> floor((M - F) / S) + 1
    ==> floor(({12, 12} - {3, 3}) / {2, 2}) + {1, 1}
    ==> {5, 5}
    DIFF = (((EXPLICIT_ROUND_DOWN - 1) * S >= I + B) ? {1, 1} : {0, 0})
    ==> ({5, 5} - {1, 1}) * {2, 2} >= {6, 6} + {3, 3} ? {1, 1} : {0,0}
    ==> {0, 0}
    O ==> EXPLICIT_ROUND_DOWN - DIFF
    ==> {5, 5} - {0, 0}
    ==> {5, 5}
  • CAFFE_ROUND_UP:
    Computation:
    M = {6, 6} + {3, 3} + {3, 3} ==> {12, 12}
    EXPLICIT_ROUND_UP ==> ceil((M - F) / S) + 1
    ==> ceil(({12, 12} - {3, 3}) / {2, 2}) + {1, 1}
    ==> {6, 6}
    DIFF = (((EXPLICIT_ROUND_UP - 1) * S >= I + B) ? {1, 1} : {0, 0})
    ==> ({6, 6} - {1, 1}) * {2, 2} >= {6, 6} + {3, 3} ? {1, 1} : {0,0}
    ==> {1, 1}
    O ==> EXPLICIT_ROUND_UP - DIFF
    ==> {6, 6} - {1, 1}
    ==> {5, 5}

The sample points are {0, 2, 4, 6, 8} in each dimension.
CAFFE_ROUND_DOWN and CAFFE_ROUND_UP have two restrictions each on usage with pooling operations. This will cause getDimensions to return an empty dimension and also to reject the network at validation time.
For more information on original reference code, see https://github.com/BVLC/caffe/blob/master/src/caffe/layers/pooling_layer.cpp

  • Restriction 1:
    CAFFE_ROUND_DOWN: B >= F is an error if (B - S) < F
    CAFFE_ROUND_UP: (B + S) >= (F + 1) is an error if B < (F + 1)
  • Restriction 2:
    CAFFE_ROUND_DOWN: (B - S) >= F is an error if B >= F
    CAFFE_ROUND_UP: B >= (F + 1) is an error if (B + S) >= (F + 1)
Enumerator
kEXPLICIT_ROUND_DOWN 

Use explicit padding, rounding output size down.

kEXPLICIT_ROUND_UP 

Use explicit padding, rounding output size up.

kSAME_UPPER 

Use SAME padding, with prePadding <= postPadding.

kSAME_LOWER 

Use SAME padding, with prePadding >= postPadding.

kCAFFE_ROUND_DOWN 

Use CAFFE padding, rounding output size down, uses prePadding value.

kCAFFE_ROUND_UP 

Use CAFFE padding, rounding output size up, uses prePadding value.

◆ PoolingType

enum nvinfer1::PoolingType : int32_t
strong

The type of pooling to perform in a pooling layer.

Enumerator
kMAX 
kAVERAGE 
kMAX_AVERAGE_BLEND 

◆ ScaleMode

enum nvinfer1::ScaleMode : int32_t
strong

Controls how shift, scale and power are applied in a Scale layer.

See also
IScaleLayer
Enumerator
kUNIFORM 

Identical coefficients across all elements of the tensor.

kCHANNEL 

Per-channel coefficients.

kELEMENTWISE 

Elementwise coefficients.

◆ ElementWiseOperation

enum nvinfer1::ElementWiseOperation : int32_t
strong

Enumerates the binary operations that may be performed by an ElementWise layer.

See also
IElementWiseLayer
Enumerator
kSUM 

Sum of the two elements.

kPROD 

Product of the two elements.

kMAX 

Maximum of the two elements.

kMIN 

Minimum of the two elements.

kSUB 

Substract the second element from the first.

kDIV 

Divide the first element by the second.

kPOW 

The first element to the power of the second element.

kFLOOR_DIV 

Floor division of the first element by the second.

kAND 

Logical AND of two elements.

kOR 

Logical OR of two elements.

kXOR 

Logical XOR of two elements.

kEQUAL 

Check if two elements are equal.

kGREATER 

Check if element in first tensor is greater than corresponding element in second tensor.

kLESS 

Check if element in first tensor is less than corresponding element in second tensor.

◆ RNNOperation

enum nvinfer1::RNNOperation : int32_t
strong

Enumerates the RNN operations that may be performed by an RNN layer.

Equation definitions

In the equations below, we use the following naming convention:

t := current time step
i := input gate
o := output gate
f := forget gate
z := update gate
r := reset gate
c := cell gate
h := hidden gate
g[t] denotes the output of gate g at timestep t, e.g.
f[t] is the output of the forget gate f.
X[t] := input tensor for timestep t
C[t] := cell state for timestep t
H[t] := hidden state for timestep t
W[g] := W (input) parameter weight matrix for gate g
R[g] := U (recurrent) parameter weight matrix for gate g
Wb[g] := W (input) parameter bias vector for gate g
Rb[g] := U (recurrent) parameter bias vector for gate g
Unless otherwise specified, all operations apply pointwise
to elements of each operand tensor.
ReLU(X) := max(X, 0)
tanh(X) := hyperbolic tangent of X
sigmoid(X) := 1 / (1 + exp(-X))
exp(X) := e^X
A.B denotes matrix multiplication of A and B.
A*B denotes pointwise multiplication of A and B.

Equations

Depending on the value of RNNOperation chosen, each sub-layer of the RNN layer will perform one of the following operations:

::kRELU
H[t] := ReLU(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
H[t] := tanh(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
i[t] := sigmoid(W[i].X[t] + R[i].H[t-1] + Wb[i] + Rb[i])
f[t] := sigmoid(W[f].X[t] + R[f].H[t-1] + Wb[f] + Rb[f])
o[t] := sigmoid(W[o].X[t] + R[o].H[t-1] + Wb[o] + Rb[o])
c[t] := tanh(W[c].X[t] + R[c].H[t-1] + Wb[c] + Rb[c])
C[t] := f[t]*C[t-1] + i[t]*c[t]
H[t] := o[t]*tanh(C[t])
z[t] := sigmoid(W[z].X[t] + R[z].H[t-1] + Wb[z] + Rb[z])
r[t] := sigmoid(W[r].X[t] + R[r].H[t-1] + Wb[r] + Rb[r])
h[t] := tanh(W[h].X[t] + r[t]*(R[h].H[t-1] + Rb[h]) + Wb[h])
H[t] := (1 - z[t])*h[t] + z[t]*H[t-1]
See also
IRNNLayer, IRNNv2Layer
Enumerator
kRELU 

Single gate RNN w/ ReLU activation function.

kTANH 

Single gate RNN w/ TANH activation function.

kLSTM 

Four-gate LSTM network w/o peephole connections.

kGRU 

Three-gate network consisting of Gated Recurrent Units.

◆ RNNDirection

enum nvinfer1::RNNDirection : int32_t
strong

Enumerates the RNN direction that may be performed by an RNN layer.

See also
IRNNLayer, IRNNv2Layer
Enumerator
kUNIDIRECTION 

Network iterations from first input to last input.

kBIDIRECTION 

Network iterates from first to last and vice versa and outputs concatenated.

◆ RNNInputMode

enum nvinfer1::RNNInputMode : int32_t
strong

Enumerates the RNN input modes that may occur with an RNN layer.

If the RNN is configured with RNNInputMode::kLINEAR, then for each gate g in the first layer of the RNN, the input vector X[t] (length E) is left-multiplied by the gate's corresponding weight matrix W[g] (dimensions HxE) as usual, before being used to compute the gate output as described by RNNOperation.

If the RNN is configured with RNNInputMode::kSKIP, then this initial matrix multiplication is "skipped" and W[g] is conceptually an identity matrix. In this case, the input vector X[t] must have length H (the size of the hidden state).

See also
IRNNLayer, IRNNv2Layer
Enumerator
kLINEAR 

Perform the normal matrix multiplication in the first recurrent layer.

kSKIP 

No operation is performed on the first recurrent layer.

◆ RNNGateType

enum nvinfer1::RNNGateType : int32_t
strong

Identifies an individual gate within an RNN cell.

See also
RNNOperation
Enumerator
kINPUT 

Input gate (i).

kOUTPUT 

Output gate (o).

kFORGET 

Forget gate (f).

kUPDATE 

Update gate (z).

kRESET 

Reset gate (r).

kCELL 

Cell gate (c).

kHIDDEN 

Hidden gate (h).

◆ UnaryOperation

enum nvinfer1::UnaryOperation : int32_t
strong

Enumerates the unary operations that may be performed by a Unary layer.

See also
IUnaryLayer
Enumerator
kEXP 

Exponentiation.

kLOG 

Log (base e).

kSQRT 

Square root.

kRECIP 

Reciprocal.

kABS 

Absolute value.

kNEG 

Negation.

kSIN 

Sine.

kCOS 

Cosine.

kTAN 

Tangent.

kSINH 

Hyperbolic sine.

kCOSH 

Hyperbolic cosine.

kASIN 

Inverse sine.

kACOS 

Inverse cosine.

kATAN 

Inverse tangent.

kASINH 

Inverse hyperbolic sine.

kACOSH 

Inverse hyperbolic cosine.

kATANH 

Inverse hyperbolic tangent.

kCEIL 

Ceiling.

kFLOOR 

Floor.

kERF 

Gauss error function.

kNOT 

Logical NOT.

◆ ReduceOperation

enum nvinfer1::ReduceOperation : int32_t
strong

Enumerates the reduce operations that may be performed by a Reduce layer.

The table shows the result of reducing across an empty volume of a given type.

Operation kFLOAT and kHALF kINT32 kINT8
kSUM 0 0 0
kPROD 1 1 1
kMAX negative infinity INT_MIN -128
kMIN positive infinity INT_MAX 127
kAVG NaN 0 -128

The current version of TensorRT usually performs reduction for kINT8 via kFLOAT or kHALF. The kINT8 values show the quantized representations of the floating-point values.

Enumerator
kSUM 
kPROD 
kMAX 
kMIN 
kAVG 

◆ SliceMode

enum nvinfer1::SliceMode : int32_t
strong

Controls how ISliceLayer handles out of bounds coordinates.

See also
ISliceLayer
Enumerator
kDEFAULT 

Fail with error when the coordinates are out of bounds. This is the default.

kWRAP 

Coordinates wrap around periodically.

◆ TopKOperation

enum nvinfer1::TopKOperation : int32_t
strong

Enumerates the operations that may be performed by a TopK layer.

Enumerator
kMAX 

Maximum of the elements.

kMIN 

Minimum of the elements.

◆ MatrixOperation

enum nvinfer1::MatrixOperation : int32_t
strong

Enumerates the operations that may be performed on a tensor by IMatrixMultiplyLayer before multiplication.

Enumerator
kNONE 

Treat x as a matrix if it has two dimensions, or as a collection of matrices if x has more than two dimensions, where the last two dimensions are the matrix dimensions.

x must have at least two dimensions.

kTRANSPOSE 

Like kNONE, but transpose the matrix dimensions.

kVECTOR 

Treat x as a vector if it has one dimension, or as a collection of vectors if x has more than one dimension.

x must have at least one dimension. The first input tensor with dimensions [M,K] used with MatrixOperation::kVECTOR is equivalent to a tensor with dimensions [M, 1, K] with MatrixOperation::kNONE, i.e. is treated as M row vectors of length K. If MatrixOperation::kTRANSPOSE is specified, then the dimensions are [M, K, 1].

The second input tensor with dimensions [M,K] used with MatrixOperation::kVECTOR is equivalent to a tensor with dimensions [M, K, 1] with MatrixOperation::kNONE, i.e. is treated as M column vectors of length K. If MatrixOperation::kTRANSPOSE is specified, then the dimensions are [M, 1, K].

◆ ResizeMode

enum nvinfer1::ResizeMode : int32_t
strong

Enumerates various modes of resize in the resize layer. Resize mode set using setResizeMode().

Enumerator
kNEAREST 

ND (0 < N <= 8) nearest neighbor resizing.

kLINEAR 

Can handle linear (1D), bilinear (2D), and trilinear (3D) resizing.

◆ LoopOutput

enum nvinfer1::LoopOutput : int32_t
strong

Enum that describes kinds of loop outputs.

Enumerator
kLAST_VALUE 

Output value is value of tensor for last iteration.

kCONCATENATE 

Output value is concatenation of values of tensor for each iteration, in forward order.

kREVERSE 

Output value is concatenation of values of tensor for each iteration, in reverse order.

◆ TripLimit

enum nvinfer1::TripLimit : int32_t
strong

Enum that describes kinds of trip limits.

Enumerator
kCOUNT 

Tensor is scalar of type kINT32 that contains the trip count.

kWHILE 

Tensor is a scalar of type kBOOL. Loop terminates when value is false.

◆ FillOperation

enum nvinfer1::FillOperation : int32_t
strong

Enumerates the tensor fill operations that may performed by a fill layer.

See also
IFillLayer
Enumerator
kLINSPACE 

Generate evenly spaced numbers over a specified interval.

kRANDOM_UNIFORM 

Generate a tensor with random values drawn from a uniform distribution.

◆ CalibrationAlgoType

enum nvinfer1::CalibrationAlgoType : int32_t
strong

enum CalibrationAlgoType

Version of calibration algorithm to use.

Enumerator
kLEGACY_CALIBRATION 
kENTROPY_CALIBRATION 
kENTROPY_CALIBRATION_2 
kMINMAX_CALIBRATION 

◆ QuantizationFlag

enum nvinfer1::QuantizationFlag : int32_t
strong

List of valid flags for quantizing the network to int8.

See also
IBuilderConfig::setQuantizationFlag(), IBuilderConfig::getQuantizationFlag()
Enumerator
kCALIBRATE_BEFORE_FUSION 

Run int8 calibration pass before layer fusion.

Only valid for IInt8LegacyCalibrator and IInt8EntropyCalibrator. We always run int8 calibration pass before layer fusion for IInt8MinMaxCalibrator and IInt8EntropyCalibrator2. Disabled by default.

◆ BuilderFlag

enum nvinfer1::BuilderFlag : int32_t
strong

List of valid modes that the builder can enable when creating an engine from a network definition.

See also
IBuilderConfig::setFlag(), IBuilderConfig::getFlag()
Enumerator
kFP16 

Enable FP16 layer selection, with FP32 fallback.

kINT8 

Enable Int8 layer selection, with FP32 fallback with FP16 fallback if kFP16 also specified.

kDEBUG 

Enable debugging of layers via synchronizing after every layer.

kGPU_FALLBACK 

Enable layers marked to execute on GPU if layer cannot execute on DLA.

kSTRICT_TYPES 

Enables strict type constraints.

kREFIT 

Enable building a refittable engine.

kDISABLE_TIMING_CACHE 

Disable reuse of timing information across identical layers.

kTF32 

Allow (but not require) computations on tensors of type DataType::kFLOAT to use TF32.

TF32 computes inner products by rounding the inputs to 10-bit mantissas before multiplying, but accumulates the sum using 23-bit mantissas. Enabled by default.

◆ ProfilingVerbosity

enum nvinfer1::ProfilingVerbosity : int32_t
strong

List of verbosity levels of layer information exposed in NVTX annotations.

See also
IBuilderConfig::setProfilingVerbosity(), IBuilderConfig::getProfilingVerbosity()
Enumerator
kDEFAULT 

Register layer names in NVTX message field.

kNONE 

Turn off NVTX traces.

kVERBOSE 

Register layer names in NVTX message field and register layer detail in NVTX JSON payload field.

◆ TacticSource

enum nvinfer1::TacticSource : int32_t
strong

List of tactic sources for TensorRT.

See also
TacticSources, IBuilderConfig::setTacticSources(), IBuilderConfig::getTacticSources()
Enumerator
kCUBLAS 

cuBLAS tactics.

Note
Disabling kCUBLAS will cause the cublas handle passed to plugins in attachToContext to be null.
kCUBLAS_LT 

cuBLAS LT tactics

◆ NetworkDefinitionCreationFlag

List of immutable network properties expressed at network creation time. NetworkDefinitionCreationFlag is used with createNetworkV2 to specify immutable properties of the network. The createNetwork() function always had an implicit batch dimension being specified by the maxBatchSize builder parameter. createNetworkV2 with kDEFAULT flag mimics that behaviour.

See also
IBuilder::createNetworkV2
Enumerator
kEXPLICIT_BATCH 

Dynamic shape support requires that the kEXPLICIT_BATCH flag is set.

With dynamic shapes, any of the input dimensions can vary at run-time, and there are no implicit dimensions in the network specification. This is specified by using the wildcard dimension value -1. Mark the network to be an explicit batch network

kEXPLICIT_PRECISION 

Setting the network to be an explicit precision network has the following implications: 1) Precision of all input tensors to the network have to be specified with ITensor::setType() function 2) Precision of all layer output tensors in the network have to be specified using ILayer::setOutputType() function 3) The builder will not quantize the weights of any layer including those running in lower precision(INT8).

It will simply cast the weights into the required precision. 4) Dynamic ranges must not be provided to run the network in int8 mode. Dynamic ranges of each tensor in the explicit precision network is [-127,127]. 5) Quantizing and dequantizing activation values between higher (FP32) and lower (INT8) precision will be performed using explicit Scale layers with input/output precision set appropriately. Mark the network to be an explicit precision network

◆ PluginType

enum nvinfer1::PluginType : int32_t
strong

The type values for the various plugins.

See also
INvPlugin::getPluginType()
Enumerator
kFASTERRCNN 

FasterRCNN fused plugin (RPN + ROI pooling).

kNORMALIZE 

Normalize plugin.

kPERMUTE 

Permute plugin.

kPRIORBOX 

PriorBox plugin.

kSSDDETECTIONOUTPUT 

SSD DetectionOutput plugin.

kCONCAT 

Concat plugin.

kPRELU 

YOLO PReLU Plugin.

kYOLOREORG 

YOLO Reorg Plugin.

kYOLOREGION 

YOLO Region Plugin.

kANCHORGENERATOR 

SSD Grid Anchor Generator.

◆ EngineCapability

enum nvinfer1::EngineCapability : int32_t
strong

Forward declaration of IPluginFactory for use by other interfaces.

List of supported engine capability flows.

The EngineCapability determines the restrictions of a network during build time for what can be executed at runtime. EngineCapability::kDEFAULT does not provide any restrictions on functionality and the resulting serialized engine can be executed with TensorRT's standard runtime APIs in the nvinfer1 namespace. EngineCapabiltiy::kSAFE_GPU provides a restricted subset of network operations that are safety certified and the resulting serialized engine can be executed with TensorRT's safe runtime APIs in the nvinfer1::safe namespace. EngineCapability::kSAFE_DLA provides a restricted subset of network operations that are DLA compatible and the resulting serialized engine can be executed using NvMediaDLA's runtime APIs. See sampleNvmedia for an example of integrating NvMediaDLA APIs with TensorRT APIs.

Enumerator
kDEFAULT 

Full capability, TensorRT mode without any restrictions using TensorRT nvinfer1 APIs.

kSAFE_GPU 

Safety restricted capability, TensorRT flow that can only run on GPU devices via TensorRT nvinfer1::safe APIs.

kSAFE_DLA 

Safety restricted capability, TensorRT flow that can only run on DLA devices via NvMediaDLA APIs.

◆ DimensionOperation

enum nvinfer1::DimensionOperation : int32_t
strong

An operation on two IDimensionExpr, which represent integer expressions used in dimension computations.

For example, given two IDimensionExpr x and y and an IExprBuilder& eb, eb.operation(DimensionOperation::kSUM, x, y) creates a representation of x+y.

See also
IDimensionExpr, IExprBuilder
Enumerator
kSUM 

Sum of the two operands.

kPROD 

Product of the two operands.

kMAX 

Maximum of the two operands.

kMIN 

Minimum of the two operands.

kSUB 

Substract the second element from the first.

kEQUAL 

1 if operands are equal, 0 otherwise.

kLESS 

1 if first operand is less than second operand, 0 otherwise.

kFLOOR_DIV 

Floor division of the first element by the second.

kCEIL_DIV 

Division rounding up.

◆ WeightsRole

enum nvinfer1::WeightsRole : int32_t
strong

How a layer uses particular Weights.

The power weights of an IScaleLayer are omitted. Refitting those is not supported.

Enumerator
kKERNEL 

kernel for IConvolutionLayer, IDeconvolutionLayer, or IFullyConnectedLayer

kBIAS 

bias for IConvolutionLayer, IDeconvolutionLayer, or IFullyConnectedLayer

kSHIFT 

shift part of IScaleLayer

kSCALE 

scale part of IScaleLayer

kCONSTANT 

weights for IConstantLayer

◆ DeviceType

enum nvinfer1::DeviceType : int32_t
strong

The device that this layer/network will execute on.

Enumerator
kGPU 

GPU Device.

kDLA 

DLA Core.

◆ OptProfileSelector

enum nvinfer1::OptProfileSelector : int32_t
strong

When setting or querying optimization profile parameters (such as shape tensor inputs or dynamic dimensions), select whether we are interested in the minimum, optimum, or maximum values for these parameters. The minimum and maximum specify the permitted range that is supported at runtime, while the optimum value is used for the kernel selection. This should be the "typical" value that is expected to occur at runtime.

See also
IOptimizationProfile::setDimensions(), IOptimizationProfile::setShapeValues()
Enumerator
kMIN 

This is used to set or get the minimum permitted value for dynamic dimensions etc.

kOPT 

This is used to set or get the value that is used in the optimization (kernel selection).

kMAX 

This is used to set or get the maximum permitted value for dynamic dimensions etc.

◆ ActivationType

enum nvinfer1::ActivationType : int32_t
strong

Enumerates the types of activation to perform in an activation layer.

Enumerator
kRELU 

Rectified linear activation.

kSIGMOID 

Sigmoid activation.

kTANH 

TanH activation.

kLEAKY_RELU 

LeakyRelu activation: x>=0 ? x : alpha * x.

kELU 

Elu activation: x>=0 ? x : alpha * (exp(x) - 1).

kSELU 

Selu activation: x>0 ? beta * x : beta * (alpha*exp(x) - alpha)

kSOFTSIGN 

Softsign activation: x / (1+|x|)

kSOFTPLUS 

Parametric softplus activation: alpha*log(exp(beta*x)+1)

kCLIP 

Clip activation: max(alpha, min(beta, x))

kHARD_SIGMOID 

Hard sigmoid activation: max(0, min(1, alpha*x+beta))

kSCALED_TANH 

Scaled tanh activation: alpha*tanh(beta*x)

kTHRESHOLDED_RELU 

Thresholded ReLU activation: x>alpha ? x : 0.

◆ DataType

enum nvinfer1::DataType : int32_t
strong

The type of weights and tensors.

Enumerator
kFLOAT 

32-bit floating point format.

kHALF 

IEEE 16-bit floating-point format.

kINT8 

8-bit integer representing a quantized floating-point value.

kINT32 

Signed 32-bit integer format.

kBOOL 

8-bit boolean. 0 = false, 1 = true, other values undefined.

◆ DimensionType

enum nvinfer1::DimensionType : int32_t
strong

The type of data encoded across this dimension.

Enumerator
kSPATIAL 

Elements correspond to different spatial data.

kCHANNEL 

Elements correspond to different channels.

kINDEX 

Elements correspond to different batch index.

kSEQUENCE 

Elements correspond to different sequence values.

◆ TensorFormat

enum nvinfer1::TensorFormat : int32_t
strong

Format of the input/output tensors.

This enum is extended to be used by both plugins and reformat-free network I/O tensors.

See also
IPluginExt::getPluginFormats(), safe::ICudaEngine::getBindingFormat()

For more information about data formats, see the topic "Data Format Description" located in the TensorRT Developer Guide (https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html).

Enumerator
kLINEAR 

Row major linear format.

For a tensor with dimensions {N, C, H, W}, the W axis always has unit stride, and the stride of every other axis is at least the the product of of the next dimension times the next stride. the strides are the same as for a C array with dimensions [N][C][H][W].

kNCHW 

Deprecated name of kLINEAR, provided for backwards compatibility and will be removed in TensorRT 8.0.

kCHW2 

Two wide channel vectorized row major format.

This format is bound to FP16. It is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+1)/2][H][W][2], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/2][h][w][c%2].

kNC2HW2 

Deprecated name of kCHW2, provided for backwards compatibility and will be removed in TensorRT 8.0.

kHWC8 

Eight channel format where C is padded to a multiple of 8.

This format is bound to FP16. It is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to the array with dimensions [N][H][W][(C+7)/8*8], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][h][w][c].

kNHWC8 

Deprecated name of kHWC8, provided for backwards compatibility and will be removed in TensorRT 8.0.

kCHW4 

Four wide channel vectorized row major format.

This format is bound to INT8 or FP16. It is only available for dimensions >= 3. For INT8, the C dimension must be a build-time constant. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+3)/4][H][W][4], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/4][h][w][c%4].

Deprecated usage:

If running on the DLA, this format can be used for acceleration with the caveat that C must be equal or lesser than 4. If used as DLA input with allowGPUFallback disable, it needs to meet line stride requirement of DLA format. Column stride in bytes should be multiple of 32.

kCHW16 

Sixteen wide channel vectorized row major format.

This format is bound to FP16. It is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+15)/16][H][W][16], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/16][h][w][c%16].

For DLA usage, this format maps to the native format for FP16, and the tensor sizes are limited to C,H,W in the range [1,8192].

kCHW32 

Thirty-two wide channel vectorized row major format.

This format is only available for dimensions >= 3. For a tensor with dimensions {N, C, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+31)/32][H][W][32], with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c/32][h][w][c%32].

For DLA usage, this format maps to the native format for INT8, and the tensor sizes are limited to C,H,W in the range [1,8192].

kDHWC8 

Eight channel format where C is padded to a multiple of 8.

This format is bound to FP16, and it is only available for dimensions >= 4. For a tensor with dimensions {N, C, D, H, W}, the memory layout is equivalent to an array with dimensions [N][D][H][W][(C+7)/8*8], with the tensor coordinates (n, c, d, h, w) mapping to array subscript [n][d][h][w][c].

kCDHW32 

Thirty-two wide channel vectorized row major format.

This format is bound to FP16 and INT8 and is only available for dimensions >= 4. For a tensor with dimensions {N, C, D, H, W}, the memory layout is equivalent to a C array with dimensions [N][(C+31)/32][D][H][W][32], with the tensor coordinates (n, c, d, h, w) mapping to array subscript [n][c/32][d][h][w][c%32].

kHWC 

Non-vectorized channel-last format.

This format is bound to FP32 and is only available for dimensions >= 3.

kDLA_LINEAR 

DLA planar format.

For a tensor with dimension {N, C, H, W}, the W axis always has unit stride. The stride for stepping along the H axis is rounded up to 64 bytes.

The memory layout is equivalent to a C array with dimensions [N][C][H][roundUp(W, 64/elementSize)] where elementSize is 2 for FP16 and 1 for Int8, with the tensor coordinates (n, c, h, w) mapping to array subscript [n][c][h][w].

kDLA_HWC4 

DLA image format.

For a tensor with dimension {N, C, H, W} the C axis always has unit stride. The stride for stepping along the H axis is rounded up to 32 bytes. C can only be 1, 3 or 4. If C == 1, it will map to grayscale format. If C == 3 or C == 4, it will map to color image format. And if C == 3, the stride for stepping along the W axis needs to be padded to 4 in elements.

When C is {1, 3, 4}, then C' is {1, 4, 4} respectively, the memory layout is equivalent to a C array with dimensions [N][H][roundUp(W, 32/C'/elementSize)][C'] where elementSize is 2 for FP16 and 1 for Int8. The tensor coordinates (n, c, h, w) mapping to array subscript [n][h][w][c].

◆ PluginVersion

enum nvinfer1::PluginVersion : uint8_t
strong
Enumerator
kV2 
kV2_EXT 

IPluginV2.

kV2_IOEXT 

IPluginV2Ext.

kV2_DYNAMICEXT 

IPluginV2IOExt.

◆ PluginFieldType

enum nvinfer1::PluginFieldType : int32_t
strong
Enumerator
kFLOAT16 

FP16 field type.

kFLOAT32 

FP32 field type.

kFLOAT64 

FP64 field type.

kINT8 

INT8 field type.

kINT16 

INT16 field type.

kINT32 

INT32 field type.

kCHAR 

char field type.

kDIMS 

nvinfer1::Dims field type.

kUNKNOWN 

◆ TensorLocation

enum nvinfer1::TensorLocation : int32_t
strong

The location for tensor data storage, device or host.

Enumerator
kDEVICE 

Data stored on device.

kHOST 

Data stored on host.

◆ ErrorCode

enum nvinfer1::ErrorCode : int32_t
strong

Error codes that can be returned by TensorRT during execution.

the type of parser error

Enumerator
kSUCCESS 

Execution completed successfully.

kUNSPECIFIED_ERROR 

An error that does not fall into any other category.

This error is included for forward compatibility

kINTERNAL_ERROR 

A non-recoverable TensorRT error occurred.

kINVALID_ARGUMENT 

An argument passed to the function is invalid in isolation.

This is a violation of the API contract

kINVALID_CONFIG 

An error occurred when comparing the state of an argument relative to other arguments.

For example, the dimensions for concat differ between two tensors outside of the channel dimension. This error is triggered when an argument is correct in isolation, but not relative to other arguments. This is to help to distinguish from the simple errors from the more complex errors. This is a violation of the API contract.

kFAILED_ALLOCATION 

An error occurred when performing an allocation of memory on the host or the device.

A memory allocation error is normally fatal, but in the case where the application provided its own memory allocation routine, it is possible to increase the pool of available memory and resume execution.

kFAILED_INITIALIZATION 

One, or more, of the components that TensorRT relies on did not initialize correctly.

This is a system setup issue.

kFAILED_EXECUTION 

An error occurred during execution that caused TensorRT to end prematurely, either an asynchronous error or other execution errors reported by CUDA/DLA.

In a dynamic system, the data can be thrown away and the next frame can be processed or execution can be retried. This is either an execution error or a memory error.

kFAILED_COMPUTATION 

An error occurred during execution that caused the data to become corrupted, but execution finished.

Examples of this error are NaN squashing or integer overflow. In a dynamic system, the data can be thrown away and the next frame can be processed or execution can be retried. This is either a data corruption error, an input error, or a range error.

kINVALID_STATE 

TensorRT was put into a bad state by incorrect sequence of function calls.

An example of an invalid state is specifying a layer to be DLA only without GPU fallback, and that layer is not supported by DLA. This can occur in situations where a service is optimistically executing networks for multiple different configurations without checking proper error configurations, and instead throwing away bad configurations caught by TensorRT. This is a violation of the API contract, but can be recoverable.

Example of a recovery: GPU fallback is disabled and conv layer with large filter(63x63) is specified to run on DLA. This will fail due to DLA not supporting the large kernel size. This can be recovered by either turning on GPU fallback or setting the layer to run on the GPU.

kUNSUPPORTED_STATE 

An error occurred due to the network not being supported on the device due to constraints of the hardware or system.

An example is running a unsafe layer in a safety certified context, or a resource requirement for the current network is greater than the capabilities of the target device. The network is otherwise correct, but the network and hardware combination is problematic. This can be recoverable. Examples:

  • Scratch space requests larger than available device memory and can be recovered by increasing allowed workspace size.
  • Tensor size exceeds the maximum element count and can be recovered by reducing the maximum batch size.

Function Documentation

◆ EnumMax< LayerType >()

template<>
constexpr int32_t nvinfer1::EnumMax< LayerType > ( )
inlineconstexpr

Maximum number of elements in LayerType enum.

See also
LayerType

◆ EnumMax< PaddingMode >()

template<>
constexpr int32_t nvinfer1::EnumMax< PaddingMode > ( )
inlineconstexpr

Maximum number of elements in PaddingMode enum.

See also
PaddingMode

◆ EnumMax< PoolingType >()

template<>
constexpr int32_t nvinfer1::EnumMax< PoolingType > ( )
inlineconstexpr

Maximum number of elements in PoolingType enum.

See also
PoolingType

◆ EnumMax< ScaleMode >()

template<>
constexpr int32_t nvinfer1::EnumMax< ScaleMode > ( )
inlineconstexpr

Maximum number of elements in ScaleMode enum.

See also
ScaleMode

◆ EnumMax< ElementWiseOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< ElementWiseOperation > ( )
inlineconstexpr

Maximum number of elements in ElementWiseOperation enum.

See also
ElementWiseOperation

◆ EnumMax< RNNOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< RNNOperation > ( )
inlineconstexpr

Maximum number of elements in RNNOperation enum.

See also
RNNOperation

◆ EnumMax< RNNDirection >()

template<>
constexpr int32_t nvinfer1::EnumMax< RNNDirection > ( )
inlineconstexpr

Maximum number of elements in RNNDirection enum.

See also
RNNDirection

◆ EnumMax< RNNInputMode >()

template<>
constexpr int32_t nvinfer1::EnumMax< RNNInputMode > ( )
inlineconstexpr

Maximum number of elements in RNNInputMode enum.

See also
RNNInputMode

◆ EnumMax< RNNGateType >()

template<>
constexpr int32_t nvinfer1::EnumMax< RNNGateType > ( )
inlineconstexpr

Maximum number of elements in RNNGateType enum.

See also
RNNGateType

◆ __attribute__()

class nvinfer1::__attribute__ ( (deprecated)  )

Application-implemented interface to compute the HW output dimensions of a layer from the layer input and parameters.

Parameters
inputDimsThe input dimensions of the layer.
kernelSizeThe kernel size (or window size, for a pooling layer) parameter of the layer operation.
strideThe stride parameter for the layer.
paddingThe padding parameter of the layer.
dilationThe dilation parameter of the layer (only applicable to convolutions).
layerNameThe name of the layer.
Returns
The output size of the layer

Note that for dilated convolutions, the dilation is applied to the kernel size before this routine is called.

◆ EnumMax< UnaryOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< UnaryOperation > ( )
inlineconstexpr

Maximum number of elements in UnaryOperation enum.

See also
UnaryOperation

◆ EnumMax< ReduceOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< ReduceOperation > ( )
inlineconstexpr

Maximum number of elements in ReduceOperation enum.

See also
ReduceOperation

◆ EnumMax< SliceMode >()

template<>
constexpr int32_t nvinfer1::EnumMax< SliceMode > ( )
inlineconstexpr

Maximum number of elements in SliceMode enum.

See also
SliceMode

◆ EnumMax< TopKOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< TopKOperation > ( )
inlineconstexpr

Maximum number of elements in TopKOperation enum.

See also
TopKOperation

◆ EnumMax< MatrixOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< MatrixOperation > ( )
inlineconstexpr

Maximum number of elements in MatrixOperation enum.

See also
DataType

◆ EnumMax< ResizeMode >()

template<>
constexpr int32_t nvinfer1::EnumMax< ResizeMode > ( )
inlineconstexpr

Maximum number of elements in ResizeMode enum.

See also
ResizeMode

◆ EnumMax< LoopOutput >()

template<>
constexpr int32_t nvinfer1::EnumMax< LoopOutput > ( )
inlineconstexpr

Maximum number of elements in LoopOutput enum.

See also
DataType

◆ EnumMax< TripLimit >()

template<>
constexpr int32_t nvinfer1::EnumMax< TripLimit > ( )
inlineconstexpr

Maximum number of elements in TripLimit enum.

See also
DataType

◆ EnumMax< FillOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< FillOperation > ( )
inlineconstexpr

Maximum number of elements in FillOperation enum.

See also
FillOperation

◆ EnumMax< CalibrationAlgoType >()

template<>
constexpr int32_t nvinfer1::EnumMax< CalibrationAlgoType > ( )
inlineconstexpr

Maximum number of elements in CalibrationAlgoType enum.

See also
DataType

◆ EnumMax< QuantizationFlag >()

template<>
constexpr int32_t nvinfer1::EnumMax< QuantizationFlag > ( )
inlineconstexpr

Maximum number of quantization flags in QuantizationFlag enum.

See also
QuantizationFlag

◆ EnumMax< BuilderFlag >()

template<>
constexpr int32_t nvinfer1::EnumMax< BuilderFlag > ( )
inlineconstexpr

Maximum number of builder flags in BuilderFlag enum.

See also
BuilderFlag

◆ EnumMax< ProfilingVerbosity >()

template<>
constexpr int32_t nvinfer1::EnumMax< ProfilingVerbosity > ( )
inlineconstexpr

Maximum number of profile verbosity levels in ProfilingVerbosity enum.

See also
ProfilingVerbosity

◆ EnumMax< TacticSource >()

template<>
constexpr int32_t nvinfer1::EnumMax< TacticSource > ( )
inlineconstexpr

Maximum number of tactic sources in TacticSource enum.

See also
TacticSource

◆ EnumMax< NetworkDefinitionCreationFlag >()

template<>
constexpr int32_t nvinfer1::EnumMax< NetworkDefinitionCreationFlag > ( )
inlineconstexpr

Maximum number of elements in NetworkDefinitionCreationFlag enum.

See also
NetworkDefinitionCreationFlag

◆ EnumMax< PluginType >()

template<>
constexpr int32_t nvinfer1::EnumMax< PluginType > ( )
inlineconstexpr

Maximum number of elements in PluginType enum.

See also
PluginType

◆ EnumMax< EngineCapability >()

template<>
constexpr int32_t nvinfer1::EnumMax< EngineCapability > ( )
inlineconstexpr

Maximum number of elements in EngineCapability enum.

See also
EngineCapability

◆ EnumMax< DimensionOperation >()

template<>
constexpr int32_t nvinfer1::EnumMax< DimensionOperation > ( )
inlineconstexpr

Maximum number of elements in DimensionOperation enum.

See also
DimensionOperation

◆ EnumMax< WeightsRole >()

template<>
constexpr int32_t nvinfer1::EnumMax< WeightsRole > ( )
inlineconstexpr

Maximum number of elements in WeightsRole enum.

See also
WeightsRole

◆ EnumMax< DeviceType >()

template<>
constexpr int32_t nvinfer1::EnumMax< DeviceType > ( )
inlineconstexpr

Maximum number of elements in DeviceType enum.

See also
DeviceType

◆ EnumMax< OptProfileSelector >()

template<>
constexpr int32_t nvinfer1::EnumMax< OptProfileSelector > ( )
inlineconstexpr

◆ EnumMax()

template<typename T >
constexpr int32_t nvinfer1::EnumMax ( )
inlineconstexpr

Forward declare IGpuAllocator for use in other interfaces.

Maximum number of elements in an enumeration type.

◆ EnumMax< ActivationType >()

template<>
constexpr int32_t nvinfer1::EnumMax< ActivationType > ( )
inlineconstexpr

Maximum number of elements in ActivationType enum.

See also
ActivationType

◆ EnumMax< DataType >()

template<>
constexpr int32_t nvinfer1::EnumMax< DataType > ( )
inlineconstexpr

Maximum number of elements in DataType enum.

See also
DataType

◆ EnumMax< DimensionType >()

template<>
constexpr int32_t nvinfer1::EnumMax< DimensionType > ( )
inlineconstexpr

Maximum number of elements in DimensionType enum.

See also
DimensionType

◆ EnumMax< TensorFormat >()

template<>
constexpr int32_t nvinfer1::EnumMax< TensorFormat > ( )
inlineconstexpr

Maximum number of elements in TensorFormat enum.

See also
TensorFormat

◆ EnumMax< TensorLocation >()

template<>
constexpr int32_t nvinfer1::EnumMax< TensorLocation > ( )
inlineconstexpr

Maximum number of elements in TensorLocation enum.

See also
TensorLocation

◆ EnumMax< ILogger::Severity >()

template<>
constexpr int32_t nvinfer1::EnumMax< ILogger::Severity > ( )
inlineconstexpr

Maximum number of elements in ILogger::Severity enum.

See also
ILogger::Severity

◆ EnumMax< ErrorCode >()

template<>
constexpr int32_t nvinfer1::EnumMax< ErrorCode > ( )
inlineconstexpr

Maximum number of elements in ErrorCode enum.

See also
ErrorCode

◆ cuErrCheck_()

void nvinfer1::cuErrCheck_ ( CUresult  stat,
const CUDADriverWrapper wrap,
const char *  file,
int  line 
)
inline
Here is the call graph for this function:
generate.D
D
Definition: 07_creating_a_model_with_the_layer_api/generate.py:96
inference.f
f
Definition: BERT/inference.py:92
generate.e
e
Definition: 05_folding_constants/generate.py:33
half_float::detail::ceil
half ceil(half arg)
Nearest integer not less than half value.
Definition: ieee_half.h:2551
generate.max
def max(self, *args)
Definition: 08_replacing_a_subgraph/generate.py:27
generate.input
input
Definition: 05_folding_constants/generate.py:26
generate.output
output
Definition: 05_folding_constants/generate.py:36
example.W
W
Definition: onnx-graphsurgeon/examples/02_creating_a_model_with_initializer/example.py:24
copyright-scan.update
def update(filename, args)
Definition: copyright-scan.py:109
generate.A
A
Definition: 07_creating_a_model_with_the_layer_api/generate.py:84
generate.min
def min(self, *args)
Definition: 08_replacing_a_subgraph/generate.py:23
nvinfer1::RNNOperation::kLSTM
@ kLSTM
Four-gate LSTM network w/o peephole connections.
generate.C
C
Definition: 07_creating_a_model_with_the_layer_api/generate.py:95
generate.B
float B
Definition: 07_creating_a_model_with_the_layer_api/generate.py:88
half_float::detail::tanh
expr tanh(half arg)
Hyperbolic tangent.
Definition: ieee_half.h:2487
nvinfer1::RNNOperation::kGRU
@ kGRU
Three-gate network consisting of Gated Recurrent Units.
nvinfer1::RNNOperation::kTANH
@ kTANH
Single gate RNN w/ TANH activation function.
half_float::detail::floor
half floor(half arg)
Nearest integer not greater than half value.
Definition: ieee_half.h:2558
generate.c
c
Definition: 05_folding_constants/generate.py:31
example.X
X
Definition: onnx-graphsurgeon/examples/01_creating_a_model/example.py:22