TensorRT  7.2.1.6
NVIDIA TensorRT
Looking for a C++ dev who knows TensorRT?
I'm looking for work. Hire me!
nvinfer1::IBuilder Class Referenceabstract

Builds an engine from a network definition. More...

Collaboration diagram for nvinfer1::IBuilder:

Public Member Functions

virtual __attribute__((deprecated)) virtual nvinfer1 void setMaxBatchSize (int32_t batchSize)=0
 Create a network definition object where all tensors have an implicit batch dimension. More...
 
virtual int32_t getMaxBatchSize () const =0
 Get the maximum batch size. More...
 
 __attribute__ ((deprecated)) virtual bool getHalf2Mode() const =0
 Query whether half2 mode is used. More...
 
 __attribute__ ((deprecated)) virtual void setDebugSync(bool sync)=0
 Set whether the builder should use debug synchronization. More...
 
 __attribute__ ((deprecated)) virtual bool getDebugSync() const =0
 Query whether the builder will use debug synchronization. More...
 
 __attribute__ ((deprecated)) virtual void setMinFindIterations(int32_t minFind)=0
 Set the number of minimization iterations used when timing layers. More...
 
 __attribute__ ((deprecated)) virtual int32_t getMinFindIterations() const =0
 Query the number of minimization iterations. More...
 
 __attribute__ ((deprecated)) virtual void setAverageFindIterations(int32_t avgFind)=0
 Set the number of averaging iterations used when timing layers. More...
 
 __attribute__ ((deprecated)) virtual int32_t getAverageFindIterations() const =0
 Query the number of averaging iterations. More...
 
virtual __attribute__((deprecated)) virtual nvinfer1 bool platformHasFastFp16 () const =0
 Build a CUDA engine from a network definition. More...
 
virtual bool platformHasFastInt8 () const =0
 Determine whether the platform has fast native int8. More...
 
virtual void destroy ()=0
 Destroy this object. More...
 
 __attribute__ ((deprecated)) virtual void setInt8Mode(bool mode)=0
 Set whether or not quantized 8-bit kernels are permitted. More...
 
 __attribute__ ((deprecated)) virtual bool getInt8Mode() const =0
 Query whether Int8 mode is used. More...
 
 __attribute__ ((deprecated)) virtual void setInt8Calibrator(IInt8Calibrator *calibrator)=0
 Set Int8 Calibration interface. More...
 
 __attribute__ ((deprecated)) virtual void setDeviceType(ILayer *layer
 Set the device that this layer must execute on. More...
 
 __attribute__ ((deprecated)) virtual DeviceType getDeviceType(const ILayer *layer) const =0
 Get the device that this layer executes on. More...
 
 __attribute__ ((deprecated)) virtual bool isDeviceTypeSet(const ILayer *layer) const =0
 whether the DeviceType has been explicitly set for this layer More...
 
 __attribute__ ((deprecated)) virtual void resetDeviceType(ILayer *layer)=0
 reset the DeviceType for this layer More...
 
 __attribute__ ((deprecated)) virtual bool canRunOnDLA(const ILayer *layer) const =0
 Checks if a layer can run on DLA. More...
 
 __attribute__ ((deprecated)) virtual void setDefaultDeviceType(DeviceType deviceType)=0
 Sets the default DeviceType to be used by the builder. More...
 
 __attribute__ ((deprecated)) virtual DeviceType getDefaultDeviceType() const =0
 Get the default DeviceType which was set by setDefaultDeviceType. More...
 
virtual int32_t getMaxDLABatchSize () const =0
 Get the maximum batch size DLA can support. More...
 
 __attribute__ ((deprecated)) virtual void allowGPUFallback(bool setFallBackMode)=0
 Sets the builder to use GPU if a layer that was supposed to run on DLA can not run on DLA. More...
 
virtual int32_t getNbDLACores () const =0
 Return the number of DLA engines available to this builder. More...
 
 __attribute__ ((deprecated)) virtual void setDLACore(int32_t dlaCore)=0
 Set the DLA core that the engine must execute on. More...
 
 __attribute__ ((deprecated)) virtual int32_t getDLACore() const =0
 Get the DLA core that the engine executes on. More...
 
virtual void setGpuAllocator (IGpuAllocator *allocator)=0
 Set the GPU allocator. More...
 
 __attribute__ ((deprecated)) virtual void setFp16Mode(bool mode)=0
 Set whether or not 16-bit kernels are permitted. More...
 
 __attribute__ ((deprecated)) virtual bool getFp16Mode() const =0
 Query whether 16-bit kernels are permitted. More...
 
 __attribute__ ((deprecated)) virtual void setStrictTypeConstraints(bool mode)=0
 Set whether or not type constraints are strict. More...
 
 __attribute__ ((deprecated)) virtual bool getStrictTypeConstraints() const =0
 Query whether or not type constraints are strict. More...
 
 __attribute__ ((deprecated)) virtual void setRefittable(bool canRefit)=0
 Set whether engines will be refittable. More...
 
 __attribute__ ((deprecated)) virtual bool getRefittable() const =0
 Query whether or not engines will be refittable. More...
 
 __attribute__ ((deprecated)) virtual void setEngineCapability(EngineCapability capability)=0
 Configure the builder to target specified EngineCapability flow. More...
 
 __attribute__ ((deprecated)) virtual EngineCapability getEngineCapability() const =0
 Query EngineCapability flow configured for the builder. More...
 
virtual nvinfer1::IBuilderConfigcreateBuilderConfig ()=0
 Create a builder configuration object. More...
 
virtual nvinfer1::ICudaEnginebuildEngineWithConfig (INetworkDefinition &network, IBuilderConfig &config)=0
 Builds an engine for the given INetworkDefinition and given IBuilderConfig. More...
 
virtual nvinfer1::INetworkDefinitioncreateNetworkV2 (NetworkDefinitionCreationFlags flags)=0
 Create a network definition object. More...
 
virtual nvinfer1::IOptimizationProfilecreateOptimizationProfile () noexcept=0
 Create a new optimization profile. More...
 
virtual void setErrorRecorder (IErrorRecorder *recorder)=0
 Set the ErrorRecorder for this interface. More...
 
virtual IErrorRecordergetErrorRecorder () const =0
 get the ErrorRecorder assigned to this interface. More...
 
virtual void reset ()=0
 Resets the builder state to default values. More...
 
virtual bool platformHasTf32 () const =0
 Determine whether the platform has TF32 support. More...
 

Public Attributes

DeviceType deviceType = 0
 

Protected Member Functions

virtual ~IBuilder ()
 Resets the builder state. More...
 

Detailed Description

Builds an engine from a network definition.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IBuilder()

virtual nvinfer1::IBuilder::~IBuilder ( )
inlineprotectedvirtual

Resets the builder state.

Deprecated:
API will be removed in TensorRT 8.0, use IBuilder::reset instead.

Member Function Documentation

◆ setMaxBatchSize()

virtual __attribute__ ((deprecated)) virtual nvinfer1 void nvinfer1::IBuilder::setMaxBatchSize ( int32_t  batchSize)
pure virtual

Create a network definition object where all tensors have an implicit batch dimension.

This method is equivalent to createNetworkV2(0U), and retained for compatibility with earlier version of TensorRT. The network does not support dynamic shapes or explicit batch sizes.

See also
INetworkDefinition, createNetworkV2
Deprecated:
API will be removed in TensorRT 8.0, use IBuilder::createNetworkV2 instead.

Set the maximum batch size.

Parameters
batchSizeThe maximum batch size which can be used at execution time, and also the batch size for which the engine will be optimized.
See also
getMaxBatchSize()

◆ getMaxBatchSize()

virtual int32_t nvinfer1::IBuilder::getMaxBatchSize ( ) const
pure virtual

Get the maximum batch size.

Returns
The maximum batch size.
See also
setMaxBatchSize()
getMaxDLABatchSize()

◆ __attribute__() [1/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query whether half2 mode is used.

See also
setHalf2Mode()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getFlag instead.

◆ __attribute__() [2/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set whether the builder should use debug synchronization.

Set the maximum workspace size.

If this flag is true, the builder will synchronize after timing each layer, and report the layer name. It can be useful when diagnosing issues at build time.

Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.
Parameters
workspaceSizeThe maximum GPU temporary memory which the engine can use at execution time.
See also
getMaxWorkspaceSize()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setMaxWorkspaceSize instead.

Get the maximum workspace size.

Returns
The maximum workspace size.
See also
setMaxWorkspaceSize()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getMaxWorkspaceSize instead.

Set whether half2 mode is used.

half2 mode is a paired-image mode that is significantly faster for batch sizes greater than one on platforms with fp16 support.

Parameters
modeWhether half2 mode is used.
See also
getHalf2Mode()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.

◆ __attribute__() [3/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query whether the builder will use debug synchronization.

See also
setDebugSync()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getFlag instead.

◆ __attribute__() [4/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set the number of minimization iterations used when timing layers.

When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in minimization.

See also
getMinFindIterations()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setMinTimingIterations instead.

◆ __attribute__() [5/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query the number of minimization iterations.

See also
setMinFindIterations()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getMinTimingIterations instead.

◆ __attribute__() [6/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set the number of averaging iterations used when timing layers.

When timing layers, the builder minimizes over a set of average times for layer execution. This parameter controls the number of iterations used in averaging.

See also
getAverageFindIterations()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setAvgTimingIterations instead.

◆ __attribute__() [7/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query the number of averaging iterations.

See also
setAverageFindIterations()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getAvgTimingIterations instead.

◆ platformHasFastFp16()

virtual __attribute__ ((deprecated)) virtual nvinfer1 bool nvinfer1::IBuilder::platformHasFastFp16 ( ) const
pure virtual

Build a CUDA engine from a network definition.

See also
INetworkDefinition ICudaEngine
Deprecated:
API will be removed in TensorRT 8.0, use IBuilder::buildEngineWithConfig instead.

Determine whether the platform has fast native fp16.

◆ platformHasFastInt8()

virtual bool nvinfer1::IBuilder::platformHasFastInt8 ( ) const
pure virtual

Determine whether the platform has fast native int8.

◆ destroy()

virtual void nvinfer1::IBuilder::destroy ( )
pure virtual

Destroy this object.

◆ __attribute__() [8/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set whether or not quantized 8-bit kernels are permitted.

During engine build int8 kernels will also be tried when this mode is enabled.

Parameters
modeWhether quantized 8-bit kernels are permitted.
See also
getInt8Mode()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.

◆ __attribute__() [9/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query whether Int8 mode is used.

See also
setInt8Mode()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getFlag instead.

◆ __attribute__() [10/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set Int8 Calibration interface.

Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setInt8Calibrator instead.

◆ __attribute__() [11/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )

Set the device that this layer must execute on.

Parameters
DeviceTypethat this layer must execute on. If DeviceType is not set or is reset, TensorRT will use the default DeviceType set in the builder.
Note
The device type for a layer must be compatible with the safety flow (if specified). For example a layer cannot be marked for DLA execution while the builder is configured for kSAFE_GPU.
See also
getDeviceType()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setDeviceType instead.

◆ __attribute__() [12/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Get the device that this layer executes on.

Returns
Returns DeviceType of the layer.
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getDeviceType instead.

◆ __attribute__() [13/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

whether the DeviceType has been explicitly set for this layer

Returns
whether the DeviceType has been explicitly set
See also
setDeviceType() getDeviceType() resetDeviceType()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::isDeviceTypeSet instead.

◆ __attribute__() [14/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

reset the DeviceType for this layer

See also
setDeviceType() getDeviceType() isDeviceTypeSet()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::resetDeviceType instead.

◆ __attribute__() [15/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Checks if a layer can run on DLA.

Returns
status true if the layer can on DLA else returns false.
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::canRunOnDLA instead.

◆ __attribute__() [16/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Sets the default DeviceType to be used by the builder.

It ensures that all the layers that can run on this device will run on it, unless setDeviceType is used to override the default DeviceType for a layer.

See also
getDefaultDeviceType()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setDefaultDeviceType instead.

◆ __attribute__() [17/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Get the default DeviceType which was set by setDefaultDeviceType.

Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getDefaultDeviceType instead.

◆ getMaxDLABatchSize()

virtual int32_t nvinfer1::IBuilder::getMaxDLABatchSize ( ) const
pure virtual

Get the maximum batch size DLA can support.

For any tensor the total volume of index dimensions combined(dimensions other than CHW) with the requested batch size should not exceed the value returned by this function.

Warning
getMaxDLABatchSize does not work with dynamic shapes.

◆ __attribute__() [18/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Sets the builder to use GPU if a layer that was supposed to run on DLA can not run on DLA.

Parameters
Allowsfallback if setFallBackMode is true else disables fallback option.
Note
GPU fallback may only be specified for non-safety modes.
See also
EngineCapability Simultaneously enabling GPU fallback and safety-restricted modes is disallowed.
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.

◆ getNbDLACores()

virtual int32_t nvinfer1::IBuilder::getNbDLACores ( ) const
pure virtual

Return the number of DLA engines available to this builder.

◆ __attribute__() [19/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set the DLA core that the engine must execute on.

Parameters
dlaCoreThe DLA core to execute the engine on (0 to N-1, where N is the maximum number of DLA cores present on the device). Default value is 0. DLA Core is not a property of the engine that is preserved by serialization: when the engine is deserialized it will be associated with the DLA core which is configured for the runtime.
See also
IRuntime::setDLACore() getDLACore()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setDLACore instead.

◆ __attribute__() [20/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Get the DLA core that the engine executes on.

Returns
If setDLACore is called, returns DLA core from 0 to N-1, else returns 0.
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getDLACore instead.

◆ setGpuAllocator()

virtual void nvinfer1::IBuilder::setGpuAllocator ( IGpuAllocator allocator)
pure virtual

Set the GPU allocator.

Parameters
allocatorSet the GPU allocator to be used by the builder. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

Note
This allocator will be passed to any engines created via the builder; thus the lifetime of the allocator must span the lifetime of those engines as well as that of the builder. If nullptr is passed, the default allocator will be used.

◆ __attribute__() [21/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set whether or not 16-bit kernels are permitted.

During engine build fp16 kernels will also be tried when this mode is enabled.

Parameters
modeWhether 16-bit kernels are permitted.
See also
getFp16Mode()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.

◆ __attribute__() [22/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query whether 16-bit kernels are permitted.

See also
setFp16Mode()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getFlag instead.

◆ __attribute__() [23/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set whether or not type constraints are strict.

When strict type constraints are in use, TensorRT will always choose a layer implementation that conforms to the type constraints specified, if one exists. If this flag is not set, a higher-precision implementation may be chosen if it results in higher performance.

If no conformant layer exists, TensorRT will choose a non-conformant layer if available regardless of the setting of this flag.

See the developer guide for the definition of strictness.

Parameters
modeWhether type constraints are strict
See also
getStrictTypeConstraints()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.

◆ __attribute__() [24/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query whether or not type constraints are strict.

See also
setStrictTypeConstraints()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getFlag instead.

◆ __attribute__() [25/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Set whether engines will be refittable.

Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setFlag instead.

◆ __attribute__() [26/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query whether or not engines will be refittable.

See also
getRefittable()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getFlag instead.

◆ __attribute__() [27/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  )
pure virtual

Configure the builder to target specified EngineCapability flow.

Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::setEngineCapability instead.

◆ __attribute__() [28/28]

nvinfer1::IBuilder::__attribute__ ( (deprecated)  ) const
pure virtual

Query EngineCapability flow configured for the builder.

See also
setEngineCapability()
Deprecated:
API will be removed in TensorRT 8.0, use IBuilderConfig::getEngineCapability instead.

◆ createBuilderConfig()

virtual nvinfer1::IBuilderConfig* nvinfer1::IBuilder::createBuilderConfig ( )
pure virtual

Create a builder configuration object.

See also
IBuilderConfig

◆ buildEngineWithConfig()

virtual nvinfer1::ICudaEngine* nvinfer1::IBuilder::buildEngineWithConfig ( INetworkDefinition network,
IBuilderConfig config 
)
pure virtual

Builds an engine for the given INetworkDefinition and given IBuilderConfig.

It enables the builder to build multiple engines based on the same network definition, but with different builder configurations.

◆ createNetworkV2()

virtual nvinfer1::INetworkDefinition* nvinfer1::IBuilder::createNetworkV2 ( NetworkDefinitionCreationFlags  flags)
pure virtual

Create a network definition object.

Creates a network definition object with immutable properties specified using the flags parameter. Providing the kDEFAULT flag as parameter mimics the behaviour of createNetwork(). CreateNetworkV2 supports dynamic shapes and explicit batch dimensions when used with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag.

Parameters
flagsBitset of NetworkDefinitionCreationFlags specifying network properties combined with bitwise OR. e.g., 1U << NetworkDefinitionCreationFlag::kEXPLICIT_BATCH
See also
INetworkDefinition, NetworkDefinitionCreationFlags

◆ createOptimizationProfile()

virtual nvinfer1::IOptimizationProfile* nvinfer1::IBuilder::createOptimizationProfile ( )
pure virtualnoexcept

Create a new optimization profile.

If the network has any dynamic input tensors, the appropriate calls to setDimensions() must be made. Likewise, if there are any shape input tensors, the appropriate calls to setShapeValues() are required. The builder retains ownership of the created optimization profile and returns a raw pointer, i.e. the users must not attempt to delete the returned pointer.

See also
IOptimizationProfile

◆ setErrorRecorder()

virtual void nvinfer1::IBuilder::setErrorRecorder ( IErrorRecorder recorder)
pure virtual

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder

◆ getErrorRecorder()

virtual IErrorRecorder* nvinfer1::IBuilder::getErrorRecorder ( ) const
pure virtual

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A default error recorder does not exist, so a nullptr will be returned if setErrorRecorder has not been called.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder

◆ reset()

virtual void nvinfer1::IBuilder::reset ( )
pure virtual

Resets the builder state to default values.

◆ platformHasTf32()

virtual bool nvinfer1::IBuilder::platformHasTf32 ( ) const
pure virtual

Determine whether the platform has TF32 support.

Member Data Documentation

◆ deviceType

DeviceType nvinfer1::IBuilder::deviceType = 0

The documentation for this class was generated from the following file: