TensorRT  7.2.1.6
NVIDIA TensorRT
Looking for a C++ dev who knows TensorRT?
I'm looking for work. Hire me!
Public Member Functions | Protected Member Functions | List of all members
nvinfer1::IRuntime Class Referenceabstract

Allows a serialized functionally unsafe engine to be deserialized. More...

Public Member Functions

virtual nvinfer1::ICudaEngine * deserializeCudaEngine (const void *blob, std::size_t size, IPluginFactory *pluginFactory) noexcept=0
 Deserialize an engine from a stream. More...
 
virtual void setDLACore (int32_t dlaCore) noexcept=0
 Set the DLA core that the deserialized engine must execute on. More...
 
virtual int32_t getDLACore () const noexcept=0
 Get the DLA core that the engine executes on. More...
 
virtual int32_t getNbDLACores () const noexcept=0
 Returns number of DLA hardware cores accessible. More...
 
virtual void destroy () noexcept=0
 Destroy this object. More...
 
virtual void setGpuAllocator (IGpuAllocator *allocator) noexcept=0
 Set the GPU allocator. More...
 
virtual void setErrorRecorder (IErrorRecorder *recorder) noexcept=0
 Set the ErrorRecorder for this interface. More...
 
virtual IErrorRecorder * getErrorRecorder () const noexcept=0
 get the ErrorRecorder assigned to this interface. More...
 
nvinfer1::ICudaEngine * deserializeCudaEngine (const void *blob, std::size_t size) noexcept
 Deserialize an engine from a stream when plugin factory is not used. More...
 

Protected Member Functions

virtual ~IRuntime ()
 

Detailed Description

Allows a serialized functionally unsafe engine to be deserialized.

Warning
Do not inherit from this class, as doing so will break forward-compatibility of the API and ABI.

Constructor & Destructor Documentation

◆ ~IRuntime()

virtual nvinfer1::IRuntime::~IRuntime ( )
inlineprotectedvirtual

Member Function Documentation

◆ deserializeCudaEngine() [1/2]

virtual nvinfer1::ICudaEngine* nvinfer1::IRuntime::deserializeCudaEngine ( const void *  blob,
std::size_t  size,
IPluginFactory *  pluginFactory 
)
pure virtualnoexcept

Deserialize an engine from a stream.

Parameters
blobThe memory that holds the serialized engine.
sizeThe size of the memory.
pluginFactoryThe plugin factory, if any plugins are used by the network, otherwise nullptr.
Returns
The engine, or nullptr if it could not be deserialized.
Here is the caller graph for this function:

◆ setDLACore()

virtual void nvinfer1::IRuntime::setDLACore ( int32_t  dlaCore)
pure virtualnoexcept

Set the DLA core that the deserialized engine must execute on.

Parameters
dlaCoreThe DLA core to execute the engine on (0 to N-1, where N is the maximum number of DLA's present on the device). Default value is 0.
See also
getDLACore()
Warning
Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.
Here is the caller graph for this function:

◆ getDLACore()

virtual int32_t nvinfer1::IRuntime::getDLACore ( ) const
pure virtualnoexcept

Get the DLA core that the engine executes on.

Returns
If setDLACore is called, returns DLA core from 0 to N-1, else returns 0.
Warning
Starting with TensorRT 8, the default value will be -1 if the DLA is not specified or unused.

◆ getNbDLACores()

virtual int32_t nvinfer1::IRuntime::getNbDLACores ( ) const
pure virtualnoexcept

Returns number of DLA hardware cores accessible.

◆ destroy()

virtual void nvinfer1::IRuntime::destroy ( )
pure virtualnoexcept

Destroy this object.

◆ setGpuAllocator()

virtual void nvinfer1::IRuntime::setGpuAllocator ( IGpuAllocator *  allocator)
pure virtualnoexcept

Set the GPU allocator.

Parameters
allocatorSet the GPU allocator to be used by the runtime. All GPU memory acquired will use this allocator. If NULL is passed, the default allocator will be used.

Default: uses cudaMalloc/cudaFree.

If nullptr is passed, the default allocator will be used.

◆ setErrorRecorder()

virtual void nvinfer1::IRuntime::setErrorRecorder ( IErrorRecorder *  recorder)
pure virtualnoexcept

Set the ErrorRecorder for this interface.

Assigns the ErrorRecorder to this interface. The ErrorRecorder will track all errors during execution. This function will call incRefCount of the registered ErrorRecorder at least once. Setting recorder to nullptr unregisters the recorder with the interface, resulting in a call to decRefCount if a recorder has been registered.

Parameters
recorderThe error recorder to register with this interface.
See also
getErrorRecorder

◆ getErrorRecorder()

virtual IErrorRecorder* nvinfer1::IRuntime::getErrorRecorder ( ) const
pure virtualnoexcept

get the ErrorRecorder assigned to this interface.

Retrieves the assigned error recorder object for the given class. A default error recorder does not exist, so a nullptr will be returned if setErrorRecorder has not been called.

Returns
A pointer to the IErrorRecorder object that has been registered.
See also
setErrorRecorder

◆ deserializeCudaEngine() [2/2]

nvinfer1::ICudaEngine* nvinfer1::IRuntime::deserializeCudaEngine ( const void *  blob,
std::size_t  size 
)
inlinenoexcept

Deserialize an engine from a stream when plugin factory is not used.

Parameters
blobThe memory that holds the serialized engine.
sizeThe size of the memory.
Returns
The engine, or nullptr if it could not be deserialized.
Here is the call graph for this function:
The documentation for this class was generated from the following file: