TensorRT (+ Triton Inference Server)
Created: 03 Jan 2023, 11:23 AM | Modified: =dateformat(this.file.mtime,"dd MMM yyyy, hh:mm a")
Tags: knowledge, tools, TinyML
Triton ⇒ serve model over HTTP or gRPC (i.e. over some network, hosted somewhere else?)
-image1.png)
https://www.nvidia.com/en-us/on-demand/session/gtcspring21-se2690/
Triton will require a Triton Server and a Triton Client
-image2.png)
-image3.png)
-image4.png)
-image5.png)
-image6.png)
-image7.png)