User Guide¶

This page describes what a regular Triton Control user can do after signing in. Administrative setup, OIDC configuration, and user management are covered in User Management and the other admin documentation pages.

Access Model¶

Users see only the Triton instances assigned to them. Administrators can see all instances and can also open user and settings pages.

For each assigned instance, viewers can inspect and use read workflows. Members and admins can additionally perform write workflows.

view dashboard and instance health
inspect Triton server metadata
view model repository state
run model inference
browse and download S3 model repository files when S3 is configured
request model load or unload operations (member/admin)
edit or upload S3 model repository files and folders when S3 is configured (member/admin)
update Triton or S3 connection for that instance (member/admin)
add Triton instances (member/admin)
delete Triton instances (admin only)

If an expected instance is missing, an administrator must assign that instance to the user account.

Main sidebar entries:

Dashboard
Triton Instances
Development (when Kubernetes actions are available)
Add Deployment (when Kubernetes actions are available)
Perf Analyzer (when Kubernetes actions are available)
Workflows (when Kubernetes actions and Argo Workflows are available)
Add Instance (button, last nav action): creates a manually managed Triton instance entry.
S3 Profiles (account menu, member/admin): reusable S3 deployment credentials.
Error Logs (admin only)

Within an instance, primary workflows are on the detail page tabs plus dedicated pages for Inference, Profile, and S3 Browser.

Dashboard¶

The dashboard gives a fleet-level view of the instances visible to the current user. It shows summary cards, a table of Triton instances, and active alerts.

For non-admin users, alerts are limited to their assigned instances. Admins see alerts across the full fleet.

Error Logs (Admin Only)¶

The Error Logs page shows recent frontend and backend application errors. Administrators can filter the list by source and select an event to inspect its details.

Frontend events are reported only while a user is signed in. They include:

unhandled Angular or JavaScript runtime errors
backend API responses with HTTP status 500 or higher
available context such as the message, stack or response detail, page or API path, HTTP method and status, user, and browser user agent

The frontend does not report handled component errors, HTTP 4xx responses, plain console.error() calls, or errors that occur before sign-in. Error reporting is best effort; if the reporting request itself fails, it is not reported again.

Backend events include unhandled request exceptions and log records at ERROR or CRITICAL level from application loggers. Sensitive lines containing common credential markers are redacted, and long values are truncated before storage.

Triton Instances¶

Instance List¶

The instance list shows each visible Triton instance with:

instance name and URL
region when configured
health status
number of models
CPU, RAM, and GPU metrics when a metrics endpoint is configured

Use search and status filters to find an instance. Open Details to inspect runtime status, metadata, models, and S3 configuration.

Metrics Behavior¶

CPU, RAM, and GPU values are shown only when the instance has a metrics endpoint configured and Triton exposes usable Prometheus metrics.

if no metrics endpoint is configured, the UI shows N/A
if the endpoint is configured but cannot be read, the detail page shows a metrics error

Example metrics endpoint:

http://triton-server:8002/metrics

If the configured URL has no path, the backend appends /metrics.

Instance Details¶

The instance detail page is tab-based. Depending on instance type and permissions, you will see:

Overview
Models
S3 Connection
Logs (self-deployed instances)

For instances added via Add Instance (existing Triton endpoints), profiling can still be executed from the Models tab using Profile when Perf Analyzer is installed.

For these manually added instances, the Logs tab is not available.

Overview Tab¶

The overview tab shows:

status, version, runtime environment, and assigned users
Triton base URL and optional metrics URL
SSL verification status
/v2/health/live and /v2/health/ready results
Triton /v2 metadata as summary or raw JSON

Members and admins can update Triton endpoint, metrics endpoint, and SSL certificate verification settings from the Edit action in the Networking card header.

Models Tab¶

The models tab shows the live Triton repository index.

If the instance is unhealthy or unreachable, Triton Control cannot read the live repository index and the tab may show no models. Check the Overview tab health details and connection errors first.

For each model version, users can:

see model state and reason returned by Triton
open the inference page
open the profile page (when Perf Analyzer is installed)
request model load (member/admin)
request model unload (member/admin)

Load/unload depends on Triton model control mode. Explicit operations require explicit model control on Triton.

S3 Connection Tab¶

Members and admins can configure S3 endpoint, bucket, credentials, optional region/prefix, SSL verification, and optional CA certificate.

When S3 is active, users can open the S3 Browser from the detail page.

For self-deployed instances created with Add Deployment, this tab controls Triton Control's S3 browser/editor connection only. It does not update the S3 model repository connection inside the already running Triton pod.

Logs Tab (Self-Deployed)¶

Self-deployed instances expose a Logs tab for Kubernetes deployment events and recent Triton pod output.

Model Inference Page¶

Open Infer from a model row to send inference through the backend proxy.

The inference page shows:

target instance and model version
resolved Triton infer URL
JSON request body editor
model configuration side panel
request latency and response JSON
per-request inference metrics (when available)

Request target:

/v2/models/<model>/versions/<version>/infer

Metrics behavior:

preferred source: Prometheus /metrics delta before/after request
fallback: Triton /v2/models/stats

Configuration side panel behavior:

if the instance has an active S3 connection, the panel shows the raw config.pbtxt file for the selected model repository path
members and admins can edit and save that config.pbtxt; viewers can inspect it
if S3 is not configured, the panel shows the read-only Triton Live API Config for the selected model/version
Live API Config is Triton's parsed runtime JSON response, not the raw config.pbtxt file

Model Profile Page (Perf Analyzer)¶

Open Profile from a model row to run model profiling.

Reference:

NVIDIA Perf Analyzer docs: https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/perf_analyzer/README.html

Current behavior:

profile button appears only when Perf Analyzer installation exists
only one profile run can execute at a time
while one run is active, other profile actions are disabled
active run can be reopened to inspect progress/results
the side configuration panel follows the same behavior as the Inference page: editable S3 config.pbtxt when S3 is configured, otherwise read-only Live API Config

S3 Browser Page¶

When S3 is configured and enabled for an instance, the S3 Browser supports:

browsing folders/files under repository prefix
creating folders (member/admin)
downloading files
deleting files and folders (member/admin)
opening editable files (.py, .pbtxt) (member/admin)
editing and saving .py and .pbtxt (member/admin)
uploading files (member/admin)
uploading folders with nested child files (member/admin)

When saving config.pbtxt, backend validation is executed against the Triton model configuration parser for that Triton version.

The same validation is used when saving config.pbtxt from the Inference or Profile page side panel.

Folder upload preserves the selected folder structure below the current S3 browser path. For example, uploading a local folder that contains resnet/1/model.plan while the browser is open at /models writes the object to /models/resnet/1/model.plan.

See Model Config Validation for details about Triton version mapping and protobuf parser generation.

Deleting a file removes that S3 object. Deleting a folder removes all S3 objects below that folder prefix. After a folder delete succeeds, the folder and its child paths are also removed from the S3 Browser tree.

Current limitation:

rename and move actions are not available in the current release

MLflow¶

The MLflow sidebar entry provides a singleton MLflow tracking server managed by Triton Control. Members and admins can install it in Kubernetes, use its UI through the authenticated backend proxy, refresh its readiness state, and uninstall it.

Installation settings include the Kubernetes resource name, MLflow image, and optional .dockerconfigjson credentials for a private image registry. The managed workload uses a 10 GiB ReadWriteOnce PersistentVolumeClaim named from the installation (for the default name, mlflow-data). MLflow stores its SQLite tracking database and local artifacts on this volume.

The Triton Control service account therefore requires namespace-scoped create, update, patch, and delete permissions for PersistentVolumeClaims. The Helm chart provides these permissions when rbac.create=true.

Uninstalling the managed MLflow instance deletes its Deployment, Service, image-pull Secret, and PersistentVolumeClaim. Export or retain required tracking data before uninstalling.

S3 Profiles¶

Members and admins can manage reusable S3 deployment profiles from the account menu. A profile stores:

profile name
endpoint
bucket
optional repository parent prefix
region
access key and encrypted secret key
path-style mode
optional CA certificate

Profiles are private to the signed-in user. Other users, including administrators, cannot list, edit, or delete them through the S3 profile API. Each user must create their own profile, even when multiple users connect to the same object store. Profiles are intended for deployment workflows, especially the code-server Triton Control Deploy extension, so users do not need to re-enter S3 credentials for every deploy. The extension shows the signed-in user's saved profiles in a dropdown and keeps manual S3 fields available in a collapsed optional section.

The stored profile values are used to upload the selected model repository from the code-server workspace and to create the Triton deployment. The resulting deployment still receives its own Kubernetes Secret for the in-pod S3 repository connection.

Add Deployment (Sidebar Entry)¶

Add Deployment is a standalone navigation entry, not an instance detail tab. It creates a self-deployed Triton workload on Kubernetes.

Prerequisite: an S3-compatible object store and bucket must already exist. The bucket, or the optional repository prefix within it, must contain models in the directory structure expected by Triton Inference Server.

In vLLM sync modes, Triton's repository path must always be the directory above the model folder. For example, version files under modela/1 are published as /models/modela/1, while Triton receives --model-repository=/models. When a vLLM S3 URL points directly at a single folder containing config.pbtxt, the sync worker adds the model-name directory automatically.

The S3 settings entered during Add Deployment are written into the Kubernetes deployment. Select a saved S3 profile for repeated deployments, or choose Manual S3 settings for a one-off connection. With a profile, the Repository prefix is the optional path inside the profile bucket and the form shows a Serving repository preview. With manual S3 settings, the prefix and final Target path preview are shown inside the manual settings section.

Models that do not use the vLLM backend use Triton's native S3 model repository directly. Enable vLLM model backend only when deploying vLLM models; Triton Control then uses the vLLM repository sync path internally and Triton reads the stable /models path. Relative model and tokenizer values in vLLM model.json files are changed to absolute paths under that directory. After the deployment is created, Triton Control cannot change that in-pod S3 repository connection in place. To change it, delete the self-deployed Triton instance and create a new deployment with the new S3 settings.

The S3 Connection tab on an instance is different: it configures how Triton Control connects to S3 for browsing, editing, and uploading model repository files. Editing that S3 connection changes Triton Control's backend access to S3; it does not rewrite the S3 connection used by the already running Triton pod.

Field	Required	Purpose	Recommended usage
Deployment name	Yes	Base name for Kubernetes resources (deployment/service/secret) and, in external backend mode, namespace.	Keep stable and DNS-safe.
Image	Yes	Triton server container image to run.	Pin explicit version tags in stage/prod.
S3 profile	Yes	Saved S3 profile or manual S3 settings.	Prefer profiles for repeated deployments.
S3 endpoint	Manual only	Object-store endpoint.	Do not include the repository prefix in this field.
Bucket	Manual only	Object-store bucket.	Keep the bucket separate from the repository prefix.
Repository prefix	Optional	S3 path prefix inside the selected bucket.	Leave empty when models are stored at the bucket root.
Serving repository / Target path	Read-only	Preview of the final S3 repository URL.	Confirm before deploy, especially with manual S3 settings.
S3 CA certificate	Optional	PEM CA certificate for HTTPS S3 endpoints.	Provide for private/self-signed/internal CAs.
Access key / Secret key / Region	Manual only	Repository credentials and region.	Use least-privilege credentials.
vLLM model backend	Optional	Enables vLLM repository handling for vLLM models.	Use only when deploying vLLM models.
Model control mode	Yes	Triton behavior (`explicit` or `poll`).	Use mode based on model operation strategy.
Startup model	Optional (`explicit`)	Model loaded at startup (`*` if empty).	Control startup footprint when needed.
Poll interval	Optional (`poll`)	Seconds between repository checks.	Tune for change rate and API load.
Ingress host/class	Optional	Expose deployment through ingress.	Use for external cluster access.
`.dockerconfigjson`	Optional	Private registry pull credentials.	Required for private images.
`requirements.txt`	Optional	Extra Python packages installed before Triton start.	Prefer dev/stage; bake into image for production.
Resources (GPU/CPU/Memory)	Optional	Kubernetes GPU limit and CPU/memory resource settings.	Enter memory as a whole Gi number; the form adds the `Gi` unit. Set GPU count to at least `1` for vLLM; the form defaults this when vLLM is enabled.

For private container registries, paste Docker registry authentication JSON into the image pull secret field (.dockerconfigjson). The following JSON is only an example of the expected Docker config shape; replace it with the registry, credentials, and optional email value for your own registry:

{
  "auths": {
    "<REGISTRY_HOST>": {
      "auth": "<BASE64_USERNAME_COLON_TOKEN>",
      "email": "<EMAIL>"
    }
  }
}

auth is the base64 encoding of username:token or username:password.

Ingress/TLS note:

Add Deployment does not create TLS certificates
configure TLS separately via ingress controller (for example cert-manager or existing TLS Secret)

When ingress is not configured, Triton Control uses internal service DNS:

http://<service>.<namespace>.svc.cluster.local:<port>

Development Workspace (Sidebar Entry)¶

Development is a standalone product feature that creates a private, Kubernetes-backed code-server workspace for the signed-in user. It is separate from development of the Triton Control application itself.

Users can create one persistent workspace, edit model repositories under /workspace, and deploy them through the bundled Triton Control Deploy extension.

Inside code-server, open the Triton Control Activity Bar view and select New Model Repository to create a starter Triton repository. The command is also available from the command palette and Explorer folder context menu. It can create a single-model repository from Python, ONNX Runtime, TensorRT, TensorRT-LLM, vLLM, or PyTorch/LibTorch templates, or an ensemble pipeline with child model folders and an ensemble config.pbtxt.

Generated config.pbtxt files are templates. Review the model artifacts, input/output tensor names, shapes, and data types before deploying. The Triton Control Activity Bar view marks discovered repositories as review model setup until scaffold placeholders or generic config template values are removed. Selecting a repository row switches to Explorer and reveals the real workspace folder; ready to deploy repositories expose deploy actions in the view and context menu.

The deploy extension has a full webview form for trusted HTTPS or localhost. For plain HTTP or an untrusted local certificate, use Triton Control: Upload Model Repository (Simple Wizard) in code-server. The simple wizard avoids webviews, uploads the repository to S3, and shows the values needed to finish in Add Deployment.

See Development Workspaces for workspace fields, Kubernetes resources, persistence, proxy behavior, deployment steps, and API details.

Perf Analyzer (Sidebar Entry)¶

Perf Analyzer is a standalone navigation entry, not an instance detail tab.

Triton Control supports a shared Perf Analyzer installation for profile runs.

one installation is managed at a time (singleton installation)
profile runs target selected instance/model/version from the Profile page

Namespace behavior:

Perf Analyzer is created in the same namespace as Triton Control
in Kubernetes, this is the namespace of the running Triton Control pod
outside Kubernetes, the namespace defaults to triton-control and can be overridden with TRITON_CONTROL_NAMESPACE, KUBERNETES_NAMESPACE, or POD_NAMESPACE

Current scope:

Add Deployment and Perf Analyzer workflows are intended for development/stage
they are not positioned as full production orchestration in this version

Perf Analyzer can also use a private registry image pull secret. Paste the same .dockerconfigjson format into the Perf Analyzer image pull secret field when the configured SDK image is stored in a private registry. This example is not a fixed template; use the Docker config JSON required by your registry:

{
  "auths": {
    "<REGISTRY_HOST>": {
      "auth": "<BASE64_USERNAME_COLON_TOKEN>",
      "email": "<EMAIL>"
    }
  }
}

Workflows (Sidebar Entry)¶

Workflows embeds the single global Argo Workflows server installed by the Triton Control Helm release. The browser connects only to Triton Control:

/api/workflows/proxy/

The backend authenticates member/admin access and proxies Argo UI, API, streaming, and WebSocket traffic to the internal Argo Server ClusterIP Service. The Argo Service is not exposed directly to the browser.

When argoWorkflows.enabled=false, the page shows that the feature is disabled. When enabled but the Argo Server is unavailable, it displays the readiness error instead of loading the iframe.

Workflows run in the same Kubernetes namespace as Triton Control. Images referenced by uploaded Workflow YAML are separate from the public Argo system images. Private Workflow images require a pull Secret in that namespace.

Add Instance is a dedicated sidebar action (shown as the last nav action). It is used to register an already running Triton endpoint in Triton Control.

Use this path when:

Triton is already deployed outside Triton Control
you want monitoring, model actions, inference, and profile workflows in the UI

Behavior after creation:

the instance appears in Triton Instances like other instances
Profile can be used from the Models tab when Perf Analyzer is installed
the Logs tab is not shown for these manually added instances

HTTPS Triton Connections¶

Set Triton endpoint in Edit Triton connection:

https://triton.example.com:8000

Metrics endpoint can also use HTTPS:

https://triton.example.com:8002/metrics

SSL flag semantics:

SSL Flag	Meaning
Off	No certificate validation (only for local/trusted test environments).
On, no certificate pasted	Validate with system trust store.
On, certificate pasted	Validate with pasted PEM CA certificate.

The certificate field expects a PEM CA certificate, not a private key.

HTTPS S3 Connections¶

S3 endpoint example:

https://s3.example.com