Dedicated Model Inference

Dedicated inference, tuned for production

Deploy any open model in minutes. Roll out safely, scale to meet demand, and keep full control. No platform team required.

Why Dedicated Inference with Together AI?

Designed for production workloads that need consistent performance and operational control.

Production control plane

Safe rollouts, autoscaling, fast cold starts, auto-rollback, and multi-region failover.

Best-in-market economics

Faster inference and more tokens per GPU deliver closed-model quality at a lower cost.

Research in production

Frontier research ships into the product continuously, so you're always on the leading edge.

Build with leading models

Explore top-performing models across text, image, video, code, and voice.

Have your own model?

Deploy custom containers on Together’s managed GPU infrastructure with automatic scaling, job queues, and built-in observability.

Key capabilities, purpose-built for AI natives

Bring any open-weight model, deploy it to a dedicated endpoint in minutes, and run it in production with full control and optimized performance.

1. Deploy in minutes
2. No DevOps required
3. Live in minutes
4. Simple configuration

Select a target model and hardware config and be live in minutes. Production-ready endpoints, no deep infra expertise.

Production-grade deployment, built in

The deployment safety a platform team would build — already built and managed.

• Zero-downtime rollouts: Canary, rolling, blue/green & automatic rollback on your thresholds.
• Shadow & A/B: Mirror live traffic to a candidate model; zero user impact.
• Multi-region failover: Declare preferred regions; traffic shifts automatically.
• SLO-driven autoscaling: Scale on TTFT, latency, and throughput, not demo loads.
• Advanced routing: Least-loaded, session affinity, prefix-cache-aware.
• Coming soon: Multi-LoRA serving.

Research that ships

Atlas

CPD

Megakernel

ThunderKittens

Deployment options

Run models using different deployment options depending on latency needs, traffic patterns, and infrastructure control.

Serverless Inference

A fully managed real-time or batch inference API with access to dozens of the most popular AI models.

• Best for variable or unpredictable traffic
• Rapid prototyping and iteration
• Cost-sensitive or early-stage production workloads

Provisioned Throughput

Reserved token capacity with SLA guarantees. Priced in PTUs, a normalized throughput unit.

• Best for production workloads
• Reliability guarantees
• Predictable pricing

Dedicated Model Inference

An inference endpoint backed by reserved, isolated compute resources and Together AI inference research.

• Best for predictable or steady traffic
• Latency-sensitive applications
• High-throughput production workloads

Dedicated Container Inference

Run inference with your own engine and model on fully-managed, scalable infrastructure.

• Best for generative media models
• Non-standard runtimes
• Custom inference pipelines

Single-tenant security and data privacy.

Our data and models remain fully under your ownership, safeguarded by robust security measures.

Inference FAQ

What is dedicated inference? Dedicated inference means running a model on GPUs reserved exclusively for your workload, ensuring consistent latency and full control.

How is dedicated inference different from serverless? Serverless is billed per token on shared capacity. Dedicated inference reserves GPUs at a fixed rate for predictable performance.

Can I deploy my own model? Yes, you can deploy custom models from Hugging Face or your own storage while Together AI manages the infrastructure.

Can I update a deployed model without downtime? Yes, deploying a new model can happen without taking the serving endpoint down, ensuring uninterrupted traffic during updates.