Blog

Inference engineering

All Model performance AI engineering Infrastructure News Community AI models Product Foundations Research

Model performance

How we built production-ready speculative decoding with TensorRT-LLM

Pankaj Gupta

Philip Kiely

Pankaj Gupta

2 others

Speculative Decoding with TensorRT-LLM

Model performance

A quick introduction to speculative decoding

Pankaj Gupta

Philip Kiely

Pankaj Gupta

2 others

Intro to Speculative Decoding

News

Introducing our Speculative Decoding Engine Builder integration for ultra-low-latency LLM inference

Abu Qader

Bryce Dubayah

Justin Yi

3 others

Speculative Decoding in Engine Builder

Model performance

Generally Available: The fastest, most accurate and cost-efficient Whisper transcription

William Gao

Derrick Yang

William Gao

3 others

The fastest Whisper

News

Introducing Custom Servers: Deploy production-ready model servers from Docker images

Bola Malek

Tianshu Cheng

2 others

Custom servers

News

Create custom environments for deployments on Baseten

Samiksha Pal

Raymond Cano

Sid Shanker

Samiksha Pal

3 others

Custom environments

News

Introducing canary deployments on Baseten

Sid Shanker

Jonathan Rochette

Raymond Cano

Sid Shanker

3 others

Canary Deployments

Infrastructure

Evaluating NVIDIA H200 Tensor Core GPUs for LLM inference

Pankaj Gupta

Philip Kiely

Pankaj Gupta

1 other

NVIDIA H200

News

Export your model inference metrics to your favorite observability tool

Helen Yang

Nicolas Gere-lamaysouette

Philip Kiely

Helen Yang

2 others

Export your inference metrics

1...20 21 22...30