TTFT
Time to First TokenThe time between sending your prompt and receiving the very first token. This is the most critical metric for interactive user experience.
← Back to Product Suite View on GitHub
CURATED LLM & AI PROMPTS FOR PERFORMANCE ENGINEERS
LLM
Performance Engineering LLM Hub
The ultimate hub for LLM performance engineering. Access a curated library of AI prompts for JMeter and k6, along with comprehensive benchmarks and performance metrics like TTFT, TPOT, and ITL.
LLM COPY READY 23 CATEGORIES
CATEGORY
AI / Productivity
PROMPTS
50+ Prompts
TOOLS
JMeter · k6 · Gatling
ACCESS
Free
Categories
Prompt library
Choose a category to open the full, copy-ready prompts.
Test Planning & Strategy
5 promptsGenerate a Performance Test Strategy Document
Open full prompt
Define SLA Targets from Business Requirements
Open full prompt
Performance Test Scope Definition
Open full prompt
Test Type Selection
Open full prompt
NFR Review Checklist
Open full prompt
Need the full prompt text? Find it on this category page in the LLM Hub.
View all 5 prompts →Reference Guide
LLM Performance Metrics
A comprehensive guide to the core metrics used to evaluate and benchmark Large Language Model performance. Understanding these is key to delivering high-quality AI experiences.
TTST
Time to Second TokenThe time gap between the first and second tokens. Measures how quickly the model transitions from processing input to active generation.
TPOT
Time Per Output TokenThe average time taken to generate each token after the first. This determines how smooth the streaming text appears to the user.
ITL
Inter-Token LatencyThe average delay between consecutive tokens. Focuses on consistency; high variance here causes 'jittery' or stuttering text.
E2E
End-to-End LatencyTotal time from request start to final token. Useful for SLAs but hides the breakdown between prefill and decode phases.
Throughput
Token & Request ThroughputThe rate of tokens or requests processed per second. Essential for system capacity planning and cost estimation.
Goodput
SLO-Meeting ThroughputThe number of requests per second that actually meet your defined performance SLOs. The 'honest' measure of user experience.
Queue Time
Request Wait TimeTime spent waiting in the inference server's queue before processing begins. Spikes under high concurrency load.
Optimizing for Use Cases
Chatbots
Prioritize TTFT for perceived speed. Users care most about how quickly the first word appears.
Batch Jobs
Focus on Throughput and E2E Latency. Cost per token is usually the primary constraint here.
Code Gen
Prioritize ITL Consistency. Stuttering generation is highly disruptive during active coding tasks.
Connect
Follow QAInsights for updates, tutorials, and community events.