May: Major Updates
Set budget & rate limits, streamline team permissions, GPT-4o, Gemini Flash, Instructor & Promptfoo integrations, and more.
May has been a month of incredible progress at Portkey - we shipped some very important features.
First up - Full Control Over Your Org & LLM Spending
We've supercharged Portkey with features that put you in the driver's seat:
Set Budget Limits & Rate Controls
- Set budget limits on provider keys to keep costs in check
- Create Rate-Limited Portkey API keys to optimize your AI service usage
Streamline Team Access and Permissions
- Generate Portkey API keys with specific permissions (e.g., "Generations-Only") for fine-grained control
- Invite team members or external collaborators with customized access levels to Portkey
New Models: Day 0 Integrations
From OpenAI to Google, the biggest model releases are here, and Portkey has Day 0 integrations for all.
- GPT-4o - Introduction cookbook
- Gemini 1.5 Flash - See how it compares
- Codestral - Invoke Mistral's new endpoint
New Providers
- ZhipuAI - Creators of ChatGLM, one of China's most capable LLMs. Docs
- Predibase - The ultimate fine-tuning platform. Docs
- MonsterAPI - Access LLMs with up to 80% lower costs. Docs
Portkey ❤️ Instructor
Portkey now natively integrates with the popular open-source Instructor library to extract structured output from LLMs. Available in Python and JS.
Deepening our Promptfoo Integration
We like Promptfoo so much, that you can now run your Promptfoo evals on 200+ different LLMs seamlessly using its native Portkey integration! (and get full-stack observability to debug and segment the results)
There's more!
- Cache Namespace - Simplify cache partitioning. Docs
- Gemini function calling - Now supported in the familiar OpenAI style! Announcement
- p50, p90 latencies - Compare latencies across providers, models, and prompts with a click. Check it out
- OpenAI Orgs & projects - You can now specify your OpenAI org and project IDs when integrating with Portkey. Docs
- Deploy Portkey in your cloud - Host and manage Portkey within your own cloud infrastructure. Docs
Reddit AMA
Portkey's CTO recently did an AMA on Reddit. There were some interesting questions!
Check out the full AMA here.
That's it! Join Portkey's Discord community to stay updated as we launch new features.
