May: Major Updates
Set budget & rate limits, streamline team permissions, GPT-4o, Gemini Flash, Instructor & Promptfoo integrations, and more.
![May: Major Updates](/blog/content/images/size/w1200/2024/07/ayush-ama.jpeg)
May has been a month of incredible progress at Portkey - we shipped some very important features.
First up - Full Control Over Your Org & LLM Spending
We've supercharged Portkey with features that put you in the driver's seat:
Set Budget Limits & Rate Controls
- Set budget limits on provider keys to keep costs in check
- Create Rate-Limited Portkey API keys to optimize your AI service usage
Streamline Team Access and Permissions
- Generate Portkey API keys with specific permissions (e.g., "Generations-Only") for fine-grained control
- Invite team members or external collaborators with customized access levels to Portkey
![](https://portkey.ai/blog/content/images/2024/07/budget-limit.gif)
New Models: Day 0 Integrations
From OpenAI to Google, the biggest model releases are here, and Portkey has Day 0 integrations for all.
- GPT-4o - Introduction cookbook
- Gemini 1.5 Flash - See how it compares
- Codestral - Invoke Mistral's new endpoint
New Providers
- ZhipuAI - Creators of ChatGLM, one of China's most capable LLMs. Docs
- Predibase - The ultimate fine-tuning platform. Docs
- MonsterAPI - Access LLMs with up to 80% lower costs. Docs
Portkey ❤️ Instructor
Portkey now natively integrates with the popular open-source Instructor library to extract structured output from LLMs. Available in Python and JS.
![](https://portkey.ai/blog/content/images/2024/07/image-1.png)
Deepening our Promptfoo Integration
We like Promptfoo so much, that you can now run your Promptfoo evals on 200+ different LLMs seamlessly using its native Portkey integration! (and get full-stack observability to debug and segment the results)
![](https://portkey.ai/blog/content/images/2024/07/image-3.png)
There's more!
- Cache Namespace - Simplify cache partitioning. Docs
- Gemini function calling - Now supported in the familiar OpenAI style! Announcement
- p50, p90 latencies - Compare latencies across providers, models, and prompts with a click. Check it out
- OpenAI Orgs & projects - You can now specify your OpenAI org and project IDs when integrating with Portkey. Docs
- Deploy Portkey in your cloud - Host and manage Portkey within your own cloud infrastructure. Docs
Reddit AMA
Portkey's CTO recently did an AMA on Reddit. There were some interesting questions!
![](https://portkey.ai/blog/content/images/2024/07/image-4.png)
Check out the full AMA here.
That's it! Join Portkey's Discord community to stay updated as we launch new features.