June at Portkey: Agents, AI Governance, and Twenty Six Cookbooks

Last month, we released 6+ integrations with AI Agent platforms, shipped crucial AI Governance features, integrated Claude 3.5, published 26 cookbooks, and more.

Here's a rundown:

Full-stack Telemetry for your Agents

Portkey now integrates with all the leading agent frameworks and gives you full-stack telemetry over everything - see granular logs & traces, tabulate costs for each run, monitor performance metrics, and debug your agents easily.

Cookbooks: Llama Agents | CrewAI | Autogen | ControlFlow | PhiData | LangChain

AI Governance

On Portkey Enterprise, we now offer advanced AI governance features -

1. Effortlessly provision LLMs for rest of the org with appropriate permissions.
2. Onboard the whole company programmatically with Admin APIs and implement RBAC.
3. See how different teams are utilising AI with our monitoring setup.

If that sounds like something you'd want for your org, schedule a call here.

New Models

Claude 3.5

Try out Anthropic's GPT4o-beating model both through the Anthropic API as well as AWS Bedrock, using Portkey.

Other exciting models:

• Qwen 2 - Alibaba's impressive model that you can also run locally. Try here.
• Gemma 2 - Google's small but mighty model that you can route to now using Portkey. Try here.
• Nemotron 340B - Nvidia's 340B(!) model with which you can create large-scale synthetic data. Try here.

That's not all!

• Portkey is now a PWA: You can install Portkey as a standalone app right from your browser.
• Prompts with Unlimited Tokens: Portkey's multimodal prompt templates now support unlimited context window.
• Fireworks Models on Prompt Playground: 80+ cutting edge LLMs from open source to private - now available on Portkey prompt playground through Fireworks.
• Request Timeout Header: Set request timeouts directly with the x-portkey-request-timeout header.
• Configs: Now, duplicate, edit and delete any saved Config.
• Changelog in the app: Browse all the latest Portkey updates right from inside the Portkey app.

Browse the full changelog here.

LLMs in Prod

We organized a packed event of top AI leaders and practitioners from Bengaluru, who came together to discuss the production challenge of building LLM apps.

• Rohit Agarwal (Portkey CEO) talked about the AI Gateway Pattern
• Rajaswa & Nitish from Postman shared their lessons building Postman's Gen AI app - Postbot
• Ganaraj talked about how he is using DSPy in production!

Subscribe to Portkey on Youtube for recordings.

That's it for this month. Stay tuned - we are shipping some very exciting features next! To  get early dibs on Portkey features, join the LLMs in Prod community below: