In the world of cloud-native applications, secrets management has evolved from a necessary evil to a critical architectural concern. Azure Key Vault stands as Microsoft’s answer to centralized secrets, keys, and certificate management, providing a secure foundation for enterprise applications. Having implemented Key Vault across dozens of production environments, I’ve come to appreciate its role […]
Read more →AKS Workload Identity
AKS workload identity is a feature of Azure Kubernetes Service (AKS) that enables you to use Azure Active Directory (AAD) to manage access to Azure resources from within a Kubernetes cluster. In this blog post, we’ll explore how AKS workload identity works and how to use it with an example code. How does AKS workload […]
Read more →RESTful AI API Design: Best Practices for LLM APIs
Designing RESTful APIs for LLMs requires careful consideration. After building 30+ LLM APIs, I’ve learned what works. Here’s the complete guide to RESTful AI API design. Figure 1: RESTful AI API Architecture Why LLM APIs Are Different LLM APIs have unique requirements: Async operations: LLM inference can take seconds or minutes Streaming responses: Need to […]
Read more →Function Calling Deep Dive: Building LLM-Powered Tools and Agents
Introduction: Function calling transforms LLMs from text generators into action-taking agents. Instead of just describing what to do, the model can actually do it—query databases, call APIs, execute code, and interact with external systems. OpenAI’s function calling (now called “tools”) and similar features from Anthropic and others let you define available functions, and the model […]
Read more →LlamaIndex: The Data Framework for Building Production RAG Applications
Introduction: LlamaIndex (formerly GPT Index) is the leading data framework for building LLM applications over your private data. While LangChain focuses on chains and agents, LlamaIndex specializes in data ingestion, indexing, and retrieval—the core components of Retrieval Augmented Generation (RAG). With over 160 data connectors through LlamaHub, sophisticated indexing strategies, and production-ready query engines, LlamaIndex […]
Read more →LLM Rate Limiting: Maximizing API Throughput Without Getting Throttled
Introduction: LLM APIs have strict rate limits—requests per minute, tokens per minute, and concurrent request limits. Hit these limits and your application grinds to a halt with 429 errors. Effective rate limiting isn’t just about staying under limits; it’s about maximizing throughput while maintaining reliability. This guide covers practical rate limiting patterns: token bucket algorithms […]
Read more →