⏬Optimize Claude and avoid hitting token/compute limits!

The Core Equation

The transcript establishes that running out of limits is a matter of total compute budget, not just raw token count. The system runs on a strict formula:

Compute Budget Used = Tokens Consumed X Model Used

To stop hitting limits, you must optimize either the tokens consumed or the model tier you are running.

Part 1: Quick Wins (Token Optimization)

1. Fix Contextual Habits

As a chat progresses, the context window fills up, exponentially draining your compute.

• Clear Tasks: Run /clear or start a new chat whenever you switch tasks.

• Work in Focus Blocks: Avoid leaving a chat for more than 5–10 minutes to maintain Claude's smart caching benefits (which expire on delayed responses).

• Adjust Effort: Lower the default execution effort (low/medium/high) in the desktop app to reduce compute per task.

• Compress Threads: Type /compact when the context window reaches roughly 60% capacity to summarize the history and save space.

2. Contextual Cleanup

Preloaded data takes up token space before you even type a word. Type /context in a fresh chat to see what is preloading, then clean it up:

• Manage MCPs: Run /mcp and delete any unused Model Context Protocol extensions.

• Trim Skills: Archive unused skills and shorten overly wordy skill descriptions.

• Optimize claude.md: This file is read on every single message. Keep it under 200 lines and focus on high-level interaction rules rather than deep project documentation.

3. Reduce Output Tokens

Output tokens are a smaller percentage of overall use but still consume budget.

• Add instructions to your claude.md telling Claude to "be concise."

• Alternatively, use community tricks or plugins (like the "Caveman" plugin) to strictly force ultra-short responses.

Part 2: System Upgrades (60% to 90% Efficiency Gains)

1. Compress Inputs via RTK

Instead of dumping raw, multi-page logs or files into Claude, use an open-source preprocessing tool like RTK.

• RTK uses deterministic computer logic to clean up text, remove boilerplate/formatting noise, and eliminate repeated text before passing it to Claude.

• Tests show this can reduce input token sizes by 60% to 90%.

2. Subagents on Minimum Viable Models (MVM)

Not every task requires a top-tier frontier model. If an AI could solve a task a year ago (e.g., basic scraping, formatting, file fetching), use a lighter model like Haiku instead of Sonnet/Opus, saving up to 90% compute.

• Define the model inside specific Claude "Skills."

• Use context: fork inside a skill to spin up a completely fresh thread, preventing the main conversation's massive context history from bloat-loading into the subtask.

3. Script-Driven Skills

For entirely repeatable tasks, transition the workflow away from AI. Use computer logic or code scripts wrapped inside a Claude skill.

Rule of thumb: Use AI for judgment, and use scripts for repeatable execution. Scripts cost zero tokens and eliminate hallucinations.

Part 3: Nuclear Enhancements (Advanced Shifts)

Enhancement	Description	Pros/Cons
1. Route Work to Codex	Install a Codex plugin to have Claude route execution-heavy, token-burning tasks to OpenAI's infrastructure.	Pros: Codex can be up to 4x more efficient for surgical code edits. Cons: Requires managing two ecosystems.
2. Images Instead of Text	Use tools like PXpipe to convert large blocks of text into an image before uploading it.	Pros: Can result in a 60–70% token reduction. Cons: Slight risk of text misinterpretation; might be patched by Anthropic.
3. Swap the Engine Entirely	Change environment variables in Claude Code to route requests to cheaper external providers (e.g., DeepSeek or GLM).	Pros: Drastically higher compute capacity per dollar. Cons: Slight drop in model intelligence; data privacy considerations.
4. Run Local Models	Route requests to an office server or hardware (like a Mac Mini) using open-source models.	Pros: Infinite, free tokens; 100% data privacy. Cons: Consumer hardware cannot run top-tier frontier models; high setup and maintenance costs ($10k+ for equivalent hardware). Not recommended for most users right now.

Summary Checklist (Speed Run)

1. Every Task Switch: Run /clear.

2. At 60% Context: Run /compact.

3. claude.md Limit: Keep it under 200 lines.

4. Pre-processing: Install RTK to compress inputs.

5. Delegation: Push grunt work to Haiku (MVM) and repeatable tasks to Scripts.

6. Heavy Coding: Route heavy edits to Codex.

Credits: https://www.youtube.com/watch?v=SFh6MMe-XcM 

Use Claude Code effectively

Use these power phrases to transform your work!

1) "Launch sub-agents" : Used to trigger parallel task execution, allowing multiple agents to handle independent tasks simultaneously rather than running them sequentially.

2) "Write me an implementation spec": Used to force the AI to create a detailed plan before building, which prevents incorrect assumptions and reduces the chances of having to redo work.

3) "Interview me": Used to shift the burden of questioning to Claude. The model will ask you the necessary questions to clarify the project, ensuring you haven't missed critical details.

4) "Verify before you build": Used to establish feedback loops. This involves updating your Claude.md file and using appropriate tools to ensure the AI can validate its output against your requirements before completion.

5) "Based on this conversation, build me a skill": Used to package repetitive processes into reusable "skills" that include a "gotchas" section to avoid repeating previous mistakes.

6) "Automate this": The most dangerous phrase. advises caution here, recommending you first perform a "taste test" (does the task require human judgment?) and an "80/20 analysis" to decide if a task is actually suitable for automation or if it should merely be augmented.

Karpathy's Method to effectively use Claude Code

Karpathy's Method to effectively use Claude Code

This framework optimizes your development loop with Claude Code by enforcing extreme alignment up front, establishing an automated quality assurance (QA) verification cycle, and closing critical guardrail gaps before any code is generated.

Step 1: The Build Spec Phase (Up-Front Alignment)

The most common point of failure in AI generation is building the wrong thing beautifully. Instead of starting with an expansive feature list, force the AI to grill you on the core constraints. This step establishes a highly modular, compartmentalized specification.

The Expanded Prompt

Roleplay as an elite Principal Software Architect and Systems Designer. Interview me to uncover the absolute core objective of this project. Do not accept vague requirements. 

Your interview process must follow these constraints:

1. One Question at a Time: Ask exactly one highly targeted question at a time. Do not overwhelm me with a list of questions.

2. Force Scope Minimization: Actively push back against scope creep. Guide me toward building a "Minimum Viable Spec" that is small, modular, and compartmentalized.

3. Explicit Decision Verification: For every key architectural or technical decision (e.g., database choice, state management, auth flow), explicitly make me choose and verify it. Do not make assumptions for me.

4. Final Spec Compilation: Only when we both agree we have hit the core target, compile the final, modular, step-by-step build spec.

Start the interview by asking me: "What is the single most important user action or core value proposition of this new tool/feature?"

Step 2: The Verify Loop (Dual-Agent Quality Assurance)

Establishing a definition of "done" is useless if there is no objective system to measure the output against it. By defining precise quality criteria, referencing a gold-standard past implementation, and leveraging the Codex plugin, you create a robust validation loop.

Pre-requisites & Setup

Ensure the Codex validation plugin is successfully active in your Claude Code environment to allow cross-agent evaluation.

The Expanded Prompt

We are about to begin the implementation phase, but we must establish our QA guardrails first. Before you write a single line of code, execute the following protocol:

1. Define Success Metrics: List the exact technical, performance, and behavioral criteria that define a "perfect" result for this task.

2. Establish the Standard: I will provide a past example of high-quality output [Insert or paste your past reference code/spec here]. Analyze its format, structure, and style, and commit to matching this exact standard.

3. Initiate the Codex Loop: Once you complete the code, you must format a payload for the Codex plugin. Ask Codex to act as an independent, adversarial code reviewer. Have Codex audit your output against our success metrics and the reference standard, and output a strict "PASS" or "FAIL with explicit issues list."

Acknowledge these three rules and summarize the criteria you will use to evaluate your own work before we start.

Step 3: Set the Environment & Close Guardrail Gaps

Before allowing Claude Code to touch your codebase, it must be fully context-aware of your environment, preferences, and project boundaries. This prompt forces a thorough audit of your repository's context files, explicitly pointing out holes and setting up hard authorization hooks.

The Expanded Prompt

Analyze my current local workspace. Specifically inspect my CLAUDE.md, my internal knowledge base/docs, my documented developer skills, and any existing safety guardrails. 

Identify the top 5 most critical information or security gaps in our current environment. For each of these 5 gaps, output a highly structured analysis in the following format:

- File Name/Path: [Where the gap lives, or where the new file should be created]

- The Problem: [Why this gap is dangerous or slows down the development context]

- The Exact Fix: [Provide the specific, copy-pasteable Markdown or configuration text to resolve the gap]

- Safety Hook Flag: [Specify if this action is a high-risk operation (e.g., destructive db runs, prod deploys, dependency additions) and draft the exact pre-execution confirmation hook needed to prevent automated bypass]

Do not generalize. Give me highly specific, actionable code and configuration blocks.

💡 Pro-Tip: Advanced Status Line Monitoring

When utilizing Claude Code, monitoring your token consumption and system usage in real time is crucial for preventing unexpected rate limits and tracking cost efficiency.

As soon as you log in, run the following command to enable the detailed, persistent status line at the bottom of your terminal:

Bash

claude
/statusline model context effort size 5 hour limit weekly limit

 

Why this matters: This configures the CLI to display active project context, real-time token spend, and current session usage metrics, allowing you to visually monitor your resource burn rate during intensive build and test cycles.

New roles in AI - Boris Cherny blog post

Boris Cherny blog post

As engineering, product, design, DS, etc. melt into a new kind of role, I was reflecting on what roles might look like in the future. For example, when I look at the Claude Code team I see what I think is five archetypes:

1. Prototyper: comes up with brand new ideas; churns out many ideas, most of which don't ship

2. Builder: quickly turns a prototype/idea into production-grade product/infra

3. Sweeper: cleans up the UI, simplifies the code and system, unships, optimizes performance

4. Grower: takes a product that has been built and iterates on it to improve Product-Market Fit

5. Maintainer: owns a mature system to make it secure, reliable, fast, and efficient as it scales

Many people span across 2 roles, and sometimes 3 roles. I also notice that these roles are not really tied to job function -- eg. across Anthropic, some designers match category 1, some 2, some 3; same for engineers, PM, DS.

A healthy team needs a mix of these, depending on the product:

- A product that is new and pre-PMF needs people that are strong at 1+2+3

- A product that is growing and has found PMF needs 2+3+4 and some 5

- A product that has strong PMF needs 3+4+5 and some 2

Maybe product roles of the future will look more like this, and less like the domain-specific roles of today?

LLMaaS, LLMOps and Enterprise AI Platform!

In this blog, I will describe LLMaaS and LLMOps and the building block of the Enterprise AI Platform! 

LLMaaS represents the idea of providing access to powerful LLM services through an internet/intranet. It's similar to other "as-a-service" models (like SaaS, PaaS, etc.), where users can access and utilize LLMs without needing to manage the underlying infrastructure or complex development.

LLMOps focuses on the operational aspects of managing and deploying LLMs in production environments. It's essentially the application of DevOps principles to the lifecycle of LLMs

In the above block diagram, I have depicted comprehensive architecture for an enterprise-grade AI platform, designed to support various stages of development and deployment of AI models, especially focusing on Large Language Models (LLMs). It's broken down into several layers, starting with infrastructure and going up to the application level:

1. Infrastructure: This layer provides the foundation for the platform, encompassing physical resources like:  Compute: Servers for running workloads, including model training, inference, and other tasks. Storage: Storage systems for datasets, model checkpoints, and other essential data.  Network: Networking infrastructure connecting various components and enabling communication.  GPU: Specialized hardware (e.g., GPUs from NVIDIA) for accelerating AI model training and inference.

2. Private Cloud/Virtualization: This layer provides a virtualized environment that leverages virtualization technologies such as VMware, Nutanix, or Oracle's VirtualBox. It enables flexible resource allocation and isolation, making it easier to manage and scale the platform.

3. Private AI Platform: This layer represents the core of the AI platform, encompassing the following components: GPU Virtualization (Ex. Nvidia Enterprise AI or VMware BitFusion): This layer further enhances GPU utilization by providing a virtualized environment for accessing GPUs. It enables resource sharing, efficient utilization, and isolation for different AI workloads. VM (Virtual Machine): Virtual machines can be used for running specific workloads or specific components that require a dedicated environment. Kubernetes: A container orchestration system that manages and scales containerized workloads, enabling the deployment and management of microservices that make up the AI platform. Container: Containers are used to package and run applications and their dependencies in a consistent and isolated manner.

4. MLOps Platforms: These are platforms specifically designed for managing the machine learning lifecycle. They provide tools and services for: Run:ai: A platform for managing and scaling GPU resources, ensuring efficient utilization and performance during model training. Kubeflow: A platform built on Kubernetes that provides tools and infrastructure for managing the machine learning pipeline, including model training, deployment, and monitoring. DOMINO: A platform that provides collaborative tools for data scientists and machine learning engineers to manage and share their work, including data, code, and models.

5. Developer Tools and Catalog: This layer provides a set of tools and resources specifically designed for AI developers: Huggingface: A popular repository and platform for sharing pre-trained models, datasets, and code for natural language processing (NLP), enabling developers to leverage existing resources and accelerate their work. PyTorch: A widely used deep learning framework that provides tools and libraries for building and training AI models. RAY: A library and framework for building distributed applications, enabling scalable training and inference of AI models. Rasa: A framework for building conversational AI chatbots, providing tools for creating and deploying chatbots that interact with users naturally. VAC: A platform for managing and deploying AI models and applications, providing tools for monitoring and managing the lifecycle of models.

6. Experimentation Platform: This layer focuses on providing tools and infrastructure for experimenting with AI models: API Gateway: Provides a secure and managed way for developers to access the platform's services, ensuring controlled access and authentication. Micro Services: The platform is structured as a collection of independent, modular services that communicate through APIs, enabling flexibility, scalability, and isolation of functionalities. Kafka: A messaging system used for real-time data streaming and communication between different components of the platform. Redis: An in-memory data store used for caching frequently accessed data to improve performance. MongoDB, MySQL: Databases for storing metadata, experiment data, and other platform-related information. Monitors: Tools for monitoring the platform's health, performance, and other key metrics, providing insights into the overall system behavior.

7. App and Data Services: This layer represents the applications and services that leverage the AI platform's capabilities for end users. They interact with the applications through various interfaces, benefiting from the platform's capabilities.

The diagram highlights the modular and interconnected nature of a modern AI platform, showcasing how different components work together to support the entire AI development lifecycle. The platform's design emphasizes scalability, flexibility, and developer experience, enabling efficient experimentation, training, and deployment of AI models, especially large language models, for various applications.

Also, please check out my other posts related to this subject

• Private AI Lab
• Power of Multi-Cloud DBaaS 

• MLOps Made Easy with Kubeflow on vSphere: Streamlining Machine Learning Workflows

• Artificial Intelligence, Machine Learning and Deep Learning for IT Operations

• Exploratory data analysis using Python

• Cloud Computing and Data Analytics fundamentals

• IT Operations Management

• MTTIC

• AIOps for Tenant and Platform Operation
• Big Data