CogCache provides the highest performance and lowest cost on the market for Azure OpenAI tokens, enabling you to unlock the full potential of generative AI:

Up to 50% reduction in costs and carbon footprint
Up to 100x faster response times, ensuring smooth and efficient operations
Gain real-time insights, track performance key metrics and view all the logged requests for easy debugging
View, control, refine and edit the output of generative AI applications via our Cognitive Caching technology
Access the most advanced LLMs with no capacity limits

How it works

CogCache leverages a cognitive caching mechanism to enhance the performance of AI-generated content. CogCache stores all LLM inputs and outputs in a cache and when similar or identical requests are made in the future, it can quickly retrieve the stored response instead of generating a new one each time, reducing the load on the AI model and speeding up the response time significantly—from the typical few seconds required to generate responses to milliseconds.

Two flavours of CogCache

1. CogCache with Caching only

You can purchase a caching only plan and you get all the benefits of CogCache, but without the LLMs. CogCache will require to bring your own Azure OpenAI deployment.

2. CogCache with Caching and Azure OpenAI LLMs

This is the recommended way to purchase CogCache. It provides the additional cost reductions of our discounted Azure OpenAI PTU-level deployments - with deeper discounts as your monthly generated token usage grows. No upfront commitment, only pay for what you use.

NOTE: Both flavours benefit the Cognitive Caching technology which allows you to view and manage the responses of LLMs and the LLM observability features which makes it easy to debug LLM based applications.

Getting started with CogCache is easy

Simply switch your code endpoints to your CogCache with the supplied key and that’s it. Your implementation doesn’t change but is faster, safer and more cost-effective.

Billing information - Caching with Azure OpenAI LLMs

IMPORTANT: The discounts are applied against the Azure OpenAI prices.

Based on the monthly usage you will be getting the following prices /1,000 tokens (input/output):

For GPT-4-Turbo:

Usage: $0 - $5,000, Price: $0.0095 / $0.0285, Discount: 5%
Usage: $5,000 - $10,000, Price: $0.00925 / $0.02775, Discount: 7.5%
Usage: $10,000 - $25,000, Price: $0.009 / $0.027, Discount: 10%
Usage: $25,000 - $50,000, Price: $0.00875 / $0.02625, Discount: 12.5%
Usage: $50,000 - unlimited, Price: $0.0085 / $0.0255, Discount: 15%

For GPT-3.5-Turbo:

Usage: $0 - $5,000, Price: $0.000475 / $0.001425, Discount: 5%
Usage: $5,000 - $10,000, Price: $0.0004625 / $0.0013875, Discount: 7.5%
Usage: $10,000 - $25,000, Price: $0.00045 / $0.00135, Discount: 10%
Usage: $25,000 - $50,000, Price: $0.000425 / $0.001275, Discount: 15%
Usage: $50,000 - unlimited, Price: $0.0004 / $0.0012, Discount: 20%

Billing information - Caching Only

Each plan comes with bundled cache entries. For additional cache entries the pricing is per unit and pay as you go model.

For more information, connect with us:

w: www.touchcast.com

e: sales@touchcast.com

Touchcast CogCacheTouchcast Inc.

Touchcast CogCache

Touchcast Inc.

Touchcast CogCache

Touchcast Inc.

Touchcast CogCache
Touchcast Inc.