Ask Sage Tokens
In this section, we will provide a comprehensive overview of tokens and their usage within the Ask Sage platform. Understanding the concept of tokens is crucial, as they serve as the primary means of accessing the platform and function the platforms currency.
Table of contents
What is a Token?
In the context of Generative Artificial Intelligence (GenAI), a token represents a unit of text that the model processes. Tokens can vary in length, ranging from a single character to an entire word, depending on the language and the model’s implementation. For instance, in English, common words like “cat” or “dog” are typically counted as single tokens, while punctuation marks and spaces are also treated as individual tokens. The model utilizes these tokens to generate, understand, and manipulate text.
For users of Ask Sage, tokens are allocated based on their subscription plan. These tokens can be used to access various platform features, such as ingesting data into a vector database (Ask Sage Dataset), or making predictions using the models available on the platform.
What is a token?
Click to see the answer
A token is roughly equivalent to 3.7 English characters.The total token count encompasses both the user’s prompt and the model’s response. For instance, if a user submits a prompt that is 10 tokens long and the model generates a response of 20 tokens, the overall token count for that interaction would be 30 tokens.
It’s important to note that the cost of tokens can vary depending on the model used. Additionally, the costs associated with tokens in the prompt and the response may differ. Therefore, users should be aware of their token usage and consider the financial implications when interacting with the models.
Example Interaction
User Prompt:
Token Count: 7 tokens
Model Response:
Token Count: 8 tokens
Total Token Count:
15 tokens
Note:
- The user prompt consists of 7 tokens.
- The model’s response consists of 8 tokens.
- The total interaction uses 15 tokens.
- Be mindful of the token costs, as they can vary by model.
- The cost of tokens in the prompt and response may differ.
What are Ask Sage Tokens?
Ask Sage tokens are unique in that they are model-agnostic and serve as the central currency for interacting with the Ask Sage platform. These tokens enable users to access and utilize any model on the platform without additional effort or extra costs. Consequently, Ask Sage tokens are consumed based on the type of operation performed, such as ingesting data into a vector database (Ask Sage Dataset) or making predictions using the models available on the platform. It is important to note that each model consumes tokens differently, depending on its complexity and the amount of data processed.
Also, Ask Sage tokens are divided into two categories:
-
Inference Tokens -
Training Tokens
Ask Sage provides users with a seamless experience and flexibility by offering a unified token system that can be used across all models on the platform. This simplifies the process of managing tokens and ensures a consistent user experience across different models as it pertains to token usage.
As a reminder, users can view their token usage and manage their subscription plans through the Tokens menu/settings tab. Here, they can access detailed information about their tokens.
- Subscription: View your current subscription plan.
- Inference Tokens: Check the number of inference tokens used and remaining.
- Training Tokens: Check the number of training tokens used and remaining.
Tokens replenish automatically on the first day of each month. Users can also purchase additional tokens if needed.
If you are associated with an Enterprise plan and not an Admin, reach out to your Admin for more tokens.
Ask Sage Inference Tokens
Ask Sage Inference Tokens are used to make predictions or inferences using the models available on the platform. These tokens are consumed when users submit prompts, use the plugins/agents to generate responses, or interact with the models in any way that requires processing text data. The consumption of inference tokens is based on the complexity of the model and the amount of data processed during the inference process.
Inference tokens are limited by the user’s subscription plan, and users can monitor their token usage in the Ask Sage platform. This feature allows users to track their token consumption and manage their usage effectively.
Users can purchase additional inference tokens if they exceed their allocated limit or require more tokens for their tasks. This flexibility ensures that users can continue to use the platform without interruptions or restrictions.
Inference tokens are not transferrable between months, so it is essential for users to utilize their tokens effectively within the allocated time frame.
Inference tokens cost varies based on the model. As well it’s important to note that the what information is being processed via a prompt has a different cost rate than the response generated by the model.
Ask Sage Training Tokens
Ask Sage Training Tokens play a pivotal role for users interested in enhancing their experience with the Retrieval-Augmented Generation (RAG) feature. These tokens are primarily used for creating and populating datasets. When you upload data to a dataset, training tokens are consumed as the data is processed into embeddings by a specialized model. The resulting vectors are then stored in a vector database, powering the RAG’s advanced query and response capabilities.
The consumption of training tokens is directly proportional to the amount of data being uploaded, such as the file size or the length of the text.
To help users monitor their training token usage, the Ask Sage platform includes a dedicated Token/Subscription section. This transparency allows for meticulous management of token allocation and usage.
It’s important to be aware that training tokens are designed for use within their allocation month and do not roll over to the next month. As such, strategic planning of training token usage is essential. Training tokens are priced at an economical rate of $0.80 per 1 million tokens, making it a cost-effective option for your data training endeavors.
Ask Sage Workbooks are similar to datasets, but note that the data sources ingested into a workbook use training tokens for the initial ingestion process. However, once the data is in the workbook, and users perform actions on the data, it will use inference tokens for any subsequent actions.
Token Refill and Subscription Plans
If you are on an Enterprise plan and are not an Admin, please contact your Admin for additional tokens. The following information is primarily intended for individual users.
Ask Sage provides users with the option to refill their tokens or upgrade their subscription plans according to their needs. Users can purchase extra tokens or move to a higher subscription tier for increased access. This flexibility ensures that users have the necessary resources to work efficiently on the platform.
Moreover, Ask Sage tokens automatically replenish on the first day of each month, giving users a fresh allocation to utilize throughout the month. This feature allows for uninterrupted use of the platform and helps users plan their tasks effectively.
Note: Tokens do not carry over from one month to the next, so it is crucial for users to make the most of their tokens within the designated time frame.