The Ingestion plugins are used to ingest user/enterprise data into custom datasets defined by the user/enterprise. These powerful tools enable you to train Ask Sage with your own data, making it more effective for your specific use cases.
Learn More: Navigate to Ingesting Data to learn more about the complete process of ingesting data into Ask Sage.
List of Ingestion Plugins & Agents
Available Plugins
PaidRequires paid subscription
CSV FilesPaid
Filter CSV columns and ingest each entry as a separate training
CSV LinesPaid
Ingest each CSV line as a separate training
Content into DatasetPaid
Train text content into specific datasets
FilePaid
Import files, chunk, summarize, and ingest content
Plain/Text ContentPaid
Import plain text, chunk, summarize, and ingest content
CSV Files
CSV Files Plugin
The CSV Files plugin is used to filter CSV columns and ingest each entry/line in a CSV file as a separate training. This allows you to selectively import specific columns from your CSV data into datasets.
1
Navigate to the Plugin
Navigate to the Ask Sage Prompt Settings section and select Prompt Templates. Then select the CSV Files plugin.
2
Upload CSV File
Click on the Choose File button to upload the CSV file you want to ingest.
3
Provide Description
Provide a short description of the CSV file you are ingesting.
4
Select Dataset
Select the dataset you want to ingest the CSV file into.
5
Submit Initial Form
Click on the Submit button to process the CSV file.
6
Select Columns
Select the columns you want to keep in the CSV file (you can select multiple columns), then click on the Submit button.
Result: After this step, the plugin will ingest and save the content of the selected columns into the dataset.
CSV Lines
CSV Lines Plugin
The CSV Lines plugin is used to ingest each entry/line in a CSV file as a separate training. This plugin processes each row individually, making it ideal for line-by-line data ingestion.
1
Navigate to the Plugin
Navigate to the Ask Sage Prompt Settings section and select Prompt Templates. Then select the CSV Lines plugin.
2
Upload CSV File
Click on the Choose File button to upload the CSV file you want to ingest.
3
Provide Description
Provide a short description of the CSV file you are ingesting.
4
Select Dataset
Select the dataset you want to ingest the CSV file into.
5
Submit
Click on the Submit button to process the CSV file.
6
Execute Prompt
Execute the prefilled prompt generated by the plugin, which will loop through each line in the CSV and execute the prompt against each line.
Next Steps: After ingesting your CSV file, you can proceed to ask questions or generate text from the data you ingested.
Content into Dataset
Content into Dataset Plugin
The Content into Dataset plugin is used to ingest text content into a specific dataset. This is ideal for text content that is not in a file format (e.g., CSV, PDF, etc.).
Use Case Example: Utilize our Summarize Website plugin to summarize website content, then ingest the summarized content into a dataset via the Content into Dataset plugin.
1
Navigate to the Plugin
Navigate to the Ask Sage Prompt Settings section and select Prompt Templates. Then select the Content into Dataset plugin.
2
Enter Text Content
Enter the text content you want to ingest into the dataset. (Recommend 500 tokens per ingestion)
3
Provide Description
Provide a short description of the text content you are ingesting.
4
Select Dataset
Select the dataset you want to ingest the text content into.
5
Submit
Click on the Submit button to process the content.
6
Execute Prompt
Execute the prefilled prompt generated by the plugin, which will ingest the text content into the dataset.
Expected Output: The expected output is similar to the CSV ingestion plugin, where you get a confirmation of the ingestion and can proceed to ask questions or generate text from the data you ingested.
File
File Ingestion Plugin
The File plugin is used to ingest file content, split it into chunks, train the chunks into a dataset, summarize the content, and ingest the summaries. This comprehensive plugin handles the entire file processing workflow.
Supported File Types: File types supported are listed in the plugin description, but you can also find the complete list of supported file types by navigating to the Ingesting Data section.
1
Navigate to the Plugin
Navigate to the Ask Sage Prompt Settings section and select Prompt Templates. Then select the File plugin.
2
Upload File
Click on the Choose File button to upload the file you want to ingest.
3
Select File Reader Strategy
Select the file reader strategy from the dropdown list:
Auto (default)
Automatically selects the most appropriate file reading strategy based on file type and content. Balances speed and accuracy.
Fast
Prioritizes speed over accuracy. Useful for quickly processing large numbers of files.
Hi_res (OCR)
High-resolution processing for OCR. Best for extracting text from images or scanned documents.
Tip: If you are unsure which strategy to choose, leave it as the default "Auto" setting.
4
File Description
Provide a short description of the file content you are ingesting.
5
Tokens Per Chunk
Enter the number of tokens you want to ingest per chunk. (Max 2,000 tokens per chunk for training)
6
Summarization Prompt
Enter the prompt you want to use to summarize the content. (Keep default if unsure)
7
Select Dataset
Select the dataset you want to ingest the file content into.
8
Submit
Click on the Submit button to process the file.
9
Execute and Confirm
Execute the prefilled prompt generated by the plugin, which will ingest the file content and prompt you to confirm the summaries.
Post-Processing Options:
/yes
Ingest the data into the dataset
/skip
Skip and re-run summarization or stop
/stop
Stop without ingesting
Note: If you choose /skip, you can re-run the summarization plugin on the summarized results and then ingest the summaries into the dataset.
Plain/Text Content
Plain/Text Content Plugin
The plain/text content plugin is used to ingest plain/text content, split it into chunks, train the chunks into a dataset, summarize the content, and ingest the summaries.
Key Difference: The main difference between this plugin and the Content into Dataset plugin is that this plugin is able to ingest very large text content by automatically chunking and summarizing it.
1
Navigate to the Plugin
Navigate to the Ask Sage Prompt Settings section and select Prompt Templates. Then select the plain/text content plugin.
2
Enter Text Content
Enter the text content you want to ingest into the dataset.
3
Provide Description
Provide a short description of the text content you are ingesting.
4
Tokens Per Chunk
Enter the number of tokens you want to ingest per chunk. (Max 2,000 tokens per chunk for training)
5
Summarization Prompt
Enter the prompt you want to use to summarize the content. (Keep default if unsure)
6
Select Dataset
Select the dataset you want to ingest the text content into.
7
Submit
Click on the Submit button to process the content.
8
Execute and Confirm
Execute the prefilled prompt generated by the plugin, which will ingest the text content and prompt you to confirm the summaries.
Processing Options: Similar to the File plugin, you will have the option to accept (/yes), skip (/skip), or stop (/stop) the summaries from being ingested into the dataset.
