From 74176c92a6ed29ef7a07fd68e9cdf5f1e5e37b1b Mon Sep 17 00:00:00 2001 From: John Alexander <174467815+ms-johnalex@users.noreply.github.com> Date: Fri, 5 Dec 2025 14:27:20 -0600 Subject: [PATCH] Update for FL and UI changes --- .../intelligentapps/images/overview/get-started.png | 4 ++-- .../images/overview/initial-view.png | 4 ++-- docs/intelligentapps/models.md | 13 +++++++------ docs/intelligentapps/overview.md | 9 +++++++-- 4 files changed, 18 insertions(+), 12 deletions(-) diff --git a/docs/intelligentapps/images/overview/get-started.png b/docs/intelligentapps/images/overview/get-started.png index c110519ec6..de4a2232fb 100644 --- a/docs/intelligentapps/images/overview/get-started.png +++ b/docs/intelligentapps/images/overview/get-started.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:06e0920a37018fe89c35b9f1944a01a12dcfb46d7b45730b480bde071d5bed61 -size 158990 +oid sha256:dd06f89a66c1986da6cd94c241e17c52954c5a274e6636b7ebbe62d7ae2c2764 +size 196082 diff --git a/docs/intelligentapps/images/overview/initial-view.png b/docs/intelligentapps/images/overview/initial-view.png index f700f4ebfa..1db97b2ec0 100644 --- a/docs/intelligentapps/images/overview/initial-view.png +++ b/docs/intelligentapps/images/overview/initial-view.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:0ff7e1c21fc0b6b855c575b74cc37e2129ad59429cc434d54affa59c1a1a7d0f -size 66069 +oid sha256:aadf5f81ee3cbf8f492e9fb5afc50c9917419ac6485f445373e0654873e1d852 +size 86134 diff --git a/docs/intelligentapps/models.md b/docs/intelligentapps/models.md index aae5905b88..5f3b7d4afe 100644 --- a/docs/intelligentapps/models.md +++ b/docs/intelligentapps/models.md @@ -1,6 +1,6 @@ --- ContentId: 52ad40fe-f352-4e16-a075-7a9606c5df3b -DateApproved: 10/03/2025 +DateApproved: 12/05/2025 MetaDescription: Find a popular generative AI model by publisher and source. Bring your own model that is hosted with a URL, or select an Ollama model. --- # Explore models in AI Toolkit @@ -12,7 +12,7 @@ Within the model catalog, you can explore and utilize models from multiple hosti - Models hosted on GitHub, such as Llama3, Phi-3, and Mistral, including pay-as-you-go options. - Models provided directly by publishers, including OpenAI's ChatGPT, Anthropic's Claude, and Google's Gemini. - Models hosted on Microsoft Foundry. -- Models downloaded locally from repositories like Ollama and ONNX. +- Models downloaded locally from repositories like Foundry Local, Ollama and ONNX. - Custom self-hosted or externally deployed models accessible via Bring-Your-Own-Model (BYOM) integration. Deploy models directly to Foundry from within the model catalog, streamlining your workflow. @@ -30,7 +30,7 @@ To find a model in the model catalog: 1. Select **MODELS** > **Catalog** to open the model catalog 1. Use the filters to reduce the list of available models - - **Hosted by**: AI Toolkit supports GitHub, ONNX, OpenAI, Anthropic, Google as model hosting sources. + - **Hosted by**: AI Toolkit supports Microsoft Foundry, Foundry Local, GitHub, ONNX, OpenAI, Ollama, Anthropic, Google, NVIDIA NIM, and Windows AI API as model hosting sources. - **Publisher**: The publisher for AI models, such as Microsoft, Meta, Google, OpenAI, Anthropic, Mistral AI, and more. - **Feature**: Supported features of the model, such as `Text Attachment`, `Image Attachment`, `Web Search`, `Structured Outputs`, and more. - **Model type**: Filter models that can run remotely or locally on CPU, GPU, or NPU. This filter depends on the local availability. @@ -48,6 +48,7 @@ To add a model from the model catalog: 1. Select the **Add** on the model card 1. The flow for adding models will be slightly different based on the providers: + - **Foundry Local**: Foundry Local downloads and runs the model, which may take a few minutes depending on your internet speed. The model is available on a localhost page and added to AI Toolkit. Learn more in [What is Foundry Local?](https://learn.microsoft.com/azure/ai-foundry/foundry-local/what-is-foundry-local?view=foundry-classic&preserve-view=true). - **GitHub**: AI Toolkit asks for your GitHub credentials to access the model repository. Once authenticated, the model is added directly into AI Toolkit. > [!NOTE] > AI Toolkit now [supports GitHub pay-as-you-go models](/docs/intelligentapps/playground.md#_github-payasyougo-model-support), so you can keep working after passing free tier limits. @@ -70,7 +71,7 @@ You can also add your own models that are hosted externally or run locally. Ther - Add custom ONNX models, such as those from Hugging Face, using AI Toolkit's [model conversion tool](/docs/intelligentapps/modelconversion.md). There are several entrypoints to add models to AI Toolkit: -- From **MY MODELS** in the tree view, hover over it and select the `+` icon. +- From **MY RESOURCES** section in the tree view, hover over **Models** and select the `+` icon. ![Screenshot of the AI Toolkit interface showing the Model Catalog toolbar with the + Add model button highlighted, indicating where users can click to add a new custom model.](./images/models/custom-1.png) - From the **Model Catalog**, select the **+ Add model** button from the tool bar. @@ -176,13 +177,13 @@ Some models require a publisher or hosting-service license and account to sign-i In this article, you learned how to: - Explore and manage generative AI models in AI Toolkit. -- Find models from various sources, including GitHub, ONNX, OpenAI, Anthropic, Google, Ollama, and custom endpoints. +- Find models from various sources, including Microsoft Foundry, Foundry Local, GitHub, ONNX, OpenAI, Anthropic, Google, Ollama, and custom endpoints. - Add models to your toolkit and deploy them to Microsoft Foundry. - Add custom models, including Ollama and OpenAI compatible models, and test them in the playground or agent builder. - Use the model catalog to view available models and select the best fit for your AI application needs. - Use filters and search to find models quickly. - Browse models by category, such as Popular, GitHub, ONNX, and Ollama. - Convert and add custom ONNX models using the model conversion tool. -- Manage models in MY MODELS, including editing, deleting, refreshing, and viewing details. +- Manage models in MY RESOURCES/Models, including editing, deleting, refreshing, and viewing details. - Start and stop the ONNX server and copy endpoints for local models. - Handle license and sign-in requirements for some models before testing them. diff --git a/docs/intelligentapps/overview.md b/docs/intelligentapps/overview.md index 8348ffb045..758ea5c881 100644 --- a/docs/intelligentapps/overview.md +++ b/docs/intelligentapps/overview.md @@ -1,6 +1,6 @@ --- ContentId: 164299e8-d27d-40b9-8b8d-a6e05df8ac69 -DateApproved: 10/03/2025 +DateApproved: 12/05/2025 MetaDescription: Build, test, and deploy AI applications with AI Toolkit for Visual Studio Code. Features model playground, prompt engineering, batch evaluation, fine-tuning, and multi-modal support for LLMs and SLMs. --- # AI Toolkit for Visual Studio Code @@ -13,7 +13,7 @@ AI Toolkit offers seamless integration with popular AI models from providers lik | Feature | Description | Screenshot | |---------|-------------|------------| -| [Model Catalog](/docs/intelligentapps/models.md) | Discover and access AI models from multiple sources including GitHub, ONNX, Ollama, OpenAI, Anthropic, and Google. Compare models side-by-side and find the perfect fit for your use case. | ![Screenshot showing the AI Toolkit Model Catalog interface with various AI model options](./images/overview/catalog.png) | +| [Model Catalog](/docs/intelligentapps/models.md) | Discover and access AI models from multiple sources including Microsoft Foundry, Foundry Local, GitHub, ONNX, Ollama, OpenAI, Anthropic, and Google. Compare models side-by-side and find the perfect fit for your use case. | ![Screenshot showing the AI Toolkit Model Catalog interface with various AI model options](./images/overview/catalog.png) | | [Playground](/docs/intelligentapps/playground.md) | Interactive chat environment for real-time model testing. Experiment with different prompts, parameters, and multi-modal inputs including images and attachments. | ![Screenshot showing the AI Toolkit Playground interface with chat messaging and model parameter controls](./images/overview/playground.png) | | [Agent Builder](/docs/intelligentapps/agentbuilder) | Streamlined prompt engineering and agent development workflow. Create sophisticated prompts, integrate MCP tools, and generate production-ready code with structured outputs. | ![Screenshot showing the Agent Builder interface for creating and managing AI agents](./images/overview/agent-builder.png) | | [Bulk Run](/docs/intelligentapps/bulkrun) | Execute batch prompt testing across multiple models simultaneously. Ideal for comparing model performance and testing at scale with various input scenarios. | ![Screenshot showing the Bulk Run interface for batch testing prompts across multiple AI models](./images/overview/bulk-run.png) | @@ -95,6 +95,11 @@ AI Toolkit opens in its own view, with the AI Toolkit icon now displayed on the - **Evaluation**: Evaluate models, prompts, and agents by comparing their outputs to ground truth data and computing evaluation metrics. - **Tracing**: Trace capabilities to help you monitor and analyze the performance of your AI applications. +- **Build Agent with GitHub Copilot**: This section enables you to use GitHub Copilot to help you build AI agents faster with AI Toolkit. It contains the following subsections: + - **Create Agent**: Opens the Chat pane and creates a prompt to build an AI agent with a Console application using GitHub Copilot. + - **Enable Tracing**: Opens the Chat pane and creates a prompt to add tracing to the current workspace using GitHub Copilot. + - **Add Evaluation Framework**: Opens the Chat pane and creates a prompt to add the evaluation framework to the current workspace using GitHub Copilot. + - **MCP Workflow**: This section contains tools you use to add an existing MCP server or to create a new one. It contains the following subsections: - **Add MCP Server**: The link for adding and working with an existing MCP server. - **Create new MCP Server**: The link for creating and deploying new MCP servers in AI Toolkit.