page_type	languages	products	urlFragment	name	description
sample	azdeveloper java bicep typescript html	azure azure-openai active-directory azure-cognitive-search azure-app-service azure-container-apps azure-kubernetes-service azure-sdks github document-intelligence azure-monitor azure-pipelines	azure-search-openai-demo-java	ChatGPT + Enterprise data (Java) on App Service, Azure Container Apps, and Azure Kubernetes Service	A Java sample app that chats with your data using OpenAI and AI Search.

ChatGPT + Enterprise data with Azure OpenAI and Azure AI Search - Java Version

This repo is the Java version of the well known ChatGPT + Enterprise data code sample originally written in python.

It demonstrates best practices for creating ChatGPT-like experiences over your own data using the Retrieval Augmented Generation pattern. It uses Azure OpenAI Service to access the ChatGPT model gpt-35-turbo, and Azure Azure AI Search for data indexing and retrieval.

This repository includes sample data so it's ready to try end to end. In this sample application we use a fictitious company called Contoso Electronics, and the experience allows its employees to ask questions about the benefits, internal policies, as well as job descriptions and roles.

Features

What this demo application does:

• Chat and Q&A interfaces
• Explores various options to help users evaluate the trustworthiness of responses with citations, tracking of source content, etc.
• Shows possible approaches for data preparation, prompt construction, and orchestration of interaction between model (ChatGPT) and retriever (Azure AI Search)
• Shows possible AI orchestration implementation using the plain Java Open AI sdk or the Java Semantic Kernel sdk
• Settings directly in the UX to tweak the behavior and experiment with options

Solution architecture and deployment options

This sample supports different architectural styles. It can be deployed as standalone app on top of Azure App Service or as a microservice event driven architecture with web frontend, AI orchestration and document ingestion apps hosted by Azure Container Apps or Azure Kubernetes Service.

• For Azure App Service deployment, see here.
• For Azure Container Apps deployment, see here.
• For Azure Kubernetes Service deployment, see here.

RAG implementation options

This repo is focused to showcase different options to implement "chat with your private documents" scenario using RAG patterns with Java, Azure OpenAI and Semantic Kernel. Below you can find the list of available implementations.

Conversational Style	RAG Approach	Description	Java Open AI SDK	Java Semantic Kernel
One Shot Ask	PlainJavaAskApproach	Use Azure AI Search and Java OpenAI APIs. It first retrieves top documents from search and use them to build a prompt. Then, it uses OpenAI to generate an answer for the user question.Several search retrieval options are available: Text, Vector, Hybrid. When Hybrid and Vector are selected an additional call to OpenAI is required to generate embeddings vector for the question.	✅	❌
Chat	PlainJavaChatApproach	Use Azure AI Search and Java OpenAI APIs. It first calls OpenAI to generate a search keyword for the chat history and then answer to the last chat question. Several search retrieval options are available: Text, Vector, Hybrid. When Hybrid and Vector are selected an additional call to OpenAI is required to generate embeddings vector for the chat extracted keywords.	✅	❌
One Shot Ask	JavaSemanticKernelWithMemoryApproach	Use Java Semantic Kernel framework with built-in MemoryStore for embeddings similarity search. A semantic function RAG.AnswerQuestion is defined to build the prompt using Memory Store vector search results.A customized version of SK built-in CognitiveSearchMemoryStore is used to map index fields populated by the documents ingestion process.	❌	This approach is currently disabled within the UI, memory feature will be available in the next java Semantic Kernel GA release
One Shot Ask	JavaSemanticKernelChainsApproach	Use Java Semantic Kernel framework with semantic and native functions chaining. It uses an imperative style for AI orchestration through semantic kernel functions chaining. InformationFinder.SearchFromQuestion native function and RAG.AnswerQuestion semantic function are called sequentially. Several search retrieval options are available: Text, Vector, Hybrid.	❌	✅
Chat	JavaSemanticKernelWithMemoryApproach	Use Java Semantic Kernel framework with built-in MemoryStore for embeddings similarity search. A semantic function RAG.AnswerConversation is defined to build the prompt using Memory Store vector search results. A customized version of SK built-in CognitiveSearchMemoryStore is used to map index fields populated by the documents ingestion process.	❌	❌ This approach is currently disabled within the UI, memory feature will be available in the next java Semantic Kernel GA release
Chat	JavaSemanticKernelChainsApproach	Use Java Semantic Kernel framework with semantic and native functions chaining. It uses an imperative style for AI orchestration through semantic kernel functions chaining. InformationFinder.SearchFromConversation native function and RAG.AnswerConversation semantic function are called sequentially. Several search retrieval options are available: Text, Vector, Hybrid.	❌	✅

Getting Started

All the available architectures and implementations come with few options to start developing and running the application locally using dev containers or in the cloud using github codespaces. For more detailed instructions, please refer to the specific architecture implementation documentation:

• App Service Getting Started
• Azure Container Apps Getting Started
• Azure Kubernetes Service Getting Started

Guidance

For platform specific guidance, please refer to the following documentation:

• App Service Guidance

• Azure Container Apps Getting Started

• Azure Kubernetes Service Getting Started

• [App Service Guidance] This sample is designed to get you started quickly and let you experiment with Java intelligent Apps RAG architectures on Azure. For production deployment, you can use the Azure Application Landing Zones (LZA) to deploy the solution maintaining best practices for security, monitoring, networking and operational excellence.

Check the chat-with-your-data-lza-app-accelerator to see how you can deploy this solution on App Service LZA or Azure Container Apps LZA.

Resources

• 📖 Revolutionize your Enterprise Data with ChatGPT: Next-gen Apps w/ Azure OpenAI and AI Search
• 📖 Azure OpenAI Service
• 📖 RAG in Azure AI Search
• 📖 Azure OpenAI client library for Java
• 📖 Semantic Kernel for Java 1.0
• 📖 Evaluating a RAG chat App
• 📖 Well Architected Framework on Azure Open AI Service
• 📺 Chat with your data using Java and Azure Open AI
• 📺 Vector Search, RAG, And Azure AI Search