: Ensure the machine running the Ollama service has adequate GPU VRAM. If Java and Ollama reside on the same server, tightly limit Java's maximum memory ( -Xmx ) to leave breathing room for Ollama's model weights in the system RAM/VRAM. Conclusion
Add the Spring AI Ollama starter to your project configuration: ollamac java work
// 1. Initialize the model pointing to your local Ollama instance OllamaChatModel = OllamaChatModel.builder() .baseUrl( "http://localhost:11434" ) .modelName( "llama3.2:1b" ) .build(); // 2. Generate a response .generate( "Explain how Java works with Ollama." ); System.out.println( "AI Response: " ); } } Use code with caution. Copied to clipboard 4. Advanced Feature: RAG (Talk to Documents) To build a "complete" professional feature, implement Retrieval-Augmented Generation (RAG) to let the AI answer questions based on your local files: Document Loading : Split local text or PDF files into chunks. Embeddings : Use Ollama’s /api/embeddings endpoint to convert text into vectors. Vector Store : Store these vectors in a local database like or an in-memory store for retrieval during chat. 5. Alternative: Spring Boot Integration If you are building a web application, use to expose the feature as a REST API: Getting Started with Ollama, Llama 3.1 and Spring AI 30 Jul 2024 — : Ensure the machine running the Ollama service
Ollama supports both text generation and text embedding models (e.g., nomic-embed-text ). Here is how a standard RAG pipeline works within a Java application: Initialize the model pointing to your local Ollama
What are you targeting (e.g., automated code review, offline chatbots, data extraction)? Which Java framework does your current project use?