{"id":5064,"date":"2024-04-29T13:48:09","date_gmt":"2024-04-29T13:48:09","guid":{"rendered":"https:\/\/www.patentclaimmaster.com\/blog\/?p=5064"},"modified":"2024-10-07T17:26:08","modified_gmt":"2024-10-07T17:26:08","slug":"patent-drafting-local-generative-ai","status":"publish","type":"post","link":"https:\/\/www.patentclaimmaster.com\/blog\/patent-drafting-local-generative-ai\/","title":{"rendered":"Patent Drafting Using Local Generative AI"},"content":{"rendered":"
\n
\"\"<\/a><\/figure>\n<\/div>\n\n\n

You can now use ClaimMaster’s +Drafting <\/em>patent drafting tools<\/a> with a completely local (i.e., offline) generative AI. Specifically, we now integrate with Ollama<\/a>, which is a free, lightweight desktop application that lets you run and interact with locally executing open-source Large Language Models (“LLMs”) in complete privacy without any cloud access. Some of the local LLMs, such as Meta’s latest Llama3<\/a>, are said to rival OpenAI GPT in accuracy and performance. Critically, when you are using ClaimMaster with Ollama for patent drafting, all document processing happens directly on your computer, so that you and your clients don’t have to worry about how your confidential IP data is transmitted and stored in the cloud. <\/strong><\/p>\n\n\n\n

Ollama has a number of key advantages for patent professionals:<\/p>\n\n\n\n

    \n
  1. It’s private<\/strong> – Ollama and its LLMs run and process all data only on your computer without any Internet access<\/em>. You can use Ollama framework to download and locally run the latest open-source LLM models, such as Meta’s Llama3<\/a>. Because it runs locally, you have complete control of your IP data.<\/li>\n\n\n\n
  2. It runs reasonably fast (with enough RAM) and is easy to use<\/strong> – you don’t need a supercomputer to run Ollama, but it helps to have plenty of RAM and a GPU. For the best results it’s recommended to have a modern PC with Windows 10\/11, a 4 or 8-core processor, an SSD with 12-50 GB of free space for models, and plenty of RAM (8GB is the bare minimum but 16+ GB is really recommended). Considering how affordable additional RAM is these days (e.g., $65 for 32GB of desktop RAM on Amazon<\/a>), it shouldn’t be much of an issue to add more RAM to your computer. If you have an Nvidia or an AMD graphics card, Ollama can further improve performance via hardware acceleration. That said, you can run Ollama on a system with lower characteristics but you’ll likely notice fairly long response times to your prompts (e.g. 1 min or more).<\/li>\n\n\n\n
  3. It’s free <\/strong>– You can use it as often and with as much text as you’d like without worrying about additional token costs, as you would when accessing OpenAI and Azure OpenAI GPT via API.<\/li>\n\n\n\n
  4. It can be used for business <\/strong>– Ollama and most of its models are licensed under permissive MIT or Apache licenses, making it well-suited for commercial use by patent professionals.<\/li>\n<\/ol>\n\n\n\n

    So if you have data privacy concerns about cloud-based OpenAI\/Azure GPT applications, using Ollama in combination with ClaimMaster seems like a no-brainer for a completely local AI solution for patent drafting. It’s free, secure, and lets you access a number of powerful open-source LLMs to help with patent drafting. All you really need is a reasonably modern PC to run Ollama smoothly. <\/p>\n\n\n\n

    \n\n\n\n

    Below are the steps needed to install and run Ollama with ClaimMaster:<\/p>\n\n\n\n

    <\/p>

    1. Download and install Ollama for Windows<\/strong>

      You can download Ollama <\/a>for Windows 10 or later from this link<\/a>. Just run the installer once you download it. \"ollamaAfter you install Ollama, it will automatically run in the background, so that applications like ClaimMaster can connect to it without further setup. You will see a neat little Ollama icon in your System Tray:\"windows<\/p> <\/li>

    2. Install one or more LLM models inside Ollama<\/strong>

      You will also need to download one or more LLMs to use with Ollama. Each LLM will normally require anywhere from 4GB to 20GB per model, depending on the complexity of the LLM. To install a particular model, open Windows Command Prompt<\/strong> or PowerShell<\/strong>, then type “ollama run <model-name><\/strong>, such as “\u201collama run llama3<\/strong>“. A list of available models is available here<\/a>. If you haven’t used a particular model yet, it might take some time for it to download and install, depending on the size of the model and how fast your Internet connection is. However, the next time you run Ollama with that model, it will run it from local storage, so it will be quite fast to start up.\"ollamaWe recommend using llama3<\/a> <\/strong>or mistral <\/strong><\/a>models for text processing, rewriting, etc. Llama3 is the latest LLM from Meta, released in April 2024. Mistral LLM is also quite good. As for vision\/image recognition, there is a multi-modal LLM called LLaVa<\/strong><\/a>. While its authors compare it to GPT Vision, in our testing it did not perform nearly as well when used for patent drafting. Specifically, even the largest 20GB version of LLM (llava:34b-v1.6) had a hard time with recognizing element numbers in figures correctly, although it could generally identify the figure subject matter. So at this time we can’t yet recommend LLaVa specifically for patent drafting, although we hope that its next versions will perform better for this task.To get help on available Ollama commands, type “ollama help”<\/strong> in command prompt. Although Ollama is easy to set up, there is a ton of options\/configurability.<\/p> <\/li>

    3. Configure ClaimMaster to connect to the local Ollama server<\/strong>

      The latest version of ClaimMaster (2024.1.0) with +Drafting <\/strong>add-on integrates directly with Ollama, so there isn’t much more to do other than installing Ollama and then selecting a particular model to run. Download the latest ClaimMaster installer from Customer Portal<\/a>. Once you install version 2024.1.0 or later, go to ClaimMaster->Patent Drafting->GPT\/LLM Settings<\/strong>:\"gpt_settings_selection\"In the LLM settings window, specify the following settings and then save them:“Source<\/strong>” should be set to Local<\/strong>“Endpoint<\/strong>” should be set to Ollama’s default server address, which is “http:\/\/localhost:11434\/api\/generate<\/strong>” (use the pre-defined drop-down option)“Model<\/strong>” should be set to “llama3″, “mistral”, “llava<\/strong>” or any other model that you’ve downloaded and installed via Ollama and want to use for patent drafting. \"ollama<\/p> <\/li>

    4. Send your prompts to the configured LLM source<\/strong>

      That’s it – once your LLM settings are configured to point to the local Ollama server, you can power up Ollama and start using ClaimMaster to send various patent drafting prompts to the locally running LLM directly from Word, as explained in other tutorials. \"claimmasterYou can create and re-use as many LLM prompts as you’d like within ClaimMaster. For example, some of the helpful things you can ask your local LLM to do include:* Re-write a text portion to fix errors and make it more eloquent* Add technical details to existing paragraph based on the discussed subject matter* Explain or define a technical term or concept* Write a paragraph or two explaining the state of problems with the current technology* Draft a section of the application based on the passed-in subject matter, such as title, background, or even claims.Notably, because we now offer the ability to integrate with different LLMs, you may notice that some prompts work better with specific LLMs than others. To help you quickly identify prompts that are specific to a particular LLM model, you can list the model name at the end of the prompt name by enclosing it in square brackets. For example, if you have configured two prompt templates Write figure description [llama]<\/strong> and Write figure description [gpt]<\/strong>, each template will only be available for selection in the Generate Text with GPT\/LLM<\/a> tool when its respective model is selected in Settings (either Llama or GPT).<\/p> <\/li><\/ol><\/div>\n\n\n\n

      Ollama is a great tool. Now that ClaimMaster can integrate with local LLMs, we plan to soon add many new features, such as improved LLM prompts for specific models, ability to train LLMs on your local documents and also using LLMs for further improving accuracy of various patent proofreading, automation, and analysis tools in ClaimMaster.<\/p>\n\n\n\n

      For now, here’s more information for using LLMs with ClaimMaster for patent drafting various sections of the patent specification:<\/p>\n\n\n\n