Using private GPT models and local LLMs with ClaimMaster

If you have your own account with OpenAI, Azure OpenAI or have installed a local LLM, ClaimMaster lets you pass your API account information for completing GPT/LLM prompts using your private GPT models (you’ll incur your own costs for the generated text). You may find this option beneficial as you’ll be able to use your custom end-points (Azure) or more advanced GPT models (ClaimMaster currently uses gpt-3.5-turbo by default), as well as set the max limit on the tokens to generate more text than allowed ClaimMaster’s default settings.

Alternatively, when running a local LLM, your data remains completely private and is not sent to the cloud for processing. Note that doing so requires you to install a program, like LM Studio or GPT4All, that can execute quantized/compressed LLMs on your computer. While running LLMs locally is slower and generates less finessed text than OpenAI GPT, for many cases it can be “good enough,” especially when used for re-writing existing text. Note that at the time of this blog post, we could not verify the exact licensing details for commercial use with LM Studio authors, so you may need to contact them directly if installing in work environment.

  1. Set up the desired GPT/LLM account

    a. OpenAI
    Create an OpenAI account or sign in. Then navigate to the API key page and “Create new secret key“, optionally naming the key. Make sure to save this somewhere safe and do not share it with anyone

    b. Azure
    Get an Azure subscription, if you don’t have it (it’s free. Create and deploy Azure GPT service.

    c. Local LLM
    Install a program allowing you to run local LLMs in server mode, such as LM Studio or GPT4All. Also download and install one of the LLMs, such as Mistral, using that program.

  2. Configure ClaimMaster to connect to the selected GPT/LLM source

    Go to ClaimMaster->Preferences, Extra Tools, Help menu, then Preferences menu. Click on Patent Drafting->GPT/LLM Settings.

    gpt_settings_selection

    Alternatively, you can open GPT/LLM settings directly from the GPT tools window.
    llm_settings

    Then, depending on the source, configure ClaimMaster GPT settings as follows to use private GPT models.

    a. OpenAI
    Select OpenAI GPT as the source. Copy & paste your private API key into the API key section. You should also specify the Model to use and Maximum # of tokens limit for prompt and return responses. You can leave the Endpoint or local address field blank.

    openai gpt settings
    Then press Save to save the settings.

    b. Azure
    Select Microsoft Azure GPT as the source. Specify your azure endpoint in the Endpoint or local address field. The endpoint should specify the full address, such as https://{YOUR_RESOURCE_NAME}.openai.azure.com/openai/deployments/{YOUR_DEPLOYMENT_NAME}/chat/completions?api-version=2024-02-15-preview. In addition, copy & paste your private API key into the API key section. You should also specify the Model to use and Maximum # of tokens limit for prompt and return responses.

    Azure GPT settings
    Then press Save to save the settings.

    c. Local LLM
    To use ClaimMaster with a locally running LLM, make sure to start the local LLM via LM Studio or a similar program in server mode. From the program, obtain the server port/address to use for the local LLM, as shown below.

    LM Studio settings

    In ClaimMaster GPT settings, select Local LLM as the source. Copy the full local address of the local LLM server from your LLM program into the Endpoint or local address field. The address would need to include the port where the LLM server is listening, as specified in the program. Please consult your program’s documentation for specific address instructions. The address would look something like http://localhost:4891/v1 or http://localhost:1235/v1/chat/completion.
    local LLM settings
    You can ignore other fields in the GPT settings window, since they are typically not used by the LLM program. Press Save to save the settings.

  3. Send your prompts to the configured GPT/LLM source

    Once your GPT/LLM settings are configured, you can start using ClaimMaster to send your prompts to the private GPT models or local LLM, as explained in other tutorials.

For more information about this feature, check out the Online Manual.