Support using ollama as an inline_completion_provider

[aitorpazos]

Check for existing issues

• Completed

Describe the feature

I am successfully using my local ollama models using assistant panel.

I would love to be able to use them as well as an inline_completion_provider.

Currently, only none, copilot or supermaven values are supported.

If applicable, add mockups / screenshots to help present your vision of the feature

No response

[MatejLach]

Some considerations that come to mind regarding this, keeping in mind the current support for Ollama:

In contrast to something like GitHub Copilot, whose entire purpose is to provide inline completions, some of the most popular models used with Ollama, such as llama3.1 DO NOT support inline completion.
For those that do, such as deepseek-coder-v2, you can make a POST request to /api/generate like:

{
  "model": "deepseek-coder-v2:latest",
  "prompt": "time := time.",   
  "suffix": "    return time;",
  "options": {
    "temperature": 0
  },
  "keep_alive": -1,
  "stream": false
}

And get back a response such as:

{
  "model": "deepseek-coder-v2:latest",
  "created_at": "2024-09-07T14:28:17.013718016Z",
  "response": "Now().Unix()\n",   // our inline completion, inserted between prompt and suffix
  "done": true,
  "done_reason": "stop",
  // metadata fields omitted 
}

For models that do not support inline completions the above request results in i.e.:

{
  "error": "llama3.1:latest does not support insert"
}

However it would be desirable to use say llama3.1 for chat while using deepseek-coder-v2 for inline completion at the same time, therefore the list of Olllama inline completion models should be separate from chat models.

Another consideration is how much sense it would make to support remote Ollama instances for inline completions. , I've got Ollama running both locally on my laptop and on a server I've got on my LAN to get access to more powerful models.
Anecdotally, the difference between local response times (~100ms, vs the server ~300ms), mean that using the LAN server for inline completions is probably impractical vs using it for chat is just fine.
It also means that it would be nice if the chat Ollama provider vs the inline completion Ollama provider did not share the same base URL.

[skewty]

This repo was mentioned in #14134.
I am including it here so it doesn't get forgotten and for easier reference.

---

Proxy that allows you to use ollama as a copilot like Github copilot

https:/bernardo-bruning/ollama-copilot

Written in Go

[xmaayy]

@skewty Seems like that wont work?
#6701

[tlvenn]

As mentioned in #16030 to address your concern @MatejLach, we should be able to configure an autocomplete model along side a chat model.

Copy pasting from the issue:

Heres an example of the config.json in Continue.dev to change autocomplete model:

  "tabAutocompleteModel": {
    "title": "Tab Autocomplete Model",
    "provider": "ollama",
    "apiBase": "http://localhost:11434",
    "model": "deepseek-coder:6.7b",
    "contextLength": 8192
  },
  "tabAutocompleteOptions": {
    "prefixPercentage": 0.5,
    "maxPromptTokens": 4096
  },