feat(provider): detect venv python via "pynvim-python" tool #35273

Problem:
Detection of the pynvim module is currently done by finding the first
Python interpreter in the `PATH` and checking if it can import pynvim.
This has several problems:
- Activation of an unrelated Python virtual environment will break
  automatic detection, unless pynvim is also installed in that
  environment.
- Installing pynvim to the expected location is difficult. User
  installation into the system-wide or user-wide Python site area is now
  deprecated.  On Ubuntu 24.04 with Python 3.12, for example, the
  command `pip install --user pynvim` now fails with the error message
  `error: externally-managed-environment`.
- Users may create a dedicated virtual environment in which to install
  pynvim, but Nvim won't detect it; instead, they must either activate
  it before launching Nvim (which interferes with the user of other
  virtual environments) or else hard-code the variable
  `g:python3_host_prog` in their `init.vim` to the path of the correct
  Python interpreter.  Neither option is desirable.

Solution:
Expose pynvim's Python interpreter on the `PATH` under the
name `pynvim-python`.  Typical user-flow:

1. User installs either uv or pipx.
2. User installs pynvim via:
   ```
   uv tool install --upgrade pynvim
   # Or:
   pipx install --upgrade pynvim
   ```

With corresponding changes in pynvim https://github.com/neovim/pynvim/issues/593
the above user-flow is all that's needed for Nvim to detect the
installed location of pynvim, even if an unrelated Python virtual
environments is activated.  It uses standard Python tooling to automate
the necessary creation of a Python virtual environment for pyenv and the
publication of `pynvim-python` to a directory on `PATH`.

(cherry picked from commit 5f8d4a248a97b7beb26a097811dce92f1b18e260)

1 files changed, 4 insertions, 0 deletions

diff --git a/runtime/lua/vim/provider/python.lua b/runtime/lua/vim/provider/python.lua
index a772b36973..afd0895c7f 100644
--- a/runtime/lua/vim/provider/python.lua
+++ b/runtime/lua/vim/provider/python.lua
@@ -83,6 +83,10 @@ function M.detect_by_module(module)
     return vim.fn.exepath(vim.fn.expand(python_exe, true)), nil
   end
 
+  if vim.fn.executable('pynvim-python') == 1 then
+    return 'pynvim-python'
+  end
+
   local errors = {}
   for _, exe in ipairs(python_candidates) do
     local error = check_for_module(exe, module)