Full driver path needed in molab

[esadek]

When using dbc in molab (a cloud-hosted marimo notebook workspace), dbc installs drivers to /tmp/uv-venv/etc/adbc/drivers, while adbc_driver_manager looks for drivers in /etc/adbc/drivers.

Here is a code example:

import subprocess

subprocess.run(["dbc", "install", "duckdb"])

Installed duckdb 1.4.0 to /tmp/uv-venv/etc/adbc/drivers

from adbc_driver_manager import dbapi

con = dbapi.connect(driver="duckdb")

Traceback (most recent call last):
  File "/tmp/marimo_73/__marimo__cell_Hbol_.py", line 3, in <module>
    con = dbapi.connect(driver="duckdb")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/uv-venv/lib/python3.12/site-packages/adbc_driver_manager/dbapi.py", line 251, in connect
    db = _lib.AdbcDatabase(**db_kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "adbc_driver_manager/_lib.pyx", line 605, in adbc_driver_manager._lib.AdbcDatabase.__init__
  File "adbc_driver_manager/_lib.pyx", line 262, in adbc_driver_manager._lib.check_error
adbc_driver_manager.ProgrammingError: NOT_FOUND: [Driver Manager] dlopen() failed: duckdb: cannot open shared object file: No such file or directory
dlopen() failed: libduckdb.so: cannot open shared object file: No such file or directory
Also searched these paths for manifests:
	not set: ADBC_DRIVER_PATH
	not enabled at build time: Conda prefix
	does not exist: /root/.config/adbc/drivers
	does not exist: /etc/adbc/drivers

To get the connection to work, the full path must be given:

con = dbapi.connect(driver="/tmp/uv-venv/etc/adbc/drivers/duckdb")

It would be great if providing the full path was not needed.

[amoeba]

Thanks for the report @esadek. I'm not sure yet what's going on but I'll take a look.

[amoeba]

This is some oddity of molab, after I type this I'll go see if I can find a bug report.

The Python driver manager package compares sys.prefix with sys.base_prefix to detect and locate a venv. If they're different, we're in a venv and sys.prefix has the location. On molab, these two values are the same which is unexpected.

At the very least, dbc is doing the right thing so that's good.

[amoeba]

A few notes:

1. I didn't find any bug reports about this
2. Local Marimo does not have this issue, just Molab.
3. If we still wanted to document Molab, we could note the user can fix their situation by running sys.prefix = os.getenv("VIRTUAL_ENV")

[amoeba]

I'm asking on their Discord about it: https://discord.com/channels/1059888774789730424/1437990865946673285/1437990865946673285.

We could consider a change in adbc_driver_manager but I'll leave this open until I hear back from someone on their team and we can go from there.

[amoeba]

I never heard got a response from anyone at Molab but I'd considered this to be a quirk or Molab and not something we necessarily want to handle. So if we document Molab, I think I'd go for my option (3) above.