Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HuggingFaceHub fails with "Error raised by inference API: Cannot override task for LLM models" #772

Closed
nokados opened this issue May 3, 2024 · 7 comments · Fixed by #784
Labels
bug Something isn't working

Comments

@nokados
Copy link

nokados commented May 3, 2024

Description

Hugging Face Hub Integration does not work. I can access HF Inference API from curl, but jupyter-ai chat fails with the ValueError: Error raised by inference API: Cannot override task for LLM models (see Context section for the full traceback). I've tried different open models: mistralai/Mistral-7B-Instruct-v0.2, bigcode/starcoder2-3b. None of them works.

I guess, the reason is that you set task argument explicitly, while langchain does not set it. Moreover, Langchain uses InferenceClient instead of deprecated InferenceApi.

Anyway, the best solution, in my opinion, is to replace langchain's HuggingFaceHub with HuggingFaceEndpoint, because the first is deprecated in langchain.

Reproduce

  1. Go to Jupyter AI Chat Settings
  2. Choose Hugging Face Hub as Language Model
  3. Set Local Model ID to bigcode/starcoder2-3b
  4. Set valid API key and click Save
  5. Go back to chat and write "Test"
  6. See error in the chat

Expected behavior

Any answer from the model served on HF Inference API

Context

Traceback From Chat
Traceback (most recent call last):
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_ai/chat_handlers/base.py", line 125, in on_message
    await self.process_message(message)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_ai/chat_handlers/default.py", line 61, in process_message
    response = await self.llm_chain.apredict(input=message.body, stop=["\nHuman:"])
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/llm.py", line 333, in apredict
    return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 157, in awarning_emitting_wrapper
    return await wrapped(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/base.py", line 428, in acall
    return await self.ainvoke(
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/base.py", line 212, in ainvoke
    raise e
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/base.py", line 203, in ainvoke
    await self._acall(inputs, run_manager=run_manager)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/llm.py", line 298, in _acall
    response = await self.agenerate([inputs], run_manager=run_manager)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/chains/llm.py", line 165, in agenerate
    return await self.llm.agenerate_prompt(
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 643, in agenerate_prompt
    return await self.agenerate(
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 1018, in agenerate
    output = await self._agenerate_helper(
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 882, in _agenerate_helper
    raise e
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 866, in _agenerate_helper
    await self._agenerate(
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain_core/language_models/llms.py", line 1338, in _agenerate
    else await self._acall(prompt, stop=stop, **kwargs)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_ai_magics/providers.py", line 684, in _acall
    return await self._call_in_executor(*args, **kwargs)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_ai_magics/providers.py", line 322, in _call_in_executor
    return await loop.run_in_executor(executor, _call_with_args)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_ai_magics/providers.py", line 647, in _call
    raise ValueError(f"Error raised by inference API: {response['error']}")
ValueError: Error raised by inference API: Cannot override task for LLM models
  • Operating System and version: Ubuntu 20.04.6 LTS
  • Browser and version: Firefox 120.0.1
  • JupyterLab version: 4.1.8
Troubleshoot Output
$PATH:
	/home/user/.pyenv/versions/3.10.5/bin
	/home/user/.pyenv/libexec
	/home/user/.pyenv/plugins/python-build/bin
	/home/user/.pyenv/plugins/pyenv-virtualenv/bin
	/home/user/.pyenv/plugins/pyenv-update/bin
	/home/user/.pyenv/plugins/pyenv-jupyter-kernel/bin
	/home/user/.pyenv/plugins/pyenv-installer/bin
	/home/user/.pyenv/plugins/pyenv-doctor/bin
	/home/user/.local/bin
	/home/user/.pyenv/plugins/pyenv-virtualenv/shims
	/home/user/.pyenv/shims
	/home/user/.pyenv/bin
	/usr/local/sbin
	/usr/local/bin
	/usr/sbin
	/usr/bin
	/sbin
	/bin
	/usr/games
	/usr/local/games
	/snap/bin

sys.path:
/home/user/.pyenv/versions/3.10.5/bin
/home/user/.pyenv/versions/3.10.5/lib/python310.zip
/home/user/.pyenv/versions/3.10.5/lib/python3.10
/home/user/.pyenv/versions/3.10.5/lib/python3.10/lib-dynload
/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages

sys.executable:
/home/user/.pyenv/versions/3.10.5/bin/python3.10

sys.version:
3.10.5 (main, Jun 16 2022, 15:48:16) [GCC 9.4.0]

platform.platform():
Linux-5.4.0-177-generic-x86_64-with-glibc2.31

which -a jupyter:
/home/user/.pyenv/versions/3.10.5/bin/jupyter
/home/user/.pyenv/shims/jupyter

pip list:
Package Version
------------------------------------- ------------------
absl-py 1.1.0
adal 1.2.7
aiofiles 22.1.0
aiohttp 3.8.4
aiosignal 1.3.1
aiosqlite 0.19.0
altair 5.0.1
annotated-types 0.6.0
anyio 3.6.1
appdirs 1.4.4
APScheduler 3.6.3
argcomplete 3.3.0
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
arrow 1.2.3
astroid 2.11.6
asttokens 2.0.5
astunparse 1.6.3
async-lru 2.0.4
async-timeout 4.0.2
attrs 23.1.0
autopep8 1.6.0
azure 4.0.0
azure-applicationinsights 0.1.1
azure-batch 4.1.3
azure-common 1.1.28
azure-core 1.24.1
azure-cosmosdb-nspkg 2.0.2
azure-cosmosdb-table 1.0.6
azure-datalake-store 0.0.52
azure-eventgrid 1.3.0
azure-graphrbac 0.40.0
azure-keyvault 1.1.0
azure-loganalytics 0.1.1
azure-mgmt 4.0.0
azure-mgmt-advisor 1.0.1
azure-mgmt-applicationinsights 0.1.1
azure-mgmt-authorization 0.50.0
azure-mgmt-batch 5.0.1
azure-mgmt-batchai 2.0.0
azure-mgmt-billing 0.2.0
azure-mgmt-cdn 3.1.0
azure-mgmt-cognitiveservices 3.0.0
azure-mgmt-commerce 1.0.1
azure-mgmt-compute 4.6.2
azure-mgmt-consumption 2.0.0
azure-mgmt-containerinstance 1.5.0
azure-mgmt-containerregistry 2.8.0
azure-mgmt-containerservice 4.4.0
azure-mgmt-cosmosdb 0.4.1
azure-mgmt-datafactory 0.6.0
azure-mgmt-datalake-analytics 0.6.0
azure-mgmt-datalake-nspkg 3.0.1
azure-mgmt-datalake-store 0.5.0
azure-mgmt-datamigration 1.0.0
azure-mgmt-devspaces 0.1.0
azure-mgmt-devtestlabs 2.2.0
azure-mgmt-dns 2.1.0
azure-mgmt-eventgrid 1.0.0
azure-mgmt-eventhub 2.6.0
azure-mgmt-hanaonazure 0.1.1
azure-mgmt-iotcentral 0.1.0
azure-mgmt-iothub 0.5.0
azure-mgmt-iothubprovisioningservices 0.2.0
azure-mgmt-keyvault 1.1.0
azure-mgmt-loganalytics 0.2.0
azure-mgmt-logic 3.0.0
azure-mgmt-machinelearningcompute 0.4.1
azure-mgmt-managementgroups 0.1.0
azure-mgmt-managementpartner 0.1.1
azure-mgmt-maps 0.1.0
azure-mgmt-marketplaceordering 0.1.0
azure-mgmt-media 1.0.1
azure-mgmt-monitor 0.5.2
azure-mgmt-msi 0.2.0
azure-mgmt-network 2.7.0
azure-mgmt-notificationhubs 2.1.0
azure-mgmt-nspkg 3.0.2
azure-mgmt-policyinsights 0.1.0
azure-mgmt-powerbiembedded 2.0.0
azure-mgmt-rdbms 1.9.0
azure-mgmt-recoveryservices 0.3.0
azure-mgmt-recoveryservicesbackup 0.3.0
azure-mgmt-redis 5.0.0
azure-mgmt-relay 0.1.0
azure-mgmt-reservations 0.2.1
azure-mgmt-resource 2.2.0
azure-mgmt-scheduler 2.0.0
azure-mgmt-search 2.1.0
azure-mgmt-servicebus 0.5.3
azure-mgmt-servicefabric 0.2.0
azure-mgmt-signalr 0.1.1
azure-mgmt-sql 0.9.1
azure-mgmt-storage 2.0.0
azure-mgmt-subscription 0.2.0
azure-mgmt-trafficmanager 0.50.0
azure-mgmt-web 0.35.0
azure-nspkg 3.0.2
azure-servicebus 0.21.1
azure-servicefabric 6.3.0.0
azure-servicemanagement-legacy 0.20.7
azure-storage-blob 1.5.0
azure-storage-common 1.4.2
azure-storage-file 1.4.0
azure-storage-queue 1.4.0
Babel 2.10.3
backcall 0.2.0
backoff 2.2.1
beautifulsoup4 4.11.1
bleach 1.5.0
blis 0.7.7
boto 2.49.0
boto3 1.24.8
boto3-stubs 1.24.8
botocore 1.27.8
botocore-stubs 1.27.8
bz2file 0.98
cachetools 4.2.2
catalogue 2.0.7
certifi 2022.5.18.1
cffi 1.15.0
chardet 4.0.0
charset-normalizer 2.0.12
click 8.1.3
clickhouse-connect 0.5.25
cloudpickle 3.0.0
cloudstorage 0.11.0
coloredlogs 15.0.1
commonmark 0.9.1
configparser 5.2.0
cryptography 37.0.2
cycler 0.11.0
cymem 2.0.6
Cython 0.29.30
dask 2024.3.1
dataclasses-json 0.5.7
DAWG-Python 0.7.2
debugpy 1.6.0
decorator 5.1.1
deepmerge 1.1.1
defusedxml 0.7.1
Deprecated 1.2.13
deprecation 2.1.0
dill 0.3.5.1
distlib 0.3.4
distributed 2024.3.1
docker-pycreds 0.4.0
docopt 0.6.2
dogpile.cache 1.1.6
duckdb 0.8.0
elasticsearch 7.17.4
elasticsearch-dsl 7.4.0
en-core-web-sm 3.3.0
entrypoints 0.4
et-xmlfile 1.1.0
executing 0.8.3
faiss-cpu 1.8.0
fastjsonschema 2.15.3
ffmpy 0.3.1
filelock 3.7.1
flake8 4.0.1
flatbuffers 1.12
fonttools 4.33.3
fqdn 1.5.1
frozenlist 1.3.3
fsspec 2023.5.0
ftfy 6.1.1
fuzzywuzzy 0.18.0
gast 0.4.0
gensim 4.2.0
gitdb 4.0.10
GitPython 3.1.37
google-api-core 2.8.1
google-auth 2.7.0
google-auth-oauthlib 0.4.6
google-cloud-core 2.3.1
google-cloud-storage 2.4.0
google-crc32c 1.3.0
google-pasta 0.2.0
google-resumable-media 2.3.3
googleapis-common-protos 1.56.2
gpt4all 2.6.0
gradio 3.40.1
gradio_client 0.4.0
greenlet 2.0.2
grpcio 1.46.3
h11 0.14.0
h5py 3.7.0
hnswlib 0.7.0
html5lib 0.9999999
httpcore 1.0.2
httptools 0.5.0
httpx 0.26.0
huggingface-hub 0.14.1
humanfriendly 10.0
idna 3.3
importlib_metadata 7.1.0
importlib-resources 6.0.1
inflection 0.5.1
ipykernel 6.15.0
ipympl 0.9.2
ipython 8.4.0
ipython-genutils 0.2.0
ipywidgets 8.0.2
iso8601 1.0.2
isodate 0.6.1
isoduration 20.11.0
isort 5.10.1
itsdangerous 2.1.2
jedi 0.17.2
jellyfish 0.9.0
Jinja2 3.1.2
jmespath 1.0.0
joblib 1.1.0
json5 0.9.8
jsonpatch 1.33
jsonpath-ng 1.6.1
jsonpointer 2.3
jsonschema 4.19.0
jsonschema-specifications 2023.7.1
jupyter_ai 2.14.1
jupyter_ai_magics 2.14.1
jupyter_client 8.3.1
jupyter_core 5.3.1
jupyter-events 0.7.0
jupyter-lsp 2.2.0
jupyter_server 2.7.3
jupyter_server_fileid 0.9.0
jupyter_server_terminals 0.4.4
jupyter_server_ydoc 0.8.0
jupyter-ydoc 0.2.5
jupyterlab 4.1.8
jupyterlab-lsp 5.0.0
jupyterlab_pygments 0.3.0
jupyterlab_server 2.27.1
jupyterlab-widgets 3.0.3
keras 2.9.0
Keras-Preprocessing 1.1.2
keystoneauth1 4.6.0
kiwisolver 1.4.3
langchain 0.1.17
langchain-community 0.0.36
langchain-core 0.1.50
langchain-text-splitters 0.0.1
langcodes 3.3.0
langdetect 1.0.9
langsmith 0.1.31
lazy-object-proxy 1.7.1
Levenshtein 0.21.1
libclang 14.0.1
lightgbm 3.3.2
linkify-it-py 2.0.2
llama-cpp-python 0.1.56
locket 1.0.0
lxml 4.9.0
lz4 4.3.2
Markdown 3.3.7
markdown-it-py 2.2.0
MarkupSafe 2.1.1
marshmallow 3.19.0
marshmallow-enum 1.5.1
matplotlib 3.5.2
matplotlib-inline 0.1.3
mccabe 0.6.1
mdit-py-plugins 0.3.3
mdurl 0.1.2
minio 6.0.0
mistune 0.8.4
monotonic 1.6
mpmath 1.3.0
msg-parser 1.2.0
msgpack 1.0.4
msrest 0.7.1
msrestazure 0.6.4
multidict 6.0.4
munch 2.5.0
murmurhash 1.0.7
mypy-boto3-s3 1.24.0
mypy-extensions 1.0.0
nbclassic 0.3.7
nbclient 0.6.4
nbconvert 6.5.0
nbformat 5.4.0
nest-asyncio 1.5.5
netifaces 0.11.0
nltk 3.7
notebook 6.4.12
notebook_shim 0.2.3
numexpr 2.8.4
numpy 1.22.4
oauthlib 3.2.0
olefile 0.46
onnxruntime 1.15.0
openapi-schema-pydantic 1.2.4
opencorpora-tools 0.6
openpyxl 3.1.2
openstacksdk 0.17.2
opt-einsum 3.3.0
orjson 3.9.15
os-service-types 1.7.0
overrides 7.3.1
packaging 23.2
pandas 1.4.2
pandocfilters 1.5.0
parso 0.7.1
partd 1.4.1
pathtools 0.1.2
pathy 0.6.1
pbr 5.9.0
pdfminer.six 20221105
pexpect 4.8.0
pickleshare 0.7.5
pika 1.2.1
Pillow 9.1.1
pip 24.0
pipenv 2022.7.4
pipx 1.5.0
platformdirs 2.5.2
pluggy 1.0.0
ply 3.11
posthog 3.0.1
preshed 3.0.6
prometheus-client 0.14.1
prompt-toolkit 3.0.29
protobuf 3.19.4
psutil 5.9.1
ptyprocess 0.7.0
pure-eval 0.2.2
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycodestyle 2.8.0
pycparser 2.21
pydantic 2.7.1
pydantic_core 2.18.2
pydocstyle 6.1.1
pydub 0.25.1
pyemd 0.5.1
pyenchant 3.2.2
pyflakes 2.4.0
Pygments 2.12.0
PyJWT 2.4.0
pylint 2.14.2
pymorphy2 0.9.1
pymorphy2-dicts-ru 2.4.417127.4579844
pymystem3 0.2.0
pypandoc 1.11
pyparsing 3.0.9
pypdf 3.16.4
pyrsistent 0.18.1
python-dateutil 2.8.2
python-docx 0.8.11
python-dotenv 1.0.0
python-json-logger 2.0.7
python-jsonrpc-server 0.4.0
python-Levenshtein 0.21.1
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.4.1
python-magic 0.4.27
python-multipart 0.0.6
python-pptx 0.6.21
python-telegram-bot 13.12
pytz 2022.1
pytz-deprecation-shim 0.1.0.post0
PyYAML 6.0
pyzmq 25.1.1
rackspaceauth 0.2.0
rackspacesdk 0.7.5
rapidfuzz 3.2.0
redis 4.3.3
referencing 0.30.2
regex 2022.6.2
requests 2.31.0
requests-oauthlib 1.3.1
requestsexceptions 1.4.0
rfc3339-validator 0.1.4
rfc3986 1.5.0
rfc3986-validator 0.1.1
rich 13.0.1
rope 1.1.1
rpds-py 0.10.2
rsa 4.8
s3transfer 0.6.0
scikit-learn 1.1.1
scipy 1.8.1
seaborn 0.11.2
semantic-version 2.10.0
Send2Trash 1.8.2
sentry-sdk 1.31.0
setproctitle 1.3.2
setuptools 62.4.0
six 1.16.0
smart-open 5.2.1
smmap 5.0.1
sniffio 1.2.0
snowballstemmer 2.2.0
sortedcontainers 2.4.0
soupsieve 2.3.2.post1
spacy-legacy 3.0.9
spacy-loggers 1.0.2
SQLAlchemy 2.0.15
srsly 2.4.3
stack-data 0.3.0
starlette 0.27.0
stevedore 3.5.0
sympy 1.12
tblib 3.0.0
tenacity 8.2.2
tensorboard 2.9.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow 2.9.1
tensorflow-estimator 2.9.0
tensorflow-gpu 2.9.1
tensorflow-io-gcs-filesystem 0.26.0
termcolor 1.1.0
terminado 0.15.0
threadpoolctl 3.1.0
tiktoken 0.6.0
tinycss2 1.1.1
tokenizers 0.13.3
toml 0.10.2
tomli 2.0.1
tomlkit 0.11.0
toolz 0.12.0
tornado 6.3.3
tqdm 4.64.0
traitlets 5.9.0
transformers 4.29.2
transliterate 1.10.2
typer 0.4.2
typing_extensions 4.6.2
typing-inspect 0.9.0
tzdata 2022.1
tzlocal 4.2
uc-micro-py 1.0.2
ujson 5.3.0
unstructured 0.6.11
uri-template 1.3.0
urllib3 1.26.9
userpath 1.9.2
uvicorn 0.22.0
uvloop 0.17.0
virtualenv 20.15.1
virtualenv-clone 0.5.7
wandb 0.15.11
wasabi 0.9.1
watchfiles 0.19.0
wcwidth 0.2.5
webcolors 1.13
webencodings 0.5.1
websocket-client 1.3.2
websockets 11.0.3
Werkzeug 2.1.2
wheel 0.37.1
widgetsnbextension 4.0.3
wordfreq 3.0.1
wrapt 1.14.1
xattr 0.9.9
xlrd 2.0.1
XlsxWriter 3.1.2
xlwt 1.3.0
y-py 0.6.0
yapf 0.32.0
yarl 1.9.2
ypy-websocket 0.8.4
zict 3.0.0
zipp 3.8.0
zstandard 0.21.0

Command Line Output
[W 2024-05-03 21:37:35.229 ServerApp] ServerApp.password config is deprecated in 2.0. Use PasswordIdentityProvider.hashed_password.
/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/_api/module_import.py:87: LangChainDeprecationWarning: Importing GuardrailsOutputParser from langchain.output_parsers is deprecated. Please replace the import with the following:
from langchain_community.output_parsers.rail_parser import GuardrailsOutputParser
  warnings.warn(
/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/langchain/document_loaders/__init__.py:36: LangChainDeprecationWarning: Importing document loaders from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.document_loaders import PyPDFLoader.

To install langchain-community run pip install -U langchain-community.
warnings.warn(
[I 2024-05-03 21:37:36.796 ServerApp] jupyter_ai | extension was successfully linked.
[I 2024-05-03 21:37:36.796 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-05-03 21:37:36.799 ServerApp] jupyter_server_fileid | extension was successfully linked.
[I 2024-05-03 21:37:36.803 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-05-03 21:37:36.806 ServerApp] jupyter_server_ydoc | extension was successfully linked.
[I 2024-05-03 21:37:36.811 ServerApp] jupyterlab | extension was successfully linked.
[W 2024-05-03 21:37:36.812 ServerApp] nbclassic | error linking extension: module 'notebook' has no attribute 'base'
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 342, in link_extension
extension.link_all_points(self.serverapp)
File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 224, in link_all_points
self.link_point(point_name, serverapp)
File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 214, in link_point
point.link(serverapp)
File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 136, in link
linker(serverapp)
File "/home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/nbclassic/notebookapp.py", line 197, in _link_jupyter_server_extension
notebook.base.handlers.IPythonHandler.get_template = get_template
AttributeError: module 'notebook' has no attribute 'base'
[I 2024-05-03 21:37:36.815 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-05-03 21:37:37.135 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-05-03 21:37:37.136 AiExtension] Configured provider allowlist: None
[I 2024-05-03 21:37:37.141 AiExtension] Configured provider blocklist: None
[I 2024-05-03 21:37:37.141 AiExtension] Configured model allowlist: None
[I 2024-05-03 21:37:37.141 AiExtension] Configured model blocklist: None
[I 2024-05-03 21:37:37.141 AiExtension] Configured model parameters: {}
[I 2024-05-03 21:37:37.168 AiExtension] Registered model provider ai21.
[I 2024-05-03 21:37:37.169 AiExtension] Registered model provider bedrock.
[I 2024-05-03 21:37:37.169 AiExtension] Registered model provider bedrock-chat.
[W 2024-05-03 21:37:37.169 AiExtension] Unable to load model provider anthropic. Please install the langchain_anthropic package.
[W 2024-05-03 21:37:37.169 AiExtension] Unable to load model provider anthropic-chat. Please install the langchain_anthropic package.
[W 2024-05-03 21:37:37.170 AiExtension] Unable to load model provider azure-chat-openai. Please install the langchain_openai package.
[I 2024-05-03 21:37:37.170 AiExtension] Registered model provider cohere.
[W 2024-05-03 21:37:37.170 AiExtension] Unable to load model provider gemini. Please install the langchain_google_genai package.
[I 2024-05-03 21:37:37.170 AiExtension] Registered model provider gpt4all.
[I 2024-05-03 21:37:37.170 AiExtension] Registered model provider huggingface_hub.
[W 2024-05-03 21:37:37.170 AiExtension] Unable to load model provider nvidia-chat. Please install the langchain_nvidia_ai_endpoints package.
[W 2024-05-03 21:37:37.171 AiExtension] Unable to load model provider openai. Please install the langchain_openai package.
[W 2024-05-03 21:37:37.171 AiExtension] Unable to load model provider openai-chat. Please install the langchain_openai package.
[I 2024-05-03 21:37:37.171 AiExtension] Registered model provider qianfan.
[I 2024-05-03 21:37:37.171 AiExtension] Registered model provider sagemaker-endpoint.
[I 2024-05-03 21:37:37.171 AiExtension] Registered model provider togetherai.
[I 2024-05-03 21:37:37.199 AiExtension] Registered embeddings model provider bedrock.
[I 2024-05-03 21:37:37.199 AiExtension] Registered embeddings model provider cohere.
[I 2024-05-03 21:37:37.199 AiExtension] Registered embeddings model provider gpt4all.
[I 2024-05-03 21:37:37.199 AiExtension] Registered embeddings model provider huggingface_hub.
[E 2024-05-03 21:37:37.200 AiExtension] Unable to load embeddings model provider class from entry point openai: No module named 'langchain_openai'.
[I 2024-05-03 21:37:37.200 AiExtension] Registered embeddings model provider qianfan.
[I 2024-05-03 21:37:37.206 AiExtension] Registered providers.
[I 2024-05-03 21:37:37.206 AiExtension] Registered jupyter_ai server extension
[I 2024-05-03 21:37:37.237 AiExtension] Initialized Jupyter AI server extension in 102 ms.
[I 2024-05-03 21:37:37.238 ServerApp] jupyter_ai | extension was successfully loaded.
[I 2024-05-03 21:37:37.240 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-05-03 21:37:37.241 FileIdExtension] Configured File ID manager: ArbitraryFileIdManager
[I 2024-05-03 21:37:37.241 FileIdExtension] ArbitraryFileIdManager : Configured root dir: /home/user
[I 2024-05-03 21:37:37.241 FileIdExtension] ArbitraryFileIdManager : Configured database path: /home/user/.local/share/jupyter/file_id_manager.db
[I 2024-05-03 21:37:37.241 FileIdExtension] ArbitraryFileIdManager : Successfully connected to database file.
[I 2024-05-03 21:37:37.241 FileIdExtension] ArbitraryFileIdManager : Creating File ID tables and indices with journal_mode = DELETE
[I 2024-05-03 21:37:37.241 FileIdExtension] Attached event listeners.
[I 2024-05-03 21:37:37.242 ServerApp] jupyter_server_fileid | extension was successfully loaded.
[I 2024-05-03 21:37:37.242 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-05-03 21:37:37.243 ServerApp] jupyter_server_ydoc | extension was successfully loaded.
[I 2024-05-03 21:37:37.244 LabApp] JupyterLab extension loaded from /home/user/.pyenv/versions/3.10.5/lib/python3.10/site-packages/jupyterlab
[I 2024-05-03 21:37:37.244 LabApp] JupyterLab application directory is /home/user/.pyenv/versions/3.10.5/share/jupyter/lab
[I 2024-05-03 21:37:37.245 LabApp] Extension Manager is 'pypi'.
[I 2024-05-03 21:37:37.255 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-05-03 21:37:37.258 ServerApp] nbclassic | extension was successfully loaded.
[I 2024-05-03 21:37:37.258 ServerApp] Serving notebooks from local directory: /home/user
[I 2024-05-03 21:37:37.258 ServerApp] Jupyter Server 2.7.3 is running at:
[I 2024-05-03 21:37:37.258 ServerApp] http://localhost:8888/lab
[I 2024-05-03 21:37:37.258 ServerApp] http://127.0.0.1:8888/lab
[I 2024-05-03 21:37:37.258 ServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 2024-05-03 21:37:37.703 ServerApp] Skipped non-installed server(s): bash-language-server, dockerfile-language-server-nodejs, javascript-typescript-langserver, jedi-language-server, julia-language-server, pyright, python-language-server, r-languageserver, sql-language-server, texlab, typescript-language-server, unified-language-server, vscode-css-languageserver-bin, vscode-html-languageserver-bin, vscode-json-languageserver-bin, yaml-language-server
[I 2024-05-03 21:37:41.017 ServerApp] Client connected. ID: someid
[I 2024-05-03 21:38:11.943 AiExtension] Switching chat language model from None to huggingface_hub:bigcode/starcoder2-3b.
[E 2024-05-03 21:38:13.027 AiExtension] Error raised by inference API: Cannot override task for LLM models
[I 2024-05-03 21:38:13.029 ServerApp] Default chat handler resolved in 1087 ms.
[C 2024-05-03 21:38:27.574 ServerApp] received signal 15, stopping
[I 2024-05-03 21:38:27.575 ServerApp] Shutting down 8 extensions
[I 2024-05-03 21:38:27.575 AiExtension] Closing Dask client.

ESC[1m> Entering new ConversationChain chain...ESC[0m
Prompt after formatting:
ESC[32;1mESC[1;3mYou are Jupyternaut, a conversational assistant living in JupyterLab to help users.
You are not a language model, but rather an application built on a foundation model from Hugging Face Hub called bigcode/starcoder2-3b.
You are talkative and you provide lots of specific details from the foundation model's context.
You may use Markdown to format your response.
Code blocks must be formatted in Markdown.
Math should be rendered with inline TeX markup, surrounded by $.
If you do not know the answer to a question, answer truthfully by responding that you do not know.
The following is a friendly conversation between you and a human.

Current conversation:

Human: Test
AI:ESC[0m

@nokados nokados added the bug Something isn't working label May 3, 2024
Copy link

welcome bot commented May 3, 2024

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.
welcome
You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

@wyycommu
Copy link

wyycommu commented May 7, 2024

Met the same problem. And I think I'm already get access to this model through "https://huggingface.co/bigcode/starcoder"

@paulohenriquecrs
Copy link

I have the same problem with different models as well (including mistralai/Mistral-7B-Instruct-v0.2 and meta-llama/Meta-Llama-3-8B)

1 similar comment
@Adilmar
Copy link

Adilmar commented May 10, 2024

I have the same problem with different models as well (including mistralai/Mistral-7B-Instruct-v0.2 and meta-llama/Meta-Llama-3-8B)

@Adilmar
Copy link

Adilmar commented May 12, 2024

I can solve it by updating the libraries:langchain and huggingface-hub

@wyycommu
Copy link

according to https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/discussions/115, changing huggingface_hub.interface_api file on 152 line i replaced api_url as as self.api_url = f"{INFERENCE_ENDPOINT}/models/{repo_id}"
it works for me.

But it seems that the length of answer is limitted. I don't know why :(

@dlqqq
Copy link
Member

dlqqq commented May 16, 2024

Thank you all for reporting this issue! We just merged a PR to fix this. It will be included in the next release, tentatively scheduled for early next week.

Note: this PR will not be backported to 1.x as JupyterLab 3 reached end-of-maintenance yesterday. See #761.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants