Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This PR fix devika #603

Open
wants to merge 23 commits into
base: main
Choose a base branch
from
Open

This PR fix devika #603

wants to merge 23 commits into from

Conversation

darrassi1
Copy link
Contributor

No description provided.

If no project is selected, return None immediately
remove backtick (`)  from generated files
add normpath to resolvle backslach compatibilite beetwen linux and windows
update save code function to resolve backslash issue
When a project is created, the folder is created immediately. 
When the "Delete Project" button is triggered, the project is deleted immediately.
Setting the temperature to 0 is not good, because the model will return the prompt repeatedly.
Setting the temperature to 0 is not good, because the model will return the prompt repeatedly.
Setting the temperature to 0 is not good, because the model will return the prompt repeatedly.

so  setting temperature to 1 will be better option.
Setting the temperature to 0 is not good, because the model will return the prompt repeatedly.
Setting the temperature to 0 is not good, because the model will return the prompt repeatedly.
Copy link

@Josephrp Josephrp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks !

@darrassi1 darrassi1 marked this pull request as draft June 16, 2024 06:33
Change the origin of frontend URL
handle more use cases of validate function of coder
this change fix this   error   File "C:\Users\pc\Desktop\stream_main\devika\src\agents\coder\coder.py", line 131, in execute
    valid_response = self.validate_response(response)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\pc\Desktop\stream_main\devika\src\agents\coder\coder.py", line 37, in validate_response
    self.logger.debug(f"Response from the model: {response}")
  File "C:\Users\pc\Desktop\stream_main\devika\src\logger.py", line 32, in debug
    self.logger.debug(message)
  File "C:\Users\pc\Desktop\stream_main\devika\.venv\Lib\site-packages\fastlogging\fastlogging.py", line 248, in debug
    self.__log(DEBUG, msg, args, kwargs)
  File "C:\Users\pc\Desktop\stream_main\devika\.venv\Lib\site-packages\fastlogging\fastlogging.py", line 238, in __log
    self._logMessage(None, (log_time, domain, level, msg, kwargs), 0)
  File "C:\Users\pc\Desktop\stream_main\devika\.venv\Lib\site-packages\fastlogging\fastlogging.py", line 423, in _logMessage
    print(message, file=self.stdout if level < ERROR else self.stderr)
  File "C:\Users\pc\Desktop\stream_main\devika\.venv\Lib\site-packages\colorama\ansitowin32.py", line 47, in write
    self.__convertor.write(text)
@darrassi1
Copy link
Contributor Author

i tested it and its working i would happy if someone develop some sand box fonctionalite to run code and internal server for each project and visualize the internal dev server in Devika browser

@darrassi1 darrassi1 marked this pull request as ready for review June 16, 2024 07:04
@darrassi1 darrassi1 mentioned this pull request Jun 16, 2024
@werruww
Copy link

werruww commented Jun 16, 2024

Is it possible to upload the entire repaired files?

@darrassi1
Copy link
Contributor Author

Do gh pr checkout 603 and you will get the update

@werruww
Copy link

werruww commented Jun 16, 2024

Do and you will get the updategh pr checkout 603

how?

@werruww
Copy link

werruww commented Jun 16, 2024

(devika) C:\Users\m\Desktop\s\devika>python devika.py
24.06.16 13:13:59: root: INFO : Initializing Devika...
24.06.16 13:13:59: root: INFO : checking configurations...
24.06.16 13:13:59: root: INFO : Initializing Prerequisites Jobs...

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "C:\Users\m\Desktop\s\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\s\devika\src\init.py", line 27, in init_devika
from src.bert.sentence import SentenceBert
File "C:\Users\m\Desktop\s\devika\src\bert\sentence.py", line 1, in
from keybert import KeyBERT
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\keybert_init_.py", line 3, in
from keybert.llm import KeyLLM
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\keybert_llm.py", line 4, in
from sentence_transformers import util
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\sentence_transformers_init
.py", line 15, in
from sentence_transformers.trainer import SentenceTransformerTrainer
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\sentence_transformers\trainer.py", line 10, in
from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1525, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1535, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\Python312\Lib\importlib_init
.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\transformers\trainer.py", line 71, in
from .optimization import Adafactor, get_scheduler
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\transformers\optimization.py", line 27, in
from .trainer_pt_utils import LayerWiseDummyOptimizer, LayerWiseDummyScheduler
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py", line 235, in
device: Optional[torch.device] = torch.device("cuda"),
C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py:235: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
device: Optional[torch.device] = torch.device("cuda"),
C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\sentence_transformers\losses\DenoisingAutoEncoderLoss.py:16: SyntaxWarning: invalid escape sequence '_'

@werruww
Copy link

werruww commented Jun 16, 2024

(devika) C:\Users\m\Desktop\s\devika>gh pr checkout 603
From https:/stitionai/devika

  • branch refs/pull/603/head -> FETCH_HEAD
    Already up to date.

@werruww
Copy link

werruww commented Jun 16, 2024

24.06.16 13:14:44: root: INFO : Loading sentence-transformer BERT models...
Traceback (most recent call last):
File "C:\Users\m\Desktop\s\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\s\devika\src\init.py", line 31, in init_devika
SentenceBert(prompt).extract_keywords()
File "C:\Users\m\Desktop\s\devika\src\bert\sentence.py", line 9, in extract_keywords
keywords = self.kw_model.extract_keywords(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\keybert_model.py", line 195, in extract_keywords
doc_embeddings = self.model.embed(docs)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\keybert\backend_sentencetransformers.py", line 67, in embed
embeddings = self.embedding_model.encode(documents, **self.encode_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\s\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in encode
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^
RuntimeError: Numpy is not available

@werruww
Copy link

werruww commented Jun 16, 2024

(devika) C:\Users\m\Desktop\s\devika\ui>pip install numpy
Requirement already satisfied: numpy in c:\python312\lib\site-packages (2.0.0)

(devika) C:\Users\m\Desktop\s\devika\ui>

@werruww
Copy link

werruww commented Jun 16, 2024

Not working after update

@werruww
Copy link

werruww commented Jun 16, 2024

I tried through Pinocchio and the same problem. Does it require gpu

@darrassi1
Copy link
Contributor Author

I tried through Pinocchio and the same problem. Does it require gpu

Do I fresh install, your problem related to Sentencetransformer and the error say , there is just missing dependency ,you should run in Devika root directory not UI directory, and run uv pip install -r requirements.txt

@darrassi1
Copy link
Contributor Author

Not working after update

It's working just you should install all requirements,

@werruww
Copy link

werruww commented Jun 17, 2024

(devika) C:\Users\m\Desktop\t\devika>uv pip install -r requirements.txt
Audited 32 packages in 43ms

(devika) C:\Users\m\Desktop\t\devika>

@werruww
Copy link

werruww commented Jun 17, 2024

(devika) C:\Users\m\Desktop\t\devika>uv pip install -r requirements.txt
Audited 32 packages in 43ms

(devika) C:\Users\m\Desktop\t\devika>gh pr checkout 603
error: Your local changes to the following files would be overwritten by checkout:
devika.py
src/agents/coder/coder.py
src/llm/gemini_client.py
src/llm/ollama_client.py
src/project.py
Please commit your changes or stash them before you switch branches.
Aborting
failed to run git: exit status 1

(devika) C:\Users\m\Desktop\t\devika>

@werruww
Copy link

werruww commented Jun 17, 2024

anaconda prompt

(base) C:\WINDOWS\system32>cd C:\Users\m\Desktop\p

(base) C:\Users\m\Desktop\p>git clone https:/stitionai/devika.git
Cloning into 'devika'...
remote: Enumerating objects: 1483, done.
remote: Counting objects: 100% (637/637), done.
remote: Compressing objects: 100% (202/202), done.
Receiving objects: 100% (1483/1483), 6.07 MiB | 3.32 MiB/s, done.
Resolving deltas: 1% (9/864), reused 449 (delta 434), pack-reused 846Resolving deltas: 0% (0/864)
Resolving deltas: 100% (864/864), done.

(base) C:\Users\m\Desktop\p>cd devika

(base) C:\Users\m\Desktop\p\devika>gh pr checkout 603
remote: Enumerating objects: 71, done.
remote: Counting objects: 100% (71/71), done.
remote: Compressing objects: 100% (65/65), done.
remote: Total 71 (delta 42), reused 15 (delta 6), pack-reused 0
Unpacking objects: 100% (71/71), 23.13 KiB | 8.00 KiB/s, done.
From https:/stitionai/devika

  • [new ref] refs/pull/603/head -> darrassi1/main
    Switched to branch 'darrassi1/main'

(base) C:\Users\m\Desktop\p\devika>uv venv
Using Python 3.11.5 interpreter at: C:\Users\m\anaconda3\python.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate

(base) C:\Users\m\Desktop\p\devika>.venv\Scripts\activate

(devika) (base) C:\Users\m\Desktop\p\devika>uv pip install -r requirements.txt
Resolved 143 packages in 7.43s
Downloaded 2 packages in 5.57s
Installed 143 packages in 1m 05s

  • annotated-types==0.7.0
  • anthropic==0.28.1
  • anyio==4.4.0
  • arabic-reshaper==3.0.0
  • asn1crypto==1.5.1
  • beautifulsoup4==4.12.3
  • bidict==0.23.1
  • blinker==1.8.2
  • cachetools==5.3.3
  • certifi==2024.6.2
  • cffi==1.16.0
  • chardet==5.2.0
  • charset-normalizer==3.3.2
  • click==8.1.7
  • colorama==0.4.6
  • cryptography==42.0.8
  • cssselect2==0.7.0
  • distro==1.9.0
  • dnspython==2.6.1
  • duckduckgo-search==6.1.6
  • eventlet==0.36.1
  • fastlogging==1.0.0
  • filelock==3.15.1
  • flask==3.0.3
  • flask-cors==4.0.1
  • flask-socketio==5.3.6
  • fsspec==2024.6.0
  • gevent==24.2.1
  • gevent-websocket==0.10.1
  • gitdb==4.0.11
  • gitpython==3.1.43
  • google-ai-generativelanguage==0.6.4
  • google-api-core==2.19.0
  • google-api-python-client==2.133.0
  • google-auth==2.30.0
  • google-auth-httplib2==0.2.0
  • google-generativeai==0.6.0
  • googleapis-common-protos==1.63.1
  • greenlet==3.0.3
  • groq==0.9.0
  • grpcio==1.64.1
  • grpcio-status==1.62.2
  • h11==0.14.0
  • html5lib==1.1
  • httpcore==1.0.5
  • httplib2==0.22.0
  • httpx==0.27.0
  • huggingface-hub==0.23.4
  • idna==3.7
  • iniconfig==2.0.0
  • intel-openmp==2021.4.0
  • itsdangerous==2.2.0
  • jinja2==3.1.4
  • jiter==0.4.2
  • joblib==1.4.2
  • keybert==0.8.5
  • lxml==5.2.2
  • markdown==3.6
  • markdown-it-py==3.0.0
  • markdownify==0.12.1
  • markupsafe==2.1.5
  • mdurl==0.1.2
  • mistletoe==1.3.0
  • mistralai==0.3.0
  • mkl==2021.4.0
  • mpmath==1.3.0
  • netlify-py==0.1.0
  • networkx==3.3
  • numpy==2.0.0
  • ollama==0.2.1
  • openai==1.34.0
  • orjson==3.10.5
  • oscrypto==1.3.0
  • packaging==24.1
  • pdfminer-six==20231228
  • pillow==10.3.0
  • playwright==1.44.0
  • pluggy==1.5.0
  • proto-plus==1.23.0
  • protobuf==4.25.3
  • pyasn1==0.6.0
  • pyasn1-modules==0.4.0
  • pycparser==2.22
  • pydantic==2.7.4
  • pydantic-core==2.18.4
  • pyee==11.1.0
  • pygments==2.18.0
  • pyhanko==0.25.0
  • pyhanko-certvalidator==0.26.3
  • pyparsing==3.1.2
  • pypdf==4.2.0
  • pypng==0.20220715.0
  • pyreqwest-impersonate==0.4.7
  • pytest==8.2.2
  • pytest-base-url==2.1.0
  • pytest-playwright==0.5.0
  • python-bidi==0.4.2
  • python-engineio==4.9.1
  • python-slugify==8.0.4
  • python-socketio==5.11.2
  • pyyaml==6.0.1
  • qrcode==7.4.2
  • regex==2024.5.15
  • reportlab==4.2.0
  • requests==2.32.3
  • rich==13.7.1
  • rsa==4.9
  • safetensors==0.4.3
  • scikit-learn==1.5.0
  • scipy==1.13.1
  • sentence-transformers==3.0.1
  • setuptools==70.0.0
  • simple-websocket==1.0.0
  • six==1.16.0
  • smmap==5.0.1
  • sniffio==1.3.1
  • soupsieve==2.5
  • sqlalchemy==2.0.30
  • sqlmodel==0.0.19
  • svglib==1.5.1
  • sympy==1.12.1
  • tbb==2021.12.0
  • text-unidecode==1.3
  • threadpoolctl==3.5.0
  • tiktoken==0.7.0
  • tinycss2==1.3.0
  • tokenizers==0.19.1
  • toml==0.10.2
  • torch==2.3.1
  • tqdm==4.66.4
  • transformers==4.41.2
  • typing-extensions==4.12.2
  • tzdata==2024.1
  • tzlocal==5.2
  • uritemplate==4.1.1
  • uritools==4.0.3
  • urllib3==2.2.2
  • webencodings==0.5.1
  • werkzeug==3.0.3
  • wsproto==1.2.0
  • xhtml2pdf==0.2.16
  • zope-event==5.0
  • zope-interface==6.4.post2

(devika) (base) C:\Users\m\Desktop\p\devika>playwright install --with-deps

(devika) (base) C:\Users\m\Desktop\p\devika>python devika.py
24.06.17 10:28:44: root: INFO : Initializing Devika...
24.06.17 10:28:44: root: INFO : checking configurations...
24.06.17 10:28:44: root: INFO : Initializing Prerequisites Jobs...

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "C:\Users\m\Desktop\p\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\p\devika\src\init.py", line 27, in init_devika
from src.bert.sentence import SentenceBert
File "C:\Users\m\Desktop\p\devika\src\bert\sentence.py", line 1, in
from keybert import KeyBERT
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_init_.py", line 3, in
from keybert.llm import KeyLLM
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_llm.py", line 4, in
from sentence_transformers import util
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers_init
.py", line 15, in
from sentence_transformers.trainer import SentenceTransformerTrainer
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\trainer.py", line 10, in
from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1525, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1535, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\Users\m\anaconda3\Lib\importlib_init
.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer.py", line 71, in
from .optimization import Adafactor, get_scheduler
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\optimization.py", line 27, in
from .trainer_pt_utils import LayerWiseDummyOptimizer, LayerWiseDummyScheduler
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py", line 235, in
device: Optional[torch.device] = torch.device("cuda"),
C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py:235: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
device: Optional[torch.device] = torch.device("cuda"),
24.06.17 10:30:31: root: INFO : Loading sentence-transformer BERT models...
Traceback (most recent call last):
File "C:\Users\m\Desktop\p\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\p\devika\src\init.py", line 31, in init_devika
SentenceBert(prompt).extract_keywords()
File "C:\Users\m\Desktop\p\devika\src\bert\sentence.py", line 9, in extract_keywords
keywords = self.kw_model.extract_keywords(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_model.py", line 195, in extract_keywords
doc_embeddings = self.model.embed(docs)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert\backend_sentencetransformers.py", line 67, in embed embeddings = self.embedding_model.encode(documents, **self.encode_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in encode
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^
RuntimeError: Numpy is not available

(devika) (base) C:\Users\m\Desktop\p\devika>

@werruww
Copy link

werruww commented Jun 17, 2024

(base) C:\Users\m>cd C:\Users\m\Desktop\p

(base) C:\Users\m\Desktop\p>git clone https:/stitionai/devika.git
Cloning into 'devika'...
remote: Enumerating objects: 1483, done.
remote: Counting objects: 100% (648/648), done.
remote: Compressing objects: 100% (205/205), done.
Receiving objects: 100% (1483/1483), 6.07 MiB | 1.54 MiB/s, done.
remote: Total 1483 (delta 519), reused 457 (delta 442), pack-reused 835
Resolving deltas: 100% (866/866), done.

(base) C:\Users\m\Desktop\p>cd devika

(base) C:\Users\m\Desktop\p\devika>uv venv
Using Python 3.11.5 interpreter at: C:\Users\m\anaconda3\python.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate

(base) C:\Users\m\Desktop\p\devika>.venv\Scripts\activate

(devika) (base) C:\Users\m\Desktop\p\devika>uv pip install -r requirements.txt
Resolved 143 packages in 1.12s
Installed 143 packages in 30.15s

  • annotated-types==0.7.0
  • anthropic==0.28.1
  • anyio==4.4.0
  • arabic-reshaper==3.0.0
  • asn1crypto==1.5.1
  • beautifulsoup4==4.12.3
  • bidict==0.23.1
  • blinker==1.8.2
  • cachetools==5.3.3
  • certifi==2024.6.2
  • cffi==1.16.0
  • chardet==5.2.0
  • charset-normalizer==3.3.2
  • click==8.1.7
  • colorama==0.4.6
  • cryptography==42.0.8
  • cssselect2==0.7.0
  • distro==1.9.0
  • dnspython==2.6.1
  • duckduckgo-search==6.1.6
  • eventlet==0.36.1
  • fastlogging==1.0.0
  • filelock==3.15.1
  • flask==3.0.3
  • flask-cors==4.0.1
  • flask-socketio==5.3.6
  • fsspec==2024.6.0
  • gevent==24.2.1
  • gevent-websocket==0.10.1
  • gitdb==4.0.11
  • gitpython==3.1.43
  • google-ai-generativelanguage==0.6.4
  • google-api-core==2.19.0
  • google-api-python-client==2.133.0
  • google-auth==2.30.0
  • google-auth-httplib2==0.2.0
  • google-generativeai==0.6.0
  • googleapis-common-protos==1.63.1
  • greenlet==3.0.3
  • groq==0.9.0
  • grpcio==1.64.1
  • grpcio-status==1.62.2
  • h11==0.14.0
  • html5lib==1.1
  • httpcore==1.0.5
  • httplib2==0.22.0
  • httpx==0.27.0
  • huggingface-hub==0.23.4
  • idna==3.7
  • iniconfig==2.0.0
  • intel-openmp==2021.4.0
  • itsdangerous==2.2.0
  • jinja2==3.1.4
  • jiter==0.4.2
  • joblib==1.4.2
  • keybert==0.8.5
  • lxml==5.2.2
  • markdown==3.6
  • markdown-it-py==3.0.0
  • markdownify==0.12.1
  • markupsafe==2.1.5
  • mdurl==0.1.2
  • mistletoe==1.3.0
  • mistralai==0.3.0
  • mkl==2021.4.0
  • mpmath==1.3.0
  • netlify-py==0.1.0
  • networkx==3.3
  • numpy==2.0.0
  • ollama==0.2.1
  • openai==1.34.0
  • orjson==3.10.5
  • oscrypto==1.3.0
  • packaging==24.1
  • pdfminer-six==20231228
  • pillow==10.3.0
  • playwright==1.44.0
  • pluggy==1.5.0
  • proto-plus==1.23.0
  • protobuf==4.25.3
  • pyasn1==0.6.0
  • pyasn1-modules==0.4.0
  • pycparser==2.22
  • pydantic==2.7.4
  • pydantic-core==2.18.4
  • pyee==11.1.0
  • pygments==2.18.0
  • pyhanko==0.25.0
  • pyhanko-certvalidator==0.26.3
  • pyparsing==3.1.2
  • pypdf==4.2.0
  • pypng==0.20220715.0
  • pyreqwest-impersonate==0.4.7
  • pytest==8.2.2
  • pytest-base-url==2.1.0
  • pytest-playwright==0.5.0
  • python-bidi==0.4.2
  • python-engineio==4.9.1
  • python-slugify==8.0.4
  • python-socketio==5.11.2
  • pyyaml==6.0.1
  • qrcode==7.4.2
  • regex==2024.5.15
  • reportlab==4.2.0
  • requests==2.32.3
  • rich==13.7.1
  • rsa==4.9
  • safetensors==0.4.3
  • scikit-learn==1.5.0
  • scipy==1.13.1
  • sentence-transformers==3.0.1
  • setuptools==70.0.0
  • simple-websocket==1.0.0
  • six==1.16.0
  • smmap==5.0.1
  • sniffio==1.3.1
  • soupsieve==2.5
  • sqlalchemy==2.0.30
  • sqlmodel==0.0.19
  • svglib==1.5.1
  • sympy==1.12.1
  • tbb==2021.12.0
  • text-unidecode==1.3
  • threadpoolctl==3.5.0
  • tiktoken==0.7.0
  • tinycss2==1.3.0
  • tokenizers==0.19.1
  • toml==0.10.2
  • torch==2.3.1
  • tqdm==4.66.4
  • transformers==4.41.2
  • typing-extensions==4.12.2
  • tzdata==2024.1
  • tzlocal==5.2
  • uritemplate==4.1.1
  • uritools==4.0.3
  • urllib3==2.2.2
  • webencodings==0.5.1
  • werkzeug==3.0.3
  • wsproto==1.2.0
  • xhtml2pdf==0.2.16
  • zope-event==5.0
  • zope-interface==6.4.post2

(devika) (base) C:\Users\m\Desktop\p\devika>playwright install --with-deps

(devika) (base) C:\Users\m\Desktop\p\devika>python devika.py
24.06.17 10:37:54: root: INFO : Initializing Devika...
24.06.17 10:37:54: root: INFO : checking configurations...
24.06.17 10:37:54: root: INFO : Initializing Prerequisites Jobs...

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "C:\Users\m\Desktop\p\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\p\devika\src\init.py", line 27, in init_devika
from src.bert.sentence import SentenceBert
File "C:\Users\m\Desktop\p\devika\src\bert\sentence.py", line 1, in
from keybert import KeyBERT
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_init_.py", line 3, in
from keybert.llm import KeyLLM
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_llm.py", line 4, in
from sentence_transformers import util
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers_init
.py", line 15, in
from sentence_transformers.trainer import SentenceTransformerTrainer
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\trainer.py", line 10, in
from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1525, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1535, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\Users\m\anaconda3\Lib\importlib_init
.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer.py", line 71, in
from .optimization import Adafactor, get_scheduler
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\optimization.py", line 27, in
from .trainer_pt_utils import LayerWiseDummyOptimizer, LayerWiseDummyScheduler
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py", line 235, in
device: Optional[torch.device] = torch.device("cuda"),
C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py:235: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
device: Optional[torch.device] = torch.device("cuda"),
24.06.17 10:38:27: root: INFO : Loading sentence-transformer BERT models...
Traceback (most recent call last):
File "C:\Users\m\Desktop\p\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\p\devika\src\init.py", line 31, in init_devika
SentenceBert(prompt).extract_keywords()
File "C:\Users\m\Desktop\p\devika\src\bert\sentence.py", line 9, in extract_keywords
keywords = self.kw_model.extract_keywords(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_model.py", line 195, in extract_keywords
doc_embeddings = self.model.embed(docs)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert\backend_sentencetransformers.py", line 67, in embed
embeddings = self.embedding_model.encode(documents, **self.encode_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in encode
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^
RuntimeError: Numpy is not available

(devika) (base) C:\Users\m\Desktop\p\devika>python devika.py
24.06.17 10:41:53: root: INFO : Initializing Devika...
24.06.17 10:41:53: root: INFO : checking configurations...
24.06.17 10:41:53: root: INFO : Initializing Prerequisites Jobs...

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "C:\Users\m\Desktop\p\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\p\devika\src\init.py", line 27, in init_devika
from src.bert.sentence import SentenceBert
File "C:\Users\m\Desktop\p\devika\src\bert\sentence.py", line 1, in
from keybert import KeyBERT
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_init_.py", line 3, in
from keybert.llm import KeyLLM
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_llm.py", line 4, in
from sentence_transformers import util
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers_init
.py", line 15, in
from sentence_transformers.trainer import SentenceTransformerTrainer
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\trainer.py", line 10, in
from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1525, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1535, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\Users\m\anaconda3\Lib\importlib_init
.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer.py", line 71, in
from .optimization import Adafactor, get_scheduler
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\optimization.py", line 27, in
from .trainer_pt_utils import LayerWiseDummyOptimizer, LayerWiseDummyScheduler
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py", line 235, in
device: Optional[torch.device] = torch.device("cuda"),
C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py:235: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
device: Optional[torch.device] = torch.device("cuda"),
24.06.17 10:42:00: root: INFO : Loading sentence-transformer BERT models...
modules.json: 100%|████████████████████████████████████████████████████████████████████████████████| 349/349 [00:00<?, ?B/s]
C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\huggingface_hub\file_download.py:157: UserWarning: huggingface_hub cache-system uses symlinks by default to efficiently store duplicated files but your machine does not support them in C:\Users\m.cache\huggingface\hub\models--sentence-transformers--all-MiniLM-L6-v2. Caching files will still work but in a degraded version that might require more space on your disk. This warning can be disabled by setting the HF_HUB_DISABLE_SYMLINKS_WARNING environment variable. For more details, see https://huggingface.co/docs/huggingface_hub/how-to-cache#limitations.
To support symlinks on Windows, you either need to activate Developer Mode or to run Python as an administrator. In order to see activate developer mode, see this article: https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development
warnings.warn(message)
config_sentence_transformers.json: 100%|███████████████████████████████████████████████████████████| 116/116 [00:00<?, ?B/s]
README.md: 100%|███████████████████████████████████████████████████████████████████████████████| 10.7k/10.7k [00:00<?, ?B/s]
sentence_bert_config.json: 100%|█████████████████████████████████████████████████████████████████| 53.0/53.0 [00:00<?, ?B/s]
config.json: 100%|█████████████████████████████████████████████████████████████████████████| 612/612 [00:00<00:00, 39.2kB/s]
model.safetensors: 100%|███████████████████████████████████████████████████████████████| 90.9M/90.9M [00:25<00:00, 3.53MB/s]
tokenizer_config.json: 100%|███████████████████████████████████████████████████████████████████████| 350/350 [00:00<?, ?B/s]
vocab.txt: 100%|██████████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 872kB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████████| 466k/466k [00:00<00:00, 1.11MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████████| 112/112 [00:00<00:00, 7.18kB/s]
1_Pooling/config.json: 100%|███████████████████████████████████████████████████████████████████████| 190/190 [00:00<?, ?B/s]
Traceback (most recent call last):
File "C:\Users\m\Desktop\p\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\p\devika\src\init.py", line 31, in init_devika
SentenceBert(prompt).extract_keywords()
File "C:\Users\m\Desktop\p\devika\src\bert\sentence.py", line 9, in extract_keywords
keywords = self.kw_model.extract_keywords(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert_model.py", line 195, in extract_keywords
doc_embeddings = self.model.embed(docs)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\keybert\backend_sentencetransformers.py", line 67, in embed
embeddings = self.embedding_model.encode(documents, **self.encode_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in encode
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\p\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^
RuntimeError: Numpy is not available

@darrassi1
Copy link
Contributor Author

After you active venv
Try uv pip uninstall keybert sentence-transformers -y

uv pip install keybert sentence-transformers

@darrassi1
Copy link
Contributor Author

Make sure this requirements is satisfied

Version's requirements
  - Python >= 3.10 and < 3.12
  - NodeJs >= 18
  - bun

@ARajgor
Copy link
Collaborator

ARajgor commented Jun 18, 2024

why set the temperature to 1?

@darrassi1
Copy link
Contributor Author

why set the temperature to 1?

When the temperature is 0, the model will always choose the most probable or likely next token based on its training, without any randomness or exploration. This can result in the model getting "stuck" in a loop, generating the same or highly similar text over and over again.

When setting temperature to 1. The validate function have low rate of fail ,
Top P and temperature need to be changed in the UI by the user , and he will have more control on the generated response output.

@darrassi1 darrassi1 marked this pull request as draft June 18, 2024 16:39
@darrassi1 darrassi1 marked this pull request as ready for review June 18, 2024 16:54
@darrassi1
Copy link
Contributor Author

Now , the temperature is set back to 0 , and I resolved the conflict

@MahmoudZaki1
Copy link

MahmoudZaki1 commented Jun 19, 2024

Now , the temperature is set back to 0 , and I resolved the conflict

Hello Younes @darrassi1 still even after updating all changes and temp set to 0 , still getting same error

RuntimeError: Numpy is not available

i think issue here with versions as numpy version i have is site-packages (2.0.0)

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

another suggestion might be multiple python versions installed in my case i found that i have 2 versions

(venv) (base) PS C:\Users\Mahmoud\Desktop\Devika\devika> pip --version
pip 24.0 from C:\Users\Mahmoud\Desktop\Devika\devika\venv\Lib\site-packages\pip (python 3.11)
(venv) (base) PS C:\Users\Mahmoud\Desktop\Devika\devika> python --version
Python 3.11.7

i might also need to uninstall one of them and when i do install and run the environments has to be all in one place

if someone had same issue and fixed by any of above, please confirm

@AVcorp
Copy link

AVcorp commented Jun 19, 2024

RuntimeError: Numpy is not available

A module that was compiled using NumPy 1.x cannot be run in NumPy 2.0.0 as it may crash. To support both 1.x and 2.x versions of NumPy, modules must be compiled with NumPy 2.0. Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to downgrade to 'numpy<2' or try to upgrade the affected module. We expect that some modules will need time to support NumPy 2.

another suggestion might be multiple python versions installed in my case i found that i have 2 versions

(venv) (base) PS C:\Users\Mahmoud\Desktop\Devika\devika> pip --version pip 24.0 from C:\Users\Mahmoud\Desktop\Devika\devika\venv\Lib\site-packages\pip (python 3.11) (venv) (base) PS C:\Users\Mahmoud\Desktop\Devika\devika> python --version Python 3.11.7

I'm not familliar with Devika, but it seems you're running it with embedded python. In that case, you need to run cmd from folder 'C:\Users\Mahmoud\Desktop\Devika\devika' and input command 'pip install numpy==1.26.4', which will replace your 2.0.0 version.
If executing command won't help, you can locate numpy folders in 'C:\Users\Mahmoud\Desktop\Devika\devika\venv\Lib\site-packages', delete them and copy correct version from different place, like prior Devika installation, or you can use system-installed python to install numpy with command, and take folders from 'C:\Users\Mahmoud\AppData\Local\Programs\Python\Python310\Lib\site-packages'. That path may vary depending on python version or if you use conda. Of course it's preferable to use the same python version as Devika - 3.11.7.
Or you can just wait until it will be coded into repo to install 1.24.6 and not the latest version, and try fresh install then.

@MahmoudZaki1
Copy link

pip install numpy==1.26.4

Thanks a lot AV :) it actually worked after installing older version and indeed it uninstalled the newer

@werruww
Copy link

werruww commented Jun 29, 2024

Requirement already satisfied: keybert in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (0.8.5)
Requirement already satisfied: sentence-transformers in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (3.0.1)
Requirement already satisfied: transformers in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (4.42.3)
Requirement already satisfied: numpy>=1.18.5 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from keybert) (1.26.4)
Requirement already satisfied: rich>=10.4.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from keybert) (13.7.1)
Requirement already satisfied: scikit-learn>=0.22.2 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from keybert) (1.5.0)
Requirement already satisfied: tqdm in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from sentence-transformers) (4.66.4)
Requirement already satisfied: torch>=1.11.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from sentence-transformers) (2.3.1)
Requirement already satisfied: scipy in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from sentence-transformers) (1.14.0)
Requirement already satisfied: huggingface-hub>=0.15.1 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from sentence-transformers) (0.23.4)
Requirement already satisfied: Pillow in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from sentence-transformers) (10.3.0)
Requirement already satisfied: filelock in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (3.15.4)
Requirement already satisfied: packaging>=20.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (24.1)
Requirement already satisfied: pyyaml>=5.1 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (6.0.1)
Requirement already satisfied: regex!=2019.12.17 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (2024.5.15)
Requirement already satisfied: requests in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (2.32.3)
Requirement already satisfied: safetensors>=0.4.1 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (0.4.3)
Requirement already satisfied: tokenizers<0.20,>=0.19 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from transformers) (0.19.1)
Requirement already satisfied: fsspec>=2023.5.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (2024.6.1)
Requirement already satisfied: typing-extensions>=3.7.4.3 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from huggingface-hub>=0.15.1->sentence-transformers) (4.12.2)
Requirement already satisfied: markdown-it-py>=2.2.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from rich>=10.4.0->keybert) (3.0.0)
Requirement already satisfied: pygments<3.0.0,>=2.13.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from rich>=10.4.0->keybert) (2.18.0)
Requirement already satisfied: joblib>=1.2.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from scikit-learn>=0.22.2->keybert) (1.4.2)
Requirement already satisfied: threadpoolctl>=3.1.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from scikit-learn>=0.22.2->keybert) (3.5.0)
Requirement already satisfied: sympy in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.11.0->sentence-transformers) (1.12.1)
Requirement already satisfied: networkx in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.11.0->sentence-transformers) (3.3)
Requirement already satisfied: jinja2 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.11.0->sentence-transformers) (3.1.4)
Requirement already satisfied: mkl<=2021.4.0,>=2021.1.1 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from torch>=1.11.0->sentence-transformers) (2021.4.0)
Requirement already satisfied: colorama in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from tqdm->sentence-transformers) (0.4.6)
Requirement already satisfied: charset-normalizer<4,>=2 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from requests->transformers) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from requests->transformers) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from requests->transformers) (2.2.2)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from requests->transformers) (2024.6.2)
Requirement already satisfied: mdurl~=0.1 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from markdown-it-py>=2.2.0->rich>=10.4.0->keybert) (0.1.2)
Requirement already satisfied: intel-openmp==2021.* in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from mkl<=2021.4.0,>=2021.1.1->torch>=1.11.0->sentence-transformers) (2021.4.0)
Requirement already satisfied: tbb==2021.* in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from mkl<=2021.4.0,>=2021.1.1->torch>=1.11.0->sentence-transformers) (2021.13.0)
Requirement already satisfied: MarkupSafe>=2.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from jinja2->torch>=1.11.0->sentence-transformers) (2.1.5)
Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (from sympy->torch>=1.11.0->sentence-transformers) (1.3.0)

(devika) C:\Users\m\Desktop\111\devika>python C:\Users\m\Desktop\111\devika\devika.py
24.06.29 15:15:24: root: INFO : Initializing Devika...
24.06.29 15:15:24: root: INFO : checking configurations...
24.06.29 15:15:24: root: INFO : Initializing Prerequisites Jobs...

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.0 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last): File "C:\Users\m\Desktop\111\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\111\devika\src\init.py", line 27, in init_devika
from src.bert.sentence import SentenceBert
File "C:\Users\m\Desktop\111\devika\src\bert\sentence.py", line 1, in
from keybert import KeyBERT
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\keybert_init_.py", line 3, in
from keybert.llm import KeyLLM
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\keybert_llm.py", line 4, in
from sentence_transformers import util
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\sentence_transformers_init
.py", line 15, in
from sentence_transformers.trainer import SentenceTransformerTrainer
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\sentence_transformers\trainer.py", line 10, in
from transformers import EvalPrediction, PreTrainedTokenizerBase, Trainer, TrainerCallback
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1525, in getattr
module = self._get_module(self._class_to_module[name])
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\transformers\utils\import_utils.py", line 1535, in get_module
return importlib.import_module("." + module_name, self.name)
File "C:\Users\m\AppData\Local\Programs\Python\Python311\Lib\importlib_init
.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\transformers\trainer.py", line 71, in
from .optimization import Adafactor, get_scheduler
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\transformers\optimization.py", line 27, in
from .trainer_pt_utils import LayerWiseDummyOptimizer, LayerWiseDummyScheduler
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py", line 235, in
device: Optional[torch.device] = torch.device("cuda"),
C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\transformers\trainer_pt_utils.py:235: UserWarning: Failed to initialize NumPy: _ARRAY_API not found (Triggered internally at ..\torch\csrc\utils\tensor_numpy.cpp:84.)
device: Optional[torch.device] = torch.device("cuda"),
24.06.29 15:15:30: root: INFO : Loading sentence-transformer BERT models...
Traceback (most recent call last):
File "C:\Users\m\Desktop\111\devika\devika.py", line 8, in
init_devika()
File "C:\Users\m\Desktop\111\devika\src\init.py", line 31, in init_devika
SentenceBert(prompt).extract_keywords()
File "C:\Users\m\Desktop\111\devika\src\bert\sentence.py", line 9, in extract_keywords
keywords = self.kw_model.extract_keywords(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\keybert_model.py", line 195, in extract_keywords
doc_embeddings = self.model.embed(docs)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\keybert\backend_sentencetransformers.py", line 67, in embed
embeddings = self.embedding_model.encode(documents, **self.encode_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in encode
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^
RuntimeError: Numpy is not available

(devika) C:\Users\m\Desktop\111\devika>

@werruww
Copy link

werruww commented Jun 29, 2024

all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in
all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings])
^^^^^^^^^^^
RuntimeError: Numpy is not available

(devika) C:\Users\m\Desktop\111\devika>pip install numpy==1.26.4
Requirement already satisfied: numpy==1.26.4 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (1.26.4)

(devika) C:\Users\m\Desktop\111\devika>

@AVcorp
Copy link

AVcorp commented Jun 30, 2024

all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\m\Desktop\111\devika.venv\Lib\site-packages\sentence_transformers\SentenceTransformer.py", line 568, in all_embeddings = np.asarray([emb.numpy() for emb in all_embeddings]) ^^^^^^^^^^^ RuntimeError: Numpy is not available

(devika) C:\Users\m\Desktop\111\devika>pip install numpy==1.26.4 Requirement already satisfied: numpy==1.26.4 in c:\users\m\appdata\local\programs\python\python311\lib\site-packages (1.26.4)

(devika) C:\Users\m\Desktop\111\devika>

You have numpy installed globally in your 'c:\users\m\appdata\local\programs\python\python311\lib\site-packages' folder, while Devika uses virtual environment.
To install numpy correctly, run cmd from devika folder and enter command 'venv\scripts\activate', then run 'pip install numpy==1.26.4', package then should be installed in 'devika\venv\Lib\site-packages' folder. Or you can copy an existing installation of numpy from 'c:\users\m\appdata\local\programs\python\python311\lib\site-packages' to 'devika\venv\Lib\site-packages'.

Copy link

@hardikns hardikns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get this merged?
Any issues?

@@ -26,7 +26,11 @@


app = Flask(__name__)
CORS(app, resources={r"/*": {"origins": ["https://localhost:3000"]}}) # Change the origin to your frontend URL
CORS(app, resources={r"/*": {"origins": # Change the origin to your frontend URL
[
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add port 3001

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants