Activity
Mon
Wed
Fri
Sun
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
What is this?
Less
More
Memberships
Agent Zero
2.5k members • Free
1 contribution to Agent Zero
4d •
A0 Docekr Ports or LiteLLM Issues?
Hello and Good Afternoon Everyone, Please forgive my wall of text. I will state that I'm not 100% familiar with docker, as I've only used it on my desktop and not in the wild. Also, when discussing technicals on AI products I do believe that "context" is important for transparency. I have installed A0 on my Debian VPS. I followed the instructions found here -> https://www.agent-zero.ai/p/docs/get-started/#installation Docker components would not install as stated. Had to first add docker source and key, then I was able to install. Followed everything else; was able to pull image and start with persistance just fine. For added layer of security I have installed Tailscale - private vpn from my home -> A0 host (for free). I was able to get to the WebUI just fine by exposing port 50080 to tailscale (tailscale serve) which gives me an https URL only available on the tailscale network. Now for my nftables firewall - I am only allowing all connections through tailscale inteface; everything else gets denied by policy. I can ssh in, ping out, etc. Secure two-way communitcation established at the host/network level. After I installed A0 and able to login to the WebUI - I went to settings as instructed to set API keys. I found the area for Embedding Model. I set my model provider (openrouter), set my model (as suggested in documentation openai/text-embedding-3-small), set my API key, and set the embedding model base URL. Click "Save". I go to open "new chat' and type 'hello' -- starts "Initializing memory database...". It continues to do this for some time. I look in the logs (docker logs --tail 100 -f agent-zero) and I see this: WARNING [huggingface_hub.utils._http] Retrying in 8s [Retry 5/5]. '(MaxRetryError('HTTPSConnectionPool(host=\'huggingface.co\', port=443): Max retries exceeded with url: /sentence-transformers/all-MiniLM-L6-v2/resolve/main/tokenizer_config.json (Caused by NameResolutionError("HTTPSConnection(host=\'huggingface.co\', port=443): Failed to resolve \'huggingface.co\' ([Errno -3] Temporary failure in name resolution)"))'), '(Request ID: f16d5bf3-43e9-493a-803a-d30b16d0f987)')' thrown while requesting HEAD https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2/resolve/main/tokenizer_config.json
0 likes • 4d
Additional information from log files --about LiteLLM User message: > hello Initializing VectorDB... Critical error occurred, retrying... This error has occurred: litellm.exceptions.InternalServerError: litellm.InternalServerError: InternalServerError: OpenAIException - Connection error. Traceback (most recent call last): Traceback (most recent call last): File "/opt/venv-a0/lib/python3.12/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions yield File "/opt/venv-a0/lib/python3.12/site-packages/httpx/_transports/default.py", line 250, in handle_request resp = self._pool.handle_request(req) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv-a0/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request raise exc from None File "/opt/venv-a0/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request response = connection.handle_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv-a0/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 101, in handle_request raise exc File "/opt/venv-a0/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 78, in handle_request stream = self._connect(request) ^^^^^^^^^^^^^^^^^^^^^^ >>> 6 stack lines skipped <<< File "/opt/venv-a0/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 1107, in make_sync_openai_embedding_request raw_response = openai_client.embeddings.with_raw_response.create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv-a0/lib/python3.12/site-packages/openai/_legacy_response.py", line 364, in wrapped return cast(LegacyAPIResponse[R], func(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv-a0/lib/python3.12/site-packages/openai/resources/embeddings.py", line 132, in create
1 like • 4d
okay, I figured this out after some sleep and research :Docker Ports: I was not letting the docker interface out past nftables. sudo nft add rule inet filter forward iifname "docker0" oifname != "docker0" ct state new accept sudo nft add rule inet filter forward ct state established,related accept Test Immediately (No Restart Needed) docker exec -it agent-zero curl -s https://huggingface.co | head -5 Restart container docker restart agent-zero Make It Persistent nft rules added via command line are lost on reboot. Save them: sudo nft list ruleset | sudo tee /etc/nftables.conf Then verify your system loads it on boot: sudo systemctl enable nftables This lets A0 talk to the outside world; while the WebUI is still only accessable through the tailscale magicdns (https) vpn. :Memory Embedding Model: After setting my embedding model, I was getting this error message: litellm.BadRequestError: Unmapped LLM provider for this endpoint. You passed model=openai/text-embedding-3-small, custom_llm_provider=openrouter. Check supported provider and route: https://docs.litellm.ai/docs/providers Why this is happening Agent Zero tries to be "smart" by automatically prefixing or stripping provider names based on what you select in the dropdown. If you select OpenRouter in the dropdown, it thinks: "Okay, I'll handle the routing." But then it sends openai/text-embedding-3-small to LiteLLM. LiteLLM looks at that and says: "Wait, you told me the provider is OpenRouter, but this model name starts with 'openai'. I don't know how to map that for an embedding call." How to actually make it work If you want to use your OpenRouter credits and not deal with their broken UI logic, do this: 1. Go to Settings -> Embedding Model. 2. Change Provider to: openai (Yes, even though you're using OpenRouter). 3. Model: openai/text-embedding-3-small 4. Base URL: https://openrouter.ai/api/v1 5. API Key: (Your OpenRouter Key)
1-1 of 1
