feat:support download huggingface files from a mirror site

This commit is contained in:
chenxinlong 2024-03-26 16:31:50 +08:00
parent e2f9bcb11d
commit 46722da9f1
2 changed files with 3 additions and 0 deletions

View File

@ -54,6 +54,7 @@ Docker specified environments are there. They are used by 'entrypoint.sh'
|CMDARGS|Arguments for [entry_with_update.py](entry_with_update.py) which is called by [entrypoint.sh](entrypoint.sh)| |CMDARGS|Arguments for [entry_with_update.py](entry_with_update.py) which is called by [entrypoint.sh](entrypoint.sh)|
|config_path|'config.txt' location| |config_path|'config.txt' location|
|config_example_path|'config_modification_tutorial.txt' location| |config_example_path|'config_modification_tutorial.txt' location|
|HF_MIRROR| huggingface mirror site domain|
You can also use the same json key names and values explained in the 'config_modification_tutorial.txt' as the environments. You can also use the same json key names and values explained in the 'config_modification_tutorial.txt' as the environments.
See examples in the [docker-compose.yml](docker-compose.yml) See examples in the [docker-compose.yml](docker-compose.yml)

View File

@ -14,6 +14,8 @@ def load_file_from_url(
Returns the path to the downloaded file. Returns the path to the downloaded file.
""" """
domain = os.environ.get("HF_MIRROR", "https://huggingface.co").rstrip('/')
url = str.replace(url, "https://huggingface.co", domain, 1)
os.makedirs(model_dir, exist_ok=True) os.makedirs(model_dir, exist_ok=True)
if not file_name: if not file_name:
parts = urlparse(url) parts = urlparse(url)