Unverified 提交 b8a2c47f authored 作者: Glenn Jocher's avatar Glenn Jocher 提交者: GitHub

Update Dockerfile `FROM pytorch/pytorch:latest` (#10902)

* Update Dockerfile `FROM pytorch/pytorch:latest` Signed-off-by: 's avatarGlenn Jocher <glenn.jocher@ultralytics.com> * isort * precommit * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * spelling * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update .pre-commit-config.yaml Signed-off-by: 's avatarGlenn Jocher <glenn.jocher@ultralytics.com> * Cleanup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cleanup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Cleanup * Cleanup * Cleanup * Cleanup --------- Signed-off-by: 's avatarGlenn Jocher <glenn.jocher@ultralytics.com> Co-authored-by: 's avatarpre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
上级 d02ee605
...@@ -4,6 +4,7 @@ ...@@ -4,6 +4,7 @@
default_language_version: default_language_version:
python: python3.8 python: python3.8
exclude: 'docs/'
# Define bot property if installed via https://github.com/marketplace/pre-commit-ci # Define bot property if installed via https://github.com/marketplace/pre-commit-ci
ci: ci:
autofix_prs: true autofix_prs: true
...@@ -28,13 +29,13 @@ repos: ...@@ -28,13 +29,13 @@ repos:
hooks: hooks:
- id: pyupgrade - id: pyupgrade
name: Upgrade code name: Upgrade code
args: [ --py37-plus ] args: [--py37-plus]
- repo: https://github.com/PyCQA/isort # - repo: https://github.com/PyCQA/isort
rev: 5.11.4 # rev: 5.11.4
hooks: # hooks:
- id: isort # - id: isort
name: Sort imports # name: Sort imports
- repo: https://github.com/pre-commit/mirrors-yapf - repo: https://github.com/pre-commit/mirrors-yapf
rev: v0.32.0 rev: v0.32.0
...@@ -50,15 +51,22 @@ repos: ...@@ -50,15 +51,22 @@ repos:
additional_dependencies: additional_dependencies:
- mdformat-gfm - mdformat-gfm
- mdformat-black - mdformat-black
exclude: "README.md|README.zh-CN.md" # exclude: "README.md|README.zh-CN.md|CONTRIBUTING.md"
- repo: https://github.com/asottile/yesqa
rev: v1.4.0
hooks:
- id: yesqa
- repo: https://github.com/PyCQA/flake8 - repo: https://github.com/PyCQA/flake8
rev: 6.0.0 rev: 5.0.4
hooks: hooks:
- id: flake8 - id: flake8
name: PEP8 name: PEP8
#- repo: https://github.com/codespell-project/codespell
# rev: v2.2.2
# hooks:
# - id: codespell
# args:
# - --ignore-words-list=crate,nd
#- repo: https://github.com/asottile/yesqa
# rev: v1.4.0
# hooks:
# - id: yesqa
差异被折叠。
差异被折叠。
...@@ -78,7 +78,7 @@ ...@@ -78,7 +78,7 @@
"source": [ "source": [
"# 1. Predict\n", "# 1. Predict\n",
"\n", "\n",
"`classify/predict.py` runs YOLOv5 Classifcation inference on a variety of sources, downloading models automatically from the [latest YOLOv5 release](https://github.com/ultralytics/yolov5/releases), and saving results to `runs/predict-cls`. Example inference sources are:\n", "`classify/predict.py` runs YOLOv5 Classification inference on a variety of sources, downloading models automatically from the [latest YOLOv5 release](https://github.com/ultralytics/yolov5/releases), and saving results to `runs/predict-cls`. Example inference sources are:\n",
"\n", "\n",
"```shell\n", "```shell\n",
"python classify/predict.py --source 0 # webcam\n", "python classify/predict.py --source 0 # webcam\n",
......
...@@ -128,9 +128,9 @@ def run( ...@@ -128,9 +128,9 @@ def run(
LOGGER.info(f"{'Class':>24}{'Images':>12}{'top1_acc':>12}{'top5_acc':>12}") LOGGER.info(f"{'Class':>24}{'Images':>12}{'top1_acc':>12}{'top5_acc':>12}")
LOGGER.info(f"{'all':>24}{targets.shape[0]:>12}{top1:>12.3g}{top5:>12.3g}") LOGGER.info(f"{'all':>24}{targets.shape[0]:>12}{top1:>12.3g}{top5:>12.3g}")
for i, c in model.names.items(): for i, c in model.names.items():
aci = acc[targets == i] acc_i = acc[targets == i]
top1i, top5i = aci.mean(0).tolist() top1i, top5i = acc_i.mean(0).tolist()
LOGGER.info(f"{c:>24}{aci.shape[0]:>12}{top1i:>12.3g}{top5i:>12.3g}") LOGGER.info(f"{c:>24}{acc_i.shape[0]:>12}{top1i:>12.3g}{top5i:>12.3g}")
# Print results # Print results
t = tuple(x.t / len(dataloader.dataset.samples) * 1E3 for x in dt) # speeds per image t = tuple(x.t / len(dataloader.dataset.samples) * 1E3 for x in dt) # speeds per image
......
...@@ -3,23 +3,33 @@ ...@@ -3,23 +3,33 @@
# Image is CUDA-optimized for YOLOv5 single/multi-GPU training and inference # Image is CUDA-optimized for YOLOv5 single/multi-GPU training and inference
# Start FROM NVIDIA PyTorch image https://ngc.nvidia.com/catalog/containers/nvidia:pytorch # Start FROM NVIDIA PyTorch image https://ngc.nvidia.com/catalog/containers/nvidia:pytorch
FROM nvcr.io/nvidia/pytorch:22.12-py3 # FROM docker.io/pytorch/pytorch:latest
RUN rm -rf /opt/pytorch # remove 1.2GB dir FROM pytorch/pytorch:latest
# Downloads to user config dir # Downloads to user config dir
ADD https://ultralytics.com/assets/Arial.ttf https://ultralytics.com/assets/Arial.Unicode.ttf /root/.config/Ultralytics/ ADD https://ultralytics.com/assets/Arial.ttf https://ultralytics.com/assets/Arial.Unicode.ttf /root/.config/Ultralytics/
# Install linux packages # Install linux packages
RUN apt update && apt install --no-install-recommends -y zip htop screen libgl1-mesa-glx ENV DEBIAN_FRONTEND noninteractive
RUN apt update
RUN TZ=Etc/UTC apt install -y tzdata
RUN apt install --no-install-recommends -y gcc git zip curl htop libgl1-mesa-glx libglib2.0-0 libpython3-dev gnupg
# RUN alias python=python3
# Install pip packages (uninstall torch nightly in favor of stable) # Create working directory
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
# Copy contents
# COPY . /usr/src/app (issues as not a .git directory)
RUN git clone https://github.com/ultralytics/yolov5 /usr/src/app
# Install pip packages
COPY requirements.txt . COPY requirements.txt .
RUN python -m pip install --upgrade pip wheel RUN python3 -m pip install --upgrade pip wheel
RUN pip uninstall -y Pillow torchtext torch torchvision RUN pip install --no-cache -r requirements.txt albumentations comet gsutil notebook \
RUN pip install --no-cache -U pycocotools # install --upgrade coremltools onnx onnx-simplifier onnxruntime openvino-dev>=2022.3
RUN pip install --no-cache -r requirements.txt albumentations comet gsutil notebook 'opencv-python<4.6.0.66' \ # tensorflow tensorflowjs \
Pillow>=9.1.0 ultralytics \
--extra-index-url https://download.pytorch.org/whl/cu113
# Create working directory # Create working directory
RUN mkdir -p /usr/src/app RUN mkdir -p /usr/src/app
...@@ -32,6 +42,9 @@ RUN git clone https://github.com/ultralytics/yolov5 /usr/src/app ...@@ -32,6 +42,9 @@ RUN git clone https://github.com/ultralytics/yolov5 /usr/src/app
# Set environment variables # Set environment variables
ENV OMP_NUM_THREADS=1 ENV OMP_NUM_THREADS=1
# Cleanup
ENV DEBIAN_FRONTEND teletype
# Usage Examples ------------------------------------------------------------------------------------------------------- # Usage Examples -------------------------------------------------------------------------------------------------------
......
...@@ -84,10 +84,6 @@ class Loggers(): ...@@ -84,10 +84,6 @@ class Loggers():
self.csv = True # always log to csv self.csv = True # always log to csv
# Messages # Messages
# if not wandb:
# prefix = colorstr('Weights & Biases: ')
# s = f"{prefix}run 'pip install wandb' to automatically track and visualize YOLOv5 🚀 runs in Weights & Biases"
# self.logger.info(s)
if not clearml: if not clearml:
prefix = colorstr('ClearML: ') prefix = colorstr('ClearML: ')
s = f"{prefix}run 'pip install clearml' to automatically track, visualize and remotely train YOLOv5 🚀 in ClearML" s = f"{prefix}run 'pip install clearml' to automatically track, visualize and remotely train YOLOv5 🚀 in ClearML"
...@@ -105,14 +101,8 @@ class Loggers(): ...@@ -105,14 +101,8 @@ class Loggers():
# W&B # W&B
if wandb and 'wandb' in self.include: if wandb and 'wandb' in self.include:
wandb_artifact_resume = isinstance(self.opt.resume, str) and self.opt.resume.startswith('wandb-artifact://')
run_id = torch.load(self.weights).get('wandb_id') if self.opt.resume and not wandb_artifact_resume else None
self.opt.hyp = self.hyp # add hyperparameters self.opt.hyp = self.hyp # add hyperparameters
self.wandb = WandbLogger(self.opt, run_id) self.wandb = WandbLogger(self.opt)
# temp warn. because nested artifacts not supported after 0.12.10
# if pkg.parse_version(wandb.__version__) >= pkg.parse_version('0.12.11'):
# s = "YOLOv5 temporarily requires wandb version 0.12.10 or below. Some features may not work as expected."
# self.logger.warning(s)
else: else:
self.wandb = None self.wandb = None
...@@ -175,7 +165,7 @@ class Loggers(): ...@@ -175,7 +165,7 @@ class Loggers():
self.comet_logger.on_pretrain_routine_end(paths) self.comet_logger.on_pretrain_routine_end(paths)
def on_train_batch_end(self, model, ni, imgs, targets, paths, vals): def on_train_batch_end(self, model, ni, imgs, targets, paths, vals):
log_dict = dict(zip(self.keys[0:3], vals)) log_dict = dict(zip(self.keys[:3], vals))
# Callback runs on train batch end # Callback runs on train batch end
# ni: number integrated batches (since train start) # ni: number integrated batches (since train start)
if self.plots: if self.plots:
...@@ -221,10 +211,10 @@ class Loggers(): ...@@ -221,10 +211,10 @@ class Loggers():
# Callback runs on val end # Callback runs on val end
if self.wandb or self.clearml: if self.wandb or self.clearml:
files = sorted(self.save_dir.glob('val*.jpg')) files = sorted(self.save_dir.glob('val*.jpg'))
if self.wandb: if self.wandb:
self.wandb.log({"Validation": [wandb.Image(str(f), caption=f.name) for f in files]}) self.wandb.log({"Validation": [wandb.Image(str(f), caption=f.name) for f in files]})
if self.clearml: if self.clearml:
self.clearml.log_debug_samples(files, title='Validation') self.clearml.log_debug_samples(files, title='Validation')
if self.comet_logger: if self.comet_logger:
self.comet_logger.on_val_end(nt, tp, fp, p, r, f1, ap, ap50, ap_class, confusion_matrix) self.comet_logger.on_val_end(nt, tp, fp, p, r, f1, ap, ap50, ap_class, confusion_matrix)
...@@ -253,7 +243,7 @@ class Loggers(): ...@@ -253,7 +243,7 @@ class Loggers():
for i, name in enumerate(self.best_keys): for i, name in enumerate(self.best_keys):
self.wandb.wandb_run.summary[name] = best_results[i] # log best results in the summary self.wandb.wandb_run.summary[name] = best_results[i] # log best results in the summary
self.wandb.log(x) self.wandb.log(x)
self.wandb.end_epoch(best_result=best_fitness == fi) self.wandb.end_epoch()
if self.clearml: if self.clearml:
self.clearml.current_epoch_logged_images = set() # reset epoch image limit self.clearml.current_epoch_logged_images = set() # reset epoch image limit
......
...@@ -23,7 +23,6 @@ And so much more. It's up to you how many of these tools you want to use, you ca ...@@ -23,7 +23,6 @@ And so much more. It's up to you how many of these tools you want to use, you ca
![ClearML scalars dashboard](https://github.com/thepycoder/clearml_screenshots/raw/main/experiment_manager_with_compare.gif) ![ClearML scalars dashboard](https://github.com/thepycoder/clearml_screenshots/raw/main/experiment_manager_with_compare.gif)
<br /> <br />
<br /> <br />
...@@ -35,15 +34,15 @@ Either sign up for free to the [ClearML Hosted Service](https://cutt.ly/yolov5-t ...@@ -35,15 +34,15 @@ Either sign up for free to the [ClearML Hosted Service](https://cutt.ly/yolov5-t
1. Install the `clearml` python package: 1. Install the `clearml` python package:
```bash ```bash
pip install clearml pip install clearml
``` ```
1. Connect the ClearML SDK to the server by [creating credentials](https://app.clear.ml/settings/workspace-configuration) (go right top to Settings -> Workspace -> Create new credentials), then execute the command below and follow the instructions: 1. Connect the ClearML SDK to the server by [creating credentials](https://app.clear.ml/settings/workspace-configuration) (go right top to Settings -> Workspace -> Create new credentials), then execute the command below and follow the instructions:
```bash ```bash
clearml-init clearml-init
``` ```
That's it! You're done 😎 That's it! You're done 😎
...@@ -60,18 +59,20 @@ pip install clearml>=1.2.0 ...@@ -60,18 +59,20 @@ pip install clearml>=1.2.0
This will enable integration with the YOLOv5 training script. Every training run from now on, will be captured and stored by the ClearML experiment manager. This will enable integration with the YOLOv5 training script. Every training run from now on, will be captured and stored by the ClearML experiment manager.
If you want to change the `project_name` or `task_name`, use the `--project` and `--name` arguments of the `train.py` script, by default the project will be called `YOLOv5` and the task `Training`. If you want to change the `project_name` or `task_name`, use the `--project` and `--name` arguments of the `train.py` script, by default the project will be called `YOLOv5` and the task `Training`.
PLEASE NOTE: ClearML uses `/` as a delimter for subprojects, so be careful when using `/` in your project name! PLEASE NOTE: ClearML uses `/` as a delimiter for subprojects, so be careful when using `/` in your project name!
```bash ```bash
python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache python train.py --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache
``` ```
or with custom project and task name: or with custom project and task name:
```bash ```bash
python train.py --project my_project --name my_training --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache python train.py --project my_project --name my_training --img 640 --batch 16 --epochs 3 --data coco128.yaml --weights yolov5s.pt --cache
``` ```
This will capture: This will capture:
- Source code + uncommitted changes - Source code + uncommitted changes
- Installed packages - Installed packages
- (Hyper)parameters - (Hyper)parameters
...@@ -94,7 +95,7 @@ There even more we can do with all of this information, like hyperparameter opti ...@@ -94,7 +95,7 @@ There even more we can do with all of this information, like hyperparameter opti
## 🔗 Dataset Version Management ## 🔗 Dataset Version Management
Versioning your data separately from your code is generally a good idea and makes it easy to aqcuire the latest version too. This repository supports supplying a dataset version ID and it will make sure to get the data if it's not there yet. Next to that, this workflow also saves the used dataset ID as part of the task parameters, so you will always know for sure which data was used in which experiment! Versioning your data separately from your code is generally a good idea and makes it easy to acquire the latest version too. This repository supports supplying a dataset version ID, and it will make sure to get the data if it's not there yet. Next to that, this workflow also saves the used dataset ID as part of the task parameters, so you will always know for sure which data was used in which experiment!
![ClearML Dataset Interface](https://github.com/thepycoder/clearml_screenshots/raw/main/clearml_data.gif) ![ClearML Dataset Interface](https://github.com/thepycoder/clearml_screenshots/raw/main/clearml_data.gif)
...@@ -112,6 +113,7 @@ The YOLOv5 repository supports a number of different datasets by using yaml file ...@@ -112,6 +113,7 @@ The YOLOv5 repository supports a number of different datasets by using yaml file
|_ LICENSE |_ LICENSE
|_ README.txt |_ README.txt
``` ```
But this can be any dataset you wish. Feel free to use your own, as long as you keep to this folder structure. But this can be any dataset you wish. Feel free to use your own, as long as you keep to this folder structure.
Next, ⚠️**copy the corresponding yaml file to the root of the dataset folder**⚠️. This yaml files contains the information ClearML will need to properly use the dataset. You can make this yourself too, of course, just follow the structure of the example yamls. Next, ⚠️**copy the corresponding yaml file to the root of the dataset folder**⚠️. This yaml files contains the information ClearML will need to properly use the dataset. You can make this yourself too, of course, just follow the structure of the example yamls.
...@@ -132,13 +134,15 @@ Basically we need the following keys: `path`, `train`, `test`, `val`, `nc`, `nam ...@@ -132,13 +134,15 @@ Basically we need the following keys: `path`, `train`, `test`, `val`, `nc`, `nam
### Upload Your Dataset ### Upload Your Dataset
To get this dataset into ClearML as a versionned dataset, go to the dataset root folder and run the following command: To get this dataset into ClearML as a versioned dataset, go to the dataset root folder and run the following command:
```bash ```bash
cd coco128 cd coco128
clearml-data sync --project YOLOv5 --name coco128 --folder . clearml-data sync --project YOLOv5 --name coco128 --folder .
``` ```
The command `clearml-data sync` is actually a shorthand command. You could also run these commands one after the other: The command `clearml-data sync` is actually a shorthand command. You could also run these commands one after the other:
```bash ```bash
# Optionally add --parent <parent_dataset_id> if you want to base # Optionally add --parent <parent_dataset_id> if you want to base
# this version on another dataset version, so no duplicate files are uploaded! # this version on another dataset version, so no duplicate files are uploaded!
...@@ -177,7 +181,7 @@ python utils/loggers/clearml/hpo.py ...@@ -177,7 +181,7 @@ python utils/loggers/clearml/hpo.py
## 🤯 Remote Execution (advanced) ## 🤯 Remote Execution (advanced)
Running HPO locally is really handy, but what if we want to run our experiments on a remote machine instead? Maybe you have access to a very powerful GPU machine on-site or you have some budget to use cloud GPUs. Running HPO locally is really handy, but what if we want to run our experiments on a remote machine instead? Maybe you have access to a very powerful GPU machine on-site, or you have some budget to use cloud GPUs.
This is where the ClearML Agent comes into play. Check out what the agent can do here: This is where the ClearML Agent comes into play. Check out what the agent can do here:
- [YouTube video](https://youtu.be/MX3BrXnaULs) - [YouTube video](https://youtu.be/MX3BrXnaULs)
...@@ -186,6 +190,7 @@ This is where the ClearML Agent comes into play. Check out what the agent can do ...@@ -186,6 +190,7 @@ This is where the ClearML Agent comes into play. Check out what the agent can do
In short: every experiment tracked by the experiment manager contains enough information to reproduce it on a different machine (installed packages, uncommitted changes etc.). So a ClearML agent does just that: it listens to a queue for incoming tasks and when it finds one, it recreates the environment and runs it while still reporting scalars, plots etc. to the experiment manager. In short: every experiment tracked by the experiment manager contains enough information to reproduce it on a different machine (installed packages, uncommitted changes etc.). So a ClearML agent does just that: it listens to a queue for incoming tasks and when it finds one, it recreates the environment and runs it while still reporting scalars, plots etc. to the experiment manager.
You can turn any machine (a cloud VM, a local GPU machine, your own laptop ... ) into a ClearML agent by simply running: You can turn any machine (a cloud VM, a local GPU machine, your own laptop ... ) into a ClearML agent by simply running:
```bash ```bash
clearml-agent daemon --queue <queues_to_listen_to> [--docker] clearml-agent daemon --queue <queues_to_listen_to> [--docker]
``` ```
...@@ -194,11 +199,11 @@ clearml-agent daemon --queue <queues_to_listen_to> [--docker] ...@@ -194,11 +199,11 @@ clearml-agent daemon --queue <queues_to_listen_to> [--docker]
With our agent running, we can give it some work. Remember from the HPO section that we can clone a task and edit the hyperparameters? We can do that from the interface too! With our agent running, we can give it some work. Remember from the HPO section that we can clone a task and edit the hyperparameters? We can do that from the interface too!
🪄 Clone the experiment by right clicking it 🪄 Clone the experiment by right-clicking it
🎯 Edit the hyperparameters to what you wish them to be 🎯 Edit the hyperparameters to what you wish them to be
⏳ Enqueue the task to any of the queues by right clicking it ⏳ Enqueue the task to any of the queues by right-clicking it
![Enqueue a task from the UI](https://github.com/thepycoder/clearml_screenshots/raw/main/enqueue.gif) ![Enqueue a task from the UI](https://github.com/thepycoder/clearml_screenshots/raw/main/enqueue.gif)
...@@ -206,7 +211,8 @@ With our agent running, we can give it some work. Remember from the HPO section ...@@ -206,7 +211,8 @@ With our agent running, we can give it some work. Remember from the HPO section
Now you can clone a task like we explained above, or simply mark your current script by adding `task.execute_remotely()` and on execution it will be put into a queue, for the agent to start working on! Now you can clone a task like we explained above, or simply mark your current script by adding `task.execute_remotely()` and on execution it will be put into a queue, for the agent to start working on!
To run the YOLOv5 training script remotely, all you have to do is add this line to the training.py script after the clearml logger has been instatiated: To run the YOLOv5 training script remotely, all you have to do is add this line to the training.py script after the clearml logger has been instantiated:
```python ```python
# ... # ...
# Loggers # Loggers
...@@ -214,16 +220,17 @@ data_dict = None ...@@ -214,16 +220,17 @@ data_dict = None
if RANK in {-1, 0}: if RANK in {-1, 0}:
loggers = Loggers(save_dir, weights, opt, hyp, LOGGER) # loggers instance loggers = Loggers(save_dir, weights, opt, hyp, LOGGER) # loggers instance
if loggers.clearml: if loggers.clearml:
loggers.clearml.task.execute_remotely(queue='my_queue') # <------ ADD THIS LINE loggers.clearml.task.execute_remotely(queue="my_queue") # <------ ADD THIS LINE
# Data_dict is either None is user did not choose for ClearML dataset or is filled in by ClearML # Data_dict is either None is user did not choose for ClearML dataset or is filled in by ClearML
data_dict = loggers.clearml.data_dict data_dict = loggers.clearml.data_dict
# ... # ...
``` ```
When running the training script after this change, python will run the script up until that line, after which it will package the code and send it to the queue instead! When running the training script after this change, python will run the script up until that line, after which it will package the code and send it to the queue instead!
### Autoscaling workers ### Autoscaling workers
ClearML comes with autoscalers too! This tool will automatically spin up new remote machines in the cloud of your choice (AWS, GCP, Azure) and turn them into ClearML agents for you whenever there are experiments detected in the queue. Once the tasks are processed, the autoscaler will automatically shut down the remote machines and you stop paying! ClearML comes with autoscalers too! This tool will automatically spin up new remote machines in the cloud of your choice (AWS, GCP, Azure) and turn them into ClearML agents for you whenever there are experiments detected in the queue. Once the tasks are processed, the autoscaler will automatically shut down the remote machines, and you stop paying!
Check out the autoscalers getting started video below. Check out the autoscalers getting started video below.
......
...@@ -23,7 +23,7 @@ pip install comet_ml ...@@ -23,7 +23,7 @@ pip install comet_ml
There are two ways to configure Comet with YOLOv5. There are two ways to configure Comet with YOLOv5.
You can either set your credentials through enviroment variables You can either set your credentials through environment variables
**Environment Variables** **Environment Variables**
...@@ -49,11 +49,12 @@ project_name=<Your Comet Project Name> # This will default to 'yolov5' ...@@ -49,11 +49,12 @@ project_name=<Your Comet Project Name> # This will default to 'yolov5'
python train.py --img 640 --batch 16 --epochs 5 --data coco128.yaml --weights yolov5s.pt python train.py --img 640 --batch 16 --epochs 5 --data coco128.yaml --weights yolov5s.pt
``` ```
That's it! Comet will automatically log your hyperparameters, command line arguments, training and valiation metrics. You can visualize and analyze your runs in the Comet UI That's it! Comet will automatically log your hyperparameters, command line arguments, training and validation metrics. You can visualize and analyze your runs in the Comet UI
<img width="1920" alt="yolo-ui" src="https://user-images.githubusercontent.com/26833433/202851203-164e94e1-2238-46dd-91f8-de020e9d6b41.png"> <img width="1920" alt="yolo-ui" src="https://user-images.githubusercontent.com/26833433/202851203-164e94e1-2238-46dd-91f8-de020e9d6b41.png">
# Try out an Example! # Try out an Example!
Check out an example of a [completed run here](https://www.comet.com/examples/comet-example-yolov5/a0e29e0e9b984e4a822db2a62d0cb357?experiment-tab=chart&showOutliers=true&smoothing=0&transformY=smoothing&xAxis=step&utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github) Check out an example of a [completed run here](https://www.comet.com/examples/comet-example-yolov5/a0e29e0e9b984e4a822db2a62d0cb357?experiment-tab=chart&showOutliers=true&smoothing=0&transformY=smoothing&xAxis=step&utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github)
Or better yet, try it out yourself in this Colab Notebook Or better yet, try it out yourself in this Colab Notebook
...@@ -65,6 +66,7 @@ Or better yet, try it out yourself in this Colab Notebook ...@@ -65,6 +66,7 @@ Or better yet, try it out yourself in this Colab Notebook
By default, Comet will log the following items By default, Comet will log the following items
## Metrics ## Metrics
- Box Loss, Object Loss, Classification Loss for the training and validation data - Box Loss, Object Loss, Classification Loss for the training and validation data
- mAP_0.5, mAP_0.5:0.95 metrics for the validation data. - mAP_0.5, mAP_0.5:0.95 metrics for the validation data.
- Precision and Recall for the validation data - Precision and Recall for the validation data
...@@ -121,7 +123,6 @@ You can control the frequency of logged predictions and the associated images by ...@@ -121,7 +123,6 @@ You can control the frequency of logged predictions and the associated images by
Here is an [example project using the Panel](https://www.comet.com/examples/comet-example-yolov5?shareable=YcwMiJaZSXfcEXpGOHDD12vA1&utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github) Here is an [example project using the Panel](https://www.comet.com/examples/comet-example-yolov5?shareable=YcwMiJaZSXfcEXpGOHDD12vA1&utm_source=yolov5&utm_medium=partner&utm_campaign=partner_yolov5_2022&utm_content=github)
```shell ```shell
python train.py \ python train.py \
--img 640 \ --img 640 \
...@@ -192,6 +193,7 @@ If you would like to use a dataset from Comet Artifacts, set the `path` variable ...@@ -192,6 +193,7 @@ If you would like to use a dataset from Comet Artifacts, set the `path` variable
# contents of artifact.yaml file # contents of artifact.yaml file
path: "comet://<workspace name>/<artifact name>:<artifact version or alias>" path: "comet://<workspace name>/<artifact name>:<artifact version or alias>"
``` ```
Then pass this file to your training script in the following way Then pass this file to your training script in the following way
```shell ```shell
...@@ -221,7 +223,7 @@ python train.py \ ...@@ -221,7 +223,7 @@ python train.py \
## Hyperparameter Search with the Comet Optimizer ## Hyperparameter Search with the Comet Optimizer
YOLOv5 is also integrated with Comet's Optimizer, making is simple to visualie hyperparameter sweeps in the Comet UI. YOLOv5 is also integrated with Comet's Optimizer, making is simple to visualize hyperparameter sweeps in the Comet UI.
### Configuring an Optimizer Sweep ### Configuring an Optimizer Sweep
......
差异被折叠。
import argparse
from wandb_utils import WandbLogger
from utils.general import LOGGER
WANDB_ARTIFACT_PREFIX = 'wandb-artifact://'
def create_dataset_artifact(opt):
logger = WandbLogger(opt, None, job_type='Dataset Creation') # TODO: return value unused
if not logger.wandb:
LOGGER.info("install wandb using `pip install wandb` to log the dataset")
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--data', type=str, default='data/coco128.yaml', help='data.yaml path')
parser.add_argument('--single-cls', action='store_true', help='train as single-class dataset')
parser.add_argument('--project', type=str, default='YOLOv5', help='name of W&B Project')
parser.add_argument('--entity', default=None, help='W&B entity')
parser.add_argument('--name', type=str, default='log dataset', help='name of W&B run')
opt = parser.parse_args()
opt.resume = False # Explicitly disallow resume check for dataset upload job
create_dataset_artifact(opt)
import sys
from pathlib import Path
import wandb
FILE = Path(__file__).resolve()
ROOT = FILE.parents[3] # YOLOv5 root directory
if str(ROOT) not in sys.path:
sys.path.append(str(ROOT)) # add ROOT to PATH
from train import parse_opt, train
from utils.callbacks import Callbacks
from utils.general import increment_path
from utils.torch_utils import select_device
def sweep():
wandb.init()
# Get hyp dict from sweep agent. Copy because train() modifies parameters which confused wandb.
hyp_dict = vars(wandb.config).get("_items").copy()
# Workaround: get necessary opt args
opt = parse_opt(known=True)
opt.batch_size = hyp_dict.get("batch_size")
opt.save_dir = str(increment_path(Path(opt.project) / opt.name, exist_ok=opt.exist_ok or opt.evolve))
opt.epochs = hyp_dict.get("epochs")
opt.nosave = True
opt.data = hyp_dict.get("data")
opt.weights = str(opt.weights)
opt.cfg = str(opt.cfg)
opt.data = str(opt.data)
opt.hyp = str(opt.hyp)
opt.project = str(opt.project)
device = select_device(opt.device, batch_size=opt.batch_size)
# train
train(hyp_dict, opt, device, callbacks=Callbacks())
if __name__ == "__main__":
sweep()
# Hyperparameters for training
# To set range-
# Provide min and max values as:
# parameter:
#
# min: scalar
# max: scalar
# OR
#
# Set a specific list of search space-
# parameter:
# values: [scalar1, scalar2, scalar3...]
#
# You can use grid, bayesian and hyperopt search strategy
# For more info on configuring sweeps visit - https://docs.wandb.ai/guides/sweeps/configuration
program: utils/loggers/wandb/sweep.py
method: random
metric:
name: metrics/mAP_0.5
goal: maximize
parameters:
# hyperparameters: set either min, max range or values list
data:
value: "data/coco128.yaml"
batch_size:
values: [64]
epochs:
values: [10]
lr0:
distribution: uniform
min: 1e-5
max: 1e-1
lrf:
distribution: uniform
min: 0.01
max: 1.0
momentum:
distribution: uniform
min: 0.6
max: 0.98
weight_decay:
distribution: uniform
min: 0.0
max: 0.001
warmup_epochs:
distribution: uniform
min: 0.0
max: 5.0
warmup_momentum:
distribution: uniform
min: 0.0
max: 0.95
warmup_bias_lr:
distribution: uniform
min: 0.0
max: 0.2
box:
distribution: uniform
min: 0.02
max: 0.2
cls:
distribution: uniform
min: 0.2
max: 4.0
cls_pw:
distribution: uniform
min: 0.5
max: 2.0
obj:
distribution: uniform
min: 0.2
max: 4.0
obj_pw:
distribution: uniform
min: 0.5
max: 2.0
iou_t:
distribution: uniform
min: 0.1
max: 0.7
anchor_t:
distribution: uniform
min: 2.0
max: 8.0
fl_gamma:
distribution: uniform
min: 0.0
max: 4.0
hsv_h:
distribution: uniform
min: 0.0
max: 0.1
hsv_s:
distribution: uniform
min: 0.0
max: 0.9
hsv_v:
distribution: uniform
min: 0.0
max: 0.9
degrees:
distribution: uniform
min: 0.0
max: 45.0
translate:
distribution: uniform
min: 0.0
max: 0.9
scale:
distribution: uniform
min: 0.0
max: 0.9
shear:
distribution: uniform
min: 0.0
max: 10.0
perspective:
distribution: uniform
min: 0.0
max: 0.001
flipud:
distribution: uniform
min: 0.0
max: 1.0
fliplr:
distribution: uniform
min: 0.0
max: 1.0
mosaic:
distribution: uniform
min: 0.0
max: 1.0
mixup:
distribution: uniform
min: 0.0
max: 1.0
copy_paste:
distribution: uniform
min: 0.0
max: 1.0
Markdown 格式
0%
您添加了 0 到此讨论。请谨慎行事。
请先完成此评论的编辑!
注册 或者 后发表评论