Skip to content
This repository was archived by the owner on Dec 14, 2023. It is now read-only.

Commit eeb5cc4

Browse files
Update README.md
1 parent 9b4c7a8 commit eeb5cc4

File tree

1 file changed

+8
-8
lines changed

1 file changed

+8
-8
lines changed

README.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
***(This is a WIP)***
44

55
### Updates
6-
6+
- **2023-3-29**: Added gradient checkpointing support.
77
- **2023-3-27**: Support for using Scaled Dot Product Attention for Torch 2.0 users.
88

99
## Getting Started
@@ -37,6 +37,13 @@ Recommended to use a RTX 3090, but you should be able to train on GPUs with <= 1
3737

3838
## Usage
3939

40+
### Preprocessing your data
41+
All videos were preprocessed using the script [here](https://github.com/ExponentialML/Video-BLIP2-Preprocessor) using automatic BLIP2 captions. Please follow the instructions there.
42+
43+
If you wish to use a custom dataloader (for instance, a folder of mp4's and captions), you're free to update the dataloader [here](https://github.com/ExponentialML/Text-To-Video-Finetuning/blob/d72e34cfbd91d2a62c07172f9ef079ca5cd651b2/utils/dataset.py#L83).
44+
45+
Feel free to share your dataloaders for others to use! It would be much appreciated.
46+
4047
### Finetuning using a training JSON
4148
```python
4249
python train.py --config ./configs/my_config.yaml
@@ -61,13 +68,6 @@ When finetuning on a single video, you should see results in half as many steps.
6168

6269
After training, you should see your results in your output directory. By default, it should be placed at the script root under `./outputs/train_<date>`
6370

64-
## Preprocessing your data
65-
All videos were preprocessed using the script [here](https://github.com/ExponentialML/Video-BLIP2-Preprocessor) using automatic BLIP2 captions. Please follow the instructions there.
66-
67-
If you wish to use a custom dataloader (for instance, a folder of mp4's and captions), you're free to update the dataloader [here](https://github.com/ExponentialML/Text-To-Video-Finetuning/blob/d72e34cfbd91d2a62c07172f9ef079ca5cd651b2/utils/dataset.py#L83).
68-
69-
Feel free to share your dataloaders for others to use! It would be much appreciated.
70-
7171
## Configuration
7272
The configuration uses a YAML config borrowed from [Tune-A-Video](https://github.com/showlab/Tune-A-Video) reposotories. Here's the gist of how it works.
7373

0 commit comments

Comments
 (0)