diff --git a/README.md b/README.md index b4f71ef..1093163 100644 --- a/README.md +++ b/README.md @@ -31,16 +31,7 @@ In our original work (NeurIPS 2023) we used GPT3.5 and GPT4 as part of our agent ## Get started: install AmadeusGPT🎻 -### [1] Set up a conda environment: - -Conda is an easy-to-use Python interface that supports launching [Jupyter Notebooks](https://jupyter.org/). If you are completely new to this, we recommend checking out the [docs here for getting conda installed](https://deeplabcut.github.io/DeepLabCut/docs/beginner-guides/beginners-guide.html#beginner-user-guide). Otherwise, proceed to use one of [our supplied conda files](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/conda). As you will see we have minimal dependencies to get started, and [here is a simple step-by-step guide](https://deeplabcut.github.io/DeepLabCut/docs/installation.html#step-2-build-an-env-using-our-conda-file) you can reference for setting it up (or see [BONUS](README.md#bonus---customized-your-conda-env) below). Here is the quick start command: - -```bash -conda env create -f amadeusGPT.yml -``` -To note, some modules AmadeusGPT can use benefit from GPU support, therefore we recommend also having an NVIDIA GPU available and installing CUDA. - -### [2] You will need an openAI key: +### [1] You will need an openAI key: **Why OpenAI API Key is needed** AmadeusGPT relies on API calls of OpenAI (we will add more LLM options in the future) for language understanding and code writing. Sign up for a [openAI API key](https://platform.openai.com/account/api-keys) [here](https://platform.openai.com/account/api-keys). @@ -58,6 +49,16 @@ import os os.environ["OPENAI_API_KEY"] = 'your api key' ``` +### [2] Set up a conda environment: + +Conda is an easy-to-use Python interface that supports launching [Jupyter Notebooks](https://jupyter.org/). If you are completely new to this, we recommend checking out the [docs here for getting conda installed](https://deeplabcut.github.io/DeepLabCut/docs/beginner-guides/beginners-guide.html#beginner-user-guide). Otherwise, proceed to use one of [our supplied conda files](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/conda). As you will see we have minimal dependencies to get started, and [here is a simple step-by-step guide](https://deeplabcut.github.io/DeepLabCut/docs/installation.html#step-2-build-an-env-using-our-conda-file) you can reference for setting it up (or see [BONUS](README.md#bonus---customized-your-conda-env) below). Here is the quick start command: + +```bash +conda env create -f amadeusGPT.yml +``` +To note, some modules AmadeusGPT can use benefit from GPU support, therefore we recommend also having an NVIDIA GPU available and installing CUDA. + + ### [3] 🪄 That's it! Now you have AmadeusGPT installed! See below on how to get started! @@ -68,16 +69,16 @@ See below on how to get started! We provide a StreamLit App, or you can use AmadeusGPT in any python interface, such as Jupyter notebooks. For this we suggest getting started from our demos: ### Try AmadeusGPT with an example Jupyter Notebook -You can git clone (or download) this repo to grab a copy and go. We provide example notebooks [here](notebook)! +You can git clone (or download) this repo to grab a copy and go. We provide example notebooks [here](notebooks)! ### Here are a few demos that could fuel your own work, so please check them out! 1) [Draw a region of interest (ROI) and ask, "when is the animal in the ROI?"](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/EPM_demo.ipynb) -2) [Use your own data](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/YourData.ipynb) - (make sure you use a GPU to run SuperAnimal if you don't have corresponding DeepLabCut keypoint files already! +2) [Use your own data](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/YourData.ipynb) - (make sure you use a GPU to run SuperAnimal if you don't have corresponding DeepLabCut keypoint files already!) 3) [Write you own integration modules and use them](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/Horse_demo.ipynb). Bonus: [source code](amadeusgpt/integration_modules). Make sure you delete the cached modules_embedding.pickle if you add new modules! 4) [Multi-Animal social interactions](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/MABe_demo.ipynb) -5) [Reuse the task program generated by LLM and run it on different videos](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/MABe_demo.ipynb) -7) You can ask one query across multiple videos. Put your keypoint files and video files (pairs) in the same folder and specify the `data_folder` as shown in this [Demo](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/custom_mouse_video.ipynb). Make sure your video file and keypoint file follows the normal DeepLabCut convention, i.e., `prefix.mp4` `prefix*.h5`. +5) [Reuse the task program generated by LLM and run it on different videos](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/Use_Task_Program.ipynb) +7) You can ask one query across multiple videos. Put your keypoint files and video files (pairs) in the same folder and specify the `data_folder` as shown in this [Demo](https://github.com/AdaptiveMotorControlLab/AmadeusGPT/tree/main/notebooks/YourData.ipynb). Make sure your video file and keypoint file follows the normal DeepLabCut convention, i.e., `prefix.mp4` `prefix*.h5`. ### Minimal example