Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions api-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,26 @@ go run main.go --taxonomy-path ~/.local/share/instructlab/taxonomy/ --rhelai --c
The `--rhelai` flag indicates that the ilab binary is available in the system's $PATH and does not require a virtual environment.
When using `--rhelai`, the `--base-dir` flag is not required since it will be in a known location at least for meow.

#### RHELAI API server easy-install

It is recommended that you make a temporary directory for yourself to facilitate the install: `mkdir -p ~/temp-apiserver-install && cd ~/temp-apiserver-install`.

We have provided some scripts that should facilitate installation of the API server on RHEL-AI. First, we will download and run a script to install `glibc-devel` as a dependency and reboot the system.

```bash
bash -c "$(curl -fsSL https://raw.githubusercontent.com/instructlab/ui/refs/heads/main/api-server/rhelai-install/install-glibc-devel.sh)"
```

After the reboot has finished we can download the other two install scripts and run the `rhelai-install.sh` as the entrypoint. Make sure to return to your directory before you start:

```bash
cd ~/temp-apiserver-install
curl -fsSL https://raw.githubusercontent.com/instructlab/ui/refs/heads/main/api-server/rhelai-install/install-go.sh
bash -c "$(curl -fsSL https://raw.githubusercontent.com/instructlab/ui/refs/heads/main/api-server/rhelai-install/rhelai-install.sh)"
```

After this, we can cleanup our temp directory as it is no longer required: `rm -rf ~/temp-apiserver-install`.

### Example command with paths

Here's an example command for running the server on a macOS machine with Metal support:
Expand Down
15 changes: 15 additions & 0 deletions api-server/rhelai-install/install-glibc-devel.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/bash
# -*- indent-tabs-mode: nil; tab-width: 2; sh-indentation: 2; -*-

# Dependency installations for RHELAI APIserver install

set -x
set -e
set -o pipefail

echo "Installing \`glibc-devel\` as a dependency of \`cgo\`."
sudo rpm-ostree install glibc-devel
echo "export CGO_ENABLED=1" >> ~/.bashrc

echo "The system needs to be rebooted for \`glibc-devel\` to be installed. Re-booting."
sudo systemctl reboot
37 changes: 37 additions & 0 deletions api-server/rhelai-install/install-go.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/bin/bash
# -*- indent-tabs-mode: nil; tab-width: 2; sh-indentation: 2; -*-

# Dependency installations for RHELAI APIserver install

GO_VERSION="1.23.5"
GO_ARCHIVE="go${GO_VERSION}.linux-amd64.tar.gz"
GO_URL="https://go.dev/dl/${GO_ARCHIVE}"
INSTALL_DIR="$HOME/.go"
PATH_EXPORT="export PATH=\$PATH:${INSTALL_DIR}/bin"

for cmd in curl tar; do
if ! command -v "$cmd" >/dev/null 2>&1; then
echo "Error: '$cmd' is not installed. Please install it and retry."
exit 1
fi
done

echo "Downloading Go ${GO_VERSION}..."
curl -LO "$GO_URL"

mkdir -p "$INSTALL_DIR"

tar --strip-components=1 -C "$INSTALL_DIR" -xzf "$GO_ARCHIVE"

rm "$GO_ARCHIVE"

# Export PATH in current shell
export PATH="$PATH:${INSTALL_DIR}/bin"
echo "Go bin directory added to PATH for the current session."

# Add PATH export to ~/.bashrc if not already present
grep -qxF "$PATH_EXPORT" "$HOME/.bashrc" || echo "$PATH_EXPORT" >> "$HOME/.bashrc"
echo "Go bin directory added to PATH in ~/.bashrc."

# Verify
go version
50 changes: 50 additions & 0 deletions api-server/rhelai-install/rhelai-install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
#!/bin/bash
# -*- indent-tabs-mode: nil; tab-width: 2; sh-indentation: 2; -*-

# Install script for the API server
## dependencies: go, git

set -x
set -e
set -o pipefail

### installations

if [ -z "$(command -v git)" ]; then
echo "please make sure \`git\` is installed."
exit 1
fi

if [ -z "$(command -v go)" ]; then
echo "\`go\` is not installed, installing."
./install-go.sh
fi

if [ -z "$TAXONOMY_PATH" ]; then
echo "Var \$TAXONOMY_PATH was not set, using default path: $HOME/.local/share/instructlab/taxonomy."
export TAXONOMY_PATH="$HOME/.local/share/instructlab/taxonomy"
fi

if [ ! -d "$TAXONOMY_PATH" ]; then
echo "\$TAXONOMY_PATH was set as $TAXONOMY_PATH, but path does not exist."
exit 1
fi

### script

if [ -d "/tmp/ui" ]; then
rm -rf /tmp/ui
fi

git clone https://github.com/instructlab/ui.git /tmp/ui
cd /tmp/ui/api-server
go mod download
go build -o ilab-api-router

CUDA_FLAG=""

if [ "$(command -v nvcc)" ] && [ -n "$(nvcc --version)" ]; then
CUDA_FLAG="--cuda"
fi

./ilab-api-router --taxonomy-path "$TAXONOMY_PATH" $CUDA_FLAG --rhelai --vllm
Loading