-
Notifications
You must be signed in to change notification settings - Fork 1
Draft: Add script for final config generation from debian and GL #206
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -73,3 +73,63 @@ The automation creates a PR if a new patch level is available. | |
|
|
||
| > [!Note] | ||
| > This is done via the [update-kernel.py](https://github.com/gardenlinux/package-linux/blob/main/update-kernel.py) tool | ||
|
|
||
| ## Config generation from debian and garden linux | ||
|
|
||
| Script(get_config.sh) takes prepare_source as input for kernel version and generates | ||
|
|
||
| 1. Final debian kernel configuration file | ||
| 2. Garden Linux kernel configuration file | ||
|
|
||
| ### Container Image | ||
| Script should be executed inside repo-debian-snapshot container | ||
| ``` | ||
| Image: ghcr.io/gardenlinux/repo-debian-snapshot | ||
| ``` | ||
|
|
||
| ### Volume Mount | ||
| ``` | ||
| -v package-linux:/workspace: Mounts the local package-linux directory to /workspace in the container | ||
| ``` | ||
|
|
||
| ### Usage Examples | ||
|
|
||
| #### For AMD64 Architecture | ||
| ``` | ||
| podman run -v package-linux:/workspace ghcr.io/gardenlinux/repo-debian-snapshot:1764836249-amd64 /workspace/get_config.sh | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. is it required to use the architecture specific tag? just using
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, if we build for the same architecture then yes we do not need arch here. If we will have to build for arm64 in x86 machine then we need this specifically. Anyways I will wrap the container call to make it simple once we finalize the way we proceed. Then I will have arch as an argument and podman will start automatically |
||
| ``` | ||
| #### For ARM64 Architecture | ||
| ``` | ||
| podman run -v package-linux:/workspace ghcr.io/gardenlinux/repo-debian-snapshot:1764836249-arm64 /workspace/get_config.sh | ||
| ``` | ||
| ### Important Notes | ||
| > Final config files are copied in package-linux folder | ||
|
|
||
| > If debian_version and kernel_version is not specified , then default version from prepare_source is used. | ||
|
|
||
| ## Comparision of kernel configs | ||
|
|
||
| Script (compare-config.py) compares 2 input config files and generates the differences in 3 sections (Added, Removed, Modified) | ||
|
|
||
| Output needs to be manually verified | ||
|
|
||
| ### Example output | ||
|
|
||
| ``` | ||
| REMOVED: | ||
| CONFIG_ACPI_EC_DEBUGFS=y | ||
| CONFIG_ACPI_VIOT=y | ||
| CONFIG_BCACHEFS_FS=m | ||
| CONFIG_BCACHEFS_POSIX_ACL=y | ||
| CONFIG_BCACHEFS_QUOTA=y | ||
| CONFIG_BCACHEFS_SIX_OPTIMISTIC_SPIN=y | ||
|
|
||
| ADDED: | ||
| CONFIG_CC_HAS_COUNTED_BY=y | ||
| CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y | ||
| CONFIG_DEBUG_INFO_BTF=y | ||
|
|
||
| CHANGED: | ||
| CONFIG_CC_VERSION_TEXT: x86_64-linux-gnu-gcc-14 (Debian 14.3.0-10) 14.3.0 -> x86_64-linux-gnu-gcc (Debian 15.2.0-9) 15.2.0 | ||
| CONFIG_GCC_VERSION: 140300 -> 150200 | ||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| #!/usr/bin/env python3 | ||
|
|
||
| import sys | ||
|
|
||
| def extract_config_values(config_path): | ||
| config_values = {} | ||
| with open(config_path, 'r') as file: | ||
| for line in file: | ||
| if not line.strip(): | ||
| continue | ||
| if line.startswith('#') and line.endswith(' is not set'): | ||
| key = line.split()[1] | ||
| config_values[key] = 'n' | ||
| continue | ||
| if '=' in line: | ||
| key, value = line.split('=', 1) | ||
| config_values[key.strip()] = value.strip().strip('"') | ||
| return config_values | ||
|
|
||
| def compare_configs(config1, config2): | ||
| print("Comparing configuration values...") | ||
|
|
||
| all_keys = set(config1.keys()) | set(config2.keys()) | ||
| added = [] | ||
| removed = [] | ||
| changed = [] | ||
|
|
||
| for key in sorted(all_keys): | ||
| val1 = config1.get(key, 'not set') | ||
| val2 = config2.get(key, 'not set') | ||
| if val1 == 'not set' and val2 != 'not set': | ||
| added.append(f"{key}={val2}") | ||
| elif val1 != 'not set' and val2 == 'not set': | ||
| removed.append(f"{key}={val1}") | ||
| elif val1 != val2: | ||
| changed.append(f"{key}: {val1} -> {val2}") | ||
| if removed: | ||
| print("REMOVED:") | ||
| for item in removed: | ||
| print(f"{item}") | ||
| print() | ||
| if added: | ||
| print("ADDED:") | ||
| for item in added: | ||
| print(f"{item}") | ||
| print() | ||
| if changed: | ||
| print("CHANGED:") | ||
| for item in changed: | ||
| print(f"{item}") | ||
|
|
||
| def main(): | ||
| if len(sys.argv) != 3: | ||
| print (sys.argv) | ||
| print("Usage: python compare.py <config1> <config2>") | ||
| sys.exit(1) | ||
| else: | ||
| config1 = sys.argv[1] | ||
| config2 = sys.argv[2] | ||
| config1_val = extract_config_values(config1) | ||
| config2_val = extract_config_values(config2) | ||
| compare_configs(config1_val, config2_val) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| #!/usr/bin/env bash | ||
| repo_dir="$(dirname "$(realpath "${BASH_SOURCE[0]}")")" | ||
|
|
||
| get_debian_vers_from_pkg_repo() { | ||
| debian_version_line=$(grep -m1 'debian_ref=' "$repo_dir/prepare_source") | ||
| echo "${debian_version_line#*=}" | tr -d '"' | ||
| } | ||
|
|
||
| get_vers_orig_from_pkg_repo() { | ||
| version_orig_line=$(grep -m1 'version_orig=' "$repo_dir/prepare_source") | ||
| echo "${version_orig_line#*=}" | tr -d '"' | ||
| } | ||
|
|
||
| deb_version=$(get_debian_vers_from_pkg_repo) | ||
| version_orig=$(get_vers_orig_from_pkg_repo) | ||
|
|
||
| arch=$(dpkg --print-architecture 2>/dev/null) | ||
|
|
||
|
|
||
| apt-get install -y --no-install-recommends quilt flex bison build-essential gcc-14 make fakeroot libncurses-dev kernel-wedge python3-jinja2 python3-dacite wget gpgv | ||
|
|
||
| apt-get remove --purge sqopv -y | ||
|
|
||
| mkdir -p /tmp | ||
|
|
||
| cd /tmp | ||
|
|
||
| git clone https://salsa.debian.org/kernel-team/linux.git | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this repo is large and a full clone is not needed here. this can be way faster if we take only the latest commit of our desired tag/branch using git clone --depth=1 --branch=debian/6.12.43-1 https://salsa.debian.org/kernel-team/linux.git
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. compare the sizes of the cloned directories. Also it is now just a matter of seconds instead of taking almost two minutes. |
||
| cd linux | ||
| git checkout $deb_version | ||
|
|
||
|
|
||
| uscan -v --download-version="$version_orig" | ||
| git clean -fdx | ||
|
|
||
|
|
||
| mkdir /tmp/src | ||
|
|
||
| xz -d < ../*.orig.tar.* | tar --extract --strip-components 1 --directory "/tmp/src" | ||
|
|
||
| cp -r debian "/tmp/src/" | ||
|
|
||
| cd /tmp/src | ||
|
|
||
| cp $repo_dir/fixes_debian/series.patch /tmp/src/ | ||
|
|
||
| patch -p1 < series.patch | ||
|
|
||
| export QUILT_PATCHES=debian/patches | ||
| quilt pop -a || true | ||
| quilt push -a | ||
|
|
||
| fakeroot make -f debian/rules debian/control | ||
|
|
||
| make -f debian/rules.gen setup_"$arch"_none_cloud-$arch | ||
| make -f debian/rules.gen setup_"$arch"_none_$arch | ||
|
|
||
| cp debian/build/build_"$arch"_none_cloud-$arch/.config $repo_dir/config_"$arch"_none_cloud-$arch | ||
| cp debian/build/build_"$arch"_none_$arch/.config $repo_dir/config_"$arch"_none_$arch | ||
|
|
||
| fakeroot make -f debian/rules clean | ||
| rm -rf "debian/config" | ||
| cp -r $repo_dir/config "debian/" | ||
|
|
||
| fakeroot make -f debian/rules debian/control | ||
|
|
||
| make -f debian/rules.gen setup_"$arch"_none_cloud-$arch | ||
| make -f debian/rules.gen setup_"$arch"_none_$arch | ||
|
|
||
| cp debian/build/build_"$arch"_none_cloud-$arch/.config $repo_dir/config_gl_"$arch"_none_cloud-$arch | ||
| cp debian/build/build_"$arch"_none_$arch/.config $repo_dir/config_gl_"$arch"_none_$arch | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
at least on mac with podman I need to provide the full path for package-linux
using
$PWD/package-linuxworks when I'm in the parent dir of package-linux