Build question

[IzzySoft]

At IzzyOnDroid we support Reproducible Builds (see: Reproducible Builds, special client support and more at IzzyOnDroid). Trying for yours, I unfortunately was not even able to build the APK:

...

Downloading linux-x64 tools...                                   1,922ms
Downloading linux-x64/font-subset tools...                         294ms
Resolving dependencies...
The current Dart SDK version is 3.7.2.

Because qrserv requires SDK version >=3.8.0-265.0.dev <4.0.0, version solving failed.
Failed to update packages.

I've used the latest stable Flutter (currently 3.29.3) with the following recipe:

build:
  - sed -r '/signingConfigs.getByName/d' -i android/app/build.gradle.kts
  - sed -r '/create\("release"\) \{/,/}/d' -i android/app/build.gradle.kts
  - FLUTTER_VERSION="3.29.3"
  - git clone -b $FLUTTER_VERSION https://github.com/flutter/flutter
  - export PUB_CACHE=$(pwd)/.pub-cache
  - flutter/bin/flutter config --no-analytics
  - flutter/bin/flutter pub get
  - flutter/bin/flutter build apk --release --target-platform android-arm

(the first two lines are just to remove the signing config, as we need an unsigned APK here). Did I miss a step? How can I see what Flutter version you are using? Maybe you could add flutter as git submodule, or add the version used to your pubspec.yaml:

environment:
    sdk: '>= 3.8.0-265.0.dev <4.0.0'
    flutter: ^3.29.3

And while I'm here: could you let us know what path you build in, as Flutter loves to embed the build path in its native libs? That would save us a few test-runs to find it via e.g. Diffoscope 😉

We'd appreciate if you could help making your build reproducible. We've also prepared some hints on reproducible builds for that.

Looking forward to your reply!

[uintdev]

Hi,

This is currently on the main channel. I had created a new commit that adds the Flutter version and updated the Dart SDK version to line up with it (the versions used on the v2.7.0 release build).

environment:
    sdk: '>= 3.9.0-96.0.dev <4.0.0'
    flutter: ^3.32.0-1.0.pre.412

I could try moving it to the stable channel, if that would ultimately help (really should be using that channel anyway).

Build path: "C:\Users\node\Projects\Projects\qrserv" (no typo here -- could've been better named)

[IzzySoft]

Build path: "C:\Users\node\Projects\Projects\qrserv"

Oh… OK, then we have to give up. We cannot match Windows paths on Linux, unfortunately. So while we might get the build-id removed with cmake (which is used here), the path would remain in the way. AFAIR there was a way to use relative paths, which might solve the problem with the leading C:, I could not find my notes on this – and I don't know if it would work with Flutter.

@linsui you came up with this approach on a different project, can you tell?

An alternative would be to use Github action (or any other CI)s to build. Those would have the added benefit of guaranteeing a clean tree on each build 😉

[linsui]

I don't know how to remove the absolute path for flutter. I don't know how to pass a CFLAG to dart anyway. Not sure if it accepts something like that.

[IzzySoft]

Thanks @linsui! I guess flags could be passed from the ndk section in build.gradle, or from a corresponding cmake section parallel to that. But I've lost the notes on this you made on another app, so I wouldn't even know how to try it. Dart seems just to prepare things, the native libs seem to be generated by cmake after it called gradlew (going by what I remember from the build logs) – though I'm not 100% sure, as the build logs from some Flutter apps do not mention cmake at all.

[linsui]

All tricks I know are documented in https://f-droid.org/docs/Reproducible_Builds/. For cmake, you can use add_compile_options("-ffile-prefix-map=${CMAKE_CURRENT_SOURCE_DIR}=."). Some pub packages use cmake to build native libs but dart itself also has embeded path. I don't know how to fix the absolute path generated by dart itself.

[IzzySoft]

Thanks again, @linsui – I'll try to remember looking up that page before pinging you next time 🙈

@uintdev that add_compile_options would need to go to CMakeLists.txt, which you don't have (at least I didn't see it). The alternative would be adding it to build.gradle:

android {
   externalNativeBuild {
      cmake {
          cFlags "-ffile-prefix-map=${rootDir}=."
          cppFlags "-ffile-prefix-map=${rootDir}=."
       }
   }
}

If you want to give it a try, that is. If it's really Dart embedding the paths, we have no chance (apart from switching to e.g. CI builds via Github actions), we've tried that in the past but never succeeded.

[uintdev]

After adding the cmake flags into the build.gradle.kts file...

android {
    defaultConfig {
        externalNativeBuild {
            cmake {
                arguments += listOf(
                    "-DCMAKE_C_FLAGS=-ffile-prefix-map=${rootDir}=.",
                    "-DCMAKE_CXX_FLAGS=-ffile-prefix-map=${rootDir}=."
                )
            }
        }
    }
}

It did not appear to do anything. Tried various other options (even those brought up in issues regarding this). Nothing. Full path would still be around under libapp.so.
I tried Linux ARM64 -- even with Rosetta 2 (which helped get it much further) as to at least change it to a path that could be worked with, but Android Studio and some of the tooling (in this case, a filesystem watcher) prevents Android builds from succeeding for one reason or another. Officially, Android Studio isn't supported on Linux machines using ARM-based CPUs yet anyway.

I was thinking of maybe just building it on the Steam Deck as that is Linux AMD64.. assuming having an immutable system won't be an issue in this case. 🤔

[IzzySoft]

Steam Deck should work (I know someone using that for "coding while traveling"). If you want a quick setup for a build system there, you're welcome to use ours. Thanks to rbuilder_setup it takes less than 5 minutes to set it up, and I could give you the full build recipe I've used to get started with it. That would kinda work like a CI then, using Podman and ephemeral containers (to counter your immutable system 😜), thus guaranteeing a clean build on each run. The setup scripts support Debian & derivatives, RPM, and also Arch and derivatives, and thus cover most of the Linux world.

PS: the path problem is solved there automatically as you define the path in the recipe:

build_home_dir: /build
build_repo_dir: /build/repo

These are the defaults, and can be adjusted as needed.

[linsui]

See flutter/flutter#140558, even if you build the app in a windows-like path, there are other problems.

[IzzySoft]

Thanks once more! I've left a comment there to move it up again in the list (the last one was from LK more than a year ago). Maybe having a second party with a different build system chiming in helps raising the priority; idk, but worth a try.

[uintdev]

I see that the Steam OS (maybe as of 3.5) includes podman, which is nice. I had got Podman Desktop installed for the sake of simpler management. Of course, CLI is always an option.
Any suggestions as to how I should have the container set up? I'm not certain as to what details I'd need in the container configuration to ensure it plays well with rbuilder. In terms of the image, I'm looking at one from 'docker.io/library/ubuntu:24.04'.

[IzzySoft]

Oh, podman would be perfect. Maybe you want to try what we use? rbuilder_setup makes the setup a breeze, done in less than 5 minutes. I could offer you the recipe I tried to start with then, it has the image definition and all.

[uintdev]

Yeah, that would be great.

[IzzySoft]

This was the last recipe I've used. You'll probably need at least to adjust the Flutter version (ideally, you'd include Flutter as git submodule instead, and then skip the clone):

---
repository: https://github.com/uintdev/qrserv.git
updates: releases
versions:
  - tag: v2.7.0
    apks:
      - apk_pattern: qrserv-.*-armeabi-v7a\.apk
        apk_url: https://github.com/uintdev/qrserv/releases/download/v2.7.0/qrserv-v2_7_0-armeabi-v7a.apk
        build:
          - sed -r '/signingConfigs.getByName/d' -i android/app/build.gradle.kts
          - sed -r '/create\("release"\) \{/,/}/d' -i android/app/build.gradle.kts
          - FLUTTER_VERSION="3.29.3"
          - git clone -b $FLUTTER_VERSION https://github.com/flutter/flutter
          - export PUB_CACHE=$(pwd)/.pub-cache
          - flutter/bin/flutter config --no-analytics
          - flutter/bin/flutter pub get
          - flutter/bin/flutter build apk --release --target-platform android-arm
          - find . -name '*.apk'
          - mv build/app/outputs/flutter-apk/app-release.apk /outputs/unsigned.apk
        build_cpus:
        build_home_dir: /build
        build_repo_dir: /build/repo
        build_timeout:
        build_user: build
        provisioning:
          android_home: /opt/sdk
          build_tools:
          cmake:
          cmdline_tools:
            version: '12.0'
            url: https://dl.google.com/android/repository/commandlinetools-linux-11076708_latest.zip
            sha256: 2d2d50857e4eb553af5a6dc3ad507a17adf43d115264b1afc116f95c92e5e258
          extra_packages:
            - curl
          image: debian:bookworm-slim
          jdk: openjdk-17-jdk-headless
          ndk:
          platform:
          platform_tools:
          tools:
          verify_gradle_wrapper: false

The find in there is just for debug. Sometimes the resulting APKs have quite specific names 😉

[uintdev]

Got a lot of things configured on the Steam Deck for that immutable-by-default sort of environment.
I got a container running and went through the setup -- went smoothly. What would I need to use in order to take in the YAML recipe file?