GH-48312: [C++][FlightRPC] Standalone ODBC MSVC CI #48313

alinaliBQ · 2025-12-03T00:42:06Z

Rationale for this change

#48312

What changes are included in this PR?

Add new ODBC workflow for Windows MSVC
Added build fixes to enable build on MSVC CI

Are these changes tested?

Tested in CI

Are there any user-facing changes?

N/A

GitHub Issue: [C++][FlightSQL][ODBC] Windows ODBC standalone CI #48312

github-actions · 2025-12-03T00:42:32Z

⚠️ GitHub issue #48278 has been automatically assigned in GitHub to PR creator.

alinaliBQ · 2025-12-03T00:42:48Z

.github/workflows/cpp_odbc.yml

+
+      #     bash -c "ci/scripts/cpp_test.sh $(pwd) $(pwd)/build"
+
+      # GH-47787 TODO Build ODBC installer


Once this PR is merged, I can rebase and uncomment below code for GH-47787 in PR #48054

Could you remove this in this PR and add this in the separated PR?

ok, removed the comments here

github-actions · 2025-12-03T00:43:48Z

⚠️ GitHub issue #48312 has been automatically assigned in GitHub to PR creator.

alinaliBQ · 2025-12-03T17:31:06Z

.github/workflows/cpp_odbc.yml

@raulcd , @kou , @lidavidm If you have time, please have a look at this PR, thank you!

Could you move this content to cpp_extra.yml?
cpp_windows.yml was extracted to a separated file because it's reused from multiple jobs. But this job isn't reused.

@kou Sure, I have moved the file to cpp_extra.yml, thanks for the context about cpp_windows.yml.

raulcd

I'll try to take a look but as an initial comment is the commit about moving from boost::optional to std::optional required for this PR?
a696be3

alinaliBQ · 2025-12-03T18:17:06Z

I'll try to take a look but as an initial comment is the commit about moving from boost::optional to std::optional required for this PR? a696be3

Yes I think so, previously when I was testing my changes on a different branch, the build was failing due to mixed usages of boost::optional and std::optional. Having this commit resolved the build failures

lidavidm · 2025-12-03T23:56:26Z

I'll try to take a look but as an initial comment is the commit about moving from boost::optional to std::optional required for this PR? a696be3

Yes I think so, previously when I was testing my changes on a different branch, the build was failing due to mixed usages of boost::optional and std::optional. Having this commit resolved the build failures

Do you want to submit this as another PR first?

alinaliBQ · 2025-12-04T00:26:10Z

Do you want to submit this as another PR first?

Sure, I have extracted the std::optional change in #48323.

kou

Why do we need separated workflow?

kou · 2025-12-04T01:38:13Z

.github/workflows/cpp_odbc.yml

+  schedule:
+    - cron: '0 13 * * *'


Why do we need this?

Since the workflow will only be run with ODBC related changes, making it run nightly will help catch any ODBC issues that occur when there are no ODBC changes merged into main.

moving it as a job inside the cpp_extra.yaml as @kou pointed on another comment allows us to run this as part of the nightlies and also trigger it on demand via adding the CI: extra label so we can always run it as part of a job if we think it might be relevant.

Thanks @raulcd and @kou for pointing me to cpp_extra.yml. I have tried to move the ODBC workflow to cpp_extra.yml. Currently it is building, will fix any build issues as I see them

I see ODBC was able to build successfully here: C++ Extra / C++ ODBC / windows (pull_request)

kou · 2025-12-04T01:39:00Z

.github/workflows/cpp_odbc.yml

+      # GH-47787 TODO Build ODBC installer
+      # ARROW_FLIGHT_SQL_ODBC_INSTALLER: ON
+      ARROW_SIMD_LEVEL: AVX2
+      CMAKE_CXX_STANDARD: "17"


Should we use 20?

ODBC is using CMAKE_CXX_STANDARD 20, but I am not sure if Arrow Flight and Flight SQL support 20 yet, so I used 17 in the workflow to be safe.

.github/workflows/cpp_odbc.yml

Remove DataSet Fix cache key Use odbc-specific cache key Fix Lint and Add odbc registration step Link appropriate GitHub issues that blocks ODBC tests Disable test phase and add schedule Run everyday 7 am Vancouver time Enable ODBC build on MSVC CI Code Clean up and enable ODBC tests in CI Still need to modify ODBCUtilEnvironment if we decide to use it. Still need to add ODBC V2 support so a different env and conn is used for ODBC 2 tests. Draft enable ODBC global setup/teardown Run ODBC test once outside of test script If ODBC test is run using MinGW Shell, segfault occurs. I am not getting any seg fault on my local MSVC Windows when I run the tests without the bash script. But if Windows CI breaks from running the standalone exe then I will look into this Prepend vcpkg to search vcpkg before `<prefix>/lib/cmake` Since we are installing dependencies on vcpkg, if `lib/cmake` is searched first, then cmake will look into that directory and use the wrong paths. Convert VCPKG Windows path to MSYS path Set `VCPKG_ROOT` in test phase Fix ODBC dll name Use `arrow_flight_sql_odbc.dll` instead of `libarrow_flight_sql_odbc.dll` which is the naming convention used on MinGW Windows Link ODBC library on all platforms On my local MSVC Windows machine, Visual Studio is used to build without needing to link ` ${ODBCINST}` explicitly, its behavior might be different from Ninja which is what CI uses. `${ODBCINST}` is likely needed by Linux as well, so adding it for all platforms. Attempt to resolve `arrow-compute-grouper-benchmark` build issue I think `if(ARROW_FLIGHT_TEST_LINKAGE STREQUAL "static")` might be more appropriate here, as I am seeing build issues due to both dynamic and static linking occurring. I see in apache@59903d0, this check is used for `arrow-filesystem-s3fs-benchmark` Disable `UNITY_BUILD` for ODBC test due to conflict on `sqlite_sql_info.cc` On some workflows, unity build is set to ON to make the build faster. This is an attempt to resolve the build issue. Remove debug messages Fix lint Add `sql_info_undef.h` to sqlite_sql_info.cc Undefine duplicates in SQLGetInfo Add `#pragma once` to odbc_test_suite.h To avoid redefinition error Attempt to resolve conflict with sql/types.h Attempt to include Arrow headers before ODBC headers to avoid conflict with arrow/flight/sql/types.h [apacheGH-48084] Replace boost::optional with std::optional Addresses comment apache#40939 (comment) Replace boost::optional with std::optional in ODBC codebase apache#48084 Fix lint Remove `BOOST_SOURCE=BUNDLED` to use vcpkg's dynamic link to boost Since we need to use link to boost dynamically, we need to remove the bundled boost library flag. Add debug messages. Remove `ARROW_BOOST_USE_SHARED = OFF` Since we have a release build, can enable boost as shared. With `ARROW_BOOST_USE_SHARED = OFF`, I am getting error ``` D:\a\arrow\arrow\build\cpp\vcpkg_installed\x64-windows\include\boost/filesystem/config.hpp(96): fatal error C1189: #error: Must not define both BOOST_FILESYSTEM_DYN_LINK and BOOST_FILESYSTEM_STATIC_LINK ``` Also increased time limit, since 2 hours don't seem to be enough to finish the vcpkg build Change to release build Getting ``` orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in unity_2_cxx.cxx.obj orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in unity_2_cxx.cxx.obj ``` when building with debug and presumably ARROW_ORC Set ARROW_BOOST_USE_SHARED to OFF `ARROW_BOOST_USE_SHARED` is restored to `OFF`, which according to Windows doc should help with the debug build now. Retore value of `ARROW_BUILD_BENCHMARKS`, though noting it is set to `OFF` in GLib MSVC workflow. Add VCPKG set up Borrowed from the Glib workflow Add `/EHsc` flag * Add back `CMAKE_CXX_STANDARD: "17"` Remove `CMAKE_CXX_STANDARD` 17 * reason: gtest can't be used with C++17. Arrow project doesn't support C++ 17 yet. Extend run-time to 2hr windows-mingw also has timeout of 2hr Empty commit to trigger CI Specify `VCPKG_BINARY_SOURCES` and `VCPKG_DEFAULT_TRIPLET` Specify VCPKG_TARGET_TRIPLET as x64 windows Set ARROW_DEPENDENCY_SOURCE to VCPKG To make it easier to manage dependencies Enable static build on MSVC `ARROW_BUILD_BENCHMARKS` prevents Flight from being built Remove `ARROW_BOOST_USE_SHARED` Having static vs. shared issue Enable Flight & Flight SQL for ODBC Enable ODBC build on MSVC CI Enable regular ctest tests Remove undef items Add concurrency and permissions to odbc yml Create cpp_odbc.yml Fix architecture in CI * Add ODBC installer MSVC CI

- Use newest `actions/checkout` Co-Authored-By: alinalibq <alina.li@improving.com>

alinaliBQ

@kou Thanks for the review! Addressed all comments.

.github/workflows/cpp_extra.yml

ci/scripts/cpp_test.sh

alinaliBQ · 2025-12-15T18:51:16Z

.github/workflows/cpp_odbc.yml

+      ARROW_BUILD_SHARED: ON
+      ARROW_BUILD_STATIC: ON


Removed static libraries here

.github/workflows/cpp_extra.yml

alinaliBQ · 2025-12-15T18:54:21Z

.github/workflows/cpp_odbc.yml

+
+      #     bash -c "ci/scripts/cpp_test.sh $(pwd) $(pwd)/build"
+
+      # GH-47787 TODO Build ODBC installer


ok, removed the comments here

alinaliBQ · 2025-12-15T18:58:17Z

cpp/cmake_modules/SetupCxxFlags.cmake

      #
      # ARROW-2986: Without /EHsc we get C4530 warning
      set(CXX_COMMON_FLAGS "/W3 /EHsc")
+      string(APPEND CMAKE_CXX_FLAGS " /EHsc")


I found this line was needed when I was working on a separate build. Locally on MSVC Visual Studio I didn't need this line, but on CI using Ninja I get C4530 warning when building GTestAlt, which was treated as an error. Adding this line resolved the issue below at FindGTestAlt.cmake.

Issue Log:

C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\__msvc_ostream.hpp(781): error C2220: the following warning is treated as an error C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.44.35207\include\__msvc_ostream.hpp(781): warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc

GitHub Action Run Log: https://github.com/apache/arrow/actions/runs/19112837562/job/54614206545

alinaliBQ · 2025-12-15T19:05:12Z

.github/workflows/cpp_odbc.yml

@kou Sure, I have moved the file to cpp_extra.yml, thanks for the context about cpp_windows.yml.

Move `include(ThirdpartyToolchain)`

kou

+1

pitrou · 2025-12-18T09:56:59Z

These builds always take very long (around 1h40), 1) are we sure vcpkg cacking is working? 2) can we stop building large unused dependencies such as aws-sdk-cpp and google-cloud-cpp?

raulcd · 2025-12-18T10:30:42Z

Just noticed this too on my PR, caching does not seem to be working, there seems to have some credential issues:

 WARNING: Your request could not be authenticated by the GitHub Packages service. Please ensure your access token is valid and has the appropriate scopes configured.
  Forbidden https://nuget.pkg.github.com/apache/ 7052ms
Response status code does not indicate success: 403 (Forbidden).
System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).

raulcd · 2025-12-18T10:32:34Z

I can also see some timeouts and I've seen some problems on other jobs connecting to github to download packages so I am unsure whether the problem is that GitHub is having issues or if there's really a problem with credentials:

Starting submission of google-cloud-cpp[core,rest-common,storage]:x64-windows@2.36.0 to 1 binary cache(s) in the background
Elapsed time to handle google-cloud-cpp:x64-windows: 5 min
google-cloud-cpp:x64-windows package ABI: 8e20db0777ec996c988874fe6425f50651387f6a355a4326a6678c64f235d355
error: "C:\ProgramData\Chocolatey\bin\nuget.exe" push -ForceEnglishOutput -Verbosity detailed -NonInteractive "D:\a\arrow\arrow\vcpkg\buildtrees\crc32c_x64-windows.1.1.2-vcpkg1b2e2cd4d288cef27d01ba4088df517007c115d52a20222fe4f7aa2621a85739.nupkg" -Timeout 100 -Source GitHub failed with exit code 1
NuGet Version: 7.0.1.1

alinaliBQ · 2025-12-18T18:09:33Z

Sorry, seems there is an issue with the caching, let me look into this issue today.

conbench-apache-arrow · 2025-12-23T19:21:10Z

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit c58bfe5.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 8 possible false positives for unstable benchmarks that are known to sometimes produce them.

github-actions bot added Component: C++ Component: FlightRPC awaiting review Awaiting review labels Dec 3, 2025

alinaliBQ commented Dec 3, 2025

View reviewed changes

alinaliBQ changed the title ~~GH-48278 : [C++][FlightRPC] work-in-progress Standalone ODBC MSVC CI~~ GH-48312 : [C++][FlightRPC] work-in-progress Standalone ODBC MSVC CI Dec 3, 2025

github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Dec 3, 2025

alinaliBQ changed the title ~~GH-48312 : [C++][FlightRPC] work-in-progress Standalone ODBC MSVC CI~~ GH-48312 : [C++][FlightRPC] Standalone ODBC MSVC CI Dec 3, 2025

alinaliBQ marked this pull request as ready for review December 3, 2025 17:29

alinaliBQ requested review from assignUser, jonkeane, kou, lidavidm and raulcd as code owners December 3, 2025 17:29

alinaliBQ commented Dec 3, 2025

View reviewed changes

raulcd reviewed Dec 3, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Dec 3, 2025

alinaliBQ mentioned this pull request Dec 3, 2025

GH-48019 : [C++][FlightRPC] work-in-progress Enable ODBC in MSVC CI #48067

Closed

alinaliBQ force-pushed the gh-48278-odbc-ci branch from fcde182 to fbdcc6e Compare December 3, 2025 22:14

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 3, 2025

alinaliBQ mentioned this pull request Dec 4, 2025

GH-48084: [C++][FlightRPC] Replace boost::optional with std::optional #48323

Merged

kou reviewed Dec 4, 2025

View reviewed changes

github-actions bot removed the awaiting change review Awaiting change review label Dec 4, 2025

github-actions bot added awaiting changes Awaiting changes and removed awaiting change review Awaiting change review labels Dec 14, 2025

alinaliBQ added 6 commits December 15, 2025 10:36

Address comments from Kou

8dc692a

Attempt to move ODBC CI to cpp_extra.yml

7841861

- Use newest `actions/checkout` Co-Authored-By: alinalibq <alina.li@improving.com>

Keep only ARROW_FLIGHT_SQL_ODBC flag on cpp_odbc.yml

a25015a

Address Kou's comments

fed94f0

Move cpp_odbc to cpp_extra

20d2e85

alinaliBQ force-pushed the gh-48278-odbc-ci branch from a257437 to 20d2e85 Compare December 15, 2025 19:29

alinaliBQ commented Dec 15, 2025

View reviewed changes

github-actions bot added awaiting change review Awaiting change review awaiting changes Awaiting changes and removed awaiting changes Awaiting changes awaiting change review Awaiting change review labels Dec 15, 2025

Address Kou's comments Dec-16

4ad7af1

Move `include(ThirdpartyToolchain)`

alinaliBQ force-pushed the gh-48278-odbc-ci branch from cd7c473 to 4ad7af1 Compare December 16, 2025 18:36

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Dec 16, 2025

kou approved these changes Dec 17, 2025

View reviewed changes

kou merged commit c58bfe5 into apache:main Dec 17, 2025
46 of 50 checks passed

kou removed the awaiting change review Awaiting change review label Dec 17, 2025

kou mentioned this pull request Dec 17, 2025

[C++][FlightSQL][ODBC] Windows ODBC standalone CI #48312

Closed

github-actions bot added the awaiting merge Awaiting merge label Dec 17, 2025

This was referenced Dec 18, 2025

[C++][FlightSQL][ODBC] MSVC CI caching issues #48594

Open

GH-48594: [C++][FlightRPC] Fix ODBC CI Long Build Time Issue #48595

Open


		# bash -c "ci/scripts/cpp_test.sh $(pwd) $(pwd)/build"

		# GH-47787 TODO Build ODBC installer

GH-48312: [C++][FlightRPC] Standalone ODBC MSVC CI #48313

GH-48312: [C++][FlightRPC] Standalone ODBC MSVC CI #48313

Conversation

alinaliBQ commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions bot commented Dec 3, 2025

Uh oh!

alinaliBQ Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 3, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

raulcd left a comment

Choose a reason for hiding this comment

Uh oh!

alinaliBQ commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lidavidm commented Dec 3, 2025

Uh oh!

alinaliBQ commented Dec 4, 2025

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

alinaliBQ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pitrou commented Dec 18, 2025

Uh oh!

raulcd commented Dec 18, 2025

Uh oh!

raulcd commented Dec 18, 2025

alinaliBQ commented Dec 3, 2025 •

edited

Loading

alinaliBQ Dec 3, 2025 •

edited

Loading

alinaliBQ commented Dec 3, 2025 •

edited

Loading