Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 27 additions & 27 deletions SerialPrograms/BuildInstructions/Build-Windows-Qt6.8.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,13 @@
The installation order here is important. While other orderings may work, this is the specific order that we have tested. And the Qt installation must be the last thing installed.

1. Install Visual Studio 2022:
1. [Download Page](https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-notes)
2. Make sure you select the C++ development tools.
1. [Download Page](https://docs.microsoft.com/en-us/visualstudio/releases/2022/release-notes)
2. Make sure you select the C++ development tools.
2. Install Windows Development SDK:
1. [Download Page](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/)
1. [Download Page](https://developer.microsoft.com/en-us/windows/downloads/windows-sdk/)
3. Install CMake:
1. [Download Page](https://cmake.org/download/)
2. When prompted select, "Add CMake to the system PATH for all users".
1. [Download Page](https://cmake.org/download/)
2. When prompted select, "Add CMake to the system PATH for all users".

## Install Qt 6.8.3:

Expand All @@ -23,16 +23,16 @@ Unlike with Qt 5.12, there is no offline installer for it. So you have two optio

If you are ok with creating an account with Qt and using their online installer, then use this method.

1. Download the online installer from here: https://www.qt.io/download-qt-Images/
3. Select the following options:
- Qt 6.8.3
- MSVC 2022 64-bit
- Sources
- Additional Libraries
- Qt Image Formats
- Qt Multimedia
- Qt Serial Port
- Qt Debug Information Files
1. Download the online installer from here: https://www.qt.io/download-qt-installer-oss/
2. Select the following options:
- Qt 6.8.3
- MSVC 2022 64-bit
- Sources
- Additional Libraries
- Qt Image Formats
- Qt Multimedia
- Qt Serial Port
- Qt Debug Information Files

![](Images/Windows-Install-Qt6.8.3-Custom.png)
![](Images/Windows-Install-Qt6.8.3-Components.png)
Expand All @@ -43,12 +43,12 @@ If you repeatedly run into an error involving "SSL handshake failed", you will n

If you are unable or unwilling to use the online installer, the alternative is to copy an installation directly into your system. To do this, you will need to download the installation from us, and copy it into your C drive.

1. Join our [Discord server](https://discord.gg/cQ4gWxN) and ask for the link to the Qt6 standalone. Someone will DM you with a link*.
1. Join our [Discord server](https://discord.gg/cQ4gWxN) and ask for the link to the Qt6 standalone. Someone will DM you with a link\*.
2. Download `Qt6.8.3.7z` and decompress it. You can use [7-zip](https://www.7-zip.org/) to decompress it. This will create a folder with the same name.
3. Move this folder to `C:\`. It will probably ask you for permissions to do it.
4. Navigate to: `C:\Qt6.8.3\Tools\QtCreator\bin\` and create a shortcut to `qtcreator.exe`. Copy this shortcut to somewhere convenient. (By default this shortcut is named, `Qt Creator 16.0.0 (Community)`)

*This Qt6 standalone file is 3GB in size and is being hosted by our staff for our own developers. We don't want the entire world converging here and overrunning the server.
\*This Qt6 standalone file is 3GB in size and is being hosted by our staff for our own developers. We don't want the entire world converging here and overrunning the server.

## Setup:

Expand All @@ -62,8 +62,8 @@ If you are unable or unwilling to use the online installer, the alternative is t
5. Click on `File` -> `Open File or Project`.
6. Navigate to `SerialPrograms` and select `CMakeLists.txt`.
7. It will then ask you to configure the project. Ensure `Desktop Qt 6.8.3 MSVC2022 64bit` and `Release with Debug Information` are selected.
- Ensure the build directory for `Desktop_Qt_6_8_2_MSVC2022_64bit-RelWithDebInfo` is located in the root of the `Arduino-Source` repo, and not buried in subfolders (e.g. `build` or `SerialPrograms`).
- Click `Configure Project`.
- Ensure the build directory for `Desktop_Qt_6_8_2_MSVC2022_64bit-RelWithDebInfo` is located in the root of the `Arduino-Source` repo, and not buried in subfolders (e.g. `build` or `SerialPrograms`).
- Click `Configure Project`.

![](Images/Windows-configure-project-qt-creator-13.png)

Expand All @@ -79,48 +79,48 @@ If you are unable or unwilling to use the online installer, the alternative is t

- Click `Projects` on the left sidebar.
- In `Build directory`, ensure the build directory for `Desktop_Qt_6_8_3_MSVC2022_64bit-RelWithDebInfo` is located in the root of the `Arduino-Source` repo, and not buried in subfolders (e.g. `build` or `SerialPrograms`).
- e.g. change the default build directory from: `Arduino-Source/SerialPrograms/build/Desktop_Qt_6_8_3_MSVC2022_64bit-RelWithDebInfo`
- e.g. change the default build directory from: `Arduino-Source/SerialPrograms/build/Desktop_Qt_6_8_3_MSVC2022_64bit-RelWithDebInfo`
- to: `Arduino-Source/build-SerialPrograms-Desktop_Qt_6_8_3_MSVC2022_64bit-RelWithDebInfo`


## Upgrading Qt components

### Installing newer Qt version

If you have already have older versions of Qt installed, but need to upgrade Qt components, you can use the Maintenance tool, which comes pre-instaled with Qt (C:\Qt\MaintenanceTool.exe).

- When prompted, in `Maintenance Actions`, choose `Add or remove components`.
- You may select the components needed, as per the instructions above, for the newer version. And de-select old versions to remove them, if you wish.
- Optional: In `Maintenance Actions`, you may also choose `Update components` as well. To update Qt Creator, and other components.

### Building with the newer Qt version

- Open the `SerialPrograms` project in Qt, as outlined in the "Setup" section above.
- In the toolbar, click `Edit` -> `Preferences`
- Under `Kits` tab, choose the newer Qt version (e.g. Desktop Qt 6.8.3)
<img src="Images/QT-kits.png">
<img src="Images/QT-kits.png">

- Under `Compiler`, set it to Microsoft Visual C++ Compiler or Visual Studio, if not already set.
- Click `Ok`, to save Kits preferences.
- Click the Monitor symbol at the bottom left
<img src="Images/QT-kit-build.png">
<img src="Images/QT-kit-build.png">

- Under `Kit`: choose the desired Qt version.
- Under `Build`: choose `Release with Debug Information`
- Click the green arrow to compile and launch the program.
- Optional: Move the `UserSettings` folder from the old build folder to the new build folder, to keep your old settings, such as the dev key.

### Upgrade compiler

- Upgrade the compiler by installing a new version of Visual Studio. Ensure `Desktop development with C++` is checked, so the latest MSVC compiler is also installed.
- Open the `SerialPrograms` project in Qt, as outlined in the "Setup" section above.
- In the toolbar, click `Edit` -> `Preferences`
- Click the `Kits` tab on the left. Then click the `Compilers` tab, click `Re-detect` to refresh the list of compilers. Then click `Apply` in the bottom right.
- Click the `Kits` tab on the left. Then click the `Compilers` tab, click `Re-detect` to refresh the list of compilers. Then click `Apply` in the bottom right.
- Back in the `Kits` tab, your newly installed compilers should now show up in the list of compilers.

When building, if you get the error `yvals_core.h:931: error: STL1001: Unexpected compiler version, expected MSVC 19.42 or newer` (or something similar), consider deleting the old build folder, and rebuilding from scratch.

<hr>

**Discord Server:**

**Discord Server:**

[<img src="https://canary.discordapp.com/api/guilds/695809740428673034/widget.png?style=banner2">](https://discord.gg/cQ4gWxN)

2 changes: 1 addition & 1 deletion SerialPrograms/Source/CommonFramework/Globals.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ namespace PokemonAutomation{

const bool IS_BETA_VERSION = true;
const int PROGRAM_VERSION_MAJOR = 0;
const int PROGRAM_VERSION_MINOR = 58;
const int PROGRAM_VERSION_MINOR = 59;
const int PROGRAM_VERSION_PATCH = 3;

const std::string PROGRAM_VERSION_BASE =
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

#include <stdint.h>
#include "Common/Compiler.h"
#include "Common/Cpp/ImageResolution.h"

namespace PokemonAutomation{

Expand All @@ -35,6 +36,7 @@ class ImageViewPlanar32{
PA_FORCE_INLINE size_t bytes_per_row () const{ return m_bytes_per_row; }
PA_FORCE_INLINE size_t width () const{ return m_width; }
PA_FORCE_INLINE size_t height () const{ return m_height; }
PA_FORCE_INLINE Resolution size () const{ return Resolution(m_width, m_height); }
PA_FORCE_INLINE size_t total_pixels () const{ return m_width * m_height; }

// Direct Pixel Access
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ const std::map<std::string, std::string> STATS_DATABASE_ALIASES{
{"Shiny Hunt Autonomous - Fishing", "PokemonSwSh:ShinyHuntAutonomousFishing"},
{"Shiny Hunt Autonomous - Overworld", "PokemonSwSh:ShinyHuntAutonomousOverworld"},
{"PokemonSV:StatsResetBloodmoon", "PokemonSV:StatsResetEventBattle"},
{"PokemonLZA:ShinyHunt-Bench", "PokemonLZA:ShinyHunt-BenchSit"},
};


Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,8 @@ WaterfillTemplateMatcher::WaterfillTemplateMatcher(
}
}

double WaterfillTemplateMatcher::rmsd(const ImageViewRGB32& image) const{
if (!image || !check_image(image)){
double WaterfillTemplateMatcher::rmsd(Resolution input_resolution, const ImageViewRGB32& image) const{
if (!image || !check_image(input_resolution, image)){
return 99999.;
}
return m_matcher->rmsd(image);
Expand Down Expand Up @@ -125,7 +125,11 @@ bool WaterfillTemplateMatcher::check_area_ratio(double candidate_area_ratio) con

return pass;
}
double WaterfillTemplateMatcher::rmsd_precropped(const ImageViewRGB32& cropped_image, const WaterfillObject& object) const{
double WaterfillTemplateMatcher::rmsd_precropped(
Resolution input_resolution,
const ImageViewRGB32& cropped_image,
const WaterfillObject& object
) const{

// XXX
// dump_debug_image(global_logger_command_line(), "CommonFramework/WaterfillTemplateMatcher", "rmsd_precropped_input", cropped_image);
Expand All @@ -142,7 +146,7 @@ double WaterfillTemplateMatcher::rmsd_precropped(const ImageViewRGB32& cropped_i
// static int c = 0;
// cout << c << endl;

double rmsd = this->rmsd(cropped_image);
double rmsd = this->rmsd(input_resolution, cropped_image);

// cout << "rmsd = " << rmsd << endl;

Expand All @@ -153,7 +157,11 @@ double WaterfillTemplateMatcher::rmsd_precropped(const ImageViewRGB32& cropped_i

return rmsd;
}
double WaterfillTemplateMatcher::rmsd_original(const ImageViewRGB32& original_image, const WaterfillObject& object) const{
double WaterfillTemplateMatcher::rmsd_original(
Resolution input_resolution,
const ImageViewRGB32& original_image,
const WaterfillObject& object
) const{

if (PreloadSettings::debug().IMAGE_TEMPLATE_MATCHING){
cout << "rmsd_original()" << endl;
Expand All @@ -172,7 +180,10 @@ double WaterfillTemplateMatcher::rmsd_original(const ImageViewRGB32& original_im
return 99999.;
}

double rmsd = this->rmsd(extract_box_reference(original_image, object));
double rmsd = this->rmsd(
input_resolution,
extract_box_reference(original_image, object)
);
if (PreloadSettings::debug().IMAGE_TEMPLATE_MATCHING){
cout << "Passed aspect and area ratio check, rmsd = " << rmsd << endl;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

#include <memory>
#include "Common/Cpp/Color.h"
#include "Common/Cpp/ImageResolution.h"
#include "CommonFramework/ImageTools/ImageBoxes.h"
#include "CommonTools/ImageMatch/ExactImageMatcher.h"

Expand Down Expand Up @@ -51,29 +52,37 @@ class WaterfillTemplateMatcher{
// In case the image is invalid, return a large value.
// It also calls the virtual function `check_image()` on the image.
// If the function returns false, then return a large value.
double rmsd(const ImageViewRGB32& image) const;
double rmsd(Resolution input_resolution, const ImageViewRGB32& image) const;

// Compute RMSD of the object on the image against the template.
// The input `cropped_image` is already cropped from a full image using the bounding box of the input waterfill `object`.
// This cropped image is compared against the template as-is.
// The waterfill object's aspect ratio and area ratio are checked against template's. Return a large value
// if the check fails.
// See `double rmsd(const ImageViewRGB32& image) const` on the details of comparing the image against the template.
virtual double rmsd_precropped(const ImageViewRGB32& cropped_image, const WaterfillObject& object) const;
virtual double rmsd_precropped(
Resolution input_resolution,
const ImageViewRGB32& cropped_image,
const WaterfillObject& object
) const;

// Compute RMSD of the object on the image against the template.
// It will crop the original image using the bounding box of the waterfill object, then compare the cropped
// image against the template as-is.
// The waterfill object's aspect ratio and area ratio are checked against template's. Return a large value
// if the check fails.
// See `double rmsd(const ImageViewRGB32& image) const` on the details of comparing the image against the template.
virtual double rmsd_original(const ImageViewRGB32& original_image, const WaterfillObject& object) const;
virtual double rmsd_original(
Resolution input_resolution,
const ImageViewRGB32& original_image,
const WaterfillObject& object
) const;

// Return the image template mesh
const ImageRGB32& image_template() const { return m_matcher->image_template(); }

protected:
virtual bool check_image(const ImageViewRGB32& image) const{ return true; };
virtual bool check_image(Resolution input_resolution, const ImageViewRGB32& image) const{ return true; };
bool check_aspect_ratio(size_t candidate_width, size_t candidate_height) const;
bool check_area_ratio(double candidate_area_ratio) const;

Expand Down
17 changes: 12 additions & 5 deletions SerialPrograms/Source/CommonTools/Images/WaterfillUtilities.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,10 +74,11 @@ std::pair<PackedBinaryMatrix, size_t> remove_center_pixels(
}

bool match_template_by_waterfill(
const ImageViewRGB32 &image,
const ImageMatch::WaterfillTemplateMatcher &matcher,
const std::vector<std::pair<uint32_t, uint32_t>> &filters,
const std::pair<size_t, size_t> &area_thresholds,
Resolution input_resolution,
const ImageViewRGB32& image,
const ImageMatch::WaterfillTemplateMatcher& matcher,
const std::vector<std::pair<uint32_t, uint32_t>>& filters,
const std::pair<size_t, size_t>& area_thresholds,
double rmsd_threshold,
std::function<bool(Kernels::Waterfill::WaterfillObject& object)> check_matched_object)
{
Expand Down Expand Up @@ -133,12 +134,18 @@ bool match_template_by_waterfill(
if (object.area > area_thresholds.second){
continue;
}
double rmsd = matcher.rmsd_original(image, object);
double rmsd = matcher.rmsd_original(input_resolution, image, object);
if (PreloadSettings::debug().IMAGE_TEMPLATE_MATCHING){
std::cout << "Object rmsd: " << rmsd << std::endl;
}

if (rmsd < rmsd_threshold){
#if 0
std::cout << "Object rmsd: " << rmsd << std::endl;
// static int c = 0;
// extract_box_reference(image, object).save("match-" + std::to_string(c++) + ".png");
#endif

detected = true;

if (check_matched_object(object)){
Expand Down
8 changes: 5 additions & 3 deletions SerialPrograms/Source/CommonTools/Images/WaterfillUtilities.h
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

#include <functional>
#include <utility>
#include "Common/Cpp/ImageResolution.h"
#include "CommonFramework/ImageTypes/BinaryImage.h"

namespace PokemonAutomation{
Expand Down Expand Up @@ -54,10 +55,11 @@ std::pair<PackedBinaryMatrix, size_t> remove_center_pixels(
// check_matched_object: if a matcher is found, pass the matched object to this function. If the function returns true, stop the
// entire template matching operation.
bool match_template_by_waterfill(
const ImageViewRGB32 &image,
const ImageMatch::WaterfillTemplateMatcher &matcher,
Resolution input_resolution,
const ImageViewRGB32& image,
const ImageMatch::WaterfillTemplateMatcher& matcher,
const std::vector<std::pair<uint32_t, uint32_t>> &filters,
const std::pair<size_t, size_t> &area_thresholds,
const std::pair<size_t, size_t>& area_thresholds,
double rmsd_threshold,
std::function<bool(Kernels::Waterfill::WaterfillObject& object)> check_matched_object);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@
#include "PokemonLGPE/Inference/Battles/PokemonLGPE_BattleArrowDetector.h"
#include "PokemonLZA/Inference/PokemonLZA_DialogDetector.h"
#include "PokemonLZA/Programs/PokemonLZA_GameEntry.h"
#include "PokemonLZA/Programs/PokemonLZA_BasicNavigation.h"



Expand Down Expand Up @@ -274,6 +275,7 @@ void TestProgram::program(MultiSwitchProgramEnvironment& env, CancellableScope&
// using namespace PokemonBDSP;
// using namespace PokemonLA;
// using namespace PokemonSV;
using namespace PokemonLZA;

[[maybe_unused]] Logger& logger = env.logger();
[[maybe_unused]] ConsoleHandle& console = env.consoles[0];
Expand All @@ -284,7 +286,13 @@ void TestProgram::program(MultiSwitchProgramEnvironment& env, CancellableScope&
VideoOverlaySet overlays(overlay);


while (true){
sit_on_bench(console, context);
}



#if 0
while (true){
ssf_press_button(context, BUTTON_ZL, 160ms, 800ms, 200ms);
ssf_press_button(context, BUTTON_PLUS, 320ms, 840ms);
Expand All @@ -293,7 +301,7 @@ void TestProgram::program(MultiSwitchProgramEnvironment& env, CancellableScope&
pbf_press_button(context, BUTTON_X, 80ms, 24ms);
pbf_press_button(context, BUTTON_B, 80ms, 24ms);
}

#endif


#if 0
Expand Down
4 changes: 3 additions & 1 deletion SerialPrograms/Source/PanelLists.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ ProgramSelect::ProgramSelect(QWidget& parent, PanelHolder& holder)
add(std::make_unique<NintendoSwitch::PokemonSV::PanelListFactory>());

add(std::make_unique<NintendoSwitch::PokemonLZA::PanelListFactory>());
add(std::make_unique<NintendoSwitch::PokemonRSE::PanelListFactory>());
if (PreloadSettings::instance().DEVELOPER_MODE){
add(std::make_unique<NintendoSwitch::PokemonRSE::PanelListFactory>());
}

add(std::make_unique<NintendoSwitch::ZeldaTotK::PanelListFactory>());

Expand Down
6 changes: 3 additions & 3 deletions SerialPrograms/Source/PokemonBDSP/PokemonBDSP_Settings.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,17 @@ GameSettings::GameSettings()
)
, m_start_game_timings("<font size=4><b>Start Game Timings:</b></font>")
, START_GAME_WAIT0(
"<b>2. Start Game Wait:</b><br>Wait this long for the game to load.",
"<b>Start Game Wait:</b><br>Wait this long for the game to load.",
LockMode::LOCK_WHILE_RUNNING,
"300 s"
)
, ENTER_GAME_MASH0(
"<b>3. Enter Game Mash:</b><br>Mash A for this long to enter the game.",
"<b>Enter Game Mash:</b><br>Mash A for this long to enter the game.",
LockMode::LOCK_WHILE_RUNNING,
"5000 ms"
)
, ENTER_GAME_WAIT0(
"<b>4. Enter Game Wait:</b><br>Wait this long for the game to enter the overworld.",
"<b>Enter Game Wait:</b><br>Wait this long for the game to enter the overworld.",
LockMode::LOCK_WHILE_RUNNING,
"300 s"
)
Expand Down
Loading