Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 92 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,109 @@

# Minecraft Mavenizer

A pure-blooded Java tool to generate a maven repository for Minecraft artifacts.

## Requirements (Delete on Release)

- Fully-fledged generation of MCPConfig-based artifacts and UserDev.
- This also include the ability to use this tool to potentially replace clean in ForgeDev.
- Code cleanup (strictly API/Main, internal code does not need a mandated cleanup as long as it is maintainable).
- Everything must adhere to the cache system. There are a few steps (albeit cheap ones) that do not use the caching system such as POM and Gradle Module Metadata generation.
- The ability to use MCP mappings (Parchment support can come sometime later, after ForgeDev is able to benefit from new toolchain).
- The groundwork for this using Gradle Module variants is already in place.
- Interface with ModLauncher and other transformer services that want to interact with Minecraft artifacts before Gradle artifact transformers.
## Preamble

## Purpose
Due to Minecraft being closed source, and (previously) obfusicated it can not be used directly as a library in the form that Mojang releases it.
This is designed to be a stand alone tool that will generate a directory that is suitable to be used as a maven respository containing Minecraft related artifacts.
The basic conept is to replace the hacks into Gradle's internals that our ForgeGradle tool was reqired to do. In doing so disconnects the core functionality from Gradle,
allowing it to be more stable considering how much Gradle likes to change its 'best practices'.

Minecraft Mavenizer solves a long-standing pain point that we've attempted to solve with ForgeGradle in the past: the ability to have a static maven repository for Minecraft artifacts. This tool is designed to be a standalone tool, without the need of a Gradle plugin, to do so.
This tool is designed as a executable jar, **not** a library that can be referenced by other java code. As such the only 'public api' is the command line arguments.
As such anything that can call a executable jar, should be able to use this. Opening the possibility for usage in other build systems such as maven. This is not tested, and may need additional changes. If people want to use this tool in other build systems. Please feel encouraged to file an issue or pull request.

For the case of Minecraft Forge, this tool will be integrated (either as a library or via JavaVersion) by ForgeGradle 7, however the skeleton is set so that even Maven projects can benefit from the Forge/MCP toolchain.
#### Broad Features
- Functional:
- MCPConfig based artifacts: Vanilla Minecraft (1.12.2+)
- UserDev3+ Forge libraries, these are versions of Forge that use ForgeGradle 3+ (Minecraft 1.13+, and some builds of 1.12.2)
- Parchment and Official mappings.
- TODO:
- Legacy Forge versions.
- Forge Gradle has gone through many eras, and before that we had a python based build system. It will take some time but the plan is to eventually support every version of Forge that has been released.
- Support Artifact Transformers allowing ModLauncher/Mixin/Whatever to apply static transformation of artifacts? Probably not needed.
- Deobfuscating arbitrary dependencies. Basically a replacement for `fg.deobf` allowing mods to be used in different mappings then they are released.
- Better cacheing management, and debug information on cache misses.

## Usage

Minecraft Mavenizer is a standalone Java tool that can be invoked through the command line. Here is an example:

```shell
java -jar minecraft-maven-0.1.0.jar --version 1.21.3-53.0.25
java -jar minecraft-maven.jar --version 1.21.3-53.0.25
```

As Forge is the main target of this tool, that is all you need to generate the Forge artifacts.

> [!WARNING]
> **There is no public API for this tool!** This is designed to solely be a CLI tool, which means that all of the implementations are internal. We reserve the right to change the internal implementation at any time.

Mavenizer is seperated into multiple `tasks` specified by the `--maven`, `--mcp`, and `--mcp_data` command line arguments.
The only task consumers should care about is the `--maven` task, and thus is the default if those arguments are omitted. And the only one I'm going to document here. The others are used in ForgeDev, our plugin for making Forge itself.

## Maven task
| Argument | Default | Description
| --------------------- | -------------------------- | ------------
| --version `String` | | The specific artifact version to generate. This is the only required argument.
| --artifact `String` | `net.minecraftforge:forge` | The artifact to generate. |
| --client | | Shorthand for `--artifact net.minecraft:client`
| --server | | Shorthand for `--artifact net.minecraft:server`
| --forge | | Shorthand for `--artifact net.minecraftforge:forge`
| --mapping-data | | Shorthand for `--artifact net.minecraft:mappings`
| --mc | | Shorthand for `--artifact net.minecraft:joined`
| --mappings `String` | `official` | Mappings to use for this artifact. Formatted as `channel:version`. If version is missing, will attempt use the detected `minecraft` version of the artifact.
| --parchment `version` | | Version of parchment mappings to use, snapshots are not supported. Shorthand for `--mappings parchment:version`
| --output `File` | `./output` | Root directory to generate the maven repository.
| --cache `File` | `./cache` | The directory to use for caching things used for building.
| --jdk-cache `File` | `./cache/jdks` | Directory to store jdks downloaded from the disco api.
| --cache-only | | Only use caches, fail if any downloads need to occur or if a task needs to do work.
| --offline | | Allows offline operations, fails if any downloads need to occur.
| --dependencies-only | | Outputs the maven containing only the Gradle Module and POM for the artifact's dependencies without outputting the artifact itself
| --global-auxiliary-variants | | Declares sources and javadoc jars as global variants, no matter the mapping version. This is used to work around gradle/gradle#35065
| --repository `String` | |**EXPERIMENTAL**: URL of a foreign maven repository to use for dependencies. The format is `name,url`. The name must not include any commas.

Supported Artifacts:
- `net.minecraftfroge:forge`
- `net.minecraft:client`
- `net.minecraft:client-extra`
- `net.minecraft:server`
- `net.minecraft:server-extra`
- `net.minecraft:joined`
- `net.minecraft:joined-extra`
- `net.minecraft:mappings`

Supported Mapping Channels:
- `official`: Official mappings provided by Mojang, located by the launcher's version.json file. Only available for versions Mojang has released the mappings for.
- `notch`: Also known as obfusicated, the raw names of the class as Mojang released them. Note: Mojang is intending to remove obfusication, so for versions that don't have obf, `notch` will not be supported.
- `srg` or `searge`: The intermediate names used by MCPConfig to make the code readable. One step above `notch`.
- `parchment`: A layer over the `official` mappings that add parameter names, and javadoc. See https://parchmentmc.org/


## What artifacts are generated?
For all artifacts, the standard maven `pom` file is generated with appropriate dependencies.
For artifacts that support mappings, we also generate a [Gradle Module File](https://docs.gradle.org/current/userguide/publishing_gradle_module_metadata.html) to specify the mapping specific variants.


### For `net.minecraft` `client`, `server`, and `joined`
- `side-version.pom`: Maven metadata pom file containing all dependencies.
- `side-version-metadata.zip`: A zip file containing metadata about this artifact.
It currently only has 2 files in it. version.properties, which contained `version=1` to know what format this zip is in. And `version.json` which is the version.json file from the Minecraft launcher.
- `side-version.jar`: The main archive for this artifact. This supports mappings, as such specific versions are referenced in the module file.
- `side-version-sources.jar`: The sources for this artifact. Not available for `notch`, `srg`, `searge` mappings. This supports mappings, as such specific versions are referenced in the module file.

### For `net.minecraft` `client-extra`, `server-extra`
- `side-version.jar`: This is a legacy file that houses all the `extra` data in the vanilla Minecraft jars. The server typically doesn't have anything in it anymore. But it used to have all the dependency libraries when Mojang released the Server jar as a normal `uberjar`. The client contains all the non-class file entries. Such as images, sounds, language files that may be in the Client vanilla jar file.


### For `net.minecraft:mappings`
- `mappings-version.jar`: A archive containing csv files mapping SRG intermediate names to the specified mapping. This is mainly intended for old versions which use SRG at runtime, and need to runtime remap reflection to mapped names.

### For `net.minecraftforge:forge`:
Currently only UserDev3 files are supported.
Generating this artifact will automatically generate the appropriate `net.minecraft:client-extra`, and `net.minecraft:mappings` artifacts.
- `forge-version.jar`: Main artifact, contains classes and a data for the game. This supports mappings, as such specific versions are referenced in the module file.
- `forge-version-sources.jar`: The sources for this artifact. This supports mappings, as such specific versions are referenced in the module file.
- `forge-version-metadata.zip`: A metadata zip containing information useful for ForgeGradle. Currently contains:
- `version.properties`: A text file containing `version=1`, This will allow us to change the format of files in this archive if needed.
- `runs.json`: Run configuration information from Userdev's config. Useful for SlimeLauncher to actually launch the game.
- `version.json`: The version.json file from the Minecraft launcher.
4 changes: 2 additions & 2 deletions settings.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ gradle.beforeProject { Project project ->
project.repositories {
mavenCentral()
maven project.gradleutils.forgeMaven
//mavenLocal()
mavenLocal()
}
}
}
Expand Down Expand Up @@ -42,7 +42,7 @@ dependencyResolutionManagement.versionCatalogs.register('libs') {
library 'utils-files', 'net.minecraftforge', 'file-utils' version '0.3.2'
library 'utils-hash', 'net.minecraftforge', 'hash-utils' version '0.1.12'
library 'utils-data', 'net.minecraftforge', 'json-data-utils' version '0.4.1'
library 'utils-logging', 'net.minecraftforge', 'log-utils' version '0.5.0'
library 'utils-logging', 'net.minecraftforge', 'log-utils' version '0.5.1'
library 'utils-os', 'net.minecraftforge', 'os-utils' version '0.1.0'
bundle 'utils', ['utils-download', 'utils-files', 'utils-hash', 'utils-data', 'utils-logging', 'utils-os']
}
Expand Down
27 changes: 11 additions & 16 deletions src/main/java/net/minecraftforge/mcmaven/cli/MCPDataTask.java
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,8 @@

// TODO [Mavenizer][MCPDataTask] This is a copy of FG6's ExtractMCPData task.
// its not the best, but I dont want to re-wrok INSTALLER_TOOLS to put the tsrg in the mappings zip
public class MCPDataTask {
public static void run(String[] args) throws Exception {
int ret = runI(args);
if (ret != 0) {
LOGGER.release();
//System.exit(ret);
}
}

private static int runI(String[] args) throws Exception {
class MCPDataTask {
static OptionParser run(String[] args, boolean getParser) throws Exception {
// TODO [MCMavenizer] Make this into a --log [level] option
LOGGER.setEnabled(Logger.Level.INFO);

Expand Down Expand Up @@ -87,10 +79,13 @@ private static int runI(String[] args) throws Exception {
.withRequiredArg();
//@formatter:on

if (getParser)
return parser;

var options = parser.parse(args);
if (options.has(helpO)) {
parser.printHelpOn(LOGGER.getInfo());
return -1;
return parser;
}

var output = options.valueOf(outputO);
Expand All @@ -106,7 +101,7 @@ private static int runI(String[] args) throws Exception {

if (artifact == null) {
LOGGER.error("Missing mcp --version or --artifact");
return -2;
return parser;
}

var mcVersion = MinecraftMaven.mcpToMcVersion(artifact.getVersion());
Expand All @@ -121,7 +116,7 @@ else if (options.has(parchmentO))
var key = options.valueOf(keyO);
if (key == null) {
LOGGER.error("Missing --key option");
return -3;
return parser;
}

var repo = new MCPConfigRepo(new Cache(cacheRoot, jdkCacheRoot), false);
Expand All @@ -144,14 +139,14 @@ else if (options.has(parchmentO))

if (path == null) {
LOGGER.error("Could not find data entry for '%s'".formatted(key));
return -4;
return parser;
}

try (ZipFile zip = new ZipFile(mcp.getData())) {
var entry = zip.getEntry(path);
if (entry == null) {
LOGGER.error("Invalid config zip, missing file: " + path);
return -5;
return parser;
}

if ("mappings".equals(key)) {
Expand Down Expand Up @@ -202,6 +197,6 @@ public String rename(IParameter value) {
}
}

return 0;
return parser;
}
}
17 changes: 10 additions & 7 deletions src/main/java/net/minecraftforge/mcmaven/cli/MCPTask.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;

import joptsimple.OptionParser;
import net.minecraftforge.mcmaven.impl.MinecraftMaven;
import net.minecraftforge.mcmaven.impl.cache.Cache;
Expand All @@ -30,8 +28,8 @@
import org.jetbrains.annotations.Nullable;

// TODO [Mavenizer][MCPTask] Cleanup. Works well but is a mess.
public class MCPTask {
public static void run(String[] args) throws Exception {
class MCPTask {
static OptionParser run(String[] args, boolean getParser) throws Exception {
// TODO [MCMavenizer] Make this into a --log [level] option
LOGGER.setEnabled(Logger.Level.INFO);

Expand Down Expand Up @@ -100,11 +98,14 @@ public static void run(String[] args) throws Exception {
.availableIf(mappingsO).withRequiredArg();
//@formatter:on

if (getParser)
return parser;

var options = parser.parse(args);
if (options.has(helpO)) {
parser.printHelpOn(LOGGER.getInfo());
LOGGER.release();
return;
return parser;
}

var output = options.valueOf(outputO);
Expand All @@ -126,7 +127,7 @@ public static void run(String[] args) throws Exception {
if (artifact == null) {
LOGGER.error("Missing mcp --version or --artifact");
LOGGER.release();
return;
return parser;
}

var repo = new MCPConfigRepo(new Cache(cacheRoot, jdkCacheRoot), false);
Expand Down Expand Up @@ -174,7 +175,7 @@ public static void run(String[] args) throws Exception {
}
}

return;
return parser;
}

var sourcesTask = side.getSources();
Expand Down Expand Up @@ -239,6 +240,8 @@ public static void run(String[] args) throws Exception {
throw new RuntimeException("Failed to generate artifact: %s".formatted(artifact), t);
}
}

return parser;
}

// TODO [Mavenizer][Extra MCPTask Files] do this better
Expand Down
25 changes: 22 additions & 3 deletions src/main/java/net/minecraftforge/mcmaven/cli/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@

import joptsimple.OptionParser;
import joptsimple.OptionSpecBuilder;
import net.minecraftforge.util.logging.Logger;

import static net.minecraftforge.mcmaven.impl.Mavenizer.LOGGER;

import java.time.Duration;
Expand Down Expand Up @@ -40,6 +42,10 @@ private static void run(String[] args) throws Exception {
var tasks = Tasks.values();
var opts = new ArrayList<OptionSpecBuilder>();

var helpO = parser.accepts("help",
"Displays this help message and exits")
.forHelp();

for (var task : tasks)
opts.add(parser.accepts(task.key, task.description));

Expand All @@ -54,12 +60,25 @@ private static void run(String[] args) throws Exception {

for (var task : tasks) {
if (options.has(task.key)) {
task.callback.run(args);
task.callback.run(args, false);
return;
}
}

parser.printHelpOn(LOGGER.getInfo());
LOGGER.release();
if (options.has(helpO)) {
LOGGER.setEnabled(Logger.Level.INFO);
parser.printHelpOn(LOGGER.getInfo());
for (var task : tasks) {
LOGGER.info();
LOGGER.info(task.key + " Task:");
LOGGER.push();
var taskParser = task.callback.run(new String[0], true);
taskParser.printHelpOn(LOGGER.getInfo());
LOGGER.pop();
}
LOGGER.release();
} else { // Default to --maven as that is the main usecase.
Tasks.MAVEN.callback.run(args, false);
}
}
}
9 changes: 7 additions & 2 deletions src/main/java/net/minecraftforge/mcmaven/cli/MavenTask.java
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
import static net.minecraftforge.mcmaven.impl.Mavenizer.LOGGER;

class MavenTask {
static void run(String[] args) throws Exception {
static OptionParser run(String[] args, boolean getParser) throws Exception {
// TODO [MCMavenizer] Make this into a --log [level] option
LOGGER.setEnabled(Logger.Level.INFO);

Expand Down Expand Up @@ -115,11 +115,14 @@ static void run(String[] args) throws Exception {
});
//@formatter:on

if (getParser)
return parser;

var options = parser.parse(args);
if (options.has(helpO)) {
parser.printHelpOn(LOGGER.getInfo());
LOGGER.release();
return;
return parser;
}

// global options
Expand Down Expand Up @@ -160,5 +163,7 @@ static void run(String[] args) throws Exception {

var mcmaven = new MinecraftMaven(output, options.has(dependenciesOnlyO), cache, jdkCache, mappings, foreignRepositories, options.has(globalAuxiliaryVariantsO));
mcmaven.run(artifact);

return parser;
}
}
5 changes: 3 additions & 2 deletions src/main/java/net/minecraftforge/mcmaven/cli/Tasks.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,16 @@

import java.util.Locale;

import joptsimple.OptionParser;

enum Tasks {
MAVEN(MavenTask::run, "Generates a maven repository for Minecraft Artifacts"),
MCP(MCPTask::run, "Generates a 'clean' sources jar from a MCPConfig pipeline"),
MCP_DATA(MCPDataTask::run, "Extracts a data file from a MCPConfig archive")
;

interface Callback {
void run(String[] args) throws Exception;
OptionParser run(String[] args, boolean getParser) throws Exception;
}
final String key;
final Callback callback;
Expand All @@ -24,5 +26,4 @@ private Tasks(Callback callback, String description) {
this.callback = callback;
this.description = description;
}

}