GH-3068: Remove `parquet-scala` #3063

Fokko · 2024-11-15T09:40:47Z

Rationale for this change

Based on the [DISCUSS] thread: https://lists.apache.org/thread/scdq9t2gvvs4glhq0qx4qcvfp62j793s

Closes #3068

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

pan3793 · 2024-11-15T10:01:34Z

Can we leave some words on README or somewhere to mention this removal?

Fokko · 2024-11-15T10:29:52Z

@pan3793 I noticed that the Scala DSL wasn't mentioned to the list, I've added it 👍

wgtmac · 2024-11-18T01:34:37Z

It seems that recent builds all fail with the following error:

[INFO] ---------------< org.apache.parquet:parquet-scala_2.12 >----------------
[INFO] Building Apache Parquet Scala 1.15.0-SNAPSHOT                    [16/18]
[INFO]   from parquet-scala/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Downloading from scala-tools.org: https://scala-tools.org/repo-releases/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom
Warning:  Checksum validation failed, expected '<!DOCTYPE' (REMOTE_EXTERNAL) but is actually '3ee5a4fdb85dc0b9c676a21a40b6f1cc2d748a03' from scala-tools.org for https://scala-tools.org/repo-releases/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom
Warning:  Could not validate integrity of download from https://scala-tools.org/repo-releases/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom
org.eclipse.aether.transfer.ChecksumFailureException: Checksum validation failed, expected '<!DOCTYPE' (REMOTE_EXTERNAL) but is actually 'e0509cfe82a6ffad07497a39b82d6e0a82a5b1ac'
    at org.eclipse.aether.connector.basic.ChecksumValidator.validateExternalChecksums (ChecksumValidator.java:179)
    at org.eclipse.aether.connector.basic.ChecksumValidator.validate (ChecksumValidator.java:109)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:484)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:414)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:260)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads (DefaultArtifactResolver.java:537)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve (DefaultArtifactResolver.java:449)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts (DefaultArtifactResolver.java:261)
    at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact (DefaultArtifactResolver.java:243)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:241)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:182)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.readArtifactDescriptor (DefaultRepositorySystem.java:269)
    at org.apache.maven.plugin.internal.DefaultPluginDependenciesResolver.resolve (DefaultPluginDependenciesResolver.java:107)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.lambda$getPluginDescriptor$0 (DefaultMavenPluginManager.java:183)
    at org.apache.maven.plugin.DefaultPluginDescriptorCache.lambda$get$0 (DefaultPluginDescriptorCache.java:72)
    at java.util.concurrent.ConcurrentHashMap.computeIfAbsent (ConcurrentHashMap.java:1660)
    at org.apache.maven.plugin.DefaultPluginDescriptorCache.get (DefaultPluginDescriptorCache.java:70)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getPluginDescriptor (DefaultMavenPluginManager.java:181)
    at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getMojoDescriptor (DefaultMavenPluginManager.java:271)
    at org.apache.maven.plugin.DefaultBuildPluginManager.getMojoDescriptor (DefaultBuildPluginManager.java:214)
    at org.apache.maven.lifecycle.internal.DefaultLifecycleMappingDelegate.calculateLifecycleMappings (DefaultLifecycleMappingDelegate.java:101)
    at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateLifecycleMappings (DefaultLifecycleExecutionPlanCalculator.java:239)
    at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateMojoExecutions (DefaultLifecycleExecutionPlanCalculator.java:197)
    at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateExecutionPlan (DefaultLifecycleExecutionPlanCalculator.java:119)
    at org.apache.maven.lifecycle.internal.DefaultLifecycleExecutionPlanCalculator.calculateExecutionPlan (DefaultLifecycleExecutionPlanCalculator.java:135)
    at org.apache.maven.lifecycle.internal.builder.BuilderCommon.resolveBuildPlan (BuilderCommon.java:93)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:100)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:73)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:53)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:118)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:261)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:173)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:101)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:903)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:280)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:203)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:255)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:201)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:361)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:314)
Warning:  Checksum validation failed, expected '<!DOCTYPE' (REMOTE_EXTERNAL) but is actually 'e0509cfe82a6ffad07497a39b82d6e0a82a5b1ac' from scala-tools.org for https://scala-tools.org/repo-releases/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom
[INFO] Downloaded from scala-tools.org: https://scala-tools.org/repo-releases/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom (23 kB at 30 kB/s)
Warning:  The POM for net.alchim31.maven:scala-maven-plugin:jar:4.9.1 is invalid, transitive dependencies (if any) will not be available: 1 problem was encountered while building the effective model
[FATAL] Non-parseable POM /home/runner/.m2/repository/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom: attribute value must start with quotation or apostrophe not M (position: START_DOCUMENT seen <!DOCTYPE html><html lang="en"  data-adblockkey=M... @1:50)  @ line 1, column 50

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Apache Parquet Java 1.15.0-SNAPSHOT:
[INFO] 
[INFO] Apache Parquet Java ................................ SUCCESS [  9.926 s]
[INFO] Apache Parquet Format Structures ................... SUCCESS [  7.546 s]
[INFO] Apache Parquet Generator ........................... SUCCESS [  1.785 s]
[INFO] Apache Parquet Common .............................. SUCCESS [  3.653 s]
[INFO] Apache Parquet Encodings ........................... SUCCESS [  9.611 s]
[INFO] Apache Parquet Column .............................. SUCCESS [ 11.506 s]
[INFO] Apache Parquet Arrow ............................... SUCCESS [  2.374 s]
[INFO] Apache Parquet Jackson ............................. SUCCESS [  1.731 s]
[INFO] Apache Parquet Hadoop .............................. SUCCESS [ 15.833 s]
[INFO] Apache Parquet Avro ................................ SUCCESS [  9.508 s]
[INFO] Apache Parquet Benchmarks .......................... SUCCESS [ 16.404 s]
[INFO] Apache Parquet Command-line ........................ SUCCESS [  9.864 s]
[INFO] Apache Parquet Pig ................................. SUCCESS [  7.967 s]
[INFO] Apache Parquet Pig Bundle .......................... SUCCESS [  2.244 s]
[INFO] Apache Parquet Protobuf ............................ SUCCESS [ 10.107 s]
[INFO] Apache Parquet Scala ............................... FAILURE [  0.791 s]
[INFO] Apache Parquet Thrift .............................. SKIPPED
[INFO] Apache Parquet Hadoop Bundle ....................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  02:01 min
[INFO] Finished at: 2024-11-17T02:48:05Z
[INFO] ------------------------------------------------------------------------
Error:  Plugin net.alchim31.maven:scala-maven-plugin:4.9.1 or one of its dependencies could not be resolved:
Error:  	1 problem was encountered while building the effective model
Error:  [FATAL] Non-parseable POM /home/runner/.m2/repository/net/alchim31/maven/scala-maven-plugin/4.9.1/scala-maven-plugin-4.9.1.pom: attribute value must start with quotation or apostrophe not M (position: START_DOCUMENT seen <!DOCTYPE html><html lang="en"  data-adblockkey=M... @1:50)  @ line 1, column 50
Error:  -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
Error: Process completed with exit code 1.

See: https://github.com/apache/parquet-java/actions/runs/11875285553/job/33092288845?pr=3064

Should we move forward with this PR?

Fokko · 2024-11-18T08:29:44Z

It looks like the Scala version is not available anymore. I would be in favor of removing the module.

I think there are no objections on the dev list either: https://lists.apache.org/thread/scdq9t2gvvs4glhq0qx4qcvfp62j793s

wgtmac

+1

Perhaps we need to create an issue for this since it is not a minor one?

Fokko · 2024-11-18T08:52:30Z

@wgtmac Good point, I've added an issue 👍

Fokko · 2024-11-20T11:31:16Z

Let's move this forward to the CI works again, thanks for the review @pan3793 and @wgtmac 👍

Fokko mentioned this pull request Nov 15, 2024

Bump Scala to 2.12.20 #3044

Closed

Fokko force-pushed the fd-remove-scala branch from 00a4573 to 0a83956 Compare November 15, 2024 10:28

pan3793 approved these changes Nov 15, 2024

View reviewed changes

wgtmac approved these changes Nov 18, 2024

View reviewed changes

apacheGH-3068: Remove parquet-scala

25589ad

Fokko force-pushed the fd-remove-scala branch from 0a83956 to 25589ad Compare November 18, 2024 08:51

Fokko changed the title ~~Remove parquet-scala~~ GH-3068: Remove parquet-scala Nov 18, 2024

Fokko merged commit 1e04ec7 into apache:master Nov 20, 2024
9 checks passed

Fokko deleted the fd-remove-scala branch November 20, 2024 11:31

ConeyLiu pushed a commit to ConeyLiu/parquet-mr that referenced this pull request Nov 26, 2024

apacheGH-3068: Remove parquet-scala (apache#3063)

87aaac8

ConeyLiu pushed a commit to ConeyLiu/parquet-mr that referenced this pull request Nov 26, 2024

apacheGH-3068: Remove parquet-scala (apache#3063)

18e3aab

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

GH-3068: Remove `parquet-scala` #3063

GH-3068: Remove `parquet-scala` #3063

Uh oh!

Fokko commented Nov 15, 2024 •

edited

Loading

Uh oh!

pan3793 commented Nov 15, 2024

Uh oh!

Fokko commented Nov 15, 2024

Uh oh!

wgtmac commented Nov 18, 2024

Uh oh!

Fokko commented Nov 18, 2024

Uh oh!

wgtmac left a comment

Uh oh!

Fokko commented Nov 18, 2024

Uh oh!

Uh oh!

Fokko commented Nov 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

GH-3068: Remove parquet-scala #3063

GH-3068: Remove parquet-scala #3063

Uh oh!

Conversation

Fokko commented Nov 15, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

pan3793 commented Nov 15, 2024

Uh oh!

Fokko commented Nov 15, 2024

Uh oh!

wgtmac commented Nov 18, 2024

Uh oh!

Fokko commented Nov 18, 2024

Uh oh!

wgtmac left a comment

Choose a reason for hiding this comment

Uh oh!

Fokko commented Nov 18, 2024

Uh oh!

Uh oh!

Fokko commented Nov 20, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

GH-3068: Remove `parquet-scala` #3063

GH-3068: Remove `parquet-scala` #3063

Fokko commented Nov 15, 2024 •

edited

Loading