Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .husky/pre-commit
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@

./mvnw sortpom:sort spotless:apply -f .teamcity
./mvnw sortpom:sort spotless:apply

git update-index --again
14 changes: 9 additions & 5 deletions .snyk
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,14 @@ ignore:
reason: Spark Core is provided dependency
expires: 2050-01-01T00:00:00.000Z
created: 2025-09-18T08:33:31.014
- 'org.apache.spark:spark-core_2.12':
<<: *spark-core
'SNYK-JAVA-COMFASTERXMLJACKSONCORE-7569538': *spark-core-exclusions
'SNYK-JAVA-COMGOOGLEPROTOBUF-8055227': *spark-core-exclusions
'SNYK-JAVA-ORGAPACHEIVY-5847858': *spark-core-exclusions
'SNYK-JAVA-ORGAPACHEZOOKEEPER-5961102': *spark-core-exclusions
'SNYK-JAVA-ORGGLASSFISHJERSEYCORE-14049172': *spark-core-exclusions
'SNYK-JAVA-IOAIRLIFT-14412703': &spark-sql-exclusions
- 'org.apache.spark:spark-sql_2.13': &spark-sql
reason: Spark SQL is provided dependency
expires: 2050-01-01T00:00:00.000Z
created: 2025-09-18T08:35:12.345
<<: *spark-sql
'SNYK-JAVA-ORGLZ4-14151788': *spark-core-exclusions
'SNYK-JAVA-ORGLZ4-14219384': *spark-core-exclusions
patch: {}
18 changes: 8 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,35 +10,33 @@ This neo4j-connector-apache-spark is Apache 2 Licensed

The documentation for Neo4j Connector for Apache Spark lives at https://github.com/neo4j/docs-spark repository.

## Building for Spark 3
## Building for Spark 4

You can build for Spark 3.x with both Scala 2.12 and Scala 2.13
You can build for Spark 4.x with Scala 2.13

```
./maven-release.sh package 2.12
./maven-release.sh package 2.13
```

These commands will generate the corresponding targets
* `spark-3/target/neo4j-connector-apache-spark_2.12-<version>_for_spark_3.jar`
* `spark-3/target/neo4j-connector-apache-spark_2.13-<version>_for_spark_3.jar`
* `spark/target/neo4j-connector-apache-spark_2.13-<version>_for_spark_4.jar`


## Integration with Apache Spark Applications

**spark-shell, pyspark, or spark-submit**

`$SPARK_HOME/bin/spark-shell --jars neo4j-connector-apache-spark_2.12-<version>_for_spark_3.jar`
`$SPARK_HOME/bin/spark-shell --jars neo4j-connector-apache-spark_2.12-<version>_for_spark_4.jar`

`$SPARK_HOME/bin/spark-shell --packages org.neo4j:neo4j-connector-apache-spark_2.12:<version>_for_spark_3`
`$SPARK_HOME/bin/spark-shell --packages org.neo4j:neo4j-connector-apache-spark_2.13:<version>_for_spark_4`

**sbt**

If you use the [sbt-spark-package plugin](https://github.com/databricks/sbt-spark-package), in your sbt build file, add:

```scala
resolvers += "Spark Packages Repo" at "http://dl.bintray.com/spark-packages/maven"
libraryDependencies += "org.neo4j" % "neo4j-connector-apache-spark_2.12" % "<version>_for_spark_3"
libraryDependencies += "org.neo4j" % "neo4j-connector-apache-spark_2.13" % "<version>_for_spark_4"
```

**maven**
Expand All @@ -50,8 +48,8 @@ In your pom.xml, add:
<!-- list of dependencies -->
<dependency>
<groupId>org.neo4j</groupId>
<artifactId>neo4j-connector-apache-spark_2.12</artifactId>
<version>[version]_for_spark_3</version>
<artifactId>neo4j-connector-apache-spark_2.13</artifactId>
<version>[version]_for_spark_4</version>
</dependency>
</dependencies>
```
Expand Down
2 changes: 1 addition & 1 deletion common/LICENSES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ libraries. For an overview of the licenses see the NOTICE.txt file.

------------------------------------------------------------------------------
Apache Software License, Version 2.0
IntelliJ IDEA Annotations
JetBrains Java Annotations
Kotlin Stdlib
Neo4j Bolt Connection (Bolt Provider reference impl)
Neo4j Bolt Connection (Pooled Source impl)
Expand Down
2 changes: 1 addition & 1 deletion common/NOTICE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ Third-party licenses
--------------------

Apache Software License, Version 2.0
IntelliJ IDEA Annotations
JetBrains Java Annotations
Kotlin Stdlib
Neo4j Bolt Connection (Bolt Provider reference impl)
Neo4j Bolt Connection (Pooled Source impl)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -734,7 +734,7 @@ class SchemaService(
tx => {
tx.run(
s"CREATE CONSTRAINT $constraintName IF NOT EXISTS FOR $asciiRepresentation REQUIRE ($props) IS $constraintType"
)
).consume()
},
sessionTransactionConfig
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ class ValidationsTest extends SparkConnectorScalaBaseTSE {
.map { _.version }
.getOrElse("UNKNOWN")
try {
Validations.validate(ValidateSparkMinVersion("3.10000"))
Validations.validate(ValidateSparkMinVersion("4.10000"))
fail(s"should be thrown a ${classOf[IllegalArgumentException].getName}")
} catch {
case e: IllegalArgumentException =>
Expand Down
Loading