Skip to content

Commit 5c78f6f

Browse files
committed
Merge branch 'main' of github.com:ClojureCivitas/clojurecivitas.github.io
2 parents 2fa5bbf + 8c4b258 commit 5c78f6f

File tree

21 files changed

+1400
-68
lines changed

21 files changed

+1400
-68
lines changed

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,4 +35,5 @@ _site/
3535
.calva/repl.calva-repl
3636
.calva/mcp-server/port
3737
.portal/vs-code.edn
38-
**/password.edn
38+
**/password.edn
39+
/src/finance/portfolio_analysis/*.edn

deps.edn

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
clj-thamil/clj-thamil {:mvn/version "0.2.0"}
1414
org.scicloj/clay {#_#_:mvn/version "2-beta54"
1515
:git/url "https://github.com/scicloj/clay.git"
16-
:git/sha "8734a5e1d5ccd255424ab174187093a66658a954"}
16+
:git/sha "26fb97b5ea2f72a98f8a64d5b8b8abf705767a39"}
1717
org.eclipse.elk/org.eclipse.elk.core {:mvn/version "0.10.0"}
1818
org.eclipse.elk/org.eclipse.elk.graph {:mvn/version "0.10.0"}
1919
org.eclipse.elk/org.eclipse.elk.graph.json {:mvn/version "0.10.0"}
@@ -38,7 +38,8 @@
3838
org.lwjgl/lwjgl-stb$natives-windows {:mvn/version "3.3.6"}
3939
org.lwjgl/lwjgl-stb$natives-macos {:mvn/version "3.3.6"}
4040
generateme/fastmath {:mvn/version "2.4.0"
41-
:exclusions [com.github.haifengl/smile-mkl org.bytedeco/openblas]}}
41+
:exclusions [com.github.haifengl/smile-mkl org.bytedeco/openblas]}
42+
clj-http/clj-http {:mvn/version "3.13.1"}}
4243

4344
:aliases
4445
{;; Build the site with `clojure -M:clay -A:markdown`

site/data_engineering/support_for_popular_data_tools/snowflake.qmd

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -45,8 +45,8 @@ If I couldn't find any such library at all, I classified it as having "no suppor
4545
| Tool Category | Supported | Support Unknown | No Support |
4646
|---------------|---------------------|--------------|---------------|
4747
| **On-cluster batch processing** | | 1. [Spark](https://spark.apache.org) (see [Spark Interop with Geni](#spark_interop_with_geni) below) | |
48-
| **On-cluster stream processing** | | 2. [Kafka Streams](https://kafka.apache.org/documentation/streams) (see [Kafka Interop with Jackdaw](#kafka_interop_with_jackdaw) below) | 3. [Spark Structured Streaming](https://spark.apache.org/streaming),\<br\>4. [Flink](https://flink.apache.org) |
49-
| **On-cluster batch and stream processing** | | | 5. [Databricks](https://databricks.com) (see [Spark Interop with Geni](#spark_interop_with_geni) below),\<br\>6. [Snowflake](https://snowflake.com) (see [Snowflake Interop](#snowflake_interop_with_a_new_clojure_api!) below) |
48+
| **On-cluster stream processing** | | 2. [Kafka Streams](https://kafka.apache.org/documentation/streams) (see [Kafka Interop with Jackdaw](#kafka_interop_with_jackdaw) below) | 3. [Spark Structured Streaming](https://spark.apache.org/streaming),<br>4. [Flink](https://flink.apache.org) |
49+
| **On-cluster batch and stream processing** | | | 5. [Databricks](https://databricks.com) (see [Spark Interop with Geni](#spark_interop_with_geni) below),<br>6. [Snowflake](https://snowflake.com) (see [Snowflake Interop](#snowflake_interop_with_a_new_clojure_api!) below) |
5050

5151
Please note, I don't wish to make any critical judgments based on either the summary analysis above or the more detailed analysis below. The goal is to understand the situation with respect to Clojure support and highlight any gaps, although I suppose I am also inadvertently highlighting the difficulties of maintaining open source software!
5252

@@ -63,8 +63,9 @@ However, I found a number of issues that would need to be addressed in order to
6363
1. Problems with [support for Databricks](https://github.com/zero-one-group/geni/issues/356), although I suspect this is related to point 1.
6464

6565
Also, in general by my criteria the support classification is "support unknown":
66+
6667
1. CI/CD build [failing.](https://github.com/zero-one-group/geni/actions)
67-
1. Version [0.0.42 api docs broken](https://cljdoc.org/d/zero.one/geni/0.0.42/doc/readme%20%20https://cljdoc.org/builds/73977), also affects version 0.0.41
68+
1. Version [0.0.42 api docs](https://cljdoc.org/d/zero.one/geni/0.0.42/doc/readme) are [broken](https://cljdoc.org/builds/73977), also affects version 0.0.41
6869
1. No commits since November 2023.
6970
1. No releases since November 2023.
7071
1. No PRs accepted or rejected since November 2023.
@@ -82,9 +83,9 @@ Also, in general by my criteria the support classification is "support unknown":
8283

8384
### Snowflake Interop with a New Clojure API!
8485

85-
Although the [Snowpark](https://docs.snowflake.com/en/developer-guide/snowpark/java) library has Java and Scala bindings, it doesn't provide anything for Clojure. As such, it's currently not possible to interact with Snowflake using the Clojure way.
86+
Although the [Snowpark](https://docs.snowflake.com/en/developer-guide/snowpark/java/index) library has Java and Scala bindings, it doesn't provide anything for Clojure. As such, it's currently not possible to interact with Snowflake using the Clojure way.
8687

87-
To address this gap, I decided to try my hand at creating a [Clojure API for Snowflake](https://github.com/alza-bitz/snowflake-clj) as part of a broader effort to improve the overall situation regarding Clojure support for popular data tools.
88+
To address this gap, I decided to try my hand at creating a [Clojure API for Snowflake](https://github.com/alza-bitz/snowpark-clj) as part of a broader effort to improve the overall situation regarding Clojure support for popular data tools.
8889

8990
The aim is to validate this approach as a foundation for enabling a wide range of data science or data engineering use cases from the Clojure REPL, in situations where Snowflake is the data warehouse of choice.
9091

site/db.edn

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -104,7 +104,24 @@
104104
:name "Jan Wedekind"
105105
:url "https://www.wedesoft.de/"
106106
:image "https://avatars.githubusercontent.com/u/28663?v=4"
107-
:links [{:icon "github" :href "https://github.com/wedesoft"}]}]
107+
:links [{:icon "github" :href "https://github.com/wedesoft"}]}
108+
{:id :kloimhardt
109+
:name "Markus Agwin Kloimwieder"
110+
:url "https://github.com/kloimhardt"
111+
:affiliation [:scicloj]}
112+
{:id :edwardaw
113+
:name "Edward Widjaja"
114+
:url "https://github.com/W-Edward"
115+
:image "https://avatars.githubusercontent.com/u/36920869?v=4"
116+
:email "edwardaw@connect.hku.hk"
117+
:affiliation [:cim]
118+
:links [{:icon "github" :href "https://github.com/W-Edward"}]}
119+
{:id :mattb
120+
:name "Matthias Buehlmaier"
121+
:affiliation [:cim]}
122+
{:id :tanvi
123+
:name "Tanvi Nagar"
124+
:affiliation [:cim]}]
108125

109126
:affiliation
110127
[{:id :clojure.core
@@ -122,7 +139,11 @@
122139
:url "https://hummi.app"}
123140
{:id :techascent
124141
:name "TechAscent"
125-
:url "https://techascent.com"}]
142+
:url "https://techascent.com"}
143+
{:id :cim
144+
:name "cim"
145+
:url "https://cim.hkubs.hku.hk/"
146+
}]
126147

127148
:topic
128149
[{:id :core

0 commit comments

Comments
 (0)