Skip to content

Commit 02cde8a

Browse files
szeigerszeiger
committed
Merge pull request #13 from SethTisue/better-readme
update and expand readme
2 parents 8873fa5 + 5898d3b commit 02cde8a

File tree

1 file changed

+164
-39
lines changed

1 file changed

+164
-39
lines changed

‎README.md‎

Lines changed: 164 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,188 @@
1-
Sculpt - Dependency graph extraction for Scala
2-
==============================================
1+
# Sculpt: dependency graph extraction for Scala
32

4-
Using the compiler plugin
5-
-------------------------
3+
Sculpt is a compiler plugin for analyzing the dependency structure of
4+
Scala source code.
65

7-
After `copyResources` in sbt, you can use the compiled plugin from another scala compiler instance. For example:
6+
The data generated by the plugin should be useful for all sorts of
7+
refactoring efforts, including carving a monolithic codebase into
8+
independent subprojects.
89

9-
scalac -Xplugin:/Users/szeiger/code/scala-sculpt/target/scala-2.11/classes:\
10-
/Users/szeiger/.ivy2/cache/io.spray/spray-json_2.11/bundles/spray-json_2.11-1.3.2.jar \
11-
-Xplugin-require:sculpt -P:sculpt:out=dep.json Dep.scala
10+
The plugin analyzes source code, not generated bytecode. The analysis
11+
code is the same code used by the incremental compiler in sbt and zinc
12+
([reference](https://github.com/gkossakowski/sbt/wiki/Incremental-compiler-notes#dependency-extraction)).
13+
Therefore, the plugin should be an accurate source of information for
14+
developers looking to reduce dependencies in order to reduce their
15+
incremental recompile times.
1216

13-
Sample interactive session
14-
--------------------------
17+
## Building the plugin from source
1518

16-
Loading a JSON model into the REPL:
19+
`sbt package` will create `target/scala-2.11/scala-sculpt_2.11-0.0.1.jar`.
20+
21+
## Using the plugin
22+
23+
You can use the compiled plugin with the Scala 2.11 compiler as follows.
24+
25+
First, make sure you have `scala-sculpt_2.11-0.0.1.jar` in your current working directory,
26+
along with `spray-json_2.11-1.3.2.jar` (which you can download
27+
[here](http://repo1.maven.org/maven2/io/spray/spray-json_2.11/1.3.2/spray-json_2.11-1.3.2.jar).
28+
29+
Then you can do e.g.:
30+
31+
scalac -Xplugin:scala-sculpt_2.11-0.0.1.jar:spray-json_2.11-1.3.2.jar \
32+
-Xplugin-require:sculpt \
33+
-P:sculpt:out=dep.json \
34+
Dep.scala
35+
36+
## Sample input and output
37+
38+
Assuming `Dep.scala` contains this source code:
39+
40+
```
41+
object Dep1 { final val x = 42 }
42+
object Dep2 { val x = Dep1.x }
43+
```
44+
45+
then the command line shown above will generate this `dep.json` file:
46+
47+
```
48+
[
49+
{"sym": ["o:Dep1"], "extends": ["pkt:scala", "tp:AnyRef"]},
50+
{"sym": ["o:Dep1", "def:<init>"], "uses": ["o:Dep1"]},
51+
{"sym": ["o:Dep1", "def:<init>"], "uses": ["pkt:java", "pkt:lang", "cl:Object", "def:<init>"]},
52+
{"sym": ["o:Dep1", "def:x"], "uses": ["pkt:scala", "cl:Int"]},
53+
{"sym": ["o:Dep1", "t:x"], "uses": ["pkt:scala", "cl:Int"]},
54+
{"sym": ["o:Dep2"], "extends": ["pkt:scala", "tp:AnyRef"]},
55+
{"sym": ["o:Dep2", "def:<init>"], "uses": ["o:Dep2"]},
56+
{"sym": ["o:Dep2", "def:<init>"], "uses": ["pkt:java", "pkt:lang", "cl:Object", "def:<init>"]},
57+
{"sym": ["o:Dep2", "def:x"], "uses": ["o:Dep2", "t:x"]},
58+
{"sym": ["o:Dep2", "def:x"], "uses": ["pkt:scala", "cl:Int"]},
59+
{"sym": ["o:Dep2", "t:x"], "uses": ["pkt:scala", "cl:Int"]}
60+
]
61+
```
62+
63+
Each line in the JSON file represents an edge between two symbols in a
64+
dependency graph.
65+
66+
The edges are of two types, `extends` and `uses`.
67+
68+
Each symbol is represented in the JSON as an array of strings, where
69+
each string represents a part of the symbol's fully qualified name.
70+
71+
So for example, in the above source code, we see that `Dep1` extends
72+
`scala.AnyRef`:
73+
74+
{"sym": ["o:Dep1"], "extends": ["pkt:scala", "tp:AnyRef"]},
75+
76+
And we see that `Dep1` uses `scala.Int` in two places:
77+
78+
{"sym": ["o:Dep1", "def:x"], "uses": ["pkt:scala", "cl:Int"]},
79+
{"sym": ["o:Dep1", "t:x"], "uses": ["pkt:scala", "cl:Int"]},
80+
81+
from this we see that `scala.Int` is used as the declared return type
82+
of `Dep1.x`, and as the inferred type of the body of `Dep1.x`.
83+
84+
For brevity, the following abbreviations are used in the JSON output:
85+
86+
### Terms
87+
88+
abbreviation | meaning
89+
-------------|--------
90+
ov | object
91+
def | def
92+
var | var
93+
mac | macro
94+
pk | package
95+
t | other term
96+
97+
### Types
98+
99+
abbreviation | meaning
100+
-------------|--------
101+
tr | trait
102+
pkt | package
103+
o | object
104+
cl | class
105+
tp | other type
106+
107+
### Other
108+
109+
The name of a constructor is always `<init>`.
110+
111+
## Graphs represented as case classes
112+
113+
The same JAR that contains the plugin also contains a suite of case
114+
classes for representing the same information in the JSON files as
115+
Scala objects.
116+
117+
We provide a `load` method for parsing a JSON file into instances
118+
of these case classes, and a `save` method for writing the instances
119+
back out to JSON.
120+
121+
These classes provide a possible starting point for graph analysis and
122+
manipulation, e.g. in the REPL.
123+
124+
### Sample interactive session
125+
126+
Now in a Scala 2.11 REPL with the same JARs on the classpath:
127+
128+
scala -classpath scala-sculpt_2.11-0.0.1.jar:spray-json_2.11-1.3.2.jar
129+
130+
If we load `dep.json` as follows, we'll see the following graph:
17131

18132
```
19133
scala> import scala.tools.sculpt.cmd._
20134
import scala.tools.sculpt.cmd._
21135
22-
scala> load("../stest/dep.json")
23-
res0: scala.tools.sculpt.model.Graph = Graph '../stest/dep.json': 11 nodes, 11 edges
136+
scala> load("dep.json")
137+
res0: scala.tools.sculpt.model.Graph = Graph 'dep.json': 11 nodes, 11 edges
24138
25139
scala> println(res0.fullString)
26-
Graph '../stest/dep.json': 11 nodes, 11 edges
140+
Graph 'dep.json': 11 nodes, 11 edges
27141
Nodes:
142+
- o:Dep1
143+
- pkt:scala.tp:AnyRef
28144
- o:Dep1.def:<init>
29-
- o:Dep2.t:x
30-
- pkt:scala.cl:Int
145+
- pkt:java.pkt:lang.cl:Object.def:<init>
31146
- o:Dep1.def:x
32-
- o:Dep2
33-
- pkt:scala.tp:AnyRef
147+
- pkt:scala.cl:Int
34148
- o:Dep1.t:x
35-
- o:Dep2.def:x
149+
- o:Dep2
36150
- o:Dep2.def:<init>
37-
- pkt:java.pkt:lang.cl:Object.def:<init>
38-
- o:Dep1
151+
- o:Dep2.def:x
152+
- o:Dep2.t:x
39153
Edges:
40-
- o:Dep2.def:<init> -[Uses]-> o:Dep2
154+
- o:Dep1 -[Extends]-> pkt:scala.tp:AnyRef
155+
- o:Dep1.def:<init> -[Uses]-> o:Dep1
41156
- o:Dep1.def:<init> -[Uses]-> pkt:java.pkt:lang.cl:Object.def:<init>
42-
- o:Dep2.def:x -[Uses]-> o:Dep2.t:x
43-
- o:Dep2.def:<init> -[Uses]-> pkt:java.pkt:lang.cl:Object.def:<init>
44-
- o:Dep2.t:x -[Uses]-> pkt:scala.cl:Int
45157
- o:Dep1.def:x -[Uses]-> pkt:scala.cl:Int
46-
- o:Dep1 -[Extends]-> pkt:scala.tp:AnyRef
47158
- o:Dep1.t:x -[Uses]-> pkt:scala.cl:Int
48159
- o:Dep2 -[Extends]-> pkt:scala.tp:AnyRef
160+
- o:Dep2.def:<init> -[Uses]-> o:Dep2
161+
- o:Dep2.def:<init> -[Uses]-> pkt:java.pkt:lang.cl:Object.def:<init>
162+
- o:Dep2.def:x -[Uses]-> o:Dep2.t:x
49163
- o:Dep2.def:x -[Uses]-> pkt:scala.cl:Int
50-
- o:Dep1.def:<init> -[Uses]-> o:Dep1
164+
- o:Dep2.t:x -[Uses]-> pkt:scala.cl:Int
51165
```
52166

53-
Removing some nodes:
167+
and we can explore the effect of removing edges from the graph using `removePaths`:
54168

55169
```
56170
scala> res0.removePaths("Dep2", "java.lang")
57171
58172
scala> println(res0.fullString)
59-
Graph '../stest/dep.json': 6 nodes, 4 edges
173+
Graph 'dep.json': 6 nodes, 4 edges
60174
Nodes:
175+
- o:Dep1
176+
- pkt:scala.tp:AnyRef
61177
- o:Dep1.def:<init>
62-
- pkt:scala.cl:Int
63178
- o:Dep1.def:x
64-
- pkt:scala.tp:AnyRef
179+
- pkt:scala.cl:Int
65180
- o:Dep1.t:x
66-
- o:Dep1
67181
Edges:
68-
- o:Dep1.def:x -[Uses]-> pkt:scala.cl:Int
69182
- o:Dep1 -[Extends]-> pkt:scala.tp:AnyRef
70-
- o:Dep1.t:x -[Uses]-> pkt:scala.cl:Int
71183
- o:Dep1.def:<init> -[Uses]-> o:Dep1
184+
- o:Dep1.def:x -[Uses]-> pkt:scala.cl:Int
185+
- o:Dep1.t:x -[Uses]-> pkt:scala.cl:Int
72186
```
73187

74188
Saving the graph back to a JSON model and loading it again:
@@ -77,20 +191,31 @@ Saving the graph back to a JSON model and loading it again:
77191
scala> save(res0, "dep2.json")
78192
79193
scala> load("dep2.json")
80-
res5: scala.tools.sculpt.model.Graph = Graph 'dep2.json': 6 nodes, 4 edges
194+
res5: scala.tools.sculpt.model.Graph = Graph 'dep2.json': 3 nodes, 2 edges
81195
82196
scala> println(res5.fullString)
83197
Graph 'dep2.json': 6 nodes, 4 edges
84198
Nodes:
199+
- o:Dep1
200+
- pkt:scala.tp:AnyRef
85201
- o:Dep1.def:<init>
86-
- pkt:scala.cl:Int
87202
- o:Dep1.def:x
88-
- pkt:scala.tp:AnyRef
203+
- pkt:scala.cl:Int
89204
- o:Dep1.t:x
90-
- o:Dep1
91205
Edges:
92-
- o:Dep1.def:x -[Uses]-> pkt:scala.cl:Int
93206
- o:Dep1 -[Extends]-> pkt:scala.tp:AnyRef
94207
- o:Dep1.def:<init> -[Uses]-> o:Dep1
208+
- o:Dep1.def:x -[Uses]-> pkt:scala.cl:Int
95209
- o:Dep1.t:x -[Uses]-> pkt:scala.cl:Int
96210
```
211+
212+
## Future work
213+
214+
Possible future directions include:
215+
216+
* user interface, e.g. via ScalaIDE integration
217+
* aggregation of dependency data at different "zoom levels" (per-package, per-file, per-class/trait/object, per-method)
218+
* identify layers and cycles
219+
* automatic identification of problematic dependencies
220+
* “what-if” analyses exploring the effect of proposed code changes
221+
* offer a means of declaring and enforcing desired architectural constraints (allowed and forbidden dependencies)

0 commit comments

Comments
 (0)