Merge branch 'tensorflow:master' into fix_edges

Author: otiliastr

README.md

@@ -71,38 +71,39 @@ learning with structured signals.
    target="_blank"><img src="http://img.youtube.com/vi/Js2WJkhdU7k/0.jpg"
                         alt="Adversarial Learning" width="180" border="2" /></a>
 
-We've also created the following hands-on colab-based tutorials that will allow
-you to interactively explore NSL:
+We've also created hands-on colab-based tutorials that will allow you to
+interactively explore NSL. Here are a few:
 
 *   [training with natural graphs](https://github.com/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_mlp_cora.ipynb)
 *   [training with synthesized graphs](https://github.com/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/graph_keras_lstm_imdb.ipynb)
 *   [adversarial learning](https://github.com/tensorflow/neural-structured-learning/blob/master/g3doc/tutorials/adversarial_keras_cnn_mnist.ipynb)
 
+You can find more examples and tutorials under the
+[examples](neural_structured_learning/examples) directory.
+
 ## Contributing to NSL
 
 Contributions are welcome and highly appreciated - there are several ways to
 contribute to TF Neural Structured Learning:
 
-*   Case studies. If you are interested in applying NSL, consider wrapping up
+*   Case studies: If you are interested in applying NSL, consider wrapping up
     your usage as a tutorial, a new dataset, or an example model that others
-    could use for experiments and/or development.
+    could use for experiments and/or development. The [examples](examples)
+    directory could be a good destination for such contributions.
 
-*   Product excellence. If you are interested in improving NSL's product
+*   Product excellence: If you are interested in improving NSL's product
     excellence and developer experience, the best way is to clone this repo,
     make changes directly on the implementation in your local repo, and then
     send us pull request to integrate your changes.
 
-*   New algorithms. If you are interested in developing new algorithms for NSL,
+*   New algorithms: If you are interested in developing new algorithms for NSL,
     the best way is to study the implementations of NSL libraries, and to think
     of extensions to the existing implementation (or alternative approaches). If
     you have a proposal for a new algorithm, we recommend starting by staging
-    your project in the `research` directory and including a colab notebook to
-    showcase the new features.
-
-    If you develop new algorithms in your own repository, we are happy to
-    feature pointers to academic publications and/or repositories that use NSL,
-    on
-    [tensorflow.org/neural_structured_learning](http://www.tensorflow.org/neural_structured_learning).
+    your project in the [research](research) directory and including a colab
+    notebook to showcase the new features. If you develop new algorithms in your
+    own repository, we would be happy to feature pointers to academic
+    publications and/or repositories using NSL from this repository.
 
 Please be sure to review the [contribution guidelines](CONTRIBUTING.md).
 
@@ -128,6 +129,11 @@ please fill this
 [form](https://docs.google.com/forms/d/1AQEcPSgmwWBJj3H2haEytF4C_fr1aotWaHjCEXpPm2A);
 we would love to hear from you.
 
+## Featured Usage
+
+Please see the [usage page](usage.md) to learn more about how NSL is being
+discussed and used in the open source community.
+
 ## Release Notes
 
 Please see the [release notes](RELEASE.md) for detailed version updates.

WORKSPACE

@@ -6,6 +6,14 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
 load("//research/carls/third_party/farmhash:workspace.bzl", farmhash = "repo")
 farmhash()
 
+# leveldb
+load("//research/carls/third_party/leveldb:workspace.bzl", leveldb = "repo")
+leveldb()
+
+# rocksdb
+load("//research/carls/third_party/rocksdb:workspace.bzl", rocksdb = "repo")
+rocksdb()
+
 # absl
 http_archive(
     name = "com_google_absl",
@@ -56,18 +64,13 @@ http_archive(
 # Use local tf to avoid error that tensorflow objects already registered.
 # Use custom protoc to make sure all protoc are built on the version of local tf
 # curl -L "https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOC_VERSION}/protoc-${PROTOC_VERSION}-linux-x86_64.zip" | sha256
-load("//research/carls:bazel/repo.bzl", "cc_tf_configure", "carls_protoc_deps")
+load("//research/carls:bazel/repo.bzl", "cc_tf_configure")
 cc_tf_configure()
 
 # Load protobuf compiler, protobuf and gRPC.
 # They MUST be in sync with TensorFlow's corresponding versions defined in
 # TENSORFLOW_DIR/tensorflow/workspace2.bzl
 
-# Protobuf compiler.
-PROTOC_VERSION = "3.9.2"
-PROTOC_SHA256 = "0d9034a3b02bd77edf5ef926fb514819a0007f84252c5e6a6391ddfc4189b904"
-carls_protoc_deps(version = PROTOC_VERSION, sha256 = PROTOC_SHA256)
-
 # Protobuffer
 http_archive(
     name = "com_google_protobuf",

g3doc/framework.md

@@ -75,9 +75,9 @@ NSL brings the following advantages:
 
 ## Step-by-step Tutorials
 
-To obtain hands-on experience with Neural Structured Learning, we have three
-tutorials that cover various scenarios where structured signals may be
-explicitly given, induced or constructed:
+To obtain hands-on experience with Neural Structured Learning, we have tutorials
+that cover various scenarios where structured signals may be explicitly given,
+constructed, or induced. Here are a few:
 
 *   [Graph regularization for document classification using natural graphs](tutorials/graph_keras_mlp_cora.ipynb).
     In this tutorial, we explore the use of graph regularization to classify
@@ -91,3 +91,7 @@ explicitly given, induced or constructed:
     In this tutorial, we explore the use of adversarial learning (where
     structured signals are induced) to classify images containing numeric
     digits.
+
+More examples and tutorials can be found in the
+[examples](https://github.com/tensorflow/neural-structured-learning/tree/master/neural_structured_learning/examples)
+directory of our GitHub repository.

neural_structured_learning/BUILD

@@ -31,6 +31,7 @@ py_library(
         ":version",
         "//neural_structured_learning/configs",
         "//neural_structured_learning/estimator",
+        "//neural_structured_learning/experimental",
         "//neural_structured_learning/keras",
         "//neural_structured_learning/lib",
         "//neural_structured_learning/tools",

neural_structured_learning/__init__.py

@@ -2,6 +2,7 @@
 
 from neural_structured_learning import configs
 from neural_structured_learning import estimator
+from neural_structured_learning import experimental
 from neural_structured_learning import keras
 from neural_structured_learning import lib
 from neural_structured_learning import tools

neural_structured_learning/examples/BUILD

@@ -0,0 +1,45 @@
+package(
+    licenses = ["notice"],  # Apache 2.0
+)
+
+py_binary(
+    name = "graph_keras_mlp_cora",
+    srcs = ["graph_keras_mlp_cora.py"],
+    python_version = "PY3",
+    deps = [
+        # package absl:app
+        # package absl/flags
+        # package absl/logging
+        # package attr
+        "//neural_structured_learning",
+        # package tensorflow
+    ],
+)
+
+py_binary(
+    name = "graph_nets_cora_graph_regularization",
+    srcs = ["graph_nets_cora_graph_regularization.py"],
+    python_version = "PY3",
+    deps = [
+        # package absl:app
+        # package absl/flags
+        # package graph_nets
+        "//neural_structured_learning",
+        "//neural_structured_learning/experimental:gnn",
+        # package tensorflow
+    ],
+)
+
+py_binary(
+    name = "graph_nets_cora_gcn",
+    srcs = ["graph_nets_cora_gcn.py"],
+    python_version = "PY3",
+    deps = [
+        # package absl:app
+        # package absl/flags
+        # package graph_nets
+        "//neural_structured_learning",
+        "//neural_structured_learning/experimental:gnn",
+        # package tensorflow
+    ],
+)

neural_structured_learning/examples/README.md

@@ -0,0 +1,31 @@
+This directory contains examples and tutorials demonstrating how to use NSL.
+
+## Example Trainers
+
+The `.py` files in this directory are all example trainers runnable end-to-end
+as Python programs:
+
+*   `adv_keras_cnn_mnist.py`: Adversarial regularization on the MNIST dataset.
+*   `graph_keras_mlp_cora.py`: Graph regularization on the Cora dataset.
+*   `graph_nets_cora_gcn.py`: Graph Convolutional Network on the Cora dataset
+    using [GraphNets](https://github.com/deepmind/graph_nets).
+*   `graph_nets_cora_graph_regularization.py`: Graph regularization on the Cora
+    dataset using [GraphNets](https://github.com/deepmind/graph_nets).
+
+## Notebooks
+
+The
+[notebooks](https://github.com/tensorflow/neural-structured-learning/tree/master/neural_structured_learning/examples/notebooks)
+subdirectory contains colab-based tutorials that allow you to explore NSL
+interactively. This subdirectory is generally where most new tutorials are
+added. Note that the tutorials shown on our
+[TensorFlow website](https://www.tensorflow.org/neural_structured_learning/framework#step-by-step_tutorials)
+are hosted under the
+[g3doc/tutorials](https://github.com/tensorflow/neural-structured-learning/tree/master/g3doc/tutorials)
+directory.
+
+## Data Preprocessing Scripts
+
+The
+[preprocess](https://github.com/tensorflow/neural-structured-learning/tree/master/neural_structured_learning/examples/preprocess)
+subdirectory contains various data preprocessing scripts.

neural_structured_learning/examples/graph_nets_cora_gcn.py

@@ -0,0 +1,57 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Example of an NSL GNN."""
+from absl import app
+from absl import flags
+import graph_nets
+import neural_structured_learning as nsl
+from neural_structured_learning.experimental import gnn
+import tensorflow as tf
+
+flags.DEFINE_string(
+    'train_examples_path',
+    None,
+    'Path to training examples.')
+flags.DEFINE_string('eval_examples_path',
+                    None,
+                    'Path to evaluation examples.')
+
+FLAGS = flags.FLAGS
+
+
+def main(argv):
+  del argv
+  neighbor_config = nsl.configs.GraphNeighborConfig(max_neighbors=3)
+  train_dataset = gnn.make_cora_dataset(
+      FLAGS.train_examples_path, shuffle=True, neighbor_config=neighbor_config)
+  eval_dataset = gnn.make_cora_dataset(FLAGS.eval_examples_path, batch_size=32)
+
+  model = gnn.GraphConvolutionalNodeClassifier(
+      seq_length=tf.data.experimental.get_structure(train_dataset)[0]
+      ['words'].shape[-1],
+      num_classes=7)
+  model.compile(
+      optimizer=tf.keras.optimizers.Adam(),
+      loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+      metrics=[
+          tf.keras.metrics.SparseCategoricalCrossentropy(from_logits=True),
+          tf.keras.metrics.SparseCategoricalAccuracy(),
+          tf.keras.metrics.SparseTopKCategoricalAccuracy(2),
+      ])
+  model.fit(train_dataset, epochs=30, validation_data=eval_dataset)
+
+
+if __name__ == '__main__':
+  graph_nets.compat.set_sonnet_version('2')
+  app.run(main)

neural_structured_learning/examples/graph_nets_cora_graph_regularization.py

@@ -0,0 +1,89 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Example of Graph Regularization with an NSL GNN."""
+import functools
+
+from absl import app
+from absl import flags
+import graph_nets
+import neural_structured_learning as nsl
+from neural_structured_learning.experimental import gnn
+import tensorflow as tf
+
+flags.DEFINE_string(
+    'train_examples_path',
+    None,
+    'Path to training examples.')
+flags.DEFINE_string('eval_examples_path',
+                    None,
+                    'Path to evaluation examples.')
+
+FLAGS = flags.FLAGS
+
+
+class NodeClassifier(tf.keras.Model):
+  """Classifier model for nodes."""
+
+  def __init__(self,
+               seq_length,
+               num_classes,
+               hidden_units=None,
+               dropout_rate=0.5,
+               **kwargs):
+    inputs = tf.keras.Input(shape=(seq_length,), dtype=tf.int64, name='words')
+    x = tf.keras.layers.Lambda(lambda x: tf.cast(x, tf.float32))(inputs)
+    for num_units in (hidden_units or [50, 50]):
+      x = tf.keras.layers.Dense(num_units, activation='relu')(x)
+      x = tf.keras.layers.Dropout(dropout_rate)(x)
+    outputs = tf.keras.layers.Dense(num_classes)(x)
+    super(NodeClassifier, self).__init__(inputs, outputs, **kwargs)
+
+
+def main(argv):
+  del argv
+  graph_reg_config = nsl.configs.GraphRegConfig(
+      neighbor_config=nsl.configs.GraphNeighborConfig(max_neighbors=3),
+      multiplier=0.1,
+      distance_config=nsl.configs.DistanceConfig(
+          distance_type=nsl.configs.DistanceType.L2,
+          reduction=tf.compat.v1.losses.Reduction.NONE,
+          sum_over_axis=-1))
+
+  train_dataset = gnn.make_cora_dataset(
+      FLAGS.train_examples_path,
+      shuffle=True,
+      neighbor_config=graph_reg_config.neighbor_config)
+  eval_dataset = gnn.make_cora_dataset(FLAGS.eval_examples_path, batch_size=32)
+
+  model = gnn.GraphRegularizationModel(
+      config=graph_reg_config,
+      node_model_fn=functools.partial(
+          NodeClassifier,
+          seq_length=tf.data.experimental.get_structure(train_dataset)[0]
+          ['words'].shape[-1],
+          num_classes=7))
+  model.compile(
+      optimizer=tf.keras.optimizers.Adam(),
+      loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+      metrics=[
+          tf.keras.metrics.SparseCategoricalCrossentropy(from_logits=True),
+          tf.keras.metrics.SparseCategoricalAccuracy(),
+          tf.keras.metrics.SparseTopKCategoricalAccuracy(2),
+      ])
+  model.fit(train_dataset, epochs=30, validation_data=eval_dataset)
+
+
+if __name__ == '__main__':
+  graph_nets.compat.set_sonnet_version('2')
+  app.run(main)

neural_structured_learning/examples/preprocess/cora/prep_data.sh

@@ -13,6 +13,8 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+# Script to download and preprocess the Cora dataset for use by NSL models.
+
 # URL for downloading Cora dataset.
 URL=https://linqs-data.soe.ucsc.edu/public/lbc/cora.tgz