Merge pull request #109 from otiliastr:fix_edges

PiperOrigin-RevId: 425667058

Author: tensorflow-copybara

research/gam/README.md

@@ -40,6 +40,21 @@ More details can be found in our
 [slides](https://drive.google.com/open?id=1tWEMoyrbLnzfSfTfYFi9eWgZWaPKF3Uu) or
 [poster](https://drive.google.com/file/d/1BZNR4B-xM41hdLLqx4mLsQ4KKJOhjgqV/view).
 
+## Updated Results
+
+A bug was discovered in the implementation of the GAM agreement regularization
+term after publication. We have fixed the bug (PR #82) and have rerun the
+affected experiments. Below are the updated results (note that the GAM* results
+are not affected).
+
+<img src="img/gam-updated-results.png " width="400">
+
+Although some of these numbers are lower than what was originally reported, the
+takeaways presented in our paper still hold: GAM adds a significant boost to the
+original base models, and also performs better than other forms of
+regularization reported in our paper. Nevertheless, we apologize for any
+inconvenience caused by this bug!
+
 ## How to run
 
 To run GAM on a graph-based dataset (e.g., Cora, Citeseer, Pubmed), from this
@@ -63,13 +78,13 @@ the dataset name accordingly: `$ tensorboard --logdir=outputs/summaries/cora`
 
 An example of such visualization for Cora with GCN + GAM model on the Pubmed
 dataset is the following:
-![Tensorboard plot](gam_gcn_pubmed.png?raw=true "GCN + GAM on Pubmed")
+![Tensorboard plot](img/gam_gcn_pubmed.png?raw=true "GCN + GAM on Pubmed")
 
 Similarly, we can run with multiple different parameter configurations and plot
 the results together for comparison. An example showing the accuracy per
 co-train iteration of a GCN + GAM model on the Cora dataset for 3 runs with 3
 different random seeds is the following:
-![Tensorboard plot](gam_gcn_cora_multiple_seeds.png?raw=true "GCN + GAM on Cora")
+![Tensorboard plot](img/gam_gcn_cora_multiple_seeds.png?raw=true "GCN + GAM on Cora")
 
 ## References
 

research/gam/gam/trainer/trainer_classification_gcn.py

@@ -871,7 +871,7 @@ def train(self, data, session=None, **kwargs):
 
   def predict(self, session, indices, is_train):
     """Make predictions for the provided sample indices."""
-    if not indices:
+    if not indices.shape[0]:
       return np.zeros((0, self.data.num_classes), dtype=np.float32)
     feed_dict = {
         self.input_indices: indices,

research/gam/gam/trainer/trainer_cotrain.py

@@ -355,12 +355,15 @@ def _select_samples_to_label(self, data, trainer_cls, session):
         assign to each of the selected nodes.
     """
     # Select the candidate samples for self-labeling, and make predictions.
-    # Remove the validation samples from the unlabeled data, if there, to avoid
-    # self-labeling them.
+    # We remove the validation and test samples from the unlabeled data,
+    # to avoid self-labeling them. We could potentially allow them to be
+    # self-labeled, but once a node is self-labeled its label is fixed for
+    # the remaining co-train iterations, so it would not take advantage
+    # of the improved versions of the model.
     indices_unlabeled = data.get_indices_unlabeled()
-    val_ind = set(data.get_indices_val())
+    eval_ind = set(data.get_indices_val()) | set(data.get_indices_test())
     indices_unlabeled = np.asarray(
-        [ind for ind in indices_unlabeled if ind not in val_ind])
+        [ind for ind in indices_unlabeled if ind not in eval_ind])
     predictions = trainer_cls.predict(
         session, indices_unlabeled, is_train=False)
 
@@ -546,8 +549,8 @@ def train(self, data, **kwargs):
 
     # Create a saver which saves only the variables that we would need to
     # restore in case the training process is restarted.
-    vars_to_save = [iter_cotrain] + trainer_agr.vars_to_save + \
-                   trainer_cls.vars_to_save
+    vars_to_save = [iter_cotrain
+                   ] + trainer_agr.vars_to_save + trainer_cls.vars_to_save
     saver = tf.train.Saver(vars_to_save)
 
     # Create a TensorFlow session. We allow soft placement in order to place
@@ -633,7 +636,9 @@ def train(self, data, **kwargs):
       logging.info(
           '--------- Cotrain step %6d | Accuracy val: %10.4f | '
           'Accuracy test: %10.4f ---------', step, val_acc, test_acc)
-
+      logging.info(
+          'Best validation acc: %.4f, corresponding test acc: %.4f at '
+          'iteration %d', best_val_acc, test_acc_at_best, iter_at_best)
       if self.first_iter_original and step == 0:
         logging.info('No self-labeling because the first iteration trains the '
                      'original classifier for evaluation purposes.')