There are two main challenges of the original decoder:

1)	Since we use MLP in the decoder, we need to assume that there is a fixed number of nodes in each graph. This  is typically done by setting a maximum number of nodes and masking the surplus nodes.

2)	When we use $\tilde{A}$ to compute the likelihood, we implicitly assume that there is some ordering over the nodes. But we don’t know the correct ordering of rows and columns when we compute reconstruction loss.

There ‘re two popular strategies to solve the second issue.


Drawback Of Original Graph Level Variational Decoder

The goal of graph level variational decoder is to define a posterior given a graph level latent embedding.  The original graph VAE proposed to combine a multi-layer perceptron (MLP) and Bernoulli distribution assumption to obtain the posterior:

$p_{\theta}(G | \mathbf{z}_G) = \prod_{(u,v)\in \mathcal{V}} \tilde{A}[u,v] A[u,v]+(1-\tilde{A}[u,v]) (1-A[u,v])$

Where $A$ is adjacency matrix and $\tilde{A}=\sigma(MLP(\mathbf{z}_G))$ is the predicted matrix of edge probabilities. The overall log-likelihood objective is equivalent to a set of independent binary cross-entropy loss.


University of Notre Dame

The node level VGAE can be modified to work with graph level representation by modifying the encoder and decoder with graph level representaions. 

Variational Graph Autoencoders (VGAE) Based On Graph-Level Latent Representations

Hamilton, W.L. Graph Representation Learning. *Synthesis Lectures on Artificial Intelligence and Machine Learning*. 2020, 14(3), 1–159. https://doi.org/10.2200/S01045ED1V01Y202009AIM046

Graph Representation Learning by William Hamilton

For any GNN model, the encoder can be modified to be a graph level variational encoder by adding a pooling layer, i.e.,

$\mu_{\mathbf{z}_G}=POOL_{\mu}(GNN_{\mu} (\mathbf{A}, \mathbf{X}))$,
$log \sigma_{\mathbf{z}_G}=POOL_{\sigma}(GNN_{\sigma} (\mathbf{A}, \mathbf{X}))$

Where $POOL: \mathbb{R}^{|V|\times d}\rightarrow \mathbb{R}^d$. Here we use two different GNNs to parameterize the mean and variance of posterior distribution, and we define a posterior for each single graph instead of each single node.


Learn Before

Related

Learn After