# Telestrations Neural Networks

Yesterday, it was board game day at the lab where I have been working recently. Everyone got together for lunch at Snakes & Lattes, a Torontonian board game cafè chain, and we spent a couple of hours laughing and chatting and, obviously, playing board games.

The lab has a go-to traditional game for the occasion: Telestrations. The game is inspired by the classic childhood’s game of Chinese whispers (or Telephone, or Wireless phone, or Gossip, there’s a bunch of different names for different countries) and its rules are pretty simple.

Everyone gets a booklet, an erasable sharpie, and a list of random terms like “flamingo” or “pipe dream” or “treehouse”. Everyone picks a word and writes it on the first page of the booklet: that’s the secret source word.

At each turn, players pass their booklet to the person on their right, and the rules are as follows:

• When you see a word, you turn the page and you have sixty seconds to draw whatever the word is;
• When you see a drawing, you turn the page and you write your best guess for what is pictured.

Players keep alternating between guessing, drawing, and passing down the booklets until every booklet has done a full round of the table and is back in the hands of the original owner. For extra fun, everybody gets to draw their secret source word at the very beginning.

In other words, it’s a written game of Chinese whispers where every other word is drawn instead of written.

There are some rules to decide who wins at the end, but the obvious source of entertainment is the complete chaos that ensues as information gets corrupted drawing after drawing. At the end of a round, not one of the original secret words ever survives.

So now the obvious, rational, almost trivial question is: what happens when you use a GAN to draw, and an image classifier to guess?
Well, here I am to show you!

# Pitfalls of Graph Neural Network Evaluation 2.0

In this post, I’m going to summarize some conceptual problems that I have found when comparing different graph neural networks (GNNs) between them.

I’m going to argue that it is extremely difficult to make an objectively fair comparison between structurally different models and that the experimental comparisons found in the literature are not always sound.

I will try to suggest reasonable solutions whenever possible, but the goal of this post is simply to make these issues appear on your radar and maybe spark a conversation on the matter.

Some of the things that I’ll say are also addressed in the original Pitfalls of Graph Neural Network Evaluation (Shchur et al., 2018), which I warmly suggest you read.