Translating Embeddings for Modeling Multi-relational Data

Whats New It proposes a simple and effective way for modeling relations as translation of head to tail using low dimensional embeddings.

How It Works

It minimize following margin baed loss: $\mathcal{L}=\sum_{(h, \ell, t) \in S} \sum_{\left(h^{\prime}, \ell, t^{\prime}\right) \in S_{(h, \ell, t)}^{\prime}}\left[\gamma+d(\boldsymbol{h}+\boldsymbol{\ell}, \boldsymbol{t})-d\left(\boldsymbol{h}^{\prime}+\boldsymbol{\ell}, \boldsymbol{t}^{\prime}\right)\right]_{+}$
- d is disimilarity measure, and h', t' are corrupt triplets. corrupt entries are created as below:
$S_{(h, \ell, t)}^{\prime}=\left\{\left(h^{\prime}, \ell, t\right) \mid h^{\prime} \in E\right\} \cup\left\{\left(h, \ell, t^{\prime}\right) \mid t^{\prime} \in E\right\}$
It experimented on two databases, Freebase and Wordnet.
- Evaluation criteria:
  - It removes head, and instead evaluate disimilarity between h'+l, and t for all entities as h'.
  - Arrange all entities h' as per score
  - It measures the rank of correct entitiy h = h'
  - It also tracks hits @ top-10
- Results shows that TransE has outperformed current SOTA techniques.
  
  Source: Author
- Also, perforamnce across 1-1, 1-Many, Many-1, and Many-Many was also encouraging.
- Few examples of the tail entities predicted are,
  
  Source: Author
- It also demonstrated capability of transfer learning, by splitting the data into two parts (keeping entities covered, but relationships are partinitoned). TransE was better among the candidates.
Source: Author

Provide feedback