This repository is the result of my work as a mentee in the Quantum Computing Mentorship Program of the Quantum Open Source Foundation.
Here, we provide a proof-of-concept for the implementation of a technique for better training Quantum Neural Networks in Pennylane's Pytorch interface known as layerwise learning.
The task selected for this PoC is the same used in the original paper for the binary classification between the handwritten digits 3 and 6 in the MNIST dataset.
For LinkedIn: My name is Felipe Oyarce. Master in Quantum Optics. Machine Learning Developer at NotCo's AI Team and Quantum Computing Enthusiast. Inspired in technological solutions to real-world problems.
Official description: I'm a Broccoli, excellent for health but I don't attract people's attention. Sometimes I cry a little.
Theodor Isacsson is a Quantum Software Developer at Xanadu, enthusiastic about pushing the limits of quantum computing software and expanding its usability and usefulness in the field. Outside of work he’s an avid traveller, enjoying adventurous trips, hiking and exploring the world.
There is a very well known problem when trying to train effectively a Quantum Neural Network (or Parametrized Quantum Circuit) called Barren Plateaus, described as the phenomenon in which the gradients of a quantum circuit vanish exponentially when the size of the circuit to train increases, this means more qubits and layers. This is a huge limitation because it requires to know very well the problem to solve in order to select properly the circuit architecture. If you want to start by random architectures you're going to face Barren Plateaus 😓.
You can visualize Barren Plateaus like a very flat surface of the loss landscape, so its almost impossible to roll down (minimize the function) over this surface.
Different strategies have been proposed for avoiding Barren Plateaus for example by using local cost function.
In layerwise learning the strategy is to gradually increase the number of parameters by adding a few layers and training them while freezing the parameters of previous layers already trained. Once you've reached the complete size of the circuit then you define a partition of the weights to train first on that subset of the weights and then on the remaining weights. Our implemention here is very simple, first we train over the first half of the circuit while freezing the other half and then we swap to train the other part.
An easy way for understanding this technique is to think that we're dividing the problem into smaller circuits to successfully avoid to fall into Barren Plateaus 💪
A layer in this approach is composed by a set of single qubit gates randomly chosen between RZ, RY and RZ rotations in each qubit and a set of CZ two qubits gates in a ladder structure. Forget about the parameters in the example image, those are the trainable parameters of each layer.
As we mentioned earlier, here the idea is to define a number of layers to add in each step of Phase I, e.g two layers per step, and optimize the circuit just considering the new parameters as trainable parameters while freezing the previous layers.
Then, the circuit is splitted into larger partitions with the parameters from Phase I, so we consider these parameters as the initial weights in the optimization. In each sweep you train a partition of the weights while freezing the other partitions. Remember that here we train the circuit in halves.
- Choose a way for encoding the data into the circuit. Here we use an Angle Encoding.
- Implement the layer structure.
- In Phase I, initialize all the weights as zero.
- Think a way of training the circuit just on subsets of the total number of parameters for Phase II.
In our implementation we transform the quantum circuit into a TorchLayer to be used within a Sequential model in Pytorch.
[1] McClean et al., 2018. Barren plateaus in quantum neural network training landscapes.
[2] Skolik et al., 2020. Layerwise learning for quantum neural networks.
[3] Tensorflow quantum Blog post about layerwise learning.
[4] Tensorflow quantum YouTube's video about layerwise learning.