Self-attention based encoder models for strong lens detection

The upcoming large scale surveys are expected to find approximately 10^5 strong gravitational systems by analysing data of many orders of magnitude larger than the current era. In this scenario,  non-automated techniques will be highly challenging and time-consuming. We propose a new automated architecture based on self-attention, which can substitute the currently used CNNs to find strong gravitational lenses. The advantages of self-attention-based encoder models over convolution neural networks are investigated and encoder models are analysed to optimise performance using the data from the Bologna Lens Challenge. From our study, we pointed out that self-Attention-based models have a clear advantage compared to simpler CNNs. A low computational cost and complexity make it a highly competing architecture to currently used CNN architecture. Moreover, introducing the encoder layers can also tackle the over-fitting problem present in the CNN's by acting as effective filters and providing better stability for the network.