LambdaNetworks: Modeling long-range Interactions without Attention

We present a general framework for capturing long-range interactions between an input and structured contextual information (e.g. a pixel surrounded by other pixels). Our method, called the lambda layer, captures such interactions by transforming available contexts into linear functions, termed lambdas, and applying these linear functions to each input separately... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK BENCHMARK
Image Classification ImageNet LambdaResNet152 Top 1 Accuracy 84.0% # 27
Number of params 35M # 36
Image Classification ImageNet LambdaResNet200 Top 1 Accuracy 84.3% # 24
Number of params 42M # 33

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet