Skip to content

Validate/enrich the positional encodings in the tokenization

The PerceiverIO uses a set of positional embeddings based on Fourier encodings.

The current version tries to be generic (all kinds of information, geo, spectral, temporal, use the same function), but has not been validated in terms of implementation or in terms of parametrization (max frequercies, etc.).

We need to rethink the kind of embedding we want to use for each kind of information. We may want to have different versions even for the same kind of information (Fourier, learned, etc.).