Huggingface Transformers

Encoders

XPM Configxpmir.text.huggingface.TransformerVocab(*, model_id, trainable, layer)

Bases: xpmir.text.Vocab

Transformer-based encoder from Huggingface

model_id: str = bert-base-uncased

Model ID from huggingface

trainable: bool

Whether BERT parameters should be trained

layer: int = 0

Layer to use (0 is the last, -1 to use them all)

XPM Configxpmir.text.huggingface.IndependentTransformerVocab(*, model_id, trainable, layer)

Bases: xpmir.text.huggingface.TransformerVocab

Encodes as [CLS] QUERY [SEP]

model_id: str = bert-base-uncased

Model ID from huggingface

trainable: bool

Whether BERT parameters should be trained

layer: int = 0

Layer to use (0 is the last, -1 to use them all)

XPM Configxpmir.text.huggingface.ContextualizedTextEncoderAdapter(*, encoder, maxlen)

Bases: xpmir.text.encoders.ContextualizedTextEncoder

encoder: xpmir.text.huggingface.ContextualizedTransformerEncoder
maxlen: int
XPM Configxpmir.text.huggingface.DualTransformerEncoder(*, model_id, trainable, layer, maxlen)

Bases: xpmir.text.huggingface.TransformerVocab, xpmir.text.encoders.DualTextEncoder

Encodes the (query, document pair) using the [CLS] token

maxlen: Maximum length of the query document pair (in tokens) or None if using the transformer limit

model_id: str = bert-base-uncased

Model ID from huggingface

trainable: bool

Whether BERT parameters should be trained

layer: int = 0

Layer to use (0 is the last, -1 to use them all)

maxlen: int

Hooks

XPM Configxpmir.text.huggingface.LayerFreezer(*, transformer, freeze_embeddings, frozen)

Bases: xpmir.letor.context.InitializationTrainingHook

This training hook class can be used to freeze some of the transformer layers

transformer: xpmir.text.huggingface.TransformerVocab

The model

freeze_embeddings: bool = False

Whether embeddings should be frozen

frozen: int = 0

Number of frozen layers (can be negative, i.e. -1 meaning until the last layer excluded, etc. / 0 means no layer)

XPM Configxpmir.text.huggingface.DistributedModelHook(*, transformer)

Bases: xpmir.context.InitializationHook

Hook to distribute the model processing

When in multiprocessing/multidevice, use torch.nn.parallel.DistributedDataParallel, otherwise use torch.nn.DataParallel.

transformer: xpmir.text.huggingface.TransformerVocab

The model