Pairwise

Trainers are responsible for defining the the way to train a learnable scorer.

XPM Configxpmir.learning.trainers.multiple.MultipleTrainer(*, hooks, model, trainers)[source]

Bases: Trainer

This trainer can be used to combine various trainers

hooks: List[xpmir.learning.context.TrainingHook] = []

Hooks for this trainer: this includes the losses, but can be adapted for other uses The specific list of hooks depends on the specific trainer

model: xpmir.learning.optim.Module

If the model to optimize is different from the model passsed to Learn, this parameter can be used – initialization is still expected to be done at the learner level

trainers: Dict[str, xpmir.learning.trainers.Trainer]

The trainers

XPM Configxpmir.letor.trainers.LossTrainer(*, hooks, model, batcher, sampler, batch_size)[source]

Bases: Trainer

Trainer based on a loss function

This trainer supposes that:

  • the sampler_iter is initialized – and is a serializable iterator over batches

hooks: List[xpmir.learning.context.TrainingHook] = []

Hooks for this trainer: this includes the losses, but can be adapted for other uses The specific list of hooks depends on the specific trainer

model: xpmir.learning.optim.Module

If the model to optimize is different from the model passsed to Learn, this parameter can be used – initialization is still expected to be done at the learner level

batcher: xpmir.learning.batchers.Batcher = xpmir.learning.batchers.Batcher()

How to batch samples together

sampler: xpmir.learning.base.Sampler

The sampler to use

batch_size: int = 16

Number of samples per batch

process_microbatch(records: BaseRecords)[source]

Combines a forward and backard

This method can be implemented by specific trainers that use the gradient. In that case the regularizer losses should be taken into account with self.add_losses.

XPM Configxpmir.letor.trainers.pointwise.PointwiseTrainer(*, hooks, model, batcher, sampler, batch_size, lossfn)[source]

Bases: LossTrainer

Pointwise trainer

hooks: List[xpmir.learning.context.TrainingHook] = []

Hooks for this trainer: this includes the losses, but can be adapted for other uses The specific list of hooks depends on the specific trainer

model: xpmir.learning.optim.Module

If the model to optimize is different from the model passsed to Learn, this parameter can be used – initialization is still expected to be done at the learner level

batcher: xpmir.learning.batchers.Batcher = xpmir.learning.batchers.Batcher()

How to batch samples together

sampler: xpmir.letor.samplers.PointwiseSampler

The pairwise sampler

batch_size: int = 16

Number of samples per batch

lossfn: xpmir.letor.trainers.pointwise.PointwiseLoss = xpmir.letor.trainers.pointwise.MSELoss(weight=1.0)

Loss function to use

Trainer

XPM Configxpmir.letor.trainers.pairwise.PairwiseTrainer(*, hooks, model, batcher, sampler, batch_size, lossfn)[source]

Bases: LossTrainer

Pairwise trainer uses samples of the form (query, positive, negative)

hooks: List[xpmir.learning.context.TrainingHook] = []

Hooks for this trainer: this includes the losses, but can be adapted for other uses The specific list of hooks depends on the specific trainer

model: xpmir.learning.optim.Module

If the model to optimize is different from the model passsed to Learn, this parameter can be used – initialization is still expected to be done at the learner level

batcher: xpmir.learning.batchers.Batcher = xpmir.learning.batchers.Batcher()

How to batch samples together

sampler: xpmir.letor.samplers.PairwiseSampler

The pairwise sampler

batch_size: int = 16

Number of samples per batch

lossfn: xpmir.letor.trainers.pairwise.PairwiseLoss

The loss function

XPM Configxpmir.letor.trainers.pairwise.DuoPairwiseTrainer(*, hooks, model, batcher, sampler, batch_size, lossfn)[source]

Bases: LossTrainer

The pairwise trainer for duobert. The iter_batch method can be the same as the pairwiseTrainer

hooks: List[xpmir.learning.context.TrainingHook] = []

Hooks for this trainer: this includes the losses, but can be adapted for other uses The specific list of hooks depends on the specific trainer

model: xpmir.learning.optim.Module

If the model to optimize is different from the model passsed to Learn, this parameter can be used – initialization is still expected to be done at the learner level

batcher: xpmir.learning.batchers.Batcher = xpmir.learning.batchers.Batcher()

How to batch samples together

sampler: xpmir.letor.samplers.PairwiseSampler

The pairwise sampler

batch_size: int = 16

Number of samples per batch

lossfn: xpmir.letor.trainers.pairwise.PairwiseLossWithTarget

The loss function

XPM Configxpmir.letor.trainers.generative.GenerativeTrainer[source]

Bases: LossTrainer

hooks: List[xpmir.learning.context.TrainingHook] = []

Hooks for this trainer: this includes the losses, but can be adapted for other uses The specific list of hooks depends on the specific trainer

model: xpmir.learning.optim.Module

If the model to optimize is different from the model passsed to Learn, this parameter can be used – initialization is still expected to be done at the learner level

batcher: xpmir.learning.batchers.Batcher = xpmir.learning.batchers.Batcher()

How to batch samples together

sampler: xpmir.letor.samplers.PairwiseSampler

The pairwise sampler

batch_size: int = 16

Number of samples per batch

Losses

XPM Configxpmir.letor.trainers.pairwise.PairwiseLoss(*, weight)[source]

Bases: Config, TorchModule

Base class for any pairwise loss

weight: float = 1.0

The weight \(w\) with which the loss is multiplied (useful when combining with other ones)

compute(scores: torch.functional.Tensor, info: TrainerContext) → torch.functional.Tensor[source]

Computes the loss

Parameters:

  • scores – A (batch x 2) tensor (positive/negative)

  • info – the trainer context

Returns:

a torch scalar

XPM Configxpmir.letor.trainers.pairwise.CrossEntropyLoss(*, weight)[source]

Bases: PairwiseLoss

Cross-Entropy Loss

Computes the cross-entropy loss

Classification loss (relevant vs non-relevant) where the logit is equal to the difference between the relevant and the non relevant document (or equivalently, softmax then mean log probability of relevant documents) Reference: C. Burges et al., “Learning to rank using gradient descent,” 2005.

warning: this loss assumes the score returned by the scorer is a logit

\[\frac{w}{N} \sum_{(s^+,s-)} \log \frac{\exp(s^+)}{\exp(s^+)+\exp(s^-)}\]
weight: float = 1.0

The weight \(w\) with which the loss is multiplied (useful when combining with other ones)

XPM Configxpmir.letor.trainers.pairwise.HingeLoss(*, weight, margin)[source]

Bases: PairwiseLoss

Hinge (or max-margin) loss

\[\frac{w}{N} \sum_{(s^+,s-)} \max(0, m - (s^+ - s^-))\]
weight: float = 1.0

The weight \(w\) with which the loss is multiplied (useful when combining with other ones)

margin: float = 1.0

The margin for the Hinge loss

XPM Configxpmir.letor.trainers.pairwise.PointwiseCrossEntropyLoss(*, weight)[source]

Bases: PairwiseLoss

Point-wise cross-entropy loss

This is a point-wise loss adapted as a pairwise one.

This loss adapts to the ranker output type:

  • If real, uses a BCELossWithLogits (sigmoid transformation)

  • If probability, uses the BCELoss

  • If log probability, uses a BCEWithLogLoss

\[\frac{w}{2N} \sum_{(s^+,s-)} \log \frac{\exp(s^+)}{\exp(s^+)+\exp(s^-)} + \log \frac{\exp(s^-)}{\exp(s^+)+\exp(s^-)}\]
weight: float = 1.0

The weight \(w\) with which the loss is multiplied (useful when combining with other ones)

XPM Configxpmir.letor.trainers.pairwise.PairwiseLossWithTarget(*, weight)[source]

Bases: PairwiseLossWithTarget

weight: float = 1.0

Pairwise

XPM Configxpmir.letor.samplers.PairwiseSampler[source]

Bases: Sampler

Abstract class for pairwise samplers which output a set of (query, positive, negative) triples

XPM Configxpmir.letor.samplers.PairwiseModelBasedSampler(*, dataset, retriever)[source]

Bases: PairwiseSampler, ModelBasedSampler

A pairwise sampler based on a retrieval model

dataset: datamaestro_text.data.ir.Adhoc

The IR adhoc dataset

retriever: xpmir.rankers.Retriever

A retriever to sample negative documents

XPM Configxpmir.letor.samplers.TripletBasedSampler(*, source)[source]

Bases: PairwiseSampler

Sampler based on a triplet source

source: datamaestro_text.data.ir.TrainingTriplets

Triplets

XPM Configxpmir.letor.samplers.PairwiseDatasetTripletBasedSampler(*, dataset)[source]

Bases: PairwiseSampler

Sampler based on a dataset where each query is associated with (1) a set of relevant documents (2) negative documents, where each negative is sampled with a specific algorithm

dataset: datamaestro_text.data.ir.PairwiseSampleDataset
XPM Configxpmir.letor.samplers.PairwiseInBatchNegativesSampler(*, sampler)[source]

Bases: BatchwiseSampler

An in-batch negative sampler constructured from a pairwise one

sampler: xpmir.letor.samplers.PairwiseSampler

The base pairwise sampler

XPM Configxpmir.letor.samplers.PairwiseSampleDatasetFromTSV(*, id, hard_negative_samples_path)[source]

Bases: PairwiseSampleDataset

Read the pairwise sample dataset from a csv file

id: str

The unique dataset ID

hard_negative_samples_path: Path

The path which stores the existing ids

XPM Configxpmir.letor.samplers.PairwiseSamplerFromTSV(*, pairwise_samples_path)[source]

Bases: PairwiseSampler

pairwise_samples_path: Path

The path which stores the existing triplets

XPM Taskxpmir.letor.samplers.ModelBasedHardNegativeSampler(*, dataset, retriever)[source]

Bases: Task, Sampler

Submit type: datamaestro_text.data.ir.PairwiseSampleDataset

Retriever-based hard negative sampler

dataset: datamaestro_text.data.ir.Adhoc

The dataset which contains the topics and assessments

retriever: xpmir.rankers.Retriever

The retriever to score of the document wrt the query

hard_negative_samples: Pathgenerated

Path to store the generated hard negatives

XPM Configxpmir.letor.samplers.hydrators.PairwiseTransformAdapter(*, sampler, adapter)[source]

Bases: PairwiseSampler

Transforms pairwise samples using an adapter

It is interesting to use this adapter since the transformation is only performed if the samples are used: when using a SkippingIterator, when recovering a checkpoint, all the records might have to be processed otherwise.

sampler: xpmir.letor.samplers.PairwiseSampler

The distillation samples without texts for query and documents

adapter: xpmir.letor.samplers.hydrators.SampleTransform

The transformation