Samplers provide samples in the form of records. They all inherit from:

XPM Configxpmir.letor.samplers.Sampler[source]

Bases: experimaestro.core.objects.Config, xpmir.utils.utils.EasyLogger

Abstract data sampler

class xpmir.letor.samplers.SerializableIterator(*args, **kwargs)[source]

Bases: Iterator[xpmir.utils.iter.ItType], Protocol[xpmir.utils.iter.ItType]

An iterator that can be serialized through state dictionaries.

This is used when saving the sampler state


XPM Configxpmir.letor.samplers.PointwiseSampler[source]

Bases: xpmir.letor.samplers.Sampler

pointwise_iter() xpmir.utils.iter.SerializableIterator[xpmir.letor.records.PointwiseRecord][source]

Iterable over pointwise records

XPM Configxpmir.letor.samplers.PointwiseModelBasedSampler(*, dataset, retriever, relevant_ratio)[source]

Bases: xpmir.letor.samplers.PointwiseSampler, xpmir.letor.samplers.ModelBasedSampler


The topics and assessments

retriever: xpmir.rankers.Retriever

The document retriever

relevant_ratio: float = 0.5

The target relevance ratio


XPM Configxpmir.letor.samplers.PairwiseSampler[source]

Bases: xpmir.letor.samplers.Sampler

Abstract class for pairwise samplers which output a set of (query, positive, negative) triples

XPM Configxpmir.letor.samplers.PairwiseModelBasedSampler(*, dataset, retriever)[source]

Bases: xpmir.letor.samplers.PairwiseSampler, xpmir.letor.samplers.ModelBasedSampler

A pairwise sampler based on a retrieval model


The topics and assessments

retriever: xpmir.rankers.Retriever

The document retriever

XPM Configxpmir.documents.samplers.BatchwiseRandomSpanSampler(*, documents, max_spansize)[source]

Bases: xpmir.documents.samplers.DocumentSampler, xpmir.letor.samplers.BatchwiseSampler

This sampler uses positive samples coming from the same documents and negative ones coming from others

Allows to (pre)-train as in co-condenser:

L. Gao and J. Callan, “Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval,” arXiv:2108.05540 [cs], Aug. 2021, Accessed: Sep. 17, 2021. [Online].

max_spansize: int = 1000

Maximum span size in number of characters

XPM Configxpmir.letor.samplers.TripletBasedSampler(*, source, index)[source]

Bases: xpmir.letor.samplers.PairwiseSampler

Sampler based on a triplet file


source: the source of the triplets index: the index (if the source is only)

XPM Configxpmir.letor.samplers.PairwiseDatasetTripletBasedSampler(*, dataset)[source]

Bases: xpmir.letor.samplers.PairwiseSampler

Sampler based on a dataset where each query is associated with (1) a set of relevant documents (2) negative documents, where each negative is sampled with a specific algorithm


Hard Negatives Sampling (Tasks)

XPM Taskxpmir.letor.samplers.ModelBasedHardNegativeSampler(*, dataset, retriever)[source]

Bases: experimaestro.core.objects.Task, xpmir.letor.samplers.Sampler

Retriever-based hard negative sampler


The dataset which contains the topics and assessments

retriever: xpmir.rankers.Retriever

The retriever to score of the document wrt the query

hard_negative_samples: Pathgenerated

Path to store the generated hard negatives

XPM Taskxpmir.letor.samplers.TeacherModelBasedHardNegativesTripletSampler(*, sampler, document_store, query_store, teacher_model)[source]

Bases: experimaestro.core.objects.Task, xpmir.letor.samplers.Sampler

For a given set of triplet, assign the score for the documents according to the teacher model

sampler: xpmir.letor.samplers.PairwiseSampler

The list of exsting hard negatives which we can sample from


The document store

query_store: xpmir.datasets.adapters.TextStore

The query_document store

teacher_model: xpmir.rankers.Scorer

The teacher model which scores the positive and negative document

hard_negative_triplet: Pathgenerated

The path to store the generated triplets


class xpmir.letor.distillation.samplers.PairwiseDistillationSample(query, documents)[source]

Bases: NamedTuple

documents: Tuple[xpmir.rankers.ScoredDocument, xpmir.rankers.ScoredDocument]

Positive/negative document with teacher scores

query: xpmir.letor.records.Query

The query

XPM Configxpmir.letor.distillation.samplers.PairwiseDistillationSamples[source]

Bases: experimaestro.core.objects.Config, Iterable[xpmir.letor.distillation.samplers.PairwiseDistillationSample]

Pairwise distillation file

XPM Configxpmir.letor.distillation.samplers.PairwiseDistillationSamplesTSV(*, id, path, with_docid, with_queryid)[source]

Bases: xpmir.letor.distillation.samplers.PairwiseDistillationSamples,

A TSV file (Score 1, Score 2, Query, Document 1, Document 2)

id: str

The unique dataset ID

path: Path

The path of the file

with_docid: bool
with_queryid: bool

Records for training

class xpmir.letor.records.PairwiseRecord(query: xpmir.letor.records.Query, positive: xpmir.letor.records.Document, negative: xpmir.letor.records.Document)[source]

Bases: object

A pairwise record is composed of a query, a positive and a negative document

class xpmir.letor.records.PointwiseRecord(query: xpmir.letor.records.Query, docid: str, content: str, score: float, relevance: Optional[float])[source]

Bases: object

A record from a pointwise sampler