Google

To illustrate span masking, consider an original sequence $$\mathbf{x}$$: 'The 0 puppies are frolicking outside the house .'. By applying this corruption method to produce $$\mathbf{x}_{\mathrm{noise}}$$, two specific spans are targeted. First, a zero-length span, indicated by '0', results in the insertion of a `[MASK]` token. Second, the contiguous span 'frolicking outside the' is entirely replaced by another single `[MASK]` token. The final corrupted sequence $$\mathbf{x}_{\mathrm{noise}}$$ becomes: 'The `[MASK]` puppies are `[MASK]` house .'.

Example of Span Masking

A text corruption technique replaces non-overlapping segments of text (spans) with a single `[MASK]` token. Given the original sentence 'The new AI assistant can write code and summarize articles.' and its corrupted version 'The [MASK] assistant can [MASK] summarize articles.', identify the two specific text spans that were replaced by the `[MASK]` tokens.

Deconstructing a Masked Sentence

A text corruption technique involves selecting non-overlapping segments of text and replacing each segment with a single `[MASK]` token. This technique can also insert a `[MASK]` token by selecting a zero-length segment. Given the original sentence and the resulting corrupted sentence below, identify the two specific text segments from the original sentence that were replaced to create the corrupted version.

Analyzing a Text Corruption Process

A text corruption technique involves selecting non-overlapping segments of text and replacing each segment with a single `[MASK]` token. This technique also allows for selecting zero-length segments, which results in the insertion of a `[MASK]` token at that position. Given the original sentence 'The quick brown fox jumps over the lazy dog.' and two segments selected for corruption—the segment 'brown fox' and a zero-length segment between 'the' and 'lazy'—what is the resulting corrupted sentence

Learn Before

Related