Google

When using Large Language Models to synthetically generate data for specific tasks, such as text classification, a potential issue is the emergence of biased predictions. This can manifest as an imbalance in the generated samples, where the majority of instances fall into a single category, leading to a skewed dataset.

Biased Predictions in LLM-based Synthetic Data Generation

To counteract the issue of biased predictions when generating synthetic data, a technique known as input inversion can be applied. This method reverses the typical generation process by first specifying the desired output (e.g., a class label) and then prompting the LLM to generate a corresponding input that fits both the instruction and the predetermined output. This approach provides better control over the distribution of generated samples, helping to create a more balanced dataset.

Input Inversion for Mitigating Data Generation Bias

Based on the scenario, analyze the primary issue with this synthetically generated dataset and explain the most likely consequence this will have on the final trained sentiment analysis model's performance.

Analyzing Bias in Synthetic Dataset Generation

A team is creating a synthetic dataset of 10,000 emails to train a spam classifier. They decide to use a large language model with the following single, general prompt, repeated 10,000 times: 'Write an example of an email.' After generation, they plan to manually label each email as 'spam' or 'not spam'.

Critique this data generation method. Identify the most significant data quality issue that is likely to arise and explain the underlying reason for this issue, connecting it to the nature of the model's training. Furthermore, describe the potential negative impact this issue would have on the performance of the final spam classification model.

Critiquing a Synthetic Data Generation Method

A team is using a large language model to generate a synthetic dataset for training a sentiment classifier. The goal is to classify user feedback into 'Positive', 'Negative', or 'Neutral' categories. After generating 10,000 examples using a general prompt to create feedback, they find that approximately 80% of the generated samples are 'Positive', 15% are 'Neutral', and only 5% are 'Negative'. Which statement best analyzes the primary issue with this generated dataset and its most likely consequ

Learn Before

Related