Skip to content
Projekte

davanstrien/awesome-synthetic-datasets

Auf GitHub ansehen

davanstrien/awesome-synthetic-datasets has 343 GitHub stars and 25 forks. Star changes are not measured yet. Last checked 2026-11-16 01:33 UTC. Last push 2026-11-04.

So werden diese Zahlen gemessen

Gemessener Sternzuwachs

in 24h
Aktualisierung verzögert
in 7 days
Aktualisierung verzögert
in 15 days
Aktualisierung verzögert
in 1 month
Aktualisierung verzögert

Nettoänderungen in überlappenden Zeiträumen, keine Tagessummen. Fehlende Messwerte bleiben leer. Sterne zeigen Interesse, nicht Qualität.

Über uns

awesome synthetic (text) datasets

Lernen

README.md

Original

Awesome Synthetic (text/vision) datasets

Generating datasets using large language models

Follow me on the 🤗 Hub


What is synthetic data?

Synthetic data refers to artificially generated data that usually aims to mimic real-world data. This data is created algorithmically, often using models or simulations, rather than collected from real-world sources. Synthetic data has been used for a long time in machine learning. Since the advent of LLMs, there has been increasing use of LLMs for producing synthetic data and for using synthetic data for training LLMs.

Resources

This repository aims to organize resources focused on helping people (including myself) get started with building synthetic datasets. As a result, it will only cover some things and will focus on pragmatic and practical resources for the most part.

Tutorials, guides and educational blog posts

Examples in this repository

Important techniques

Important Datasets

TinyStories

TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities.

OpenHermes-2.5

The Open Hermes 2.5 dataset is a continuation of the Open Hermes 1 dataset, at a much larger scale, much more diverse, and much higher quality compilation, reaching 1M, primarily synthetically generated instruction and chat samples.

Cosmopedia

Cosmopedia is a dataset of synthetic textbooks, blogposts, stories, posts and WikiHow articles generated by Mixtral-8x7B-Instruct-v0.1.The dataset contains over 30 million files and 25 billion tokens, making it the largest open synthetic dataset to date.

A reproduction of Textbooks Are All You Need. More detail in this blog post

WebSight

WebSight is a large synthetic dataset containing HTML/CSS codes representing synthetically generated English websites, each accompanied by a corresponding screenshot.

More details in this blog post.

synthetic_text_to_sql

gretelai/synthetic_text_to_sql is a rich dataset of high quality synthetic Text-to-SQL samples, designed and generated using Gretel Navigator, and released under Apache 2.0. Please see our release blogpost for more details.

Salesforce/xlam-function-calling-60k

A dataset of 60,000 generated function-calling examples across 21 categories and 3,673 APIs.

Libraries, code and tools

This list isn't compressive and tries to focus on either actively developed libraries or libraries/code examples that demonstrate a particular approach well.

distilabel

⚗️ distilabel is a framework for synthetic data and AI feedback for AI engineers that require high-quality outputs, full data ownership, and overall efficiency.

This is a very flexible library that is actively being developed and improved. It supports a large number of LLM providers. It includes many common synthetic data generation techniques from papers such as Self-Instruct and EvolInstruct.

NeMo Data Designer

NeMo Data Designer is a Python library for generating high-quality synthetic datasets.

A production-focused library from NVIDIA for creating diverse, realistic synthetic data. Key features include dependency-aware generation (for correlated fields), built-in validation with Python/SQL validators, LLM-as-a-judge quality scoring, and multi-provider support (NVIDIA, OpenAI, Hugging Face). Used to prepare pre and post training datasets for NVIDIA Nemotron models.

Torque

An open-source Typescript library for declarative and typesafe synthetic dataset generation. It allows to compose reusable conversations with a declarative DSL and generate production-ready datasets like you build UIs.

DataDreamer

An open-source Python library for prompting, synthetic data generation, and training workflows. It is designed to be simple, extremely efficient, and research-grade.

DataGenFlow

🌱 DataGenFlow is a free, open-source tool to generate and validate datasets with full transparency. Build generation pipelines by connecting built-in blocks, using templates, or creating custom blocks for specialized workflows.

Flexible low-code tool for building synthetic data generation pipelines, with support for various LLM providers. Fully customizable and extensible with your own code blocks or generation flows.

FeatureBench

An open-source pipeline for turning GitHub repositories into test-validated coding tasks. It discovers repositories, builds reproducible Docker environments, and generates tasks through extensible adapters. A feature-development adapter is included, with a unified CLI for data generation, agent inference, and evaluation.

llm-swarm

Manage scalable open LLM inference endpoints in Slurm clusters

This library is primarily focused on scaling synthetic text generation using Slurm clusters. This library was used to generate Cosmopedia

Domain Specific Dataset Project

This is a project to bootstrap the creation of domain-specific datasets for training models. The goal is to create a set of tools that help users to collaborate with domain experts. Includes a UI tool for creating a pipeline for generating a synthetic dataset focused on a particular domain.

AutoPrompt

A framework for prompt tuning using Intent-based Prompt Calibration

This framework aims to help you automatically generate high-quality, detailed prompts. It uses a refinement process, where it iteratively builds a dataset of challenging edge cases and optimizes the prompt accordingly. This approach aims to reduce the manual effort in prompt engineering and reduce prompt sensitivity.

Self-Contrast

Self-Contrast is an innovative method that offers an annotation-free approach for aligning with human preference.

This is the code accompanying the Starling team's paper "Extensive Self-Contrast Enables Feedback-Free Language Model Alignment". The "Nectar" synthetic dataset is used to train a reward model, which is then used to train a large language model by Reinforcement Learning with AI Feedback (RLAIF).

Important Papers

Some important papers about synthetic data generation. Note I'm not trying to add every possible paper on the topic, but rather focus on those that introduced important techniques or had a big impact (particularly "in the wild"). I am collecting a longer list of papers in this Hugging Face Collection.