From Zero to a Million Jokes: Loading Hugging Face Datasets into CameoDB

One CLI command loads one million Reddit jokes from Hugging Face into CameoDB, schema detected and fully indexed, so you can search the whole set at once.

Update

What’s new since this post (v0.3.8). Schema detection got smarter: repeated short values become string fields, the id column is chosen by its values, and schema detect --report explains every choice. data load gained --parallel to keep several batches in flight, --id to build an id from one or more columns, and --recreate, so the schema and summary you see today will look a little different from the ones below.

01 · Data Loading

You've Got CameoDB Running. Now What?

If you followed the Quickstart, you already have CameoDB running and a books index loaded. That's your proof-of-concept. Now let's go from a handful of records to one million.

The dataset: SocialGrep/one-million-reddit-jokes on Hugging Face. A single CSV file with joke titles, body text, scores, subreddits, and timestamps. Perfect for demonstrating CameoDB's ability to detect schemas and ingest data directly from a URL.

02 · Data Loading

Step 1: Detect the Schema

Point CameoDB's CLI at the raw CSV URL. The schema detect command reads the header row and samples data to infer field types automatically:

schema detect https://huggingface.co/datasets/SocialGrep/one-million-reddit-jokes/resolve/main/one-million-reddit-jokes.csv

CameoDB returns a full schema definition with 10 detected fields:

02 · Data Loading · continued

Step 1: Detect the Schema

{
 "routing_field_name": "id",
 "fields": {
 "id": { "field_type": "Text", "indexed": true, "stored": true, "tokenizer": "raw" },
 "title": { "field_type": "Text", "indexed": true, "stored": false },
 "selftext": { "field_type": "Text", "indexed": true, "stored": false },
 "score": { "field_type": "I64", "indexed": true, "fast": true },
 "created_utc": { "field_type": "I64", "indexed": true, "fast": true },
 "subreddit": { "field_type": "Boolean", "indexed": true },
 "type": { "field_type": "Text", "indexed": true },
 "permalink": { "field_type": "Text", "indexed": true },
 "domain": { "field_type": "Text", "indexed": true },
 "url": { "field_type": "Text", "indexed": true }
 }
}

Notice: score and created_utc are detected as I64 with fast fields enabled, meaning they support range queries and sorting. Text fields are fully indexed for search.

03 · Data Loading

Step 2: Load One Million Records

Now the single command that does everything: creates the index, applies the schema, downloads the CSV, and streams all rows in batches:

data load jokes https://huggingface.co/datasets/SocialGrep/one-million-reddit-jokes/resolve/main/one-million-reddit-jokes.csv
Schema was missing; detected and applied schema to index 'jokes'
Ingestion complete for index 'jokes': loaded=1000000 failed=0 (batch size 4000)

That's it. One million records, zero failures. CameoDB auto-detected the schema on first contact and streamed the data in batches of 4,000. The jokes index didn't exist before this command. It was created on the fly.

04 · Data Loading

Step 3: Search Instantly

The index is immediately queryable. Let's find football jokes:

search jokes title:football limit 5 return title, selftext
{
 "hits": [
 { "_score": 10.50, "title": "Football", "selftext": "[removed]" },
 { "_score": 10.41, "title": "Football", "selftext": "As a woman passed her daughter's closed bedroom door..." },
 { "_score": 9.91, "title": "Fart Football", "selftext": "An old married couple no sooner hit the pillows..." },
 // ... 2 more results
 ],
 "hits_returned": 5,
 "total_hits": 1330,
 "took_ms": 11,
 "stats": { "shards": { "total": 4, "responded": 4, "failed": 0 } }
}
04 · Data Loading · continued

Step 3: Search Instantly

1,330 football jokes found across 4 shards in 11 milliseconds. The data was distributed automatically. No configuration, no manual sharding, no external tooling.

05 · Data Loading

The Takeaway

Three commands. That's the entire workflow from discovering a dataset on Hugging Face to running full-text search queries against a million records. CameoDB handles schema inference, index creation, batch ingestion, and distributed search, all from the CLI.