I Built My Own NotebookLM-Style Article-to-Podcast Pipeline

The first time I heard a NotebookLM audio overview — two AI hosts having a genuinely listenable conversation about a document I'd uploaded — my reaction wasn't "wow, magic." It was "wait, I can build the useful 80% of this for my own content."

Some context: I'm a Japanese software engineer in the Bay Area (payments at PayPal, now at fintech startup Atlas), and I run a personal learning pipeline where an LLM writes me a study article every day and publishes it to my portfolio site (I wrote about that separately). The articles are good, but my best uninterrupted learning time is the commute and the gym — times when I can't read. What I wanted was exactly what NotebookLM does: every article automatically turned into a two-person podcast dialogue, in natural Japanese, waiting in the article page's audio player.

So I built it: Firebase Functions (TypeScript) + Firestore + Cloud Storage + OpenAI APIs. An LLM rewrites the article as a host/guest dialogue script, each line is synthesized with a different OpenAI TTS voice, the MP3s are concatenated, uploaded to Cloud Storage, and the article document gets the audio metadata. A Firestore trigger makes it fully automatic — a new article appears, and a few minutes later it has a podcast.

This post covers the pipeline end to end, plus the engineering details that turned out to matter most: content-hash-based regeneration skipping, a small status state machine, prompt versioning as a cache-busting input, and the honest limitations of doing audio this simply.

Pipeline overview

 study_articles/{id} created (ja, published)
        │  Firestore onDocumentCreated trigger
        ▼
 runAudioGeneration(articleId, template)
        │
        ├── content hash unchanged? ──▶ skip (audio already up to date)
        ▼
 article.audioStatus = "generating"
        │
        ▼
 LLM: article → HOST/GUEST dialogue script
 (template: tech / concept / book)
        │  parse "HOST: …" / "GUEST: …" lines
        ▼
 per-line TTS (OpenAI)
   host  → voice "alloy" ──┐
   guest → voice "nova" ───┤  one MP3 buffer per line
                           ▼
                    Buffer.concat
                           │
                           ▼
 Cloud Storage upload (public, immutable cache)
        │  delete previous audio file if any
        ▼
 article doc update:
   audio { url, duration, contentHash, template,
           model, promptVersion, hostVoice, guestVoice }
   audioStatus = "ready"

Two LLM-ish calls per article: one chat/completions-style call to write the script, then one TTS call per dialogue line. Everything else is plumbing — but the plumbing is where the interesting decisions live.

Step 1: Rewriting the article as a dialogue

Feeding an article to TTS directly produces a lecture — technically audio, but nothing you'd choose to listen to. The NotebookLM insight is that dialogue is the listenable format: questions reset your attention, and the back-and-forth chunks information naturally.

The script generator flattens the structured article (title, summary, intro, sections, conclusion, key takeaways) into plain text and asks the model to rewrite it as a script in a rigid format — HOST: and GUEST: lines, nothing else. There are three prompt templates depending on the content type:

• tech — the default for my study articles. Host plays a senior-engineer listener asking sharp clarifying questions (explicitly not fake-naive ones); guest is a peer engineer walking through design decisions and failure modes.
• concept — for theory-heavy articles; the dialogue focuses on the essence of the idea and where it applies.
• book — for reading notes; host plays the "so why should I care about this book?" role, and the structure is author's claims → my interpretation → my pushback.

The prompts encode rules I only discovered by listening to bad output:

1. No invented analogies. LLMs desperately want to say "it's like a restaurant." My articles are written for working engineers; the script must stay in technical vocabulary even when simplifying.
2. Never read code aloud. Listening to someone speak TypeScript is misery. The rule: replace any code with a one-line description of what it's for ("the part that mixes the user ID into the cache key").
3. No visual references. "As the table above shows…" is meaningless in audio. Trade-off tables become "there are three options: first…, second…".
4. Cover everything; pad nothing. Early versions summarized aggressively. The current prompt forbids omission — every section and every key takeaway must appear — while also banning filler back-channel ("I see", "interesting") used just to stretch length.
5. Structure for ears. Announce topic and conclusion up front; two to four sentences per turn; end with the host reciting the key takeaways as "today's points."

The output is parsed with a deliberately boring regex, and validated:

export function parseScript(raw: string): ScriptLine[] {
  const lines: ScriptLine[] = [];
  // Accepts both ASCII ":" and full-width "：" — the model uses both.
  const re = /^\s*(HOST|GUEST)\s*[:：]\s*(.+)$/gim;
  let m: RegExpExecArray | null;
  while ((m = re.exec(raw)) !== null) {
    const text = m[2].trim();
    if (text) lines.push({ speaker: m[1].toLowerCase() as Speaker, text });
  }
  return lines;
}

const lines = parseScript(raw);
if (lines.length < 4) {
  // A "script" with 3 lines means the model ignored the format.
  // Refuse loudly instead of synthesizing garbage audio.
  throw new Error(`Script must contain at least 4 dialogue turns (got ${lines.length})`);
}

I considered asking for JSON output instead, but the line format has a nice property: it degrades gracefully. If the model adds a stray preamble, the regex simply skips it. The minimum-lines check catches the catastrophic case.

Step 2: Two voices, line by line

Each parsed line is synthesized separately, because the host and guest use different voices — alloy for the host, nova for the guest. Two distinct voices are what make the dialogue sound like a dialogue; with one voice, the question-answer structure just sounds like someone arguing with themselves.

The detail that made the biggest difference for Japanese output is the TTS instructions parameter. OpenAI's general-purpose voices handle Japanese, but by default they carry a noticeable English-speaker accent and flat prosody. Giving each role an explicit speaking-style instruction — in Japanese — pushed quality from "robotic translation" to "acceptable podcast":

const VOICES = { host: "alloy", guest: "nova" } as const;

const JA_STYLE = {
  host:
    "Speak like a native-Japanese podcast MC: natural, easy to follow, " +
    "warm and upbeat, slightly brisk tempo, light pauses at punctuation. " +
    "Do not use an English accent.",
  guest:
    "Speak like a native-Japanese technical commentator: calm and " +
    "intellectual, soft assertions, pronounce technical terms carefully. " +
    "Do not use an English accent.",
} as const;
// (In the real code these instructions are written in Japanese.)

async function synthesizeDialogue(lines: ScriptLine[]) {
  const chunks: Buffer[] = [];
  for (const { speaker, text } of lines) {
    const res = await openai.audio.speech.create({
      model: TTS_MODEL,
      voice: VOICES[speaker],
      input: text,
      response_format: "mp3",
      instructions: JA_STYLE[speaker],
    });
    chunks.push(Buffer.from(await res.arrayBuffer()));
  }

  const mp3 = Buffer.concat(chunks);
  // Duration estimated from size at ~32kbps (~4000 bytes/sec).
  // Good enough for a UI label; not sample-accurate.
  const estimatedDuration = Math.round(mp3.length / 4000);
  return { mp3, estimatedDuration };
}

Two confessions about this code, both deliberate:

The "concat" is literally Buffer.concat. No ffmpeg, no crossfades, no loudness normalization — MP3 buffers are byte-concatenated. This works better than it has any right to: most decoders play concatenated MP3 streams fine, and turn-taking boundaries are natural places for tiny discontinuities anyway. A proper audio person would wince. For a personal pipeline, shipping without an ffmpeg binary in the Functions bundle was the right trade.

Duration is estimated from byte count. Parsing MP3 frame headers for exact duration wasn't worth it for a player label. Divide by an assumed bitrate, round, done.

Step 3: The content-hash skip

The most load-bearing piece of engineering in this pipeline is deciding when not to run it. TTS for a long article is the expensive step — minutes of wall time, nontrivial API cost. Regenerating identical audio because someone pressed the button twice, or a trigger re-fired, would be pure waste.

Every generated audio stores a contentHash: a SHA-256 over everything that determines the output —

function buildHashInput(article: Article, template: Template, model: string): string {
  const body = [
    article.title,
    article.summary,
    article.introduction,
    ...article.sections.map((s) => `${s.title}\n${s.content}`),
    article.conclusion,
    ...article.keyTakeaways,
  ].join("\n---\n");

  // Anything that changes the output must be part of the hash:
  // article content, dialogue template, prompt version, TTS/script model.
  return [body, template, `v${PROMPT_VERSION}`, model].join("|");
}

// At the top of runAudioGeneration:
if (!force && article.audio?.template === template) {
  const hash = sha256(buildHashInput(article, template, article.audio.model));
  if (article.audio.contentHash === hash) {
    return { audio: article.audio, skipped: true,
             reason: "Audio already up-to-date for this content and template" };
  }
}

The elegant side effect is what happens when any input changes. Edit a section of the article? Hash differs, next run regenerates. Switch the script model? Regenerates. And — my favorite — bump the prompt version constant, and every existing audio silently becomes stale:

// prompts.ts
// Bumped when prompt content changes meaningfully — participates in
// contentHash so existing audio is detected as stale and regenerated.
export const PROMPT_VERSION = 3;

Prompt versioning here isn't documentation; it's a cache key. When I rewrote the tech template to ban invented analogies, I bumped PROMPT_VERSION from 2 to 3 and every article's audio regenerated with the better script on its next touch — no migration script, no manual invalidation. A force flag exists for "I just want it regenerated" moments, bypassing the check entirely.

Step 4: Status state machine and cleanup

Generation takes minutes, and the frontend needs to render something meaningful the whole time. The article document carries a tiny state machine: audioStatus ∈ {generating, ready, failed} plus an optional audioError.

await articleRef.update({
  audioStatus: "generating",
  audioError: FieldValue.delete(),   // clear any stale error
});

try {
  const script = await generateDialogueScript(article, template);
  const { mp3, estimatedDuration } = await synthesizeDialogue(script.lines);

  // Delete the previous audio file BEFORE uploading the new one —
  // files are timestamp-named, so old ones would otherwise leak forever.
  if (article.audio?.storagePath) {
    await deleteAudio(article.audio.storagePath).catch(() => {
      /* best-effort: an orphaned file is not worth failing the run */
    });
  }

  const uploaded = await uploadAudio(articleId, mp3); // public, cache: immutable

  await articleRef.update({
    audio: {
      storageUrl: uploaded.url,
      storagePath: uploaded.path,
      duration: estimatedDuration,
      script: script.raw,          // stored for display / debugging
      template,
      contentHash,
      model: script.model,
      promptVersion: PROMPT_VERSION,
      hostVoice: "alloy",
      guestVoice: "nova",
      generatedAt: now(),
    },
    audioStatus: "ready",
  });
} catch (err) {
  await articleRef.update({
    audioStatus: "failed",
    audioError: String(err).slice(0, 500),  // truncate: error text is UI data,
  });                                        // not a log dump
  throw err;
}

Small choices with outsized value:

• Error truncation. A failed OpenAI call can produce an enormous error string. The document stores at most 500 characters — enough for the UI and for triage; full details go to Cloud Logging.
• Stale-file deletion. Audio files are named with a timestamp under study-article-audio/{articleId}/, so regeneration would silently accumulate orphans without the explicit delete. It's best-effort: a leaked file is a worse reason to fail a run than a missing delete.
• Immutable cache headers. Because a regeneration produces a new filename (and thus a new URL), the old URL never changes content — so Cache-Control: public, max-age=31536000, immutable is safe and the player never fights the CDN.
• The script is stored on the document. Being able to read exactly what dialogue was synthesized, months later, has settled every "why does this episode sound weird" question in seconds.

Step 5: Fully automatic via a Firestore trigger

The final piece removes the human. An onDocumentCreated trigger on the articles collection fires whenever the daily article generator publishes something new:

export const onArticleCreated = onDocumentCreated(
  "study_articles/{articleId}",
  async (event) => {
    const article = event.data?.data();
    if (!article) return;
    if (article.status !== "published") return;   // drafts don't get audio
    if (article.language !== "ja") return;        // JA-only for now
    if (article.audio) return;                    // already has audio

    try {
      await runAudioGeneration(event.params.articleId, "tech", false);
    } catch {
      // Do NOT rethrow: the failure is already persisted on the article
      // as audioStatus="failed", and rethrowing would make the platform
      // retry a deterministic failure.
    }
  }
);

The guards are the design: only published articles, only Japanese (the speaking-style instructions are Japanese-specific for now), never double-generate. And the catch block swallows the error on purpose — runAudioGeneration already wrote audioStatus: "failed" to the document, so the failure is visible where it matters, and letting the trigger error out would just make the platform retry something deterministic. Combined with the previous post's pipeline, the full chain is: Linear issue → scheduled article generation → Firestore write → trigger → podcast. I file an issue from my phone; the next morning there's an article and an episode.

Honest limitations

• Raw MP3 concatenation. No crossfade, no BGM, no intro jingle, no loudness normalization across lines. Occasionally you can hear a slightly abrupt seam between speakers. NotebookLM's production polish is genuinely better; this is the 80% version.
• Per-line synthesis is slow. Lines are synthesized sequentially — a long article means 60+ TTS calls and several minutes of wall time. Fine for a background trigger, and safely inside the 540-second function timeout, but nobody would call it snappy. Parallelizing with a small concurrency limit is the obvious fix I haven't needed yet.
• Japanese accent quality. The instructions parameter helps a lot, but a general-purpose multilingual TTS still isn't a dedicated Japanese engine. Pitch-accent errors on technical katakana terms slip through. It clears my "listenable on a commute" bar, not a "publish as a real podcast" bar.
• Byte-based duration. The estimate can drift by several seconds on long episodes. Harmless for a label; wrong tool if you ever need chapters or precise seeking.
• One template per run, chosen statically. The trigger always uses the tech template; picking concept/book requires the manual endpoint. Auto-classifying the article type is on the list.

Future ideas, roughly in order of appeal: concurrency-limited parallel TTS; an RSS feed so episodes land in a real podcast app (the killer feature I miss most); lightweight ffmpeg post-processing for gaps and loudness; auto-selecting the dialogue template from article content; and English audio with English-appropriate voice instructions.

Summary

• Dialogue is the trick, not the voices. Article → host/guest script → per-line two-voice TTS gets you a NotebookLM-style result with two API surfaces (one LLM call, N TTS calls) and plain TypeScript plumbing.
• Prompt rules that mattered: no invented analogies, never read code aloud, no visual references, cover everything, no filler. Each rule exists because I heard the failure it prevents.
• Hash everything that determines the output — content, template, prompt version, model — and skip generation when the hash matches. Prompt versioning doubles as cache invalidation: bump one constant and stale audio regenerates itself.
• A three-state audioStatus machine with truncated errors keeps the UI honest during multi-minute generations; deleting the prior audio file prevents silent storage leaks; new-filename-per-generation makes immutable caching safe.
• Firestore triggers make it fully hands-off, but write the idempotency guards (published? right language? already has audio?) and swallow trigger errors deliberately once the failure is persisted on the document.
• Ship the simple version: Buffer.concat for audio and byte-count duration are "wrong," and they've served every single day without complaint.

The unlock wasn't any single technique — it was noticing that "read on a commute" and "listen on a commute" are different products, and the second one was one weekend of plumbing away.