Working with audio and images

What you will learn How to use the audio and image paths, and the one thing that only works here.

Why these get their own chapter

Chat and embeddings in the previous chapter return text. Audio and images return files, which changes how you handle them.

• Responses are large (wav, png)
• They take a long time
• You have to decide where the result goes

Text to speech

Forget --output and binary data floods your terminal. A common mistake.

Split long text

Feeding in a long script at once takes a while, and a failure midway means starting over. Generating per paragraph and joining is more robust.

Joining is a job for an audio tool in the terminal.

Speech to text

This is the first step of turning a meeting recording into meeting notes.

graph TD
  A["Recording"] --> B["Speech → text"]
  B --> C["Raw transcript"]
  C --> D["Hand to a task<br/>'organise into action items'"]
  D --> E["Meeting notes"]

The second step is creating a task via the REST API. Joining the model API to the task API like this is the shape you will use most often in practice.

Cloned voices — only here

This is not in the OpenAI specification, so its path is not under /v1/ either.

Register your own voice, or a chosen one, and select it as the voice. It lets you produce many pieces consistently in the same voice.

⚠ A voice is personal data. Cloning someone else's voice requires their consent. That holds for internal material too, and more so for anything published. Do not make one without consent.

Image generation

The response carries image addresses. Downloading and saving them is a separate step.

Generating in batches

For many images, keep the prompts in a file.

That buys you three things.

Bury the prompts in code and you lose all three.

Editing and upscaling

Generate many at low resolution, upscale only the keepers — better on both time and cost.

Where the output files go

This is what tangles most often in practice.

It takes a long time

So work like this is better run on a schedule with only the result checked. It is not something to sit and wait for.

Common mistakes

Regenerating everything after a partial failure

Three of fifty failed and the whole batch runs again. Make it skip files that already exist. That requires predictable filenames.

Not recording the prompt

A great image comes out and you do not know which prompt made it. Keep the input file alongside.

Not getting voice consent

Written above, but worth repeating. Cloned voices presume consent.

Check yourself

1. Why split a long script for speech?

2. What do you gain by keeping image prompts in a file?

3. What must you confirm before using a cloned voice?

Now make all of this run by itself → Building pipelines with schedules