Music

AI Music and Audio Generation

Music Generation Models
| Model | Type | Access |
|-------|------|--------|
| Suno | Text-to-song | Commercial |
| Udio | Text-to-song | Commercial |
| MusicGen (Meta) | Text-to-music | Open source |
| AudioCraft (Meta) | Audio suite | Open source |
| Stable Audio | Text-to-audio | Commercial |

MusicGen Usage

from audiocraft.models import MusicGen

model = MusicGen.get_pretrained("facebook/musicgen-medium")
model.set_generation_params(duration=30)  # 30 seconds

# Text to music
audio = model.generate(["upbeat electronic dance track with synths"])

# Music continuation (melody conditioning)
audio = model.generate_continuation(
    prompt="electronic dance music",
    audio=existing_audio,
    duration=15
)

Sound Effects Generation

# AudioGen for sound effects
from audiocraft.models import AudioGen

model = AudioGen.get_pretrained("facebook/audiogen-medium")
audio = model.generate(["thunderstorm with heavy rain and distant thunder"])

Key Capabilities
| Capability | Description |
|------------|-------------|
| Text-to-music | Description to audio |
| Melody continuation | Extend existing music |
| Style transfer | Apply genre/style |
| Stem separation | Isolate vocals, drums, etc. |
| Audio enhancement | Upscaling, denoising |

Stem Separation

from demucs.api import Separator

separator = Separator(model="htdemucs")
stems = separator.separate_audio_file("song.mp3")
# Returns: drums, bass, vocals, other

Use Cases
| Use Case | Approach |
|----------|----------|
| Background music | MusicGen with style prompts |
| Sound design | AudioGen for effects |
| Music production | Continuation, variation |
| Content creation | Royalty-free generation |
| Gaming | Adaptive music generation |

Considerations
| Factor | Consideration |
|--------|---------------|
| Copyright | Training data concerns |
| Licensing | Check commercial use rights |
| Quality | Still evolving, varies by genre |
| Length | Usually limited (30s-3min) |
| Control | Limited fine control |

Best Practices
- Provide detailed style descriptions
- Iterate with continuation for longer pieces
- Post-process with traditional tools
- Consider mixing generated with human-created
- Check licensing for commercial use