Your Spotify history is not yours

I’ve been using Spotify since 2012 and was curious about what tracks have stuck with me, what I might have blasted non-stop for a month and then never listened to again, and how my playlists have evolved.

I was also mildly annoyed with Spotify. I like the core product, and discovering new music has never been easier. But I’ve watched them shove podcasts everywhere in the app, I’ve been declining their AI features, and I’ve seen ads added to a service I’m already paying for. I would rather have more control over my music than another surface for podcasts.

Realistically, Spotify is one of those subscriptions I’ll be paying for the rest of my life. I wanted to experiment with my library without depending on Spotify also wanting to build those features.

Spotify offers an API for your fifty most recently played tracks. To get the older history, you have to request a privacy export and wait for it to arrive. I wanted the whole record available locally, with new plays added as I listened. That became Acetate, an application for keeping track of my Spotify listening history, library, and playlists.

I’ll freely admit laziness played a part: I much prefer collecting songs to sorting them, and figured self-maintaining genre playlists would be nice to have. With a local copy I could keep working on that even if Spotify changed what was available through its API again.

Fourteen years in a zip file

Spotify have an export option for your Extended Streaming History which includes the date and time you listened to songs, the song and artist name, Spotify URI, how many milliseconds you listened for, what platform you listened on, what country you listened from and whether you skipped songs. You can request an export here.¹

Make sure you’re requesting the correct export. Spotify offer a generic “download your account data” but you’ll want the Extended Streaming History export specifically.²

When I downloaded my data it went back to 2012. After filtering out tracks played for less than thirty seconds, I had 155,831 plays. Acetate stores the raw events and applies that filter when querying them.

Importing this data meant I could start asking questions like:

• Which songs have I come back to over the years?
• When did I start listening to an artist religiously?
• Are there songs I thought I liked but only listened to once?

I wanted to retain the ability to define “listening to a track” differently in future. Maybe I’ll be interested in skipped tracks, or want to distinguish tracks I started from tracks I listened to the whole way through. For now I’m just keeping the events. I’d be annoyed to need another fourteen-year export because I’d decided a twenty-second listen was interesting after all.

Keeping the archive alive

Every quarter-hour Acetate queries Spotify’s recently-played API³ and adds any new items to the database. A uniqueness constraint on the played-at timestamp, track ID, and source ensures repeat queries don’t result in duplicates.

The endpoint allows pagination using timestamps as cursors, but my poller just grabs the latest fifty tracks. It doesn’t step through a cursor to ensure nothing was missed. If the computer is switched off for a while and I listen to more than fifty tracks, Acetate might not capture them all. For a long gap, my workaround is to request another export and import it.

The recently-played API doesn’t expose play duration, so I’ve left that NULL (ms_played is available in the export). I don’t think it’s good to use 0, as that would mean we’re assuming it’s a skip (which isn’t true for all of these events).

One song, several identities

Different releases of the same track (e.g. deluxe or compilation versions of albums) may have different Spotify track IDs.⁴

I didn’t realise this at first and my initial listening stats were aggregating on Spotify’s track ID. So rather than seeing a song I’d listened to fifty times, I’d see three different songs I’d listened to twenty, eighteen and twelve times respectively.

Acetate attempts to group these entries into recordings using stable identifiers where possible, preferably ISRCs.⁵ I don’t delete the Spotify IDs when I do this—they’re how I get back from a recording to its catalog entries and individual plays.

This condensed roughly 48,000 Spotify track IDs down to 43,077 recordings.

This is the unglamorous core of data science. It’s mostly data cleaning under the hood.

Deletion is data

Spotify’s data export only provides the current state of my playlists, not previous versions. I wanted to know what I’d removed or rearranged too, which meant collecting the changes myself.

To determine whether a playlist has changed since Acetate last saw it, the application calculates a hash of the ordered list of track IDs and compares it to the most recent version. If it’s different, a new full snapshot is saved. Playlist versions are append-only in Acetate’s database.⁶ Because the list is ordered, moving a track triggers a version just as adding or removing one does.

This means we can see when:

• A playlist has disappeared.
• A song has been removed.
• You’ve unliked a song. (Ever swiped the wrong way and forgotten which song it was, all within two seconds?)
• The order has changed.
• A catalog entry has been replaced by another track ID.

I can’t tell from the data whether a song has been deliberately removed from a playlist or whether it was an accidental swipe, but at this scale storing full playlist data every time a playlist changes isn’t particularly onerous so I’m happy to leave it for now.

Acetate uses SQLite for storage, which I figure would be plenty for most people’s Spotify data, and at least it’s easy to back up.

After implementing embeddings, I started getting intermittent database is locked errors across the various components that touch the database (poller, backfills, preview fetcher, dashboard). WAL mode allows for concurrent readers while there’s an active write transaction, but there can only ever be one write transaction at a time.⁷ Turns out my embedding worker was keeping a write transaction open while computing the next embedding (~40s of inference time!), even though it wasn’t actually writing anything to the database during that time. Committing the last embedding result right away before computing the next one solved the problem.

The product built on top

With Spotify data aggregated into a usable form, I could start building things. Turns out in July 2025 I explored the Daft Punk back catalogue pretty thoroughly. Still sad I never got to see them live though.

Wrapped can be for any year, quarter, month or week. Doesn’t have to be December. Doesn’t require Spotify’s involvement or temporarily turning the Spotify app into Snapchat.

“Lost gems” shows you songs you listened to a lot in the past, but have drifted away from since. Your Liked Songs archive updates itself each month. Smart Playlists let you specify a condition (e.g. liked more than a year ago, not played for ninety days, limit to fifty songs) and Acetate will keep a Spotify playlist in sync with songs matching that condition. We do validation to ensure playlists being written to are in a registry of playlists Acetate is allowed to manage so you don’t accidentally trash a carefully curated playlist you made.

Features like these are what justify the existence of Acetate to me. I’m not sure I’d keep it alive for the sake of data-hoarding Spotify data, but I know I’d at least fix bugs if Smart Playlists stopped working, as they’re genuinely useful to me.

I still use Spotify to play music; Acetate isn’t trying to be a streaming service. But now I can analyse what I’ve listened to and build features around it without depending on Spotify’s involvement. I’m happy to rent the playback. I’d rather keep the history.

The next feature I wanted was to generate playlists based on genre. Writing playlists to Spotify is easy enough, but I realised I needed to work out what genre each song was, and unfortunately genre data seemed to be mostly missing.

Footnotes

1. Spotify documents the fields included in its lifetime Extended Streaming History export, including timestamps, milliseconds played, track identity, platform, and skip information. ↩

2. Article 20 of the GDPR provides a data-portability right for qualifying personal data. The export is useful; it does not make the rest of the ingestion and maintenance work disappear. ↩

3. Spotify’s Get Recently Played Tracks endpoint returns play-history objects with cursor-based pagination. ↩

4. Spotify notes in its Web API reference that duplicate tracks, such as versions from a single and an album, are rated independently for popularity. ↩

5. ISRC is the industry identifier for a particular sound recording. It survives ordinary changes in distributor, file format, or bitrate, although remixes and materially different versions may receive their own codes. The International ISRC Registration Authority explains the distinction. ↩

6. Acetate hashes the ordered track-ID list as a compact change-detection fingerprint. Order matters: moving a track should trigger a new snapshot even when the membership is unchanged. The digest detects change; the saved full snapshot explains it. ↩

7. SQLite’s WAL documentation explains that readers and a writer can proceed concurrently, but there can only be one writer at a time. ↩