I'm sure nothing changed, but I'm noticing that when requesting favorites via the API (e.g. gelbooru.com:443/index.ph...7&limit=100&pid=0), the results are sorted by "favorite", rather than sorted by "added". This means trying to scrape/download new favorites is impossible, and requires checking the entire list (and indeed, I now see gallery-dl painfully checking tens of thousands of links trying to find new favorites).

I could have sworn the API results used to be sorted by date added so that you could simply start at the end and download new favorites until you hit one (or a series of several) that you've seen.

Regardless of whether this is a new behavior or not, is there some way to sort the results by date added, so I can suggest an enhancement to gallery-dl?

The API has never been able to sort by the date you favorited it. It's never had access to that information. In fact, it doesn't have any information about favorites - getting it to be able to search them at all despite favorites not being indexed was a hack.

I figured it hadn't actually changed and was just my imagination; I'll suggest to gallery-dl that they pull the whole list first, sort by date-added client-side, and then start downloading backwards from the end. Should have roughly the same effect, just with the need to pull a bunch of pages up-front.

Honestly, a lot of optimizations they could make on how they pull favs. My own script is much more efficient, but I don't collect metadata as gallery-dl does, so I still use it. The best code to maintain is the code that isn't yours. ;)

I think they are talking about the favorite API endpoint specifically, not the search results.

Yes; gallery-dl uses the favorite API; does it return results differently? Did I use the wrong URL in the example above?

So from what I'm hearing, there's really no way - with any API - to efficiently pull updates to a favorites collection? I only ask as while I want to keep my offline favorites backup up-to-date, I also don't want to hammer Gelbooru with 40000 requests (two per favorite) needlessly. If that's the only way then I'll schedule it to run weekly or something to lessen the load.

Discussing this with the author of gallery-dl, we're seeing very different behavior across two different user id's - his comes back sorted by favorite date (as expected), but mine comes back sorted by post id. Any idea what's up with this?
github.com/mikf/gallery-dl/discussions/4769

And as strange as this began, as of today it's returning items - at least the most recently added - in order of favoriting, so as far as I'm concerned it's now working as intended (and doesn't require me to hammer the servers to make updates). Still, very strange.