Feat/manga recommendations

CHANGELOG.md

@@ -10,9 +10,85 @@ The format is simplified version of [Keep a Changelog](https://keepachangelog.co
 
 ## [Unreleased]
 
+## [1.11.0]
+
+### Additions
+- **Recommendations**, as a source of its own under "For you" in Browse. Learns a taste profile
+  from what you read (reading is the driver; tracker scores only scale it; a dropped series counts
+  against its tags; what you read long ago fades), then asks four public catalogues — AniList,
+  MangaUpdates, Kitsu and MyAnimeList — for the best-rated and most-read series under every tag
+  you are into, merges what they agree on (a series all four name scores highest), and finds each
+  one in the sources your library uses, with MangaDex alternative titles as a fallback for pt-BR
+  names. Your sources' own genre filters are searched too, deeper for stronger tags, and the
+  trackers' "similar to" suggestions for your strongest seeds round it out. Everything is read in
+  detail before it is shown and ranked on its full tags, tag pairs you like together, and the
+  community's score
+- Recommendations learn from you: long-press a cover for *Not interested* (its tags lose weight,
+  relative to how much your library backs them) or *Already read*; adding to the library from the
+  window reinforces the tags it was recommended for. Verdicts can be forgotten on the tags screen
+- "Your tags" screen: every learned tag with its weight and how many series back it; drag a
+  slider to fix a weight, add a tag, strike one that is not a genre, long-press to select and
+  merge tags you consider the same. Merges, bans and weights are yours and are backed up
+- Recommendations are built in batches of fifty up to the number you choose (25 to 500; Settings ›
+  Browse › Recommendations), on a schedule (every week to every six months, at your hour, Wi-Fi
+  only) that carries on the next day by itself when a big target needs more than one night, or on
+  demand: "Fetch 50 more" adds a batch, "Rebuild from scratch" starts over. Stage-by-stage
+  progress, a notification when done, and the list refreshes itself when the library grows by ten
+  or more. How far back a "recent start" may go in the catalogues is a setting too
+- Recommendation filters: only with these tags (only tags something in the list carries), include
+  saved-but-unread, sort by best match / newest / fewest chapters, and "hide sources with
+  unreliable dates" (on by default). A long-press on a cover says why it is there: tags or "like
+  X", chapter count, last chapter, community score, and whether it is new, gone quiet or a bit
+  outside your usual
+- Trackers enrich the library overnight: their genres and tags join the entry's own (as a custom
+  edit, so a refresh keeps them), and every other name they know feeds migration and merged-source
+  search — which is where a scanlator's own Portuguese title finally finds the series elsewhere
+- Merged sources: the app looks overnight for library entries on your other sources and offers a
+  one-tap merge on the entry's page
+- Release calendar: month view (toggle in the app bar), "Arrived today", "On hiatus", a "Only what
+  I'm caught up on" filter (also honoured by the widget and the morning digest), a "probably"
+  estimate for entries too new to have a rhythm (borrowed from the source's other series), and
+  tracker status (AniList, MAL, Kitsu, MangaUpdates) overriding a source that never updated its own
+- Webtoon reader: "continue reading" returns to the exact spot on a tall page, a tenth of a
+  screen early, instead of the top of the page
+- Dracula / Alucard theme (follows the system: Alucard light, Dracula dark)
+- Category rules, "Your tags" and the release calendar share the app's standard app bar
+
+### Changes
+- "Hide library items" in Browse is now on by default: what you already have on a source no
+  longer shows up when browsing that source
+- Recommendations skip one-shots (by tag, title or chapter names), series with no chapters, and
+  series gone quiet for longer than their size allows (three months for a new one, up to two years
+  for a long one; finished series are exempt). Series from sources that report no real dates are
+  ranked lower and hidden by default, and their chapter count does not make them "new". Every
+  rebuild starts from nothing: the previous list and the rows it left behind are discarded first
+- Tag names are unified across languages (Ação / Action / Acción), and source metadata dressed as
+  a genre ("Classificação: Sugestivo", scanlator names, years) is dropped
+- Merged-source search no longer offers the entry's own source or sources already merged in, and
+  tries the series' other names on a source that found nothing under the library title
+- A weekly series that posted late once keeps expecting its usual weekday
+- One slot in five on the recommendation list goes to something a little outside your usual, so
+  the list keeps surprising instead of narrowing on the same five tags
+
 ### Fixes
 - Tapping an extension group header did nothing: the row's ripple was on a child view, and a
   clickable child swallows the touch before it reaches the row the click listener is on
+- Sources that report every chapter as uploaded "0 minutes ago" no longer rewrite every chapter on
+  each update, pollute "recently updated", or fool the release calendar and the recommendations:
+  such dates are treated as unknown, and the calendar learns from when chapters actually arrive
+- Release calendar: a source that stamps a date as midnight UTC no longer lands a day early west
+  of Greenwich; webtoon "current page" was one behind when the image was taller than the screen
+- Release estimates are recalculated once when the estimator changes, instead of staying wrong
+  until each series' next fetch
+- Recommendations: a stale list could survive a rebuild that finished while the screen was
+  covered; a re-rank could overwrite a batch the build had just written; looking a title up in the
+  sources ran one source at a time with no timeout, so one silent source could hold a build for
+  an hour; "Fetch 50 more" did nothing once the target was met
+- Tag names from sources and from your own typing are sanitised before they are sent to AniList
+  or Kitsu
+
+### Other
+- Chapter memo is treated as source-owned; per-chapter reading offsets live in a preference
 
 ## [1.10.3]
 

app/build.gradle.kts

@@ -19,7 +19,7 @@ fun runCommand(command: String): String {
 }
 
 @Suppress("PropertyName")
-val _versionName = "1.10.3"
+val _versionName = "1.11.0"
 val betaCount by lazy {
     val betaTags = runCommand("git tag -l --sort=refname v${_versionName}-b*")
 
@@ -47,7 +47,7 @@ base {
 android {
     defaultConfig {
         applicationId = "eu.kanade.tachiyomi"
-        versionCode = 162
+        versionCode = 163
         versionName = _versionName
         testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
         multiDexEnabled = true

app/src/main/java/eu/kanade/tachiyomi/App.kt

@@ -61,6 +61,8 @@ import eu.kanade.tachiyomi.util.system.AuthenticatorUtil
 import eu.kanade.tachiyomi.util.system.GLUtil
 import eu.kanade.tachiyomi.util.system.ImageUtil
 import eu.kanade.tachiyomi.util.system.launchIO
+import karasu.domain.manga.interval.ReleaseEstimate
+import karasu.domain.manga.interval.RecalculateReleaseEstimates
 import eu.kanade.tachiyomi.util.system.localeContext
 import eu.kanade.tachiyomi.util.system.notification
 import eu.kanade.tachiyomi.util.system.setToDefault
@@ -153,6 +155,14 @@ open class App : Application(), DefaultLifecycleObserver, SingletonImageLoader.F
             with(TachiyomiWidgetManager()) { this@App.init() }
         }
 
+        // The release estimator changed since these estimates were written: redo them once.
+        if (preferences.releaseEstimateVersion().get() != ReleaseEstimate.ESTIMATOR_VERSION) {
+            scope.launchIO {
+                runCatching { Injekt.get<RecalculateReleaseEstimates>().await() }
+                    .onSuccess { preferences.releaseEstimateVersion().set(ReleaseEstimate.ESTIMATOR_VERSION) }
+            }
+        }
+
         // Show notification to disable Incognito Mode when it's enabled
         preferences.incognitoMode().changes()
             .onEach { enabled ->

app/src/main/java/eu/kanade/tachiyomi/appwidget/ReleasesGlanceWidget.kt

@@ -109,7 +109,8 @@ class ReleasesGlanceWidget : GlanceAppWidget() {
             .toSet()
         val grace = ReleaseEstimate.graceOf(preferences.releaseMissGraceDays().get())
         val calendar = runCatching {
-            getReleaseSchedule.await(categories).calendar(dayCount = 7, grace = grace)
+            getReleaseSchedule.await(categories, onlyCaughtUp = preferences.releaseCalendarOnlyCaughtUp().get())
+                .calendar(dayCount = 7, grace = grace)
         }
             .getOrNull()
             ?: return ReleasesWidgetData(today = true, covers = emptyList())

app/src/main/java/eu/kanade/tachiyomi/data/library/LibraryUpdateJob.kt

@@ -31,6 +31,7 @@ import eu.kanade.tachiyomi.data.download.DownloadJob
 import eu.kanade.tachiyomi.data.download.DownloadManager
 import eu.kanade.tachiyomi.data.koreader.KoreaderSyncJob
 import eu.kanade.tachiyomi.data.migration.AutoMigrateJob
+import eu.kanade.tachiyomi.data.recommendation.RecommendationJob
 import eu.kanade.tachiyomi.data.notification.Notifications
 import eu.kanade.tachiyomi.data.preference.DEVICE_BATTERY_NOT_LOW
 import eu.kanade.tachiyomi.data.preference.DEVICE_CHARGING
@@ -860,6 +861,7 @@ class LibraryUpdateJob(private val context: Context, workerParams: WorkerParamet
             // do with how often the library is refreshed. Its own hour is its off switch, and
             // re-registering here is what survives a reboot or an app update.
             AutoMigrateJob.setupTask(context)
+            RecommendationJob.setupTask(context)
         }
 
         fun cancelAllWorks(context: Context) {

app/src/main/java/eu/kanade/tachiyomi/data/library/ReleaseDigestJob.kt

@@ -50,7 +50,8 @@ class ReleaseDigestJob(private val context: Context, workerParams: WorkerParamet
         // Day one of the calendar rather than a fresh query, so the notification and the screen
         // it opens can never disagree — including about overdue entries, which the calendar puts
         // on today because the chapter has not arrived and is therefore still coming.
-        val today = Injekt.get<GetReleaseSchedule>().await(categories)
+        val schedule = Injekt.get<GetReleaseSchedule>().await(categories, onlyCaughtUp = preferences.releaseCalendarOnlyCaughtUp().get())
+        val today = schedule
             .calendar(
                 dayCount = 1,
                 grace = ReleaseEstimate.graceOf(preferences.releaseMissGraceDays().get()),

app/src/main/java/eu/kanade/tachiyomi/data/migration/AutoMigrateJob.kt

@@ -23,6 +23,7 @@ import eu.kanade.tachiyomi.data.track.TrackManager
 import eu.kanade.tachiyomi.domain.manga.models.Manga
 import eu.kanade.tachiyomi.smartsearch.SmartSearchEngine
 import eu.kanade.tachiyomi.source.CatalogueSource
+import eu.kanade.tachiyomi.source.RecommendationSource
 import eu.kanade.tachiyomi.source.SourceManager
 import eu.kanade.tachiyomi.source.model.SManga
 import eu.kanade.tachiyomi.ui.main.MainActivity
@@ -143,7 +144,8 @@ class AutoMigrateJob(private val context: Context, workerParams: WorkerParameter
      * confident about, so listing pt-BR before English is what makes pt-BR win a tie.
      */
     private fun candidateSources(languages: List<String>, exclude: Long): List<CatalogueSource> {
-        val installed = sourceManager.getCatalogueSources().filter { it.id != exclude }
+        val installed = sourceManager.getCatalogueSources()
+            .filter { it.id != exclude && it.id != RecommendationSource.ID }
         return languages.flatMap { lang -> installed.filter { it.lang.equals(lang, ignoreCase = true) } }
     }
 
@@ -296,7 +298,6 @@ class AutoMigrateJob(private val context: Context, workerParams: WorkerParameter
          * twice a year and never walk it back. Landing exactly on the hour counts as passed, so
          * rescheduling at the moment the job fires books the next slot, not the same one again.
          */
-        @VisibleForTesting
         fun millisUntil(day: Int, hour: Int, now: ZonedDateTime = ZonedDateTime.now()): Long {
             val todayAt = now.truncatedTo(ChronoUnit.DAYS).withHour(hour)
             var next = if (todayAt.isAfter(now)) todayAt else todayAt.plusDays(1)

app/src/main/java/eu/kanade/tachiyomi/data/notification/Notifications.kt

@@ -71,6 +71,12 @@ object Notifications {
      * library moved on its own. That is worth being able to keep loud while muting the rest.
      */
     const val CHANNEL_AUTO_MIGRATE = "auto_migrate_channel"
+
+    /** A rebuild of the recommendations the reader asked for has finished. */
+    const val CHANNEL_RECOMMENDATIONS = "recommendations_channel"
+    const val ID_RECOMMENDATIONS = -107
+    const val CHANNEL_RECOMMENDATIONS_PROGRESS = "recommendations_progress_channel"
+    const val ID_RECOMMENDATIONS_PROGRESS = -108
     const val ID_AUTO_MIGRATE = -106
 
     /**
@@ -199,6 +205,21 @@ object Notifications {
             ).apply {
                 group = GROUP_LIBRARY
             },
+            NotificationChannel(
+                CHANNEL_RECOMMENDATIONS,
+                context.getString(MR.strings.recommendations),
+                NotificationManager.IMPORTANCE_DEFAULT,
+            ).apply {
+                group = GROUP_LIBRARY
+            },
+            NotificationChannel(
+                CHANNEL_RECOMMENDATIONS_PROGRESS,
+                context.getString(MR.strings.recommendation_refreshing),
+                NotificationManager.IMPORTANCE_LOW,
+            ).apply {
+                group = GROUP_LIBRARY
+                setShowBadge(false)
+            },
             NotificationChannel(
                 CHANNEL_BACKUP_RESTORE_PROGRESS,
                 context.getString(MR.strings.progress),

app/src/main/java/eu/kanade/tachiyomi/data/preference/PreferencesHelper.kt

@@ -245,6 +245,50 @@ class PreferencesHelper(val context: Context, val preferenceStore: PreferenceSto
      */
     fun releaseScheduleCategories() = preferenceStore.getStringSet("release_schedule_categories", emptySet())
 
+    /** Calendar as a month grid rather than the list of days. */
+    fun releaseCalendarMonthView() = preferenceStore.getBoolean("release_calendar_month_view", false)
+
+    /** Calendar, widget and digest leave out entries the reader is behind on. */
+    fun releaseCalendarOnlyCaughtUp() = preferenceStore.getBoolean("release_calendar_only_caught_up", false)
+
+    /** The estimator whose output the stored estimates came from; a newer one recalculates once. */
+    fun releaseEstimateVersion() = preferenceStore.getInt(Preference.appStateKey("release_estimate_version"), 0)
+
+    /** JSON object of chapter id to how far into its last read page the reader was (0..1). */
+    fun readerPageOffsets() = preferenceStore.getString("reader_page_offsets", "{}")
+
+    /** Hour of the day the recommendations are rebuilt, or -1 for never. */
+    fun recommendationHour() = preferenceStore.getInt("recommendation_hour", 3)
+
+    /** Library categories the taste profile is learned from. Empty means the whole library. */
+    fun recommendationCategories() = preferenceStore.getStringSet("recommendation_categories", emptySet())
+
+    /** How many recommendations to build up to. Bigger takes more nights; the job resumes itself. */
+    fun recommendationTarget() = preferenceStore.getInt("recommendation_target", 150)
+
+    /** How far back a series may have started to count as a recent start in the catalogue pass. */
+    fun recommendationRecentYears() = preferenceStore.getInt("recommendation_recent_years", 3)
+
+    /** Days between rebuilds. */
+    fun recommendationIntervalDays() = preferenceStore.getInt("recommendation_interval_days", 15)
+
+    fun recommendationOnlyOnWifi() = preferenceStore.getBoolean("recommendation_only_on_wifi", true)
+
+    /** JSON object of lowercase canonical tag to the name of the group the reader merged it into. */
+    fun recommendationTagMerges() = preferenceStore.getString("recommendation_tag_merges", "{}")
+
+    /** JSON list of the reader's verdicts on recommendations. A preference so it is backed up. */
+    fun recommendationFeedback() = preferenceStore.getString("recommendation_feedback", "[]")
+
+    /** JSON object of manga id to what its trackers know (tags, other names). Backed up with the rest. */
+    fun recommendationTrackerExtras() = preferenceStore.getString("recommendation_tracker_extras", "{}")
+
+    /** Tags the reader struck from the profile for good: things a source called a genre that are not one. */
+    fun recommendationBannedTags() = preferenceStore.getStringSet("recommendation_banned_tags", emptySet())
+
+    /** JSON object of tag to weight (-1..1): the reader's overrides on the learned taste profile. */
+    fun recommendationTagWeights() = preferenceStore.getString("recommendation_tag_weights", "{}")
+
     fun libraryUpdateMangaRestriction() = preferenceStore.getStringSet("library_update_manga_restriction", setOf(MANGA_HAS_UNREAD, MANGA_NON_COMPLETED, MANGA_NON_READ))
 
     fun libraryUpdateCategories() = preferenceStore.getStringSet("library_update_categories", emptySet())
@@ -423,7 +467,7 @@ class PreferencesHelper(val context: Context, val preferenceStore: PreferenceSto
 
     fun onlySearchPinned() = preferenceStore.getBoolean(Keys.onlySearchPinned, false)
 
-    fun hideInLibraryItems() = preferenceStore.getBoolean("browse_hide_in_library_items", false)
+    fun hideInLibraryItems() = preferenceStore.getBoolean("browse_hide_in_library_items", true)
 
     // Tutorial preferences
     fun shownFilterTutorial() = preferenceStore.getBoolean("shown_filter_tutorial", false)