Skip to content

feat: add llm for post translation feature #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cirex-web merged 3 commits into from
Mar 27, 2026
Merged

feat: add llm for post translation feature #67

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ac08d6f
Starter code with button for translation feature in P4
MichaelHilton Mar 25, 2026
d727ea4
feat: connect with our running backend instance
cirex-web Mar 26, 2026
3b4a9f6
fix: on error, don't translate
cirex-web Mar 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions public/openapi/components/schemas/PostObject.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,10 @@ PostObject:
For posts received via ActivityPub, it is the url of the original piece of content.
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
sourceContent:
type: string
nullable: true
Expand Down Expand Up @@ -206,6 +210,10 @@ PostDataObject:
description: A topic identifier
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
timestamp:
type: number
anonymous:
Expand Down
5 changes: 5 additions & 0 deletions public/openapi/components/schemas/TopicObject.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,9 @@ TopicObject:
type: string
isLocal:
type: boolean
translatedContent:
type: string
nullable: true
timestampISO:
type: string
description: An ISO 8601 formatted date string (complementing `timestamp`)
Expand Down Expand Up @@ -170,6 +173,8 @@ TopicObject:
example: "#f44336"
index:
type: number
isEnglish:
type: boolean
nullable: true
tags:
type: array
Expand Down
8 changes: 8 additions & 0 deletions public/openapi/read/categories.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,10 @@ get:
type: number
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
timestampISO:
type: string
description: An ISO 8601 formatted date string (complementing `timestamp`)
Expand Down Expand Up @@ -171,6 +175,10 @@ get:
type: number
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
sourceContent:
type: string
nullable: true
Expand Down
8 changes: 8 additions & 0 deletions public/openapi/read/index.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ get:
type: number
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
timestampISO:
type: string
description: An ISO 8601 formatted date string (complementing `timestamp`)
Expand Down Expand Up @@ -174,6 +178,10 @@ get:
type: number
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
sourceContent:
type: string
nullable: true
Expand Down
5 changes: 5 additions & 0 deletions public/openapi/read/unread.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,9 @@ get:
anonymous:
type: number
description: Whether this post was made anonymously (1 for anonymous, 0 for not)
translatedContent:
type: string
nullable: true
timestampISO:
type: string
description: An ISO 8601 formatted date string (complementing `timestamp`)
Expand Down Expand Up @@ -202,6 +205,8 @@ get:
example: "#f44336"
index:
type: number
isEnglish:
type: boolean
tags:
type: array
items:
Expand Down
4 changes: 4 additions & 0 deletions public/openapi/write/posts/pid.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,10 @@ get:
description: A topic identifier
content:
type: string
isEnglish:
type: boolean
translatedContent:
type: string
timestamp:
type: number
flagId:
Expand Down
16 changes: 16 additions & 0 deletions public/src/client/topic.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,12 +72,28 @@ define('forum/topic', [
addCrosspostsHandler();

$(window).on('scroll', utils.debounce(updateTopicTitle, 250));
configurePostToggle();

handleTopicSearch();

hooks.fire('action:topic.loaded', ajaxify.data);
};

function configurePostToggle() {
$('.topic').on('click', '.view-translated-btn', function () {
// Toggle the visibility of the next .translated-content div
$(this).closest('.sensitive-content-message').next('.translated-content').toggle();
// Optionally, change the button text based on visibility
var isVisible = $(this).closest('.sensitive-content-message').next('.translated-content').is(':visible');
if (isVisible) {
$(this).text('Hide the translated message.');
} else {
$(this).text('Click here to view the translated message.');
}
});
}


function handleTopicSearch() {
require(['mousetrap'], (mousetrap) => {
if (config.topicSearchEnabled) {
Expand Down
5 changes: 4 additions & 1 deletion src/posts/create.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ const categories = require('../categories');
const groups = require('../groups');
const activitypub = require('../activitypub');
const utils = require('../utils');
const translate = require('../translate');

module.exports = function (Posts) {
Posts.create = async function (data) {
Expand All @@ -18,6 +19,8 @@ module.exports = function (Posts) {
const timestamp = data.timestamp || Date.now();
const isMain = data.isMain || false;
let hasAttachment = false;
const [isEnglish, translatedContent] = await translate.translate(data);


if (!uid && parseInt(uid, 10) !== 0) {
throw new Error('[[error:invalid-uid]]');
Expand All @@ -28,7 +31,7 @@ module.exports = function (Posts) {
}

const pid = data.pid || await db.incrObjectField('global', 'nextPid');
let postData = { pid, uid, tid, content, sourceContent, timestamp };
let postData = { pid, uid, tid, content, sourceContent, timestamp, isEnglish, translatedContent};

if (data.toPid) {
postData.toPid = data.toPid;
Expand Down
6 changes: 6 additions & 0 deletions src/posts/data.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,5 +100,11 @@ function modifyPost(post, fields) {
post.uploads = [];
}
}
// Mark post as "English" if decided by translator service or if it has no info
post.isEnglish = post.isEnglish == 'true' || post.isEnglish === undefined;
// If translatedContent is undefined, default to empty string (no translation needed for English posts)
if (post.translatedContent === undefined) {
post.translatedContent = '';
}
}
}
2 changes: 1 addition & 1 deletion src/posts/summary.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ module.exports = function (Posts) {
options.escape = options.hasOwnProperty('escape') ? options.escape : false;
options.extraFields = options.hasOwnProperty('extraFields') ? options.extraFields : [];

const fields = ['pid', 'tid', 'toPid', 'url', 'content', 'sourceContent', 'uid', 'timestamp', 'deleted', 'upvotes', 'downvotes', 'replies', 'handle', 'anonymous'].concat(options.extraFields);
const fields = ['pid', 'tid', 'toPid', 'url', 'content', 'sourceContent', 'uid', 'timestamp', 'deleted', 'upvotes', 'downvotes', 'replies', 'handle', 'anonymous', 'isEnglish', 'translatedContent'].concat(options.extraFields);

let posts = await Posts.getPostsFields(pids, fields);
posts = posts.filter(Boolean);
Expand Down
18 changes: 18 additions & 0 deletions src/translate/index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

/* eslint-disable strict */
//var request = require('request');

const translatorApi = module.exports;



translatorApi.translate = async function (postData) {

@yuriiShmal yuriiShmal Mar 26, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You got the backend working? Above and beyond, I see.

@cirex-web cirex-web Mar 27, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup it's hosted on our VM

@cirex-web cirex-web Mar 27, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had to run ollama serve before starting the translation python service

const TRANSLATOR_API = 'http://17313-team11.s3d.cmu.edu:5000/';
try {
const response = await fetch(TRANSLATOR_API + '/?content=' + postData.content);
const data = await response.json();
return [data.is_english.toString(), data.translated_content];
} catch (e) {
return [true, postData.content]; // default as english
}
};
8 changes: 8 additions & 0 deletions vendor/nodebb-theme-harmony-2.1.35/templates/partials/topic/post.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,14 @@

<div class="content text-break" component="post/content" itemprop="text">
{posts.content}
{{{if !posts.isEnglish }}}
<div class="sensitive-content-message">
<a class="btn btn-sm btn-primary view-translated-btn">Click here to view the translated message.</a>
</div>
<div class="translated-content" style="display:none;">
{posts.translatedContent}
</div>
{{{end}}}
</div>

<div component="post/footer" class="post-footer border-bottom pb-2">
Expand Down
Loading